10/3/2025 by MHA
Automating Azure Data Factory Deployment: NodeTool, Build & Release Pipelines with Bicep
Modern CI/CD practices with Azure Data Factory typically use a build pipeline to generate validated artifacts and a release pipeline to deploy those artifacts into target environments (Dev, Test, Prod). NodeTool is used in the build pipeline to guarantee the required Node.js version is installed, supporting packages like @microsoft/azure-data-factory-utilities that enable automated ARM template generation.
Build Pipeline: Setup with NodeTool
The build pipeline YAML should include NodeTool to install Node.js and npm tasks to install ADFUtilities.
Sample build pipeline steps:
steps:
# configuring node
- task: NodeTool@0
inputs:
versionSpec: '18.x'
displayName: 'Install Node.js'
- task: Npm@1
inputs:
command: 'install'
verbose: true
workingDir: '$(workingDir)'
displayName: 'Install npm package'
# validating artifacts
- task: Npm@1
inputs:
command: 'custom'
customCommand: 'run build validate $(workingDir) /subscriptions/$(subscriptionId)/resourceGroups/$(resourceGroupName)/providers/Microsoft.DataFactory/factories/$(dataFactoryName)'
workingDir: '$(workingDir)'
displayName: 'Validate'
# generating ARM Templates from source code
- task: Npm@1
inputs:
command: 'custom'
customCommand: 'run build export $(workingDir) /subscriptions/$(subscriptionId)/resourceGroups/$(resourceGroupName)/providers/Microsoft.DataFactory/factories/$(dataFactoryName) artifacts'
workingDir: '$(workingDir)'
displayName: 'Generate ARM template'
- task: CopyFiles@2
inputs:
SourceFolder: '$(workingDir)/artifacts'
Contents: '**'
TargetFolder: '$(build.artifactstagingdirectory)/application'
displayName: 'Copying application artifact'
# building bicep into ARM
- task: AzureCLI@2
displayName: 'Build bicep artifact'
inputs:
azureSubscription: $(serviceConnection)
scriptType: 'pscore'
scriptLocation: 'inlineScript'
inlineScript: |
$file = "$(Build.SourcesDirectory)/AzureInfrastructure/IaC/Main.bicep"
New-Item -ItemType Directory -Force -Path $(build.artifactstagingdirectory)/infrastructure
az bicep build --file $file --outdir $(build.artifactstagingdirectory)/infrastructure
- task: PublishPipelineArtifact@1
inputs:
targetPath: '$(build.artifactstagingdirectory)'
artifact: 'datafactory'
publishLocation: 'pipeline'
This generates and publishes ARM templates as build artifacts for further deployment.
Release Pipeline: Deploy ARM Templates
The release pipeline uses these build artifacts to deploy Bicep-defined infrastructure and Data Factory resources.
- Download build artifact
- Deploy infrastructure using Bicep via Azure CLI or ARM tasks
- Deploy ARM templates to Data Factory using AzureResourceManagerTemplateDeployment
Example release strategy deployment steps:
strategy:
runOnce:
preDeploy:
steps:
- checkout: self
- task: DownloadPipelineArtifact@2 #downloading artifacts created in build stage
inputs:
source: 'current'
path: '$(Pipeline.Workspace)'
- task: AzureResourceManagerTemplateDeployment@3
displayName: Creating ADF Infrastructure by IaC
inputs:
deploymentScope: 'Resource Group'
azureResourceManagerConnection: '$(serviceConnection)'
subscriptionId: '$(subscriptionId)'
action: 'Create Or Update Resource Group'
resourceGroupName: '$(resourceGroupName)'
location: 'norwayeast'
templateLocation: 'Linked artifact'
csmFile: '$(Pipeline.Workspace)\datafactory\infrastructure\Main.json'
overrideParameters: '-domain $(domain) -env $(env)'
deploymentMode: 'Incremental'
deploymentOutputs: 'ArmOutputs'
deploy:
steps:
- task: AzureResourceManagerTemplateDeployment@3
displayName: 'Deploy ADF Artifacts'
inputs:
deploymentScope: 'Resource Group'
azureResourceManagerConnection: '$(serviceConnection)'
subscriptionId: '$(subscriptionId)'
action: 'Create Or Update Resource Group'
resourceGroupName: '$(resourceGroupName)'
location: 'norwayeast'
templateLocation: 'Linked artifact'
csmFile: '$(Pipeline.Workspace)/datafactory/application/ARMTemplateForFactory.json'
csmParametersFile: '$(Pipeline.Workspace)/datafactory/application/ARMTemplateParametersForFactory.json'
overrideParameters: '-factoryName $(dataFactoryName)'
deploymentMode: 'Incremental'
This decoupling of build and release ensures artifacts are validated, secured, and can be promoted to multiple stages with parameters and secret management.
Best Practices
- Use NodeTool for predictable Node.js installs on agents.
- Separation of build (validate/generate) and release (deploy) pipelines supports compliance and rollback.
- Store secrets and connection strings in Azure Key Vault, not variables or source code.
- Parameterize configurations per environment and automate overrides in the release pipeline.
- Always validate ADF artifacts before ARM/Bicep deployment for reliability.
References
- Automated publishing for continuous integration and delivery - Microsoft Docs
- NodeTool@0 - Node.js tool installer - Microsoft Docs
- Azure Data Factory Deployment using Azure DevOps & ADFUtilities
- How to Set Up CI/CD for Azure Data Factory Using Azure DevOps
- ADF Build - Create YAML CICD Pipeline - part 1
- Azure Data Factory (ADF)— Continuous integration and delivery (CI/CD)