Menu

Lessons learned when creating reusable YAML-templates

I recently wrote about how to utilize reusable YAML-templates in Azure DevOps. Shared source control repository for YAML templates gives a great value from re-usability point of view. This blog post concentrates more how you should formulate YAML-templates to make them reusable as possible. 

Template hierarchy

First a few words about template hierarchy and how I have typically structured templates.

Orchestrator is an application specific YAML based CI/CD pipeline which main responsibility is to orchestrate and control build and deployment process. Basically orchestrator template contains Stages which can be different application environments where application will be deployed. Orchestrator consumes reusable templates to execute generic operations like building, deploying, executing integration tests or database migrations.

undefined

Consider these tips when creating reusable YAML-templates

These tips are created based on my previous experience when I have created and refactored existing YAML based pipelines.

1. Remember single responsibility principal

Create templates which have responsibility over a single part. For example you should have database migration template which only executes database migration nothing else. It's much easier to maintain templates when they have clear responsible and overall templates are also smaller.

2. Use aggregate templates

Purpose of the aggregate template is to hide logic behind one template so consumption of the template is straightforward and easy for consumer. Diagram below illustrates an aggregate template called "deploy.yaml" which deploys infrastructure and application at once. Deploy.yaml template itself consumes separate "deploy-infra.yaml" and "deploy-app.yaml" templates according to single responsibility principal.

undefined

3. Use parameters to make template reusable

Add all configuration related things to parameters. Do not hard code any Build Agent path etc. inside templates.

4. Pass JobName and DependsOn to template as a parameter

When JobName and DependsOn parameters are passed to the template you can easily control from the orchestrator template which is the execution order of the templates inside the stage. JobName is a technical name of the job and can be used with DependsOn to control dependency. When DependsOn parameter is determined as an object type you can then use multiple values.

Template:

parameters: 
    - name: jobName
      type: string
    - name: jobDisplayName
      type: string
    - name: dependsOn
      type: object

jobs:
    - job: ${{parameters.jobName}}
      displayName: ${{parameters.jobDisplayName}}
      dependsOn: ${{parameters.dependsOn}}

Orchestrator:

When template supports JobName and DependsOn parameters you can control execution order in orchestrator like this.

stages:
    - stage: DeploymentTest
      displayName: Deploy infra, application and database migrations to test
      jobs:
          - template: "Deploy/deploy.yaml@SharedYamlTemplates"
            parameters:
                jobName: 'DeployApplicationToTest'
                JobDisplayName: 'Deploy application to test'
                enviromentName: 'TEST'
                serviceConnection: ${{variables.test_serviceConnection}}
                appServiceName: 'app-service-test'
                infraParameterFile: 'test-parameters.json'
                azureSubscriptionId: ${{variables.test_azureSubscriptionId}}
                azureRG: ${{variables.test_azureResourceGroup}}
                dependsOn: 

          - template: "DatabaseMigrations/database-migration.yaml@SharedYamlTemplates"
            parameters:
                jobName: 'DatabaseMigrationToTest'
                JobDisplayName: 'Execute database migration to test'
                enviromentName: 'TEST'
                databaseServerName: 'databaseservertest'
                databaseName: 'testdatabase'
                projectName: 'Database.Migrations'
                serviceConnection: ${{variables.test_serviceConnection}}
                dependsOn: 
                    - DeployApplicationToTest # Database migration will be executed after DeployApplicationToTest

          - template: "Tests/integration-test.yaml@SharedYamlTemplates"
            parameters:
                jobName: 'IntegrationTestsToTest'
                jobDisplayName: 'Execute integration test to test'
                dependsOn:
                    - DeployApplicationToTest # Integration tests will be executed after DeployApplicationToTest and DatabaseMigrationToTest
                    - DatabaseMigrationToTest 

5. Add IsEnabled parameter

When template has IsEnabled parameter you can easily control from the orchestrator whether this template should be executed or not. Sometimes you might need to execute template only when code is commited ex. to specific branch.

parameters: 
    - name: jobName
      type: string
    - name: jobDisplayName
      type: string
    - name: dependsOn
      type: object
    - name: isEnabled
      type: string 

jobs:
    - job: ${{parameters.jobName}}
      displayName: ${{parameters.jobDisplayName}}
      dependsOn: ${{parameters.dependsOn}}
      condition: eq(${{parameters.isEnabled}}, true)

Comments