Streamline your DevOps workflow with Terraform Pipelines and Automated Environment Agent Registration on Azure

As a DevOps Engineer, I have a passion for automating tasks and simplifying my workflow. My situation is somewhat unique because I am configuring flexible virtual machine (VM) environments for specific purposes. While you’re reading this, you might be thinking, ‘Hmm, containers could address this issue.’ And you’re right—they can. However, due to specific requirements from clients or audiences, VMs may still be necessary. That’s why I devised a solution to automatically register Azure DevOps Environment VM Resources using ADO YAML pipelines, Terraform, Azure Virtual Machines, Custom Virtual Machine Extensions, and pipeline agent registration. In this article, I’ll walk you through how I accomplished this and highlight the benefits it brings.

The Approach

The challenge I aimed to address was deploying application infrastructure natively and dynamically configuring the app atop that infrastructure. While containerization is a popular choice, my specific use case required a more monolithic approach using virtual machines (VMs). To minimize licensing costs and skill gaps for future support, I deliberately avoided third-party Configuration Management tools. Instead, I leveraged OS-native tooling for app configuration.

Since I provisioned VMs using Terraform, I sought a streamlined, ‘one-click’ solution for building a VM, deploying the app, and configuring it. Enter the Microsoft Custom Script VM Extension—a valuable tool. At the end of the Terraform process, this extension executes an Azure DevOps Agent registration script on the VM. My pipeline then allows time for the VM to configure itself within my ADO project’s Pipeline Environment as a Deployment resource. Subsequently, another pipeline job verifies that the VM is operational. Finally, an App Deployment stage connects to the VM, facilitating application deployment.

This pattern proves advantageous because, as I make changes or updates to my app during its lifecycle, I can pass an Environment Resource Tag as a parameter to the same pipeline. This ensures that configuration adjustments apply only to tagged resources within the VM pool, even if multiple resources exist.

Provisioning and Terraform Configuration

To establish the foundation of our Infrastructure as Code (IaC) solution, I’ve turned to Terraform for provisioning. If you’ve worked with Terraform, you’re likely aware of its meticulous handling of state. In typical development scenarios, backend configuration is hardcoded in the backend.tf file. While the remote backend is the recommended approach, I took an additional step: I removed the backend configuration entirely from backend.tf and instead stored the necessary information as pipeline variables within an Azure DevOps Pipelines Library.

This approach allows me to dynamically initialize my backend state. By doing so, my YAML configuration becomes highly flexible and can adapt to any provided state configuration. This flexibility aligns well with the principles of a DevOps/SRE environment, where continuous integration and continuous deployment (CI/CD) concepts play a pivotal role in managing infrastructure.

Let’s clarify: the Terraform configuration itself isn’t overly complex, but adhering to the DRY principle (i.e., ‘Don’t Repeat Yourself’) is crucial. In my Terraform setup, I minimize configuration repetition at the application lifecycle level. Why? Because I often encounter app and compute configurations that require slight customization to meet different requirements.

However, at the lower layers, reusability becomes paramount. I achieve this by modularizing my Terraform components based on infrastructure layers. Here’s why this approach matters:

1. Foundational Infrastructure Layers: By modularizing at this level, I ensure that the right configurations—such as network settings, standardized disks, custom images, and firewalls—are managed by dedicated infrastructure teams. Each layer focuses on its specific responsibilities.
2. Leveraging Terraform Modules: I place my trust in infrastructure teams to use Terraform modules for lower-level infrastructures. These modules handle the ‘heavy lifting,’ allowing us to automate repetitive tasks efficiently.
3. App-Level Modularization: However, I don’t modularize components at the app level indiscriminately. Instead, I consider the variability of the configuration. If the app’s configuration remains relatively stable, I avoid unnecessary modularization. But when dealing with more dynamic or variable configurations, I embrace modularity to maintain flexibility.

My Terraform approach aims to balance specificity and reusability, ensuring efficient management in a DevOps/SRE environment.

VM Registration and Deployment Setup

The core concept revolves around provisioning Azure VMs, whether on Windows or Linux, using a custom VM extension. This extension executes either a PowerShell script or a Bash script, depending on your distribution.

Initially, I adapted the ADO Environment Registration script provided by Azure DevOps Environments. However, this script, designed to be a single line, proved cumbersome to work with and lacked effective debugging capabilities. To address this, I refactored the script into a more readable and flexible format, allowing me to execute it with the necessary parameters.

The primary purpose of this script is to register the VM as an Environment Resource within Azure DevOps. To achieve this, I utilize a Personal Access Token (PAT) with appropriate permissions for adding Environment Resources. While Azure DevOps currently lacks an alternative automated registration mechanism, using a PAT is significantly more secure than relying on traditional “username” and “password” credentials. For operational or production scenarios, I strongly recommend employing a service account in Azure DevOps to generate the PAT.

To enable the agent to execute remote commands on the target resource via a deployment pipeline, additional credentials are essential. Furthermore, the script ensures that the agent runs as a service. This way, even after system restarts, the Environment Resource gracefully comes back online.

Security, Durability, Flexibility

For security and convenience, all credentials are randomly generated for each build and securely stored in Azure Key Vault, referenceable by naming conventions related to the specific resources provisioned. This ‘hands-off’ approach allows me to dynamically create and manage VMs without manual configuration or concerns about security vulnerabilities. It also enables simple, yet secure machine access for VM and App Administrators.

The beauty of this approach lies in its full parameterization. My pipeline becomes a flexible pattern, adaptable to various contexts and scenarios.

VM Health Check and Deployment Management

The subsequent step involves using an Azure CLI task within my ADO YAML pipeline to verify the status of the VM and ensure it is running. Once the VM is ready, I pass essential variables—such as credential information and machine details—to the next stage of my pipeline. In this subsequent stage, I execute a Deployment job to deploy my application onto the VM.

Here’s how the process unfolds:

1. VM Status Check: The Azure CLI task queries the VM’s status. If it’s operational, we proceed.
2. Passing Variables: Armed with the necessary variables, we move to the deployment phase. These variables include credentials and specifics about the target machine.
3. Application Deployment: The Deployment job targets the designated environment and adheres to the resource tag(s) specified in my pipeline parameters. This dynamic approach allows me to deploy to any VM created via Terraform—no hardcoded values or pipeline configuration changes required.
4. Resource Management: Following this pattern, any VM that adheres to the registration process can have its app deployments managed seamlessly. The registered Environment Resource streamlines the entire process.
5. Environment Resource Tags: For further precision, I can specify an Environment Resource Tag to precisely target deployments.

Solution Advantages

In summary, this solution has several advantages:

• It is scalable: I can create as many VMs as I need with Terraform and register them as Environment Resources automatically.
• It is secure: I don’t have to expose any sensitive information or credentials in my pipeline or in my VMs. Everything is stored in Azure Key Vault and accessed only when needed.
• It is flexible: I can use any VM configuration that I want with Terraform and pass in my own script parameters, which helps filter down to the exact Environment Resource registration type required for my uses. Since ADO Agent software is somewhat complicated and has many options, depending on whether registration is for a pipeline agent, environment resource, deployment agent, etc., it helps to narrow down my options through a custom registration script.
• It is reusable: I can use the same pipeline and the same Terraform configuration for different projects or environments, by changing only the pipeline parameters.

Watch Out For…

Some “gotchas” to look for:

• Azure DevOps permissions: This can be messy if you aren’t careful, especially when accessing resources in cross-project configuration within an organization. However, this can be a challenge with any solution in Azure DevOps.
• Pipeline variables: Read up on pipeline variables. When you create multi-stage pipelines, and then output Terraform variables, you need to make sure you have the proper syntax for referencing those variables in later jobs, or even pipeline stages.
• Registration script syntax w/ Azure VM Extension: The Azure VM Custom Script Extension works well in this case, but because the commands to execute the script from Terraform use JSON syntax, you need to properly escape special characters. TIP: I found GitHub Copilot Chat or Bing Chat codex to be helpful with this…

My primary goal is to contribute to the DevOps community by sharing my insights and experiences. I trust that you’ll find this solution both valuable and intriguing. If you’re curious to delve deeper into my approach, don’t hesitate to reach out. Whether you have questions or feedback, feel free to share them in the comments—I’m all ears.