Set Connection References and Environment Variables in Pipelines

Since Microsoft has released both Connection References and Environment Variables the process of setting them in a Pipeline wasn’t very straightforward. Scott Durow (Twitter & LinkedIn) explained a possible solution to it in one of his blog posts (Connection References with ALM – Mind the Gap!). In this blog post, I will explain a different way to set Connection References and Environment Variables in Pipelines.

The approach I am explaining and demonstrating is as of writing (2021-12-01) still in preview.

Update: As of 2021-12-15 the explained approach is GA. See MS announcement.

Since parts of the approach are still in preview you would use it at your own risk. I personally figure this still might be better than handling everything manually, but more about that later.

Problem

First of all, we should talk a bit about the problems that we currently have when it comes to ALM and Connection References as well as Environment Variables.

Manual deployment

When you are doing manual deployments it should work pretty well most of the time. When you import a solution that contains either Connection References you don’t have associated with a connection or Environment Variables where neither a default value nor current value exists an additional step in the import will be shown where you can configure those things.

Automated deployment

Bigger problems arise when it comes to automated deployments. Since there isn’t a human interaction possible no one could associate connections to connection references or assign values to environment variables.

This means those things have to happen after the deployment is finished. Either manually or through some PowerShell Scripting (some might call it “voodoo”) as Scott describes in the mentioned blog.

A result is that all Power Automate flows using a connection reference without an associated connection will get inactivated during import.

Don’t get me wrong, I do like the idea of both of them. The problem was/is “only” the ALM part.

Solution

Fortunately, Microsoft listens to feedback and comes to the rescue. The Import Solution Task of the Power Platform Build Tools does have two new configurations UseDeploymentSettingsFile and DeploymentSettingsFile.

What you have to do is basically create a settings file where you map Environment Variables to values and Connection References to in your target environment already existing connections.

The step will then perform the assigning and association part itself. This should lead to smooth deployments including activated flows.

Let’s see how we could use this part of the Import Solution task.

Prerequisits

To be able to follow this blog post we need to make sure we have some prerequisites in place.

ALM Process

I will base my explanation on the Basic ALM process I explained in an earlier blog post.

To summarize it very shortly:

The mentioned Basic ALM process does contain 3 build/release pipelines

• Create export from DEV
• Build Managed Solution
• Release (to Test)

During the further explanations within this blog post, we will do changes in the last two of those pipelines.

Power Platform CLI

In addition, you will need the Power Platform CLI installed on your machine.

Solution

Prepare a solution containing at least one Environment Variable and one Connection Reference to be able to test the process.

My solution looks like the following and is called “DevToolsTest“.

Create Settings file

The first step would be to create our Settings file. There are several ways of doing it (as described in the docs). We will do it based on a solution file.

Export Solution

We export the solution we created as part of the prerequisites unmanaged and save it somewhere on our machine.

Run pac

Open a Command Prompt (or PowerShell) in the folder which contains the solution zip file.

The following command will create a settings file (called settings-test.json) based on your solution in the same folder. You have to replace the name of the zip file with yours.

Make sure you have the latest version of the pac installed by executing “pac install latest” first.

pac solution create-settings --solution-zip ./DevToolsTest_1_0_0_1.zip --settings-file ./settings-test.json

In my case, the settings file looks like this

{
  "EnvironmentVariables": [
    {
      "SchemaName": "bebe_EnvVarTest",
      "Value": ""
    }
  ],
  "ConnectionReferences": [
    {
      "LogicalName": "bebe_sharedcommondataserviceforapps_e904a",
      "ConnectionId": "",
      "ConnectorId": "/providers/Microsoft.PowerApps/apis/shared_commondataserviceforapps"
    }
  ]
}

What you can see is that there is an array of Environment Variables where you could add the desired value. The file also contains an array of Connection References, what missing here is the ID of the connection (in the target environment) you would like to associate.

For the ease of this post, I will hard code all of that. For sure you could dynamically set/fill this file as a part of your pipeline. What I found most practical is having a settings file for every target environment. Much like the app.config one knows from C# projects.

If you have several static settings files those have to be handled manually in the future. For example when new Environment Variables or Connection References should be added.

The settings file should then be stored within your repository. I use a Settings folder for it. If you follow my “Folder structure of a Dataverse project“, I would put it under “PowerPlatform”

Find connection id

One step which could be a bit confusing is finding the internal Id of a connection. It is though as easy as opening the connection in the target environment and copying the Id from the url.

Finished settings file

My finished settings file looks like the following.

{
  "EnvironmentVariables": [
    {
      "SchemaName": "bebe_EnvVarTest",
      "Value": "Test Value"
    }
  ],
  "ConnectionReferences": [
    {
      "LogicalName": "bebe_sharedcommondataserviceforapps_*****",
      "ConnectionId": "6ea97b9cf5ae4bc0ad52************",
      "ConnectorId": "/providers/Microsoft.PowerApps/apis/shared_commondataserviceforapps"
    }
  ]
}

Change Pipeline

So far so good, but what do we have to change in our pipeline to be able to use this.

Since we use static settings files we “only” have to add them to the artefact of our second pipeline (Build Managed solution) and use it in our Import Solution task of the release pipeline.

Build Managed solution

We have to change the following two things in this pipeline.

Copy Files

As the penultimate task in our pipeline, we add a “copy file” task.

As “Source Folder” we select the folder in our repository where we stored the settings file(s).

As “Contents” we write two stars (**) to fetch everything.

As “Target Folder” choose the artefact staging directory and “Settings” as the subfolder

$(Build.ArtifactStagingDirectory)\Settings

Publish Pipeline Artifacts

We also have to alter the last task (Publish Pipeline Artifacts) in a way that takes the whole staging folder and not only the solution zip file.

To do so just change “File or directory path” to the following.

$(Build.ArtifactStagingDirectory)

Release to Test

All that is left to set Connection References and Environment Variables in Pipelines is to change our Release to Test release pipeline to actually use the settings file.

To do so we check the checkbox “Use deployment settings file” and choose the correct file from the artifact.

Conclusion

As you can see this approach to set Connection References and Environment Variables in Pipelines is much easier than running different PowerShell scripts. That’s at least what I think. As mentioned this part of the import step is still in preview, but hopefully, it will be GA soon. Since 2021-12-15 this approach is GA and can safely be used in Production.

I hope this post helped you to come a bit closer to your ALM goal. Please feel free to contact me if you have any further questions.