We can use pull request templates to help guide developers as they create pull requests. However, instructions could be ignored or have to be manually checked by the reviewer. Ideally, requesters would have to explicitly confirm that an action is unnecessary before continuing.

In this post, I will take you through how we can automate posting comments (via an Azure Pipeline) only if our CHANGELOG.md file has not been updated as part of the pull request. When requiring pull request comments to be resolved before completion, it can act as a non-intrusive reminder while still allowing people to bypass it with a reason if relevant.

Permissions

For the Azure pipeline to interact with Azure DevOps API and post comments to the pull request we need to ensure that the Build Service account used to run all pipelines has the correct permissions set.

The build service account is created when creating the project and is named [ProjectName] Build Service. We will need to ensure that this account has the permission Contribute to pull requests set to allow, as the default is set to deny. This can be done by navigating to the Repositories section of Project Settings and then selecting Security.

Once this is done we can use the build service account in our Azure Pipeline to interact with the API through the $(System.AccessToken).

For more information on the Build Service account permissions see the official documentation.

The second setting that we should update is turning on the Require a comment resolution setting in the branch policy. This will force the requester to resolve all comments before the pull request can be completed. This can be found in the Policies tab for a given branch in the project.

Building the Pipeline

We will need to create an Azure DevOps YAML pipeline that will run our script. This code is an inline PowerShell script, although you could use any scripting language or use a separate script file.

At the pipeline’s core is the API call to post a message to the pull request.

trigger: none

steps:
- checkout: none

- task: PowerShell@2
  condition: eq(variables['Build.Reason'], 'PullRequest')
  displayName: Post Message to PR
  env:
    SYSTEM_ACCESSTOKEN: $(System.AccessToken)  
  inputs:
      targetType: 'inline'
      script: |
        $url = "$(System.CollectionUri)$(System.TeamProject)/_apis/git/repositories/$(Build.Repository.Name)/pullRequests/$(System.PullRequest.PullRequestId)/threads?api-version=6.0"
        $body = @{
            "comments" = @(
                @{
                    "parentCommentId" = 0
                    "content" = "We have noticed you have not updated the CHANGELOG.md file 😱`n`nPlease confirm if this is not needed."
                }
            )
        } | ConvertTo-Json
        $headers = @{
            Authorization = "Bearer $env:SYSTEM_ACCESSTOKEN"
            "Content-Type" = "application/json"
        }
        Invoke-RestMethod -Uri $url -Method Post -Headers $headers -Body $body

If we leave the code like this, it will post a message every time this is run. We need to add some logic to check if the file has been added, or a comment has been left.

trigger: none

steps:
- checkout: none

- task: PowerShell@2
  condition: eq(variables['Build.Reason'], 'PullRequest')
  displayName: Post Message to PR
  env:
    SYSTEM_ACCESSTOKEN: $(System.AccessToken)  
  inputs:
      targetType: 'inline'
      script: |
        $headers = @{
            Authorization = "Bearer $env:SYSTEM_ACCESSTOKEN"
            "Content-Type" = "application/json"
        }

        #First check if any file includes CHANGELOG.md
        
        #Get all commits in the the pull request
        $commitsUrl = "$(System.CollectionUri)$(System.TeamProject)/_apis/git/repositories/$(Build.Repository.Name)/pullRequests/$(System.PullRequest.PullRequestId)/commits?api-version=6.0"
        $commits = Invoke-RestMethod -Uri $commitsUrl -Method Get -Headers $headers

        #Check files and for each commit if they include CHANGELOG.md
        foreach ($commit in $commits.value) {
            $commitUrl = "$(System.CollectionUri)$(System.TeamProject)/_apis/git/repositories/$(Build.Repository.Name)/commits/$($commit.commitId)/changes?api-version=6.0"
            $files = Invoke-RestMethod -Uri $commitUrl -Method Get -Headers $headers

            $changelogExists = $files.changes | Where-Object { $_.item.path -like "*CHANGELOG.md" }
            if ($changelogExists -ne $null) {
              Write-Host "Changes include CHANGELOG.md, skipping comment"
              exit 0
            }
        }
  
        $threadUrl = "$(System.CollectionUri)$(System.TeamProject)/_apis/git/repositories/$(Build.Repository.Name)/pullRequests/$(System.PullRequest.PullRequestId)/threads?api-version=6.0"
        $message="We have noticed you have not updated the CHANGELOG.md file 😱`n`nPlease confirm if this is not needed."
      
        #Check if comment already exists
        $existingComments = Invoke-RestMethod -Uri $threadUrl -Method Get -Headers $headers
        $commentExists = $existingComments.value | Where-Object { $_.comments.content -eq $message }
        if ($commentExists -ne $null) {
            Write-Host "Comment already exists"
            exit 0
        }

        #Post a new message if the file is not found and the comment does not exist
        Write-Host "Posting a new message to PR"
        $body = @{
          "comments" = @(
            @{
              "parentCommentId" = 0
              "content" = $message
            }
          )
          "status" = "active"
        } | ConvertTo-Json
      
        Invoke-RestMethod -Uri $threadUrl -Method Post -Headers $headers -Body $body

I have chosen not to fail the build if the file is not found as this is a soft validation. We are just reminding the requester that it is missing as it may be valid not to have updated the file, for example, if the change is in the readme.

Once we have added the pipeline to our git repository we need to register the pipeline. This can be done by navigating to the Pipelines section of the project and then selecting New Pipeline.

Configuration

Now that we have a pipeline that can run our script, the final step is to configure it to run on the pull request creation.

If you are using Azure DevOps for source control this must be done in the Azure DevOps UI (as described here) while GitHub and Bitbucket Cloud allow it to be configured in the pipeline YAML.

In Azure DevOps navigate to the Repos section of your project settings and then select Branches. Here you can set the policy for the branch, including the validation pipeline to run. Select to add a new validation with the plus button.

Then select the pipeline that you wish to trigger on pull request creation or updates.

Final Result

If we create a branch and a pull request without a CHANGELOG.md file we will see the following comment posted to the pull request. The creator can then resolve the comment if they have a valid reason for not updating the file, or choose to add the file.

Further updates will not push a new comment to the pull request.

Alternatives

When looking at a solution it is always good to consider the alternatives and why they may or may not work for you.

As mentioned earlier we could have used pull request templates. These are a great guide however cannot enforce that the requester has read them. They also do not allow for dynamic content based on the changes in the pull request. The upside is they are very easy to implement.

It is possible to configure webhooks in Azure DevOps that are called when a pull request is created or updated. I have used this for other tasks however it is more complex to set up and maintain. It also requires the external system to have permission to interact with Azure DevOps if you wish to automate tasks such as posting comments.

Finally, another option is to use client-side git-hooks to check the changes before they are even pushed. This can work well in some cases, however you need to distribute these to all developers and they can be bypassed. Server-side git-hooks are another option but these are not supported by Azure DevOps.

Summary and Next Steps

In this post, we have looked at how we can automate posting comments on a pull request in Azure DevOps if our conditions are met. This can be used to remind requesters of requirements or to provide additional information to the reviewer. By using the Build Service account and the Azure DevOps API we can automate this process and ensure that it is consistent across all pull requests.

We could take this further by auto-resolving the comment. Alternatively, we could use the technique to validate the content of the file, adding extra reviewers or even blocking the pull request if the conditions are not met.

For more information about the DevOps rest API for pull requests see the official documentation.

Title Photo by Phil Hearing on Unsplash