Branch Previews with Google App Engine and GitHub Actions

Branch Previews with Google App Engine and GitHub Actions

Leveraging GitHub Actions for easy-to-use, automated branch preview deployments

- 12 mins

:mega: This post is also available on dev.to!


Shortly after I returned to school, early in October 2019 I started working part-time remotely for Sumus, a property management company based out of Lethbridge, Alberta. My role was primarily as a software developer on a investor portal they wanted to build. I wasn’t starting from scratch - there was already a sizable codebase going, and a simple deployment set up on Google App Engine.

Right off the bat I had a number of tech-debt-related issues I wanted to address before I started developing new features, one of which was automating this deployment process. App Engine does not seem to have a great way of doing this outside of using their source control, so I decided to do this myself.

As I progressed on the automation of the App Engine deployment, I realized branch previews were not that much more of a hassle to set up, so I got those up and running as well - Heroku has a nice article about why staging environments are nice to have. This blog post will cover some of the work I did on this front, and hopefully give a good idea about how you can go about creating a similar setup for your own projects if you want. Our project consists of a React frontend serviced by a Node.js backend, so my post will lean a bit towards that particular setup, but should apply to a variety of different stacks.

Here’s a sneak peak of the end result:


The Problem

First off, a quick intro to App Engine. This was my first encounter with App Engine, so this won’t be the best rundown, but in a nutshell App Engine seems like a reasonably priced way to deploy your application in a serverless fashion with the flexibility to scale to your needs. It also offers nice out-of-the-box integration with Google’s other monitoring offerings, which is a nice plus.

Most of the official documentation seems to indicate that deployment happens primarily through:

The old process for deploying our application involved making sure I had all my credentials and stuff set up, and running:

gcloud app deploy app.yaml

There was no real way short of making sure I wasn’t axing someone else’s deployment or notifying everyone of what is currently active short of shooting a Slack announcement and hoping it’s seen and handled appropriately. I felt like the entire process would be more comfortable if it was automated and tied to source control, so that:

Solution

I didn’t actually start off with leveraging GitHub Actions for automating this process - my first iteration used CircleCI, where we run our tests, style checks, and and so on. This had the advantage of allowing me to stage deployments based on whether or not previous checks pass:

Unfortunately this was eating up a huge chunk of our pipeline minutes - as you can see in the image above, the appengine_stage job takes more than 97% of each build when a branch is configured to stage. This brought us uncomfortably close to hitting the CircleCI free tier, so I ended up moving it to GitHub Actions to split up our workloads.

Staging and Release

I first ran into the concept of branch previews working on the UBC Launch Pad website, where we leveraged Netlify’s great branch preview feature. It was a fantastic way to do some live testing and get feedback quickly, so I leveraged branch previews again during my time with nwPlus working on the nwHacks 2019 website, where I used a tool I worked on, ubclaunchpad/inertia, to quickly stage previews for the nwPlus design team to provide feedback on.

Now that I’m back to working on websites, I figured branch previews would come in useful here again (and they have so far!). To accomodate this, I introduced some extra steps to our deployment flow:

Versioning Frontends and Backends

A bit of a conundrum when deploying multiple versions of a multi-component service is making sure that they talk to the correct instances - for example, a branch preview deployment probably does not want to have its frontend talk to the backend of a different deployment if you are trying to demonstrate a new feature.

For service-to-service deployments, this is fairly straight forward - App Engine provides a variety of default environment variables you can use to interpret the appropriate backend to talk to. We can take advantage of how App Engine addresses un-promoted deployments:

https://${version}-dot-${project}.appspot.com # unpromoted
https://${project}.appspot.com                # promoted

In a multi-service setup, you also get an additional ${component} piece attached to the address:

https://${version}-dot-${component}-dot-${project}.appspot.com # unpromoted
https://${component}-dot-${project}.appspot.com # unpromoted

Then, following by using a “versioning” scheme of version=branch_name, we can easily determine where the desired service should be located, and point our requests to the correct address.

The only hurdle to this is for frontends. We have to know the version at buildtime, which sadly App Engine’s default build feature does not provide, so you’ll have to either:

The latter is probably best practice anyway, since you want to optimize your App Engine setup for fast instance start times, but in case you are running your builds in App Engine (our deployments were previously) this is a minor hurdle to be aware of.

Automation

GitHub Actions + App Engine

GitHub Actions is a pretty new product, which I guess is probably GitHub’s answer to GitLab’s CI/CD features. I’ve come to like it a lot more for anything outside of running your tests and whatnot, since it has a lot of interesting hooks and triggers based on normal GitHub activity that you can leverage, but for this example I won’t be using many of those. If you’re following along you might want to consult the official workflow syntax documentation.

Anyway, to get started I set up a staging workflow:

# .github/workflows/appengine-stage.yml
name: appengine-stage
on:
  push:
    branches:
    - master
    # insert branches to stage here

All I really want to do here is declare what branches I want to stage, and make staging additional branches just a matter of adding it to the configuration in your PR (and removing it when you’re done). I’m thinking of using PR labels for this, but haven’t figured out a good way to do it yet.

The first step is to actually grab your branch name. GitHub only provides you with the commit’s reference, which takes the form of:

refs/heads/${branch_name}

So we’ll want to extract it with a script:

steps:
- name: Extract branch name
  id: get_branch
  shell: bash
  run: echo "##[set-output name=branch;]$(echo ${GITHUB_REF#refs/heads/} | tr / -)"

There’s a couple of things going on here:

Then, in other steps you can access the branch name like so:

stage-${{ steps.get_branch.outputs.branch }}

All that’s really left to do is run the deployment.

- uses: actions-hub/gcloud@268.0.0
  env:
    APPLICATION_CREDENTIALS: ${{ secrets.GCLOUD_SERVICE_KEY }}
  with:
    args: app deploy client/app.yaml server/app.yaml --no-promote --quiet --version stage-${{ steps.get_branch.outputs.branch }}

The release workflow is very similar, except it runs on releases and generates version names based on the tagged version:

# .github/workflows/appengine-release.yml
name: appengine-release
on:
  release:
    types: [ published ]
jobs:
  release:
    # ...
    steps:
    - name: Extract tag name
      id: get_tag
      shell: bash
      run: echo "##[set-output name=tag;]$(echo ${GITHUB_REF#refs/tags/} | tr . -)"

I also have a separate workflow for pruning previews. Since previews are typically set up for pull requests, the prune job runs when pull requests close (ideally it should be on branch deletion, but there doesn’t seem to be a simple trigger for that at the moment):

# .github/workflows/appengine-prune.yml
name: appengine-prune
on:
  pull_request:
    types: [ closed ]
jobs:
  prune:
    # ...
    steps:
    # ...
    - uses: actions-hub/gcloud@268.0.0
      env:
        APPLICATION_CREDENTIALS: ${{ secrets.GCLOUD_SERVICE_KEY }}
      with:
        args: app versions delete stage-${{ steps.get_branch.outputs.branch }} --quiet

GitHub Deployments

As a bit of a stretch goal, I wanted to be able to see the deployments within the GitHub UI, just like with the Netlify branch previews. For example:

There’s another example of this at the top of this article. It’s mostly a small quality of life thing, but the more I thought about it the more I wanted it so…

Anyway, this feature is called “GitHub Deployments”. I tried a bunch of available Actions from the marketplace, for working with this, but for some reason I couldn’t really get any of them to work the way I wanted, which is to:

I was probably holding them all wrong, but after a few hours I just went ahead and wrote my own Action, bobheadxi/deployments, for doing exactly what I wanted. Then all I had to do was add a step before and after each of my workflows:

jobs:
  deploy:
    steps:
    - uses: bobheadxi/deployments@master
      id: deployment
      with:
        step: start
        token: ${{ secrets.GITHUB_TOKEN }}
        env: release-${{ steps.get_tag.outputs.tag }}
        transient: true
        desc: Setting up staging deployment for ${{ steps.get_tag.outputs.tag }}
    # ... as before
    - name: Update deployment status
      uses: bobheadxi/deployments@master
      if: always()
      with:
        step: finish
        token: ${{ secrets.GITHUB_TOKEN }}
        status: ${{ job.status }}
        env: ${{ steps.deployment.outputs.env }}
        env_url: https://release-${{ steps.get_tag.outputs.tag }}-dot-project.appspot.com
        deployment_id: ${{ steps.deployment.outputs.deployment_id }}

For pruning, I needed to be able to go and deactivate all deployments associated with the preview environment. Since I owned the action I just added the feature.

- uses: bobheadxi/deployments@master
  with:
    step: deactivate-env
    token: ${{ secrets.GITHUB_TOKEN }}
    env: stage-${{ steps.get_branch.outputs.branch }}
    desc: Deployment was pruned

And that was it! As a bonus, notifications for these deployments show up in Slack via the GitHub integration:

Wrapup

There’s definitely a bunch of caveats in this approach, and if the resources are available to you it might be easier to use a platform like Heroku to do all this hard work for you. That said, this was a fun hack and has made staging previews for the team to assess and rolling out releases a lot safer-feeling and less of a hassle.

Robert Lin

Robert Lin

Enthusiastic eater of lunches. Please feel free to reach out to me at robert@bobheadxi.dev!

rss facebook twitter github youtube tumblr mail spotify lastfm instagram linkedin google google-plus pinterest medium vimeo stackoverflow reddit quora quora