Updated 02/04/21 Thanks to an astute reader who reached out to me informing me why the tests didn’t run any faster post cache and also provided the fix! Noted below.

In my previous post: https://markphelps.me/posts/migrating-from-travis-to-github-actions/ I described my journey migrating my open source Go project, Flipt, to GitHub Actions from Travis.

While this has turned out great, one thing that I did find missing using Actions was the ability to use a dependency cache to speed up builds. Travis has had this ability for awhile now and I previously took advantage of it when I used to run my builds for Flipt on Travis.

Well it turns out that the GitHub Actions team has quietly released the ability to save and restore cached dependencies using Actions!! The github/actions/cache repository explains how you can setup caching as part of your workflows.

I just discovered the cache action yesterday and was eager to try and set it up in my Flipt workflows.

Pre-Cache

Since Flipt uses Go Modules and does not vendor its dependencies, each build run using GitHub Actions would previously have to download all required modules from the Go Module mirror or GitHub itself:

This was pretty wasteful as the modules used don’t change all that much, but they were still being downloaded each time regardless. This also meant that the build was dependent on the speed that these modules could be downloaded, along with their availability.

Build Cache to the Rescue

Here’s the end result:

# Get values for cache paths to be used in later steps
- id: go-cache-paths
  run: |
    echo "::set-output name=go-build::$(go env GOCACHE)"
    echo "::set-output name=go-mod::$(go env GOMODCACHE)"

- name: Checkout
  uses: actions/checkout@v2

# Cache go build cache, used to speedup go test
- name: Go Build Cache
  uses: actions/cache@v2
  with:
    path: ${{ steps.go-cache-paths.outputs.go-build }}
    key: ${{ runner.os }}-go-build-${{ hashFiles('**/go.sum') }}

# Cache go mod cache, used to speedup builds
- name: Go Mod Cache
  uses: actions/cache@v2
  with:
    path: ${{ steps.go-cache-paths.outputs.go-mod }}
    key: ${{ runner.os }}-go-mod-${{ hashFiles('**/go.sum') }}

The required parts are:

1. Setting the path correctly
2. Choosing the right cache key

Setting the Path(s) Correctly

In order for the cache to work, you need to tell it which files you want cached. In our case we want to cache all of the Go modules that are used in our build. This brings up the important question:

Where are Go module source files stored?

I found the answer after some googling.

spoiler: modules are stored at $GOPATH/pkg/mod.

It seems the $GOPATH is still useful for something!

With this new knowledge, I first tried setting path to:

path: $GOPATH/pkg/mod

Which didn’t work. It seems that Actions wasn’t able to resolve $GOPATH here. Perhaps this was a misconfiguration or overlook of something on my part, but I looked for a different solution regardless.

I needed to find a solution that didn’t depend on environment variables to set the path correctly. I came upon this (long) issue in the action/setup-go repo which is used to well, setup Go in Actions.

This lead me to this comment which states that:

Recent Go versions already default this to $HOME/go

So, with that in mind I updated my action to set the path to:

path: ~/go/pkg/mod

And all was well!

Updated 02/04/21 I actually switched this over to use Actions outputs syntax so that I would always have the correct values for the Go build and mod cache. This is reflected in the above Gist.

Choosing the Right Cache Key(s)

The last part of setting up a successful caching strategy is to select the right cache key. The cache key is used to determine when we can rely on the files cached and when we cannot.

Simply put:

You want to use the cache whenever possible, but ‘bust’ the cache when dependencies change.

In Go’s case, we can depend on the go.sum file to tell us when any of our dependencies change, since it lists all modules with their versions, along with hashes of their contents.

Thankfully, the action/cache action provides a nice helper function hashFiles() which will create a MD5 checksum of the contents of whatever file or directory you give it. This is exactly what we need to generate a good hash key to use as our cache key since any change to the go.sum file would result in a new MD5 checksum value.

I updated the mod cache key to:

key: ${{ runner.os }}-go-mod-${{ hashFiles('**/go.sum') }}

This will generate cache keys that contain:

1. The OS that the action is running on (Linux, Windows, MacOS, etc)
2. The MD5 checksum of the go.sum file.

note: It is important that **/ prefixes go.sum in the argument to hashFiles() in order for Actions to find the file.

Results

After all was configured properly, I pushed a new commit and was able to see that actions/cache was working as expected! Note the new test time!

A couple things to note:

1. go test no longer had to download all the modules before running, which was the point of all this.
2. The build was slightly faster than the no cache build, however this small speedup could be attributed to many things beyond our control such as build machine usage during the time of our runs.
3. go test actually uses the Go build cache, not only the module cache when running tests. That’s why it’s important to also cache the Go build cache if you want to see speedup when running your tests. You can find where this cache lives by running the go env GOCACHE command. Thank you astute reader for pointing this out and contacting me!

Wrap Up

To me, the addition of caching to GitHub Actions means that it is fully capable of replacing most of the products from existing CI providers and I look forward to what the Actions team has in store for the future!