Reduce your build time by leveraging the cache on Google Cloud
Use docker cache mechanisms to save build time and repeated processes.
Following on from my post over at the Mechanical Rock blog, I thought I’d add in another post on how to leverage cloudbuild and GCR Google Container Registry to reuse docker layers to get fast feedback cycles.
In this case, I’ll be referring to my open source application, Secret Santa which is a ruby on rails based application. I’m currently in the process of moving it from Heroku to GCP. I’m currently evaluating CloudRun and Kubernetes for it, but that is a post for another time.
Originally my cloudbuild file started out like this:
steps:
- name: 'gcr.io/cloud-builders/docker'
args: [
'build', '--file', 'Dockerfile-prod',
'-t', 'gcr.io/$PROJECT_ID/secretsanta-web:latest',
'.'
]
images:
- 'gcr.io/$PROJECT_ID/secretsanta-web'
And the cloudbuild console output reported a 3 minute, 28 second build.
ID CREATE_TIME DURATION SOURCE IMAGES STATUS
cd674c50-04da-4c41-9ca4-db3829b0dc25 2019-04-19T02:37:33+00:00 3M28S gs://secretsanta-web_cloudbuild/source/1555641444.5-0de18f77531e47cab03ba1ae358b14ea.tgz gcr.io/secretsanta-web/secretsanta-web (+1 more) SUCCESS
By injecting another step before the container packaging starts, and using the docker --cache-from
command we can speed up the build by providing a cache for the container to build from rather than re-doing the same, unchanged layers each time.
My updated cloudbuild yaml file now has two steps. It pulls in the container, then uses it as a cache image and runs the build. If it can’t pull the image for whatever reason, the step still completes, with a zero exit code, so the build will still run, it just won’t be cached. This is denoted by the || exit 0
in the arguments provided to the docker command.
steps:
- name: 'gcr.io/cloud-builders/docker'
entrypoint: 'bash'
args:
- '-c'
- |
docker pull gcr.io/$PROJECT_ID/secretsanta-web:latest || exit 0
- name: 'gcr.io/cloud-builders/docker'
args: [
'build', '--file', 'Dockerfile-prod',
'--cache-from', 'gcr.io/$PROJECT_ID/secretsanta-web:latest',
'-t', 'gcr.io/$PROJECT_ID/secretsanta-web:latest',
'.'
]
images:
- 'gcr.io/$PROJECT_ID/secretsanta-web'
We’ve now got a build time of around 27 seconds. So we’ve saved more than three minutes just by tweaking a few lines of our build script. Awesome.
ID CREATE_TIME DURATION SOURCE IMAGES STATUS
a651a8d3-d657-49d2-b5bb-4e0e84e00e3b 2019-04-19T02:45:29+00:00 27S gs://secretsanta-web_cloudbuild/source/1555641921.26-683b498d119449c29cfcf490e3e5c552.tgz gcr.io/secretsanta-web/secretsanta-web (+1 more) SUCCESS