5

I have a docker Dockerfile in which I want to clone a Git repo. If the Git repo is not updated I would like Docker to use its cache. If the repo is updated then building the Docker Image should use the Git repo updates.

Based on this stackoverflow question:

How to prevent Dockerfile caching git clone

I have added the following to my Dockerfile:

#invalidate cache if micropython is updated
ADD https://api.github.com/repos/dwjbosman/micropython/git/refs/heads/master /root/micropython_version.json

RUN git clone --recurse-submodules https://github.com/dwjbosman/micropython.git /root/micropython && \
    cd /root/micropython && \
    git submodule update

I use a makefile with targets to build the image. First I pull the previous image from docker hub:

docker pull $(REGISTRY)/$(DOCKER_IMAGE)

and then I use this pulled image as a cache for the new build:

docker build --cache-from $(DOCKER_IMAGE) -t $(DOCKER_IMAGE) 

Afterwards the new image is pushed to Docker hub. I am building the images in a CI environment (Gitlab). If I check the logs the ADD step always invalidates the cache (and the checksum changes) even if no updates were done in the Git repo.

I have checked the output of "https://api.github.com/repos/dwjbosman/micropython/git/refs/heads/master". The contents does not seem to change.

Is there some hidden property (time perhaps?) which still changes?

4
  • 1
    I don't have issues running this locally with the classic builder (though buildkit has issues that would break the build). Are you sure the layers exist in the builder from the previous build, or is Gitlab using ephemeral build nodes? Commented Oct 28, 2019 at 1:04
  • I updated the question and added the image build steps Commented Oct 28, 2019 at 8:25
  • Maybe something with the permission of the file, like in this one?: devops.stackexchange.com/a/4812 Commented Oct 29, 2019 at 16:06
  • How far down in your Dockerfile is the ADD line? Could it be that your layers are not being reused because --cache-from doesn't include the the base image? -- github.com/moby/moby/issues/20316#issuecomment-358260810 Commented Apr 29, 2020 at 10:04

0

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.