docker-compose external build context with relative dockerfile

Question

What I'm trying to accomplish is this: I want to cache the current users ~/.aws directory inside a container so that I can use them during the build of another container.

I have the following docker-compose.yml:

version: "3.7"
services:
  worker:
    depends_on:
      - aws
  aws:
    build:
      context: ~/.aws
      dockerfile: ./ctx.dockerfile
      args:
        - workdir=/root/.aws

These are the contents of ctx.dockerfile:

FROM alpine:3.9

ARG workdir
WORKDIR ${workdir}

COPY . .

And in my worker service Dockerfile I have the following:

...
COPY --from=aws_ctx:local /root/.aws /root/.aws
...

The Problem

docker compose isn't treating the dockerfile path in the aws service as relative to the docker-compose.yml, it is instead assuming it is relative to the context path. Is there anyway I can have docker-compose load the ctx.dockerfile from same directory as docker-compose.yml AND set the context the way that I am?

I'm up for changing my approach to the problem, but I have a few contstraints:

1. any solution must be workable on Windows, OSX, and Linux
2. any solution must only require docker and/or docker-compose, I can't run a shell script beforehand

Answer 1

Is there anyway I can have docker-compose load the ctx.dockerfile from same directory as docker-compose.yml AND set the context the way that I am?

AFAIK: No, there isn't.

Everything that the Dockerfile interacts with on build time must be in the defined context. So, you need .aws and the current folder where the docker-compose.yml etc. lives to be in the same context, i.e. the context would need to be the highest level of your relevant directory structure and then you would have to define relative paths to the files you need (the Dockerfiles and .aws).

Maybe you could set /home/$USER as your build context (or even higher level, depending on where your Dockerfiles etc. live), but then you would also have to create a .dockerignore file and ignore everything in the context besides .aws and the current folder... As you can see, this would be a mess and not very reproducible.

I would suggest to use a volume instead of COPYing the ~/.aws folder inside your container.

Example:

nico@lapap12:~$ ls -l ~/.aws
total 0
-rw-r--r-- 1 nico nico 0 May 22 17:45 foo.bar

docker-compose.yml:

version: "3.7"

services:
  allinone:
    image: alpine:latest
    volumes:
      - ~/.aws:/tmp/aws:ro
    command: ls -l /tmp/aws

nico@lapap12:~/local/so$ docker-compose up
Creating so_allinone_1 ... done
Attaching to so_allinone_1
allinone_1  | total 0
allinone_1  | -rw-r--r--    1 1000     1000             0 May 22 15:45 foo.bar
so_allinone_1 exited with code 0

You could go from there and copy the content of /tmp/aws to /root/.aws if you want to change this folder's content in the container, but don't want to touch it on the actual host.