How to Configure Cgroup V2 limits on docker-compose containers

Question

I want to configure cgroups V2 resource limitation on a Docker-Compose container. How do I do this?

Answer 1

INTRO:

I eat my own dog food and have been using this solution for a few years on publicly-facing docker-compose containers. So it's sufficiently battle-tested and the test evidence at the end proves that it does.

This 2-step solution takes only 2 minutes to config Cgroup V2 on your Docker-Compose containers

REQUIREMENTS:

Just (2) requirements:

• Docker Host OS: A SystemD host which supports Cgroup V2. I use Ubuntu 24.04.1 LTS with this solution
• docker-compose version: A docker-compose file formatted using v3.9. But any version which supports the croup_parent directive should work

Validated on earlier versions than those above, but should be portable without modification on any SystemD Linux distro supporting Cgroup v2

Note that config is container in-specific!!!: Although I run the solution on Alpine Linux & Ubuntu AARCH64 containers, it could be on other arch Docker containers- there is NO config or dependencies at the container level.

STEP 1: HOST Config:

Create a file called "docker.slice" in the following path:

/etc/systemd/system/docker.slice

Below is a general-purpose config; tweak to sensible values for your specific use-case:

[Unit]
Description=Slice that limits docker resources 
Before=slices.target

[Slice]
MemoryAccounting=true
CPUAccounting=true
TasksMax=30%
AllowedCPUs=1-3
# NOTE: If slice restricted to 3 of 4 cores, max avail. CPUQuota= 300%, NOT 400%
CPUQuota=300%
# CPUWeight value range: 1-10000
CPUWeight=8000
MemoryLow=1G
MemoryHigh=2G
MemoryMax=2G
# IOWeight value range: 1-10000
IOWeight=5000

Read your new changes:

    systemctl daemon-reload

STEP 2 DOCKER-COMPOSE Config:

In the docker-compose file, in the cgroup_parent directive, specify the name of the systemd file created in Step 1: docker.slice

    cgroup_parent: docker.slice

Jason formatting: I'm inserting (4) spaces before adding the cgroup_parent directive on the 5th space in my own docker-compose file.

Restart the docker-compose containers.

Now when you execute:

systemctl status docker.slice

You'll see your container running under Cgroups resource control. YourCgroups V2 config for your docker-compose container is finished. That's it!

(OPTIONAL STEP) CONFIG- CONTAINER:

Rebuild container with stress-ng to validate resource limitation is working. Delete stress-ng from your container build process after testing completed.

But if you're happy the solution works as proposed after reviewing my test evidence below, you can just skip this step.

TESTING:

Stress Testing:

We'll use stress-ng inside the container to prove our Cgroup limits are being enforced. There are a gazillion stress-ng commands we can use to test various things, but I'll just offer a single test as thin't meant to be a 'stress-ng` HowTo.

Enter the container :

docker exec -it <Container ID from "docker ps"> sh

Or if bash is available inside your container (need to expressly add it in Alpine Linux):

docker exec -it <Container ID from "docker ps"> bash

Once inside the container we're testing for Cgroup resource limitation, execute the stress test:

stress-ng --cpu 0 --iomix 4 --vm 16 --vm-bytes 4G --timeout 180s --metrics

Note that we set our resource limitation for memory to a value of a max of 2G in docker.slice. Thus, we'll cane the memory by using a value of 4G in the stress-ng command above. If all is configured correctly, we should never see anything above 2G.

Monitoring Resource Limitation:

In a new terminal window/tab, execute the following monitoring commands to validate limitations are being enforced correctly.

NOTE: Below commands are to be executed from THE HOST - NOT inside the container!

systemctl status docker.slice
systemd-cgtop

Some screen grabs are offered below to evidence our Cgroups V2 config is working correctly. Please note however that the output after all the stress-ng commands is truncated for illustrative clarity.

systemctl status docker.slice:

systemd-cgtop:

dmesg |grep -i oom Output:

Remark oom-kill:constraint=CONSTRAINT_MEMCG. Cgroups is clobbering stress-ng when it tries to pass our memory threshold:

CONCLUSION:

It took me a while to get Cgroups working in docker-compose due to the lack of any comprehensive HowTo. Since I was porting OpenLDAP to AARCH64, I figured it was worth taking a moment to document the process I follow to save others from pulling their hair out of their heads getting it working. ;-)

Anyhoo, HTH other enterprise folks who were searching in vain for some config help-

-Terrence Houlahan