1

I'm having a problem with JSON format produced by docker compose ps --format json command.

Here's my example docker-compose.yml file:

version: '3.5'

services:
  nginx:
    image: nginx
    links: 
      - php

  php:
    image: php:fpm

I start my containers by docker compose up -d. When I run docker compose ps it produces the correct output:

NAME                  IMAGE     COMMAND                                          SERVICE   CREATED         STATUS         PORTS
test_docker-nginx-1   nginx     "/docker-entrypoint.sh nginx -g 'daemon off;'"   nginx     4 minutes ago   Up 4 minutes   80/tcp
test_docker-php-1     php:fpm   "docker-php-entrypoint php-fpm"                  php       4 minutes ago   Up 4 minutes   9000/tcp

I wanted to customize the above output a little bit by using --format json and jq tool. However it appeared that docker compose ps --format json produces an incorrect output:

{"Command":"\"/docker-entrypoint.sh nginx -g 'daemon off;'\"","CreatedAt":"2023-09-20 14:32:55 +0200 CEST","ExitCode":0,"Health":"","ID":"956ee02dc6beb2b32fb15dac9cfb3be37bf09bcfd5fe05d9c23b31ba2fbc2108","Image":"nginx","Labels":"com.docker.compose.project.working_dir=/var/www/test_docker,com.docker.compose.version=2.21.0,com.docker.compose.config-hash=0483cc81f74ebb9392330ce8680a4d528986eecb650af475b34bdfe90bfa1dc3,com.docker.compose.container-number=1,com.docker.compose.depends_on=php:service_started:true,com.docker.compose.image=sha256:f5a6b296b8a29b4e3d89ffa99e4a86309874ae400e82b3d3993f84e1e3bb0eb9,com.docker.compose.project=test_docker,com.docker.compose.project.config_files=/var/www/test_docker/docker-compose.yml,maintainer=NGINX Docker Maintainers \[email protected]\u003e,com.docker.compose.oneoff=False,com.docker.compose.service=nginx","LocalVolumes":"0","Mounts":"","Name":"test_docker-nginx-1","Names":"test_docker-nginx-1","Networks":"test_docker_default","Ports":"80/tcp","Publishers":[{"URL":"","TargetPort":80,"PublishedPort":0,"Protocol":"tcp"}],"RunningFor":"6 minutes ago","Service":"nginx","Size":"0B","State":"running","Status":"Up 6 minutes"}
{"Command":"\"docker-php-entrypoint php-fpm\"","CreatedAt":"2023-09-20 14:32:53 +0200 CEST","ExitCode":0,"Health":"","ID":"9ace40ff342457bceeff0433fd882cf65e819ac85cef4216df8815b8630bb7df","Image":"php:fpm","Labels":"com.docker.compose.depends_on=,com.docker.compose.image=sha256:c6b042c2a60f05833e6b5041cb5fbd83ef70f42f9b510decb364dc7306dd8da2,com.docker.compose.project.config_files=/var/www/test_docker/docker-compose.yml,com.docker.compose.config-hash=816ffa2d0d86f46676ce2aba77f5d4ded1e538e99c78aee30db77ba4deb5835e,com.docker.compose.container-number=1,com.docker.compose.oneoff=False,com.docker.compose.project=test_docker,com.docker.compose.project.working_dir=/var/www/test_docker,com.docker.compose.service=php,com.docker.compose.version=2.21.0","LocalVolumes":"0","Mounts":"","Name":"test_docker-php-1","Names":"test_docker-php-1","Networks":"test_docker_default","Ports":"9000/tcp","Publishers":[{"URL":"","TargetPort":9000,"PublishedPort":0,"Protocol":"tcp"}],"RunningFor":"6 minutes ago","Service":"php","Size":"0B","State":"running","Status":"Up 6 minutes"}

It is formatted as {...} {...}, but I would rather expect something like [{...}, {...}].

I tested the same on other machines and it works perfectly OK there, so I assume it must be something with my installation / version / configuration.

Some details:

$ docker version
Client: Docker Engine - Community
 Version:           24.0.6
 API version:       1.43
 Go version:        go1.20.7
 Git commit:        ed223bc
 Built:             Mon Sep  4 12:31:44 2023
 OS/Arch:           linux/amd64
 Context:           default

Server: Docker Engine - Community
 Engine:
  Version:          24.0.6
  API version:      1.43 (minimum version 1.12)
  Go version:       go1.20.7
  Git commit:       1a79695
  Built:            Mon Sep  4 12:31:44 2023
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.24
  GitCommit:        61f9fd88f79f081d64d6fa3bb1a0dc71ec870523
 runc:
  Version:          1.1.9
  GitCommit:        v1.1.9-0-gccaecfc
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

$ docker compose version
Docker Compose version v2.21.0

$ cat ~/.docker/config.json 
{
    "auths": {
        [redacted]
    }
}

Below is the output for the same docker-compose.yml file, ran on another server. It has a correct format, but in the same time has a few differences (e.g. no Labels or RunningFor properties, Created vs. CreatedAt, etc.)

$ docker compose ps --format json
[{"ID":"e51334d871cba0ea4cdcd5003b9293b9de7ae01cef0cf95f1df3949f8762066f","Name":"docker_test_nginx_1","Image":"nginx","Command":"/docker-entrypoint.sh nginx -g 'daemon off;'","Project":"docker_test","Service":"nginx","Created":1695218753,"State":"running","Status":"Up 5 hours","Health":"","ExitCode":0,"Publishers":[{"URL":"","TargetPort":80,"PublishedPort":0,"Protocol":"tcp"}]},{"ID":"438c39a82b6c102eb1b4b0923696b8330a7a2fbcacacb645e7dff69589ea61be","Name":"docker_test_php_1","Image":"php:fpm","Command":"docker-php-entrypoint php-fpm","Project":"docker_test","Service":"php","Created":1695218749,"State":"running","Status":"Up 5 hours","Health":"","ExitCode":0,"Publishers":[{"URL":"","TargetPort":9000,"PublishedPort":0,"Protocol":"tcp"}]}]

Is there anything I should check to make it to produce a properly formatted JSON? Any help will be appreciated!

Improve this question

2 Answers 2

Reset to default
5

Unfortunately, this is an expected behavior and breaking change introduced in docker compose 2.21. We are currently dealing with the same issue..

See https://github.com/docker/compose/issues/10958 for more info.

In the same vein from boppy's answer, here's one possible solution that solves both docker compose ps --format json output (no matter which docker compose's version basically):

BEFORE:

docker compose ps -a --format json | jq <...any additional transfo, if any...>

AFTER (the fix):

docker compose ps -a --format json | jq -sc '.[] | if type=="array" then .[] else . end' | jq -s | jq <...any additional transfo, if any...>

For sure, this adds more processing, but in reality, I haven't seen too much impact so far, this will depends on the number of records that you are dealing with.

Improve this answer
Sign up to request clarification or add additional context in comments.

1 Comment

Actually, you don' need shell pipes. You could use jq's, like ... | jq -s '.[] | if type=="array" then . else [.] end | <your additional filters>
3

jq offers a switch to merge multi-input JSON feeds into an array. It's the -s (slurp) option:

docker compose ps --format json | jq -s .
Improve this answer

5 Comments

Thanks for the answer! This really helps with the output customization issue. However I'd also like to find a solution to the base problem.
I assume that internally every container will create one call to a sub-routine to provide information. This info is not gathered, but the output is generated after each line. But as I'm not 100% certain, I hope to get corrected by s.o. with deeper architectural knowledge in docker ;)
This doesn't seem to be a case. Like I mentioned, I ran the exact same test on other computers and it produced perfectly correct [{...}, {...}] output.
Are you sure? I cannot reproduce docker compose to return an array on any machine. BUT I can reproduce that docker-compose in fact does! The one WITH the dash is an external script, while the white-space one is the new, docker-binary internal thingy...
I am sure, yes. I will add the output to my original post.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.