efficient way to compose all frames in a video to a single image

Question

I've got a 720p video file that is about 20 minutes long. I want to compose each frame in the video using a single operator (e.g. lighten, darken, etc.) to produce a single image output.

My thought was that I could use Imagemagick's convert function to do this, using something like this:

convert video.mp4 -compose Lighten output.jpg

But after running for a few minutes it runs out of memory (and I've got a lot of memory and I've set policy to allow most of it to be used). Under the hood it appears to be calling ffmpeg, but whatever is happening it seems to be very inefficient, requiring tens of gigabytes of RAM for what should be loading a couple 720p frame into RAM at a time (one for the composite, and one for the new frame).

What's a quicker and more efficient way to compose video into an image?

Answer 1

This can be done effectively by ffmpeg, although it requires knowing exactly how many frames need to be combined. A tblend filter must be set up to combine adjacent frames; each filter will divide the number of frames in half.

In my case, I had 72,000 frames (20 minutes @ 60fps). I picked 16 stages as this will divide the video down by 65536. The tblend documentation says:

By default, the output terminates when the longest input terminates.

From this, I would expect that I should divide by 131072, as the last division should result in a short input which should be combined with the full length one properly. However, using 17 stages results in the output Output file is empty, nothing was encoded so I dropped it to 16 at which point I got one image output.

The final command was:

ffmpeg -i $I -vf "tblend=lighten,framestep=2,tblend=lighten,framestep=2,tblend=lighten,framestep=2,tblend=lighten,framestep=2,tblend=lighten,framestep=2,tblend=lighten,framestep=2,tblend=lighten,framestep=2,tblend=lighten,framestep=2,tblend=lighten,framestep=2,tblend=lighten,framestep=2,tblend=lighten,framestep=2,tblend=lighten,framestep=2,tblend=lighten,framestep=2,tblend=lighten,framestep=2,tblend=lighten,framestep=2,tblend=lighten,framestep=2,setpts=1*PTS" -f image2 $O.png

Where $I is the input file and $O is the input file stripped of its extension.

For comparison with convert, using ffmpeg using the above command on a 20 minute video file took slightly over a minute and negligible memory to process.