1

I'm doing a loop over ffmpeg to convert a bunch of videos, and I would like to highlight some terms in ffmpeg's output. But the solution I tried always made the live progress of the conversion disappear.

What I have tried is mainly to redirect stderr to stdout, and grep or ack like this (ffmpeg's command is simplified for readability) :

ffmpeg -i input.mp4 ouput.mkv 2>&1 | ack --passthru --color "pcm_s16le|aac"

or

ffmpeg -i input.mp4 ouput.mkv 2>&1 | grep -E "pcm_s16le|aac|$"
(note the |$ hack that allow to match everything without coloring it, acting like a passthrough)

And what I mean by live progress info is the line that look like this :

frame=190 fps=3.7 q=-0.0 size=308kB time=06:08.66 bitrate=677kbits/s speed=0.17x

So, is there a way to highlight some words that output, and keep the live progress ?

Improve this question
2
  • Can you verify that your patterns are correctly defined? If you save the ffmpeg output to a file and then run the ack or grep commands, does it produce the output you are expecting? Commented Feb 13, 2019 at 8:51
  • Yes, the patterns work. Even in the console I see the right words highlighted. The issue is that I don't know how that progress info seems to not be seen by grep nor ack. You can see it yourself with ffmpeg 2>&1 | grep -E "ffmpeg|$" for example. Commented Feb 13, 2019 at 12:18

2 Answers 2

Reset to default
2

The problem you're seeing is almost certainly due to the progress info being printed with CR (carriage-return, ASCII 13) line endings, instead of the traditional Unix LF (line-feed, ASCII 10). This is done to allow each new progress update to overprint the last.

Unfortunately, this means the entire set of progress updates looks to grep, ack and all other line-oriented tools as a single line. That's why they seem to "lock up" and print nothing until ffmpeg finishes, and I can't think of a similar highlighting tool that operates on anything but a standard Unix line basis.

You could try something like:

ffmpeg ... | tr \\015 \\012 | grep ...

which transforms all CRs to LFs. This lets grep et al see each progress update as a separate line, at the cost of also seeing each update as a separate line, filling your terminal:

frame=   85 fps=0.0 q=28.0 size=       0kB time=00:00:03.79 bitrate=   0.1kbits/s dup=1 drop=0 speed=7.55x    
frame=  129 fps=129 q=28.0 size=       0kB time=00:00:05.65 bitrate=   0.1kbits/s dup=1 drop=0 speed=5.64x    
frame=  172 fps=111 q=28.0 size=     256kB time=00:00:07.42 bitrate= 282.6kbits/s dup=1 drop=0 speed=4.81x    
frame=  213 fps=104 q=28.0 size=     512kB time=00:00:09.17 bitrate= 457.3kbits/s dup=1 drop=0 speed=4.49x    
frame=  254 fps= 99 q=28.0 size=     512kB time=00:00:10.85 bitrate= 386.3kbits/s dup=1 drop=0 speed=4.22x    
frame=  295 fps= 96 q=28.0 size=     768kB time=00:00:12.56 bitrate= 500.7kbits/s dup=1 drop=0 speed=4.08x    
frame=  333 fps= 93 q=28.0 size=    1024kB time=00:00:14.14 bitrate= 593.1kbits/s dup=1 drop=0 speed=3.94x    
frame=  382 fps= 93 q=28.0 size=    1024kB time=00:00:16.19 bitrate= 518.1kbits/s dup=1 drop=0 speed=3.93x    
frame=  428 fps= 93 q=28.0 size=    1280kB time=00:00:18.11 bitrate= 579.0kbits/s dup=1 drop=0 speed=3.92x    
frame=  473 fps= 92 q=28.0 size=    1536kB time=00:00:20.00 bitrate= 628.9kbits/s dup=1 drop=0 speed=3.91x    
frame=  519 fps= 92 q=28.0 size=    1536kB time=00:00:21.93 bitrate= 573.8kbits/s dup=1 drop=0 speed= 3.9x    
frame=  567 fps= 92 q=28.0 size=    1792kB time=00:00:23.91 bitrate= 613.9kbits/s dup=1 drop=0 speed=3.89x    
frame=  601 fps= 90 q=28.0 size=    2048kB time=00:00:25.32 bitrate= 662.6kbits/s dup=1 drop=0 speed= 3.8x    
frame=  637 fps= 89 q=28.0 size=    2304kB time=00:00:26.83 bitrate= 703.3kbits/s dup=1 drop=0 speed=3.75x    
frame=  684 fps= 89 q=28.0 size=    2304kB time=00:00:28.77 bitrate= 655.9kbits/s dup=1 drop=0 speed=3.74x    
frame=  720 fps= 84 q=-1.0 size=    2924kB time=00:00:30.01 bitrate= 798.0kbits/s dup=1 drop=0 speed= 3.5x    
...

You'd probably also want to disable stdio buffering between the components in this pipeline for maximum responsivness:

stdbuf -o0 ffmpeg ... 2>&1 | stdbuf -i0 -o0 tr \\015 \\012 | stdbuf -i0 -o0 grep ...
Improve this answer
7
  • That's a nice idea. Doing it with sed would allow to do the reverse after the grep, and keep the \r. Except doing a grep or ack after the tr command is really weird. Looks like flush issues. And I tried with sed, it didn't output anything :/ Commented Feb 13, 2019 at 15:00
  • @Cqoicebordel "And I tried with sed, it didn't output anything" Well, sed is also line-oriented. so it can't do tr's job in this scenario. Commented Feb 13, 2019 at 15:47
  • @Cqoicebordel As for "flush issues", I just added a note to my answer about disabling stdio buffering. Commented Feb 13, 2019 at 16:00
  • The flush issues are indeed solved. But the grep command don't seem to work after that. the ack one works, but I can't do anything after it, even just adding > file.txt doesn't work (empty file). So, there is something weird, and I don't know what is it :/ Commented Feb 13, 2019 at 16:30
  • @Cqoicebordel What does your command line look like? I tested with stdbuf -o0 ffmpeg -i test.mkv test.mp4 2>&1 | stdbuf -i0 -o0 tr \\015 \\012 | stdbuf -i0 -o0 grep -E --color=always "pcm_s16le|aac|$" | stdbuf -i0 tee file.txt. It displayed the entire ffmpeg output very responsively, and file.txt even captured all of grep's highlighting. Commented Feb 13, 2019 at 16:41
0

  • Adrian's answer is very helpful, but I lost the dialogue, the prompt 

    File 'output.mkv' already exists. Overwrite ? [y/N]

  • With the tool vialog, that is using xterm and its log file, I could make it work all the way

    stdbuf -o0 vialog ffmpeg -t 10 -i input.mp4 output.mkv |
stdbuf -i0 -o0 tr -s '\015' '\012' |
stdbuf -i0 -o0 grep --color -e 'time' -e 'speed' -e 'Audio:' -e 'Video:' -e '$'

    • with the dialogue in the xterm window

    • with high-lighting in the terminal window, where vialog is started (which makes it a console window of xterm),

  • You find the tool vialog via this link,

    Wake me up when a [slow] command line process wants my attention?

    You may want to modify vialog to do more exactly what you want.

Improve this answer

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.