transcode movies efficiently when 2 GPUs are available

Question

I have a Kodi video library where each movie is in its own folder because I had to place .nfo files with links to TMDB in each folder to ensure correct identification. The movies are in their original Blu-ray resolution, stored on a drive shared over Samba on a gigabit LAN.

I need to transcode all these files with FFmpeg to max. 1334×750 px.

Setup: Intel Core i-7 3930K, 2 x NVIDIA GTX 980 6 GB GPU, KDE on Debian Testing, custom FFmpeg compiled with h264_nvenc enabled. Although the GPUs are connected with an SLI bridge, they're not in SLI mode due to NVIDIA's Linux driver limitation (v550.163.01). GPU 0 is used by the system, GPU 1 is idle.

How to do it efficiently?

Answer 1

I've tried combinations of hwaccel and h264_nvenc.

• hwaccel is for decoding
• all streams other than video (i.e. audio and subtitles) are just copied without transcoding
• scale=1334:-2 tells the filter to scale the input to 1334 pixels wide and to calculate the height divisible by 2 while maintaining original aspect ratio.
• as the question is about GPU acceleration, I've omitted the classic ffmpeg invocation, processed purely on the CPU.

Results

1. Slow: CPU load 90%, transcode speed 4x.

   ffmpeg -hwaccel cuda -i "in.mkv" -map 0 -codec:a copy -codec:s copy \
   -filter:v scale=1334:-2 "out.mkv"
   

2. Faster: CPU load 30%, transcode speed 11x

   ffmpeg -hwaccel cuda -i "in.mkv" -map 0 -codec:a copy -codec:s copy \
   -codec:v h264_nvenc -gpu 1 -filter:v scale=1334:-2 "out.mkv"
   

3. Fastest: CPU load 80%, transcode speed 12x

   ffmpeg -i "in.mkv" -map 0 -codec:a copy -codec:s copy \
   -codec:v h264_nvenc -gpu 1 -filter:v scale=1334:-2 "out.mkv"
   

Script

#!/bin/bash

# Rescale files in subdirectories (Kodi library) using 1334 as driving width 
# from which height is calculated. Output format is mkv.

for directory in * ; do
  [[ -d "$directory" ]] || continue
  for file in "$directory"/*.{mkv,mp4,mpg,webm} ; do
    [[ -e "$file" ]] || continue
      ffmpeg -i "$file" -map 0 -codec:a copy -codec:s copy -codec:v h264_nvenc -gpu 1 -filter:v scale="1334:-2" "$file"-w1334.mkv
  done
done

Answer 2

Example modified script writing to local /tmp, then moving to the server if the transcode was successful:

#!/bin/bash

# Rescale files in subdirectories (Kodi library) using 1334
# as driving width from which height is calculated.
# Output format is mkv.

for directory in * ; do
  [[ -d "$directory" ]] || continue
  for file in "$directory"/*.{mkv,mp4,mpg,webm} ; do
    [[ -e "$file" ]] || continue
      filename=$(basename "$file")
      output_file="/tmp/${filename%.*}-w1334.mkv"

      ffmpeg -i "$file" -map 0 -codec:a copy \
        -codec:s copy -codec:v h264_nvenc -gpu 1 \
        -filter:v scale="1334:-2" "$output_file" &&
            mv "$output_file" "$directory"
  done
done

I also modified the script to avoid horizontal scrolling on SE due to long lines.

I find this also makes long and complicated command lines like those often used with ffmpeg to be much more readable and easier to understand and debug and modify.

Another bash script readability tip with commands like ffmpeg (and rsync and many others where long command lines are common) is to put some or all of the args in an array (do this before the main loop, it only needs to be done once, not repeatedly) and use "${arrayname[@]}" in the ffmpeg command line, e.g.

#!/bin/bash

# Rescale files in subdirectories (Kodi library) using 1334
# as driving width from which height is calculated.
# Output format is mkv.

ffargs=(-map 0
        -codec:a copy
        -codec:s copy
        -codec:v h264_nvenc
        -gpu 1
        -filter:v
        scale="1334:-2"
       )

for directory in * ; do
  [[ -d "$directory" ]] || continue
  for file in "$directory"/*.{mkv,mp4,mpg,webm} ; do
    [[ -e "$file" ]] || continue
      filename=$(basename "$file")
      output_file="/tmp/${filename%.*}-w1334.mkv"

      ffmpeg -i "$file" "${ffargs[@]}" "$output_file" &&
          mv "$output_file" "$directory"
  done
done

BTW, I'm not sure why you're bothering to check if the input file exists - the script is looping over filenames found in the directory, so it's pretty much guaranteed to exist. If you need that at all, it might be more useful to check that it's a regular (i.e. not a symlink or pipe or whatever), non-empty file that is readable by your uid rather than just testing for existence, e.g. instead of [[ -e "$file" ]]:

[[ -f "$file" && -s "$file" && -r "$file" ]] || continue

And maybe also check if the directory is both readable & writable by you:

[[ -d "$directory" && -r "$directory" && -w "$directory" ]] || continue