Most source video is 8-bit YUV 4:2:0, but that is not a good format for image processing due to sub-sampled chroma. Almost no routines support processing on that (not frei0r or Qt, only sometimes libavfilter). One way to avoid this is to do more processing in 8-bit YUV 4:2:0, but the new standard in video is 10-bit HDR. People can strive to port routines to be more pixel format independent, which is very difficult to do while also optimizing performance. If you want to improve performance of libswscale, then ask FFmpeg, and good luck as it is already quite optimized. If you want to use GPU and OpenColorIO for pixel format conversion, well, that is an enhancement request. I hope to spend a fair amount of time in 2020 to improve image processing performance. OpenColorIO might be a part of that depending on the rest of the pipeline running on GPU to reduce the costly transfer of heavy image data transfer between CPU and GPU memory. When profiling, it is often more useful to look for surprises and not the areas that you already know are costly. For example, should sws_init_context() be as high as it is?