[Feature Request] Add support for higher bitdepth codecs to the resize capture filter

[mikekoetter]

The resize capture filter seems to have select few codecs it supports, though it doesn't appear to be documented anywhere, the resize_utils suggests it would only support RGB, RGBA, I420, and UYVY/YUYV.

I'm not familiar with the OpenCV code here, would it be difficult to extend this to support higher bitrate codecs such as R10K?

[MartinPulec]

Conversions are now deployed for resize capture filter (commit d1a8e93 and earlier) , I believe it should work for R10k/R12L. It is converted to 16-bit RGB, which OpenCV supports but not back to the original pixel format – it is assumed, that it is later compressed, anyways. In case you'd send uncompressed and 16-bit is undesirable, you'd need to append the conversion back explicitly, eg. `-F resize:1280x720,change_pixfmt:R10k.

Second note - for more than 8 bit formats, currently only 16-bit RGB is used to interact with OpenCV, not sure if OpenCV consumes something like Y416 (I may look tomorrow), so that there is currently a restriction that the UltraGrid conversions to RG48 (16-bit RGB) must exist, which is the case for R10k and R12L but not notably for v210. I also may add it later if it turns out that OpenCV doesn't support 16 bit YCbCr codecs – anyways, as currently implemented, CV still converts YCbCr like UYVY to RGB; don't know if it knows better than UG (eg. from the performance POV; maybe it can have some OpenCL accelerations if enabled in CV)

[mikekoetter]

Ah that conversion is interesting - often our incoming 12 bit signals are the ones that we also need to scale in both resolution and bit depth. Previously I would set the codec on the decklink input to R10K, how it seems if you also want to scale dimensionally the correct workflow would be to let the decklink pick it up 12 bit at full res and then let the capture filter do both of the scaling functions.

[mikekoetter]

I just pulled in main and built it with no issues. Initial testing with a 2048 R10K to 1080p R10K works well.

[MartinPulec]

which is the case for R10k and R12L but not notably for v210

I've just added a conversion v210->RG48, effectively causing that the resize is possible in full-depth.

if it turns out that OpenCV doesn't support 16 bit YCbCr codecs

I've tried a little bit more – it seems like it supports 16-bit YCbCr input, but not in the formats that UltraGrid offers (Y216 and Y416); 3 channel YUV should work. But it may not matter, anyways, because the resize capture filter would just convert to RGB with OpenCV prior to actual resizing in that case. Currently it does UltraGrid just instead of OpenCV.