Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

wrong color-space metadata after encode from RGB #281

Open
MartinPulec opened this issue Oct 23, 2023 · 9 comments
Open

wrong color-space metadata after encode from RGB #281

MartinPulec opened this issue Oct 23, 2023 · 9 comments

Comments

@MartinPulec
Copy link

this refers to CESNET/UltraGrid#341

hi, when QSV is compressing from a RGB format (BGRA for now; but applies also to X2RGB), it converts unconditionally to limited-range YUV BT.601, but sets stream metadata according to AVCodecContext::color_primaries and AVCodecContext::color_range. If those are not set explicitly, it sets full-range to metadata, anyways.

Please see following test case:

  1. input
    ffmpeg -y -f lavfi -i color=#ff0000 -t 1 -pix_fmt bgra [params] -c:v hevc_qsv out.h265

  2. output evaluation

    mediainfo --Language=raw --Full --Inform='Video;%matrix_coefficients%,%colour_range%' out.h265 &&
     ffmpeg -y -loglevel quiet -i out.h265 -strict -1 -frames 1 out.y4m; sed '1,2d' < out.y4m | od -t x1 
    

input RGB=[0xff,0x00,0x00] corresponds¹ to full-range YCbCr BT.601 [76,85,255]=[0x4c,0x55,0xff], limited range [65,91,237]=[0x51,0x5b,0xed] ²

¹ computed using this coefficient; Cb,Cr coefficients are offset by 128
² or Y=67=0x53 if using Y range 16-240, which I think is correct

evaluation:

  1. no params for 1. above - [0x51,0x5b,0xef], mediainfo: ",Full"
  2. -colorspace bt709 - [0x51,0x5b,0xef], mediainfo: "BT.709,Full"
  3. -colorspace bt709 -color_range tv- (ditto), mediainfo: "BT.709,Limited"
  4. -color_range tv - (ditto), mediainfo: ",Limited"
  5. -colorspace smpte170m -color_range tv - (ditto), mediainfo: "BT.601,Limited"

So, in all cases, the output is in limitted-range YCbCr 601, but metadata are correct only in case 5 (and also 4, assuming that implicit value is OK).

PS: -vf lavfi -i color=#ff0000 -t 1 -pix_fmt bgra seems to convert to [0x00,0x00,0xfd,0xff] - the swscale conversion is not entirely accurate but I don't know how to produce single-color RGB directly, without conversion from YUV (-vf lavfi -i color produces YUV)

@xhaihao
Copy link
Contributor

xhaihao commented Oct 25, 2023

Thanks for catching this issue, and I'll look into it.

@alatteri
Copy link

alatteri commented Nov 6, 2023

Hi @xhaihao

Any update on this issue?

Thanks.

@xhaihao
Copy link
Contributor

xhaihao commented Nov 7, 2023

-colorspace <arg>, -color_range <arg> etc are used to specify the color properties only, there is no color conversion here.

For example:

$ ffmpeg -y -loglevel verbose -f lavfi -i color=#ff0000 -t 1 -pix_fmt bgra out.0.rgb

The output is bgra(pc, gbr/unknown/unknown, progressive)

$ ffmpeg -y -loglevel verbose -f lavfi -i color=#ff0000 -t 1 -pix_fmt bgra -colorspace bt709 out.1.rgb

The output is bgra(pc, bt709/unknown/unknown, progressive)

$ ffmpeg -y -loglevel verbose -f lavfi -i color=#ff0000 -t 1 -pix_fmt bgra -colorspace bt709 -color_range tv out.2.rgb

The output is bgra(tv, bt709/unknown/unknown, progressive)

The color properties are different in the above examples, however out.0.rgb, out.1.rgb and out.2.rgb are same.

$ md5sum out.*
ef62d4f82747b3211118b905c649b5b0  out.0.rgb
ef62d4f82747b3211118b905c649b5b0  out.1.rgb
ef62d4f82747b3211118b905c649b5b0  out.2.rgb

@xhaihao
Copy link
Contributor

xhaihao commented Nov 7, 2023

PS: -vf lavfi -i color=#ff0000 -t 1 -pix_fmt bgra seems to convert to [0x00,0x00,0xfd,0xff] - the swscale conversion is not entirely accurate but I don't know how to produce single-color RGB directly, without conversion from YUV (-vf lavfi -i color produces YUV)

You may use -f lavfi -i color=#ff0000,format=bgra to produce single-color RGB directly

@MartinPulec
Copy link
Author

thank you much for looking into this

-colorspace , -color_range etc are used to specify the color properties only, there is no color conversion here.

I am aware of the fact that it just sets input metadata.

Anyways, if I deduced it correctly, so QSV does unconditional conversion from (full-range) RGB to limited-range BT.601 YCbCr (please correct me, if I am wrong). In this case, I believe that in the HEVC metadata there should be limited-range YCbCr, not full-range (optionally also BT.601). (The conversion will be correct only for input RGB pc/bt601, of course.)

Currently even though the input RGB using pc/bt601 is converted to limited-range YCbCr, while in codestream metadata remains PC/full range (seems to be just copied from the AVCodecContext::color_range). Well, as a workaround, I could set the source color range to AVCOL_RANGE_MPEG (limited) to enforce correct output codec metadata, but this will be just pretended. Please also note that not setting anything explicitly causes the output to have undefined color primaries (which is OK) but still telling to be full-range YCbCr.

@alatteri
Copy link

Hi @xhaihao,

Any thoughts on Martin's latest information?

Thanks,
Alan

@xhaihao
Copy link
Contributor

xhaihao commented Nov 21, 2023

@alatteri Yes the driver does an internal conversion, however the default range (full range) is set by FFmpeg, not QSV. Note the two commands below have different color ranges, it is full range in 1 however it is limited range in 2

  1. $ ffmpeg -y -f lavfi -i color=#ff0000 -t 1 -pix_fmt bgra -c:v hevc_qsv out.h265

  2. $ ffmpeg -y -f lavfi -i color=#ff0000,format=bgra -t 1 -c:v hevc_qsv out.h265

@XinfengZhang Any thought about this issue ? Is it possible to expose the capability in VA-API level so that user may control the conversion ?

@XinfengZhang
Copy link

plan to add a new interfaces to report the color space support for a given config + format
something like vaGetSurfaceFormatAttributes [refer: https://github.com/intel/libva/pull/589/files]
then add a field to specify the color space in SPS.

TBH, because no requirement previously , I defer it.

@alatteri
Copy link

alatteri commented Dec 10, 2023

Hi @XinfengZhang

When do you think you'll be able to get this merged? I see it has been pending for a very long time.

Thanks,
Alan

TBH, because no requirement previously , I defer it.

@MartinPulec Do you think this is the fix we need? ->

plan to add a new interfaces to report the color space support for a given config + format
something like vaGetSurfaceFormatAttributes [refer: https://github.com/https://github.com/intel/libva/pull/589/files]
then add a field to specify the color space in SPS.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants