-
Notifications
You must be signed in to change notification settings - Fork 369
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Implemented hardware decoding #1685
Conversation
12 tests are also failing on my physical MacOS machine. The errors are the same as Github Actions |
Sorry about that. I thought they were existing failures but I forgot to do a |
All tests pass locally for me now. Also made |
This implements hardware decoding continuing from the work of @rvillalba-novetta and @mikeboers in main...rvillalba-novetta:PyAV:hwaccel (and children commits)
44f8397
to
741834d
Compare
I've cleaned up the things that remained a problem. |
if HW_DEVICE is None: | ||
av.codec.hwaccel.dump_hwdevices() | ||
print("Please set HW_DEVICE.") | ||
exit() | ||
|
||
assert HW_DEVICE in av.codec.hwaccel.hwdevices_available, f"{HW_DEVICE} not available." |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think the example is a little outdated here:
- there is no
dump_hwdevices
member inav.codec.hwaccel
av.codec.hwaccel.hwdevices_available
is acyfunction
, not a list, set or dict
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
#1689 should fix it.
The HWAccel and HWAccelContext merge is also creating problems with multiple streams sharing one context. I'm preparing a new PR to fix these issues. |
If you're doing a second PR, could you also have a look at #1690? |
This implements hardware decoding continuing from the original work of @rvillalba-novetta and refactoring by @mikeboers in main...rvillalba-novetta:PyAV:hwaccel (and children commits)
I've completed the refactoring, fixed a few bugs, and brought the changes up to date with both PyAV at HEAD and ffmpeg 7.
I've also added a test and an example script. The test cannot be run in CI for obvious reasons (GitHub runners don't have GPUs), but I've manually run it on Mac, Windows, and Linux (WSL).
I understand that @mikeboers didn't find it worthwhile back in the days, but the situation has changed significantly since then - now almost every machine has a supported decoder, and modern encodings (HEVC, VP9, AV1) are so compute-intensive to decode that they have practically been designed for hardware decoding. As an example, I've been working with 4K HEVC videos (standard format coming out of action cameras and phones these days), and my Apple M3 can only decode them at 15 fps, while VideoToolbox easily does 100+ fps. I believe now and especially going forward, hardware decoding will be a necessity for almost any application that cares about performance or need realtime decoding.
Performance tests with a 4K HEVC video (using the new example script) -
MacBook Air M3:
CPU (M3) - 16.3 fps
GPU (M3, videotoolbox) - 121.4 fps
Windows:
CPU (AMD Ryzen 5 3500X) - 15.6 fps
GPU (NVIDIA RTX 3060 Ti, using d3d11va) - 49.1 fps
Linux (WSL):
CPU (AMD Ryzen 5 3500X) - 15.5 fps
GPU (NVIDIA RTX 3060 Ti, using cuda) - 91.3 fps
Notes:
Thanks!