add a "Use Vorbis" switch #151
Conversation
AAC is currently hard-coded as the codec used for transcoding. In low-bandwidth environments, where transcoding is usually a must, AAC performs very badly though, especially the built-in FFmpeg AAC encoder. qaac and maybe FDK AAC are a bit better, but due to codec limitations, even those high-performance codecs are outperformed by Vorbis on low bitrates (also using those would require more patching in Jellyfin, and using non-free software, so that wouldn't really work). Vorbis is a free (as in freedom) audio format, and the libvorbis codec implementation is also free (also as in freedom). Jellyfin supports using it out-of-the-box. While opus would've been a better choice, I and some friends currently have some trouble getting it to work properly on the Jellyfin server side of things, so I figured many other users do as well, which is why Opus, even if it performs even better, hasn't been chosen. A switch has been added into the Transcoding Settings Menu that toggles between the widely-supported AAC and the higher-quality Vorbis. Settings are stored in the Hive Database (I do not have Dart and Flutter installed, so 1) this is untested and 2) the FinampModels.g.dart hasn't been re-generated yet).
|
Thanks for this, just made a few changes:
For some reason, I can't get an audio output on the Android emulator, so I've attached a build below. Could you check that it works fine on your server? |
|
Oh, no, Jellyfin uses |
|
For some reason AAC transcodes aren't playing any sound for me as well, I'll look into it tomorrow on a real device |
|
Testing on a Xiaomi Mi 6 running LineageOS 18.1 (Android 11) I get the following errors with Vorbis transcoding enabled: Vorbis transcodes succeed on server side, after a while the .ogg files are deleted though (I assume it's due to no playback progress reported): One time I've also experienced the player skipping through the queue, but I'm unable to reproduce this. |
|
Is this on the build provided in the .zip or the one after the |
|
The second one. Lack of HLS could be the culprit, as AAC audio appears in transcodes directory as a bunch of .ts files and a .m3u8 playlist, meanwhile Vorbis audio is transcoded into single .ogg file. |
|
Yeah, the error is it complaining that it hasn't received proper HLS stuff. Does Vorbis support HLS? From what I can tell, HLS only supports AAC, MP3, AC-3 or EC-3 for audio. We could make it so that just_audio loads a regular audio source, but then we'd go back to seeking not working properly with transcoded audio. I'm considering making a gstreamer backend for just_audio, which may be able to handle non-HLS transcodes better (haven't actually tested, but Exoplayer/iOS don't do a great job so it's probably pretty easy to beat) |
|
Non-transcoded audio is also not HLSed though, so, without having looked into the program logic too much, couldn't we just do whatever we're already doing with non-transcoded audio with Vorbis transcoded audio as well? If yes, that'd be a cleaner solution imo |
|
Having looked into the code, the latest commit should also mitigate this by simply setting |
|
Now playback works, but seeking is broken - slider keeps moving forward from selected timestamp, but song is played from the start. |
|
To me, that sounds like seeking into a region which isn't loaded on the device, either because of local buffering or server side transcoding progress. Not much that can be done about it (other than maybe resetting the slider), I'm afraid... Or are you getting a specific error from the seek function? |
|
This is what I meant by "we'd go back to seeking not working properly with transcoded audio". From what I can tell, Jellyfin doesn't return a length for a transcoded file (because the file doesn't fully exist yet) and Exoplayer/iOS both rely on the length to handle seeking. HLS solves this by properly segmenting the audio, but that of course doesn't work with Vorbis. The proper way to handle seeking on non-HLS transcodes is to restart the stream at an offset (https://github.com/MediaBrowser/Emby/wiki/Audio-Streaming#seeking), which may be possible but it would be very jank. We could add a disclaimer that seeking won't work when Vorbis transcoding is enabled, but that's a bit lazy. Random note - transcoding currently performs like this in the release build, I really need to get 0.6 out lol |
|
This might actually seem like this is true at first glance. However, Ogg Vorbis NEVER encodes duration information - thus none is written, both for transcoded and direct-played audio. You can easily verify this by wget'ing some transcoded endpoint from your Jellyfin server and grabbing some full Ogg Vorbis file, and then doing ExoPlayer waits for the End of Stream (EOS) packet and determines the length using that - and so do FFmpeg and many others. |
|
Just tested it in an emulator and seeking restarts the stream even if the stream has completed. This is likely an issue with ExoPlayer. IMO, this is fine as long as we just add a note saying that seeking will not work. I'm considering writing a GStreamer backend for just_audio, which may handle seeking better. |
|
As for Opus not working from Jellyfin, it looks like it never specified the codec when transcoding, so if you were requesting an OGG it would default to Vorbis. This PR is probably too stale to really merge now, and I'm currently working on transcoded downloads which may change how this PR could be implemented. If my Jellyfin PR gets merged, it would make sense to add extra options, such as Opus, as we'll finally be able to stream Opus on iOS through the CAF container. |
|
Just did a quick test, Jellyfin can return OPUS from a HLS stream, makes this a no-brainer for Android (assuming it can play Opus-encoded HLS). I'll look into updating this PR at some point, hopefully before this PR turns two years old 🙃 |
you have about two months left 😁 |
|
haha I'll probably roll it into #496 🙃 |
|
Okay, so the new download system already supports transcoded downloads using OPUS, and it's really impressive how little bitrate it needs. We probably need to extend the scale to support even lower values... |
|
Currently working on #915 which changes the default segment container from MPEG-TS to Fragmented MP4 (fMP4). This seems to improve codec compatibility. This might also make OPUS or Vorbis viable for regular transcoded streaming on Android and iOS. |
AAC is currently hard-coded as the codec used for transcoding. In
low-bandwidth environments, where transcoding is usually a must, AAC
performs very badly though, especially the built-in FFmpeg AAC encoder.
qaac and maybe FDK AAC are a bit better, but due to codec limitations,
even those high-performance codecs are outperformed by Vorbis on low
bitrates (also using those would require more patching in Jellyfin, and
using non-free software, so that wouldn't really work).
Vorbis is a free (as in freedom) audio format, and the libvorbis codec
implementation is also free (also as in freedom). Jellyfin supports
using it out-of-the-box.
While opus would've been a better choice, I and some friends currently
have some trouble getting it to work properly on the Jellyfin server
side of things, so I figured many other users do as well, which is why
Opus, even if it performs even better, hasn't been chosen.
A switch has been added into the Transcoding Settings Menu that toggles
between the widely-supported AAC and the higher-quality Vorbis. Settings
are stored in the Hive Database (I do not have Dart and Flutter
installed, so 1) this is untested and 2) the FinampModels.g.dart hasn't
been re-generated yet).