whisper-cpp: add ROCm and Vulkan support, rename from openai-whisper-cpp #365033
+204
−117
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
aviallon
force-pushed
the
feature/rework-whisper-cpp
branch
from
a67bf75 to
7246230
Compare
aviallon
force-pushed
the
feature/rework-whisper-cpp
branch
2 times, most recently
from
9058dfa to
d05678a
Compare
|
|
If someone could test on Darwin, it would be awesome. |
ofborg
bot
added
8.has: clean-up
8.has: package (new)
ofborg
bot
added
the
11.by: package-maintainer
|
Great job! I tested it on Darwin, but unfortunately, it didn't work: |
It looks like the RPATH is not set correctly. I'll also see if I can create a KVM-OSX VM to test the derivation directly. |
sersorrel
reviewed
Dec 15, 2024
There was a problem hiding this comment.
tested Vulkan support with:
$ nixpkgs-review pr 365033
$ mkdir /tmp/whisper
$ ./results/whisper-cpp-vulkan-x86_64-linux/bin/whisper-cpp-download-ggml-model base.en /tmp/whisper/
$ ./results/whisper-cpp-vulkan-x86_64-linux/bin/whisper-cpp -m /tmp/whisper/ggml-base.en.bin -m /tmp/whisper/ggml-base.en.bin ~/Downloads/jfk.wav
and it seems to work ok:
ggml_vulkan: Found 1 Vulkan devices:
ggml_vulkan: 0 = AMD Radeon RX 7900 XTX (RADV NAVI31) ((null)) | uma: 1 | fp16: 1986290034 | warp size: 0
with rocmSupport = true, it seems to work fine as well (though it takes ~25 minutes to build locally):
ggml_cuda_init: found 1 ROCm devices:
Device 0: AMD Radeon RX 7900 XTX, compute capability 11.0, VMM: no
kurnevsky
reviewed
Dec 15, 2024
|
The vanilla build works (and fixes the SIMD issues thank you) However the |
I'm a bit swamped with other stuff right now, so I won't be able to dive into debugging this further. Sorry! |
aviallon
force-pushed
the
feature/rework-whisper-cpp
branch
from
d05678a to
bfc4257
Compare
|
@PapayaJackal I tried adding |
|
$ nix build --file /nix/store/w1hgvm5rb3rblya8if90cdx5lvwyf4wc-nixpkgs-review-2.12.0/lib/python3.12/site-packages/nixpkgs_review/nix/review-shell.nix --nix-path 'nixpkgs=/Users/rodrigo/.cache/nixpkgs-review/pr-365033-1/nixpkgs nixpkgs-overlays=/var/folders/by/9xt1lyys0qvds8zzbx60c58m0000gn/T/tmppcq_ecwm' --extra-experimental-features 'nix-command no-url-literals' --no-link --keep-going --no-allow-import-from-derivation --argstr local-system aarch64-darwin --argstr nixpkgs-path /Users/rodrigo/.cache/nixpkgs-review/pr-365033-1/nixpkgs --argstr nixpkgs-config-path /var/folders/by/9xt1lyys0qvds8zzbx60c58m0000gn/T/tmp0qmt4adr.nix --argstr attrs-path /Users/rodrigo/.cache/nixpkgs-review/pr-365033-1/attrs.nix
error: builder for '/nix/store/dinzb2x13v9ssxvp36p7m76fk35dc13a-whisper-cpp-1.7.2.drv' failed with exit code 1;
last 10 log lines:
> ^^^^^^^^^^^^^^^
> File "/nix/store/whmwqp5qa1lpadz0ik6wfjgjzbbgjh1z-python3-3.12.7-env/lib/python3.12/site-packages/elftools/elf/elffile.py", line 84, in __init__
> self._identify_file()
> File "/nix/store/whmwqp5qa1lpadz0ik6wfjgjzbbgjh1z-python3-3.12.7-env/lib/python3.12/site-packages/elftools/elf/elffile.py", line 570, in _identify_file
> elf_assert(magic == b'\x7fELF', 'Magic number does not match')
> File "/nix/store/whmwqp5qa1lpadz0ik6wfjgjzbbgjh1z-python3-3.12.7-env/lib/python3.12/site-packages/elftools/common/utils.py", line 80, in elf_assert
> _assert_with_exception(cond, msg, ELFError)
> File "/nix/store/whmwqp5qa1lpadz0ik6wfjgjzbbgjh1z-python3-3.12.7-env/lib/python3.12/site-packages/elftools/common/utils.py", line 143, in _assert_with_exception
> raise exception_type(msg)
> elftools.common.exceptions.ELFError: Magic number does not match
For full logs, run 'nix-store -l /nix/store/dinzb2x13v9ssxvp36p7m76fk35dc13a-whisper-cpp-1.7.2.drv'.
error: builder for '/nix/store/gdcg3r5pwz8m8w0vbf6jk7cy6w76jylc-whisper-cpp-1.7.2.drv' failed with exit code 1;
last 10 log lines:
> ^^^^^^^^^^^^^^^
> File "/nix/store/whmwqp5qa1lpadz0ik6wfjgjzbbgjh1z-python3-3.12.7-env/lib/python3.12/site-packages/elftools/elf/elffile.py", line 84, in __init__
> self._identify_file()
> File "/nix/store/whmwqp5qa1lpadz0ik6wfjgjzbbgjh1z-python3-3.12.7-env/lib/python3.12/site-packages/elftools/elf/elffile.py", line 570, in _identify_file
> elf_assert(magic == b'\x7fELF', 'Magic number does not match')
> File "/nix/store/whmwqp5qa1lpadz0ik6wfjgjzbbgjh1z-python3-3.12.7-env/lib/python3.12/site-packages/elftools/common/utils.py", line 80, in elf_assert
> _assert_with_exception(cond, msg, ELFError)
> File "/nix/store/whmwqp5qa1lpadz0ik6wfjgjzbbgjh1z-python3-3.12.7-env/lib/python3.12/site-packages/elftools/common/utils.py", line 143, in _assert_with_exception
> raise exception_type(msg)
> elftools.common.exceptions.ELFError: Magic number does not match
For full logs, run 'nix-store -l /nix/store/gdcg3r5pwz8m8w0vbf6jk7cy6w76jylc-whisper-cpp-1.7.2.drv'.
error: 2 dependencies of derivation '/nix/store/dasbp7ay07cbiw4l4rp56bxfk5ink0jp-review-shell.drv' failed to build |
|
@aviallon, this is the first time I've run those commands, and my environment might also be broken. Please let me know if there is anything more I can do to help |
|
@rodfersou thank you very much. |
|
@aviallon I run the command for full logs looks like the target macos (12) is different from the one I have (11)
[2/55] Building CXX object src/CMakeFiles/whisper.coreml.dir/coreml/whisper-encoder.mm.o
/tmp/nix-build-whisper-cpp-1.7.2.drv-1/source/src/coreml/whisper-encoder.mm:31:44: warning: 'whisper_encoder_impl' is only available on macOS 12.0 or newer [-
Wunguarded-availability-new]
const void * data = CFBridgingRetain([[whisper_encoder_impl alloc] initWithContentsOfURL:url_model configuration:config error:nil]);
^~~~~~~~~~~~~~~~~~~~
/tmp/nix-build-whisper-cpp-1.7.2.drv-1/source/src/coreml/whisper-encoder-impl.h:41:12: note: 'whisper_encoder_impl' has been marked as being introduced in macOS 12.0 here, but the deployment target is macOS 11.0.0
@interface whisper_encoder_impl : NSObject
^
/tmp/nix-build-whisper-cpp-1.7.2.drv-1/source/src/coreml/whisper-encoder.mm:31:44: note: enclose 'whisper_encoder_impl' in an @available check to silence this warning
const void * data = CFBridgingRetain([[whisper_encoder_impl alloc] initWithContentsOfURL:url_model configuration:config error:nil]);
^~~~~~~~~~~~~~~~~~~~
/tmp/nix-build-whisper-cpp-1.7.2.drv-1/source/src/coreml/whisper-encoder.mm:65:9: warning: 'whisper_encoder_implOutput' is only available on macOS 12.0 or newer [-Wunguarded-availability-new]
whisper_encoder_implOutput * outCoreML = [(__bridge id) ctx->data predictionFromLogmel_data:inMultiArray error:nil];
^~~~~~~~~~~~~~~~~~~~~~~~~~
/tmp/nix-build-whisper-cpp-1.7.2.drv-1/source/src/coreml/whisper-encoder-impl.h:29:12: note: 'whisper_encoder_implOutput' has been marked as being introduced in macOS 12.0 here, but the deployment target is macOS 11.0.0
@interface whisper_encoder_implOutput : NSObject<MLFeatureProvider>
^
/tmp/nix-build-whisper-cpp-1.7.2.drv-1/source/src/coreml/whisper-encoder.mm:65:9: note: enclose 'whisper_encoder_implOutput' in an @available check to silence this warning
whisper_encoder_implOutput * outCoreML = [(__bridge id) ctx->data predictionFromLogmel_data:inMultiArray error:nil];
^~~~~~~~~~~~~~~~~~~~~~~~~~ |
|
@aviallon can we try with SDK12 instead? |
|
this is what happens when try to run the failed binary ...failed_results/whisper-cpp-aarch64-darwin/bin
$ ./whisper-cpp
dyld[10672]: Library not loaded: libwhisper.1.dylib
Referenced from: <3DC12AEA-5D8F-3F2A-8F4F-964BFAD19413> /nix/store/xp339mpr218ppnkm6fcrr9j3c6ckhx09-whisper-cpp-1.7.2/bin/whisper-cpp
Reason: tried: 'libwhisper.1.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OSlibwhisper.1.dylib' (no such file), 'libwhisper.1.dylib' (no such file), '/usr/local/lib/libwhisper.1.dylib' (no such file), '/usr/lib/libwhisper.1.dylib' (no such file, not in dyld cache), '/nix/store/xp339mpr218ppnkm6fcrr9j3c6ckhx09-whisper-cpp-1.7.2/bin/libwhisper.1.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/nix/store/xp339mpr218ppnkm6fcrr9j3c6ckhx09-whisper-cpp-1.7.2/bin/libwhisper.1.dylib' (no such file), '/nix/store/xp339mpr218ppnkm6fcrr9j3c6ckhx09-whisper-cpp-1.7.2/bin/libwhisper.1.dylib' (no such file), '/usr/local/lib/libwhisper.1.dylib' (no such file), '/usr/lib/libwhisper.1.dylib' (no such file, not in dyld cache)
[1] 10672 abort ./whisper-cpp |
|
follow the full logs whisper-cpp-aarch64-darwin.log |
|
@aviallon this should help https://stackoverflow.com/a/11585225 |
rodfersou
reviewed
Dec 22, 2024
|
Looking forward to this landing, please let us know if we can assist. |
|
|
|
Quite frankly, I can't fix macOS support. |
|
I daily drive an I suppose it's best to mark it broken for now to not delay this PR any further While this PR is not merged yet, I think it will be a good idea to bump the version to |
|
@aviallon did you try my suggestions? I'm pretty sure adding the environment variables I told should fix.. |
rodfersou
suggested changes
Jan 12, 2025
|
@rodfersou I can try, but it is unlikely to make any difference, as the issue is caused by a wrong RPATH due to the binaries not being part of CMake's install target. |
|
did you notice the typo that you keep LLAMA instead of WHISPER in the environment variable? |
aviallon
force-pushed
the
feature/rework-whisper-cpp
branch
from
bfc4257 to
aa40380
Compare
|
@rodfersou @lucasew I reworked the PR to patch |
|
@rodfersou According to https://logs.ofborg.org/?attempt_id=d3509326-6aa2-439d-a4a1-7388ad8189a1&key=nixos%2Fnixpkgs.365033, it doesn't even build for x86_64-darwin... |
|
Wait, apple-sdk has changed. I have to rework things. |
aviallon
force-pushed
the
feature/rework-whisper-cpp
branch
2 times, most recently
from
f72f772 to
d97db76
Compare
|
$ nixpkgs-review pr --post-result 365033
$ git -c fetch.prune=false fetch --no-tags --force https://github.com/NixOS/nixpkgs master:refs/nixpkgs-review/0 pull/365033/head:refs/nixpkgs-review/1
$ git worktree add /Users/rodrigo/.cache/nixpkgs-review/pr-365033-5/nixpkgs 489c04e40ca17f831e45cefc4a72e8884bf7a51a
Preparing worktree (detached HEAD 489c04e40ca1)
Updating files: 100% (45779/45779), done.
HEAD is now at 489c04e40ca1 miniflux: 2.2.4 -> 2.2.5 (#375556)
$ git merge --no-commit --no-ff d97db76924bd0185e1f9aebb1ecd3520cd1acea2
Auto-merging pkgs/top-level/aliases.nix
Auto-merging pkgs/top-level/all-packages.nix
Automatic merge went well; stopped before committing as requested
Nothing to be built.
Link to currently reviewing PR:
https://github.com/NixOS/nixpkgs/pull/365033
Posting result comment on https://github.com/NixOS/nixpkgs/pull/365033
$ /nix/store/c4kw7qpffsq7vd6brwgkfnglvj85ivqk-nix-2.24.11/bin/nix-shell --argstr local-system aarch64-darwin --argstr nixpkgs-path /Users/rodrigo/.cache/nixpkgs-review/pr-365033-5/nixpkgs --argstr nixpkgs-config-path /var/folders/by/9xt1lyys0qvds8zzbx60c58m0000gn/T/tmpn9m2a4bs.nix --argstr attrs-path /Users/rodrigo/.cache/nixpkgs-review/pr-365033-5/attrs.nix --nix-path 'nixpkgs=/Users/rodrigo/.cache/nixpkgs-review/pr-365033-5/nixpkgs nixpkgs-overlays=/var/folders/by/9xt1lyys0qvds8zzbx60c58m0000gn/T/tmpc3n6y285' /nix/store/w1hgvm5rb3rblya8if90cdx5lvwyf4wc-nixpkgs-review-2.12.0/lib/python3.12/site-packages/nixpkgs_review/nix/review-shell.nix
[nix-shell:~/.cache/nixpkgs-review/pr-365033-5]$ |
|
|
@aviallon I'm confused, not sure if it's working |
Rewrite to use CMake build Add aviallon to whisper-cpp's maintainers
aviallon
force-pushed
the
feature/rework-whisper-cpp
branch
from
d97db76 to
27a56b6
Compare
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Rewrite to use CMake build
Add aviallon to whisper-cpp's maintainers
Fixes #357736
Fixes #349669
Closes #358174
Things done
nix.conf? (See Nix manual)sandbox = relaxedsandbox = truenix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage./result/bin/)Add a 👍 reaction to pull requests you find important.