Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Windows 10 AMD ROCm not loading model when it did a week ago #1200

Open
mtgKerk opened this issue Nov 4, 2024 · 1 comment
Open

Windows 10 AMD ROCm not loading model when it did a week ago #1200

mtgKerk opened this issue Nov 4, 2024 · 1 comment

Comments

@mtgKerk
Copy link

mtgKerk commented Nov 4, 2024

What happens

Last week i could still properly run models with Koboldcpp ROCm (which i hope you can with here as well since issues are not enabled there), but when i tried today with the same settings, i get the following console output.

***Welcome to KoboldCpp - Version 1.77.yr0-ROCm
For command line arguments, please refer to --help
***
Auto Selected HIP Backend...

Auto Recommended GPU Layers: 31
Attempting to use hipBLAS library for faster prompt ingestion. A compatible AMD GPU will be required.
Initializing dynamic library: koboldcpp_hipblas.dll
==========
Namespace(model='', model_param='E:/AI/koboldcpp/models/Lumimaid-Magnum-12B.q8_0.gguf', port=5001, port_param=5001, host='', launch=False, config=None, threads=7, usecublas=['normal', '0', 'mmq'], usevulkan=None, useclblast=None, usecpu=False, contextsize=4096, gpulayers=31, tensor_split=None, checkforupdates=False, ropeconfig=[0.0, 10000.0], blasbatchsize=512, blasthreads=7, lora=None, noshift=False, nommap=False, usemlock=False, noavx2=False, debugmode=0, onready='', benchmark=None, prompt='', promptlimit=100, multiuser=1, remotetunnel=False, highpriority=False, foreground=False, preloadstory=None, quiet=False, ssl=None, nocertify=False, mmproj=None, password=None, ignoremissing=False, chatcompletionsadapter=None, flashattention=True, quantkv=0, forceversion=0, smartcontext=False, unpack='', nomodel=False, showgui=False, skiplauncher=False, hordemodelname='', hordeworkername='', hordekey='', hordemaxctx=0, hordegenlen=0, sdmodel='', sdthreads=7, sdclamped=0, sdvae='', sdvaeauto=False, sdquant=False, sdlora='', sdloramult=1.0, whispermodel='', hordeconfig=None, sdconfig=None, noblas=False)
==========
Loading model: E:\AI\koboldcpp\models\Lumimaid-Magnum-12B.q8_0.gguf

The reported GGUF Arch is: llama
Arch Category: 0

---
Identified as GGUF model: (ver 6)
Attempting to Load...
---
Using automatic RoPE scaling for GGUF. If the model has custom RoPE settings, they'll be used directly instead!
It means that the RoPE values written above will be replaced by the RoPE values indicated after loading.
System Info: AVX = 1 | AVX_VNNI = 0 | AVX2 = 1 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | AVX512_BF16 = 0 | AMX_INT8 = 0 | FMA = 1 | NEON = 0 | SVE = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | RISCV_VECT = 0 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 1 | SSSE3 = 1 | VSX = 0 | MATMUL_INT8 = 0 | LLAMAFILE = 0 |
Traceback (most recent call last):
  File "koboldcpp.py", line 5090, in <module>
  File "koboldcpp.py", line 4733, in main
  File "koboldcpp.py", line 920, in load_model
OSError: exception: access violation reading 0x0000000000000000
[PYI-11872:ERROR] Failed to execute script 'koboldcpp' due to unhandled exception!

System info

I am on Windows 10 latest update (as of today, 04.11.) using an AMD Ryzen 7 7800X3D with the latest driver (so AVX should be enabled though apparently only AVX and AVX2 according to the console output), graphics card is AMD Radeon RX 7900 XTX in case that is somehow relevant.

The only things that changed from the last time it worked hardware or software related is todays windows update.

What works

I can run on Failsafe mode with either regular Kobold or Kobold_rocm but it's slow enough to be annoying.

What doesn't work

I scanned through most seemingly related issues and found nothing that worked/helped. Most related issues are seemingly irrelevant since it still worked last week and nothing major should have changed.
I tried updating my Kobold version, changing any setting that wasn't default, some things that are probably irrelevant like virtualization being enabled, checking if my CPU has AVX support and others that i don't remember.

What i need

Any solutions or ideas what might have changed in the last week are appreciated.

Thank you in advance

@LostRuins
Copy link
Owner

Does vulkan work? Give that a try.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants