You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Last week i could still properly run models with Koboldcpp ROCm (which i hope you can with here as well since issues are not enabled there), but when i tried today with the same settings, i get the following console output.
***Welcome to KoboldCpp - Version 1.77.yr0-ROCm
For command line arguments, please refer to --help
***
Auto Selected HIP Backend...
Auto Recommended GPU Layers: 31
Attempting to use hipBLAS library for faster prompt ingestion. A compatible AMD GPU will be required.
Initializing dynamic library: koboldcpp_hipblas.dll
==========
Namespace(model='', model_param='E:/AI/koboldcpp/models/Lumimaid-Magnum-12B.q8_0.gguf', port=5001, port_param=5001, host='', launch=False, config=None, threads=7, usecublas=['normal', '0', 'mmq'], usevulkan=None, useclblast=None, usecpu=False, contextsize=4096, gpulayers=31, tensor_split=None, checkforupdates=False, ropeconfig=[0.0, 10000.0], blasbatchsize=512, blasthreads=7, lora=None, noshift=False, nommap=False, usemlock=False, noavx2=False, debugmode=0, onready='', benchmark=None, prompt='', promptlimit=100, multiuser=1, remotetunnel=False, highpriority=False, foreground=False, preloadstory=None, quiet=False, ssl=None, nocertify=False, mmproj=None, password=None, ignoremissing=False, chatcompletionsadapter=None, flashattention=True, quantkv=0, forceversion=0, smartcontext=False, unpack='', nomodel=False, showgui=False, skiplauncher=False, hordemodelname='', hordeworkername='', hordekey='', hordemaxctx=0, hordegenlen=0, sdmodel='', sdthreads=7, sdclamped=0, sdvae='', sdvaeauto=False, sdquant=False, sdlora='', sdloramult=1.0, whispermodel='', hordeconfig=None, sdconfig=None, noblas=False)
==========
Loading model: E:\AI\koboldcpp\models\Lumimaid-Magnum-12B.q8_0.gguf
The reported GGUF Arch is: llama
Arch Category: 0
---
Identified as GGUF model: (ver 6)
Attempting to Load...
---
Using automatic RoPE scaling for GGUF. If the model has custom RoPE settings, they'll be used directly instead!
It means that the RoPE values written above will be replaced by the RoPE values indicated after loading.
System Info: AVX = 1 | AVX_VNNI = 0 | AVX2 = 1 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | AVX512_BF16 = 0 | AMX_INT8 = 0 | FMA = 1 | NEON = 0 | SVE = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | RISCV_VECT = 0 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 1 | SSSE3 = 1 | VSX = 0 | MATMUL_INT8 = 0 | LLAMAFILE = 0 |
Traceback (most recent call last):
File "koboldcpp.py", line 5090, in <module>
File "koboldcpp.py", line 4733, in main
File "koboldcpp.py", line 920, in load_model
OSError: exception: access violation reading 0x0000000000000000
[PYI-11872:ERROR] Failed to execute script 'koboldcpp' due to unhandled exception!
System info
I am on Windows 10 latest update (as of today, 04.11.) using an AMD Ryzen 7 7800X3D with the latest driver (so AVX should be enabled though apparently only AVX and AVX2 according to the console output), graphics card is AMD Radeon RX 7900 XTX in case that is somehow relevant.
The only things that changed from the last time it worked hardware or software related is todays windows update.
What works
I can run on Failsafe mode with either regular Kobold or Kobold_rocm but it's slow enough to be annoying.
What doesn't work
I scanned through most seemingly related issues and found nothing that worked/helped. Most related issues are seemingly irrelevant since it still worked last week and nothing major should have changed.
I tried updating my Kobold version, changing any setting that wasn't default, some things that are probably irrelevant like virtualization being enabled, checking if my CPU has AVX support and others that i don't remember.
What i need
Any solutions or ideas what might have changed in the last week are appreciated.
Thank you in advance
The text was updated successfully, but these errors were encountered:
What happens
Last week i could still properly run models with Koboldcpp ROCm (which i hope you can with here as well since issues are not enabled there), but when i tried today with the same settings, i get the following console output.
System info
I am on Windows 10 latest update (as of today, 04.11.) using an AMD Ryzen 7 7800X3D with the latest driver (so AVX should be enabled though apparently only AVX and AVX2 according to the console output), graphics card is AMD Radeon RX 7900 XTX in case that is somehow relevant.
The only things that changed from the last time it worked hardware or software related is todays windows update.
What works
I can run on Failsafe mode with either regular Kobold or Kobold_rocm but it's slow enough to be annoying.
What doesn't work
I scanned through most seemingly related issues and found nothing that worked/helped. Most related issues are seemingly irrelevant since it still worked last week and nothing major should have changed.
I tried updating my Kobold version, changing any setting that wasn't default, some things that are probably irrelevant like virtualization being enabled, checking if my CPU has AVX support and others that i don't remember.
What i need
Any solutions or ideas what might have changed in the last week are appreciated.
Thank you in advance
The text was updated successfully, but these errors were encountered: