You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
CUDA_VISIBLE_DEVICES=0,1 和 --sharded true 设置后报错:
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cuda:1!
The text was updated successfully, but these errors were encountered:
通过TGI托管的模型,启动方式为:
BNB_CUDA_VERSION=122 CUDA_VISIBLE_DEVICES=0,1 text-generation-launcher --model-id /data/llms/codeshell-7b-chat --tokenizer-config-path /data/llms/codeshell-7b-chat/tokenizer_config.json --sharded true --trust-remote-code --port=8080
CUDA_VISIBLE_DEVICES=0,1 和 --sharded true 设置后报错:
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cuda:1!
The text was updated successfully, but these errors were encountered: