multiple gpus loading

[hitxujian]

is support multiple gpus inference, why not suport load model with device_map='auto',

[HaozheZhao]

I'm sorry, but your question is a bit confusing to me : (
Could you please provide more details about your problem or the function you want to solve? It would be helpful if you could also provide some related code.

[hitxujian]

model = InstructBlipForConditionalGeneration.from_pretrained(
model_ckpt,device_map='auto',
config=config,**{"torch_dtype": torch.bfloat16})

ValueError: InstructBlipForConditionalGeneration does not support device_map='auto'. To implement support, the modelclass needs to implement the _no_split_modules attribute.

[lucasmgomez]

I also need to use multiple gpu's to load the model. Is that supported?