need summarize all(all videocards) gpu memory count when rpc

[lexasub]

Hello, we need summarize all(all videocards) gpu memory count when rpc. When rpc server running, we may have any count of gpu, we need count all memory.

llama.cpp/examples/rpc/rpc-server.cpp

Line 116 in c05e8c9

ggml_backend_cuda_get_device_memory(0, free_mem, total_mem);

[lexasub]

the task seems to be more complicated than I thought, in the first approximation, it will be necessary to raise several backends ggml_backend_rpc_start_server(backend, endpoint.c_str(), free_mem, total_mem), which is not good, it is necessary to “adapt” ggml_backend_cuda_init/ggml_backend.