Add host traces to high-level profilings

[szutenberg]

run vllm server with
HABANA_PROF_CONFIG=$PWD/cfg.json VLLM_TORCH_PROFILER_DIR=$PWD HABANA_PROFILE=1 VLLM_PROFILER_ENABLED=full

cfg.json:

{
    "Plugins": [
        {
            "lib": "libhost_profiler.so",
            "name": "HostProfiler",
            "values": {
                "api_group": {
                    "HCCL": {
                        "value": false
                    },
                    "HLTHUNK": {
                        "value": false
                    },
                    "SYNAPSE": {
                        "value": false
                    }
                },
                "start_disabled": {
                    "value": true
                }
            },
            "enable": true
        },
        {
            "lib": "libhw_trace.so",
            "name": "HwTrace",
            "values": {
                "generalOptions": {
                    "traceBufferSize": {
                        "value": "0x8000000"
                    },
                    "profilePhase": {
                        "value": "profileApi"
                    }
                },
                "archProfileUnits": {
                    "gaudi3": {
                        "CS": {
                            "enable": {
                                "value": false
                            },
                            "Gaudi3CSAdvancedProfiling": {
                                "value": 0
                            }
                        }
                    }
                }
            },
            "enable": false
        }
    ]
}

Start profiling with:

curl -X POST http://localhost:8080/start_profile

Stop profiling with:

curl -X POST http://localhost:8080/stop_profile

[mswiniarsk]

With regard to the cfg.json file - what do you think about putting it into vllm-hpu-extension, so that we have a default one and everyone does not have to create one for its own?

[mswiniarsk]

Great idea with overriding the torch profiler handler to achieve this behavior!

[mswiniarsk]

I see that this function does not use any HPUWorker class's arguments (self), so it can be extracted from HPUWorker logic and treated as a sperate. Since we already have quite a complex logic in HPUModelRunner and HPUWorker, could you move this code to vllm-hpu-extension/.../profiler.py?

[szutenberg]

See line 133:

events = self.model_runner.profiler.profiling_trace_events

[mswiniarsk]

Looks a bit hacky. Can't we simple run
high_level_profiler.start('internal', 'start_profiler')
high_level_profiler.stop('internal', 'start_profiler')