Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[BUGFIX]fix FP8 failing issue on habana_main [PatchedVLLMKVCache fwd …
…rror] (#502) Fix argument incompatible issue for FP8 ``` ERROR 11-11 04:29:13 engine.py:143] File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1556, in _wrapped_call_impl ERROR 11-11 04:29:13 engine.py:143] return self._call_impl(*args, **kwargs) ERROR 11-11 04:29:13 engine.py:143] File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1606, in _call_impl ERROR 11-11 04:29:13 engine.py:143] result = forward_call(*args, **kwargs) ERROR 11-11 04:29:13 engine.py:143] TypeError: PatchedVLLMKVCache.forward() missing 2 required positional arguments: 'block_indices' and 'block_offset' ``` FIX #453 https://github.com/HabanaAI/vllm-fork/blob/habana_main/README_GAUDI.md#troubleshooting-tweaking-hpu-graphs
- Loading branch information