Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Multi models support for upstream #590

Open
wants to merge 211 commits into
base: multi_models_upstream
Choose a base branch
from
Open

Multi models support for upstream #590

wants to merge 211 commits into from

Conversation

xuechendi
Copy link

@xuechendi xuechendi commented Dec 4, 2024 •
edited by github-actions bot
Loading

No description provided.

B-201 and others added 30 commits November 18, 2024 12:57
@B-201
[Model][LoRA]LoRA support added for glm-4v (vllm-project#10418)
5be4e52 
Signed-off-by: B-201 <[email protected]>
@Isotr0py
[Model] Remove transformers attention porting in VITs (vllm-project#1…
e7ebb66 
…0414)

Signed-off-by: Isotr0py <[email protected]>
@B-201
[Doc] Update doc for LoRA support in GLM-4V (vllm-project#10425)
4186be8 
Signed-off-by: B-201 <[email protected]>
@youkaichao
[5/N][torch.compile] torch.jit.script --> torch.compile (vllm-project…
7851b45 
…#10406)

Signed-off-by: youkaichao <[email protected]>
@ismael-dm
[Doc] Add documentation for Structured Outputs (vllm-project#9943)
31894a2 
Signed-off-by: ismael-dm <[email protected]>
@andrew
Fix open_collective value in FUNDING.yml (vllm-project#10426)
4f686d1 
Signed-off-by: Andrew Nesbitt <[email protected]>
@mgoin
[Model][Bugfix] Support TP for PixtralHF ViT (vllm-project#10405)
281cc4b 
Signed-off-by: mgoin <[email protected]>
@yma11
[Hardware][XPU] AWQ/GPTQ support for xpu backend (vllm-project#10107)
6b2d25e 
Signed-off-by: yan ma <[email protected]>
@angusYuhao
[Kernel] Explicitly specify other value in tl.load calls (vllm-projec…
c2170a5 
…t#9014)

Signed-off-by: Angus Wang <[email protected]>
@LucasWilkinson
[Kernel] Initial Machete W4A8 support + Refactors (vllm-project#9855)
96d999f 
Signed-off-by: Lucas Wilkinson <[email protected]>
@youkaichao
[3/N][torch.compile] consolidate custom op logging (vllm-project#10399)
a03ea40 
Signed-off-by: youkaichao <[email protected]>
@youkaichao
[ci][bugfix] fix kernel tests (vllm-project#10431)
2298e69 
Signed-off-by: youkaichao <[email protected]>
@rickyyx
[misc] partial prefix & random input generation benchmark (vllm-proje…
90a6c75 
…ct#9929)

Signed-off-by: rickyx <[email protected]>
@khluu
[ci/build] Have dependabot ignore all patch update (vllm-project#10436)
284203f 
We have too many dependencies and all patch updates can be a little noisy. This is to have dependabot ignore all patch version updates.
@jeejeelee
[Bugfix]Fix Phi-3 BNB online quantization (vllm-project#10417)
7eb719d 
Signed-off-by: Jee Jee Li <[email protected]>
@MengqingCao
[Platform][Refactor] Extract func get_default_attn_backend to `Plat…
8c1fb50 
…form` (vllm-project#10358)

Signed-off-by: Mengqing Cao <[email protected]>
@mgoin
Add openai.beta.chat.completions.parse example to structured_outputs.…
74f8c2c 
…rst (vllm-project#10433)
@tjohnson31415
[Bugfix] Guard for negative counter metrics to prevent crash (vllm-pr…
272e31c 
…oject#10430)

Signed-off-by: Travis Johnson <[email protected]>
@jeejeelee
[Misc] Avoid misleading warning messages (vllm-project#10438)
382b6a4 
Signed-off-by: Jee Jee Li <[email protected]>
@russellb
[Doc] Add the start of an arch overview page (vllm-project#10368)
5390d66
@youkaichao
[misc][plugin] improve plugin loading (vllm-project#10443)
25f9c78 
Signed-off-by: youkaichao <[email protected]>
@zhouyuan
[CI][CPU] adding numa node number as container name suffix (vllm-proj…
b461465 
…ect#10441)

Signed-off-by: Yuan Zhou <[email protected]>
@xiyuan-lee
[BugFix] Fix hermes tool parser output error stream arguments in some…
f028dff 
… cases (vllm-project#10395) (vllm-project#10398)

Signed-off-by: xiyuan lee <[email protected]>
@patrickvonplaten
[Pixtral-Large] Pixtral actually has no bias in vision-lang adapter (v…
11fd7ea 
…llm-project#10449)
@mikejuliet13 @B-201
@Isotr0py @youkaichao @ismael-dm @andrew @mgoin @yma11 @angusYuhao @LucasWilkinson @rickyyx @khluu @jeejeelee @MengqingCao @tjohnson31415 @russellb
Fix: Build error seen on Power Architecture (vllm-project#10421)
1ea291a 
Signed-off-by: Manjul Mohan <[email protected]>
Signed-off-by: B-201 <[email protected]>
Signed-off-by: Isotr0py <[email protected]>
Signed-off-by: youkaichao <[email protected]>
Signed-off-by: ismael-dm <[email protected]>
Signed-off-by: Andrew Nesbitt <[email protected]>
Signed-off-by: mgoin <[email protected]>
Signed-off-by: yan ma <[email protected]>
Signed-off-by: Angus Wang <[email protected]>
Signed-off-by: Lucas Wilkinson <[email protected]>
Signed-off-by: rickyx <[email protected]>
Signed-off-by: Jee Jee Li <[email protected]>
Signed-off-by: Mengqing Cao <[email protected]>
Signed-off-by: Travis Johnson <[email protected]>
Co-authored-by: Manjul Mohan [email protected] <[email protected]>
Co-authored-by: B-201 <[email protected]>
Co-authored-by: Isotr0py <[email protected]>
Co-authored-by: youkaichao <[email protected]>
Co-authored-by: ismael-dm <[email protected]>
Co-authored-by: Andrew Nesbitt <[email protected]>
Co-authored-by: Michael Goin <[email protected]>
Co-authored-by: Yan Ma <[email protected]>
Co-authored-by: Angus Wang <[email protected]>
Co-authored-by: Lucas Wilkinson <[email protected]>
Co-authored-by: Ricky Xu <[email protected]>
Co-authored-by: Kevin H. Luu <[email protected]>
Co-authored-by: Jee Jee Li <[email protected]>
Co-authored-by: Mengqing Cao <[email protected]>
Co-authored-by: Travis Johnson <[email protected]>
Co-authored-by: Russell Bryant <[email protected]>
@russellb
[Doc] fix link for page that was renamed (vllm-project#10455)
fd9f124 
Signed-off-by: Russell Bryant <[email protected]>
@youkaichao
[6/N] torch.compile rollout to users (vllm-project#10437)
803f37e 
Signed-off-by: youkaichao <[email protected]>
@russellb
[Core] Avoid metrics log noise when idle (vllm-project#8868)
efa9084 
Signed-off-by: Russell Bryant <[email protected]>
@ElizaWszola
[Model][Quantization] HQQ support through Marlin kernel expansion (vl…
b00b33d 
…lm-project#9766)

Signed-off-by: ElizaWszola <[email protected]>
@maxdebayser
Change granite chat template to keep json list formatting for tool ca…
a324d3a 
…lls (vllm-project#10452)

Signed-off-by: Max de Bayser <[email protected]>
yma11 and others added 24 commits December 2, 2024 17:53
@yma11 @youkaichao
[Misc][XPU] Avoid torch compile for XPU platform (vllm-project#10747)
519cc6c 
Signed-off-by: yan ma <[email protected]>
Co-authored-by: youkaichao <[email protected]>
@janimo
Fix openvino on GPU (vllm-project#10793)
9b14d97
@Isotr0py @DarkLight1337
[Model] Add TP and BNB quantization support to LlavaMultiModalProject…
4c05edb 
…or (vllm-project#10834)

Signed-off-by: Isotr0py <[email protected]>
Co-authored-by: Cyrus Leung <[email protected]>
@mgoin
[Bugfix] Prevent benchmark_throughput.py from using duplicated random…
4433195 
… prompts (vllm-project#10753)
@zixuanzhang226
[Model] support bitsandbytes quantization with minicpm model (vllm-pr…
d746268 
…oject#10842)

Signed-off-by: Ubuntu <[email protected]>
@jeejeelee
[Bugfix] Fix QKVParallelLinearWithShardedLora bias bug (vllm-project#…
a4cf256 
…10844)

Signed-off-by: Jee Jee Li <[email protected]>
@youkaichao
[core][distributed] add pynccl broadcast (vllm-project#10843)
21fe7b4 
Signed-off-by: youkaichao <[email protected]>
@youkaichao
[torch.compile] remove compilation_context and simplify code (vllm-pr…
dc5ce86 
…oject#10838)

Signed-off-by: youkaichao <[email protected]>
@russellb @DarkLight1337
[Doc] Add github links for source code references (vllm-project#10672)
ef51831 
Signed-off-by: Russell Bryant <[email protected]>
Signed-off-by: DarkLight1337 <[email protected]>
Co-authored-by: DarkLight1337 <[email protected]>
@DarkLight1337
[Misc] Remove deprecated names (vllm-project#10817)
3257d44 
Signed-off-by: DarkLight1337 <[email protected]>
@aarnphm @mgoin
[Core][Performance] Add XGrammar support for guided decoding and set …
9323a31 
…it as default (vllm-project#10785)

Signed-off-by: Aaron Pham <[email protected]>
Signed-off-by: mgoin <[email protected]>
Co-authored-by: mgoin <[email protected]>
@zhengy001
[Speculative Decoding] Move indices to device before filtering output (…
f6084f6 
…vllm-project#10850)

Co-authored-by: Yang Zheng(SW)(Alex) <[email protected]>
@alexm-neuralmagic @mgoin @ywang96
[V1] VLM - Run the mm_mapper preprocessor in the frontend process (vl…
3bc94ca 
…lm-project#10640)

Signed-off-by: Roger Wang <[email protected]>
Co-authored-by: Michael Goin <[email protected]>
Co-authored-by: Roger Wang <[email protected]>
@yma11
[MISC][XPU] quick fix for XPU CI (vllm-project#10859)
2f2cdc7 
Signed-off-by: yan ma <[email protected]>
@mgoin
[Bugfix] Only require XGrammar on x86 (vllm-project#10865)
7090c27 
Signed-off-by: mgoin <[email protected]>
@tomeras91
[Frontend] correctly record prefill and decode time metrics (vllm-pro…
7c32b68 
…ject#10853)

Signed-off-by: Tomer Asida <[email protected]>
@gshtras
[Build][Bugfix] Using the correct type hint (vllm-project#10866)
a061fe6 
Signed-off-by: Gregory Shtrasberg <[email protected]>
@xuechendi @aarnphm
[Benchmark] Benchmark structured output with datasets (vllm-project#1…
381ac93 
…0557)

Signed-off-by: Aaron Pham <[email protected]>
Signed-off-by: Chendi Xue <[email protected]>
Co-authored-by: Aaron Pham <[email protected]>
@tlrmchlsmth
[CI/Build] Replace mean with torch.all in test_pynccl.py (vllm-projec…
d2bd88b 
…t#10876)

Signed-off-by: Tyler Michael Smith <[email protected]>
@wangxiyuan
Drop ROCm load format check (vllm-project#10767)
b5b647b 
Signed-off-by: wangxiyuan <[email protected]>
@khluu
[ci/build] Change queue name for Release jobs (vllm-project#10875)
fa2dea6
@khluu
[ci/build] Job to build and push release image (vllm-project#10877)
c9ca4fc
@o2363286
[bugfix] fixed parameter “n” when set parameter “bestof” > 1 (vllm-pr…
8db957e 
…oject#10854)

Signed-off-by: jianzheng <[email protected]>
@khluu
[ci/build] Update vLLM postmerge ECR repo (vllm-project#10887)
c92acb9
@xuechendi
Copy link
Author

@kzawora-intel , is it possible to do a force-merge? I found the created multi_models_upstream is based on habana_main. But since this Branch is targeted to base on upstream codes.

@xuechendi xuechendi force-pushed the multi_models_rebase branch from 0915219 to 655b610 Compare December 4, 2024 17:16
jikunshang and others added 4 commits December 4, 2024 20:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.