Code

Nov 18 rebase#485

Code

Merged

kzawora-intel merged 187 commits intohabana_mainHabanaAI/vllm-fork:habana_mainfrom private/kzawora/nov_12_rebaseHabanaAI/vllm-fork:private/kzawora/nov_12_rebaseCopy head branch name to clipboardNov 18, 2024

+20,768-7,375

Commits

Commits on Nov 6, 2024

• [Model][LoRA]LoRA support added for LlamaEmbeddingModel (vllm-project#10071)

  Show description for 2003cc3
  jeejeelee

  authored
  2003cc3

  Copy full SHA for 2003cc3

• [Model] Add Idefics3 support (vllm-project#9767)

  Show description for a5bba7d
  

  jeejeelee

  and

  B-201

  authored
  a5bba7d

  Copy full SHA for a5bba7d

• [Model][LoRA]LoRA support added for Qwen2VLForConditionalGeneration (vllm-project#10022)

  Show description for 406d4cc
  ericperfect

  authored
  406d4cc

  Copy full SHA for 406d4cc

• Remove ScaledActivation for AWQ (vllm-project#10057)

  Show description for 399c798
  mgoin

  authored
  399c798

  Copy full SHA for 399c798

• [CI/Build] Drop Python 3.8 support (vllm-project#10038)

  Show description for 098f94d
  

  russellb

  and

  DarkLight1337

  authored
  098f94d

  Copy full SHA for 098f94d

• [CI/Build] change conflict PR comment from mergify (vllm-project#10080)

  Show description for 87bd7e0
  russellb

  authored
  87bd7e0

  Copy full SHA for 87bd7e0

• [V1] Make v1 more testable (vllm-project#9888)

  Show description for d58268c
  joerunde

  authored
  d58268c

  Copy full SHA for d58268c

• [CI/Build] Always run the ruff workflow (vllm-project#10092)

  Show description for 74f2f8a
  russellb

  authored
  74f2f8a

  Copy full SHA for 74f2f8a

Commits on Nov 7, 2024

• [core][distributed] add stateless_init_process_group (vllm-project#10072)

  Show description for 719c1ca
  youkaichao

  authored
  719c1ca

  Copy full SHA for 719c1ca

• [Bugfix] Fix FP8 torch._scaled_mm fallback for torch>2.5 with CUDA<12.4 (vllm-project#10095)

  Show description for 4ab3256
  mgoin

  authored
  4ab3256

  Copy full SHA for 4ab3256

• [Misc][XPU] Upgrade to Pytorch 2.5 for xpu backend (vllm-project#9823)

  Show description for d3859f1
  

  yma11

  and

  jikunshang

  authored
  d3859f1

  Copy full SHA for d3859f1

• [Frontend] Adjust try/except blocks in API impl (vllm-project#10056)

  Show description for 29862b8
  njhill

  authored
  29862b8

  Copy full SHA for 29862b8

• [Hardware][CPU] Update torch 2.5 (vllm-project#9911)

  Show description for a4b3e0c
  bigPYJ1151

  authored
  a4b3e0c

  Copy full SHA for a4b3e0c

• [doc] add back Python 3.8 ABI (vllm-project#10100)

  Show description for e7b84c3
  youkaichao

  authored
  e7b84c3

  Copy full SHA for e7b84c3

• [V1][BugFix] Fix Generator construction in greedy + seed case (vllm-project#10097)

  Show description for 1fa020c
  njhill

  authored
  1fa020c

  Copy full SHA for 1fa020c

• [Misc] Consolidate ModelConfig code related to HF config (vllm-project#10104)

  Show description for db7db4a
  DarkLight1337

  authored
  db7db4a

  Copy full SHA for db7db4a

• [CI/Build] re-add codespell to CI (vllm-project#10083)

  Show description for 104d729
  russellb

  authored
  104d729

  Copy full SHA for 104d729

• Doc: Improve benchmark documentation (vllm-project#9927)

  Show description for d7263a1
  rafvasq

  authored
  d7263a1

  Copy full SHA for d7263a1

• [Core][Distributed] Refactor ipc buffer init in CustomAllreduce (vllm-project#10030)

  Show description for 6192e9b
  hanzhi713

  authored
  6192e9b

  Copy full SHA for 6192e9b

• [CI/Build] Improve mypy + python version matrix (vllm-project#10041)

  Show description for e036e52
  russellb

  authored
  e036e52

  Copy full SHA for e036e52

• Adds method to read the pooling types from model's files (vllm-project#9506)

  Show description for aa9078f
  

  flaviabeo

  and

  maxdebayser

  authored
  aa9078f

  Copy full SHA for aa9078f

• [Frontend] Fix multiple values for keyword argument error (vllm-project#10075) (vllm-project#10076)

  Show description for 0dfba97
  DIYer22

  authored
  0dfba97

  Copy full SHA for 0dfba97

• [Hardware][CPU][bugfix] Fix half dtype support on AVX2-only target (vllm-project#10108)

  Show description for a6f332d
  bigPYJ1151

  authored
  a6f332d

  Copy full SHA for a6f332d

• [Bugfix] Make image processor respect mm_processor_kwargs for Qwen2-VL (vllm-project#10112)

  Show description for 999df95
  li-plus

  authored
  999df95

  Copy full SHA for 999df95

• [Misc] Add Gamma-Distribution Request Generation Support for Serving Benchmark. (vllm-project#10105)

  Show description for a62bc01
  

  spliii

  and

  ywang96

  authored
  a62bc01

  Copy full SHA for a62bc01

• [Frontend] Tool calling parser for Granite 3.0 models (vllm-project#9027)

  Show description for ae62fd1
  maxdebayser

  authored
  ae62fd1

  Copy full SHA for ae62fd1

• [Feature] [Spec decode]: Combine chunked prefill with speculative decoding (vllm-project#9291)

  Show description for 9d43afc
  NickLucche

  authored
  9d43afc

  Copy full SHA for 9d43afc

• [CI/Build] Always run mypy (vllm-project#10122)

  Show description for de0e61a
  russellb

  authored
  de0e61a

  Copy full SHA for de0e61a

• [CI/Build] Add shell script linting using shellcheck (vllm-project#7925)

  Show description for 3be5b26
  russellb

  authored
  3be5b26

  Copy full SHA for 3be5b26

• [CI/Build] Automate PR body text cleanup (vllm-project#10082)

  Show description for a2f1f3b
  russellb

  authored
  a2f1f3b

  Copy full SHA for a2f1f3b

• Bump actions/setup-python from 5.2.0 to 5.3.0 (vllm-project#9745)

  Show description for 97b8475
  dependabot[bot]

  authored
  97b8475

  Copy full SHA for 97b8475

• Online video support for VLMs (vllm-project#10020)

  Show description for 28b2877
  

  3 peopleauthored
  28b2877

  Copy full SHA for 28b2877

• Bump actions/checkout from 4.2.1 to 4.2.2 (vllm-project#9746)

  Show description for 93bff42
  dependabot[bot]

  authored
  93bff42

  Copy full SHA for 93bff42

Commits on Nov 8, 2024

• [Misc] report relevant env vars in collect_env.py tool (vllm-project#9293)

  ycool

  authored
  073a472

  Copy full SHA for 073a472

• [V1] Add all_token_ids attribute to Request (vllm-project#10135)

  Show description for 42b4f46
  WoosukKwon

  authored
  42b4f46

  Copy full SHA for 42b4f46

• [V1] Prefix caching (take 2) (vllm-project#9972)

  Show description for 201fc07
  comaniac

  authored
  201fc07

  Copy full SHA for 201fc07

• [CI/Build] Give PR cleanup job PR write access (vllm-project#10139)

  Show description for 6bb52b0
  russellb

  authored
  6bb52b0

  Copy full SHA for 6bb52b0

• [Doc] Update FAQ links in spec_decode.rst (vllm-project#9662)

  Show description for 40d0e74
  whyiug

  authored
  40d0e74

  Copy full SHA for 40d0e74

• [Bugfix] Add error handling when server cannot respond any valid tokens (vllm-project#5895)

  DearPlanet

  authored
  ad39bd6

  Copy full SHA for ad39bd6

• [Misc] Fix ImportError causing by triton (vllm-project#9493)

  MengqingCao

  authored
  7371749

  Copy full SHA for 7371749

• [Doc] Move CONTRIBUTING to docs site (vllm-project#9924)

  Show description for 3a7f15a
  russellb

  authored
  3a7f15a

  Copy full SHA for 3a7f15a

• Fixes a typo about 'max_decode_seq_len' which causes crashes with cuda graph. (vllm-project#9285)

  Show description for da07a9e
  sighingnow

  authored
  da07a9e

  Copy full SHA for da07a9e

• Add hf_transfer to testing image (vllm-project#10096)

  mgoin

  authored
  aea6ad6

  Copy full SHA for aea6ad6

• [Misc] Fix typo in vllm-project#5895 (vllm-project#10145)

  Show description for f4c2187
  DarkLight1337

  authored
  f4c2187

  Copy full SHA for f4c2187

• [Bugfix][XPU] Fix xpu tp by introducing XpuCommunicator (vllm-project#10144)

  Show description for f10797c
  yma11

  authored
  f10797c

  Copy full SHA for f10797c

• [Model] Expose size to Idefics3 as mm_processor_kwargs (vllm-project#10146)

  Show description for 1ff4aed
  Isotr0py

  authored
  1ff4aed

  Copy full SHA for 1ff4aed

• [V1]Enable APC by default only for text models (vllm-project#10148)

  Show description for 208ce62
  ywang96

  authored
  208ce62

  Copy full SHA for 208ce62

• [CI/Build] Update CPU tests to include all "standard" tests (vllm-project#5481)

  Show description for b489fc3
  DarkLight1337

  authored
  b489fc3

  Copy full SHA for b489fc3

• Fix edge case Mistral tokenizer (vllm-project#10152)

  patrickvonplaten

  authored
  0535e5f

  Copy full SHA for 0535e5f

• Disable spec-decode + chunked-prefill for draft models with tensor parallelism > 1 (vllm-project#10136)

  Show description for f677862
  sroy745

  authored
  f677862

  Copy full SHA for f677862

• [Misc] Improve Web UI (vllm-project#10090)

  Show description for 6b30471
  rafvasq

  authored
  6b30471

  Copy full SHA for 6b30471

• [V1] Fix non-cudagraph op name (vllm-project#10166)

  Show description for b5815c8
  WoosukKwon

  authored
  b5815c8

  Copy full SHA for b5815c8

• [CI/Build] Ignore .gitignored files for shellcheck (vllm-project#10162)

  Show description for 87713c6
  ProExpertProg

  authored
  87713c6

  Copy full SHA for 87713c6

• Rename vllm.logging to vllm.logging_utils (vllm-project#10134)

  flozi00

  authored
  e1b5a82

  Copy full SHA for e1b5a82

• [torch.compile] Fuse RMSNorm with quant (vllm-project#9138)

  Show description for 4f93dfe
  

  ProExpertProg

  and

  youkaichao

  authored
  4f93dfe

  Copy full SHA for 4f93dfe

• [Bugfix] SymIntArrayRef expected to contain concrete integers (vllm-project#10170)

  Show description for 10b67d8
  bnellnm

  authored
  10b67d8

  Copy full SHA for 10b67d8

Commits on Nov 9, 2024

• [Kernel][Triton] Add Triton implementation for scaled_mm_triton to support fp8 and int8 SmoothQuant, symmetric case (vllm-project#9857)

  Show description for 127c074
  rasmith

  authored
  127c074

  Copy full SHA for 127c074

• [CI/Build] Adding timeout in CPU CI to avoid CPU test queue blocking (vllm-project#6892)

  Show description for d7edca1
  

  bigPYJ1151

  and

  DarkLight1337

  authored
  d7edca1

  Copy full SHA for d7edca1

• [0/N] Rename MultiModalInputs to MultiModalKwargs (vllm-project#10040)

  Show description for e0191a9
  DarkLight1337

  authored
  e0191a9

  Copy full SHA for e0191a9

• [Bugfix] Ignore GPTQ quantization of Qwen2-VL visual module (vllm-project#10169)

  Show description for f83fecc
  mgoin

  authored
  f83fecc

  Copy full SHA for f83fecc

• [CI/Build] Fix VLM broadcast tests tensor_parallel_size passing (vllm-project#10161)

  Show description for 47672f3
  Isotr0py

  authored
  47672f3

  Copy full SHA for 47672f3

• [Doc] Adjust RunLLM location (vllm-project#10176)

  Show description for 49d2a41
  DarkLight1337

  authored
  49d2a41

  Copy full SHA for 49d2a41

• [5/N] pass the whole config to model (vllm-project#9983)

  Show description for 1a95f10
  youkaichao

  authored
  1a95f10

  Copy full SHA for 1a95f10

• [CI/Build] Add run-hpu-test.sh script (vllm-project#10167)

  Show description for 8e1529d
  xuechendi

  authored
  8e1529d

  Copy full SHA for 8e1529d

• [Bugfix] Enable some fp8 and quantized fullgraph tests (vllm-project#10171)

  Show description for f192aeb
  bnellnm

  authored
  f192aeb

  Copy full SHA for f192aeb

• [bugfix] fix broken tests of mlp speculator (vllm-project#10177)

  Show description for bd46357
  youkaichao

  authored
  bd46357

  Copy full SHA for bd46357

• [doc] explaining the integration with huggingface (vllm-project#10173)

  Show description for 8a4358e
  youkaichao

  authored
  8a4358e

  Copy full SHA for 8a4358e

• bugfix: fix the bug that stream generate not work (vllm-project#2756)

  caijizhuo

  authored
  9e37266

  Copy full SHA for 9e37266

• [Frontend] add add_request_id middleware (vllm-project#9594)

  Show description for d88bff1
  cjackal

  authored
  d88bff1

  Copy full SHA for d88bff1

• [Frontend][Core] Override HF config.json via CLI (vllm-project#5836)

  Show description for b09895a
  

  KrishnaM251

  and

  DarkLight1337

  authored
  b09895a

  Copy full SHA for b09895a

• [CI/Build] Split up models tests (vllm-project#10069)

  Show description for 51c2e1f
  DarkLight1337

  authored
  51c2e1f

  Copy full SHA for 51c2e1f

Commits on Nov 10, 2024

• [ci][build] limit cmake version (vllm-project#10188)

  Show description for 9fa4bdd
  youkaichao

  authored
  9fa4bdd

  Copy full SHA for 9fa4bdd

• [Doc] Fix typo error in CONTRIBUTING.md (vllm-project#10190)

  Show description for 1968202
  FuryMartin

  authored
  1968202

  Copy full SHA for 1968202

• [doc] Polish the integration with huggingface doc (vllm-project#10195)

  Show description for bfb7d61
  

  CRZbulabula

  and

  youkaichao

  authored
  bfb7d61

  Copy full SHA for bfb7d61

• [Misc] small fixes to function tracing file path (vllm-project#9543)

  Show description for 20cf2f5
  

  ShawnD200

  and

  youkaichao

  authored
  20cf2f5

  Copy full SHA for 20cf2f5

Commits on Nov 11, 2024

• [misc] improve cloudpickle registration and tests (vllm-project#10202)

  Show description for 73b9083
  youkaichao

  authored
  73b9083

  Copy full SHA for 73b9083

• [Doc] Fix typo error in vllm/entrypoints/openai/cli_args.py (vllm-project#10196)

  yansh97

  authored
  ad9a78b

  Copy full SHA for ad9a78b

• [doc] improve debugging code (vllm-project#10206)

  Show description for f0f2e56
  youkaichao

  authored
  f0f2e56

  Copy full SHA for f0f2e56

• [6/N] pass whole config to inner model (vllm-project#10205)

  Show description for f89d18f
  youkaichao

  authored
  f89d18f

  Copy full SHA for f89d18f

• Bump the patch-update group with 5 updates (vllm-project#10210)

  Show description for 9804ac7
  dependabot[bot]

  authored
  9804ac7

  Copy full SHA for 9804ac7

• [Hardware][CPU] Add embedding models support for CPU backend (vllm-project#10193)

  Show description for 58170d6
  Isotr0py

  authored
  58170d6

  Copy full SHA for 58170d6

• [LoRA][Kernel] Remove the unused libentry module (vllm-project#10214)

  Show description for 36e4acd
  jeejeelee

  authored
  36e4acd

  Copy full SHA for 36e4acd

• [V1] Allow tokenizer_mode and trust_remote_code for Detokenizer (vllm-project#10211)

  Show description for 5fb1f93
  ywang96

  authored
  5fb1f93

  Copy full SHA for 5fb1f93

• [Bugfix][Hardware][CPU] Fix broken encoder-decoder CPU runner (vllm-project#10218)

  Show description for 2cebda4
  Isotr0py

  authored
  2cebda4

  Copy full SHA for 2cebda4

• [Metrics] add more metrics (vllm-project#4464)

  Show description for 874f551
  

  3 peopleauthored
  874f551

  Copy full SHA for 874f551

• [Doc] fix doc string typo in block_manager swap_out function (vllm-project#10212)

  yyccli

  authored
  36fc439

  Copy full SHA for 36fc439

• [core][distributed] add stateless process group (vllm-project#10216)

  Show description for e6de978
  youkaichao

  authored
  e6de978

  Copy full SHA for e6de978

• Bump actions/setup-python from 5.2.0 to 5.3.0 (vllm-project#10209)

  Show description for 25144ce
  dependabot[bot]

  authored
  25144ce

  Copy full SHA for 25144ce

• [V1] Fix detokenizer ports (vllm-project#10224)

  Show description for f9dadfb
  WoosukKwon

  authored
  f9dadfb

  Copy full SHA for f9dadfb

• [V1] Do not use inductor for piecewise CUDA graphs (vllm-project#10225)

  Show description for d7a4f22
  WoosukKwon

  authored
  d7a4f22

  Copy full SHA for d7a4f22

• [v1][torch.compile] support managing cudagraph buffer (vllm-project#10203)

  Show description for 330e82d
  

  youkaichao

  and

  WoosukKwon

  authored
  330e82d

  Copy full SHA for 330e82d

• [V1] Use custom ops for piecewise CUDA graphs (vllm-project#10227)

  Show description for fe15729
  WoosukKwon

  authored
  fe15729

  Copy full SHA for fe15729

• Add docs on serving with Llama Stack (vllm-project#10183)

  Show description for 4800339
  

  terrytangyuan

  and

  russellb

  authored
  4800339

  Copy full SHA for 4800339

• [misc][distributed] auto port selection and disable tests (vllm-project#10226)

  Show description for 8a7fe47
  youkaichao

  authored
  8a7fe47

  Copy full SHA for 8a7fe47

• [V1] Enable custom ops with piecewise CUDA graphs (vllm-project#10228)

  Show description for 9d5b4e4
  WoosukKwon

  authored
  9d5b4e4

  Copy full SHA for 9d5b4e4

• Make shutil rename in python_only_dev (vllm-project#10233)

  Show description for 08f93e7
  shcheglovnd

  authored
  08f93e7

  Copy full SHA for 08f93e7

• [V1] AsyncLLM Implementation (vllm-project#9826)

  Show description for 6ace6fb
  

  4 peopleauthored
  6ace6fb

  Copy full SHA for 6ace6fb

• [doc] update debugging guide (vllm-project#10236)

  Show description for d1c6799
  youkaichao

  authored
  d1c6799

  Copy full SHA for d1c6799

Commits on Nov 12, 2024

• [Doc] Update help text for --distributed-executor-backend (vllm-project#10231)

  Show description for 9cdba96
  russellb

  authored
  9cdba96

  Copy full SHA for 9cdba96

• [1/N] torch.compile user interface design (vllm-project#10237)

  Show description for eea55cc
  youkaichao

  authored
  eea55cc

  Copy full SHA for eea55cc

• [Misc][LoRA] Replace hardcoded cuda device with configurable argument (vllm-project#10223)

  Show description for 7f5edb5
  jeejeelee

  authored
  7f5edb5

  Copy full SHA for 7f5edb5

• Splitting attention kernel file (vllm-project#10091)

  Show description for 812c981
  

  maleksan85

  and

  Aleksandr Malyshev

  authored
  812c981

  Copy full SHA for 812c981

• [doc] explain the class hierarchy in vLLM (vllm-project#10240)

  Show description for 3a28f18
  youkaichao

  authored
  3a28f18

  Copy full SHA for 3a28f18

• [CI][CPU]refactor CPU tests to allow to bind with different cores (vllm-project#10222)

  Show description for d201d41
  zhouyuan

  authored
  d201d41

  Copy full SHA for d201d41

• [BugFix] Do not raise a ValueError when tool_choice is set to the supported none option and tools are not defined. (vllm-project#10000)

  Show description for 36c513a
  gcalmettes

  authored
  36c513a

  Copy full SHA for 36c513a

• Merge remote-tracking branch 'upstream/main' into HEAD

  kzawora-intel

  committed
  65a920e

  Copy full SHA for 65a920e

• [Misc]Fix Idefics3Model argument (vllm-project#10255)

  Show description for a838ba7
  jeejeelee

  authored
  a838ba7

  Copy full SHA for a838ba7

• [Bugfix] Fix QwenModel argument (vllm-project#10262)

  Show description for 176fcb1
  DamonFool

  authored
  176fcb1

  Copy full SHA for 176fcb1

• [Frontend] Add per-request number of cached token stats (vllm-project#10174)

  zifeitong

  authored
  47db6ec

  Copy full SHA for 47db6ec

• [V1] Use pickle for serializing EngineCoreRequest & Add multimodal inputs to EngineCoreRequest (vllm-project#10245)

  Show description for 7c65527
  WoosukKwon

  authored
  7c65527

  Copy full SHA for 7c65527

• [Encoder Decoder] Update Mllama to run with both FlashAttention and XFormers (vllm-project#9982)

  Show description for b41fb9d
  sroy745

  authored
  b41fb9d

  Copy full SHA for b41fb9d

• [LoRA] Adds support for bias in LoRA (vllm-project#5733)

  Show description for 8a06428
  

  followumesh

  and

  Umesh Deshpande

  authored
  8a06428

  Copy full SHA for 8a06428

• [V1] Enable Inductor when using piecewise CUDA graphs (vllm-project#10268)

  Show description for 1f55e05
  WoosukKwon

  authored
  1f55e05

  Copy full SHA for 1f55e05

• [doc] fix location of runllm widget (vllm-project#10266)

  Show description for 96ae0ea
  youkaichao

  authored
  96ae0ea

  Copy full SHA for 96ae0ea

• [doc] improve debugging doc (vllm-project#10270)

  Show description for 1808145
  youkaichao

  authored
  1808145

  Copy full SHA for 1808145

• Revert "[ci][build] limit cmake version" (vllm-project#10271)

  youkaichao

  authored
  377b74f

  Copy full SHA for 377b74f

Commits on Nov 13, 2024

• [V1] Fix CI tests on V1 engine (vllm-project#10272)

  Show description for 112fa0b
  WoosukKwon

  authored
  112fa0b

  Copy full SHA for 112fa0b

• [core][distributed] use tcp store directly (vllm-project#10275)

  Show description for 0d4ea3f
  youkaichao

  authored
  0d4ea3f

  Copy full SHA for 0d4ea3f

• [V1] Support VLMs with fine-grained scheduling (vllm-project#9871)

  Show description for bbd3e86
  

  WoosukKwon

  and

  ywang96

  authored
  bbd3e86

  Copy full SHA for bbd3e86

• Bump to compressed-tensors v0.8.0 (vllm-project#10279)

  Show description for 56a955e
  dsikka

  authored
  56a955e

  Copy full SHA for 56a955e

• [Doc] Fix typo in arg_utils.py (vllm-project#10264)

  Show description for 032fcf1
  xyang16

  authored
  032fcf1

  Copy full SHA for 032fcf1

• [Model] Add support for Qwen2-VL video embeddings input & multiple image embeddings input with varied resolutions (vllm-project#10221)

  Show description for 3945c82
  imkero

  authored
  3945c82

  Copy full SHA for 3945c82

• [Model] Adding Support for Qwen2VL as an Embedding Model. Using MrLight/dse-qwen2-2b-mrl-v1 (vllm-project#9944)

  Show description for 1b886aa
  

  FurtherAI

  and

  FurtherAI

  authored
  1b886aa

  Copy full SHA for 1b886aa

• [Core] Flashinfer - Remove advance step size restriction (vllm-project#10282)

  pavanimajety

  authored
  b6dde33

  Copy full SHA for b6dde33

• [Model][LoRA]LoRA support added for idefics3 (vllm-project#10281)

  Show description for d909acf
  B-201

  authored
  d909acf

  Copy full SHA for d909acf

• [V1] Add missing tokenizer options for Detokenizer (vllm-project#10288)

  Show description for bb7991a
  ywang96

  authored
  bb7991a

  Copy full SHA for bb7991a

• [1/N] Initial prototype for multi-modal processor (vllm-project#10044)

  Show description for 0b8bb86
  DarkLight1337

  authored
  0b8bb86

  Copy full SHA for 0b8bb86

• [Bugfix] bitsandbytes models fail to run pipeline parallel (vllm-project#10200)

  Show description for ac49b59
  HoangCongDuc

  authored
  ac49b59

  Copy full SHA for ac49b59

Commits on Nov 14, 2024

• [Bugfix] Fix tensor parallel for qwen2 classification model (vllm-project#10297)

  Show description for 15bb833
  Isotr0py

  authored
  15bb833

  Copy full SHA for 15bb833

• [misc] error early for old-style class (vllm-project#10304)

  Show description for 504ac53
  youkaichao

  authored
  504ac53

  Copy full SHA for 504ac53

• [Misc] format.sh: Simplify tool_version_check (vllm-project#10305)

  Show description for e0853b6
  russellb

  authored
  e0853b6

  Copy full SHA for e0853b6

• [Frontend] Pythonic tool parser (vllm-project#9859)

  Show description for f67ce05
  mdepinet

  authored
  f67ce05

  Copy full SHA for f67ce05

• [BugFix]: properly deserialize tool_calls iterator before processing by mistral-common when MistralTokenizer is used (vllm-project#9951)

  Show description for 52b48c1
  gcalmettes

  authored
  52b48c1

  Copy full SHA for 52b48c1

• [Model] Add BNB quantization support for Idefics3 (vllm-project#10310)

  Show description for 294bf46
  

  B-201

  and

  jeejeelee

  authored
  294bf46

  Copy full SHA for 294bf46

• [ci][distributed] disable hanging tests (vllm-project#10317)

  Show description for 29f3ef2
  youkaichao

  authored
  29f3ef2

  Copy full SHA for 29f3ef2

• [CI/Build] Fix CPU CI online inference timeout (vllm-project#10314)

  Show description for 03025c0
  Isotr0py

  authored
  03025c0

  Copy full SHA for 03025c0

• [CI/Build] Make shellcheck happy (vllm-project#10285)

  Show description for 675d603
  DarkLight1337

  authored
  675d603

  Copy full SHA for 675d603

• [Docs] Publish meetup slides (vllm-project#10331)

  Show description for 1dbae03
  WoosukKwon

  authored
  1dbae03

  Copy full SHA for 1dbae03

• Support Roberta embedding models (vllm-project#9387)

  Show description for 4a18fd1
  

  maxdebayser

  and

  flaviabeo

  authored
  4a18fd1

  Copy full SHA for 4a18fd1

Commits on Nov 15, 2024

• [Perf] Reduce peak memory usage of llama (vllm-project#10339)

  Show description for b2e0ad3
  andoorve

  authored
  b2e0ad3

  Copy full SHA for b2e0ad3

• [Bugfix] use AF_INET6 for OpenAI Compatible Server with ipv6 (vllm-project#9583)

  Show description for 554af92
  jxpxxzj

  authored
  554af92

  Copy full SHA for 554af92

• [Tool parsing] Improve / correct mistral tool parsing (vllm-project#10333)

  patrickvonplaten

  authored
  11cd1ae

  Copy full SHA for 11cd1ae

• [Bugfix] Fix unable to load some models (vllm-project#10312)

  Show description for 972112d
  DarkLight1337

  authored
  972112d

  Copy full SHA for 972112d

• [bugfix] Fix static asymmetric quantization case (vllm-project#10334)

  Show description for bf2ddc6
  

  ProExpertProg

  and

  danieldk

  authored
  bf2ddc6

  Copy full SHA for bf2ddc6

• [Misc] Change RedundantReshapesPass and FusionPass logging from info to debug (vllm-project#10308)

  Show description for 2885ba0
  tlrmchlsmth

  authored
  2885ba0

  Copy full SHA for 2885ba0

• [Model] Support Qwen2 embeddings and use tags to select model tests (vllm-project#10184)

  DarkLight1337

  authored
  b40cf64

  Copy full SHA for b40cf64

• [Bugfix] Qwen-vl output is inconsistent in speculative decoding (vllm-project#10350)

  skylee-01

  authored
  2ec8827

  Copy full SHA for 2ec8827

• [Misc] Consolidate pooler config overrides (vllm-project#10351)

  Show description for 2ac6d0e
  DarkLight1337

  authored
  2ac6d0e

  Copy full SHA for 2ac6d0e

• [Build] skip renaming files for release wheels pipeline (vllm-project#9671)

  Show description for 02dbf30
  simon-mo

  authored
  02dbf30

  Copy full SHA for 02dbf30

• Add default value to avoid Falcon crash (vllm-project#5363) (vllm-project#10347)

  Show description for 3d158cd
  wchen61

  authored
  3d158cd

  Copy full SHA for 3d158cd

• [Misc] Fix import error in tensorizer tests and cleanup some code (vllm-project#10349)

  Show description for b311efd
  DarkLight1337

  authored
  b311efd

  Copy full SHA for b311efd

• [Doc] Remove float32 choice from --lora-dtype (vllm-project#10348)

  Show description for 2690855
  xyang16

  authored
  2690855

  Copy full SHA for 2690855

• [Bugfix] Fix fully sharded LoRA bug (vllm-project#10352)

  Show description for 1d65ec7
  jeejeelee

  authored
  1d65ec7

  Copy full SHA for 1d65ec7

• [Misc] Fix some help info of arg_utils to improve readability (vllm-project#10362)

  ShangmingCai

  authored
  f2056f7

  Copy full SHA for f2056f7

• [core][misc] keep compatibility for old-style classes (vllm-project#10356)

  Show description for 3a763ba
  youkaichao

  authored
  3a763ba

  Copy full SHA for 3a763ba

• [Bugfix] Ensure special tokens are properly filtered out for guided structured output with MistralTokenizer (vllm-project#10363)

  Show description for 691a3ec
  gcalmettes

  authored
  691a3ec

  Copy full SHA for 691a3ec

• [Misc] Bump up test_fused_moe tolerance (vllm-project#10364)

  Show description for 79ee45b
  ElizaWszola

  authored
  79ee45b

  Copy full SHA for 79ee45b

• [Misc] bump mistral common version (vllm-project#10367)

  Show description for a6221a1
  simon-mo

  authored
  a6221a1

  Copy full SHA for a6221a1

• [Docs] Add Nebius as sponsors (vllm-project#10371)

  Show description for c76ac49
  simon-mo

  authored
  c76ac49

  Copy full SHA for c76ac49

• [Frontend] Add --version flag to CLI (vllm-project#10369)

  Show description for a067f85
  russellb

  authored
  a067f85

  Copy full SHA for a067f85

• [Doc] Move PR template content to docs (vllm-project#10159)

  Show description for 3e8d14d
  russellb

  authored
  3e8d14d

  Copy full SHA for 3e8d14d

• [Docs] Misc updates to TPU installation instructions (vllm-project#10165)

  mikegre-google

  authored
  4f168f6

  Copy full SHA for 4f168f6

Commits on Nov 16, 2024

• [Frontend] Automatic detection of chat content format from AST (vllm-project#9919)

  Show description for 32e46e0
  DarkLight1337

  authored
  32e46e0

  Copy full SHA for 32e46e0

• [doc] add doc for the plugin system (vllm-project#10372)

  Show description for 755b853
  youkaichao

  authored
  755b853

  Copy full SHA for 755b853

• [misc][plugin] improve log messages (vllm-project#10386)

  Show description for 2f427c2
  youkaichao

  authored
  2f427c2

  Copy full SHA for 2f427c2

• [BugFix] [Kernel] Fix GPU SEGV occuring in fused_moe kernel (vllm-project#10385)

  Show description for 1d75472
  rasmith

  authored
  1d75472

  Copy full SHA for 1d75472

• [Misc] Update benchmark to support image_url file or http (vllm-project#10287)

  Show description for 8b6725b
  kakao-steve-ai

  authored
  8b6725b

  Copy full SHA for 8b6725b

• [Misc] Medusa supports custom bias (vllm-project#10361)

  skylee-01

  authored
  b98d89e

  Copy full SHA for b98d89e

• [Bugfix] Fix M-RoPE position calculation when chunked prefill is enabled (vllm-project#10388)

  Show description for 361c29e
  imkero

  authored
  361c29e

  Copy full SHA for 361c29e

• [V1] Add code owners for V1 (vllm-project#10397)

  Show description for 661a34f
  WoosukKwon

  authored
  661a34f

  Copy full SHA for 661a34f

Commits on Nov 17, 2024

• [2/N][torch.compile] make compilation cfg part of vllm cfg (vllm-project#10383)

  Show description for 4fd9375
  youkaichao

  authored
  4fd9375

  Copy full SHA for 4fd9375

• [V1] Refactor model executable interface for all text-only language models (vllm-project#10374)

  Show description for 643ecf7
  ywang96

  authored
  643ecf7

  Copy full SHA for 643ecf7

• [CI/Build] Fix IDC hpu [Device not found] issue (vllm-project#10384)

  Show description for 905d0f0
  xuechendi

  authored
  905d0f0

  Copy full SHA for 905d0f0

• [Bugfix][CPU] Fix CPU embedding runner with tensor parallel (vllm-project#10394)

  Show description for cf349c4
  Isotr0py

  authored
  cf349c4

  Copy full SHA for cf349c4

• [platforms] refactor cpu code (vllm-project#10402)

  Show description for 8d74b5a
  youkaichao

  authored
  8d74b5a

  Copy full SHA for 8d74b5a

• [Hardware] [HPU]add mark_step for hpu (vllm-project#10239)

  Show description for 76aab90
  jikunshang

  authored
  76aab90

  Copy full SHA for 76aab90

• [Bugfix] Fix mrope_position_delta in non-last prefill chunk (vllm-project#10403)

  Show description for 80d85c5
  imkero

  authored
  80d85c5

  Copy full SHA for 80d85c5

• [Misc] Enhance offline_inference to support user-configurable paramet… (vllm-project#10392)

  Show description for d1557e6
  wchen61

  authored
  d1557e6

  Copy full SHA for d1557e6

Commits on Nov 18, 2024

• [Misc] Add uninitialized params tracking for AutoWeightsLoader (vllm-project#10327)

  Show description for c4e4643
  Isotr0py

  authored
  c4e4643

  Copy full SHA for c4e4643

• [Bugfix] Ignore ray reinit error when current platform is ROCm or XPU (vllm-project#10375)

  Show description for 47826ca
  HollowMan6

  authored
  47826ca

  Copy full SHA for 47826ca

• [4/N][torch.compile] clean up set_torch_compile_backend (vllm-project#10401)

  Show description for 51bb12d
  youkaichao

  authored
  51bb12d

  Copy full SHA for 51bb12d

• [VLM] Report multi_modal_placeholders in output (vllm-project#10407)

  Show description for c7dec92
  lk-chen

  authored
  c7dec92

  Copy full SHA for c7dec92

• [Model] Remove redundant softmax when using PoolingType.STEP (vllm-project#10415)

  Maybewuss

  authored
  01aae1c

  Copy full SHA for 01aae1c

• Merge remote-tracking branch 'origin/habana_main' into HEAD

  kzawora-intel

  committed
  9ebcb9b

  Copy full SHA for 9ebcb9b

• Merge remote-tracking branch 'upstream/main' into HEAD

  kzawora-intel

  committed
  295cabe

  Copy full SHA for 295cabe

• Merge remote-tracking branch 'origin/habana_main' into HEAD

  kzawora-intel

  committed
  8155ba7

  Copy full SHA for 8155ba7

• format.sh

  kzawora-intel

  committed
  3400180

  Copy full SHA for 3400180