Skip to content

Issues: ray-project/llmperf

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

28 Open 2 Closed
28 Open 2 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

Support other scenarios
#78 opened Oct 28, 2024 by yw-furiosa
how to test vllm interface
#77 opened Oct 25, 2024 by ZTurboX
Using max_concurrency or AsyncActor?
#76 opened Oct 21, 2024 by chiragjn
Wrong generated prompt length
#75 opened Oct 18, 2024 by PietroFerr
Error while using pip
#70 opened Sep 16, 2024 by avi7611
1
Contributions guidelines
#69 opened Sep 16, 2024 by miloszwatroba
Azure OpenAi Endpoint Support
#67 opened Aug 22, 2024 by gujju62
When max-num-completed-requests is not divisible by num-concurrent-requests, an error will occur.
#63 opened Aug 9, 2024 by huangdi614
How to use llmperf to test ollama performance (TTFT, etc)
#58 opened Jun 21, 2024 by alexhegit
1
Subsequent requests cannot be sent until 'num_concurrent_requests' requests have all finished
#56 opened Jun 18, 2024 by llsj14
5
Divide by zero: request_metrics[common_metrics.REQ_OUTPUT_THROUGHPUT] = num_output_tokens / request_metrics[common_metrics.E2E_LAT]
#55 opened Jun 13, 2024 by yaronr
7
Sagemaker client issue
#53 opened Jun 7, 2024 by SuchethaChintha
2
error in request_metrics dictionary implementation
#49 opened May 14, 2024 by Durga2Dash
Vertex AI API needs to be updated.
#48 opened May 14, 2024 by Durga2Dash
Since it does not have a 'setup.py' nor a 'setup.cfg', it cannot be installed in editable mode PEP 660
#47 opened May 9, 2024 by focusunsink
1
unable to get the benchmark result
#44 opened Apr 4, 2024 by dipshirajput
1
Blocking on pending requests despite block == false
#43 opened Apr 3, 2024 by dacorvo
1
HUGGINGFACE set
#38 opened Mar 12, 2024 by capyun
bug of counting output tokens
#35 opened Feb 22, 2024 by irasin
llmperf not working for concurrent users
#34 opened Feb 13, 2024 by nkanike07
Bug: Hugging Face TGI not working
#33 opened Feb 7, 2024 by ptrmayer
Concurrency level is not handled properly
#32 opened Jan 26, 2024 by alexeykudinkin
Add memory bandwidth utilization metric
#31 opened Jan 25, 2024 by mmcclean-aws
Handling custom codes arguments like trust_remote_code
#30 opened Jan 11, 2024 by Akash08naik
usage for local models .
#29 opened Jan 11, 2024 by Akash08naik
ProTip! Mix and match filters to narrow down what you’re looking for.