Skip to content

Commit

Permalink
more work
Browse files Browse the repository at this point in the history
  • Loading branch information
jmhessel committed Aug 14, 2024
1 parent 50eda29 commit 5466fd9
Showing 1 changed file with 1 addition and 3 deletions.
4 changes: 1 addition & 3 deletions src/llmperf/ray_clients/openai_chat_completions_client.py
Original file line number Diff line number Diff line change
Expand Up @@ -114,11 +114,9 @@ def llm_request(self, request_config: RequestConfig) -> Dict[str, Any]:
output_throughput = tokens_received / total_request_time

except Exception as e:
import traceback
stack_trace = traceback.format_exc()
metrics[common_metrics.ERROR_MSG] = error_msg
metrics[common_metrics.ERROR_CODE] = error_response_code
raise ValueError(f"Warning Or Error: {e} \n {stack_trace}")
print(f"Warning Or Error: {e}")
print(error_response_code)

metrics[common_metrics.INTER_TOKEN_LAT] = sum(time_to_next_token) #This should be same as metrics[common_metrics.E2E_LAT]. Leave it here for now
Expand Down

0 comments on commit 5466fd9

Please sign in to comment.