Skip to content

Commit

Permalink
add rest skip_codes doc; make skip_codes take precedence over ratelim…
Browse files Browse the repository at this point in the history
…it_codes
  • Loading branch information
leondz committed Nov 12, 2024
1 parent 0577ca0 commit 645b585
Show file tree
Hide file tree
Showing 2 changed files with 7 additions and 5 deletions.
1 change: 1 addition & 0 deletions docs/source/garak.generators.rest.rst
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ Uses the following options from ``_config.plugins.generators["rest.RestGenerator
* ``response_json_field`` - (optional) Which field of the response JSON should be used as the output string? Default ``text``. Can also be a JSONPath value, and ``response_json_field`` is used as such if it starts with ``$``.
* ``request_timeout`` - How many seconds should we wait before timing out? Default 20
* ``ratelimit_codes`` - Which endpoint HTTP response codes should be caught as indicative of rate limiting and retried? ``List[int]``, default ``[429]``
* ``skip_codes`` - Which endpoint HTTP response code should lead to the generation being treated as not possible and skipped for this query. Takes precedence over ``skip_codes``.

Templates can be either a string or a JSON-serialisable Python object.
Instance of ``$INPUT`` here are replaced with the prompt; instances of ``$KEY``
Expand Down
11 changes: 6 additions & 5 deletions garak/generators/rest.py
Original file line number Diff line number Diff line change
Expand Up @@ -194,17 +194,18 @@ def _call_model(
"timeout": self.request_timeout,
}
resp = self.http_function(self.uri, **req_kArgs)
if resp.status_code in self.ratelimit_codes:
raise RateLimitHit(
f"Rate limited: {resp.status_code} - {resp.reason}, uri: {self.uri}"
)

elif resp.status_code in self.skip_codes:
if resp.status_code in self.skip_codes:
logging.debug(
f"REST skip prompt: {resp.status_code} - {resp.reason}, uri: {self.uri}"
)
return [None]

elif resp.status_code in self.ratelimit_codes:
raise RateLimitHit(
f"Rate limited: {resp.status_code} - {resp.reason}, uri: {self.uri}"
)

elif str(resp.status_code)[0] == "3":
raise NotImplementedError(
f"REST URI redirection: {resp.status_code} - {resp.reason}, uri: {self.uri}"
Expand Down

0 comments on commit 645b585

Please sign in to comment.