Skip to content

Commit

Permalink
Reimplement retries
Browse files Browse the repository at this point in the history
  • Loading branch information
hakanensari committed Dec 17, 2024
1 parent 9212e9b commit c1bbbf3
Show file tree
Hide file tree
Showing 58 changed files with 605 additions and 890 deletions.
16 changes: 15 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -102,10 +102,24 @@ end

### Rate limiting

Amazon’s Selling Partner API (SP-API) imposes [rate limits][rate-limits] on most operations. Peddler respects these limits and automatically backs off when throttled. You can override the default rate limit by passing a `:rate_limit` to the operation. You can also provide an optional `:tries` argument to specify the number of request attempts, including retries, before giving up. By default, we retry once.
Amazon’s Selling Partner API (SP-API) imposes [rate limits][rate-limits] on most operations. Peddler respects these limits and will automatically back off when throttled. You can override the default rate limit by passing a `:rate_limit` to the operation.

You can also provide an optional `:retries` argument when initializing an API to specify the number of retries if throttled. By default, this is set to 0, meaning no retries will be attempted. If set to a positive value, Peddler will retry the request that many times if throttled, backing off based on the specified rate limit.

**Note:** This functionality requires version 6 of the underlying [HTTP library][httprb]. As of writing, this is not released yet. To use rate limiting, point to their main branch on GitHub.

Example usage:

```ruby
api = Peddler.orders_v0(aws_region, access_token, retries: 3)
api.get_orders(
marketplaceIds: ["ATVPDKIKX0DER"],
createdAfter: "2023-01-01T00:00:00Z"
)
```

In this example, if the request to `get_orders` is throttled, Peddler will retry the request up to 3 times, backing off according to the rate limit specified by Amazon.

### The APIs

Peddler provides a class for each API version under an eponymous namespace. Below is a list of the more important APIs, along with brief descriptions and code examples to help you get started.
Expand Down
11 changes: 0 additions & 11 deletions lib/generator/parameter_builder.rb
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,6 @@ def build
parameters = parameters.select { |p| p["name"] }
if @rate_limit
parameters << build_rate_limit_param
parameters << build_tries_param
end
parameters << build_notification_type_param if needs_notification_type?

Expand All @@ -32,16 +31,6 @@ def build_rate_limit_param
}
end

def build_tries_param
{
"name" => "tries",
"type" => "Integer",
"required" => false,
"description" => "Total request attempts, including retries",
"default" => 2,
}
end

def build_notification_type_param
{
"name" => "notification_type",
Expand Down
2 changes: 1 addition & 1 deletion lib/generator/templates/operation.rb.erb
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ path = "<%= path %>"
}.compact
<% end -%>
<% if rate_limit %>
meter(rate_limit, tries:).<%= verb %>(<%= request_args.join(", ") %>)
meter(rate_limit).<%= verb %>(<%= request_args.join(", ") %>)
<% else %>
<%= verb %>(<%= request_args.join(", ") %>)
<% end %>
Expand Down
15 changes: 11 additions & 4 deletions lib/peddler/api.rb
Original file line number Diff line number Diff line change
Expand Up @@ -25,11 +25,17 @@ class << self
# @return [String]
attr_reader :access_token

# Number of retries if throttled (default: 0)
#
# @return [Integer]
attr_reader :retries

# @param [String] aws_region
# @param [String] access_token
def initialize(aws_region, access_token)
def initialize(aws_region, access_token, retries: 0)
@endpoint = Endpoint.find(aws_region)
@access_token = access_token
@retries = retries
@sandbox = false
end

Expand Down Expand Up @@ -67,13 +73,14 @@ def http
# Throttles with a rate limit and retries when the API returns a 429
#
# @param [Float] requests_per_second
# @param [Integer] tries
# @return [self]
def meter(requests_per_second, tries: 2)
def meter(requests_per_second)
return self if retries.zero?

# HTTP v6.0 will implement retriable. Until then, point to their GitHub repo, or it's a no-op.
# https://github.com/httprb/http/pull/790
delay = sandbox? ? 0.2 : 1.0 / requests_per_second
retriable(delay:, tries:, retry_statuses: [429])
retriable(delay:, tries: retries + 1, retry_statuses: [429])

self
end
Expand Down
16 changes: 6 additions & 10 deletions lib/peddler/apis/amazon_warehousing_and_distribution_2024_05_09.rb
Original file line number Diff line number Diff line change
Expand Up @@ -22,15 +22,14 @@ class AmazonWarehousingAndDistribution20240509 < API
# @param sku_quantities [String] If equal to `SHOW`, the response includes the shipment SKU quantity details.
# Defaults to `HIDE`, in which case the response does not contain SKU quantities
# @param rate_limit [Float] Requests per second
# @param tries [Integer] Total request attempts, including retries
# @return [Peddler::Response] The API response
def get_inbound_shipment(shipment_id, sku_quantities: nil, rate_limit: 2.0, tries: 2)
def get_inbound_shipment(shipment_id, sku_quantities: nil, rate_limit: 2.0)
path = "/awd/2024-05-09/inboundShipments/#{shipment_id}"
params = {
"skuQuantities" => sku_quantities,
}.compact

meter(rate_limit, tries:).get(path, params:)
meter(rate_limit).get(path, params:)
end

# Retrieves a summary of all the inbound AWD shipments associated with a merchant, with the ability to apply
Expand All @@ -48,10 +47,9 @@ def get_inbound_shipment(shipment_id, sku_quantities: nil, rate_limit: 2.0, trie
# @param max_results [Integer] Maximum number of results to return.
# @param next_token [String] Token to retrieve the next set of paginated results.
# @param rate_limit [Float] Requests per second
# @param tries [Integer] Total request attempts, including retries
# @return [Peddler::Response] The API response
def list_inbound_shipments(sort_by: nil, sort_order: nil, shipment_status: nil, updated_after: nil,
updated_before: nil, max_results: 25, next_token: nil, rate_limit: 1.0, tries: 2)
updated_before: nil, max_results: 25, next_token: nil, rate_limit: 1.0)
path = "/awd/2024-05-09/inboundShipments"
params = {
"sortBy" => sort_by,
Expand All @@ -63,7 +61,7 @@ def list_inbound_shipments(sort_by: nil, sort_order: nil, shipment_status: nil,
"nextToken" => next_token,
}.compact

meter(rate_limit, tries:).get(path, params:)
meter(rate_limit).get(path, params:)
end

# Lists AWD inventory associated with a merchant with the ability to apply optional filters.
Expand All @@ -76,10 +74,8 @@ def list_inbound_shipments(sort_by: nil, sort_order: nil, shipment_status: nil,
# @param next_token [String] Token to retrieve the next set of paginated results.
# @param max_results [Integer] Maximum number of results to return.
# @param rate_limit [Float] Requests per second
# @param tries [Integer] Total request attempts, including retries
# @return [Peddler::Response] The API response
def list_inventory(sku: nil, sort_order: nil, details: nil, next_token: nil, max_results: 25, rate_limit: 2.0,
tries: 2)
def list_inventory(sku: nil, sort_order: nil, details: nil, next_token: nil, max_results: 25, rate_limit: 2.0)
path = "/awd/2024-05-09/inventory"
params = {
"sku" => sku,
Expand All @@ -89,7 +85,7 @@ def list_inventory(sku: nil, sort_order: nil, details: nil, next_token: nil, max
"maxResults" => max_results,
}.compact

meter(rate_limit, tries:).get(path, params:)
meter(rate_limit).get(path, params:)
end
end
end
Expand Down
50 changes: 20 additions & 30 deletions lib/peddler/apis/aplus_content_2020_11_01.rb
Original file line number Diff line number Diff line change
Expand Up @@ -28,9 +28,8 @@ class AplusContent20201101 < API
# other parameter will cause the request to fail. When no nextPageToken value is returned there are no more
# pages to return. A pageToken value is not usable across different operations.
# @param rate_limit [Float] Requests per second
# @param tries [Integer] Total request attempts, including retries
# @return [Peddler::Response] The API response
def search_content_documents(marketplace_id, page_token: nil, rate_limit: 10.0, tries: 2)
def search_content_documents(marketplace_id, page_token: nil, rate_limit: 10.0)
cannot_sandbox!

path = "/aplus/2020-11-01/contentDocuments"
Expand All @@ -39,17 +38,16 @@ def search_content_documents(marketplace_id, page_token: nil, rate_limit: 10.0,
"pageToken" => page_token,
}.compact

meter(rate_limit, tries:).get(path, params:)
meter(rate_limit).get(path, params:)
end

# Creates a new A+ Content document.
#
# @param marketplace_id [String] The identifier for the marketplace where the A+ Content is published.
# @param post_content_document_request [Hash] The content document request details.
# @param rate_limit [Float] Requests per second
# @param tries [Integer] Total request attempts, including retries
# @return [Peddler::Response] The API response
def create_content_document(marketplace_id, post_content_document_request, rate_limit: 10.0, tries: 2)
def create_content_document(marketplace_id, post_content_document_request, rate_limit: 10.0)
cannot_sandbox!

path = "/aplus/2020-11-01/contentDocuments"
Expand All @@ -58,7 +56,7 @@ def create_content_document(marketplace_id, post_content_document_request, rate_
"marketplaceId" => marketplace_id,
}.compact

meter(rate_limit, tries:).post(path, body:, params:)
meter(rate_limit).post(path, body:, params:)
end

# Returns an A+ Content document, if available.
Expand All @@ -69,9 +67,8 @@ def create_content_document(marketplace_id, post_content_document_request, rate_
# @param marketplace_id [String] The identifier for the marketplace where the A+ Content is published.
# @param included_data_set [Array<String>] The set of A+ Content data types to include in the response.
# @param rate_limit [Float] Requests per second
# @param tries [Integer] Total request attempts, including retries
# @return [Peddler::Response] The API response
def get_content_document(content_reference_key, marketplace_id, included_data_set, rate_limit: 10.0, tries: 2)
def get_content_document(content_reference_key, marketplace_id, included_data_set, rate_limit: 10.0)
cannot_sandbox!

path = "/aplus/2020-11-01/contentDocuments/#{content_reference_key}"
Expand All @@ -80,7 +77,7 @@ def get_content_document(content_reference_key, marketplace_id, included_data_se
"includedDataSet" => included_data_set,
}.compact

meter(rate_limit, tries:).get(path, params:)
meter(rate_limit).get(path, params:)
end

# Updates an existing A+ Content document.
Expand All @@ -91,10 +88,9 @@ def get_content_document(content_reference_key, marketplace_id, included_data_se
# @param marketplace_id [String] The identifier for the marketplace where the A+ Content is published.
# @param post_content_document_request [Hash] The content document request details.
# @param rate_limit [Float] Requests per second
# @param tries [Integer] Total request attempts, including retries
# @return [Peddler::Response] The API response
def update_content_document(content_reference_key, marketplace_id, post_content_document_request,
rate_limit: 10.0, tries: 2)
rate_limit: 10.0)
cannot_sandbox!

path = "/aplus/2020-11-01/contentDocuments/#{content_reference_key}"
Expand All @@ -103,7 +99,7 @@ def update_content_document(content_reference_key, marketplace_id, post_content_
"marketplaceId" => marketplace_id,
}.compact

meter(rate_limit, tries:).post(path, body:, params:)
meter(rate_limit).post(path, body:, params:)
end

# Returns a list of ASINs related to the specified A+ Content document, if available. If you do not include the
Expand All @@ -122,10 +118,9 @@ def update_content_document(content_reference_key, marketplace_id, post_content_
# other parameter will cause the request to fail. When no nextPageToken value is returned there are no more
# pages to return. A pageToken value is not usable across different operations.
# @param rate_limit [Float] Requests per second
# @param tries [Integer] Total request attempts, including retries
# @return [Peddler::Response] The API response
def list_content_document_asin_relations(content_reference_key, marketplace_id, included_data_set: nil,
asin_set: nil, page_token: nil, rate_limit: 10.0, tries: 2)
asin_set: nil, page_token: nil, rate_limit: 10.0)
cannot_sandbox!

path = "/aplus/2020-11-01/contentDocuments/#{content_reference_key}/asins"
Expand All @@ -136,7 +131,7 @@ def list_content_document_asin_relations(content_reference_key, marketplace_id,
"pageToken" => page_token,
}.compact

meter(rate_limit, tries:).get(path, params:)
meter(rate_limit).get(path, params:)
end

# Replaces all ASINs related to the specified A+ Content document, if available. This may add or remove ASINs,
Expand All @@ -149,10 +144,9 @@ def list_content_document_asin_relations(content_reference_key, marketplace_id,
# @param marketplace_id [String] The identifier for the marketplace where the A+ Content is published.
# @param post_content_document_asin_relations_request [Hash] The content document ASIN relations request details.
# @param rate_limit [Float] Requests per second
# @param tries [Integer] Total request attempts, including retries
# @return [Peddler::Response] The API response
def post_content_document_asin_relations(content_reference_key, marketplace_id,
post_content_document_asin_relations_request, rate_limit: 10.0, tries: 2)
post_content_document_asin_relations_request, rate_limit: 10.0)
cannot_sandbox!

path = "/aplus/2020-11-01/contentDocuments/#{content_reference_key}/asins"
Expand All @@ -161,7 +155,7 @@ def post_content_document_asin_relations(content_reference_key, marketplace_id,
"marketplaceId" => marketplace_id,
}.compact

meter(rate_limit, tries:).post(path, body:, params:)
meter(rate_limit).post(path, body:, params:)
end

# Checks if the A+ Content document is valid for use on a set of ASINs.
Expand All @@ -170,10 +164,9 @@ def post_content_document_asin_relations(content_reference_key, marketplace_id,
# @param asin_set [Array<String>] The set of ASINs.
# @param post_content_document_request [Hash] The content document request details.
# @param rate_limit [Float] Requests per second
# @param tries [Integer] Total request attempts, including retries
# @return [Peddler::Response] The API response
def validate_content_document_asin_relations(marketplace_id, post_content_document_request, asin_set: nil,
rate_limit: 10.0, tries: 2)
rate_limit: 10.0)
cannot_sandbox!

path = "/aplus/2020-11-01/contentAsinValidations"
Expand All @@ -183,7 +176,7 @@ def validate_content_document_asin_relations(marketplace_id, post_content_docume
"asinSet" => asin_set,
}.compact

meter(rate_limit, tries:).post(path, body:, params:)
meter(rate_limit).post(path, body:, params:)
end

# Searches for A+ Content publishing records, if available.
Expand All @@ -196,9 +189,8 @@ def validate_content_document_asin_relations(marketplace_id, post_content_docume
# other parameter will cause the request to fail. When no nextPageToken value is returned there are no more
# pages to return. A pageToken value is not usable across different operations.
# @param rate_limit [Float] Requests per second
# @param tries [Integer] Total request attempts, including retries
# @return [Peddler::Response] The API response
def search_content_publish_records(marketplace_id, asin, page_token: nil, rate_limit: 10.0, tries: 2)
def search_content_publish_records(marketplace_id, asin, page_token: nil, rate_limit: 10.0)
cannot_sandbox!

path = "/aplus/2020-11-01/contentPublishRecords"
Expand All @@ -208,7 +200,7 @@ def search_content_publish_records(marketplace_id, asin, page_token: nil, rate_l
"pageToken" => page_token,
}.compact

meter(rate_limit, tries:).get(path, params:)
meter(rate_limit).get(path, params:)
end

# Submits an A+ Content document for review, approval, and publishing.
Expand All @@ -218,17 +210,16 @@ def search_content_publish_records(marketplace_id, asin, page_token: nil, rate_l
# any A+ content identifier.
# @param marketplace_id [String] The identifier for the marketplace where the A+ Content is published.
# @param rate_limit [Float] Requests per second
# @param tries [Integer] Total request attempts, including retries
# @return [Peddler::Response] The API response
def post_content_document_approval_submission(content_reference_key, marketplace_id, rate_limit: 10.0, tries: 2)
def post_content_document_approval_submission(content_reference_key, marketplace_id, rate_limit: 10.0)
cannot_sandbox!

path = "/aplus/2020-11-01/contentDocuments/#{content_reference_key}/approvalSubmissions"
params = {
"marketplaceId" => marketplace_id,
}.compact

meter(rate_limit, tries:).post(path, params:)
meter(rate_limit).post(path, params:)
end

# Submits a request to suspend visible A+ Content. This neither deletes the content document nor the ASIN
Expand All @@ -239,17 +230,16 @@ def post_content_document_approval_submission(content_reference_key, marketplace
# any A+ content identifier.
# @param marketplace_id [String] The identifier for the marketplace where the A+ Content is published.
# @param rate_limit [Float] Requests per second
# @param tries [Integer] Total request attempts, including retries
# @return [Peddler::Response] The API response
def post_content_document_suspend_submission(content_reference_key, marketplace_id, rate_limit: 10.0, tries: 2)
def post_content_document_suspend_submission(content_reference_key, marketplace_id, rate_limit: 10.0)
cannot_sandbox!

path = "/aplus/2020-11-01/contentDocuments/#{content_reference_key}/suspendSubmissions"
params = {
"marketplaceId" => marketplace_id,
}.compact

meter(rate_limit, tries:).post(path, params:)
meter(rate_limit).post(path, params:)
end
end
end
Expand Down
Loading

0 comments on commit c1bbbf3

Please sign in to comment.