413 PAYLOAD_TOO_LARGE using a pre-signed URL

[hihosilvers]

Hi,
I'm getting a 413 when trying to upload an image of 1.9 MB to S3Mock using a pre-signed URL.

When I try to upload the same photo using the aws-sdk it works fine.
But when I try to use the pre-signed URL I'm getting a 413.

Container logging:

Has anyone encountered this before?

Pre-signed URL was generated with the following arguments (using boto3 but the arguments should be the same as in Java):

presigned_post_info = client.generate_presigned_post(
    Bucket=bucket_name,
    Key=key,
    Fields={"acl": acl, "Content-Type": "image/jpeg"},
    Conditions=[
        {"acl": acl},
        {"success_action_status": "201"},
        {"Content-Type": "image/jpeg"},
    ],
    ExpiresIn=expires_in,
) 

[afranken]

I see that you used generate_presigned_post to generate a URL that you want to use to upload data.

The put object API accepts only PUT requests:
https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutObject.html
https://docs.aws.amazon.com/AmazonS3/latest/userguide/PresignedUrlUploadObject.html

There are a few APIs that accept POST requests, S3Mock will try to best match the incoming request to a matching API given the parameters and the HTTP method of the request, so we can't return a good error message here.

[hihosilvers]

@afranken thanks for your reply.

Ok, so it looks like a production S3 endpoint should accept a POST with a form to create an object. The presigned_post URL is a bit different than a typical pre-signed URL generated through the aws-sdk because the URL itself is just the bucket address, the body of the request includes further form data (acl, Content-Type) and you can also attach conditions (e.g. content-length-range)

https://boto3.amazonaws.com/v1/documentation/api/latest/guide/s3-presigned-urls.html#generating-a-presigned-url-to-upload-a-file

It might be worthwhile taking a look at potentially adding this functionality. While the Java aws-sdk doesn't generate this type of presigned URL (yet?), packages for other languages, e.g. for Python, Javascript, Ruby, and perhaps others, do.

The POST form method of creating an object has the benefit that it's a lot more flexible, allowing you to create complex rules for Content-Type, specifying file size requirements, etc.
More info here (scroll down about half the page):
https://fourtheorem.com/the-illustrated-guide-to-s3-pre-signed-urls/

[afranken]

ah, I wasn't aware that there is a separate API for browser uploads that specifies a POST Object:
https://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-UsingHTTPPOST.html

It should be possible to support this, but I'm currently working on versioning support, so this may take some time.