Since AWSSDK.S3 3.7.412 our putObjectAsyc(...) method is broken

[SteveW94]

Describe the bug

Hi,

we use a rather simple putting method (getting method looks similar) to put objects in our blob-storage.

Content-Type="text/plain"

It worked fine, since ages.
Since version 3.7.412 it is broken and produces the following error every time:

"The provided 'x-amz-content-sha256' header does not match what was computed"

The version before, 3.7.411.7 works just fine as expected with absolutely no code change

Regression Issue

• Select this option if this issue appears to be a regression.

Expected Behavior

Objects get putted and retrieved from and to the blobstorage

Current Behavior

"The provided 'x-amz-content-sha256' header does not match what was computed" Exception occurs

Reproduction Steps

Use putObjectAsync() Method as described above for example

Possible Solution

No response

Additional Information/Context

No response

AWS .NET SDK and/or Package version used

AWSSDK.S3 3.7.412

Targeted .NET Platform

.NET6

Operating System and version

Windows 10

[dscpinheiro]

Are you using S3 or is your blobstorage a 3rd-party implementation? We made a change last week so that the SDK automatically calculates checksums for all PutObject calls: #3610

You can also enable debug logs to see the exact request the SDK is making (if you want to compare before and after version 3.7.412):

using Amazon;
using Amazon.Runtime;
using Amazon.Runtime.CredentialManagement;
using Amazon.S3;

AWSConfigs.LoggingConfig.LogTo = LoggingOptions.Console;
AWSConfigs.LoggingConfig.LogMetricsFormat = LogMetricsFormatOption.JSON;
AWSConfigs.LoggingConfig.LogResponses = ResponseLoggingOption.Always;
AWSConfigs.LoggingConfig.LogMetrics = true;

using var s3 = new AmazonS3Client();
await s3.PutObjectAsync(...);

[dscpinheiro]

This is the difference between requests after the S3 default integrity update:

Before:

PUT
/before-crc.txt

content-length:7214
content-type:text/plain
host:dspin-us-west-2-sdk-test-bucket.s3.us-west-2.amazonaws.com
user-agent:aws-sdk-dotnet-coreclr/3.7.411.7 ua/2.0 os/windows#10.0.22631.0 md/ARCH#X64 lang/.NET_Core#9.0.1 md/aws-sdk-dotnet-core#3.7.400.79 api/S3#3.7.411.7 cfg/retry-mode#legacy md/ClientAsync cfg/init-coll#1
x-amz-content-sha256:STREAMING-AWS4-HMAC-SHA256-PAYLOAD
x-amz-date:20250120T141234Z
x-amz-decoded-content-length:7039

content-length;content-type;host;user-agent;x-amz-content-sha256;x-amz-date;x-amz-decoded-content-length
STREAMING-AWS4-HMAC-SHA256-PAYLOAD

After:

PUT
/after-crc.txt

content-length:7335
content-type:text/plain
host:dspin-us-west-2-sdk-test-bucket.s3.us-west-2.amazonaws.com
user-agent:aws-sdk-dotnet-coreclr/3.7.412.2 ua/2.0 os/windows#10.0.22631.0 md/ARCH#X64 lang/.NET_Core#9.0.1 md/aws-sdk-dotnet-core#3.7.401.2 api/S3#3.7.412.2 cfg/retry-mode#legacy md/ClientAsync cfg/init-coll#1 m/U,Z,b
x-amz-content-sha256:STREAMING-AWS4-HMAC-SHA256-PAYLOAD-TRAILER
x-amz-date:20250120T141418Z
x-amz-decoded-content-length:7039
x-amz-sdk-checksum-algorithm:CRC32
x-amz-trailer:x-amz-checksum-crc32

content-length;content-type;host;user-agent;x-amz-content-sha256;x-amz-date;x-amz-decoded-content-length;x-amz-sdk-checksum-algorithm;x-amz-trailer
STREAMING-AWS4-HMAC-SHA256-PAYLOAD-TRAILER

---

@SteveW94 This still works as expected when using Amazon S3, but I wonder if the implementation you're using doesn't handle the change in the x-amz-content-sha256 header.

[SteveW94]

Thank you for your fast answer.

To do likewise, I can say, we use the well established MinIO library/implementation

https://min.io/

[dscpinheiro]

I see. MinIO doesn't support the new checksum functionality yet, although they have an open issue to address that: minio/minio#20845

For now, your options are:

• Wait for the above issue to be merged / released (recommended - seems like they already have an open PR)
• Pin the AWSSDK.S3 version to version 3.7.411.7 or lower
• Use one of the config options for data protection: https://docs.aws.amazon.com/sdkref/latest/guide/feature-dataintegrity.html (setting the request_checksum_calculation to WHEN_REQUIRED will tell the .NET SDK not to calculate checksums for PutObject calls)

[SteveW94]

Thanks a lot for the fast clarification.
As we will be stuck on the current Min.IO version for the next years probably even, I pinned it for now.
But I think, I will go for your hint with the WHEN_REQUIRED setting as soon as I had time to have some insight in it.

I think this can be closed already with that :)

[github-actions]

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.