Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: access graph fails to fetch s3 bucket details #50758

Merged
merged 3 commits into from
Jan 6, 2025

Conversation

tigrato
Copy link
Contributor

@tigrato tigrato commented Jan 6, 2025

With SDK V1, AWS get bucket details such as policies, acls, status... didn't require the specification of the target s3 bucket location. With the recent changes to support newer versions of the AWS SDK, the get bucket details started to fail with the following error:

BucketRegionError: incorrect region, the bucket is not in 'ap-south-1' region at endpoint '', bucket is in 'eu-central-1' region
status code: 301, request id: QS5C24H12ZV3VNM4, host id: mzVDk4010MPTCFdxyE/XwERX9W35MSge85PG+h5Jvwyvi7MhxdXLaysb2PTZCMY9r1ngBi6Gv6g=, failed to fetch bucket "cyz" acls polic

This PR uses HeadBucket to retrieve the location of the s3 bucket and then uses the bucket location client to retrieve the s3 bucket details.

Fixes #50757

Changelog: Fixes a bug where S3 bucket details fail to fetch due to incorrect bucket region.

With SDK V1, AWS get bucket details such as policies, acls, status...
didn't require the specification of the target s3 bucket location. With
the recent changes to support newer versions of the AWS SDK, the get
bucket details started to fail with the following error:

```
BucketRegionError: incorrect region, the bucket is not in 'ap-south-1' region at endpoint '', bucket is in 'eu-central-1' region
status code: 301, request id: QS5C24H12ZV3VNM4, host id: mzVDk4010MPTCFdxyE/XwERX9W35MSge85PG+h5Jvwyvi7MhxdXLaysb2PTZCMY9r1ngBi6Gv6g=, failed to fetch bucket "cyz" acls polic
```

This PR uses `HeadBucket` to retrieve the location of the s3 bucket and
then uses the bucket location client to retrieve the s3 bucket details.

Fixes #50757
@tigrato tigrato disabled auto-merge January 6, 2025 10:50
@tigrato tigrato enabled auto-merge January 6, 2025 11:38
lib/srv/discovery/fetchers/aws-sync/s3.go Outdated Show resolved Hide resolved
lib/srv/discovery/fetchers/aws-sync/s3.go Outdated Show resolved Hide resolved
lib/srv/discovery/fetchers/aws-sync/s3.go Outdated Show resolved Hide resolved
lib/cloud/aws/policy_statements.go Show resolved Hide resolved
lib/srv/discovery/fetchers/aws-sync/s3.go Outdated Show resolved Hide resolved
Copy link

github-actions bot commented Jan 6, 2025

Amplify deployment status

Branch Commit Job ID Status Preview Updated (UTC)
tigrato/fix-sdk2-s3-list-errs 9c07d2e 2 ✅SUCCEED tigrato-fix-sdk2-s3-list-errs 2025-01-06 12:07:37

@tigrato tigrato force-pushed the tigrato/fix-sdk2-s3-list-errs branch from 2b78c73 to 9c07d2e Compare January 6, 2025 12:02
@tigrato tigrato added this pull request to the merge queue Jan 6, 2025
Merged via the queue into master with commit 4d3e7d3 Jan 6, 2025
44 checks passed
@tigrato tigrato deleted the tigrato/fix-sdk2-s3-list-errs branch January 6, 2025 12:41
@public-teleport-github-review-bot

@tigrato See the table below for backport results.

Branch Result
branch/v15 Failed
branch/v16 Failed
branch/v17 Create PR

tigrato added a commit that referenced this pull request Jan 6, 2025
* fix: access graph fails to fetch s3 bucket details

With SDK V1, AWS get bucket details such as policies, acls, status...
didn't require the specification of the target s3 bucket location. With
the recent changes to support newer versions of the AWS SDK, the get
bucket details started to fail with the following error:

```
BucketRegionError: incorrect region, the bucket is not in 'ap-south-1' region at endpoint '', bucket is in 'eu-central-1' region
status code: 301, request id: QS5C24H12ZV3VNM4, host id: mzVDk4010MPTCFdxyE/XwERX9W35MSge85PG+h5Jvwyvi7MhxdXLaysb2PTZCMY9r1ngBi6Gv6g=, failed to fetch bucket "cyz" acls polic
```

This PR uses `HeadBucket` to retrieve the location of the s3 bucket and
then uses the bucket location client to retrieve the s3 bucket details.

Fixes #50757

* fix tests

* handle review comments
tigrato added a commit that referenced this pull request Jan 6, 2025
* fix: access graph fails to fetch s3 bucket details

With SDK V1, AWS get bucket details such as policies, acls, status...
didn't require the specification of the target s3 bucket location. With
the recent changes to support newer versions of the AWS SDK, the get
bucket details started to fail with the following error:

```
BucketRegionError: incorrect region, the bucket is not in 'ap-south-1' region at endpoint '', bucket is in 'eu-central-1' region
status code: 301, request id: QS5C24H12ZV3VNM4, host id: mzVDk4010MPTCFdxyE/XwERX9W35MSge85PG+h5Jvwyvi7MhxdXLaysb2PTZCMY9r1ngBi6Gv6g=, failed to fetch bucket "cyz" acls polic
```

This PR uses `HeadBucket` to retrieve the location of the s3 bucket and
then uses the bucket location client to retrieve the s3 bucket details.

Fixes #50757

* fix tests

* handle review comments
github-merge-queue bot pushed a commit that referenced this pull request Jan 6, 2025
* fix: access graph fails to fetch s3 bucket details

With SDK V1, AWS get bucket details such as policies, acls, status...
didn't require the specification of the target s3 bucket location. With
the recent changes to support newer versions of the AWS SDK, the get
bucket details started to fail with the following error:

```
BucketRegionError: incorrect region, the bucket is not in 'ap-south-1' region at endpoint '', bucket is in 'eu-central-1' region
status code: 301, request id: QS5C24H12ZV3VNM4, host id: mzVDk4010MPTCFdxyE/XwERX9W35MSge85PG+h5Jvwyvi7MhxdXLaysb2PTZCMY9r1ngBi6Gv6g=, failed to fetch bucket "cyz" acls polic
```

This PR uses `HeadBucket` to retrieve the location of the s3 bucket and
then uses the bucket location client to retrieve the s3 bucket details.

Fixes #50757

* fix tests

* handle review comments
github-merge-queue bot pushed a commit that referenced this pull request Jan 6, 2025
* fix: access graph fails to fetch s3 bucket details

With SDK V1, AWS get bucket details such as policies, acls, status...
didn't require the specification of the target s3 bucket location. With
the recent changes to support newer versions of the AWS SDK, the get
bucket details started to fail with the following error:

```
BucketRegionError: incorrect region, the bucket is not in 'ap-south-1' region at endpoint '', bucket is in 'eu-central-1' region
status code: 301, request id: QS5C24H12ZV3VNM4, host id: mzVDk4010MPTCFdxyE/XwERX9W35MSge85PG+h5Jvwyvi7MhxdXLaysb2PTZCMY9r1ngBi6Gv6g=, failed to fetch bucket "cyz" acls polic
```

This PR uses `HeadBucket` to retrieve the location of the s3 bucket and
then uses the bucket location client to retrieve the s3 bucket details.

Fixes #50757

* fix tests

* handle review comments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Access Graph AWS sync fails with incorrect s3 bucket region
3 participants