[RESOLVED] Network disruption🚨

[matmicro]

If you are looking for help, please confirm the following...

• I confirm I have searched the Docs, Codemagic Sample Projects, and GitHub Discussions.
• I confirm the software versions on my local computer match the ones in CI environment (e.g. Xcode version, Gradle version). See here
• I confirm I am able to do a fresh clone of the repository and build the project on my local computer.

Which mobile framework are you using?

Flutter (Dart)

Steps to reproduce

Try to installing SDK

I tried several times, wait and retray again.
But still failing.

Expected results

Download & Install

Actual results

== Use Flutter version 3.16.0 ==

curl --fail --show-error --silent --output flutter_macos_arm64_3.16.0-stable.zip --connect-timeout 10.00 --max-time 120.00 https://storage.googleapis.com/flutter_infra_release/releases/stable/macos/flutter_macos_arm64_3.16.0-stable.zip
curl: (28) Operation timed out after 120000 milliseconds with 88259261 out of 1143511989 bytes received

Downloading Flutter release timed out

Build failed :|
Downloading Flutter release timed out

Build id (optional)

[himesh-cm]

Hey @matmicro, All services are working normally,
Can you please share the build ID with me for further investigation? You can find it under your build details page's 'ID' section. 
Navigate to the 'Builds' section on the left-hand menu, select the build you want to view, and click the copy icon.

[matmicro]

Hi,
Please find here a list of IDs failing to download stuff:
Failing during Build:
665a2fa56240149c2bf02d3f

Failing during installing SDK:
6659f54d0699407d82be4b7a
6659e72317e0eb0cc568bc1d
6659e08c5d26ccf22077712a

Please let me know if you identified the root cause.
Regards

[himesh-cm]

Thanks for sharing. I am looking into this. Meanwhile, you could try again and let me know if you still have the issues today.

[mikhail-tokarev]

Hey @matmicro
There was an elevation in network errors in Atlanta datacenter, we will ask our partners to investigate it. Sorry for the inconvenience!

[Bonsaikitt3n]

This just started happening to us:
== Use Flutter version 3.19.5 ==

curl --fail --show-error --silent --output flutter_macos_arm64_3.19.5-stable.zip --connect-timeout 10.00 --max-time 120.00 https://storage.googleapis.com/flutter_infra_release/releases/stable/macos/flutter_macos_arm64_3.19.5-stable.zip
curl: (28) Operation timed out after 120009 milliseconds with 799531446 out of 1168633011 bytes received

Downloading Flutter release timed out

Build failed :|
Downloading Flutter release timed out

[Bonsaikitt3n]

Is there a hard timeout of 120000 milliseconds? Can you just bump that up?

[Bonsaikitt3n]

This one worked: 665f7883d7d5d35df1d4c5eb

[helinanever]

We are still experiencing intermittent network failures in our Atlanta data center causing timeouts and "Something Went Wrong" errors. Builds routed to machines in other data centers are not affected by this issue.
Unfortunately, our data center partner has not been able to identify the root cause of the issue yet and we cannot provide an ETA on when the issue will be fixed. However, we are continuing to work closely with them to address the problem.
Please also see our status page at https://status.codemagic.io/.

[johnstef99]

Why not cache the released flutter versions internally on some codemagic infra? Wouldn't that mean faster build time? Why abuse google storage?

[mikhail-tokarev]

The error with Flutter SDK is just the tip of the iceberg, and different network operations perform slower than expected, so we can't cache the entire Internet (unfortunately)

[Bonsaikitt3n]

curl: (28) Operation timed out after 120000 milliseconds.

Can you bump this? Looks like it going but it's slow. This is starting to really affect our workflow.

[mikhail-tokarev]

We've received an update from the Atlanta engineers saying that they were able to find an Internet Service Provider path with higher latency than other ISPs, which seems to be the root cause of the problem.

They are making the required adjustments. Thank you for your patience!

[mikhail-tokarev]

They adjusted some more IPS paths, and from our monitoring, we haven't seen any errors in the last hour. However, we will continue to monitor.

Sorry again for all this troubles!

[matmicro]

Sorry, but still facing network issues.

Build id: 6669e47e9b97d76983877b0e

fatal: unable to access 'https://github.com/*****.git/': Failed to connect to github.com port 443 after 75010 ms: Couldn't connect to server

And also some of my builds are 2x slower than usually.