Chromium on ARM - Still waiting to connect to Chromium, retrying in 1 second

[TonyBrobston]

We have been building a custom image off cypress/factory for awhile now and installing chromium. As of recently this has stopped working.

I started to create a reproducible example and then realized that the arm chromium example here also produced the problem for me: https://github.com/cypress-io/cypress-docker-images/tree/master/examples/chromium

Running the last command from that readme produces: Still waiting to connect to Chromium, retrying in 1 second

I had my peer do these same steps on his machine and it produced the same result.

It is unclear to me whether this is a Cypress, Chromium, or Cypress Docker issue.

I tried also using more verbose logging: docker run --env DEBUG="cypress:*" -it --rm --entrypoint bash test-chromium -c "npx cypress run --browser chromium"

I see cypress:network:connect received error on connect, retrying { iteration: 0, delay: 100, err: Error: connect ECONNREFUSED 127.0.0.1:42149 at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1495:16) { errno: -111, code: 'ECONNREFUSED', syscall: 'connect', address: '127.0.0.1', port: 42149 } } +0ms

...and I see this: cypress:launcher:browsers chromium stderr: *** stack smashing detected ***: terminated

[MikeMcC399]

@TonyBrobston

Please let us know which host operating system and hardware you are using to run Docker.

[TonyBrobston]

@TonyBrobston

Please let us know which host operating system and hardware you are using to run Docker.

@MikeMcC399
MacBook Pro
Apple M1 Max

[MikeMcC399]

@TonyBrobston

Thanks for confirming you're running under macOS with Apple M1!

I tested as far as I was able, however I don't have arm64 in any usable way available to me. Cypress Docker images don't ship any arm64-compatible browsers, due to on-going availability issues.

The GitHub Actions CI environments don't provide the necessary testing environment:

• GitHub Actions only allows running Docker images under Linux, so Docker under macOS can't be tested.

• The GitHub Runner image macos-14 for arm64 includes Google Chrome, but not Chromium. Cypress works correctly with macos-14 and Chrome 131.

• Chromium 131 with cypress/base:22.12.0 (latest) running against Cypress 13.15.1 shows no issue on amd64 hardware. See logs below:

Logs for amd64

[+] Building 0.8s (11/11) FINISHED                                                                                          docker:default
 => [internal] load build definition from Dockerfile                                                                                  0.0s
 => => transferring dockerfile: 253B                                                                                                  0.0s
 => [internal] load metadata for docker.io/cypress/base:latest                                                                        0.0s
 => [internal] load .dockerignore                                                                                                     0.0s
 => => transferring context: 2B                                                                                                       0.0s
 => [internal] load build context                                                                                                     0.7s
 => => transferring context: 24.03MB                                                                                                  0.7s
 => [1/6] FROM docker.io/cypress/base:latest                                                                                          0.0s
 => CACHED [2/6] COPY . /opt/app                                                                                                      0.0s
 => CACHED [3/6] WORKDIR /opt/app                                                                                                     0.0s
 => CACHED [4/6] RUN apt-get update              # Update package index                                                               0.0s
 => CACHED [5/6] RUN apt-get install chromium -y # Install Chromium                                                                   0.0s
 => CACHED [6/6] RUN npx cypress install         # Install Cypress binary                                                             0.0s
 => exporting to image                                                                                                                0.0s
 => => exporting layers                                                                                                               0.0s
 => => writing image sha256:07b36897a4d4e90da75290a781af44cc8b3033282e8e2612b448c7c3c25c57ca                                          0.0s
 => => naming to docker.io/library/test-chromium                                                                                      0.0s


DevTools listening on ws://127.0.0.1:42273/devtools/browser/c87f4d35-95a7-45ab-92a0-48f94983ffad

====================================================================================================

  (Run Starting)

  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ Cypress:        13.15.1                                                                        │
  │ Browser:        Chromium 131 (headless)                                                        │
  │ Node Version:   v22.12.0 (/usr/local/bin/node)                                                 │
  │ Specs:          1 found (spec.cy.js)                                                           │
  │ Searched:       cypress/e2e/**/*.cy.{js,jsx,ts,tsx}                                            │
  └────────────────────────────────────────────────────────────────────────────────────────────────┘


────────────────────────────────────────────────────────────────────────────────────────────────────

  Running:  spec.cy.js                                                                      (1 of 1)


  test local demo page
    ✓ heading (106ms)


  1 passing (149ms)


  (Results)

  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ Tests:        1                                                                                │
  │ Passing:      1                                                                                │
  │ Failing:      0                                                                                │
  │ Pending:      0                                                                                │
  │ Skipped:      0                                                                                │
  │ Screenshots:  0                                                                                │
  │ Video:        false                                                                            │
  │ Duration:     0 seconds                                                                        │
  │ Spec Ran:     spec.cy.js                                                                       │
  └────────────────────────────────────────────────────────────────────────────────────────────────┘


====================================================================================================

  (Run Finished)


       Spec                                              Tests  Passing  Failing  Pending  Skipped
  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ ✔  spec.cy.js                               151ms        1        1        -        -        - │
  └────────────────────────────────────────────────────────────────────────────────────────────────┘
    ✔  All specs passed!                        151ms        1        1        -        -        -

[MikeMcC399]

@TonyBrobston

I was able to reproduce an error situation Error: connect ECONNREFUSED 127.0.0.1 on CircleCI using the machine image ubuntu-2204:2024.11.1, resource_class: arm.medium and the latest version of Cypress 13.16.1. Chromium 131.0.6778.85 is the current packaged version from Debian.

I'm confirming an issue with Chromium on ARM:

  (Run Starting)

  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ Cypress:        13.16.1                                                                        │
  │ Browser:        Chromium 131 (headless)                                                        │
  │ Node Version:   v22.12.0 (/usr/local/bin/node)                                                 │
  │ Specs:          1 found (spec.cy.js)                                                           │
  │ Searched:       cypress/e2e/**/*.cy.{js,jsx,ts,tsx}                                            │
  └────────────────────────────────────────────────────────────────────────────────────────────────┘


────────────────────────────────────────────────────────────────────────────────────────────────────

  Running:  spec.cy.js                                                                      (1 of 1)
There was an error reconnecting to the Chrome DevTools protocol. Please restart the browser.

Error: connect ECONNREFUSED 127.0.0.1:35713
    at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1495:16)

The same test is successful with resource_class: medium (which is amd64).

The test is based on the directory examples/chromium, updated to Cypress 13.16.1.

It is unclear to me whether this is a Cypress, Chromium, or Cypress Docker issue.

The above test indicates that it isn't specifically an Apple M1 issue. It would need further tests in a non-Docker ARM64 environment to narrow down the cause of the issue. I'm unsure if I can further contribute to this issue, since I don't have any native ARM64 hardware available to me. Perhaps there are others who could assist in testing here?

[MikeMcC399]

I'm also seeing Chromium related error messages:

  cypress:launcher:browsers chromium stderr: *** stack smashing detected ***: terminated +52ms
  cypress:launcher:browsers chromium stderr: [500:500:1205/124058.781106:ERROR:gpu_process_host.cc(982)] GPU process exited unexpectedly: exit_code=134 +42ms

Full debug log attached:

chromium_arm_debug.zip

[MikeMcC399]

@TonyBrobston

Searching on https://bugs.debian.org/cgi-bin/pkgreport.cgi?pkg=chromium;dist=unstable I found:

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1088974 submitted Dec 3, 2024, Severity: serious
chromium: 131.0.6778.85-1~deb12u1 in bookworm-security fails to start on arm64: stack smashing detected

This appears to correspond to the issue we are seeing here.

You can subscribe to the bug on the bug report page.

I will leave this issue open until it has been reported resolved by Debian / Chromium. There would be no change needed to Cypress Docker images or to Cypress if it turns out that the bug is indeed the root cause.

[MikeMcC399]

@TonyBrobston

This may help you in the interim. I tested it on amd64 and arm64 (under Ubuntu):

Workaround

Until Debian releases a fixed version of Chromium for Debian 12.x arm64:

Replace the following line:

cypress-docker-images/examples/chromium/Dockerfile

Line 5 in c61c638

RUN apt-get install chromium -y # Install Chromium

with:

RUN apt-get install chromium=130.0.6723.91-1~deb12u1 chromium-common=130.0.6723.91-1~deb12u1 -y # Install older Chromium

[TonyBrobston]

@MikeMcC399 Thank you for this! This has allowed me to move forward.

I spent a couple of hours over the last few days racking my brain on this. I even tried RUN apt install -y chromium=[version number], but I couldn't seem to find a valid version number and kept getting an error from the install saying not a valid candidate or something along those lines. Could you link me to where you found this version number for future reference?

I also saw that bug you linked about stack smashing from December 3rd, but I wasn't sure if that was related or not.

I had thought this was a chromium specific issue, but I wasn't sure at the time where to even submit a bug; though it appears to be bugs.debian.org.

Anyhow, thanks for the help!

[MikeMcC399]

@TonyBrobston

Thank you for this! This has allowed me to move forward.

You're welcome and I'm glad that this is working for you!

I spent a couple of hours over the last few days racking my brain on this. I even tried RUN apt install -y chromium=[version number], but I couldn't seem to find a valid version number and kept getting an error from the install saying not a valid candidate or something along those lines. Could you link me to where you found this version number for future reference?

The working version was mentioned in the bug report https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1088974. I wouldn't have guessed otherwise:

On arm64 systems, the version of chromium currently in bookworm-security
(131.0.6778.85-1~deb12u1) fails to start with several "stack smashing detected"
errors. The bookworm version (130.0.6723.91-1~deb12u1) works fine instead.

I did however need some trial and error before I got the installation command to work!

I also saw that bug you linked about stack smashing from December 3rd, but I wasn't sure if that was related or not.

Your debug hints in your original post were helpful in finding the bug report, so thanks for the details you provided!

I had thought this was a chromium specific issue, but I wasn't sure at the time where to even submit a bug; though it appears to be bugs.debian.org.

I have never personally needed to report a Debian bug, so I can only point you to https://www.debian.org/Bugs/Reporting for instructions.

Anyhow, thanks for the help!

😄

[MikeMcC399]

@TonyBrobston

Debian bug - 1088974 is now closed as fixed.

Would you like to check if it is fixed for you without using the workaround of downgrading?

I am now able to run successfully on CircleCI with Chromium 131 on arm64.

---

https://packages.debian.org/search?keywords=chromium shows a new release:

bookworm (stable) (web): web browser
131.0.6778.108-1~deb12u1 [security]: amd64 arm64 armhf i386 ppc64el

The crashing version, as listed in Debian bug - 1088974, was
131.0.6778.85-1~deb12u1