assertion failed [!result.is_error]: Failed to create temporary file

[Thom-x]

Gloo Edge Product

Enterprise

Gloo Edge Version

v1.17.0

Kubernetes Version

v1.27.9

Describe the bug

Me and my team are developping locally on MacBook Pro M3 (arm64).
When running with rosetta emulation (for performance benefits) gateway-proxy fail to start with (only this log is displayed nothing else).

assertion failed [!result.is_error]: Failed to create temporary file
(ThreadContextFcntl.cpp:84 create_tempfile)
Stream closed EOF for ts-gloo/gateway-proxy-56c5b87fc4-2bbj6 (gateway-proxy)

It does the same for fed console.

I know we can disable this option to make it works, but because we are running a lot of services the performance drawback is too important and we can't do that.

I see that solo-io#3486 has been updated recently to be part of 1.18, do you think it will solve this issue ?

I tried compiling envoy-gloo to be able to have a native arm64 image/binary but it fail with make fast-build-arm, I don't want to go into a rabbit hole so I stopped here.

---

See here and here for mention of this exception,

Expected Behavior

Pod should be starting as ususal.

Steps to reproduce the bug

helm repo add glooe "https://storage.googleapis.com/gloo-ee-helm" --force-update
helm upgrade -i glooe glooe/gloo-ee --version 1.17.0 --namespace ts-gloo --create-namespace \
   --set-string license_key=$GLOO_LICENSE_KEY \
   --set gloo.gatewayProxies.gatewayProxy.service.type=ClusterIP \
   --set global.extensions.glooRedis.enableAcl=false \
   --set global.extensions.extAuth.enabled=true \
   --set gloo-fed.glooFedApiserver.enable=false	 \
   --set global.extensions.extAuth.requestTimeout=5s

Additional Environment Detail

MacBook Pro M3 (arm64) with Docker for Desktop 27.0.03 and Rosetta emulation,
Minikube installed using brew v1.33.1.
Helm installed with brew v3.15.3.

Additional Context

No response

[kevin-shelaga]

Good morning, for now you must disable Rosetta to run Gloo locally on apple silicon

[Thom-x]

I know that it can solve the issue (that's what I said).
But the performances are awful without Rosetta emulation, so this is a blocker for us.
Could you at least have a look to know the root cause and find a workaround (building the image for arm64, even if it is not 100% stable at least we could be able to dev).

[nfuden]

docker/roadmap#384

[Thom-x]

I tested adding EXPERIMENTAL_DOCKER_DESKTOP_FORCE_QEMU=1 to the container like suggested in docker/roadmap#384, and it works 😶

So, we can just add this env and we should be able to run Rosetta 2 except for this pod.

TL:DR We need to add EXPERIMENTAL_DOCKER_DESKTOP_FORCE_QEMU=1 to run it and keep Rosetta everywhere else.

helm install ...
      --set gloo.gatewayProxies.gatewayProxy.kind.deployment.customEnv[0].name=EXPERIMENTAL_DOCKER_DESKTOP_FORCE_QEMU \
      --set-string gloo.gatewayProxies.gatewayProxy.kind.deployment.customEnv[0].value=1 \

[prathyk]

Do we have some way to use gloo on a mac m1 (arm64)? I'm using colima and hence I cannot use the docker workaround mentioned above

[qikijames]

I'm using lima with vz and rosetta on mac m1 and I also hit this issue