Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue with DNS Caching After Internet Outage in go-livepeer Node #3291

Open
FranckUltima opened this issue Dec 3, 2024 · 0 comments
Open
Labels
status: triage this issue has not been evaluated yet

Comments

@FranckUltima
Copy link

Issue Description

Hello,

I've encountered an issue with my orchestrator node using go-livepeer. After a scheduled maintenance by my internet provider, my orchestrator node didn't resume its service normally and got stuck in a loop with the following error:

E1203 11:40:11.668550 1 block_watcher.go:123] blockwatch.Watcher error encountered - trying again on next polling interval err="Post "https://arb-mainnet.g.alchemy.com/v2/YOUR_API_KEY\": context deadline exceeded"

To restore the connection to the RPC, I had to manually restart the Docker container (version 0.8.0). This problem occurs with different RPC providers, including Alchemy.
Interestingly, one of my nodes (AI) that I forgot to restart eventually managed to reconnect to the RPC after approximately 15 minutes.

Suspected Cause

Mike suggested that the issue might be related to DNS caching. When the internet connection is restored, the DNS entries may be outdated, causing the node to fail to reconnect properly until the container is restarted.

Proposed Solution

One proposed solution is to adjust the health check endpoint to test the node's ability to stream and block poll RPC requests. If the health check detects issues, the Docker container could be configured to automatically restart, ensuring the node attempts to reconnect without manual intervention.

Thank you!

@FranckUltima FranckUltima changed the title ssue with DNS Caching After Internet Outage in go-livepeer Node Issue with DNS Caching After Internet Outage in go-livepeer Node Dec 3, 2024
@github-actions github-actions bot added the status: triage this issue has not been evaluated yet label Dec 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
status: triage this issue has not been evaluated yet
Projects
None yet
Development

No branches or pull requests

1 participant