Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Node Setup Problem #2386

Closed
cadndata opened this issue Oct 28, 2024 · 2 comments
Closed

Node Setup Problem #2386

cadndata opened this issue Oct 28, 2024 · 2 comments
Labels
s-triage Needs to be reviewed, designed and prioritized

Comments

@cadndata
Copy link

What is the bug?
Node is not working anymore. Hubble application is attempting to download the latest database snapshot but receives a 404 error (not found). Specifically, it’s trying to access "https://download.farcaster.xyz/snapshots/MAINNET/DB_SCHEMA_2/latest.json," but the file cannot be located at that URL

Please see the error below

hubble-1 | {"message":"{"level":30,"time":1730109958922,"pid":28,"hostname":"71018ed088c2","msg":"Stopped watching contract events"}\n","timestamp":"2024-10-28T10:05:58.922Z","type":"out","process_id":0,"app_name":"hubble"}
hubble-1 | {"message":"{"level":30,"time":1730109958922,"pid":28,"hostname":"71018ed088c2","msg":"Stopped watching block numbers"}\n","timestamp":"2024-10-28T10:05:58.922Z","type":"out","process_id":0,"app_name":"hubble"}
hubble-1 | {"message":"{"level":30,"time":1730109958922,"pid":28,"hostname":"71018ed088c2","msg":"Stopped fname events provider"}\n","timestamp":"2024-10-28T10:05:58.922Z","type":"out","process_id":0,"app_name":"hubble"}
hubble-1 | {"message":"{"level":30,"time":1730109958922,"pid":28,"hostname":"71018ed088c2","msg":"Sync engine stopped"}\n","timestamp":"2024-10-28T10:05:58.922Z","type":"out","process_id":0,"app_name":"hubble"}
hubble-1 | {"message":"{"level":30,"time":1730109958922,"pid":28,"hostname":"71018ed088c2","msg":"L2EventsProvider stopped"}\n","timestamp":"2024-10-28T10:05:58.922Z","type":"out","process_id":0,"app_name":"hubble"}
hubble-1 | {"message":"{"level":30,"time":1730109958922,"pid":28,"hostname":"71018ed088c2","msg":"Stopped libp2p..."}\n","timestamp":"2024-10-28T10:05:58.922Z","type":"out","process_id":0,"app_name":"hubble"}
hubble-1 | {"message":"{"level":30,"time":1730109958922,"pid":28,"hostname":"71018ed088c2","msg":"stopping engine"}\n","timestamp":"2024-10-28T10:05:58.922Z","type":"out","process_id":0,"app_name":"hubble"}
hubble-1 | {"message":"{"level":30,"time":1730109958922,"pid":28,"hostname":"71018ed088c2","msg":"engine stopped"}\n","timestamp":"2024-10-28T10:05:58.922Z","type":"out","process_id":0,"app_name":"hubble"}
hubble-1 | {"message":"{"level":30,"time":1730109958922,"pid":28,"hostname":"71018ed088c2","msg":"Hubble stopped, exiting normally"}\n","timestamp":"2024-10-28T10:05:58.922Z","type":"out","process_id":0,"app_name":"hubble"}
hubble-1 | 2024-10-28T10:05:58: PM2 log: App name:hubble id:0 disconnected
hubble-1 | 2024-10-28T10:05:59: PM2 log: App [hubble:0] exited with code [1] via signal [SIGINT]
hubble-1 | 2024-10-28T10:06:00: PM2 log: 0 application online, retry = 3
hubble-1 | 2024-10-28T10:06:02: PM2 log: 0 application online, retry = 2
hubble-1 | 2024-10-28T10:06:04: PM2 log: 0 application online, retry = 1
hubble-1 | 2024-10-28T10:06:06: PM2 log: 0 application online, retry = 0
hubble-1 | 2024-10-28T10:06:06: PM2 log: Stopping app:hubble id:0
hubble-1 | 2024-10-28T10:06:06: PM2 log: pid=28 msg=process killed
hubble-1 | 2024-10-28T10:06:06: PM2 log: PM2 successfully stopped
hubble-1 exited with code 0
hubble-1 | 2024-10-28T10:06:09: PM2 log: Launching in no daemon mode
hubble-1 | 2024-10-28T10:06:09: PM2 log: App [hubble:0] starting in -cluster mode-
hubble-1 | 2024-10-28T10:06:09: PM2 log: App [hubble:0] online
hubble-1 | {"message":"\n Hubble Startup Checks\n","timestamp":"2024-10-28T10:06:12.573Z","type":"out","process_id":0,"app_name":"hubble"}
hubble-1 | {"message":"------------------------\n","timestamp":"2024-10-28T10:06:12.574Z","type":"out","process_id":0,"app_name":"hubble"}
hubble-1 | {"message":"✅ | Farcaster: 2024.10.16 Hubble: 1.16.1\n","timestamp":"2024-10-28T10:06:12.574Z","type":"out","process_id":0,"app_name":"hubble"}
hubble-1 | {"message":"✅ | Detected 15GB of RAM\n","timestamp":"2024-10-28T10:06:12.574Z","type":"out","process_id":0,"app_name":"hubble"}
hubble-1 | {"message":"✅ | Directory /home/node/app/apps/hubble/.rocks is writable\n","timestamp":"2024-10-28T10:06:12.574Z","type":"out","process_id":0,"app_name":"hubble"}
hubble-1 | {"message":"✅ | Directory /home/node/app/apps/hubble/.hub is writable\n","timestamp":"2024-10-28T10:06:12.575Z","type":"out","process_id":0,"app_name":"hubble"}
hubble-1 | {"message":"✅ | Found PeerId 12D3KooWSWPyAHGRJu616Sdz3FQJYwJeaVxSyJa5PnTdsb8ar38g\n","timestamp":"2024-10-28T10:06:12.585Z","type":"out","process_id":0,"app_name":"hubble"}
hubble-1 | {"message":"✅ | Hubble Monitoring enabled\n","timestamp":"2024-10-28T10:06:12.588Z","type":"out","process_id":0,"app_name":"hubble"}
hubble-1 | {"message":"✅ | Network is MAINNET(1)\n","timestamp":"2024-10-28T10:06:12.588Z","type":"out","process_id":0,"app_name":"hubble"}
hubble-1 | {"message":"✅ | Bootstrapping from 4 peers\n","timestamp":"2024-10-28T10:06:12.593Z","type":"out","process_id":0,"app_name":"hubble"}
hubble-1 | {"message":"✅ | Hub Operator FID is 522239(cemvural)\n","timestamp":"2024-10-28T10:06:12.798Z","type":"out","process_id":0,"app_name":"hubble"}
hubble-1 | {"message":"✅ | Connected to L1 Ethereum node\n","timestamp":"2024-10-28T10:06:13.043Z","type":"out","process_id":0,"app_name":"hubble"}
hubble-1 | {"message":"✅ | Connected to L2 OP Mainnet node\n","timestamp":"2024-10-28T10:06:13.230Z","type":"out","process_id":0,"app_name":"hubble"}
hubble-1 | {"message":"\nMonitor Your Node\n","timestamp":"2024-10-28T10:06:13.292Z","type":"out","process_id":0,"app_name":"hubble"}
hubble-1 | {"message":"----------------\n","timestamp":"2024-10-28T10:06:13.294Z","type":"out","process_id":0,"app_name":"hubble"}
hubble-1 | {"message":"🔗 | Grafana at http://localhost:3000\n","timestamp":"2024-10-28T10:06:13.295Z","type":"out","process_id":0,"app_name":"hubble"}
hubble-1 | {"message":"\n Starting Hubble\n","timestamp":"2024-10-28T10:06:13.295Z","type":"out","process_id":0,"app_name":"hubble"}
hubble-1 | {"message":"------------------\n","timestamp":"2024-10-28T10:06:13.296Z","type":"out","process_id":0,"app_name":"hubble"}
hubble-1 | {"message":"Please wait... This may take several minutes\n","timestamp":"2024-10-28T10:06:13.296Z","type":"out","process_id":0,"app_name":"hubble"}
hubble-1 | {"message":"{"level":60,"time":1730109973908,"pid":29,"hostname":"71018ed088c2","err":{"type":"Error","message":"Failed to start hub: failed to fetch snapshot metadata - Failed to get latest snapshot from https://download.farcaster.xyz/snapshots/MAINNET/DB_SCHEMA_11/latest.json [AxiosError: Request failed with status code 404]","stack":"Error: Failed to start hub\n at file:///home/node/app/apps/hubble/src/cli.ts:682:14\n at /home/node/app/node_modules/neverthrow/dist/index.cjs.js:113:102\n at processTicksAndRejections (node:internal/process/task_queues:95:5)\n at Command. (file:///home/node/app/apps/hubble/src/cli.ts:680:25)\ncaused by: HubError: failed to fetch snapshot metadata - Failed to get latest snapshot from https://download.farcaster.xyz/snapshots/MAINNET/DB_SCHEMA_11/latest.json [AxiosError: Request failed with status code 404]\n at Hub.attemptCatchupSyncWithSnapshot (file:///home/node/app/apps/hubble/src/hubble.ts:935:18)\n at processTicksAndRejections (node:internal/process/task_queues:95:5)\n at Hub.start (file:///home/node/app/apps/hubble/src/hubble.ts:632:27)\n at Command. (file:///home/node/app/apps/hubble/src/cli.ts:680:25)"},"msg":"Failed to start hub"}\n","timestamp":"2024-10-28T10:06:13.983Z","type":"out","process_id":0,"app_name":"hubble"}
hubble-1 | {"message":"{"level":60,"time":1730109973985,"pid":29,"hostname":"71018ed088c2","reason":"Hub Startup failed","msg":"shutting down hub"}\n","timestamp":"2024-10-28T10:06:13.985Z","type":"out","process_id":0,"app_name":"hubble"}

How can it be reproduced? (optional)
Tried in a new server and reproduced the error. You can reproduce the the error when you want to set up the node

Additional context (optional)
Node worked for weeks perfectly. This problem occured a few days ago

@github-actions github-actions bot added the s-triage Needs to be reviewed, designed and prioritized label Oct 28, 2024
@bibo7086
Copy link

Duplicate of #2383 and #1930.

@sds
Copy link
Member

sds commented Oct 31, 2024

Our snapshot CDN was broken a few days ago and is now fixed.

@sds sds closed this as completed Oct 31, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
s-triage Needs to be reviewed, designed and prioritized
Projects
None yet
Development

No branches or pull requests

3 participants