Failed
Nov 17 16:03:55 INF Starting Lava
Nov 17 16:03:55 INF Executing Command: ./scripts/start_env_dev.sh
Nov 17 16:03:55 INF Waiting for Lava
Nov 17 16:04:05 INF Waiting for Lava
Nov 17 16:04:15 INF Waiting for Lava
Nov 17 16:04:25 INF Waiting for Lava
Nov 17 16:04:35 INF Waiting for Lava
Nov 17 16:04:45 INF Waiting for Lava
Nov 17 16:04:55 INF Waiting for Lava
Nov 17 16:05:05 INF Waiting for Lava
Nov 17 16:05:15 INF Waiting for Lava
Nov 17 16:05:25 INF Waiting for Lava
Nov 17 16:05:35 INF Waiting for Lava
Nov 17 16:05:45 INF Waiting for Lava
Nov 17 16:05:55 INF Waiting for Lava
Nov 17 16:06:05 INF Starting Lava OK
Nov 17 16:06:05 INF Staking Lava
Nov 17 16:06:05 INF Executing Command: ./scripts/test/init_e2e.sh
Nov 17 16:07:13 INF stakeLava OK
ETH1
provider lava@14e36fkzxyklq85p5ea6z4mywrs02haknwx03d0 [{127.0.0.1:2225 1 [] [jsonrpc] []}]
provider lava@162kmv9y67c3l3qpnfm3yp7m4uz577nk74eugkv [{127.0.0.1:2222 1 [] [jsonrpc] []}]
provider lava@1cvejj00nl7lkqtweyexqjfatqlj9gs8czrucuf [{127.0.0.1:2221 1 [debug] [jsonrpc] [archive]}]
provider lava@1tuj2fpzpy0unwvkzmu3r4w8qgmcmpsem42uuhv [{127.0.0.1:2224 1 [] [jsonrpc] []}]
provider lava@1w644jchpd6mnjcj50sfwmmmshgxhgzdrugxhff [{127.0.0.1:2223 1 [] [jsonrpc] []}]
LAV1
provider lava@17vcc0jstmssvh3am03strrhjsg00448wm5nmjw [{127.0.0.1:2261 1 [] [grpc rest tendermintrpc] []}]
provider lava@17z48m6s3gf8ftr8qy74vvr3g0rxg8p585geqyy [{127.0.0.1:2262 1 [] [grpc rest tendermintrpc] []}]
provider lava@18aqp3nun00eyt2rk9mv6u46xk0dqq5mpnzr7ds [{127.0.0.1:2265 1 [] [grpc rest tendermintrpc] []}]
provider lava@1lde5cfk3mhhfkdujxn69uskqllclecjpuw47j5 [{127.0.0.1:2263 1 [] [grpc rest tendermintrpc] []}]
provider lava@1xsm9mfwmj9v48s85y64y59zfwccrer6rumscpm [{127.0.0.1:2264 1 [] [grpc rest tendermintrpc] []}]
Nov 17 16:07:14 INF Staking Lava OK
Nov 17 16:07:14 INF RUNNING TESTS
Nov 17 16:07:14 INF Executing Command: /opt/hostedtoolcache/go/1.20.14/x64/bin/go test ./testutil/e2e/proxy/. -v -timeout 0 eth
Nov 17 16:07:14 INF startJSONRPCProxy OK
Nov 17 16:07:14 INF Waiting JSONRPC Consumer
Nov 17 16:07:15 INF Waiting JSONRPC Consumer
Nov 17 16:07:16 INF Waiting JSONRPC Consumer
Nov 17 16:07:17 INF Waiting JSONRPC Consumer
Nov 17 16:07:18 INF Waiting JSONRPC Consumer
Nov 17 16:07:19 INF Waiting JSONRPC Consumer
Nov 17 16:07:20 INF Waiting JSONRPC Consumer
Nov 17 16:07:21 INF Waiting JSONRPC Consumer
Nov 17 16:07:22 INF Waiting JSONRPC Consumer
Nov 17 16:07:23 INF Waiting JSONRPC Consumer
Nov 17 16:07:24 INF Waiting JSONRPC Consumer
Nov 17 16:07:25 INF Waiting JSONRPC Consumer
Nov 17 16:07:26 INF Waiting JSONRPC Consumer
Nov 17 16:07:27 INF Waiting JSONRPC Consumer
Nov 17 16:07:28 INF Waiting JSONRPC Consumer
Nov 17 16:07:29 INF Waiting JSONRPC Consumer
Nov 17 16:07:30 INF Waiting JSONRPC Consumer
Nov 17 16:07:31 INF Waiting JSONRPC Consumer
Nov 17 16:07:32 INF Waiting JSONRPC Consumer
Nov 17 16:07:33 INF Waiting JSONRPC Consumer
Nov 17 16:07:33 INF JSONRPCProxy OK
Nov 17 16:07:33 INF Validated proxy is alive got response res=16482926
Nov 17 16:07:33 DBG Executing command /home/runner/go/bin/lavap rpcprovider ./testutil/e2e/e2eConfigs/provider/jsonrpcProvider1.yml --cache-be 127.0.0.1:2777 --chain-id=lava --from servicer1 --geolocation 1 --log_level debug
Nov 17 16:07:33 DBG Executing command /home/runner/go/bin/lavap rpcprovider ./testutil/e2e/e2eConfigs/provider/jsonrpcProvider2.yml --cache-be 127.0.0.1:2777 --chain-id=lava --from servicer2 --geolocation 1 --log_level debug
Nov 17 16:07:33 DBG Executing command /home/runner/go/bin/lavap rpcprovider ./testutil/e2e/e2eConfigs/provider/jsonrpcProvider3.yml --cache-be 127.0.0.1:2777 --chain-id=lava --from servicer3 --geolocation 1 --log_level debug
Nov 17 16:07:33 DBG Executing command /home/runner/go/bin/lavap rpcprovider ./testutil/e2e/e2eConfigs/provider/jsonrpcProvider4.yml --cache-be 127.0.0.1:2777 --chain-id=lava --from servicer4 --geolocation 1 --log_level debug
Nov 17 16:07:33 DBG Executing command /home/runner/go/bin/lavap rpcprovider ./testutil/e2e/e2eConfigs/provider/jsonrpcProvider5.yml --cache-be 127.0.0.1:2777 --chain-id=lava --from servicer5 --geolocation 1 --log_level debug
Nov 17 16:07:33 INF Waiting Provider 127.0.0.1:2221
Nov 17 16:07:35 INF Waiting Provider 127.0.0.1:2221
Nov 17 16:07:37 INF Waiting Provider 127.0.0.1:2221
Nov 17 16:07:39 INF Waiting Provider 127.0.0.1:2221
Nov 17 16:07:41 INF Waiting Provider 127.0.0.1:2221
Nov 17 16:07:43 INF Waiting Provider 127.0.0.1:2221
Nov 17 16:07:45 INF Waiting Provider 127.0.0.1:2221
Nov 17 16:07:47 INF Waiting Provider 127.0.0.1:2221
Nov 17 16:07:49 INF Waiting Provider 127.0.0.1:2221
Nov 17 16:07:51 INF Waiting Provider 127.0.0.1:2221
Nov 17 16:07:53 INF Waiting Provider 127.0.0.1:2221
Nov 17 16:07:55 INF Waiting Provider 127.0.0.1:2221
Nov 17 16:07:57 INF Waiting Provider 127.0.0.1:2221
Nov 17 16:07:57 INF Waiting Provider 127.0.0.1:2222
Nov 17 16:07:57 INF Waiting Provider 127.0.0.1:2223
Nov 17 16:07:57 INF Waiting Provider 127.0.0.1:2224
Nov 17 16:07:57 INF startJSONRPCProvider OK
Nov 17 16:07:57 INF Executing Command: /home/runner/go/bin/lavap rpcconsumer ./testutil/e2e/e2eConfigs/consumer/ethConsumer1.yml --cache-be 127.0.0.1:2778 --chain-id=lava --from user1 --geolocation 1 --log_level debug --allow-insecure-provider-dialing
Nov 17 16:07:57 INF startJSONRPCConsumer OK
Nov 17 16:07:57 INF Waiting JSONRPC Consumer
Nov 17 16:07:58 INF Waiting JSONRPC Consumer
Nov 17 16:07:59 INF Waiting JSONRPC Consumer
Nov 17 16:08:00 INF Waiting JSONRPC Consumer
Nov 17 16:08:01 INF Waiting JSONRPC Consumer
Nov 17 16:08:02 INF JSONRPCConsumer1 OK
Nov 17 16:08:02 INF Validated proxy is alive got response res=16482926
Nov 17 16:08:02 INF Executing Command: /home/runner/go/bin/lavap rpcprovider ./testutil/e2e/e2eConfigs/provider/lavaProvider6 --cache-be 127.0.0.1:2777 --chain-id=lava --from servicer6 --geolocation 1 --log_level debug
Nov 17 16:08:02 INF Executing Command: /home/runner/go/bin/lavap rpcprovider ./testutil/e2e/e2eConfigs/provider/lavaProvider7 --cache-be 127.0.0.1:2777 --chain-id=lava --from servicer7 --geolocation 1 --log_level debug
Nov 17 16:08:02 INF Executing Command: /home/runner/go/bin/lavap rpcprovider ./testutil/e2e/e2eConfigs/provider/lavaProvider8 --cache-be 127.0.0.1:2777 --chain-id=lava --from servicer8 --geolocation 1 --log_level debug
Nov 17 16:08:02 INF Executing Command: /home/runner/go/bin/lavap rpcprovider ./testutil/e2e/e2eConfigs/provider/lavaProvider9 --cache-be 127.0.0.1:2777 --chain-id=lava --from servicer9 --geolocation 1 --log_level debug
Nov 17 16:08:02 INF Executing Command: /home/runner/go/bin/lavap rpcprovider ./testutil/e2e/e2eConfigs/provider/lavaProvider10 --cache-be 127.0.0.1:2777 --chain-id=lava --from servicer10 --geolocation 1 --log_level debug
Nov 17 16:08:02 INF Waiting Provider 127.0.0.1:2261
Nov 17 16:08:04 INF Waiting Provider 127.0.0.1:2261
Nov 17 16:08:06 INF Waiting Provider 127.0.0.1:2261
Nov 17 16:08:06 INF Waiting Provider 127.0.0.1:2262
Nov 17 16:08:07 INF Waiting Provider 127.0.0.1:2263
Nov 17 16:08:07 INF Waiting Provider 127.0.0.1:2264
Nov 17 16:08:07 INF Waiting Provider 127.0.0.1:2265
Nov 17 16:08:07 INF startLavaProviders OK
Nov 17 16:08:07 INF Executing Command: /home/runner/go/bin/lavap rpcconsumer ./testutil/e2e/e2eConfigs/consumer/lavaConsumer1.yml --cache-be 127.0.0.1:2778 --chain-id=lava --from user3 --geolocation 1 --log_level debug --allow-insecure-provider-dialing
Nov 17 16:08:07 INF startRPCConsumer OK
Nov 17 16:08:07 INF Waiting TENDERMINT Consumer
Nov 17 16:08:08 INF Waiting TENDERMINT Consumer
Nov 17 16:08:09 INF Waiting TENDERMINT Consumer
Nov 17 16:08:10 INF Waiting TENDERMINT Consumer
Nov 17 16:08:11 INF Waiting TENDERMINT Consumer
Nov 17 16:08:11 INF checkTendermintConsumer OK
Nov 17 16:08:11 INF Waiting REST Consumer
Nov 17 16:08:11 INF checkRESTConsumer OK
Nov 17 16:08:11 INF Waiting GRPC Consumer
Nov 17 16:08:11 INF checkGRPCConsumer OK
Nov 17 16:08:11 INF Starting JSONRPC Tests
Nov 17 16:08:27 INF JSONRPC TEST OK
Nov 17 16:08:27 INF Starting TENDERMINT Tests
Nov 17 16:08:42 INF TENDERMINTRPC TEST OK
Nov 17 16:08:42 INF Starting TENDERMINTRPC URI Tests
Nov 17 16:08:57 INF TENDERMINTRPC URI TEST OK
Nov 17 16:08:57 INF Starting REST Tests
Nov 17 16:09:12 INF REST TEST OK
Nov 17 16:09:12 INF Starting GRPC Tests
Nov 17 16:09:27 INF GRPC TEST OK
Nov 17 16:09:27 INF Executing Command: /home/runner/go/bin/lavap cache 127.0.0.1:2778 --log_level debug
Nov 17 16:09:27 INF Waiting Cache 127.0.0.1:2778
Nov 17 16:09:29 INF Waiting Cache 127.0.0.1:2778
Nov 17 16:09:29 INF startConsumerCache OK
Nov 17 16:09:29 INF Executing Command: /home/runner/go/bin/lavap cache 127.0.0.1:2777 --log_level debug
Nov 17 16:09:29 INF Waiting Cache 127.0.0.1:2777
Nov 17 16:09:31 INF Waiting Cache 127.0.0.1:2777
Nov 17 16:09:31 INF startProviderCache OK
Nov 17 16:09:31 INF Waiting TENDERMINT Consumer
Nov 17 16:09:31 INF checkTendermintConsumer OK
Nov 17 16:09:31 INF Waiting REST Consumer
Nov 17 16:09:31 INF checkRESTConsumer OK
Nov 17 16:09:31 INF Waiting GRPC Consumer
Nov 17 16:09:32 INF checkGRPCConsumer OK
Nov 17 16:09:32 INF Starting JSONRPC Tests
Nov 17 16:09:48 INF JSONRPC TEST OK
Nov 17 16:09:48 INF Starting TENDERMINT Tests
Nov 17 16:10:03 INF TENDERMINTRPC TEST OK
Nov 17 16:10:03 INF Starting TENDERMINTRPC URI Tests
Nov 17 16:10:18 INF TENDERMINTRPC URI TEST OK
Nov 17 16:10:18 INF Starting REST Tests
Nov 17 16:10:33 INF REST TEST OK
Nov 17 16:10:33 INF Starting GRPC Tests
Nov 17 16:10:48 INF GRPC TEST OK
Nov 17 16:10:48 INF Starting Lava over Lava Tests
Nov 17 16:10:48 INF Executing Command: ./scripts/test/init_e2e_lava_over_lava.sh
SEP1
provider lava@14e36fkzxyklq85p5ea6z4mywrs02haknwx03d0 [{127.0.0.1:2125 1 [] [jsonrpc] []}]
provider lava@162kmv9y67c3l3qpnfm3yp7m4uz577nk74eugkv [{127.0.0.1:2122 1 [] [jsonrpc] []}]
provider lava@1cvejj00nl7lkqtweyexqjfatqlj9gs8czrucuf [{127.0.0.1:2121 1 [] [jsonrpc] []}]
provider lava@1tuj2fpzpy0unwvkzmu3r4w8qgmcmpsem42uuhv [{127.0.0.1:2124 1 [] [jsonrpc] []}]
provider lava@1w644jchpd6mnjcj50sfwmmmshgxhgzdrugxhff [{127.0.0.1:2123 1 [] [jsonrpc] []}]
Nov 17 16:11:15 INF Lava Over Lava Test OK
Nov 17 16:11:15 INF Starting Relay Response Integrity Tests
Nov 17 16:11:15 INF TENDERMINT RESPONSE CROSS VERIFICATION OK
Nov 17 16:11:15 INF REST RESPONSE CROSS VERIFICATION OK
Nov 17 16:11:15 INF GRPC RESPONSE CROSS VERIFICATION OK
Nov 17 16:11:15 INF Starting WebSocket Subscription Test
Nov 17 16:11:15 INF Setting up web socket client 1
Nov 17 16:11:15 DBG Dialed WebSocket Successful response="&{Status:101 Switching Protocols StatusCode:101 Proto:HTTP/1.1 ProtoMajor:1 ProtoMinor:1 Header:map[Access-Control-Allow-Origin:[*] Connection:[Upgrade] Date:[Sun, 17 Nov 2024 16:11:15 GMT] Sec-Websocket-Accept:[45Y3yKB+g96gNAufho55XTsZHDo=] Upgrade:[websocket]] Body:{Reader:0xc001ebbe90} ContentLength:0 TransferEncoding:[] Close:false Uncompressed:false Trailer:map[] Request:0xc000b84c00 TLS:<nil>}" url=ws://127.0.0.1:3340/websocket
Nov 17 16:11:15 INF Start listening for NewBlock messages on web socket 1
Nov 17 16:11:15 INF Subscribing to NewBlock events on web socket 1
Nov 17 16:11:15 INF Setting up web socket client 2
Nov 17 16:11:15 DBG Dialed WebSocket Successful response="&{Status:101 Switching Protocols StatusCode:101 Proto:HTTP/1.1 ProtoMajor:1 ProtoMinor:1 Header:map[Access-Control-Allow-Origin:[*] Connection:[Upgrade] Date:[Sun, 17 Nov 2024 16:11:15 GMT] Sec-Websocket-Accept:[hABoWiQoUUbAsa1sYq6tKU3Kvy8=] Upgrade:[websocket]] Body:{Reader:0xc002efe360} ContentLength:0 TransferEncoding:[] Close:false Uncompressed:false Trailer:map[] Request:0xc000b84f00 TLS:<nil>}" url=ws://127.0.0.1:3340/websocket
Nov 17 16:11:15 INF Start listening for NewBlock messages on web socket 2
Nov 17 16:11:15 INF Subscribing to NewBlock events on web socket 2
Nov 17 16:11:15 INF Setting up web socket client 3
Nov 17 16:11:15 DBG Dialed WebSocket Successful response="&{Status:101 Switching Protocols StatusCode:101 Proto:HTTP/1.1 ProtoMajor:1 ProtoMinor:1 Header:map[Access-Control-Allow-Origin:[*] Connection:[Upgrade] Date:[Sun, 17 Nov 2024 16:11:15 GMT] Sec-Websocket-Accept:[Xx7bqNbt47ClOEp5hXvwRR760aQ=] Upgrade:[websocket]] Body:{Reader:0xc002efe810} ContentLength:0 TransferEncoding:[] Close:false Uncompressed:false Trailer:map[] Request:0xc000b85200 TLS:<nil>}" url=ws://127.0.0.1:3340/websocket
Nov 17 16:11:15 INF Start listening for NewBlock messages on web socket 3
Nov 17 16:11:15 INF Subscribing to NewBlock events on web socket 3
Nov 17 16:11:15 INF Setting up web socket client 4
Nov 17 16:11:15 DBG Dialed WebSocket Successful response="&{Status:101 Switching Protocols StatusCode:101 Proto:HTTP/1.1 ProtoMajor:1 ProtoMinor:1 Header:map[Access-Control-Allow-Origin:[*] Connection:[Upgrade] Date:[Sun, 17 Nov 2024 16:11:15 GMT] Sec-Websocket-Accept:[xDZRqJKHcmhjzYlxpk4GGBMy3/I=] Upgrade:[websocket]] Body:{Reader:0xc00202d7d0} ContentLength:0 TransferEncoding:[] Close:false Uncompressed:false Trailer:map[] Request:0xc000b85900 TLS:<nil>}" url=ws://127.0.0.1:3340/websocket
Nov 17 16:11:15 INF Start listening for NewBlock messages on web socket 4
Nov 17 16:11:15 INF Subscribing to NewBlock events on web socket 4
Nov 17 16:11:15 INF Setting up web socket client 5
Nov 17 16:11:15 DBG Dialed WebSocket Successful response="&{Status:101 Switching Protocols StatusCode:101 Proto:HTTP/1.1 ProtoMajor:1 ProtoMinor:1 Header:map[Access-Control-Allow-Origin:[*] Connection:[Upgrade] Date:[Sun, 17 Nov 2024 16:11:15 GMT] Sec-Websocket-Accept:[OYe0YNmbNYJBlIBbvzfvovIqjeQ=] Upgrade:[websocket]] Body:{Reader:0xc00280c540} ContentLength:0 TransferEncoding:[] Close:false Uncompressed:false Trailer:map[] Request:0xc0005a5300 TLS:<nil>}" url=ws://127.0.0.1:3340/websocket
Nov 17 16:11:15 INF Start listening for NewBlock messages on web socket 5
Nov 17 16:11:15 INF Subscribing to NewBlock events on web socket 5
Nov 17 16:11:15 INF Sleeping for 12 seconds to receive blocks
Nov 17 16:11:27 DBG Looping through subscription containers subscriptionContainers="[0xc001409df0 0xc0014304f0 0xc001430be0 0xc001129670 0xc0012171c0]"
Nov 17 16:11:27 INF Making sure both clients received at least 10 blocks
Nov 17 16:11:27 INF Making sure both clients received at least 10 blocks
Nov 17 16:11:27 INF Making sure both clients received at least 10 blocks
Nov 17 16:11:27 INF Making sure both clients received at least 10 blocks
Nov 17 16:11:27 INF Making sure both clients received at least 10 blocks
Nov 17 16:11:27 INF Unsubscribing from NewBlock events on web socket 1
Nov 17 16:11:27 INF Sleeping for 7 seconds to make sure unsubscribed client stops receiving blocks
Nov 17 16:11:34 INF Closing web socket 1
Nov 17 16:11:34 INF Closing web socket 2
Nov 17 16:11:34 INF Closing web socket 3
Nov 17 16:11:34 INF Closing web socket 4
Nov 17 16:11:34 INF Closing web socket 5
Nov 17 16:11:34 INF WebSocket Subscription Test OK
Nov 17 16:11:34 INF Starting QoS Tests
Nov 17 16:11:35 INF QOS CHECK OK
Nov 17 16:11:35 INF Sleeping Until All Rewards are collected
Nov 17 16:11:56 INF sleepUntilNextEpoch OK
Nov 17 16:12:16 INF sleepUntilNextEpoch OK
Nov 17 16:12:35 INF sleepUntilNextEpoch OK
Nov 17 16:12:56 INF sleepUntilNextEpoch OK
Nov 17 16:12:56 INF Restarting lava to emergency mode
Nov 17 16:12:56 INF Adding EmergencyMode Start Line to log_name=01_stakeLava
Nov 17 16:12:56 INF Successfully verified EmergencyMode Start Line in the last 20 lines line=19 log_name=01_stakeLava
Nov 17 16:12:56 INF Adding EmergencyMode Start Line to log_name=9_QoS_04
Nov 17 16:12:56 INF Successfully verified EmergencyMode Start Line in the last 20 lines line=19 log_name=9_QoS_04
Nov 17 16:12:56 INF Adding EmergencyMode Start Line to log_name=8_authAccount05
Nov 17 16:12:56 INF Successfully verified EmergencyMode Start Line in the last 20 lines line=1 log_name=8_authAccount05
Nov 17 16:12:56 INF Adding EmergencyMode Start Line to log_name=8_authAccount06
Nov 17 16:12:56 INF Successfully verified EmergencyMode Start Line in the last 20 lines line=1 log_name=8_authAccount06
Nov 17 16:12:56 INF Adding EmergencyMode Start Line to log_name=03_EthProvider_04
Nov 17 16:12:56 INF Successfully verified EmergencyMode Start Line in the last 20 lines line=19 log_name=03_EthProvider_04
Nov 17 16:12:56 INF Adding EmergencyMode Start Line to log_name=05_LavaProvider_04
Nov 17 16:12:56 INF Successfully verified EmergencyMode Start Line in the last 20 lines line=19 log_name=05_LavaProvider_04
Nov 17 16:12:56 INF Adding EmergencyMode Start Line to log_name=8_authAccount00
Nov 17 16:12:56 INF Successfully verified EmergencyMode Start Line in the last 20 lines line=1 log_name=8_authAccount00
Nov 17 16:12:56 INF Adding EmergencyMode Start Line to log_name=8_authAccount01
Nov 17 16:12:56 INF Successfully verified EmergencyMode Start Line in the last 20 lines line=1 log_name=8_authAccount01
Nov 17 16:12:56 INF Adding EmergencyMode Start Line to log_name=8_authAccount03
Nov 17 16:12:56 INF Successfully verified EmergencyMode Start Line in the last 20 lines line=1 log_name=8_authAccount03
Nov 17 16:12:56 INF Adding EmergencyMode Start Line to log_name=8_authAccount08
Nov 17 16:12:56 INF Successfully verified EmergencyMode Start Line in the last 20 lines line=1 log_name=8_authAccount08
Nov 17 16:12:56 INF Adding EmergencyMode Start Line to log_name=9_QoS_07
Nov 17 16:12:56 INF Successfully verified EmergencyMode Start Line in the last 20 lines line=19 log_name=9_QoS_07
Nov 17 16:12:56 INF Adding EmergencyMode Start Line to log_name=03_EthProvider_02
Nov 17 16:12:56 INF Successfully verified EmergencyMode Start Line in the last 20 lines line=19 log_name=03_EthProvider_02
Nov 17 16:12:56 INF Adding EmergencyMode Start Line to log_name=05_LavaProvider_05
Nov 17 16:12:56 INF Successfully verified EmergencyMode Start Line in the last 20 lines line=19 log_name=05_LavaProvider_05
Nov 17 16:12:56 INF Adding EmergencyMode Start Line to log_name=9_QoS_01
Nov 17 16:12:56 INF Successfully verified EmergencyMode Start Line in the last 20 lines line=19 log_name=9_QoS_01
Nov 17 16:12:56 INF Adding EmergencyMode Start Line to log_name=8_authAccount02
Nov 17 16:12:56 INF Successfully verified EmergencyMode Start Line in the last 20 lines line=1 log_name=8_authAccount02
Nov 17 16:12:56 INF Adding EmergencyMode Start Line to log_name=8_authAccount04
Nov 17 16:12:56 INF Successfully verified EmergencyMode Start Line in the last 20 lines line=1 log_name=8_authAccount04
Nov 17 16:12:56 INF Adding EmergencyMode Start Line to log_name=8_authAccount07
Nov 17 16:12:56 INF Successfully verified EmergencyMode Start Line in the last 20 lines line=1 log_name=8_authAccount07
Nov 17 16:12:56 INF Adding EmergencyMode Start Line to log_name=02_jsonProxy
Nov 17 16:12:56 INF Successfully verified EmergencyMode Start Line in the last 20 lines line=19 log_name=02_jsonProxy
Nov 17 16:12:56 INF Adding EmergencyMode Start Line to log_name=9_QoS_02
Nov 17 16:12:56 INF Successfully verified EmergencyMode Start Line in the last 20 lines line=19 log_name=9_QoS_02
Nov 17 16:12:56 INF Adding EmergencyMode Start Line to log_name=9_QoS_06
Nov 17 16:12:56 INF Successfully verified EmergencyMode Start Line in the last 20 lines line=19 log_name=9_QoS_06
Nov 17 16:12:56 INF Adding EmergencyMode Start Line to log_name=03_EthProvider_05
Nov 17 16:12:56 INF Successfully verified EmergencyMode Start Line in the last 20 lines line=19 log_name=03_EthProvider_05
Nov 17 16:12:56 INF Adding EmergencyMode Start Line to log_name=05_LavaProvider_02
Nov 17 16:12:56 INF Successfully verified EmergencyMode Start Line in the last 20 lines line=19 log_name=05_LavaProvider_02
Nov 17 16:12:56 INF Adding EmergencyMode Start Line to log_name=05_LavaProvider_03
Nov 17 16:12:56 INF Successfully verified EmergencyMode Start Line in the last 20 lines line=19 log_name=05_LavaProvider_03
Nov 17 16:12:56 INF Adding EmergencyMode Start Line to log_name=08_Consumer_Cache
Nov 17 16:12:56 INF Successfully verified EmergencyMode Start Line in the last 20 lines line=3 log_name=08_Consumer_Cache
Nov 17 16:12:56 INF Adding EmergencyMode Start Line to log_name=9_QoS_05
Nov 17 16:12:56 INF Successfully verified EmergencyMode Start Line in the last 20 lines line=19 log_name=9_QoS_05
Nov 17 16:12:56 INF Adding EmergencyMode Start Line to log_name=03_EthProvider_01
Nov 17 16:12:56 INF Successfully verified EmergencyMode Start Line in the last 20 lines line=19 log_name=03_EthProvider_01
Nov 17 16:12:56 INF Adding EmergencyMode Start Line to log_name=07_lavaOverLava
Nov 17 16:12:56 INF Successfully verified EmergencyMode Start Line in the last 20 lines line=19 log_name=07_lavaOverLava
Nov 17 16:12:56 INF Adding EmergencyMode Start Line to log_name=9_QoS_03
Nov 17 16:12:56 INF Successfully verified EmergencyMode Start Line in the last 20 lines line=19 log_name=9_QoS_03
Nov 17 16:12:56 INF Adding EmergencyMode Start Line to log_name=03_EthProvider_03
Nov 17 16:12:56 INF Successfully verified EmergencyMode Start Line in the last 20 lines line=19 log_name=03_EthProvider_03
Nov 17 16:12:56 INF Adding EmergencyMode Start Line to log_name=06_RPCConsumer_01
Nov 17 16:12:56 INF Successfully verified EmergencyMode Start Line in the last 20 lines line=19 log_name=06_RPCConsumer_01
Nov 17 16:12:56 INF Adding EmergencyMode Start Line to log_name=00_StartLava
Nov 17 16:12:56 INF Successfully verified EmergencyMode Start Line in the last 20 lines line=19 log_name=00_StartLava
Nov 17 16:12:56 INF Adding EmergencyMode Start Line to log_name=04_jsonConsumer_01
Nov 17 16:12:56 INF Successfully verified EmergencyMode Start Line in the last 20 lines line=19 log_name=04_jsonConsumer_01
Nov 17 16:12:56 INF Adding EmergencyMode Start Line to log_name=05_LavaProvider_01
Nov 17 16:12:56 INF Successfully verified EmergencyMode Start Line in the last 20 lines line=19 log_name=05_LavaProvider_01
Nov 17 16:12:56 INF Adding EmergencyMode Start Line to log_name=09_Provider_Cache
Nov 17 16:12:56 INF Successfully verified EmergencyMode Start Line in the last 20 lines line=3 log_name=09_Provider_Cache
Nov 17 16:12:56 INF Adding EmergencyMode Start Line to log_name=9_QoS_00
Nov 17 16:12:56 INF Successfully verified EmergencyMode Start Line in the last 20 lines line=19 log_name=9_QoS_00
Nov 17 16:12:56 INF Adding EmergencyMode Start Line to log_name=9_QoS_08
Nov 17 16:12:56 INF Successfully verified EmergencyMode Start Line in the last 20 lines line=19 log_name=9_QoS_08
Nov 17 16:12:56 INF stopLava OK
Nov 17 16:12:56 INF Executing Command: ./scripts/test/emergency_mode.sh 100000
Nov 17 16:12:56 INF Waiting for Lava
Nov 17 16:12:56 INF startLava OK
Nov 17 16:13:06 INF Starting Lava OK
Nov 17 16:13:06 INF Executing Command: /home/runner/go/bin/lavap rpcconsumer ./testutil/e2e/e2eConfigs/consumer/lavaConsumerEmergency1.yml --chain-id=lava --from user5 --geolocation 1 --log_level debug --allow-insecure-provider-dialing
Nov 17 16:13:06 INF startRPCEmergencyConsumer OK
Nov 17 16:13:06 INF Waiting REST Consumer
Nov 17 16:13:07 INF Waiting REST Consumer
Nov 17 16:13:07 INF checkRESTConsumer OK
Nov 17 16:13:07 INF getLastBlockTime OK
Nov 17 16:13:07 INF Waiting for finishing current epoch and waiting for 2 more virtual epochs
Nov 17 16:13:18 INF 0 : VIRTUAL EPOCH ENDED
Nov 17 16:13:42 INF 1 : VIRTUAL EPOCH ENDED
Nov 17 16:14:06 INF 2 : VIRTUAL EPOCH ENDED
Nov 17 16:14:07 INF Adding EmergencyMode End Line to log_name=11_RPCEmergencyConsumer_01
Nov 17 16:14:07 INF Adding EmergencyMode End Line to log_name=03_EthProvider_04
Nov 17 16:14:07 INF Adding EmergencyMode End Line to log_name=05_LavaProvider_04
Nov 17 16:14:07 INF Adding EmergencyMode End Line to log_name=8_authAccount00
Nov 17 16:14:07 INF Adding EmergencyMode End Line to log_name=8_authAccount01
Nov 17 16:14:07 INF Adding EmergencyMode End Line to log_name=8_authAccount03
Nov 17 16:14:07 INF Adding EmergencyMode End Line to log_name=8_authAccount08
Nov 17 16:14:07 INF Adding EmergencyMode End Line to log_name=9_QoS_07
Nov 17 16:14:07 INF Adding EmergencyMode End Line to log_name=10_StartLavaInEmergencyMode
Nov 17 16:14:07 INF Adding EmergencyMode End Line to log_name=03_EthProvider_02
Nov 17 16:14:07 INF Adding EmergencyMode End Line to log_name=05_LavaProvider_05
Nov 17 16:14:07 INF Adding EmergencyMode End Line to log_name=9_QoS_01
Nov 17 16:14:07 INF Adding EmergencyMode End Line to log_name=8_authAccount02
Nov 17 16:14:07 INF Adding EmergencyMode End Line to log_name=8_authAccount04
Nov 17 16:14:07 INF Adding EmergencyMode End Line to log_name=8_authAccount07
Nov 17 16:14:07 INF Adding EmergencyMode End Line to log_name=02_jsonProxy
Nov 17 16:14:07 INF Adding EmergencyMode End Line to log_name=9_QoS_02
Nov 17 16:14:07 INF Adding EmergencyMode End Line to log_name=9_QoS_06
Nov 17 16:14:07 INF Adding EmergencyMode End Line to log_name=03_EthProvider_05
Nov 17 16:14:07 INF Adding EmergencyMode End Line to log_name=05_LavaProvider_02
Nov 17 16:14:07 INF Adding EmergencyMode End Line to log_name=05_LavaProvider_03
Nov 17 16:14:07 INF Adding EmergencyMode End Line to log_name=08_Consumer_Cache
Nov 17 16:14:07 INF Adding EmergencyMode End Line to log_name=9_QoS_05
Nov 17 16:14:07 INF Adding EmergencyMode End Line to log_name=03_EthProvider_01
Nov 17 16:14:07 INF Adding EmergencyMode End Line to log_name=07_lavaOverLava
Nov 17 16:14:07 INF Adding EmergencyMode End Line to log_name=9_QoS_03
Nov 17 16:14:07 INF Adding EmergencyMode End Line to log_name=03_EthProvider_03
Nov 17 16:14:07 INF Adding EmergencyMode End Line to log_name=06_RPCConsumer_01
Nov 17 16:14:07 INF Adding EmergencyMode End Line to log_name=00_StartLava
Nov 17 16:14:07 INF Adding EmergencyMode End Line to log_name=04_jsonConsumer_01
Nov 17 16:14:07 INF Adding EmergencyMode End Line to log_name=05_LavaProvider_01
Nov 17 16:14:07 INF Adding EmergencyMode End Line to log_name=09_Provider_Cache
Nov 17 16:14:07 INF Adding EmergencyMode End Line to log_name=9_QoS_00
Nov 17 16:14:07 INF Adding EmergencyMode End Line to log_name=9_QoS_08
Nov 17 16:14:07 INF Adding EmergencyMode End Line to log_name=01_stakeLava
Nov 17 16:14:07 INF Adding EmergencyMode End Line to log_name=9_QoS_04
Nov 17 16:14:07 INF Adding EmergencyMode End Line to log_name=8_authAccount05
Nov 17 16:14:07 INF Adding EmergencyMode End Line to log_name=8_authAccount06
Nov 17 16:14:07 INF REST RELAY TESTS OK
Nov 17 16:14:07 DBG success writing to file bytesWritten=9381 fileName=9_QoS_06 lines=9381
Nov 17 16:14:07 INF Found Emergency start line Line index=292 file=9_QoS_06
Nov 17 16:14:07 INF Found Emergency end line Line index=293 file=9_QoS_06
Nov 17 16:14:07 DBG success writing to file bytesWritten=11770082 fileName=02_jsonProxy lines=11770082
Nov 17 16:14:07 INF Found Emergency start line Line index=7776 file=02_jsonProxy
Nov 17 16:14:07 INF Found Emergency end line Line index=9341 file=02_jsonProxy
Nov 17 16:14:07 DBG success writing to file bytesWritten=9381 fileName=9_QoS_02 lines=9381
Nov 17 16:14:07 INF Found Emergency start line Line index=292 file=9_QoS_02
Nov 17 16:14:07 INF Found Emergency end line Line index=293 file=9_QoS_02
Nov 17 16:14:07 DBG success writing to file bytesWritten=6050910 fileName=05_LavaProvider_03 lines=6050910
Nov 17 16:14:07 INF Found Emergency start line Line index=27976 file=05_LavaProvider_03
Nov 17 16:14:07 INF Found Emergency end line Line index=28115 file=05_LavaProvider_03
Nov 17 16:14:07 DBG success writing to file bytesWritten=274 fileName=08_Consumer_Cache lines=274
Nov 17 16:14:07 INF Found Emergency start line Line index=3 file=08_Consumer_Cache
Nov 17 16:14:07 INF Found Emergency end line Line index=4 file=08_Consumer_Cache
Nov 17 16:14:07 DBG success writing to file bytesWritten=14084 fileName=9_QoS_05 lines=14084
Nov 17 16:14:07 INF Found Emergency start line Line index=414 file=9_QoS_05
Nov 17 16:14:07 INF Found Emergency end line Line index=415 file=9_QoS_05
Nov 17 16:14:07 DBG success writing to file bytesWritten=27466 fileName=03_EthProvider_05 lines=27466
Nov 17 16:14:07 INF Found Emergency start line Line index=212 file=03_EthProvider_05
Nov 17 16:14:07 INF Found Emergency end line Line index=222 file=03_EthProvider_05
Nov 17 16:14:07 DBG success writing to file bytesWritten=5439354 fileName=05_LavaProvider_02 lines=5439354
Nov 17 16:14:07 INF Found Emergency start line Line index=27839 file=05_LavaProvider_02
Nov 17 16:14:07 INF Found Emergency end line Line index=27963 file=05_LavaProvider_02
Nov 17 16:14:07 DBG success writing to file bytesWritten=14106 fileName=9_QoS_03 lines=14106
Nov 17 16:14:07 INF Found Emergency start line Line index=414 file=9_QoS_03
Nov 17 16:14:07 INF Found Emergency end line Line index=415 file=9_QoS_03
Nov 17 16:14:07 DBG success writing to file bytesWritten=66196 fileName=03_EthProvider_01 lines=66196
Nov 17 16:14:07 INF Found Emergency start line Line index=424 file=03_EthProvider_01
Nov 17 16:14:07 INF Found Emergency end line Line index=433 file=03_EthProvider_01
Nov 17 16:14:07 DBG success writing to file bytesWritten=2287 fileName=07_lavaOverLava lines=2287
Nov 17 16:14:07 INF Found Emergency start line Line index=96 file=07_lavaOverLava
Nov 17 16:14:07 INF Found Emergency end line Line index=97 file=07_lavaOverLava
Nov 17 16:14:07 DBG success writing to file bytesWritten=35670 fileName=03_EthProvider_03 lines=35670
Nov 17 16:14:07 INF Found Emergency start line Line index=270 file=03_EthProvider_03
Nov 17 16:14:07 INF Found Emergency end line Line index=278 file=03_EthProvider_03
Nov 17 16:14:07 DBG success writing to file bytesWritten=51206038 fileName=06_RPCConsumer_01 lines=51206038
Nov 17 16:14:07 INF Found Emergency start line Line index=186630 file=06_RPCConsumer_01
Nov 17 16:14:07 INF Found Emergency end line Line index=186638 file=06_RPCConsumer_01
Nov 17 16:14:07 DBG success writing to file bytesWritten=5562658 fileName=05_LavaProvider_01 lines=5562658
Nov 17 16:14:07 INF Found Emergency start line Line index=28369 file=05_LavaProvider_01
Nov 17 16:14:07 INF Found Emergency end line Line index=28541 file=05_LavaProvider_01
Nov 17 16:14:07 DBG success writing to file bytesWritten=274 fileName=09_Provider_Cache lines=274
Nov 17 16:14:07 INF Found Emergency start line Line index=3 file=09_Provider_Cache
Nov 17 16:14:07 INF Found Emergency end line Line index=4 file=09_Provider_Cache
Nov 17 16:14:07 DBG success writing to file bytesWritten=14098 fileName=9_QoS_00 lines=14098
Nov 17 16:14:07 INF Found Emergency start line Line index=414 file=9_QoS_00
Nov 17 16:14:07 INF Found Emergency end line Line index=415 file=9_QoS_00
Nov 17 16:14:07 DBG success writing to file bytesWritten=9381 fileName=9_QoS_08 lines=9381
Nov 17 16:14:07 INF Found Emergency start line Line index=292 file=9_QoS_08
Nov 17 16:14:07 INF Found Emergency end line Line index=293 file=9_QoS_08
Nov 17 16:14:07 DBG success writing to file bytesWritten=1076161 fileName=00_StartLava lines=1076161
Nov 17 16:14:07 DBG success writing to file bytesWritten=71232 fileName=04_jsonConsumer_01 lines=71232
Nov 17 16:14:07 INF Found Emergency start line Line index=195 file=04_jsonConsumer_01
Nov 17 16:14:07 INF Found Emergency end line Line index=203 file=04_jsonConsumer_01
Nov 17 16:14:07 DBG success writing to file bytesWritten=333 fileName=8_authAccount05 lines=333
Nov 17 16:14:07 INF Found Emergency start line Line index=1 file=8_authAccount05
Nov 17 16:14:07 INF Found Emergency end line Line index=2 file=8_authAccount05
Nov 17 16:14:07 DBG success writing to file bytesWritten=331 fileName=8_authAccount06 lines=331
Nov 17 16:14:07 INF Found Emergency start line Line index=1 file=8_authAccount06
Nov 17 16:14:07 INF Found Emergency end line Line index=2 file=8_authAccount06
Nov 17 16:14:07 DBG success writing to file bytesWritten=8402 fileName=01_stakeLava lines=8402
Nov 17 16:14:07 INF Found Emergency start line Line index=413 file=01_stakeLava
Nov 17 16:14:07 INF Found Emergency end line Line index=414 file=01_stakeLava
Nov 17 16:14:07 DBG success writing to file bytesWritten=14094 fileName=9_QoS_04 lines=14094
Nov 17 16:14:07 INF Found Emergency start line Line index=414 file=9_QoS_04
Nov 17 16:14:07 INF Found Emergency end line Line index=415 file=9_QoS_04
Nov 17 16:14:07 DBG success writing to file bytesWritten=333 fileName=8_authAccount00 lines=333
Nov 17 16:14:07 INF Found Emergency start line Line index=1 file=8_authAccount00
Nov 17 16:14:07 INF Found Emergency end line Line index=2 file=8_authAccount00
Nov 17 16:14:07 DBG success writing to file bytesWritten=332 fileName=8_authAccount01 lines=332
Nov 17 16:14:07 INF Found Emergency start line Line index=1 file=8_authAccount01
Nov 17 16:14:07 INF Found Emergency end line Line index=2 file=8_authAccount01
Nov 17 16:14:07 DBG success writing to file bytesWritten=333 fileName=8_authAccount03 lines=333
Nov 17 16:14:07 INF Found Emergency start line Line index=1 file=8_authAccount03
Nov 17 16:14:07 INF Found Emergency end line Line index=2 file=8_authAccount03
Nov 17 16:14:07 DBG success writing to file bytesWritten=332 fileName=8_authAccount08 lines=332
Nov 17 16:14:07 INF Found Emergency start line Line index=1 file=8_authAccount08
Nov 17 16:14:07 INF Found Emergency end line Line index=2 file=8_authAccount08
Nov 17 16:14:07 DBG success writing to file bytesWritten=251782 fileName=11_RPCEmergencyConsumer_01 lines=251782
Nov 17 16:14:07 INF Found Emergency end line Line index=817 file=11_RPCEmergencyConsumer_01
Nov 17 16:14:07 DBG success writing to file bytesWritten=33955 fileName=03_EthProvider_04 lines=33955
Nov 17 16:14:07 INF Found Emergency start line Line index=262 file=03_EthProvider_04
Nov 17 16:14:07 INF Found Emergency end line Line index=270 file=03_EthProvider_04
Nov 17 16:14:07 DBG success writing to file bytesWritten=5382190 fileName=05_LavaProvider_04 lines=5382190
Nov 17 16:14:07 INF Found Emergency start line Line index=27477 file=05_LavaProvider_04
Nov 17 16:14:07 INF Found Emergency end line Line index=27633 file=05_LavaProvider_04
Nov 17 16:14:07 DBG success writing to file bytesWritten=9381 fileName=9_QoS_01 lines=9381
Nov 17 16:14:07 INF Found Emergency start line Line index=292 file=9_QoS_01
Nov 17 16:14:07 INF Found Emergency end line Line index=293 file=9_QoS_01
Nov 17 16:14:07 DBG success writing to file bytesWritten=331 fileName=8_authAccount02 lines=331
Nov 17 16:14:07 INF Found Emergency start line Line index=1 file=8_authAccount02
Nov 17 16:14:07 INF Found Emergency end line Line index=2 file=8_authAccount02
Nov 17 16:14:07 DBG success writing to file bytesWritten=333 fileName=8_authAccount04 lines=333
Nov 17 16:14:07 INF Found Emergency start line Line index=1 file=8_authAccount04
Nov 17 16:14:07 INF Found Emergency end line Line index=2 file=8_authAccount04
Nov 17 16:14:07 DBG success writing to file bytesWritten=333 fileName=8_authAccount07 lines=333
Nov 17 16:14:07 INF Found Emergency start line Line index=1 file=8_authAccount07
Nov 17 16:14:07 INF Found Emergency end line Line index=2 file=8_authAccount07
Nov 17 16:14:07 DBG success writing to file bytesWritten=14109 fileName=9_QoS_07 lines=14109
Nov 17 16:14:07 INF Found Emergency start line Line index=414 file=9_QoS_07
Nov 17 16:14:07 INF Found Emergency end line Line index=415 file=9_QoS_07
Nov 17 16:14:07 DBG success writing to file bytesWritten=7133 fileName=10_StartLavaInEmergencyMode lines=7133
Nov 17 16:14:07 INF Found Emergency end line Line index=48 file=10_StartLavaInEmergencyMode
Nov 17 16:14:07 DBG success writing to file bytesWritten=26353 fileName=03_EthProvider_02 lines=26353
Nov 17 16:14:07 INF Found Emergency start line Line index=207 file=03_EthProvider_02
Nov 17 16:14:07 INF Found Emergency end line Line index=215 file=03_EthProvider_02
Nov 17 16:14:07 DBG success writing to file bytesWritten=5643161 fileName=05_LavaProvider_05 lines=5643161
Nov 17 16:14:07 INF Found Emergency start line Line index=28231 file=05_LavaProvider_05
Nov 17 16:14:07 INF Found Emergency end line Line index=28351 file=05_LavaProvider_05
ERROR:  Nov 17 16:11:34 ERR Got error from srv.Send() error="rpc error: code = Unavailable desc = transport is closing" GUID=10068720810661214158