LND Out of Memory with last version

[croll83]

Hi,

I updated the lncli-web to last version yesterday, but as soon as I open the webpage, lnd get killed because out of memory.

I'm running without docker on the following configuration:

• Ubuntu 16.04
• 2VCPU 4GB RAM
• Bitcoin Core 0.16.0
• LND last version (Master - update yesterday)
• Node 6.10.0

I upgraded the VM with 4vCPU and 8GB Ram after the error and it still persists.

It was working as a charme with the gitlab code of 3 months ago, until yesterday (with last version of lnd). Now since there's no tags or branches to checkout, I'm not able to get it back working...

As soon as I open the webpage of lncli-web, it successfully retrieve the nodeinfo data tab, but it fails to retrieve Netwok Info tab.

This is the log of lnd:

fatal error: runtime: out of memory

runtime stack:
runtime.throw(0xf73577, 0x16)
/usr/lib/go-1.10/src/runtime/panic.go:619 +0x81
runtime.sysMap(0xc430600000, 0x124c590000, 0x0, 0x1766ed8)
/usr/lib/go-1.10/src/runtime/mem_linux.go:216 +0x20a
runtime.(*mheap).sysAlloc(0x174d400, 0x124c590000, 0x7f37e9b7d890)
/usr/lib/go-1.10/src/runtime/malloc.go:470 +0xd4
runtime.(*mheap).grow(0x174d400, 0x9262c7, 0x0)
/usr/lib/go-1.10/src/runtime/mheap.go:907 +0x60
runtime.(*mheap).allocSpanLocked(0x174d400, 0x9262c7, 0x1766ee8, 0x7ffd7ac599d0)
/usr/lib/go-1.10/src/runtime/mheap.go:820 +0x301
runtime.(*mheap).alloc_m(0x174d400, 0x9262c7, 0xffffffffffff0101, 0x7ffd7ac59a00)
/usr/lib/go-1.10/src/runtime/mheap.go:686 +0x118
runtime.(*mheap).alloc.func1()
/usr/lib/go-1.10/src/runtime/mheap.go:753 +0x4d
runtime.(*mheap).alloc(0x174d400, 0x9262c7, 0x7ffd7a010101, 0x4161bc)
/usr/lib/go-1.10/src/runtime/mheap.go:752 +0x8a
runtime.largeAlloc(0x124c58c22c, 0x450101, 0x7f37e9bbf6c8)
/usr/lib/go-1.10/src/runtime/malloc.go:826 +0x94
runtime.mallocgc.func1()
/usr/lib/go-1.10/src/runtime/malloc.go:721 +0x46
runtime.systemstack(0x0)
/usr/lib/go-1.10/src/runtime/asm_amd64.s:409 +0x79
runtime.mstart()
/usr/lib/go-1.10/src/runtime/proc.go:1170

goroutine 71931 [running]:
runtime.systemstack_switch()

[mably]

I'm running a btcd/lnd/lncli-web stack on a 2 GB RAM VPS without any problem.

All components updated to latest master.

After a few hours, btcd takes about 50% mem, lnd 5% and lncli-web 5% too.

2 GB is not enough for building though, but should be ok for running.

[croll83]

that's very strange... I didn't have any problem before pulling last master of lncli-web.

LND seems to work fine, I successfully run some rpc commands like describegraph or queryroutes and I have results without problems.

It's seems some looping activity done by lncli-web on mybox.

What could be the issue?

[mably]

My NodeJS version is 7.8.0, that's the only difference I can see with your configuration.

May be you could try to upgrade to a more recent NodeJS version.

[croll83]

NAK.

With Node 7.8.0 I can't build gRPC:

npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@^1.0.0 (node_modules/chokidar/node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for [email protected]: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})
npm ERR! Linux 4.4.0-130-generic
npm ERR! argv "/home/bitcoin/.nvm/versions/node/v7.8.0/bin/node" "/home/bitcoin/.nvm/versions/node/v7.8.0/bin/npm" "install"
npm ERR! node v7.8.0
npm ERR! npm v4.2.0
npm ERR! code ELIFECYCLE
npm ERR! errno 1

npm ERR! [email protected] install: node-pre-gyp install --fallback-to-build --library=static_library
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] install script 'node-pre-gyp install --fallback-to-build --library=static_library'.
npm ERR! Make sure you have the latest version of node.js and npm installed.
npm ERR! If you do, this is most likely a problem with the grpc package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR! node-pre-gyp install --fallback-to-build --library=static_library
npm ERR! You can get information on how to open an issue for this project with:
npm ERR! npm bugs grpc
npm ERR! Or if that isn't available, you can get their info via:
npm ERR! npm owner ls grpc
npm ERR! There is likely additional logging output above.

npm ERR! Please include the following file with any support request:
npm ERR! /home/bitcoin/.npm/_logs/2018-07-13T12_35_41_883Z-debug.log

[mably]

If you could try to identify the particular commit that causes the regression that could help fixing it.

[croll83]

Guys no luck at all.

Some tests I did:

• use node 7.8.0 on Ubuntu 16.04 -> npm install fails during grpc build (uses grpc package too new).
• use node 6.5.1 -> npm install succeed (probably take an old version of grpc). Same out-of-memory problem on lnd.
• use node 7.8.0 with package.json modified (grpc 1.1.1) -> npm install ok. Same out-of-memory problem on lnd.

Tested on a different box (still Ubuntu 16.04), everything from scratch (lnd fresh build up to master, lncli fresh build with 7.8.0 and grpc 1.1.1 -> still fail. out-of-memory for lnd that get killed.

Any idea on additional tests?