Web3.py on AWS Lambda

[greese]

• Version: current
• Python: 2.7
• OS: linux - was lambda

What was wrong?

Dependencies on cytoolz (which depends on a C compiler) make deployment on AWS lambda not work, whereas web3.js is able to deploy on AWS lambda.

The specific error is:

Unable to import module 'mylambda': /var/task/cytoolz/functoolz.so: undefined symbol: PyFPE_jbuf

How can it be fixed?

No clue. Not sure what the dependency on cytoolz does.

[carver]

I don't know much about the lambda options. Is it py2-only? If so, web3.py v4 won't work on it anyway, and v3 is in only being maintained with critical security patches.

This suggests that CPython was compiled without a necessary flag (not necessarily that cytoolz needs to be compiled):
https://stackoverflow.com/a/47703373/8412986

[greese]

Actually, I'm doing both Python 3.6 and Python 2.7 builds. I just reported the 2.7. Here's the 3.6 output from lambda:

Unable to import module 'sparq.lambdas.rates': No module named '_pysha3'

Now, the pysha3 stuff is actually installed:

(p3) ubuntu@host:~/deploy/p3$ find . -name \*pysha3\* -print
./lib/python3.5/site-packages/_pysha3.cpython-35m-x86_64-linux-gnu.so
./lib/python3.5/site-packages/pysha3-1.0.2.dist-info
(p3) ubuntu@host:~/deploy/p3$ pip install pysha3
Requirement already satisfied: pysha3 in ./lib/python3.5/site-packages

[carver]

Is sparq.lambdas.rates your module? It's a little weird that it's importing a sha3 module directly. If you need it for something, import sha3 instead of _pysha3 (but you shouldn't, you should just be able to use Web3.sha3()). Anyway, this may be some other thing unrelated to web3.py. What call is triggering the error? Do you still get the error if the only thing your code does is call Web3.sha3(text='testing')?

The full stack trace and code is needed to debug further.

I'm guessing some kind of module naming conflict issue, but don't have enough information.

[greese]

Yeah. I went ahead and created a new handler that looks like this:

from web3 import *

def lambda_handler(event, context):
    return Web3.sha3(text='testing')

And I got the same:

Unable to import module 'hello': No module named '_pysha3'

[greese]

As far as a stack trace, it's not even loading the handler module, so it's not getting to a stack trace point.

[carver]

Reading this SO post made me think about the environment. Maybe it can't import because it was built with the wrong target environment?

I'm trying to guess the target build environment based on the file name:

./lib/python3.5/site-packages/_pysha3.cpython-35m-x86_64-linux-gnu.so

I'm a newb at python C extensions, but I'm guessing the 35 in there means Python 3.5 and you mentioned that your Lambda is running in Py3.6. Everything else looks right, I assume Lambda it runs in linux, on a 64-bit cpu with cpython (not pypy).

[greese]

Good catch. I'll have a look at that.

[greese]

This may take longer than I'd like. For some reason, when installing ethereum, I am getting the no Python.h thing, but python3-dev is installed. Grrrr...

[carver]

Hm, well we're inching toward completely removing the dependency on pyethereum so keep an eye on #505 .

[greese]

OK, that last one was me failing to realize I needed a python3.6-dev specific to 3.6.

I am still getting the C compiler issue in Python 3:

Unable to import module 'sparq.lambdas.rates': /var/task/cytoolz/functoolz.cpython-36m-x86_64-linux-gnu.so: undefined symbol: PyFPE_jbuf

[carver]

So back to my first comment: looks like you need a CPython that is compiled with --with-fpectl.

[greese]

This gets beyond my Python skill set. I've done some googling, but can't figure out how to do that. Any quick instructions? Thanks.

[carver]

I've never had to compile CPython on my machines, so I'm guessing it's actually a default setting. Either try to download the python binaries instead of using whatever is coming default, or you could try this guide (which I imagine works for 3.6 just fine as well):

https://passingcuriosity.com/2015/installing-python-from-source/

[greese]

Thanks!

Also, I found a more detailed description of the issue here: https://bugs.python.org/issue29137

I'll pass on exact instructions in this thread when I get this working. For now, it appears certain there's no web3.py issue.

[greese]

The short answer is that I have it working. It requires a cytools build from source.

The long answer:

The Python environment within AWS lambda is 3.6 and built WITHOUT --with-fpectl, but most environments are built with it. To build a proper lambda environment, here are the steps:

1. You need Ubuntu 17.04. It's the only official AMI that can support 3.6. The AWS Linux ones that exist for Lambda do not support 3.6. Go figure. You actually need to upgrade an Ubuntu 16.04 to 17.04. Fun.
2. Do a normal virtualenv configuration of your 3.6 environment.
3. BEFORE pip installing ethereum or web3.py, you need to manually build cytoolz.

git clone https://github.com/pytoolz/cytoolz.git
cd cytoolz
git checkout release
make

for file in VITUALENVROOT/lib/python3.6/site-packages/cytoolz/*; do
  cp -R cytoolz/`basename $file`VIRTUALENVROOT/lib/python3.6/site-packages/cytoolz/;
done

5. pip install ethereum -y
6. pip install web3 -y
7. [if you are deploying new contracts] pip install -y py-solc
8. Package up your lambda

Building cytoolz on your Ubuntu system with figure it all out.

[greese]

Closing this out as it isn't web3.py issue.

[carver]

Great, thanks for posting the writeup here!

[sot528]

Thank you for your post.
And FYI: I could use just pip for ethereum & web3 when I used Amazon Linux docker image for make .zip file.

• https://github.com/sot528/lambda-web3py/blob/master/Dockerfile

[iteratelance]

@greese

I tried this approach, albeit using an older version of Ubuntu 16.04 but was able to get everything to compile and work locally by installing python3.6-dev, build-essential, and libssl-dev. However, unless I'm misreading, doesn't your script assume that cytoolz is already in the virtualenv and copies files that exist in the new build directory to overwrite existing files in the virtualenv?

Did you run the make command within the virtualenv?

I tried mv cytoolz VIRTUALENVROOT/lib/python3.6/site-packages/ but to no avail. Works locally but not on Lambda.

Also the snippet appears to be missing a space between $file and VIRTUALENVROOT unless again I'm in error.

for file in VITUALENVROOT/lib/python3.6/site-packages/cytoolz/*; do
  cp -R cytoolz/`basename $file` VIRTUALENVROOT/lib/python3.6/site-packages/cytoolz/;
done

[iteratelance]

I was able to get this working simply by starting with an Ubuntu 18.04 image and adding the following packages via
sudo apt-get update && sudo apt-get install

        - python3.6-dev
        - build-essential
        - libssl-dev
        - python3-pip
        - locales

Then just ran virtualenv and pip as usually and all was well.

[0xRadix]

hey @iteratelance, @greese, running into the same issue, and I do have a basic question on how to resolve this.

Here is what I did so far:

1. Added all Python packages needed (ethereum, web3, py-solc, etc.) on an EC2 instance running Ubuntu 18.04
2. Ran virtualenv on the same EC2 instance and used pip to install the same packages mentioned above
3. Created an archive that contains all the necessary libraries (step 8 here: https://docs.aws.amazon.com/pinpoint/latest/developerguide/tutorials-importing-data-create-python-package.html)

What I am having difficulty understanding is how do we make this EC2 environment available for a given AWS Lambda function? Do we just take the zip, copy it over from EC2 to my local machine, and then upload it to AWS Lambda? Should we be using Docker instead (i.e. https://docs.aws.amazon.com/lambda/latest/dg/images-create.html)?

[0xRadix]

hey @iteratelance, @greese, running into the same issue, and I do have a basic question on how to resolve this.

Here is what I did so far:

1. Added all Python packages needed (ethereum, web3, py-solc, etc.) on an EC2 instance running Ubuntu 18.04

2. Ran virtualenv on the same EC2 instance and used pip to install the same packages mentioned above

3. Created an archive that contains all the necessary libraries (step 8 here: https://docs.aws.amazon.com/pinpoint/latest/developerguide/tutorials-importing-data-create-python-package.html)

What I am having difficulty understanding is how do we make this EC2 environment available for a given AWS Lambda function? Do we just take the zip, copy it over from EC2 to my local machine, and then upload it to AWS Lambda? Should we be using Docker instead (i.e. https://docs.aws.amazon.com/lambda/latest/dg/images-create.html)?

Was able to figure this out. For anyone else that may be running into the same issue:

1. In the EC2 instance, install all the packages you need
2. Zip all your dependences found in venv/lib/pythonX.X/site-packages, and upload them to AWS S3
3. Have your Lambda function use this environment (at the time of this writing, the UI allows you to reference whatever you uploaded onto S3, just copy/paste the link)

[kclowes]

@0xRadix Thanks for coming back and posting!

[GraphicalDot]

There is a more easy way to do it now.
Creating an environment on CLoud9 similar to the lambda function and then creating a zip file from the python packages.

https://aws.amazon.com/premiumsupport/knowledge-center/lambda-import-module-error-python/

[ayjammoulMTS]

@GraphicalDot I tried that in addition to all the other approaches mentioned yet nothing seemed to work, what might be the issue?