My thoughts on the Python FDK

[paulfelix]

I want to share some thoughts on the fdk-python design and some experiences (so far) developing my own Python FDK. This is not a proposal to make any changes to the fdk-python, just thoughts.

Two substantial changes to the latest fdk-python include:

1. Homegrown async http code
2. A change in the way the function gets executed in order to facilitate lazy loading of the function's modules.

The main driver for these changes, from my understanding, is to a) improve cold startup time up to the point when the function is listening on the unix socket, and b) support Python 3 asyncio programming in the function itself.

I feel the implemented solutions to these problems are unnecessary and result in too much extra code that has to be developed/maintained/fixed. Specifically, the homegrown async http code is unnecessary, and the lazy importing approach (using a Python 3.7 feature) is also unnecessary. It also sets the Python FDK programming model apart from the FDKs for other languages.

Here's my experience so far developing my own Python FDK. I think it also solves the same problems while using much less code and avoiding a change to the function programming model. The code shown here is working code, but still just proof-of-concept, as it lacks extra error handling, return types, etc.

I chose to use the eventlet wsgi package to listen to http over the unix socket. Here's my main code fo far:

import os
import sys
import socket
from eventlet import wsgi, listen
from .application import Application


def run(handler):
    fn_listener = os.environ.get('FN_LISTENER')
    if not fn_listener:
        sys.exit('FN_LISTENER is not set')

    socket_file = fn_listener.lstrip('unix:')
    sock = listen(socket_file, family=socket.AF_UNIX)
    wsgi.server(sock, Application(handler))

Here's the basic wsgi application code. It constructs a Request object from the environment variables (code not shown) that is passed to the function's handler.

import sys
import json
import logging
from .request import Request

logger = logging.getLogger()


class Application:
    def __init__(self, handler):
        self._handler = handler

    def __call__(self, environ, start_response):
        try:
            request = Request(environ)
            response = self._handler(request)
            start_response('200 OK', [('Content-Type', 'application/json')])
            response_body = json.dumps(response).encode()
            return [response_body]
        except Exception:
            logger.exception('Error executing function')
        finally:
            sys.stdout.flush()
            sys.stderr.flush()

Here is the function's func.py code:

import fnpy


def handler(request):
    from myfunction import execute
    return execute(request)


if __name__ == "__main__":
    fnpy.run(handler)

Notice that the function's modules are not imported until the handler is called. Unless I'm missing something, that's all we need for lazy loading. The next time the handler is called, the myfunction module will have been loaded already, so the import will be a no-op.

Using my FDK, I have been working on a function that handles GraphQL queries and makes concurrent calls to other services using Python asyncio. The time from cold-start container loading to the first handler call is < 1sec.

I have been able to run my function on OSX and CentOS platforms with the same performance on both.

[denismakogon]

Hi and thanks for the feedback.
I’d like to answer to certain statements from your feedback:

First of all, as for now Fn ships 1 request at the time to a function. However, nobody said that it’s not going to change in months.
It means that having a sync server (which is definitely easy to implement, indeed) will make us re-do the whole FDK as soon as protocol will change, which brings the question like why an FDK is not capable to do more than one request at the time?

So, between two options: “sync server” VS “concurrent server that can do sync request as well” I would personally pick the one that is capable to do more.

The next concern here is an HTTP framework choice:

Homegrown async http code

Unfortunately, non of the existing HTTP async concurrent frameworks on a market can deal with cold start thing. They all are huge filled with unneeded code, tons of code. It’s a shame that Python standard lib doesn’t have good choice of async HTTP framework, so, we have to find our own way.

Homegrown. Why? Well, say hi to cold start time, we have pretty strong requirement: FDK must start within 3 seconds. So, I tried to use this framework as 3rd-party dependency, however, the cold start time was beyond 4 second, which is not acceptable for sure. But, moving this framework to an FDK repo made the whole startup time lower, less than 2.5 seconds. This is a win, but what’s the cost? A cost of a trade-off.

eventlet VS asyncio

This is a battle for year. Eventlet been around python 2 for a while, but tulip(asyncio) appeared not so long ago.

However, eventlet lost its battle to become de-facto native core technology.

So, there’s only one problem - due to certain disagreement Python standard lib doesn’t have native async http server. That’s the problem we need to bring to python cores rather than arguing here, don’t you think so?

Note: The biggest advantage over eventlet is that asyncio is built-in into a language and a lot of tools being developed with it, the amount of libs with asyncio is way more bigger than tools with eventlet. Just check how many tools live at aio-libs space as an example of the technology adoption.

[denismakogon]

It also sets the Python FDK programming model apart from the FDKs for other languages.

That’s true and I foresee that for any interpreted language we have officially supported we would concider changing the way it woks by adopting the concept of delayed imports, of course, if that feature is supported.

[paulfelix]

Thanks for the reply.

First, let's put to rest the notion that you need a homegrown solution to achieve the needed startup performance. My FDK proves that you don't.

Secondly, if you insist on an async server using Python 3 asyncio specifically (I still don't see the need, but...), I'm sure there's a better solution based on tools with community support instead of code developed by one individual. You say asyncio has won the battle and is built in, but then I see all this code you had to write, and that concerns me. Maybe check out this article as perhaps a better solution? not sure:

https://medium.com/@pgjones/an-asyncio-socket-tutorial-5e6f3308b8b0

Finally, My FDK proves that the change in the FDK programming model to support lazy importing is COMPLETELY UNNECESSARY for the Python FDK.

[denismakogon]

My FDK proves that you don't.

As soon as we'd found asyncio-based HTTP framework that can start within less than 3 seconds, we'd remove our own code immediately, but changing asyncio to eventlet is not a solution.

Maybe check out this article as perhaps a better solution? not sure https://medium.com/@pgjones/an-asyncio-socket-tutorial-5e6f3308b8b0

Well, if you'd check our HTTP framework you'd see that we use the same interface (asyncio.Protocol) for implementing an HTTP server, but our code is better because it does implement keep-alive connection handling which is crucial for an FDK.

Finally, My FDK proves that the change in the FDK programming model to support lazy importing is COMPLETELY UNNECESSARY for the Python FDK.

to be clear, your code proves that every import inside of an HTTP request handler would not cause any problems because they are all inside of a function with limited availability scope (when our FDK doesn't limit the scope).

In any case, placing all imports inside of a function is not pythonic way to solve imports cold start issue.

[rdallman]

thanks @paulfelix for your feedback.

we did try https://medium.com/@pgjones/an-asyncio-socket-tutorial-5e6f3308b8b0 but it wasn't quite sufficient for our needs, as Denis pointed out, we pretty much ended up copying a lot of stuff from sanic and chose instead to just basically vendor sanic and strip some of the stuff we did not need to reduce the cold start time.

very good feedback on the imports issue, if the python3.7 bit becomes a blocker for users that seems like a pretty sane route to go (just tell users to import in handler). I'm skeptical as well of requiring python3.7, as it hasn't been out for long and certain enterprise users may need more flexibility on supported python versions that have been around for a while longer. honestly, would prefer this approach for flexibility alone, as 3.5+ will support the async stuff fine which was one of our main requirements.

I agree that for the FDK in their current state of just running 1 function at a time, asyncio doesn't seem necessary at all to me, either, and would allow us to not need the hassle of the async stuff at all. python itself seems very fragmented, and it's not clear the best path forward with the fdk python for a lot of reasons (python 2 support people want, some people want async, some don't need it - don't mean to spark this discussion, just reiterating why this kind of feedback is useful)

again, thanks a lot for digging in and pasting your honest feedback, one of my greatest fears with the python fdk (or any fdk, really) is users just going and implementing their own - if that happens, then we need to reconsider why that's the case and work to fix that, the goal is for these to be useful and any of our ideologies shouldn't get in the way of that!