pygbag within dockerized Python env

[andreagalle]

Since I planned to dev and run everything from a docker container alnoda-workspace this package would perfectly fits to make some amazing Python games without having to export the graphic session somehow.

After importing pygame & pygbag everything works fine if I am running the container from a Linux host, but not from a Windows one. The console on my Chrome browser shows that the server fails to retrieve some .js resources.

I am going to edit this post, to provide more information on how to reproduce this issue, what’s the error, but I would at least have a feedback on the possible scenario:

1. Does it requires a couple fixes on the pygbag side? Which modules should I focus to give my contribution?
2. Is it just a matter of properly using pygbag options, e.g. bind or the issue resides on the Docker side (deployed container and/or host machine)?

This SO post seems connected to this issue:

https://stackoverflow.com/questions/24319662/from-inside-of-a-docker-container-how-do-i-connect-to-the-localhost-of-the-mach

[pmp-p]

pygbag needs "localhost" address literally in the url bar to work correctly, a lot of conditionnal blocks depend on that. even 127.0.0.1 would not work correctly for getting .js / camera or ssl downgrades and with CORS restrictions that browser applies.

Maybe try pygame-embed as host-less alternative, see example here https://pygame-web.github.io/showroom/test_embed.html it can access files on-the-fly without need for packing

i don't use docker but it seems to me you need to redirect the port 8000 from the guest to 127.0.0.1:8000 on the host. ssh client can usually help doing that it has great tunnel creation abilities

[andreagalle]

Actually what I am doing is (from a Windows host):

• running the (dockerized) python workspace

PS C:\Users\andrea> docker run --name space-1 -d -p 8020-8040:8020-8040 alnoda/python-workspace

• going into the docker container

PS C:\Users\andrea> docker container exec -it space-1 bash

Then, from within the dockerized (ubuntu) python env:

• importing python modules, git cloning chess pygame project and pygbag-ging it!

abc@c2e1f93a3fac:/$ pip install pygame
abc@c2e1f93a3fac:/$ pip install pygbag --user --upgrade
abc@c2e1f93a3fac:/$ cd ../project && git clone https://github.com/pmp-p/pygame-pychess-wasm.git
abc@c2e1f93a3fac:/$ python -m pygbag --bind 172.17.0.3 --port 8030 pygame-pychess-wasm

Where 172.17.0.3 is the above docker container IP address I retrieved running ifconfig command within it.
Finally, the following output comes after the http://localhost:8030/ URL is entered in my 109.0.5414.75 (64 bit - Official Build) Chrome browser

8030...
...
...
packing 47 files complete

    building from local cached template https://pygame-web.github.io/archives/0.6/default.tmpl
    cached at /home/project/pygame-pychess-wasm/build/web-cache/fc90079097e20d74f2893615f8b9f12f.tmpl
    result files will be in /home/project/pygame-pychess-wasm/build/web


    using icon from local cached file https://pygame-web.github.io/archives/0.6/favicon.png
    cached at /home/project/pygame-pychess-wasm/build/web-cache/4e882f7a2cf19180a48bf1d703dfd4bd.png
Not using SSL
Serving HTTP on 172.17.0.3 port 8030 (http://172.17.0.3:8030/) ...
172.17.0.1 - - [16/Jan/2023 21:19:17] "GET / HTTP/1.1" 200 -
REPLACING /home/project/pygame-pychess-wasm/build/web/index.html https://pygame-web.github.io http://172.17.0.3:8030/

and these errors are shown within the console

Moreover, accessing the same resource from the browser, using localhost instead of the docker container IP address (as above) serves the required resource http://localhost:8030//archives/0.6/pythons.js as you can see ...

Contrary, running the above commands from a Linux host, the behaviour is completely different and everything (surprinsingly) works!

Both the http://localhost:8030/ and http://172.17.0.3:8030/ URLs redirect to the nedded .js and the example works as expected.

NB redirecting to port 8030 is necessary since, from within the dockerized python env, just couple ports (8030 and 8031) are exposed outside. However, this is unrelevant and port redirection doesn't seem an issue, right here.

[andreagalle]

@pmp-p I suppose I should focus on the testserver.py module, to make it handling requests to resources using localhost in place of the bind address passed as argument.
I double-checked the issue doesn't reside on the Docker side, since within the above dockerized python env running this dummy server, serves under http://localhost:8030/ the Hello World! HTML page as expected:

from http.server import HTTPServer, BaseHTTPRequestHandler

class RequestHandler(BaseHTTPRequestHandler):
    def _send_response(self, content):
        self.send_response(200)
        self.send_header('Content-type', 'text/html')
        self.end_headers()
        self.wfile.write(content)

    def do_GET(self):
        content = b'<html><body><h1>Hello, World!</h1></body></html>'
        self._send_response(content)

def run(server_class=HTTPServer, handler_class=RequestHandler, ip='172.17.0.3', port=8030):
    server_address = (ip, port)
    httpd = server_class(server_address, handler_class)
    print(f'Starting httpd on {ip}:{port}...')
    httpd.serve_forever()
    
run() # run as : python scriptname.py

[pmp-p]

if you bind the server to a special ip then it won't be bound to localhost anymore I don't think you should set it all all ( or try 0.0.0.0 instead) . This is not a really a pygbag parameter it's inherited from the python http test server. The "bind" pygbag uses is actually the --cdn parameter ( where it fetches from to fill cache ).

Regarding what address is sent to the browser, there's no parameter at all it (should) reuse the base url asked by browser and found in the headers => that's why in the screenshot the browser ask for files directly on 172.17.0.3 bypassing pygbag proxy.

http://172.17.0.3:8030/ is certainly not valid browser side : you must get it to ask everything on locahost:8030 so it does not make direct network requests and lower security barrier ( lots of things non-ssl are only tolerated on "localhost" based url)

[gabriel-v]

If it helps, I have a docker setup but instead of running the python web server, it runs nginx and an auto-reloader.

To use that repo: init git submodules, then ./build.sh then ./run.sh.

I use the --cdn argument to point to localhost:8000, so I also clone the archive files in the Docker image, and serve them with this nginx config.

If --host 0.0.0.0 does not work and you cannot figure out the docker vs. windows networking, you can make a static build, install nginx/apache on your windows machine directly, and have it serve the correct files.

[gabriel-v]

@andreagalle added powershell run example: https://github.com/gabriel-v/python-wasm-pygame-experiments/tree/7b9311d43c8e6b52c12247ea035dcc38c82fdf13

The container also builds pygbag from scratch - you can customize it by changing the fork under lib/pygbag

confirmed at a friend's windows machine the container works as expected.

the hot reloading only works on windows with the in-container code editor, as inotifywait does not trigger events on a windows folder mount.

[andreagalle]

Thanks a lot @gabriel-v , I'll try it out no later than tonight!

From what I can see you used another base image, different from https://hub.docker.com/r/alnoda/python-workspace : do you think that could be the issue?

I really liked that image, having a "complete" python workspace.

[gabriel-v]

that image is built from this dockerfile which is just debian + python

You can create your own image by changing that FROM, and also here and here to your own image repository (or comment out the docker push)

You probably want to remove the postgres-13 install from dockerfile (not used atm), and maybe try to simply install pygbag from pip instead of building it.

Or, you could start from the debian base image known to work, and install whatever you want from that other place here.

[gabriel-v]

The image you linked seems to be based on ubuntu 20 which probably doesn't have python3.11 in the container.

Through the building code, cmake and pygbag CLI might work with python 3.9 or lower - see if it works!

PS build takes ~45min on my machine

[andreagalle]

Thanks a lot for the additional details, I had a look at your Dockerfile and I'll try it out "as is" (wof... 45 mins! :P ).

Wonderful, thanks again for the hints on the python version needed by pygbag module.

Concerning the alnoda/python-workspace image I was talking about, this Docker image was meant to give the students a complete and "ready to go" python env without anything to install apart from pygame and pygbag.

If the problem resides on this Docker image I'll try to work it out!

[gabriel-v]

if the students will only be running code in the browser, then they will have those pre-installed for browser.

Also keep in mind anything you install in the docker image with pip install will not work in the browser - for that you only have the packages in archive list here.

So the students won't be using the zsh / other things packaged in your docker image.

Try out the browser editor i packaged in the image (see readme) and edit with it "src/test-hotreload/game.py" and see game code instantly reset itself - that would be cool for the students, right?

[gabriel-v]

In that VS Code they have git, terminals (bash and ssh) that run in the container and work wonderfully.

To run code in the browser, they either edit the code in src/* and it auto-refreshes, or they open some link with #debug and see the big black python shell from pygbag.

Either way, you can ask them to write browser code that interacts with the container flask server from here, or give them an API backend yourself and make multiplayer games.

See pygame-web/pkg-porting-wasm#5 for details on hot reload implementation - you can find examples of platform.fopen used to make HTTP calls and parse json

[andreagalle]

Thanks a lot @gabriel-v I have to read your suggestions more carefully. Then I would be happy to share with you what I've done so far.

Any other suggestion from you would be really appreciated!

[andreagalle]

I made it, it was just necessary to edit this line within the testserver.py module.

There I had to replace BPROXY with BMYPROXY defined as global within the run_code_server function given the "http://<host>:<port>" value, utf-8 encoded.

Running pygbag from within the container, the <host> value must be:

• localhost to access it from the host itself
• the host IP address to access it from another client on the same local network

always passing the container IP address to the pygbag --bind parameter as well as the desired --port parameter, that has to match the <port> above.

I will submit a pull request ASAP, to handle this case with a couple of additional parameters. What do you think @pmp-p ?

[pmp-p]

the host IP address to access it from another client on the same local network

it's probably usefull but i'm quite sure camera access will break if anything else than "localhost" is in url of end user and still on http+lan.

Also I still have trouble to figure out what docker is bringing for something that is 100% prebuilt and runs only client side. Do you have some schematics of what you are doing ?

if it is only about handling students files in a work session there are probably much simpler solutions.

[andreagalle]

Not it is just about having pygame working without having to install anything, but Docker. This way, relying on proper Docker images you don't have even to have installed python itself.
Thus, since Docker containers lacks of a GUI, it was necessary to make use of your wonderful pygbag module. This way you can develop and play your videogames using just a Browser (and Docker). Does it not make sense for you? Which kind of schematics are you willing for?

[andreagalle]

Moreover, yesterday, I was able to dev/run/play everything from the Browser only (without having to install even Docker) making use of GitHub codespaces (you will find more info here).

I had to slightly adjust your pygbag module, almost the same way. Don't you think it would turn usefull to have such a feature for pygbag as an option to pass through arguments?

[pmp-p]

Which kind of schematics are you willing for?

Well what part/use of pygbag really requires docker ? The python module was only made for packaging "final" product in a quick way.
I'd be interested to remove the need for docker if i can, maybe it's directly related to pygame-web/pkg-porting-wasm#3

I think people - not yet packaging for a game release - should just use a browser + online pygbag runtime ( currently hosted on github pages ) with codesandbox.io/github.dev or just some network/usb drive folder dropped on Chromium browers integrated source editor ( it works fine with python code ).
it is also really easy to add custom filesystems with browserfs.

my own testing zone on codesandbox https://codesandbox.io/s/happy-hooks-mb850n?file=/test-hg_fsurl.html that runs live without packaging.

pygbag - python module side - is really for localhost use when you want minimal network traffic while testing ( or eventually go offline) or access IoT with ws:// or camera/audio recording. Which are usefull but still unexplored corner cases.

[pmp-p]

future of pygbag is run from browser or wasi cmdline, not from docker.