Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Problems launching container on arm64 processor #504

Open
pavlis opened this issue Feb 6, 2024 · 1 comment
Open

Problems launching container on arm64 processor #504

pavlis opened this issue Feb 6, 2024 · 1 comment

Comments

@pavlis
Copy link
Collaborator

pavlis commented Feb 6, 2024

I encountered a serious error trying to launch the current container on an arm64 architecture machine (new Apple machine) in emulation mode. I have done the nearly same operation before when the container was built for arm64 and didn't have this problem.

For the record I launched the container with this long incantation because I was working with dask distributed:

docker run -p 8888:8888 -p 8787:8787 --platform linux/amd64 --mount src=`pwd`,target=/home,type=bind mspass/mspass

When I connect to jupyter in the usual way it is fine until I try to open a notebook. I then get this nasty error window pop up in the browser:

image

When I look at the terminal where I launched docker I get this that I think indicates what is failing:

[I 2024-02-06 12:44:19.493 ServerApp] Skipped non-installed server(s): bash-language-server, dockerfile-language-server-nodejs, javascript-typescript-langserver, jedi-language-server, julia-language-server, pyright, python-language-server, python-lsp-server, r-languageserver, sql-language-server, texlab, typescript-language-server, unified-language-server, vscode-css-languageserver-bin, vscode-html-languageserver-bin, vscode-json-languageserver-bin, yaml-language-server
[I 2024-02-06 12:44:54.632 LabApp] Build is up to date
[I 2024-02-06 12:45:08.624 ServerApp] Writing notebook-signing key to /data/db/.local/share/jupyter/notebook_secret
[W 2024-02-06 12:45:08.650 ServerApp] Notebook mspass_tutorial/notebooks/parallel_processing_basics.ipynb is not trusted
Operation not permitted (src/thread.cpp:281)
Operation not permitted (src/thread.cpp:281)
qemu: uncaught target signal 6 (Aborted) - core dumped
/usr/sbin/start-mspass.sh: line 38:    42 Aborted                 jupyter lab ${NOTEBOOK_ARGS}
Operation not permitted (src/thread.cpp:281)
qemu: uncaught target signal 6 (Aborted) - core dumped
Current Mongosh Log ID:	65c229d9a798bdf3193ff683
Connecting to:		mongodb://127.0.0.1:27017/admin?directConnection=true&serverSelectionTimeoutMS=2000&appName=mongosh+2.1.3
MongoNetworkError: connect ECONNREFUSED 127.0.0.1:27017

The mongo log shows it was definitely disconnected, but I'm not sure it is what caused the abort. That may just be a symptom.

@pavlis
Copy link
Collaborator Author

pavlis commented Feb 6, 2024

More evidence this is a MongoDB problem. Tried this on a different mac (my laptop) that also has an arm64 chip. Slightly different behavior but still a problem.

Container launched fine this time until I tried to connect to MongoDB in the usual way. Then I got the following (Note cut-and-paste took more than I expected for some unknown reason):

[Skip to left side bar](http://127.0.0.1:8888/lab/workspaces/auto-G/tree/getting_started.ipynb#)
>
/
Name
Last Modified

MsPASS Getting Started Tutorial
Gary L. Pavlis, Indiana University and Yinzhi (Ian) Wang, TACC
Preliminaries

This tutorial assumes you have already done the following:

    Installed docker.
    Run the commmand docker pull mspass/mspass
    In a terminal cd to the working directory where you saved this tutorial.
    Launched docker with this incantation:

  docker run -p 8888:8888 \
      --mount src=`pwd`,target=home,type=bind mspass/mspass

    Connected to the container to get this tutorial running as described in the User Manual.

Our user manual and github wiki pages give more details on how to do the above. Go there if you have any issues. This tutorial assumes that was completed and you are running this notebook while connected to MsPASS running within the docker container.
Download data with obspy
Overview of this section

MsPASS leans heavily on obspy. In particular, in this section we will use obspy's web services functions to download waveform data, station metadata, and source metadata. The approach we are using here is to stage these data to your local disk. The dataset we will assemble is the mainshock and 7 days of larger aftershocks of the Tohoku earthquake. The next section then covers how we import these data into the MsPASS framework to allow them to be processed.
Select, download, and save source data in MongoDB

As noted we are focusing on the Tohoku earthquake and its aftershocks. That earthquake's origin time is approximately March 11, 2011, at 5:46:24 UTC. The ISC epicenter is 38.30N, 142.50E. We will then apply obspy's get_events function with the following time and area filters:

    Starttime March 11, 2011, 1 hour before the origin time.
    End time 60 days after the mainshock origin time.
    Epicenters within + or - 3 degrees Latitude
    Epicenters within + or - 3 degrees of Longitude.
    Only aftershocks larger than 6.5

Here is the incantation in obspy to do that:
from obspy import UTCDateTime
from obspy.clients.fdsn import Client
client=Client("IRIS")
t0=UTCDateTime('2011-03-11T05:46:24.0')
starttime=t0-3600.0
endtime=t0+(60.0)*(24.0)*(3600.0)
lat0=38.3
lon0=142.5
minlat=lat0-3.0
maxlat=lat0+3.0
minlon=lon0-3.0
maxlon=lon0+3.0
minmag=6.5

cat=client.get_events(starttime=starttime,endtime=endtime,
        minlatitude=minlat,minlongitude=minlon,
        maxlatitude=maxlat,maxlongitude=maxlon,
        minmagnitude=minmag)
print(cat)

13 Event(s) in Catalog:
2011-04-11T08:16:12.690000Z | +36.953, +140.584 | 6.7  MW
2011-04-07T14:32:44.100000Z | +38.251, +141.730 | 7.1  MW
...
2011-03-11T05:51:20.500000Z | +37.310, +142.240 | 6.8  None
2011-03-11T05:46:23.200000Z | +38.296, +142.498 | 9.1  MW
To see all events call 'print(CatalogObject.__str__(print_all=True))'

We can save these easily into MongoDB for use in later processing with this simple command.
from mspasspy.db.database import Database
import mspasspy.client as msc
dbclient=msc.Client()
db = dbclient.get_database('getting_started')

<jemalloc>: MADV_DONTNEED does not work (memset will be used instead)
<jemalloc>: (This is the expected behaviour if you are running under QEMU)

That was eventually followed by a server timeout consistent with mongod crashing.

I suspect the reason the other notebook crashed immediately is that jupyter lab preserves state and I selecting a notebook that had both a mongodb and and dask distributed connection running when it was last closed. That, by the way, is yet another example of why we need a container that doesn't demand jupyter lab as the front end.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant