ssl.SSLWantReadError: The operation did not complete (read)

[FilipDem]

Describe the issue

I got regular ssl.SSLWantReadError on the get_vehicles().
I am using version 0.16.3, but got it also with other versions.
See traceback below.

Traceback (most recent call last):
  File "/home/pi/domoticz/plugins/Bmw/plugin.py", line 276, in handleTasks
    asyncio.run(self.myBMW.get_vehicles())
  File "/usr/lib/python3.9/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.9/asyncio/base_events.py", line 642, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.9/dist-packages/bimmer_connected/account.py", line 112, in get_vehicles
    await self._init_vehicles()
  File "/usr/local/lib/python3.9/dist-packages/bimmer_connected/account.py", line 84, in _init_vehicles
    vehicle_list_response = await client.post(
  File "/usr/local/lib/python3.9/dist-packages/httpx/_client.py", line 1892, in post
    return await self.request(
  File "/usr/local/lib/python3.9/dist-packages/httpx/_client.py", line 1574, in request
    return await self.send(request, auth=auth, follow_redirects=follow_redirects)
  File "/usr/local/lib/python3.9/dist-packages/httpx/_client.py", line 1661, in send
    response = await self._send_handling_auth(
  File "/usr/local/lib/python3.9/dist-packages/httpx/_client.py", line 1686, in _send_handling_auth
    request = await auth_flow.__anext__()
  File "/usr/local/lib/python3.9/dist-packages/bimmer_connected/api/authentication.py", line 85, in async_auth_flow
    await self.login()
  File "/usr/local/lib/python3.9/dist-packages/bimmer_connected/api/authentication.py", line 134, in login
    token_data = await self._login_row_na()
  File "/usr/local/lib/python3.9/dist-packages/bimmer_connected/api/authentication.py", line 156, in _login_row_na
    r_oauth_settings = await client.get(
  File "/usr/local/lib/python3.9/dist-packages/httpx/_client.py", line 1801, in get
    return await self.request(
  File "/usr/local/lib/python3.9/dist-packages/httpx/_client.py", line 1574, in request
    return await self.send(request, auth=auth, follow_redirects=follow_redirects)
  File "/usr/local/lib/python3.9/dist-packages/httpx/_client.py", line 1661, in send
    response = await self._send_handling_auth(
  File "/usr/local/lib/python3.9/dist-packages/httpx/_client.py", line 1689, in _send_handling_auth
    response = await self._send_handling_redirects(
  File "/usr/local/lib/python3.9/dist-packages/httpx/_client.py", line 1726, in _send_handling_redirects
    response = await self._send_single_request(request)
  File "/usr/local/lib/python3.9/dist-packages/httpx/_client.py", line 1763, in _send_single_request
    response = await transport.handle_async_request(request)
  File "/usr/local/lib/python3.9/dist-packages/httpx/_transports/default.py", line 373, in handle_async_request
    resp = await self._pool.handle_async_request(req)
  File "/usr/local/lib/python3.9/dist-packages/httpcore/_async/connection_pool.py", line 216, in handle_async_request
    raise exc from None
  File "/usr/local/lib/python3.9/dist-packages/httpcore/_async/connection_pool.py", line 196, in handle_async_request
    response = await connection.handle_async_request(
  File "/usr/local/lib/python3.9/dist-packages/httpcore/_async/connection.py", line 99, in handle_async_request
    raise exc
  File "/usr/local/lib/python3.9/dist-packages/httpcore/_async/connection.py", line 76, in handle_async_request
    stream = await self._connect(request)
  File "/usr/local/lib/python3.9/dist-packages/httpcore/_async/connection.py", line 154, in _connect
    stream = await stream.start_tls(**kwargs)
  File "/usr/local/lib/python3.9/dist-packages/httpcore/_backends/anyio.py", line 80, in start_tls
    raise exc
  File "/usr/local/lib/python3.9/dist-packages/httpcore/_backends/anyio.py", line 71, in start_tls
    ssl_stream = await anyio.streams.tls.TLSStream.wrap(
  File "/usr/local/lib/python3.9/dist-packages/anyio/streams/tls.py", line 132, in wrap
    await wrapper._call_sslobject_method(ssl_object.do_handshake)
  File "/usr/local/lib/python3.9/dist-packages/anyio/streams/tls.py", line 140, in _call_sslobject_method
    result = func(*args)
  File "/usr/lib/python3.9/ssl.py", line 944, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLWantReadError: The operation did not complete (read) (_ssl.c:1123)

Expected behavior

Which Home Assistant version are you using?

(using Domoticz)

What was the last working version of Home Assistant Core?

No response

What is your region?

Rest of world

MyBMW website

• I can still successfully login to the BMW MyBMW website and the car status is available there.
• I have MyBMW enabled for my vehicle.

Number of cars

• I have 2 or more cars linked to the MyBMW account.
• I have a Mini vehicle linked to my account.
• I have a Toyota Supra vehicle linked to my account.

Output of bimmer_connected fingerprint

No response

Anything in the logs that might be useful for us?

No response

Additional information

No response

[rikroe]

I guess this is either an issue of the BMW servers or at least your connectivity to them.

Nothing really I would know we can change. By default, the timeout is 5s (which should be plenty).

[FilipDem]

OK, thanks for the answers. It is not at my side... I have now build in retries...
I would have expected that the error was captured inside the get_vehicles(), but all fine & clear with your explanation.

[FilipDem]

I am still getting troubles and after investigating I might have found a track of the origin. As all was working nice a long time and now I am getting the problem, I migth be linked with an update to bullseye (and thus Python 3.9). I found back
python/cpython#88216 wrt to
ssl.SSLWantReadError.

This issue is a regression in Python 3.9. It was recently fixed in main/3.10, but I'm opening this issue to request that it is fixed in 3.9 as well since it breaks certain Python scripts running in WeeChat.

I have a C application which is using the Python/C API and is running multiple python subinterpreters. One of those subinterpreters is running a script which is reading data from a non-blocking ssl socket. When there is no more data to read, trying to read throws ssl.SSLWantReadError which is handled by the script. However, if a script in another subinterpreter imports _ssl, the SSLWantReadError exceptions thrown in the first script now have a different class instance, so they are not catched anymore.

As my Domoticz server is based on Bullseye (Python 3.9), would it be possible to have a workaround? I am willing to implement retries, but it seems that this exception destroys sth fundamentally so that a restart of the of the server is the only way out (which is not practical). So if it would be possible to catch this error it would be nice...

For more info about my implementation (that ran since a long time without any problem): Domoticz Plugin

Thanks

F

[rikroe]

Hi Filip,

I fear I have no clue how to work around this in the library. As we do not build our own HTTP connections but just reusing httpx, it's very far from me.
If this is a Python 3.9 issue, I would add an alternative/second python version to your distribution (so existing default python3.9 is not touched, but you can use python3.10).

See e.g. https://tiltingatwindmills.dev/how-to-install-an-alternate-python-version/.
I only would check to get the latest patch release and ensure to compile with configure --enable-optimizations.

Then run your scripts not via python3 script.py but python3.10 script.py.

Richard

[FilipDem]

The problem comes with the call to get_vehicles() as I am doing it in loop every 10 min. The problem is that a simple retry from my side doesn't work if I capture the problem... Only a complete reset works (even redoing first a MyBMWAccount fails always then). So I thought that there is some "bmw context" lost during the "crash" and if you could capture the problem yourself and make a retry possible, it could help (I can do the retry myself of course)... But not sure, you know better your code.

I also thought about the second python, but Domoticz uses an embedded python... It is not a simple execution of a script. So this makes it all more complex and I am afraid of side effects.