InsteonPLM fails with pyserial version > 3.2.0

[rstanley75]

When I upgraded to the most recent Home Assistant 0.49.1 I started to get an error that insteon_plm could not be configured, with this in the home-assistant.log

2017-07-26 22:26:22 ERROR (MainThread) [homeassistant.setup] Error during setup of component insteon_plm
Traceback (most recent call last):
  File "/opt/hass/lib/python3.4/site-packages/homeassistant/setup.py", line 187, in _async_setup_component
    result = yield from component.async_setup(hass, processed_config)
  File "/opt/hass/lib/python3.4/site-packages/homeassistant/components/insteon_plm.py", line 44, in async_setup
    import insteonplm
  File "/home/hass/.homeassistant/deps/insteonplm/__init__.py", line 6, in <module>
    from .connection import Connection      # noqa: F401
  File "/home/hass/.homeassistant/deps/insteonplm/connection.py", line 5, in <module>
    import serial.aio
  File "/home/hass/.homeassistant/deps/serial/aio.py", line 20, in <module>
    import logger
ImportError: No module named 'logger'

I tried downgrading Home Assistant to 0.48.1 which was working previously but I kept getting the same error. In a moment of desperation I tried a fresh install of Home Assistant 0.48.1 in a new container to no avail.

So I started digging and found this in the changelog for pyserial (https://github.com/pyserial/pyserial/blob/master/CHANGES.rst)

Version 3.2.1 2016-10-14

Improvements:

remove serial.aio in favor of separate package, pyserial-asyncio

I don't know what the heck it means, except the error references aio.py. I tried pyserial v3.2.0:

pip3 install --upgrade pyserial==3.2.0 --target ~/.homeassistant/deps
hass -skip-pip

To my amazement insteon_plm started working again! To be sure I tried one iteration more recent:

pip3 install --upgrade pyserial==3.2.1 --target ~/.homeassistant/deps

insteon_plm failed with version 3.2.1.

Forcing the downgrade to pyserial 3.2.0 seems to be a valid workaround for now. This is where I'm lost, as I have no idea where to proceed from here to fix the issue in insteon_plm itself.

[larizzo]

Try install pyserial-asyncio package what I understand is that the serial.aio was split into a dfifferent package so doing this should solve the issue but I haven't had a chance to try:
pip3 install --upgrade pyserial==3.2.1 --target ~/.homeassistant/deps
pip3 install --upgrade pyserial-asyncio --target ~/.homeassistant/deps

basically the insteonplm package needs to add pyserial-asyncio as a dependency if that works.

[nugget]

This looks like a packaging problem in the pyserial project. I've opened an issue with that project here: pyserial/pyserial#254

There's a stale aio.py file in the pyserial-3.4 whl file which should not be there.

[lanrongwen]

Good thing I read this, I was about to update my install... Now I know what to look for when I get around to it. :)

[teharris1]

I upgraded pyserial-asyncio and pyserial per the direction provided by @nugget. Now I am getting an error that set_write_buffer_limits is NotImplementedError in /usr/lib/python3.5/asyncio/transports.py. I am running Debian with a virtualenv installation so I am not sure why this is even referencing /usr/lib/python3.5. Souldn't everything be self contained in /srv/homeassistant?

[j3kestrel]

To the original post/problem, I found this worked as a work-around: pip3 install logger

I am using the following patch for the set_write_buffer_limits fault:

--- protocol.py.orig    2017-07-29 17:44:52.454129414 -0400
+++ protocol.py 2017-07-29 17:52:07.422150061 -0400
@@ -218,9 +218,17 @@
         self.log.info('Connection established to PLM')
         self.transport = transport

-        self.transport.set_write_buffer_limits(128)
-        limit = self.transport.get_write_buffer_size()
-        self.log.debug('Write buffer size is %d', limit)
+        try:
+            self.transport.set_write_buffer_limits(128)
+        except NotImplementedError:
+            self.log.warning('Unable to set_write_buffer_limits().  Performanc\
e may be degraded.')
+
+        try:
+            limit = self.transport.get_write_buffer_size()
+            self.log.debug('Write buffer size is %d', limit)
+        except NotImplementedError:
+            self.log.debug('Unable to report get_write_buffer_size().')
+
         self.get_plm_info()
         self.load_all_link_database()

It is on my TODO list to submit a pull request for the above patch.

[nugget]

If you're seeing the "logger" error then the solution is not to install the logger package. All that does is mask the actual problem.

The key is downgrading pyserial to a version at or below 3.2.0. Anything newer than that contains a super-ancient file from back before pyserial split off into pyserial-asyncio. With the bugged versions of pyserial, the erroneous aio.py file supersedes the current pyserial-asyncio stuff which results in missing API methods (like get_write_buffer_size) which didn't exist back then. It also includes the typo trying to import a "logger" package that isn't actually needed (that was a bug in pyserial that has been fixed since November 2015).

[teharris1]

Yes, I now downgraded to 3.2.0 and it is all working. I did not implement the patch described by @j3kestrel

[j3kestrel]

Well, glad I commented before putting much more effort in to it! That might explain at least some of the un-reliability I've seen? It will be a few days before I know, but I'll go try to fix things up.

Thanks!

[j3kestrel]

I removed the logging package that I installed:
pip3 unintall logging
Restored the original copy of protocol.py and downgraded to pyserial 3.2.0:
pip3 install --upgrade pyserial==3.2.0

Seems to be working.

[rstanley75]

I'm still running into this error from time to time. Just today I had to downgrade pyserial using @j3kestrel's solution and all I had done was reboot.

[teharris1]

@rstanley75 Hope all is well. Not sure how this is happening since the insteonplm code specifically references 3.2.0. Might you have another module that references later version? Or might you have an auto update process running?

I asked the pyserial team again when they plan to update it.