Add Linux Support using D-Bus / bluez

[tthiery]

Methodology: https://stackoverflow.com/questions/53933345/utilizing-bluetooth-le-on-raspberry-pi-using-net-core

[rickjansen-dev]

I'm working on getting this library to work on a raspberry pi using bluez/dbus.
I do have some success, on a local version I can get it to connect to a technic hub with a L motor and get that to actually turn.

There's still some issues, the device is disconnected in most cases during value notification setup, discovery is also a bit fiddly. But once it gets connected and setup, it actually works. I'm still trying to figure out the cause of the issues.

Also, my current implementation uses https://github.com/hashtagchris/DotNet-BlueZ as a whole, but I found some issues that i'm trying to work around/fix. Since that package is a small and possibly incomplete abstraction layer on top of DBus anway, my goal is to implement this directly on top of Dbus (using Tmds.DBus) myself, after I get this working properly.

I just wanted to let you know i'm putting some effort into this currently and i'll create a PR once I have something that actually works all the time.

[tthiery]

Awesome. Create a draft pull request at any time.

When you need any help, just ask here!

One more question: how you learned about this library? I have many new contributors this week and I am curious what drove you here! 🙂

[rickjansen-dev]

I created a draft PR to track changes: #106

I found this library kind of by accident 🙂 I want to do some automation of my lego trains ever since I converted from 12v era stuff to powered up sometime last year or so. Recently I got inspired and wanted to connect my trains to some model train automation software (rocrail most likely). My end goal is to connect sensors to detect trains and servo's to control points to a raspberry pi which then communicate with rocrail or something else and i can control everything from within there. Now for the trains themselves, i figured i could use the same raspberry pi with bluetooth to control the trains.
So, since .net core works great on the raspberry and I really like the .net ecosystem it was kind of a no brainer to search for a C#/.NET powered up library. Found yours, seems to perfectly fit my needs.

There's no real reason for me to need to use a raspberry & linux for this, since i could just use my windows laptop instead, but that's only half the fun right 🤷‍♂️ 🙂

[tthiery]

That's cool. I would love some contributions in the train area. I am not so sure if this hub-device model is the right thing for trains. Maybe thinking different (like a huge array of devices (trains, rail switches...) with automatically managed BLE and Protocol layer below). Check #4 .

[rickjansen-dev]

I basically restarted implementing this, using DBus directly. It is much more reliable and I have a proper sense of what's going on now. I can get a working connection to a technic hub and run the gyro example succesfully every time.
Besides lots of refactoring (🙂), there's a couple of issues I need to figure out how to fix:

• Enabling change handlers on the service fails/hangs/task never completes when the device is discovering.
• Stopping discovery fails/hangs/task never completes. This is an issue because it is needed to work around above point
• Enabling discovery only raises events for devices that are never connected to before. Getting existing devices works in a different way

Regarding that last point, could you shed some light on exactly what the Discover() method on IPoweredUpBluetoothAdapter is supposed to do? It seems there's no seperation between discovering existing devices and new devices? I'm not sure how this works in the windows bluetooth stack, but in BlueZ there's a clear seperation. This also is kind of an issue, since it will automatically find existing devices, regardless if they are turned on or not, so it will try to connect immediately.
I think I could work around this by removing existing LWP devices before starting discovery and then start discovery and treat each device as a completely new one, but to me this seems a bit silly and a crude way of dealing with this (and will be slower too).

[tthiery]

I am also not perfectly happy with the API. Discover, find, ... To many interfaces, like you said.

I am open for a breaking change here and a new API surface.

My requirements would be:

• easy for beginners
• UI selection possible
• directly connect a known (typed) device without much scanning

[tthiery]

DiscoverAsync basically awaits the first of its type and otherwise invokes Discover with callback.

And what WinRT does... Only WinRT knows :) The Bluetooth interface is not too advanced there, it is after all over a generic driver over all kind of BT devices. Will have a look.

[rickjansen-dev]

So, I investigated further and figured out that there were deadlocks when awaiting a few specific methods related to connecting and watching property changes for a device.
To be more specific, calling .Wait() here results in deadlocks:

powered-up/src/SharpBrick.PoweredUp/PoweredUpHost.cs

Line 58 in 9bb4714

onDiscovery(hub).Wait();

To get stuff working, I changed the discovery handler passed to Discover() to a Func returning a Task so it can be awaited, see: https://github.com/vuurbeving/powered-up/blob/cca3e7f81e91f84b2e7b3d846ca7250627831dd8/src/SharpBrick.PoweredUp/Bluetooth/IPoweredUpBluetoothAdapter.cs#L9

The code in my fork now is a complete mess, but... it works (for a single specific test scenario that is). Exact same behavior every time also. I haven't tested the windows bluetooth stack with a async discovery handler yet, maybe that breaks 🙄.

Now to clean this up and get a proper bluez support for this library, I think maybe we should define a proper api surface first, like you mentioned. Due to the very async nature of bluez, dbus and well, bluetooth itself, I'd like the approach to be 100% async. Also we should take care to apply ConfigureAwait properly where needed. Especially when working with a UI.

The only issue I currently have is that for existing devices, using bluez I can get the manufacturerdata (even after the device is disconnected, off and not broadcasting). But after a reboot I cannot get the manufacturerdata, it will fail. So the device will be there as an existing device, but no manufacturer data can be retrieved, so I can't actually properly determine if it's a LWP device or not. To do this, I need to actually connect to the device.
So if I implement the bluez Discover method by returning existing devices first, and subsequently newly discovered devices, this will be an issue, since for an existing device, I do not know for sure if it's a LWP device. The only thing I know about a device at this point is a BT address and a name ('Technic Hub'). To me it seems bad practice to return a list of (could be many!) devices without even knowing if they are actual LWP devices or not. And the alternative to tell the user of the library to use his/her bluetooth adapter only for LWP devices is not acceptable either 😀
Nevermind this, I can get the service UUIDs, even after reboot. It should be safe to rely on this to determine whether a device is a LWP device or not. We do however probably need to be able to deal with non-existing manufacturer data.
Currently it will crash here:

powered-up/src/SharpBrick.PoweredUp/Hubs/HubFactory.cs

Line 18 in 9bb4714

hub.Configure(0x00);

[tthiery]

Regards Threading

I hate threading. To death ;). I investigated the threading behavior of WinRT by googling and documentation reading. NOTHING. The callback from the WinRT notification handler ... is that a task/thread for my pleasure (humble opinion: WinRT case) or part of the event loop (I guess dbus case) down to the driver.

• I agree, we need to define the surface API regards threading / async behavior. Can you work out a proposal for this specific case of the bluetooth adapter. I will then implement & test it for WinRT.
• ConfigureAwait. Hell, I forgot. So long in backend / logic land without SyncContexts. But you are right, prime UI logics are WPF and WinForms which actually have one. Let us file a separate issue and get it done for the whole library.

Regards the devices

• Detect if it is a LWP devices should be possible using scanning for service matches.
• For non-existing manufacturer data: Could not you use the existing device (with matching service pattern) and then do active discovery scan and await its feedback incl. the manufacturer data? And why not do always just active scanning w/o relying on previously know devices?

[rickjansen-dev]

I'll create a PR for the just the few specific changes needed for bluez/dbus to ever work/not deadlock, this can then be tested with WinRT stack, hopefully it simply works. The api surface

About ConfigureAwait, I guess it's best to create a seperate issue to track this, as currently I do not specifically need it right now.

I do already scan for devices with the lego specific service uuid as a filter, that works very well 👍.
The issue with the manufacturer data is simply due to the way bluez & the dbus integration work. But your questions got me thinking again (thanks! 😃) and I think I know a way around the issues, will test it tonight or so.

Basically the issue comes down to this:
On very first discovery, a device object is created by bluez, this emits a signal which I am listening to (it's is a signal InterfaceAdded emitted by the the adapter). If you scan afterwards and turn on the device, this signal will never be emitted again, because the device object already exists. This is by design and it is how it is supposed to work. The only way to ever get the InterfaceAdded signal again is by removing the device first.
During the scan, the device will send the manufacturer info and other properties though (I confirmed this using the bluetooth/dbus tooling but not in .net yet) and I should leverage this to listen for PropertyChanged signals coming from the device. Initially I assumed I would only need to listen to the property changed signal when actually connecting to the device and setting up, not during the discovery stage (which I wrongly assumed to be an adapter-only thing, event-wise).
It's all quite clear to me now and implementing it should be straightforward-ish.

[tthiery]

I merged #116. Was working like charm.

[justxi]

Hi, I created programs to control 42100, 42099, 42109 and to test 42114 with bricknil (janvrany fork) for Linux (Gentoo).
Now I wanted to create a program for the city train (60197) but the hub seems to disconnect while set up (happens only on the "Powered Up Hub" not on "Powered Up Technic Hub"). Most of the time I can start the motor or change the led color one time, before all stops. It seems that you had a similar problem(?).
Is there a short descriptions on how to install and run your sharpbrick patch for Linux?