Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added support for Google´s Bumble Bluetooth Controller stack #1681

Draft
wants to merge 1 commit into
base: develop
Choose a base branch
from

Conversation

vChavezB
Copy link

@vChavezB vChavezB commented Nov 1, 2024

The backend supports direct use with Bumble. The HCI Controller is managed by the Bumble stack and the transport layer can be defined by the user (e.g. VHCI, Serial, TCP, android-netsim).

Use cases for this backend are:

  1. Bluetooth Functional tests without Hardware. Example of Bluetooth stacks that
    support virtualization are Android Emulator and Zephyr RTOS.
  2. Virtual connection between different HCI Controllers that are not in the same
    radio network (virtual or physical).

To enable this backend the env. variable BLEAK_BUMBLE must be set.

At the moment this is an experimental backend and would like to get some feedback from users and the devs of bleak :)

@vChavezB vChavezB marked this pull request as draft November 1, 2024 19:40
@vChavezB vChavezB force-pushed the bumble branch 11 times, most recently from b76d7f3 to 62411ce Compare November 1, 2024 22:41
@vChavezB
Copy link
Author

vChavezB commented Nov 1, 2024

Fixed some linting and doc issues.

@JPHutchins
Copy link
Contributor

It looks like you may not have updated the lock file. Please run poetry lock --no-update, commit, and push. thanks!

Copy link
Contributor

@JPHutchins JPHutchins left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks amazing!

Starting with a first round of type safety. While Bleak does not have 100% type coverage, I won't support any new python - in any context - that does not have 100% type coverage. That's just my opinion, and I'm happy to PR for it if it's too much of a lift.

I'll stay tuned to this and test it out on HW ASAP!

Comment on lines 13 to 15
def get_default_transport():
return "tcp-server:_:1234"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can it be:

def get_default_transport() -> str:
    return "tcp-server:_:1234"

or better:

def get_default_transport() -> Literal["tcp-server:_:1234"]:
    return "tcp-server:_:1234"

Why does it need a function?

from bumble.link import LocalLink
from bumble.transport import open_transport

transports = {}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What are these? Bumble has poor type safety, but it seems we can get some of it:

async def open_transport(name: str) -> Transport:
    """
    Open a transport by name.
    The name must be <type>:<metadata><parameters>
    Where <parameters> depend on the type (and may be empty for some types), and
    <metadata> is either omitted, or a ,-separated list of <key>=<value> pairs,
    enclosed in [].
    If there are not metadata or parameter, the : after the <type> may be omitted.
    Examples:
      * usb:0
      * usb:[driver=rtk]0
      * android-netsim

    The supported types are:
      * serial
      * udp
      * tcp-client
      * tcp-server
      * ws-client
      * ws-server
      * pty
      * file
      * vhci
      * hci-socket
      * usb
      * pyusb
      * android-emulator
      * android-netsim
    """

Likely this API should not use strings. Aren't these really enums?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Like why are these serialized to strings? OMG, it parses them:

scheme, *tail = name.split(':', 1)
    spec = tail[0] if tail else None
    metadata = None
    if spec:
        # Metadata may precede the spec
        if spec.startswith('['):
            metadata_str, *tail = spec[1:].split(']')
            spec = tail[0] if tail else None
            metadata = dict([entry.split('=') for entry in metadata_str.split(',')])

🤦‍♀️

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When Guido left Google he should have taken Python with him...

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think bumble uses directly strings. See here.

Copy link
Contributor

@JPHutchins JPHutchins Nov 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually there are hidden imports here which is going to be a problem for compiling binaries... lazy imports are nice until they aren't.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think bumble uses directly strings. See here.

Agreed - that API is bumming me out. It should deserialize to some well defined types and then pass those to open the transport. I see it was done this way to allow user input from the command line into this function.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking into this more, I don't seen any reason to use the open_transport function. Possibly can do away with the stuff in the __init__.py file entirely. Can the user be responsible to import the transport from Bumble and inject it themselves?

Copy link
Author

@vChavezB vChavezB Nov 2, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I checked the Transports from bumble and I am not sure its trivial. The implementation specific transports are defined locally inside the function that opens each transport.

Example for the TCP Server

https://github.com/google/bumble/blob/b78f8951430e8b4c0f9ef14818666dbede97b009/bumble/transport/tcp_server.py#L65

This requires a PR to the bumble project to allow users to access the Transport implementations or copy paste all transports and make them accessible while its not possible directly from the bumble project.

What I could do in the meantime is add enums to the strings of the transport with argument adapter and a separate argument string for the specific parameters that the transport needs.

from bumble.transport import open_transport

transports = {}
link = LocalLink()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know if these mutable objects are intended to export to the entire application. If they must be at file scope, then some safety can be added, for example:

_link: Final = LocalLink()

Comment on lines 30 to 32
def get_link():
# Assume all transports are linked
return link
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

def get_link() -> LocalLink:
    # Assume all transports are linked
    return link

I think I see that these getters are intended to protect the global object.

Comment on lines 17 to 28
async def start_transport(transport: str):
if transport not in transports.keys():
transports[transport] = await open_transport(transport)
Controller(
"ext",
host_source=transports[transport].source,
host_sink=transports[transport].sink,
link=link,
)

return transports[transport]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

async def start_transport(transport: str) -> Transport:

Comment on lines 85 to 107
async def disconnect(self) -> bool:
"""Disconnect from the specified GATT server.

Returns:
Boolean representing connection status.

"""
await self._dev.disconnect(
self._connection, HCI_REMOTE_USER_TERMINATED_CONNECTION_ERROR
)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

missing return

bleak/backends/bumble/client.py Show resolved Hide resolved
bleak/backends/bumble/client.py Show resolved Hide resolved
"""
descriptor = self.services.get_descriptor(handle)
if not descriptor:
raise BleakError("Descriptor {} was not found!".format(handle))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Prefer f strings

if not descriptor:
raise BleakError("Descriptor {} was not found!".format(handle))
val = await descriptor.obj.read_value()
logger.debug("Read Descriptor {0} : {1}".format(handle, val))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Prefer f-strings

@JPHutchins
Copy link
Contributor

Re: the overall API. If this is merged, I think that it should be importable as a separate class. BleakClient can remain as an OS-selected class, but BumbleClient should be separately importable. BumbleClient would be fully compatible with BleakClient - that is, they implement the same interface, Protocol, ABC, whatever you want to call it.

@vChavezB
Copy link
Author

vChavezB commented Nov 1, 2024

I agree, the Bumble client is largely OS independent (except for the transports that use the Linux HCI device). So it could be imported separately.

Thanks for the feedback on type safety! It really helps to remove ambiguity in Python and help other developers understand the intent of the code. I will apply the suggestions you mentioned.

@JPHutchins
Copy link
Contributor

JPHutchins commented Nov 1, 2024

I agree, the Bumble client is largely OS independent (except for the transports that use the Linux HCI device). So it could be imported separately.

Thanks for the feedback on type safety! It really helps to remove ambiguity in Python and help other developers understand the intent of the code. I will apply the suggestions you mentioned.

Glad you like it! I was annoyed when I first saw it being required in projects but now I'm a "kool-aid drinker" 🤣! It does in fact eliminate an entire class of runtime bugs that Python applications tend to suffer from.

You'll need mypy or pyright LSPs/extensions running to really see anything. By default, if a function has no return type specified, the linters ignore the typing in the function body. Best to simply specify a return type for all functions, even if it is None.

@vChavezB vChavezB force-pushed the bumble branch 2 times, most recently from dd907d4 to 7cf2739 Compare November 2, 2024 11:38
@vChavezB
Copy link
Author

vChavezB commented Nov 2, 2024

I have applied some of the suggestions from @JPHutchins. I will also check the typesafety of the changes in this PR.

@vChavezB vChavezB force-pushed the bumble branch 5 times, most recently from 7a1fc78 to d49fe6f Compare November 2, 2024 22:16
Copy link
Collaborator

@dlech dlech left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks really nice!

I think it will be useful for people who have devices that don't work so well with the various OS Bluetooth stacks. And it could be useful for providing a test backend for anyone that needs some serious testing.

I think that it should be importable as a separate class.

I disagree with this suggestion. I would like to be able to run any script with or without the Bumble backend, so no code changes should be required to use the Bumble backend. I also don't want a bunch of features being added that aren't supported on other backends, so having a separate class should not be necessary.

docs/backends/android.rst Show resolved Hide resolved
from bleak.backends.descriptor import BleakGATTDescriptor


class BleakGATTCharacteristicBumble(BleakGATTCharacteristic):
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've been hoping we could get rid of the subclasses of BleakGATTService, BleakGATTCharacteristic and BleakGATTDescriptor and just use that class directly everywhere since the subclasses are 90% duplicate code.

Not sure if that is something that is feasible to do before this PR or not.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently I put this PR in Draft to get some feedback. If you plan to refactor the classes I am open to rebase my fork and change the backend.

bleak/backends/bumble/client.py Outdated Show resolved Hide resolved
await self._dev.power_on()
await self._dev.connect(self.address)

self.services: BleakGATTServiceCollection = await self.get_services()
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If getting services fails, we should disconnect.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you mean by disconnect?

should I do wrap it around a try statement ?

or do you mean if the BLE device does not have any services we should disconnect?

Copy link
Author

@vChavezB vChavezB Nov 4, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did not find any similar logic in the backends that refer to check if the service fails. The only one is the WinRT where this is raised and is specific for Windows:

From bumble I always get a List of services, if not, then its empty and thus will have an empty BleakGATTServiceCollection


"""
await start_transport(self._adapter)
await self._dev.power_on()
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm... power isn't something Bleak has had to deal with before. Since up to now, we have only used OS APIs, it has been up to the OS to control power.

Likely this should be moved to a new adapter class. See #1060

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is because Bumble does not interact with the OS. It has its own HCI stack, so I have to "turn on" the Bumble BLE device, which internally resets the bumble HCI controller.

https://github.com/google/bumble/blob/b78f8951430e8b4c0f9ef14818666dbede97b009/bumble/device.py#L2199

Copy link
Author

@vChavezB vChavezB Nov 4, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If there is a BleakAdapter proposed I am glad to use it for this PR

bleak/backends/bumble/utils.py Outdated Show resolved Hide resolved

.. note::

To select this backend set the environmental variable ``BLEAK_BUMBLE``.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The environment variable should contain the transport info.

The idea is that users should just have to set the environment variable and use any Bleak script with the Bumble backend without having to modify any code.

Copy link
Author

@vChavezB vChavezB Nov 4, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My idea was that I could use bumble for functional testing with a virtual link. The user can then modify their code to select the appropiate transport.

Also what happens if the user wants to use both bumble and their physical OS HCI Controller.

Example:

  • Native HCI from Linux : hci0
  • HCI Serial controller that does not work well with Linux : bumble over serial

I opted for the bumble backend with an env. variable for tests only. it would be good to exactly define the use cases of bumble for bleak.

  • Do we accept bumble as a virtual transport to extend the limitations of OS drivers and hardware issues?
  • Should bumble cohabitate with the backend of each native OS ?
    • Use case for this is when you still want to use the native hardware from your OS and as mentioned a serial/USB controller.
  • Should bumble be flexible to use multiple transports?
    • This allows for example to connect an application running over Android simulator, a VHCI (linux), a serial HCI Controller and real hardware on the same network. This would for example enhance functional cross-platform testing of applications.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will add an env. variable to define use of bumble + transport options. In addition, I will just mention in the docs that if a specific backend must be selected then it can be changed over the BleakClient argument backend 👍

docs/backends/bumble.rst Show resolved Hide resolved
pyproject.toml Outdated
@@ -38,6 +38,7 @@ bleak-winrt = { version = "^1.2.0", markers = "platform_system=='Windows'", pyth
"winrt-Windows.Foundation.Collections" = { version = "^2", markers = "platform_system=='Windows'", python = ">=3.12" }
"winrt-Windows.Storage.Streams" = { version = "^2", markers = "platform_system=='Windows'", python = ">=3.12" }
dbus-fast = { version = ">=1.83.0, < 3", markers = "platform_system == 'Linux'" }
bumble = "^0.0.201"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this should be an optional dependency (i.e. only installed if pip install bleak[bumble]). Otherwise it is going to bring in a lot of other dependencies that most people don't need.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, seems fine by me. Then I would add this in the docs for the backend on how to install.

tests/bleak/backends/bumble/test_scanner.py Outdated Show resolved Hide resolved
@vChavezB
Copy link
Author

vChavezB commented Nov 4, 2024

Resolved some of the reviews pending is:

@vChavezB vChavezB force-pushed the bumble branch 2 times, most recently from 1fcba15 to e6c8182 Compare November 6, 2024 23:27
Copy link
Contributor

@JPHutchins JPHutchins left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good! Will be testing with an nrf52840dk momentarily!

Overall, a lot of time will be saved by getting mypy working in your editor.

@property
def service_uuid(self) -> str:
"""The uuid of the Service containing this characteristic"""
return self.__svc.uuid
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Type error, got UUID, expected str

Comment on lines 218 to 221
value = await self._peer.read_characteristics_by_uuid(characteristic.obj.uuid)
value = bytearray(value[0])
logger.debug(f"Read Characteristic {characteristic.uuid} : {value}")
return value
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, the peer returns a list of bytes, then we extract the first one and return it? What is in the rest of the list?

There are type errors here because value is mutated. Best to use a different variable name for the data that's extracted from the list of bytes.

bleak/backends/bumble/client.py Outdated Show resolved Hide resolved
bleak/backends/bumble/descriptor.py Outdated Show resolved Hide resolved
@property
def uuid(self) -> str:
"""UUID for this descriptor"""
return str(self.obj.uuid)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"DescriptorProxy" has no attribute "uuid"


device = self.create_or_update_device(
str(advertisement.address),
local_name,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Argument 2 to "create_or_update_device" of "BaseBleakScanner" has incompatible type "list[UUID] | tuple[UUID, bytes] | bytes | str | int | tuple[int, int] | tuple[int, bytes] | Appearance | None"; expected "str"

Comment on lines 88 to 90
for service_data in advertisement.data.get_all(AdvertisingData.SERVICE_DATA):
service_uuid, data = service_data
service_data[str(service_uuid)] = data
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Incompatible types in assignment (expression has type "list[UUID] | tuple[UUID, bytes] | bytes | str | int | tuple[int, int] | tuple[int, bytes] | Appearance", variable has type "dict[str, list[UUID] | tuple[UUID, bytes] | bytes | str | int | tuple[int, int] | tuple[int, bytes] | Appearance]")

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Service data expects dict[str, bytes, but the for loop mutates the dict and fills some of the values with strings. Is this intentional?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Most of the type errors can be resolved by avoiding the reuse of variable names for objects that are different types.


def __init__(self, obj: ServiceProxy):
super().__init__(obj)
self.__characteristics = []
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Replace all __ with _. Add type annotation.

Comment on lines 35 to 40
def add_characteristic(self, characteristic: CharacteristicProxy):
"""Add a :py:class:`~BleakGATTCharacteristicWinRT` to the service.

Should not be used by end user, but rather by `bleak` itself.
"""
self.__characteristics.append(characteristic)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Argument 1 of "add_characteristic" is incompatible with supertype "BleakGATTService"; supertype defines the argument type as "BleakGATTCharacteristic"

Comment on lines 7 to 23
"""
Convert a native Bumble UUID to a string representation.
Reason for this conversion is that the string representation (__str__) done in bumble
is not a standard UUID string representation.
In addition, the byte representation from data member uuid_bytes of bumble.core.UUID
is represented in reverse order.
Example:
Shortened 16 bit UUID: 0x1800Bumble UUID class
bumble UUID string representation: 'UUID-16:1800 (Generic Access)'
bumble UUID bytes representation: b'\x00\x18'
"""
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This docstring isn't getting parsed correctly, needs to be updated to conform to google spec.

Comment on lines 13 to 15
def __init__(self, obj: DescriptorProxy, characteristic: CharacteristicProxy):
super(BleakGATTDescriptorBumble, self).__init__(obj)
self.obj: Final = obj
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Update to:

    def __init__(self, obj: DescriptorProxy, characteristic: CharacteristicProxy):
        super().__init__(obj)

which is the same, AFAICT.

@JPHutchins
Copy link
Contributor

@vChavezB @dlech What do you think about committing FW images for some common platforms? If you'd rather not, we can create a repo that is solely for that.

@dlech
Copy link
Collaborator

dlech commented Nov 13, 2024

What do you think about committing FW images for some common platforms?

Sounds like a job for a separate repo.

@JPHutchins
Copy link
Contributor

Build HCI USB sample for NRF52840DK:

west build -b nrf52840dk/nrf52840 zephyr/samples/bluetooth/hci_usb

Flash

west flash

NRF52840DK device shows up!

image

Confirmed poetry run python -m examples.discover running normally on the regular BT adapter. Now will set env variables to test the Bumble backend!

$env:BLEAK_BUMBLE="usb:2FE3:0008"
poetry run python -m examples.discover
scanning for 5 seconds, please wait...
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "C:\Users\jp\repos\bleak\examples\discover.py", line 51, in <module>
    asyncio.run(main(args))
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.12_3.12.2032.0_x64__qbz5n2kfra8p0\Lib\asyncio\runners.py", line 194, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.12_3.12.2032.0_x64__qbz5n2kfra8p0\Lib\asyncio\runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.12_3.12.2032.0_x64__qbz5n2kfra8p0\Lib\asyncio\base_events.py", line 687, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "C:\Users\jp\repos\bleak\examples\discover.py", line 20, in main
    devices = await BleakScanner.discover(
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\jp\repos\bleak\bleak\__init__.py", line 320, in discover
    async with cls(**kwargs) as scanner:
               ^^^^^^^^^^^^^
  File "C:\Users\jp\repos\bleak\bleak\__init__.py", line 158, in __aenter__
    await self._backend.start()
  File "C:\Users\jp\repos\bleak\bleak\backends\bumble\scanner.py", line 115, in start
    await start_transport(self._adapter)
  File "C:\Users\jp\repos\bleak\bleak\backends\bumble\__init__.py", line 112, in start_transport
    transports[transport_cmd] = await open_transport(transport_cmd)
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\jp\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\Local\pypoetry\Cache\virtualenvs\bleak-dSQ9Yv1H-py3.12\Lib\site-packages\bumble\transport\__init__.py", line 93, in open_transport
    transport = await _open_transport(scheme, spec)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\jp\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\Local\pypoetry\Cache\virtualenvs\bleak-dSQ9Yv1H-py3.12\Lib\site-packages\bumble\transport\__init__.py", line 165, in _open_transport
    return await open_usb_transport(spec)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\jp\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\Local\pypoetry\Cache\virtualenvs\bleak-dSQ9Yv1H-py3.12\Lib\site-packages\bumble\transport\usb.py", line 456, in open_usb_transport
    raise TransportInitError('device not found')
bumble.transport.common.TransportInitError: device not found

Tried lower case hex for VID/PID as well.

Reference is here: https://google.github.io/bumble/transports/usb.html

OK, updated to:

$env:BLEAK_BUMBLE="usb:0"

And now I get:

scanning for 5 seconds, please wait...
!!! failed to open USB device: LIBUSB_ERROR_NOT_SUPPORTED [-12]
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "C:\Users\jp\repos\bleak\examples\discover.py", line 51, in <module>
    asyncio.run(main(args))
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.12_3.12.2032.0_x64__qbz5n2kfra8p0\Lib\asyncio\runners.py", line 194, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.12_3.12.2032.0_x64__qbz5n2kfra8p0\Lib\asyncio\runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.12_3.12.2032.0_x64__qbz5n2kfra8p0\Lib\asyncio\base_events.py", line 687, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "C:\Users\jp\repos\bleak\examples\discover.py", line 20, in main
    devices = await BleakScanner.discover(
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\jp\repos\bleak\bleak\__init__.py", line 320, in discover
    async with cls(**kwargs) as scanner:
               ^^^^^^^^^^^^^
  File "C:\Users\jp\repos\bleak\bleak\__init__.py", line 158, in __aenter__
    await self._backend.start()
  File "C:\Users\jp\repos\bleak\bleak\backends\bumble\scanner.py", line 115, in start
    await start_transport(self._adapter)
  File "C:\Users\jp\repos\bleak\bleak\backends\bumble\__init__.py", line 112, in start_transport
    transports[transport_cmd] = await open_transport(transport_cmd)
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\jp\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\Local\pypoetry\Cache\virtualenvs\bleak-dSQ9Yv1H-py3.12\Lib\site-packages\bumble\transport\__init__.py", line 93, in open_transport
    transport = await _open_transport(scheme, spec)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\jp\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\Local\pypoetry\Cache\virtualenvs\bleak-dSQ9Yv1H-py3.12\Lib\site-packages\bumble\transport\__init__.py", line 165, in _open_transport
    return await open_usb_transport(spec)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\jp\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\Local\pypoetry\Cache\virtualenvs\bleak-dSQ9Yv1H-py3.12\Lib\site-packages\bumble\transport\usb.py", line 536, in open_usb_transport
    device = found.open()
             ^^^^^^^^^^^^
  File "C:\Users\jp\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\Local\pypoetry\Cache\virtualenvs\bleak-dSQ9Yv1H-py3.12\Lib\site-packages\usb1\__init__.py", line 2055, in open
    mayRaiseUSBError(libusb1.libusb_open(self.device_p, byref(handle)))
  File "C:\Users\jp\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\Local\pypoetry\Cache\virtualenvs\bleak-dSQ9Yv1H-py3.12\Lib\site-packages\usb1\__init__.py", line 127, in mayRaiseUSBError
    __raiseUSBError(value)
  File "C:\Users\jp\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\Local\pypoetry\Cache\virtualenvs\bleak-dSQ9Yv1H-py3.12\Lib\site-packages\usb1\__init__.py", line 119, in raiseUSBError
    raise __STATUS_TO_EXCEPTION_DICT.get(value, __USBError)(value)
usb1.USBErrorNotSupported: LIBUSB_ERROR_NOT_SUPPORTED [-12]

Installing Zadig 🤣 ...

Tried libusb-win32 driver - discover script hangs until I unplug the board.

Tried libusbK driver - same

Tried WinUSB - same

Could test from linux first, for example, then try Windows.

@vChavezB
Copy link
Author

vChavezB commented Nov 13, 2024

Looking good! Will be testing with an nrf52840dk momentarily!

Overall, a lot of time will be saved by getting mypy working in your editor.

Thanks for the review. I will definitely check again with the typesafety extensions in pycharm. It would be nice to get some field tests to check if everything works.

So far I have only tested with virtual devices (Zephyr posix and renode).

@JPHutchins
Copy link
Contributor

Tried lower case hex for VID/PID as well.

Would you believe that the PID is 000B but I'd set 0008 every time? 😭. Will run through again ASAP.

@JPHutchins
Copy link
Contributor

JPHutchins commented Nov 17, 2024

Would you believe that the PID is 000B but I'd set 0008 every time? 😭. Will run through again ASAP.

OK, picking up where I left off. Starting by trying to revert the drivers to Windows defaults. It's on libwdi now, after unisntalling and restart.

Set logging and correct USB VID:

$env:BLEAK_LOGGING=1
$env:BLEAK_BUMBLE="usb:2fe3:000B"
poetry run python -m examples.discover
scanning for 5 seconds, please wait...

Still hang. No Ctrl-C. At least it's find the USB device now. Can exit by removing the Bumble device.

Call stack reveals it's maybe doing some USB stuff (I didn't include whole stack)

!!! IN transfer not completed: status=4
Exception ignored on calling ctypes callback function: <bound method USBTransfer.__callbackWrapper of <class 'usb1.USBTransfer'>>
Traceback (most recent call last):
  File "C:\Users\jp\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\Local\pypoetry\Cache\virtualenvs\bleak-dSQ9Yv1H-py3.12\Lib\site-packages\usb1\__init__.py", line 327, in __callbackWrapper
    callback(self)
  File "C:\Users\jp\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\Local\pypoetry\Cache\virtualenvs\bleak-dSQ9Yv1H-py3.12\Lib\site-packages\bumble\transport\usb.py", line 280, in transfer_callback
    self.loop.call_soon_threadsafe(self.on_transport_lost)
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.12_3.12.2032.0_x64__qbz5n2kfra8p0\Lib\asyncio\base_events.py", line 840, in call_soon_threadsafe
    self._check_closed()
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.12_3.12.2032.0_x64__qbz5n2kfra8p0\Lib\asyncio\base_events.py", line 541, in _check_closed
    raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed
!!! IN transfer not completed: status=5
Exception ignored on calling ctypes callback function: <bound method USBTransfer.__callbackWrapper of <class 'usb1.USBTransfer'>>
Traceback (most recent call last):
  File "C:\Users\jp\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\Local\pypoetry\Cache\virtualenvs\bleak-dSQ9Yv1H-py3.12\Lib\site-packages\usb1\__init__.py", line 327, in __callbackWrapper
    callback(self)
  File "C:\Users\jp\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\Local\pypoetry\Cache\virtualenvs\bleak-dSQ9Yv1H-py3.12\Lib\site-packages\bumble\transport\usb.py", line 280, in transfer_callback
    self.loop.call_soon_threadsafe(self.on_transport_lost)
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.12_3.12.2032.0_x64__qbz5n2kfra8p0\Lib\asyncio\base_events.py", line 840, in call_soon_threadsafe
    self._check_closed()
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.12_3.12.2032.0_x64__qbz5n2kfra8p0\Lib\asyncio\base_events.py", line 541, in _check_closed
    raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed
Exception in thread Thread-1 (run):
Traceback (most recent call last):
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.12_3.12.2032.0_x64__qbz5n2kfra8p0\Lib\threading.py", line 1075, in _bootstrap_inner
    self.run()

So... maybe it's blocking on some USB transfer that never happens.

Adding logging to examples.discover:

logging.basicConfig(level=logging.DEBUG)

I mean... this looks promising!

poetry run python -m examples.discover
DEBUG:asyncio:Using proactor: IocpProactor
scanning for 5 seconds, please wait...
DEBUG:bumble.link:new controller: <bumble.controller.Controller object at 0x0000029E537B7EF0>
DEBUG:bumble.transport.usb:USB Device: Bus 004 Device 019: ID 2fe3:000b
DEBUG:bumble.transport.usb:selected endpoints: configuration=1, interface=0, setting=0, acl_in=0x82, acl_out=0x01, events_in=0x81,
DEBUG:bumble.transport.usb:current configuration = 1
DEBUG:bumble.transport.usb:starting USB event loop
DEBUG:bumble.link:new controller: <bumble.controller.Controller object at 0x0000029E532F6570>
DEBUG:bumble.drivers:Probing driver class: rtk
DEBUG:bumble.drivers.rtk:USB metadata not found
DEBUG:bumble.drivers:Probing driver class: intel
DEBUG:bumble.drivers.intel:USB metadata not sufficient
DEBUG:bumble.host:### HOST -> CONTROLLER: HCI_RESET_COMMAND
DEBUG:bumble.controller:<<< [Scanner] HOST -> CONTROLLER: HCI_RESET_COMMAND
DEBUG:bumble.controller:>>> [Scanner] CONTROLLER -> HOST: HCI_COMMAND_COMPLETE_EVENT:
  num_hci_command_packets: 1
  command_opcode:          HCI_RESET_COMMAND
  return_parameters:       HCI_SUCCESS
DEBUG:bumble.host:### CONTROLLER -> HOST: HCI_COMMAND_COMPLETE_EVENT:
  num_hci_command_packets: 1
  command_opcode:          HCI_RESET_COMMAND
  return_parameters:       HCI_SUCCESS
DEBUG:bumble.host:### HOST -> CONTROLLER: HCI_READ_LOCAL_SUPPORTED_COMMANDS_COMMAND
DEBUG:bumble.controller:<<< [Scanner] HOST -> CONTROLLER: HCI_READ_LOCAL_SUPPORTED_COMMANDS_COMMAND
DEBUG:bumble.controller:>>> [Scanner] CONTROLLER -> HOST: HCI_COMMAND_COMPLETE_EVENT:
  num_hci_command_packets: 1
  command_opcode:          HCI_READ_LOCAL_SUPPORTED_COMMANDS_COMMAND
  return_parameters:       002000800000c000000000e4000000a822000000000000040000f7ffff7f00000030f0f9ff01008004000000000000000000000000000000000000000000000000
DEBUG:bumble.host:### CONTROLLER -> HOST: HCI_COMMAND_COMPLETE_EVENT:
  num_hci_command_packets: 1
  command_opcode:          HCI_READ_LOCAL_SUPPORTED_COMMANDS_COMMAND
  return_parameters:
    status:             HCI_SUCCESS
    supported_commands: 2000800000c000000000e4000000a822000000000000040000f7ffff7f00000030f0f9ff01008004000000000000000000000000000000000000000000000000
DEBUG:bumble.host:### HOST -> CONTROLLER: HCI_LE_READ_LOCAL_SUPPORTED_FEATURES_COMMAND
DEBUG:bumble.controller:<<< [Scanner] HOST -> CONTROLLER: HCI_LE_READ_LOCAL_SUPPORTED_FEATURES_COMMAND
DEBUG:bumble.controller:>>> [Scanner] CONTROLLER -> HOST: HCI_COMMAND_COMPLETE_EVENT:
  num_hci_command_packets: 1
  command_opcode:          HCI_LE_READ_LOCAL_SUPPORTED_FEATURES_COMMAND
  return_parameters:       00ff49010000000000
DEBUG:bumble.host:### CONTROLLER -> HOST: HCI_COMMAND_COMPLETE_EVENT:
  num_hci_command_packets: 1
  command_opcode:          HCI_LE_READ_LOCAL_SUPPORTED_FEATURES_COMMAND
  return_parameters:
    status:      HCI_SUCCESS
    le_features: ff49010000000000
DEBUG:bumble.host:### HOST -> CONTROLLER: HCI_READ_LOCAL_VERSION_INFORMATION_COMMAND
DEBUG:bumble.controller:<<< [Scanner] HOST -> CONTROLLER: HCI_READ_LOCAL_VERSION_INFORMATION_COMMAND
DEBUG:bumble.controller:>>> [Scanner] CONTROLLER -> HOST: HCI_COMMAND_COMPLETE_EVENT:
  num_hci_command_packets: 1
  command_opcode:          HCI_READ_LOCAL_VERSION_INFORMATION_COMMAND
  return_parameters:       0009000009ffff0000
DEBUG:bumble.host:### CONTROLLER -> HOST: HCI_COMMAND_COMPLETE_EVENT:
  num_hci_command_packets: 1
  command_opcode:          HCI_READ_LOCAL_VERSION_INFORMATION_COMMAND
  return_parameters:
    status:             HCI_SUCCESS
    hci_version:        9
    hci_subversion:     0
    lmp_version:        9
    company_identifier: 65535
    lmp_subversion:     0
DEBUG:bumble.host:### HOST -> CONTROLLER: HCI_READ_LOCAL_SUPPORTED_FEATURES_COMMAND
DEBUG:bumble.controller:<<< [Scanner] HOST -> CONTROLLER: HCI_READ_LOCAL_SUPPORTED_FEATURES_COMMAND
DEBUG:bumble.controller:>>> [Scanner] CONTROLLER -> HOST: HCI_COMMAND_COMPLETE_EVENT:
  num_hci_command_packets: 1
  command_opcode:          HCI_READ_LOCAL_SUPPORTED_FEATURES_COMMAND
  return_parameters:       000000000060000000
DEBUG:bumble.host:### CONTROLLER -> HOST: HCI_COMMAND_COMPLETE_EVENT:
  num_hci_command_packets: 1
  command_opcode:          HCI_READ_LOCAL_SUPPORTED_FEATURES_COMMAND
  return_parameters:
    status:       HCI_SUCCESS
    lmp_features: 0000000060000000
DEBUG:bumble.host:### HOST -> CONTROLLER: HCI_SET_EVENT_MASK_COMMAND:
  event_mask: ff9ffbbf07f8bf3d
DEBUG:bumble.controller:<<< [Scanner] HOST -> CONTROLLER: HCI_SET_EVENT_MASK_COMMAND:
  event_mask: ff9ffbbf07f8bf3d
DEBUG:bumble.controller:>>> [Scanner] CONTROLLER -> HOST: HCI_COMMAND_COMPLETE_EVENT:
  num_hci_command_packets: 1
  command_opcode:          HCI_SET_EVENT_MASK_COMMAND
  return_parameters:       HCI_SUCCESS
DEBUG:bumble.host:### CONTROLLER -> HOST: HCI_COMMAND_COMPLETE_EVENT:
  num_hci_command_packets: 1
  command_opcode:          HCI_SET_EVENT_MASK_COMMAND
  return_parameters:       HCI_SUCCESS
DEBUG:bumble.host:### HOST -> CONTROLLER: HCI_LE_SET_EVENT_MASK_COMMAND:
  le_event_mask: fffff7ff07000000
DEBUG:bumble.controller:<<< [Scanner] HOST -> CONTROLLER: HCI_LE_SET_EVENT_MASK_COMMAND:
  le_event_mask: fffff7ff07000000
DEBUG:bumble.controller:>>> [Scanner] CONTROLLER -> HOST: HCI_COMMAND_COMPLETE_EVENT:
  num_hci_command_packets: 1
  command_opcode:          HCI_LE_SET_EVENT_MASK_COMMAND
  return_parameters:       HCI_SUCCESS
DEBUG:bumble.host:### CONTROLLER -> HOST: HCI_COMMAND_COMPLETE_EVENT:
  num_hci_command_packets: 1
  command_opcode:          HCI_LE_SET_EVENT_MASK_COMMAND
  return_parameters:       HCI_SUCCESS
DEBUG:bumble.host:### HOST -> CONTROLLER: HCI_READ_BUFFER_SIZE_COMMAND
DEBUG:bumble.controller:<<< [Scanner] HOST -> CONTROLLER: HCI_READ_BUFFER_SIZE_COMMAND
DEBUG:bumble.controller:>>> [Scanner] CONTROLLER -> HOST: HCI_COMMAND_COMPLETE_EVENT:
  num_hci_command_packets: 1
  command_opcode:          HCI_READ_BUFFER_SIZE_COMMAND
  return_parameters:       001b000040000000
DEBUG:bumble.host:### CONTROLLER -> HOST: HCI_COMMAND_COMPLETE_EVENT:
  num_hci_command_packets: 1
  command_opcode:          HCI_READ_BUFFER_SIZE_COMMAND
  return_parameters:
    status:                                HCI_SUCCESS
    hc_acl_data_packet_length:             27
    hc_synchronous_data_packet_length:     0
    hc_total_num_acl_data_packets:         64
    hc_total_num_synchronous_data_packets: 0
DEBUG:bumble.host:HCI ACL flow control: hc_acl_data_packet_length=27,hc_total_num_acl_data_packets=64
DEBUG:bumble.host:### HOST -> CONTROLLER: HCI_LE_READ_BUFFER_SIZE_COMMAND
DEBUG:bumble.controller:<<< [Scanner] HOST -> CONTROLLER: HCI_LE_READ_BUFFER_SIZE_COMMAND
DEBUG:bumble.controller:>>> [Scanner] CONTROLLER -> HOST: HCI_COMMAND_COMPLETE_EVENT:
  num_hci_command_packets: 1
  command_opcode:          HCI_LE_READ_BUFFER_SIZE_COMMAND
  return_parameters:       001b0040
DEBUG:bumble.host:### CONTROLLER -> HOST: HCI_COMMAND_COMPLETE_EVENT:
  num_hci_command_packets: 1
  command_opcode:          HCI_LE_READ_BUFFER_SIZE_COMMAND
  return_parameters:
    status:                           HCI_SUCCESS
    hc_le_acl_data_packet_length:     27
    hc_total_num_le_acl_data_packets: 64
DEBUG:bumble.host:HCI LE ACL flow control: hc_le_acl_data_packet_length=27,hc_total_num_le_acl_data_packets=64
DEBUG:bumble.host:### HOST -> CONTROLLER: HCI_LE_READ_SUGGESTED_DEFAULT_DATA_LENGTH_COMMAND
DEBUG:bumble.controller:<<< [Scanner] HOST -> CONTROLLER: HCI_LE_READ_SUGGESTED_DEFAULT_DATA_LENGTH_COMMAND
DEBUG:bumble.controller:>>> [Scanner] CONTROLLER -> HOST: HCI_COMMAND_COMPLETE_EVENT:
  num_hci_command_packets: 1
  command_opcode:          HCI_LE_READ_SUGGESTED_DEFAULT_DATA_LENGTH_COMMAND
  return_parameters:       001b004801
DEBUG:bumble.host:### CONTROLLER -> HOST: HCI_COMMAND_COMPLETE_EVENT:
  num_hci_command_packets: 1
  command_opcode:          HCI_LE_READ_SUGGESTED_DEFAULT_DATA_LENGTH_COMMAND
  return_parameters:
    status:                  HCI_SUCCESS
    suggested_max_tx_octets: 27
    suggested_max_tx_time:   328
DEBUG:bumble.host:### HOST -> CONTROLLER: HCI_LE_WRITE_SUGGESTED_DEFAULT_DATA_LENGTH_COMMAND:
  suggested_max_tx_octets: 251
  suggested_max_tx_time:   2120
DEBUG:bumble.controller:<<< [Scanner] HOST -> CONTROLLER: HCI_LE_WRITE_SUGGESTED_DEFAULT_DATA_LENGTH_COMMAND:
  suggested_max_tx_octets: 251
  suggested_max_tx_time:   2120
DEBUG:bumble.controller:>>> [Scanner] CONTROLLER -> HOST: HCI_COMMAND_COMPLETE_EVENT:
  num_hci_command_packets: 1
  command_opcode:          HCI_LE_WRITE_SUGGESTED_DEFAULT_DATA_LENGTH_COMMAND
  return_parameters:       HCI_SUCCESS
DEBUG:bumble.host:### CONTROLLER -> HOST: HCI_COMMAND_COMPLETE_EVENT:
  num_hci_command_packets: 1
  command_opcode:          HCI_LE_WRITE_SUGGESTED_DEFAULT_DATA_LENGTH_COMMAND
  return_parameters:       HCI_SUCCESS
DEBUG:bumble.host:### HOST -> CONTROLLER: HCI_READ_BD_ADDR_COMMAND
DEBUG:bumble.controller:<<< [Scanner] HOST -> CONTROLLER: HCI_READ_BD_ADDR_COMMAND
DEBUG:bumble.controller:>>> [Scanner] CONTROLLER -> HOST: HCI_COMMAND_COMPLETE_EVENT:
  num_hci_command_packets: 1
  command_opcode:          HCI_READ_BD_ADDR_COMMAND
  return_parameters:       00000000000000
DEBUG:bumble.host:### CONTROLLER -> HOST: HCI_COMMAND_COMPLETE_EVENT:
  num_hci_command_packets: 1
  command_opcode:          HCI_READ_BD_ADDR_COMMAND
  return_parameters:
    status:  HCI_SUCCESS
    bd_addr: 00:00:00:00:00:00/P
DEBUG:bumble.device:BD_ADDR: 00:00:00:00:00:00/P
DEBUG:bumble.device:LE Random Address: F0:F1:F2:F3:F4:F5
DEBUG:bumble.host:### HOST -> CONTROLLER: HCI_LE_SET_RANDOM_ADDRESS_COMMAND:
  random_address: F0:F1:F2:F3:F4:F5
DEBUG:bumble.controller:<<< [Scanner] HOST -> CONTROLLER: HCI_LE_SET_RANDOM_ADDRESS_COMMAND:
  random_address: F0:F1:F2:F3:F4:F5
DEBUG:bumble.controller:new random address: F0:F1:F2:F3:F4:F5
DEBUG:bumble.controller:>>> [Scanner] CONTROLLER -> HOST: HCI_COMMAND_COMPLETE_EVENT:
  num_hci_command_packets: 1
  command_opcode:          HCI_LE_SET_RANDOM_ADDRESS_COMMAND
  return_parameters:       HCI_SUCCESS
DEBUG:bumble.host:### CONTROLLER -> HOST: HCI_COMMAND_COMPLETE_EVENT:
  num_hci_command_packets: 1
  command_opcode:          HCI_LE_SET_RANDOM_ADDRESS_COMMAND
  return_parameters:       HCI_SUCCESS
DEBUG:bumble.host:### HOST -> CONTROLLER: HCI_LE_SET_SCAN_PARAMETERS_COMMAND:
  le_scan_type:           1
  le_scan_interval:       96
  le_scan_window:         96
  own_address_type:       RANDOM
  scanning_filter_policy: 0
DEBUG:bumble.controller:<<< [Scanner] HOST -> CONTROLLER: HCI_LE_SET_SCAN_PARAMETERS_COMMAND:
  le_scan_type:           1
  le_scan_interval:       96
  le_scan_window:         96
  own_address_type:       RANDOM
  scanning_filter_policy: 0
DEBUG:bumble.controller:>>> [Scanner] CONTROLLER -> HOST: HCI_COMMAND_COMPLETE_EVENT:
  num_hci_command_packets: 1
  command_opcode:          HCI_LE_SET_SCAN_PARAMETERS_COMMAND
  return_parameters:       HCI_SUCCESS
DEBUG:bumble.host:### CONTROLLER -> HOST: HCI_COMMAND_COMPLETE_EVENT:
  num_hci_command_packets: 1
  command_opcode:          HCI_LE_SET_SCAN_PARAMETERS_COMMAND
  return_parameters:       HCI_SUCCESS
DEBUG:bumble.host:### HOST -> CONTROLLER: HCI_LE_SET_SCAN_ENABLE_COMMAND:
  le_scan_enable:    1
  filter_duplicates: 0
DEBUG:bumble.controller:<<< [Scanner] HOST -> CONTROLLER: HCI_LE_SET_SCAN_ENABLE_COMMAND:
  le_scan_enable:    1
  filter_duplicates: 0
DEBUG:bumble.controller:>>> [Scanner] CONTROLLER -> HOST: HCI_COMMAND_COMPLETE_EVENT:
  num_hci_command_packets: 1
  command_opcode:          HCI_LE_SET_SCAN_ENABLE_COMMAND
  return_parameters:       HCI_SUCCESS
DEBUG:bumble.host:### CONTROLLER -> HOST: HCI_COMMAND_COMPLETE_EVENT:
  num_hci_command_packets: 1
  command_opcode:          HCI_LE_SET_SCAN_ENABLE_COMMAND
  return_parameters:       HCI_SUCCESS
DEBUG:bumble.host:### HOST -> CONTROLLER: HCI_LE_SET_SCAN_ENABLE_COMMAND:
  le_scan_enable:    0
  filter_duplicates: 0
DEBUG:bumble.controller:<<< [Scanner] HOST -> CONTROLLER: HCI_LE_SET_SCAN_ENABLE_COMMAND:
  le_scan_enable:    0
  filter_duplicates: 0
DEBUG:bumble.controller:>>> [Scanner] CONTROLLER -> HOST: HCI_COMMAND_COMPLETE_EVENT:
  num_hci_command_packets: 1
  command_opcode:          HCI_LE_SET_SCAN_ENABLE_COMMAND
  return_parameters:       HCI_SUCCESS
DEBUG:bumble.host:### CONTROLLER -> HOST: HCI_COMMAND_COMPLETE_EVENT:
  num_hci_command_packets: 1
  command_opcode:          HCI_LE_SET_SCAN_ENABLE_COMMAND
  return_parameters:       HCI_SUCCESS

So maybe there are 2 problems left:

  • not exiting after 5 seconds as specified
  • no devices are discovered - maybe a callback is not set?

@JPHutchins
Copy link
Contributor

So, it looks like Bumble uses a flexible event callback system. Unfortunately, they're using strings again, but here's the magic string: "advertisement". Found in source as

self.emit('advertisement', advertisement)

So it seems like intended usage may be with a closure, like this:

    async def start(self) -> None:

        def on_advertisement(advertisement: Advertisement):
            logger.debug(f"Received advertisement: {advertisement}")

            service_uuids: List[str] = []
            service_data: Dict[str, AdvertisingDataObject] = {}
            local_name = advertisement.data.get(AdvertisingData.COMPLETE_LOCAL_NAME)
            if not local_name:
                local_name = advertisement.data.get(
                    AdvertisingData.SHORTENED_LOCAL_NAME
                )
            manuf_data = advertisement.data.get(
                AdvertisingData.MANUFACTURER_SPECIFIC_DATA
            )
            for uuid_type in SERVICE_UUID_TYPES:
                adv_uuids = advertisement.data.get(uuid_type)
                if adv_uuids is None:
                    continue
                if not isinstance(adv_uuids, list):
                    continue
                for uuid in adv_uuids:  # type: UUID
                    if uuid not in service_uuids:
                        service_uuids.append(str(uuid))

            for service_data in advertisement.data.get_all(
                AdvertisingData.SERVICE_DATA
            ):
                service_uuid, data = service_data
                service_data[str(service_uuid)] = data

            advertisement_data = AdvertisementData(
                local_name=local_name,
                manufacturer_data=manuf_data,
                service_data=service_data,
                service_uuids=service_uuids,
                tx_power=advertisement.tx_power,
                rssi=advertisement.rssi,
                platform_data=(None, None),
            )

            device = self.create_or_update_device(
                str(advertisement.address),
                local_name,
                {},
                advertisement_data,
            )
            self.call_detection_callbacks(device, advertisement_data)

        self.device.on("advertisement", on_advertisement)

        await start_transport(self._adapter)
        await self.device.power_on()
        await self.device.start_scanning(active=self._scan_active)

That said, I still have no success. After adding some logging to bumble and pyee I can confirm that no events are being emitted, so I'm not really sure where to look next.

@JPHutchins
Copy link
Contributor

Aw, I see this was supposed to work with multiple inheritance of Device.Listener. I would favor dependency injection instead, for clarity: self._listener = OurListenerImpl().

@JPHutchins
Copy link
Contributor

Can confirm the scanner (Zephyr FW) is working well with Bumble's example!

python -m examples.run_scanner usb:2fe3:000B

@JPHutchins
Copy link
Contributor

I think this can be refactored to align with the example more closely: https://github.com/google/bumble/blob/5e959d638e6a9c99e62536d0a3472cf4e6616ccf/examples/run_scanner.py#L36-L78

Specifically, class consrtuctors in Python async should not do any IO. So we can wait until some action is taken to open connections and do async IO, for example.

@JPHutchins
Copy link
Contributor

The following scanner implementation is working well for me. Obviously would need to be adapted to fit with the rest of the API, but it's nice to see it running.

# SPDX-License-Identifier: MIT
# Copyright (c) 2024 Victor Chavez

import logging
import os
from typing import Dict, Final, List, Literal, Optional

from bumble.core import AdvertisingData, AdvertisingDataObject
from bumble.device import Advertisement, Device
from bumble.hci import Address
from bumble.transport import open_transport

from bleak.backends.scanner import (
    AdvertisementData,
    AdvertisementDataCallback,
    BaseBleakScanner,
)

logger = logging.getLogger(__name__)

SERVICE_UUID_TYPES = [
    AdvertisingData.COMPLETE_LIST_OF_128_BIT_SERVICE_CLASS_UUIDS,
    AdvertisingData.COMPLETE_LIST_OF_16_BIT_SERVICE_CLASS_UUIDS,
    AdvertisingData.COMPLETE_LIST_OF_32_BIT_SERVICE_CLASS_UUIDS,
    AdvertisingData.INCOMPLETE_LIST_OF_16_BIT_SERVICE_CLASS_UUIDS,
    AdvertisingData.INCOMPLETE_LIST_OF_32_BIT_SERVICE_CLASS_UUIDS,
    AdvertisingData.INCOMPLETE_LIST_OF_128_BIT_SERVICE_CLASS_UUIDS,
]

# Arbitrary BD_ADDR for the scanner device
SCANNER_BD_ADDR = "F0:F1:F2:F3:F4:F5"


class BleakScannerBumble(BaseBleakScanner):
    """
    Interface for Bleak Bluetooth LE Scanners

    Args:
        detection_callback:
            Optional function that will be called each time a device is
            discovered or advertising data has changed.
        service_uuids:
            Optional list of service UUIDs to filter on. Only advertisements
            containing this advertising data will be received.
    Keyword Args:
        adapter (BumbleTransport): Bumble transport adapter to use.
    """

    def __init__(
        self,
        detection_callback: Optional[AdvertisementDataCallback],
        service_uuids: Optional[List[str]],
        scanning_mode: Literal["active", "passive"],
        **kwargs,
    ):
        super().__init__(detection_callback, service_uuids)

        self._scanning_mode: Final = scanning_mode

        self._device: Optional[Device] = None

    async def on_connection(self, connection):
        pass

    async def start(self) -> None:

        def on_advertisement(advertisement: Advertisement):
            logger.debug(f"Received advertisement: {advertisement}")

            service_uuids: List[str] = []
            service_data: Dict[str, AdvertisingDataObject] = {}
            local_name = advertisement.data.get(AdvertisingData.COMPLETE_LOCAL_NAME)
            if not local_name:
                local_name = advertisement.data.get(
                    AdvertisingData.SHORTENED_LOCAL_NAME
                )
            manuf_data = advertisement.data.get(
                AdvertisingData.MANUFACTURER_SPECIFIC_DATA
            )
            for uuid_type in SERVICE_UUID_TYPES:
                adv_uuids = advertisement.data.get(uuid_type)
                if adv_uuids is None:
                    continue
                if not isinstance(adv_uuids, list):
                    continue
                for uuid in adv_uuids:
                    if uuid not in service_uuids:
                        service_uuids.append(str(uuid))

            for service_data in advertisement.data.get_all(
                AdvertisingData.SERVICE_DATA
            ):
                service_uuid, data = service_data
                service_data[str(service_uuid)] = data

            advertisement_data = AdvertisementData(
                local_name=local_name,
                manufacturer_data=manuf_data,
                service_data=service_data,
                service_uuids=service_uuids,
                tx_power=advertisement.tx_power,
                rssi=advertisement.rssi,
                platform_data=(None, None),
            )

            device = self.create_or_update_device(
                str(advertisement.address),
                local_name,
                {},
                advertisement_data,
            )
            self.call_detection_callbacks(device, advertisement_data)

        hci_transport: Final = await open_transport(os.environ["BLEAK_BUMBLE"])

        self._device = Device.with_hci(
            "scanner_dev",
            Address(SCANNER_BD_ADDR),
            hci_transport.source,
            hci_transport.sink,
        )

        self._device.on("advertisement", on_advertisement)

        await self._device.power_on()
        await self._device.start_scanning(active=self._scanning_mode == "active")

    async def stop(self) -> None:
        if self._device is None:
            raise RuntimeError("Scanner not started")

        await self._device.stop_scanning()
        await self._device.power_off()

        self._device = None

    def set_scanning_filter(self, **kwargs) -> None:
        # Implement scanning filter setup
        pass

@JPHutchins
Copy link
Contributor

There's another wrinkle with Bumble. The library defines open_transport as an async function, but this is not necessary in the case of opening a USB transport - that is, open_usb_transport has no awaits.

Defining the USBTransport class in a factory function is a problem.

@vChavezB
Copy link
Author

Thanks for the tests and suggestions to make it work with usb 👍

@vChavezB
Copy link
Author

vChavezB commented Nov 17, 2024

I have done some more type safety changes, and moved operation calls of the Bumble Controller stack to function calls.

In addition, I removed the listeners and use the on function to get access to the function calls through the emitters of Bumble.

@vChavezB
Copy link
Author

vChavezB commented Nov 17, 2024

The following scanner implementation is working well for me. Obviously would need to be adapted to fit with the rest of the API, but it's nice to see it running.

I think the modifications you made were due to having an HCI controller (Zephyr MCU).

graph LR
    A[Bleak Bumble - HCI Host] <-->B[Zephyr - HCI Controller]<--> C[Bluetooth Radio]
Loading

For my use case its the other way around.

graph LR
    A[Bleak Bumble - HCI Controller] <-->B[Zephyr - HCI Host]
Loading

Perhaps there should be an env variable or setting to set the backend mode:

  1. Communicate with an HCI controller: Used when your native OS drivers does not support the HCI controller.
  2. Communicating with a HCI host: Used for virtualization and cross-platform functional tests with bluetooth.

@vChavezB
Copy link
Author

vChavezB commented Dec 29, 2024

Just a short update. You can now set the HCI Mode (Host or controller).

To set as HCI Host env. variable BLEAK_BUMBLE_HOST can be set or via argument.

@vChavezB vChavezB force-pushed the bumble branch 4 times, most recently from 1a4da61 to 168ed72 Compare January 3, 2025 14:10
The backend supports direct use with Bumble. The HCI Controller
is managed by the Bumble stack and the transport layer can
be defined by the user (e.g. VHCI, Serial, TCP, android-netsim).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants