Provide distribution packages

[ygoe]

I'm looking for a USB WiFi adapter that supports dual band, ac and has a detachable antenna. There are some products available but none have their chip indicated. None grant Linux support. Only some buyers find out and report the chip used, and sometimes it's one that might be covered by this or other drivers. When looking for chips already supported by Linux Wireless (basically just select Mediatek MT76xx), it's impossible to find such products.

I have the impression that Linux devs are making drivers very easily accessible for which no products exist. And the drivers for widely available products are hard to identify, find and actually use. Most custom build approaches fall apart immediately when the kernel was upgraded, which might happen through regular security updates. If that WiFi is your only connection to the internet, the updates effectively breaks the device.

Why aren't there packages in Linux distributions like Ubuntu, Armbian or Raspberry Pi OS that allow easy usage of widely available USB WiFi adapters? (The powerful ones, not the cheap low-end stuff that reaches just a few metres.) There should be a package for this driver (and others) that can simply be installed and that automatically plays nice with updated kernels. Maybe even with plug&play. I really don't understand the problem here. Looks like Linux devs are not using WiFi.

[cilynx]

The first thing to keep in mind here is that many Linux devs, including everyone contributing to this repo, are volunteers. We have day jobs, families, and all of the other commitments of real life. The fact that open source software exists and is maintained at all is a small miracle.

The next complicating factor is that most manufacturers of WiFi dongles have basically no incentive or interest in working with the Linux / open-source community. Compared to the Windows userbase who is happy with proprietary drivers, we're a tiny market that they're not going to make any money from. We're also the ones who are going to reverse engineer their stuff and publish their weaknesses, further disincentivizing them from working with us.

Now, for those manufacturers who do release open-source drivers (such as this one), they tend to be recycled garbage code. This very codebase has artifacts related to several adapters that aren't actually supported by this driver because Realtek doesn't start clean with each family of devices -- they just modify the old stuff until it works. That leaves it on us, the community, to figure out what matters and what doesn't when working with the code.

After that, we have the fact that the open-source drivers released by the manufacturers tend to have non-trivial errors, bad practices, etc. in them. They are written by humans, after all, and they're not used to their code every seeing the light of day. Thus, to make them compile cleanly without warnings or errors, we have to go through and clean all that up. You'd think this would be a one-time job, but the manufacturers do occasionally put out driver updates and they of course do not include our cleanup. Whenever they do, we have to go start all over again on a new garbage codebase, diff it with the previous version, figure out what they changed, which cleanups we need to apply, any new cleanup we need to do, etc.

Next, why kernel updates break things. The Linux kernel is ever-changing to make it faster, more secure, etc. Sometimes this means that the kernel devs change the APIs that modules (like WiFi drivers) use to communicate with the rest of the kernel. This means someone (again, volunteers because the manufacturers don't care) has to figure out what broke, go research the API changes, update and test the driver module. Of course not everyone will be running the newest kernel, so we have to support all of the old kernels as well, hence all of the if (LINUX_VERSION_CODE < KERNEL_VERSION(5, 10, 0)) sort of stuff you'll find all throughout the code.

When it comes to packaging for various distros, there are as many different packaging specifications as their are distros and every one of those distros requires alignment with their custom kernels. It's quite common for distro maintainers to apply security patches from later kernel versions to earlier kernel versions and tack on a distro-specific suffix to the earlier KERNEL_VERSION and ship it that way, so now in our wifi driver module, we'd have to look for every possible flavor of every possible kernel version from every possible distro. It would be a full-time job for someone just to maintain a single driver like this one and guarantee it would work with every kernel from every distro. This is why the distro communities tend to maintain packages for their distros as opposed to the upstream module owners. Occasionally, you'll find module owners who build packages for the distro they use, but that requires that they stay abreast of the packaging standards and custom kernel tweaks.

Finally, availability. You are absolutely correct that cheap, proprietary garbage absolutely floods the market while well-supported open-source hardware is neigh impossible to find. You're also correct that "high-power" devices tend to not be supported well while the lowest-common-denominator is. This is again due to manufacturers doing everything they can to make money and open-source developers doing everything we can to cover as broad of a swath of available devices as we can. With wifi (and a lot of other things), that "high-power" functionality comes from manufacturers developing their hardware outside of published and agreed-upon specifications. They have proprietary methods of beam forming, error correction, interference compensation, etc. and they're going to do everything they can to ensure their competitors don't find out how those things work, so all of that gets bundled up into a proprietary binary firmware blob and that's what you get. No open-source for the "high-power" functionality. Open-source developers spend most of our time trying to cover the standards to make sure standards-compliant devices work. This means that most devices you can get will have at least a baseline of functionality.

If someone made a truly open-source hardware wifi platform, they'd be lifted up by the community, but the truth of the matter is that wifi is hard. It takes a lot of work and equipment to make EM waves wiggle exactly the way you want them to and it's just simply not in the grasp of the DIY community today and large manufacturers that do have the skill and equipment are not incentivized to do it. Even the Pine Phone, a triumph of open hardware, still uses a Realtek chip with proprietary firmware for wifi.

So, no, it's not that we don't use wifi. It's that it's a hard problem. If you would like to contribute distro packages, pipelines to create them, updates to add support for new and custom kernel versions without breaking the old ones, @MaxG87 and I will be happy to review your pull requests and improve the repo for everyone.

[MaxG87]

That's a awesome answer and I can only second that! Upgrading to driver version 5.8 was a major pain and I even introduced a regression there 🙁. So yes, working with this code base is indeed hard.

I think this is a good occasion to state that I am no expert on Linux dev tasks or WiFi. I can compile this driver, new kernels and fix compile errors. I definitely cannot add or fix features, because the code base is too complex for me.

Last but not least: This project has a way of distributing it's product. It's DKMS, which is cross-distribution, handles kernel updates and is fully abstracted away by deploy.sh. I once did not merge a PR that claimed to provide scripts which would generate Debian packages because I could not see what deb-packages would provide beyond what is provided by DKMS. However, I am very interested here. What do you find lacking with DKMS? Which use case is not covered?

Also, adding this driver as a package to distros, e.g. to Debian would bring all the burden of maintainership. How would one handle security issues? How bug reports? Timely so? It would be a huge responsibility.

As a last note: breaking changes are introduced only with new kernel versions. I have never encountered a bugfix version that broke this driver. However, I mostly use self-compiled kernels since I learned how to do so for this project.

[ygoe]

I understand the work of managing all this. And unfortunately I'm far from able to provide any of this. My only previous experience was with a wlan driver of a Logilink adapter that was provided as binary but for each specific kernel version. So if the kernel was updated to a new version, the old installed driver might stop working. If this wlan interface is your only link to the device, an upgrade basically bricks it until you get there and can connect a cable.

Meanwhile I tried out this driver on two different devices, a Raspberry Pi 3B+ and an Orange Pi Zero running Armbian (Ubuntu). If I understood this dkms right, it should recompile the driver if a new kernel version is installed, so it should always work after an upgrade. Is this correct? What happens if the build fails because something has changed too much in the kernel? Will the kernel upgrade be rejected then, so that somebody gets a chance to notice the issue? How are incompatibilities with new kernel versions detected?

On the Raspberry Pi, I played a good citizen and first went through the installation process before attaching the wlan adapter. Everything went fine, but communication over that network stopped after some 3 minutes. This is reproducible under different network conditions. So it's not working here for now. I haven't investigated this further or tried possible alternatives yet.

On the Orange Pi (Armbian), I just forgot to install the driver and, in my despair about the unusable devices (should I return them or not?), plugged the adapter in. Then I noticed that it just worked, no installation required here. I don't know what driver is included there, but it's useful without any additional work already. And it's stable over many minutes at least. So I have some working solution at least.

BTW, with "powerful" devices I just meant dual-band 5 GHz ac. Not because it's advanced (actually it's old today) but because most adapters just support 2.4 GHz n WiFi.

[n2qcn]

Armbian uses this exact repo n2qcn/build@91a791c

[lachesis]

I would also love working debian packages so I could just do "apt install rtl88x2bu" or similar. These packages can hook DKMS as e.g. openzfs already does. This would be especially wonderful for something like the Raspberry Pi.