KBFS: broken integration with Finder on macOS < 12 & > 13: read this for a very kludgy workaround! #25830
Comments
A very long explanation on the causes of this issueIf you have nothing more important to read, and are genuinely interested in understanding more about the inner workings of the way Keybase is integrated with macOS, read on... I promise it's going to be very boring and long-winded!. What is broken, and why?Simply put, the integration of Keybase and Finder relies upon a technology known as Filesystem in Userspace (FUSE), originally developed for the Linux kernel. This is a technology that allows novel filesystems to be implemented safely outside the kernel, and presented to the underlying system as if they were 'normal' files — without requiring device drivers, kernel patches, extra services running, etc. — just like a 'normal' filesystem, so to speak. In fact, when well-implemented, the operating system is not even really aware of what exactly it's accessing or how the magic happens; a file is a file, after all. It has been successfully ported to other Unixes, as well as to macOS (there are at least three variants; more on that below), and there is an equivalent technology for Windows known as the Dokan Library, which, these days, also features a FUSE-compatible layer. This means that (in theory!) if someone develops a new filesystem (such as Keybase, with KBFS) it can easily work, without change, across practically all platforms and architectures. In practice, it's not that easy. Although FUSE support has been part of the Linux kernel for many, many years now, Apple frowns at anything not invented by them. And while Darwin (the operating system) is open source, the macOS kernel is not. However, Apple has generously allowed developers to 'plug' into the kernel the so-called 'kernel extensions', which technically run on a safe environment 'very close' to the kernel — but not too close! The variants of FUSE for macOS generally use a kernel extension to provide the necessary hooks for the FUSE API to work, and from that layer upwards, it's a standard FUSE API from the perspective of the developers. Apple doesn't like kernel extensions and has threatened to get rid of them in future versions of macOS (there are replacement mechanisms, known as 'system extensions', but they don't exactly replicate all the functionality), but, until Benjamin Fleischer, the developer/maintainer of FUSE for macOS, switches to the 'new' technology, we're stuck with some variant of his kernel extensions to get a working FUSE API. Needless to say, from the point of view of Keybase (the company), this is exactly what they need to allow their own cryptographically-signed, network-based filesystem to be mounted on any operating system (including mobile phones!). They just need to worry about interconnecting between KBFS and FUSE. And then, each platform will have its own way of supplying the required FUSE API — and that is not something Keybase needs to worry much about any longer. Now, in the Mac world, there is a catch. Every time there is a kernel upgrade, you need a kernel extension specially compiled for that specific kernel, and which might work on future versions — but will never work on older ones. This is due to the way Apple modifies its kernel and changes the APIs for the kernel extensions; they change (sometimes dramatically, when the architecture changes — from PowerPC to Intel and now ARM64) over the years, and that means that developers of kernel extensions have to keep them up to date with the 'latest and greatest' versions. Fortunately, in general, there is no need to specifically develop everything from scratch; in most scenarios, all that is required is just to recompile the extension with the Xcode SDK for the new version. Or hope that the old version somehow still works. You can use older SDKs to compile extensions for new kernels — strangely, that seems to be the most common scenario (also used by Keybase). To make things more complex in the Mac universe: one of the first widespread kernel extensions, then known as 'Google MacFUSE for Mac OSX', was abandoned by its authors ca. 2009, but the source code was made freely available by Google. A new project was created by the aforementioned Benjamin Fleischer, it was renamed OSXFUSE, and was kept up to date (using the Linux kernel implementation as the reference). Then Apple started getting paranoid with digital signatures for all installed software (except whatever you happened to compile yourself). Because those signatures require a paid certificate from Apple, it meant that many open-source project managers couldn't afford the luxury of signing all the releases; instead, they just supplied the open-source code, and we had to compile it on our own. This is tricky — it requires having Xcode (free) installed, with the appropriate SDK (also free), but potentially also a digital signature (not free!) and spending a lot of time tinkering with everything so that it works, spewing an installable binary after much munching and crunching over many minutes. Or hours, on old computers. To side-step this issue, the team behind the OSXFUSE project made a dramatic change on their software base. While most of the code is still open source, there is something deep in the core — signed by the team — which is not, and it’s being distributed on binary form only. These components are still free to use, free to distribute, free to do whatever you wish to it (even resell it, if that's your plan), but limited to non-commercial usage: you just don't get the source code for it. Why? Because it was signed by the main developer himself, who cannot afford to 'give away' their keys to everybody. So they reached a compromise: continuing further development on the now rebranded "macFUSE" project, which is partially free and open-source, but has a closed-source component, only published in binary form, which is free for non-commercial use; while still keeping around (with no further development) the 'old' 100% free & open source OSXFUSE. Apple, of course, went a notch further in their paranoia. Now it was not only necessary to digitally sign one's software; to be able to make it available on the App Store, or at least prevent macOS from complaining about 'unknown developers', the software, after being signed, has to be notarised by Apple — which appends their own digital signature only on software they 'approve'. I'm not sure if this has an extra cost (I believe not) but it obviously means that Apple will be quite aware of what is being distributed and how. And on top of that, kernel extensions require a special digital signature from Apple (because they’re dangerous, after all!), which, allegedly, is quite hard to obtain. Benjamin Fleischer was lucky enough to be one of them, so his work under closed-sourced macFUSE could go on, while still being able to distribute the new kernel extension for free (for non-commercial use). You can read a bit about the paradigm change on this article. Keybase (the company) now faced a dilemma. They obviously could just get macFUSE being shipped with Keybase (both the open-source and the closed-source components) — possibly paying something to Fleischer. Since OSXFUSE doesn't get any further patches (and allegedly has some serious security issues — well, according to Homebrew, at least), macFUSE is now the 'preferred' version to enable FUSE under macOS. Also — and this might not have been obvious! — you just need one version of FUSE in your system: everything that uses FUSE to provide 'virtual' filesystems will 'talk' to whatever is providing the FUSE API. That would mean, for Keybase, that everybody who already had macFUSE installed on their systems (which is not that unlikely; it is quite useful and popular in many scenarios) would never need to worry about it. Others might just get a warning/notice to install it — or, better, if Keybase detected the presence of a Homebrew installation (possibly MacPorts also works; I haven't checked), it could automatically launch it to install macFUSE, and not worry about OS versions and architectures — that's the job of the Homebrew team! Alas, the truth is that the number of users who are actively updating their preferred FUSE version is much smaller than the number of active Keybase users. And to make matters worse... many may have old/outdated versions of FUSE, or not the quite correct one, or just downloaded it from a different repository with its own installation quirks... and that meant Keybase wouldn't work. And they (the company) would have to give tech support over things they didn't control. Instead, they opted to include their own FUSE kernel extension (a similar approach taken by other software developers, such as Atakama). But here they stumbled upon another problem: macFUSE, as said, has a closed-source component. With a digital signature. Now, to be able to distribute it with Keybase, the company would have to double-sign the whole package — or else, macOS (and the user!) would be confused, getting one package signed by two different developers. I'm aware that there is a way for creating a chain of signatures — ultimately notarised by Apple, of course — to avoid this double-developer issue, but it seems reasonable to assume that Keybase (the company) could not 'force' the developers of macFUSE to adapt their own software if and when required, while sticking to Keybase's own release schedule. They aren't Keybase employees, after all. (Never mind that, these days, macFUSE gets more regular releases than Keybase, but that wasn't true before 2020.) Keybase's solution, therefore, required a single code source, that they could sign in toto, without requiring anyone outside the company to cooperate or collaborate in that process. This meant abandoning macFUSE, and embrace the venerable old OSXFUSE instead. Keybase doesn't do much with the code: essentially, they only do a search for the OSXFUSE tag inside the code and change it to KBFUSE instead. I believe that one or two patches are applied, to fix the most dreadful security issues in OSXFUSE, but, in general, the code is the same, and works exactly with the same API, and, most importantly, you can replace one by the other, and Keybase will still work. This is what they internally do, when they suspect that, somehow, their slightly-tweaked version of OSXFUSE doesn't seem to work in some scenarios, and try to use the original kernel extension instead, just to see where the problem is. So, after all, where is the problem, then? Shortly before Christmas 2022, Keybase (former?) developer @mmaxim was taking a look at some filed issues from Ventura users, who couldn't get Finder to mount the virtual KBFS with FUSE. He correctly figured out that OSXFUSE was not prepared to go beyond macOS 11 (Big Sur). This meant a change of plans, namely, adapting the macFUSE 4.X code to work with Keybase instead. How exactly that was supposed to work is beyond my understanding — at least, not without getting rid of the closed-source component — and I most certainly didn't check what exactly he did. The truth is that he managed to compile a kernel extension that worked under 12 and 13, thus making Ventura users very happy. He did make a mistake of some sort — a missing bundle — which was corrected minutes later. Then a release was officially produced and made available to anyone who had the app on auto-update and... @mmaxim disappeared from the face of the Earth, without trace. Immediately, anyone not on macOS Ventura started to complain and post issue after issue, saying that the latest and greatest Keybase release was 'broken'. And it surely was. Be it by choice, or simply as a last-minute-patch-before-my-Christmas-vacation-from-which-I-might-not-come-back-so-soon, the truth is that @mmaxim only left a kernel extension available for macOS 12 (it just happens that 13 also works with the same kernel extension — but nothing that came before that!). All the other kernel extensions — as said, OSXFUSE supports everything from 10.5 upwards — were simply not included in the bundle, and I'm sure that this was pure neglect; after all, supporting all macOS versions will just require 160 Kbytes on disk (at least, the thin binaries). Incidentally, @mmaxim also wanted to 'fix' the issues of those running Big Sur on Apple Silicon — alas, that didn't work as expected, since he 'forgot' to include a fat binary for macOS 11. Fortunately, it's presumable that Big Sur users on Apple Silicon have moved on to newer macOS versions (Monterey and Ventura, both supported) — until they upgraded to the Sonoma Beta, which won't work with Keybase's KBFUSE any longer (granted, Keybase has no obligation to support beta software, but Sonoma will be launched in a few months, so... it would be better to start updating the code now). The disappearance of @mmaxim is actually a bit baffling. You might have noticed that @chrisnojima has been active as always, since early 2023 doubling his efforts to squash bugs left and right; however, he focuses mostly on the Electron GUI side of things. Maybe the main reason for the lack of an official release is the huge number of issues that nobody had time to reply to yet — we can only speculate on that. So, now you know what was broken. How do we fix it? The 'official' fix is not available to usTo do a proper fix, of course, requires recompiling the code, and this time, making sure that the KBFUSE bundle includes the missing kexts. In theory, this should just be a simple change. In practice, @mmaxim claims to have bumped the FUSE kext to '4.4.X' which is surprising — as said, this is supposed to be possible only with the closed-source component. I haven't checked what exactly has changed; the truth is that, on the source code tree, at the date of writing this,I can only find the binaries for the kernel extension. I can’t see where the source is. I have no idea if @mmaxim just copied the binaries from macFUSE and added them to the package. It looks like that for sure, but perhaps the actual source code is in a non-obvious place? If it isn’t, why should they bother to include so many string replacement scripts to change from “osxfuse”/"macFUSE" (with many different possible capitalisations) to “kbfuse”? Or is that just legacy code for the “old” OSXFUSE, which is not being called any longer?
Or do they have already an agreement with Fleischer? After all, the building script assumes that there is source code “somewhere”: Line 12 in 4ce9e3e This source code package is not available to the general public, so one has to assume that it’s something downloaded separately from a ‘private’ website, to which Keybase employees have access, but not the general public. In any case, this would require a full recompilation and a repackaging — or else, macOS will complain that the cryptographic seal had been broken. Still, it's frustrating to see that perhaps less than 1% of the whole package needs to be changed (macFUSE still supports older versions of macOS, so, presumably, the source should be able to compile the kexts). If @mmaxim hadn't disappeared, I'm sure that he could fix his mistakes in a matter of minutes (and then, of course, take hours to compile & fully test all releases, even though most of that happens semi-automatically these days, not requiring him to stare in front of a computer screen). For an outsider, it means delving deeper into the codebase, and trying to understand how much can be salvaged to apply a fix. I also personally lack the ability to do cross-compilation to different versions of macOS or different architectures — which ultimately means that I could only do a version that happened to work on my Mac but couldn't guarantee that it would work on anybody else's Mac. To make matters even more confusing... have you taken a look at what exactly the Keybase app contains? It's a little Russian Doll, layers within layers. Besides the Keybase.app proper — essentially an Electron executable — there are a few command-line tools which also get installed and put on the path. But there is more. Inside the app's contents there are two further apps, namely, an installer and an upgrader. These are full-blown macOS apps that get co-installed with the 'main' app. And it goes deeper: inside the installer app, there is the above-mentioned installable bundle, which installs The dreaded error box saying that the Finder integration has failed is essentially communication between the 'main' app and its subsumed 'child' apps. Keybase, the main app, checks if it has After that, when Keybase terminates, it cleans the configuration after itself, and asks to do the reverse (i.e. unregister the helper applications and remove the files for But this is not all that happens under the hood. A big question I had was — how does a userspace application acquire permissions to mount a (virtual) filesystem, without asking the user to authorise a privileged operation? The answer wasn’t that obvious, but here Keybase is not especially original. What it also launches is a Helper daemon. This is a typical solution for a common problem. So, when a user gives permission to install Keybase, not only the kernel extension is plugged into its place, but the Helper software also gets permission to run in privileged (superuser) mode. That way, it will never be necessary to ask the user for permission: the Redirector, for instance, can run as the user, asking the Helper to make any necessary changes to mount the filesystem on its customary place (e.g. Anyway, if any of the above fails, then Keybase app gets 'confused' and doesn't quite know if Finder mounted the volume or not. When it attempts to do something on the KBFS filesystem, and it doesn’t work, then it tries again. Sometimes, it might even trigger a new attempt to call How this ought to work (but doesn’t)Keybase, in its many variants (OS & architecture), is usually configured from several sources:
While most of the command-line arguments can be seen with The most important, though, for our scenario, is Therefore, the overall solution for this issue should be quite simple: just install your preferred FUSE extension (kernel or even possibly a modern so-called ‘system extension’), figure out where it’s installed (easy if you did it via Homebrew!), and pass that parameter to That’s what sensible programmers would have done, right? And probably they did, but without any sort of documentation (not even any clue where on the source code this may be called), it’s next-to-impossible to figure out where such parameters can be changed by the user. I tried the obvious — changing Worse than that, I couldn’t even understand where exactly those parameters might be stored on disk. While it’s easy to figure out how things work for the However, it’s naturally easy to see what parameters As you know by now, these parameters are not there at all, because Maybe Keybase (the company) has a good reason for doing things this way, but I fail to understand it. What I can say is that Again, one would assume that these parameters are read from somewhere (disk, environment, possibly even command-line arguments to So, at this point, I was stuck: I couldn’t find any place to configure One tiny little ray of hope was the following experiment: what if I launch And, in fact, during a tiny slice of time (at around 6 AM my time, after looking for a solution for several hours), Yay? No, not really. A fraction of a second later, Keybase writes While this is terribly annoying in our scenario, consider how Apple needs to give technical support to clueless users. Apple doesn’t really want to deal with complex configuration problems: macOS, after all, should ‘work out of the box’ and be tamper-proof, even by the legitimate owner of the Mac. This behaviour, therefore, makes sense on an operating system that is supposed not to allows users (legitimate or malicious) to tinker too much with it. It’s also of no avail to attempt to kill the The kludgeMy ‘hack’, therefore, was quite simple — since I had no other options left. If Keybase only let the However, the ‘cleaning up’ after leaving Keybase makes some sense. One presumes that if the application is shut down (‘completely shut down’), users expect that nothing else will be running (well, eventually, an exception can be made for the tiny FUSE kext, which has to be explicitly removed, but if it’s not being actively called, it won’t consume any resources). That means not only no Keybase, but also no Helper applications, no KBFS, no mounted volumes, nothing. There are obviously ways for doing that without having to delete every configuration file for Why exactly Keybase didn’t go that route is beyond my understanding. I thought it could be something related to users not wanting to run Keybase at login — but that’s not even an issue with It would also obviously allow users to change that configuration file 😉 (which is, after all, the whole point of placing these files under a user’s Library!). Maybe Keybase really, really, really doesn’t want anybody tampering with that file? Well... I don’t want Keybase to tamper with my files, either! After a few failed experiments — the file gets overwritten even if it has no write permissions, or even if it’s owned by root — there seemed just a way to keep the file there: make it immutable. For those that aren’t aware of the many layers of protection on the Mac filesystem, there are at least three I’m aware of. The first level is common to all Unixes, i.e. the file permissions that can be set to owner/group/all. Since any user can become a privileged user, and that means giving them the ability to break anything (or everything!) beyond repair, e.g. On top of that ‘basic’ level, Apple has implemented file attributes, available using the But there is another layer of protection, which Apple inherited from BSD 4.4, and which are known as flags. These apparently are around since 2006 (possibly earlier!), and can be viewed with Although the 2006 man page cautions the user that not all utilities respect these BSD flags (and can therefore override them), the good news is that, whatever Keybase uses to generate the WIth that ‘trick’ — setting There are, of course, limitations. And, aye, this is not 100% stable — not by far! Every now and then, Keybase stops communicating with These are inconveniences, especially if you use the KBFS actively, i.e. with some kind of application directly writing on it (such as, say, Notes...) — because when the connection goes down, you might lose your work. An application reading/writing to a file on KBFS at the moment it loses connection will very likely enter an infinite loop, expecting the connection to be up again. Re-launching Possible future solutionsI’m not going to delve deep into this. My hope is that someone at Keybase at least takes notice of this issue, and, as a consequence, decides to either revert commit #25338, or, in a much more reasonable way, allow us to modify many more configuration parameters. One thing that has intrigued me is how the latest Keybase versions are able to ship a version of 4.X, which allegedly includes the issue with the digital signature affixed on a closed-source component — because OSXFUSE does not support (directly, at least) Apple Silicon, only macFUSE 4.X does that. But client/go/kbfs/libfuse/mounter_osx.go Line 86 in 4ce9e3e client/go/kbfs/libfuse/mounter_osx.go Line 39 in 4ce9e3e There are not many references to how exactly Even weirder... to ‘talk’ to the FUSE kext, Keybase uses the popular Go library from bazil.org/fuse. The maintainer of the bazil-fuse project, @tv42, seems to have dropped all macOS support on his library: bazil/fuse#224 While I couldn’t test it myself, however, it seems that ‘pre-3.3.0’ support is still in bazil-fuse. Or else, how could Keybase use it? Unless, of course, if they are just using an old version of bazil-fuse (and don’t dare to upgrade it!). Not surprisingly, this dilemma is faced by many other open-source projects that use macFUSE. Here is an interesting discussion/debate between the developers of The bazil-fuse maintainer, however, admitted back in 2016 that there might be a compromise, since recent (i.e. 4+) versions of macFUSE do, indeed, support a very complex mounting mechanism, which apparently works similarly in both macOS and Linux. I’m afraid this is way too deep-level in the respective kernels (which are so different!), but I can understand that, over the years, thanks to some convergence of efforts, there are now some more common (or at least more familiar) kernel APIs. They must be different at some level, though, because, as said, the kernels are completely different beasts. FreeBSD inherits the same codebase (BSD 4.4) as macOS/Darwin, but there are radical differences between both: Darwin uses a microkernel architecture based on Mach (XNU, to be more precise), something that Apple inherited from Steve Jobs’ work back when he founded NeXT. FreeBSD, by contrast, has a monolithic kernel — in fact, the same approach taken by Linux, which comes from a completely different family (namely, SysV Unix). Interestingly, there was some convergence between the Linux and FreeBSD kernels, mostly because of the Internet: the TCP/IP support on BSD was far superior to anything else in the mid-1990s, and, since it was free and open source, even Microsoft Windows used it (seriously!). Linux started using its own solution, but BSD’s was by far more superior. This led to the kernel developers from Linux and FreeBSD to come together and adapt their kernels so that many things are similar, even if their origin is completely different (and it shows!). What this means is that, in some regards, the “BSD philosophy” is followed by the Linux kernel, and this has become familiar to those who use Linux only, and are scared of macOS/Darwin — which also uses the “BSD philosophy”, just a conceptually totally different kernel. Obviously, there is the easy way out: dump bazil-fuse altogether, use go-fuse instead, and let the users decide what ‘flavour’ of kernel extension they prefer. Go-fuse supports them all, and their maintainers are trying to essentially support all FUSE implementations they can. Needless to say, this is not a drop-in replacement; a lot of code would have to be refactored to take it into account. Anyway, I digress (but this was the “futurology” section, so I’ve got an excuse!). I really didn’t delve that deep in Keybase’s December patch — just merely scratched the surface — and therefore haven’t reached a conclusion yet. One thing is for sure: without a bit more help from the Keybase developers, it’s hard to ‘fix’ the code. My best suggestion, therefore, would be to specifically target Because everything else can remain in place: the Keybase application doesn’t care what Or, alternatively, change the Thanks for reading till the very end! Digitally yours, — Gwyn Note: on the comments below, I was warned (correctly) that certain expressions in this text could be considered offensive and drama-inciting, and therefore I deleted or changed them (they didn't really add anything to the overall context). There was no intention to offend anyone, of course, much less those who have kept their projects around for years and years and having hundreds of thousands or millions of users relying upon their code, often published as free and open-source. I'm fond of hyperbole, but I agree that this time I should have been more careful, more neutral, and, especially, pay more attention to the real human beings who live behind their nicknames. Nobody benefitted from those expressions and turns of phrase, so I have eliminated them all, as far as I could tell; if you find anything that is still offensive, drama-inciting, exaggerated, wrongly explained or presented, or simply factually wrong, please let me know. Like so many of you, the only thing I want is to get a fix, a patch, something to keep everything working; unfortunately, I'm terribly limited in my knowledge in how things work at a deep level, close to the kernel, and don't even know where to start. In any case, I'm still more inclined to believe that it's the Keybase code that should be changed — not the FUSE extensions for Go, or the kernel extensions injected into the macOS kernel — but that would definitely require a repackaging and a profound refactoring on how the application is distributed/published in the macOS ecosystem. That is only up to Keybase (the company) to deal with, not those who created all those pieces of software, without which Keybase's KBFS would never work — and it was quite offensive for me to direct my rants to them, who have absolutely nothing to do with Keybase and Keybase's decisions on how their software is installed. Again, my apologies; I'll endeavour to be more careful and more respectful in the future. |
|
Replace osxfuse with fuse-t (http://fuse-t.org/). No kext is required. Integration with basil-fuse can be easily implemented. |
You're right, Alex — this is very likely the way to go in the future, and which Apple also recommends. Your trick of using the NFSv4 subsystem and apply a FUSE layer on top of it is pure genius :) FYI, locally, I have installed your package as well from Homebrew ;) but, sadly, Keybase's Granted, it'll be up to Keybase to replace one closed-source component by another :) but that's up to them :) |
|
By sheer chance, I was browsing through Master Anacrolix's ever-fascinating list of repositories, and I noticed that he 'fixed' bazil.org/fuse to continue to work under macOS, releasing his own fork: https://github.com/anacrolix/fuse There you go, Keybase — no need to switch Go FUSE libraries at all, just get the drop-in replacement, and all your woes are gone! (well, and so are ours, of course) To-do (for self): fork just the |
|
Actually there's already a fork at https://github.com/keybase/fuse |
Not that Keybase has bothered much to explain anything, they just install their own fork and that's it. |
Also, my apologies; the original fork with working macFUSE 4+ support is from @zegl, but he has archived his fork over two years ago and never updated it; it was probably just meant to submit a PR, and when that was sort of rejected by @tv42, @zegl's fork was abandoned. @anacrolix picked up where @zegl left it, brought it up to date with the current version of bazil/fuse, and changed all dependencies, so that his fork can work as a drop-in replacement for bazil/fuse. |
|
@GwynethLlewelyn thank you, yes that is the correct read. I believe zegl's fixes apply to the most recent possible bazil.org/fuse possible before a major rewrite is necessary. I made it possible to import it without replace directives or a workspace. A minor improvement but necessary for very long lived forks. |
|
@anacrolix Here you go, sent you the PR |
|
With the amount of effort that the @GwynethLlewelyn went into detailing this issue I sincerely hope this gets fixed :) |
|
Describing others as "very angry" and "drama" is drama-seeking behavior. Please consider who is the source of drama when using such tone. |
You're right — my apologies, after reading some of your replies, I was projecting my own perceptions and wrongly assumed that I read there a certain degree of annoyance/irritation with the way Darwin/macOS work at the (micro)kernel level. I'm removing those adjectives now. GitHub will preserve their history, of course, but at least, people reading this issue for the first time will only read it in a more neutral tone. |
|
It's sad and somewhat frustrating that the internals of Linux and macOS/Mach don't happen to align enough to make FUSE work well on macOS. I don't think I have more emotions about that topic than that. They're independently evolved systems where commonalities are superficial and/or result of convergent evolution; it'd be surprising if their internals aligned by luck. Linux and *BSD are more aligned because they've been happily at least partially "borrowing" each others' best inventions on how to structure the kernel VFS etc. The FUSE protocol is very much a codification of how the Linux kernel VFS works internally; it's purpose is to export that API to userspace in a safe manner, not to be a universal filesystem driver interface. |
|
I installed Fuse T and could gain access again to the chats and files, despites the error still pops up. |
|
On (iMac late-2013 with the last installable macOS) macOS Catalina, 10.15.7, Homebrew (I already had it) informed me that it is "not upgrading osxfuse, because the latest version is already installed". I'm paused at this moment wondering whether any further actions are needed. (Will test.) On my other ten years old Mac (Mac Pro late-2013 with the last installable macOS) I've got macOS Monterey 12.7.3, and have never had this problem (that has been afflicting the iMac, I guess for over a year. Tested: KBFS problem is not fixed yet. Will proceed with the next steps (after having checked that I have the latest osxfuse already installed. … (Still working on it.) to load, but … something that may be relating to the wording of the original Keybase KBFS installation error, that tells me that there aren't any device slots available, and that they may be taken up by apps such as VMWare, VirtualBox, … VPN programs and Intel HAXM, as I found all these other launch agents, some of which match that descripton. Some of these I don't even use anymore. Maybe this will work if I delete the unused ones? |
|
@seanPhill, regarding those questions:
Ok, there seems to be some confusion here :) What is the result of running:
I'm just asking that because, well, as far as I know, the last version of Theoretically, newer kernels are supposed to be able to run older kernel extensions without complaining much. This is certainly the case with macOS Big Sur 11.7.9 (the last version that Apple bothered to upgrade; it's unsupported now). Thus, you should use:
instead (see my original post). The command-line
Nevertheless, feel free to delete the ones you don't use, especially everything that pertains to applications you have deleted from your disk long ago. The boot process will only waste its time running scripts for applications that don't exist any more. You can make its life easier (and, in return, get a slightly shorter boot time!) by removing whatever you don't use anymore. The message you mention (see the snapshots in this thread) is rather useless and quite cryptic. Essentially what it means is that Keybase couldn't talk to the osxfuse extension — in the current context, of course. At the application level, the error you mention for "not having enough slots", or whatever it says, is overengineered — while you might lack such ports (it's always possible!), the much more likely interpretation is "I've tried to check for the kernel extension I need, but something is wrong and I can't read from it" … meaning that the osxfuse extension is not loaded as a kernel extension, thus Keybase is unable to talk to it. |
|
fwiw, Keybase v6.5.0-20241007140341+f10651d043 on 15.0.1 is the first Keybase client in a long time for which I haven't had to do some sort of shenanigans to get working on arm64. #28028 looks like the PR with the fix? well, except that I had to reboot into the recovery environment in order to enable system extensions, but that felt oddly more wholesome than cramming in FUSE drivers from Homebrew or whatnot. |
Hello all,
After seven frustrating months without the ability to use the Finder integration on macOS Big Sur (currently at 11.7.9 (20G1426)), I finally found a workaround! I'm posting this here because every month someone complains about the integration of Keybase with Finder not working any longer with either very old or very new versions of macOS.
TL;DR (well, not really, the workaround has many steps to follow)
(way longer explanations on the comment below, if you really wish to know more)
The issue
In order for Keybase to mount KBFS as a 'regular' filesystem (as viewed by Finder), it uses Filesystem in Userspace (FUSE). This requires a kernel extension to be installed, and every version of macOS requires at least a recompilation of that extension in order to work in subsequent versions. You can grab FUSE from elsewhere, but for several reasons, Keybase bundles its own version.
Unfortunately, on Dec 12, 2022, @mmaxim, in an attempt to get a quick fix for macOS Ventura users, published, in quick succession, the commits #25335 and #25338. These fixed the issue for Ventura, while breaking it for essentially everybody else. @mmaxim has been strangely absent — he doesn't reply to messages and has barely committed anything during 2023 (and nothing related to Keybase). As far as we can publicly see, the last time he logged in to GitHub was back in March. @chrisnojima, who publishes at least one new commit every day (sometimes, even more than one), is apparently unaware of the Big Mess that @mmaxim left behind, or simply hasn't got time to do a real fix.
Partial list of issues filed from users affected with this problem
Some are directly related, others indirectly; the list still continues to be expanded.
If you've seen the following popup:
then you're welcome to the club! You can see it's not very exclusive — lots of people have experienced exactly the same issue (and still are!) and are waiting for a fix. Here is a sample:
Note: the above list may be very incomplete, I essentially followed a chain of issues that referred to each other; there might be many more who never made any reference — meaning that the author of the issue never bothered to search here — and thus I couldn't find it. But it should give the Keybase developers some pause for thought.
The only known workaround so far is to install a previous version of Keybase, in the hope that it will still bring the "old" Keybase FUSE port which still works with older kernels (as it should). The newer versions of Keybase will always overwrite any attempts of "fixing" the Keybase FUSE kernel extension (by replacing it with a working one); thus my kludgy workaround below.
Going 'back in time' to an old version may not be always possible, of course, and that's why I'm suggesting a very kludgy workaround, which should work with every future version of Keybase (I'm testing it with the nightly builds).
All that the Keybase devs need to do is to partially revert the changes made in early December 2022, keeping the old kernel support in, while adding support for newer kernels (just like the current macFUSE version does).
My kludgy workaround
I have found a very, very kludgy workaround for those who simply cannot live without this feature (myself included!) and would give an arm and a leg just to get it fixed.
Fair warning: this is not for the faint of heart. You might break everything (I didn't, but I guess I was lucky) and never be able to use Keybase again. Or even your Mac. I don't know, and I won't help you out if you have a problem; accept this solution 'as is' and don't complain if it bricked your Mac — you're doing this on your sole responsibility, and, who knows, you might not even get Apple to fix your Mac during the warranty time. Caveat utilitor.
That said, I'm not sure if this will work on anything but macOS Big Sur, because that's the version I've got. However, I'm pretty sure that the following macOS versions should work: 10.5, 10.6, 10.7, 10.8, 10.9, 10.10, 10.11, 10.12, 10.13, 10.14, 10.15, 10.16, 11.X. Anything above that... well, I cannot say, but at least the latest Keybase version works 'out of the box' with 12 and 13. You're out of luck with 14, though.
cd ~/Library/LaunchAgents/. There should be three files there, starting withkeybase...keybase.kbfs.plist: e.g.cp keybase.kbfs.plist ~/Desktop(or wherever you prefer).ps xaww | grep -i Keybasea few times, just to see if some process is lingering (and, if it is, force-quit it from the shell prompt with akill -9 <pid>, or even withsudo kill -9 <pid>).~/Desktop/keybase.kbfs.plist.There might be some slight variants, but it should essentially be what's above.
8. Locate
<key>ProgramArguments</key>, and, below, inside the<array>, you can see the parameters being passed tokbfs. We want to add a new parameter, right at the start. Just before<string>-debug</string>, add a new line with<string>-use-system-fuse</string>. Save it! Your file should now look like this:cat ~/Desktop/keybase.kbfs.plistand check that no weird characters have been added. It should be a pure ASCII file. If you're paranoid, you can even do afile ~/Desktop/keybase.kbfs.plistwhich should sayexported SGML document text, ASCII text. The part that is important is theASCII text. If you getdataorbinaryorRich Text Formatinstead, then you have probably inserted some characters you shouldn't; either fix it or delete it and restart from scratch (i.e., back to 1.).cp ~/Desktop/keybase.kbfs.plist ~/Library/LaunchAgents/. Confirm that it's in the right place!chflags uimmutable ~/Library/LaunchAgents/keybase.kbfs.plist. Check that this worked properly (the dollar sign is your prompt, do not copy & paste it by mistake):$ /bin/ls -lO ~/Library/LaunchAgents/keybase.kbfs.plist -rw-r--r-- 1 myusername staff uchg 1217 26 Jul 11:53 /Users/myusername/Library/LaunchAgents/keybase.kbfs.plistThe crucial flag there is the
uchg(which means unchangeable). So long as that flag is there, nobody can touch that file. Ever. Not even the superuser! You can try that on your own —sudo rm -f ~/Library/LaunchAgents/keybase.kbfs.plistshould give an error. (If you wish to know more about what this does, you'll have to read it on the long version below.) Nevermind the date or the file size, they will obviously be different on your system.Ok, this got rid of Keybase's own FUSE package. Now we have to install the 'correct' kernel extension for FUSE:
brew install osxfuse. Ignore ifbrewcomplains about security issues and nags you to install the much more recent version, now rebrandedmacFUSE. Ignore it. You want OSXFUSE. I believe that the version bundled by Homebrew is 3.11.2.sw_vers -productVersionfrom the shell, if you prefer. If you're on Big Sur, for instance, you should have something like 11.7.9.osxfuse; note that Homebrew might give different results for different versions of macOS and/or architectures. Here is what I have in an Intel MacBook Pro (mid-2014) with Big Sur:You wish to use the version that matches your version of macOS (note that sometimes kernel extensions work across versions, thus the symbolic links).
5. Now load the kernel extension. Cross your fingers! For instance, if your macOS version is 10.12, then you should run the following command:
Apple Silicon (ARM-based) Macs may need to follow further instructions. For your reference (in case of link rot), this is what Apple's manual says:
Also note (if you have Apple Silicon) that the Keybase-provided FUSE kernel extension is a 'fat binary' supporting Intel and ARM64. Homebrew apparently ships the extensions only for whatever architecture you've got, but it's worth figuring out if that's true: if you do a
file /Library/Filesystems/osxfuse.fs/Contents/Extensions/10.12/osxfuse.kext/Contents/MacOS/osxfuse, it should tell you what architecture it has been compiled for. Under my old Intel Mac, Homebrew just provided me with a 'thin' binary for x86_64; YMMV.6. Check if it was properly loaded:
Again, what matters here is that you actually get that line displayed — numbers, etc, will be different.
7. If your macOS version doesn't have the
kmutilcommand, you can use the (deprecated)kextload(instead ofkmutil load) andkextstat -b(instead ofkmutil showloaded --bundle-identifier).All right! Now everything should be in place. Most people will require a reboot at this stage, others will have some luck merely by launching Keybase...
If you do a reboot, the first thing to check is if the kext has been properly loaded, using the command shown on 6. above. This ought to be working.
After a reboot, some magic should have happened, even before you start Keybase. You see, because we left the immutable file on ~/Library/LaunchAgents, the system launcher ought to have launched
kbfs, the Keybase FileSystem daemon, which provides the 'bridge' between Keybase, Finder, and the FUSE kernel extension. You can check that it's running:If you do look at the logs, you will see a lot of errors, which is to be expected: Keybase hasn't been launched yet, and that means no authentication was performed, thus
kbfscannot do anything until that happens.You should also now get a new icon on the Preferences Pane, at the bottom, saying "FUSE". When you click on it, it should tell you the version of FUSE that has been installed by Homebrew. Do not click on update. I know it's tempting, but you do not want a different version — not until Keybase releases a permanent fix.
It is also possible that at some stage you'll encounter a popup like this one:
It might not be exactly that (I forgot to take a snapshot, so I'm reusing an image from a different context), but the purpose will be the same: authorise your Mac to run a kernel extension signed by Benjamin Fleischer, a legitimate Apple-approved developer, who has been doing versions of FUSE for Mac for many, many years. The box is a bit cryptic, and sometimes it's not easy to switch to the Preferences and click on the right place that gives permission to run the kernel extension. Or sometimes you have it all greyed out and have to click on the lock to be allowed to make any changes... and when you do that, the message to allow the extension to be run disappears! It's terrible, I know, but the alternative is to give permission via the command line instead:
Why 3T5GSNBU6W? Well, that 'magic number' actually identifies Benjamin Fleischer — that's his 'Team ID' as registered by Apple. If you wish to see who else has registered Team IDs on your system, you can see them all here:
sudo sqlite3 /private/var/db/SystemPolicyConfiguration/KextPolicy "select * from kext_policy;"It shouldn't surprise you to see Keybase, Inc. listed there as well, with the Team ID 99229SGT5K.
Note that this is basically just a list of extensions that are allowed to run; they aren't necessarily active (with the command above, the third column should show extensions allowed to run — 1 — and those that have been at some point installed in the system but aren't allowed to run any more — 0). Therefore, it doesn't matter if you have multiple versions or variants of FUSE for Mac on that list (including the one from Keybase!). What matters is that the one we want to run has permissions to do so.
All right, everything is in place now — it's time to start Keybase!
A lot of things will happen, most of which will be very confusing to follow, especially if you bother to look at the logs. What Keybase will now try to do is to change the immutable file to launch its own kernel extension instead; this will give some errors (possibly on the logs). Since we have our version of
kbfscorrectly configured to 'talk' to the correct kernel extension, Keybase won't be able to start its own — it will try, though, and get impossibly confused, as now twokbfsare running on the system, one of which non-operational. In one circumstance, Keybase even attempted to get my permission to reinstall the kernel extension, hoping to 'fix' the mess (according to its perspective); I allowed it to do so, confident that it couldn't do much about the immutable file — and, indeed, that's exactly what happened.In my case, during this 'crash of titans', 'our' tweaked version of
kbfsmay shortly notice that it has been granted authentication to connect to Keybase's remote filesystem, and start happily to tell that to Finder. This might only last a few moments, though, because the 'flawed'kbfsinstance will try to 'steal' that authentication for itself, and both will compete to get hold of Finder (potentially crashing it, BTW). Anyway, while it's fun to watch (with the logs open, no less!), it's not what we wish to have, so the trick is to put an end to it all, and this is accomplished simply with(you can even do a
sudo killall -9 kbfsfor good measure)What happens next is actually also amusing. Of course, when
kbfsdies — all instances of it — your Finder access will not work with the remote folder, and, again, it's not impossible that Finder won't crash again. However, there is now a difference: the mechanism that launcheskbfsis not Keybase, but rather the system's launcher, which is set to launchkbfsover and over again, every time it 'dies' for whatever reason. Keybase, therefore, doesn't interfere with the process any more — it knows that, sooner or later (usually sooner!),kbfswill be up again. Also,kbfsis a helper function for Finder — as you have noticed, during all these past few months, you could always access your files, either from the GUI (under Files) or from the command-line (withkeybase fs (...)series of commands). It's only Finder that needskbfs, so Keybase leaves it in peace.Eventually, therefore, the system launcher will re-launch
kbfsagain — but this time, it only has our immutable file to load, and therefore it will cleanly launchkbfsconnected to the correct kernel extension — and Finder will be immediately aware of the service!In theory, this is something you only need to do once — possibly killing
kbfsagain after you install a newer version of Keybase, which might try to run its own installation script again and fight for the right to bind to Finder. In practice, I would guess that every now and then you will need to happily kill some processes, restart Keybase, or even reboot, in order to fix things. I've noticed some inconsistent behaviour after 'waking up' the PowerBook which had been in 'deep sleep'. Also, if you close Keybase (or even just the GUI) and start it again, it might trigger the conflicts once more.If by any chance Keybase kicks your 'tweaked'
kbfsout of the process list, and tries to launch its own untweaked version, then you can try doing the following:Eventually, one of the last three commands will kick the 'correct'
kbfsinto place. Be prepared to do more frequent reboots, though; this hacky configuration may leave Finder unresponsive (as warned earlier) and utterly confused about what mounts are still valid; while sometimes Force Quit on Finder will work, other times you have no choice but to reboot.As said in the very beginning — I told you that this was a kludgy workaround, right? 😸
There you go. That's all. The rest is reference material for those really interested in more information, and how I figured out this very crude and ugly way of 'fixing' things.
Oh, and make sure you send @mmaxim a few messages, asking him to revert the mess he did in December 2022.
(And I apologise for the length of this issue; there are simply way too many steps to take until it works)
The text was updated successfully, but these errors were encountered: