| title |
|---|
The ML Mindshare Championship |
[June 2021]({{ .OriginalReq.URL.Path }}.md)
Let me fire up a VM to simulate the popularity war for ML mindshare. Puts on announcer hat:
--
Ladies and gentlemen! Announcing our World Champion For ML Mindshare 2021! It was a close race, but the dust has settled and the results are in. As of 2021, Pytorch is in the lead by a huge margin! crowd cheers. Everyone loves it, the API is simple, and Facebook is committed to win the war for ML mindshare at any cost. But the key factor was something Google never seems to believe, but decades of history has shown to be crucial and Google keeps ignoring: "Every teenager with a GPU learns Pytorch first, and quickly learns from their friends to stay the hell away from Tensorflow!" Wow! crowd goes wild!
Yes, it was a brave showing by Tensorflow, our 2015 World Champion. But in 2015, the goal was to launch an ML API as quickly as possible, and Pytorch was almost a year behind. The correct decision was to launch, pretend like everyone loves how it works, and fix it later. A bold move! Cloud TPUs bravely did the same thing in 2019, and in 2021 it's working out very well for them, since no one has to pretend anymore. But Tensorflow never could capture that teenager mindshare, and businesses kept pretending to love tensorflow (crowd goes "ohhh,") which left Pytorch wide open for a resounding victory! (cheer)
But, ladies and gents, in 2021, we have a new underdog. Though Pytorch has a massive lead, it has no performance at scale! Crowd boos, jeers, throws 🍅 at announcer. Now, now... please remain calm. holds up shield, 🔪 stabs it with a twanggg; announcer continues
You see, Facebook has gone "all-in" on "GPUs are the future of ML training!" Wow, what a decision by Facebook. It seems reasonable, until you think about literally any aspect of large-scale ML training! What a turn of events. In a huge upset, the Cloud TPU team has brilliantly recognized and exploited this massive oversight by Facebook, and they have Google-scale industrial strength to manufacture a small continent of TPUs! Two continents, actually! Three if you count asia-east-1, which apparently has TPUs!
This turn of events somehow leaves Google in an exceptionally strong position. Literally all they have to do is capture that teenager ML mindshare, and then wait a decade! What do teenagers like? Hot rods and <author is male, idly wonders if female teenagers often like hot rods, decides probably ok>!
That's right, speed. Everything about a teenager is fast: skateboards, relationships, you name it! Facebook has somehow left themselves wide open for attack in this area. And here comes our Cloud TPU underdogs.
Does Facebook have nothing to worry about? For awhile, it was seeming like Google couldn't pull it off! You see, the only possible API that goes fast on TPUs other than Tensorflow, is Jax. And in 2019, it was unclear that Jax would ever see the light of day outside Google. crowd gasps. Yes, you see, Google suffers from being bigger than your mom at scale! It's so massive that there is a whole Internal Google Kingdom, and Jax was only fast inside this kingdom, and Google seemed to be cool with this decision. Uh oh! That means the only way TPUs could win this mindshare battle is if teenagers somehow love Tensorflow. Ain't no teen touchin' no TPU REPL unless it can do basic things without exploding, and no such "TPU REPL" ever existed for Tensorflow! Crowd goes ohhhh.
But, our brave heroes from the Jax team have somehow managed to Pokemon-evolve a unique mutation: design skill! Crowd is now on their feet, chanting "Ooh! Ooh! Ooh!" in unison. That's right ladies and gentlemen, you heard it correctly: Google has the speed, and the design skill, and they've made TPUs available to every teenager with SSH and a credit card! Is it through sheer dumb luck, or is there a mastermind at work, deep in the bowels of Google Kingdom?
Crowd is now brawling, Facebook Red Shirts punching and clawing the Tensorflow Blue shirts.
intercom buzzes, house lights come up, deadpan monotone voice comes on: "Ladies and gentlemen, please, do not attack the tensorflow fans in the crowd. They are all like 40 and work in huge offices, so they'll never win a brawl." house lights dim, crowd quiets down and sits.
Who can say! Whether by luck or by merit, we hereby declare that Jax is the house favorite for the upcoming 2024 ML Mindshare Championship! Both Facebook Reds and Tensorflow Blues now throw knives at announcer.
With Jax on TPU VMs being in an extremely promising position, all Google has to do is "not screw up the design!" Will they pull it off?!
Crowd goes AaaahAaahah, stadium collapses, VM reboots.
--
Hmm. The simulation was inconclusive. I suppose we'll have to wait until 2024 to see how the mindshare wars play out.