Compute is arguably the hardest thing to get your hands on right now. Especially as a college student, I didn’t have a budget to rent GPUs, and I wish I could say I had a couple of H100s to borrow. But from a project I was working on at the time, I realized we have GPUs all around us.
GPUs are plentiful if you look in the right places. WebGL, and the modernized WebGPU, power Figma and Google Maps 3D view, along with many of the in-browser games you might have played after Flash was deprecated. That means almost every browser has access to GPU power, including your computer, your phone, and in some cases your smart TV. But for the most part, those machines sit idle for a good chunk of the day, if not most of it. That led me to a question I kept coming back to: what would it take to let these machines participate in training a serious model?
At first, I was shocked nobody had done this. Connect enough machines, divide up the work, and train — it sounded fairly easy. In practice, almost every part of it is difficult.
Your personal device does not behave like a machine in a datacenter, beyond the obvious reasons. Someone closes their laptop, a phone goes to sleep, or a network connection drops. Workers in this kind of network can appear or vanish at any moment, so there is no stable cluster you can reserve and expect to find unchanged when you get back.
The hardware differs across the board as well. Your browser does not have access to CUDA, let alone 80GB of memory. The environment is far more constrained. The training path we found that actually held together was TensorFlow.js running on WebGL, which meant designing around small amounts of memory and FP32 compute.
Then there is the model itself. A seriously sized transformer will not fit inside a single browser tab, so it has to be split across multiple devices. Activations move forward, gradients move backward, and the whole system has to stay busy while that traffic crosses commodity internet connections.
That creates another problem. The obvious way to synchronize distributed training is also the easiest way to destroy a home network. Sending full gradients between every worker after every step simply moves too much data. Most of the optimization has to happen locally, and when machines do communicate, the updates have to be as small as possible.
But the constraint I kept coming back to had nothing to do with training. It had to do with installation.
In an ideal world, I could ask family and friends to join my network and help me train something. There are almost certainly more people willing to lend idle GPU time than there are people willing to install Python, configure CUDA, clone a repository, and work from a terminal. If contributing compute requires setup, most of those machines will never join.
So the interface had to be a URL.
While I was working on this, I reached out to the only person I knew who would be willing to tackle a problem this ridiculous: Ryan Rodriguez. Funnily enough, he was already thinking about the same problem, having watched the Mac mini frenzy sweep the world after the release of OpenClaw. Together, we built Swarm, a distributed training system that runs inside the browser.
You open a tab. Your GPU becomes a worker. There is nothing to install.
Swarm splits transformer layers across GPU-backed Chrome workers and trains them using 1F1B pipeline parallelism. Each stage runs locally through TensorFlow.js inside a Web Worker, while activations and gradients travel directly between peers over WebRTC.
To keep communication from overwhelming the system, we use DiLoCo so that most optimization happens locally and workers synchronize only every few hundred steps. SparseLoCo compresses those updates further, reducing synchronization traffic by roughly 65x.
Memory was another limiting factor. Gradient checkpointing cut activation memory by about 60%, and 8-bit Adam reduced optimizer-state memory by 4x. Together, those changes made it possible to train a model that could not fit inside any individual browser worker by spreading it across several of them instead.
Eventually, we ran a 774M-parameter GPT-2 Large model across six Chrome workers distributed across three AWS regions.
That experiment was the part that made the idea feel less hypothetical.
Swarm is still very much a work in progress, and we would love to have more people contribute to it. The goal is not to prove that a collection of browser tabs can replace a well-run GPU cluster. Datacenters exist for good reasons.
What interests me is the space on the other side of that.
There is an enormous amount of compute sitting in machines that were never designed to be part of a training cluster, owned by people who will probably never SSH into a server. If participating becomes as simple as opening a webpage, that compute suddenly becomes available in a way it was not before.
As the race to build better models continues, I suspect we will see more strange, improvised ways of finding and using compute.
Swarm is one attempt to see how far one of those paths can go.