This happens a lot. Every once in a while there is a technical “trend” that spikes on Twitter and everyone virtue signals to support. A recent example is Jev, or OpenClaw. Sometimes it’s a straight-up scam, like the old Reflection Llama 70B finetune that made Matt Shumer infamous.
Another one that’s such a distilled example, to me, is TurboQuant. It was this paper Google released on reducing the size of KV cache with a few stacked techniques: https://arxiv.org/abs/2504.19874
Hence the “Quant” part. And it generated a frenzy. CNBC was talking about it. Inference frameworks had to deal with hundreds of vibecoded PRs, everyone wanted it and was Tweeting about how it was going to leap LLMs forward.
…And it was nonsense.
Well. The paper wasn’t nonsense. It was a good paper.
But it mostly wasn’t new, or radical. Some engines (like exllama and some llama.cpp forks) had already implemented most of it (cache quantization, hadamard rotation) for YEARS, and no one cared. It worked alright, it was fine. But by the time Google published the paper, new attention schemes were making it entirely irrelevant anyway, yet people were acting like it was some kind of revolution.
And it may sound different, a linux distro vs a paper. But this is basically Omarchy.
The people hyping it (and funding it) don’t understand the first thing about it. They aren’t actually using it, probably. But it showed up in their Twitter feed, and now they’re compelled to virtue signal with their support for the Next Big Thing.
This happens a lot. Every once in a while there is a technical “trend” that spikes on Twitter and everyone virtue signals to support. A recent example is Jev, or OpenClaw. Sometimes it’s a straight-up scam, like the old Reflection Llama 70B finetune that made Matt Shumer infamous.
Another one that’s such a distilled example, to me, is TurboQuant. It was this paper Google released on reducing the size of KV cache with a few stacked techniques: https://arxiv.org/abs/2504.19874
Hence the “Quant” part. And it generated a frenzy. CNBC was talking about it. Inference frameworks had to deal with hundreds of vibecoded PRs, everyone wanted it and was Tweeting about how it was going to leap LLMs forward.
…And it was nonsense.
Well. The paper wasn’t nonsense. It was a good paper.
But it mostly wasn’t new, or radical. Some engines (like exllama and some llama.cpp forks) had already implemented most of it (cache quantization, hadamard rotation) for YEARS, and no one cared. It worked alright, it was fine. But by the time Google published the paper, new attention schemes were making it entirely irrelevant anyway, yet people were acting like it was some kind of revolution.
And it may sound different, a linux distro vs a paper. But this is basically Omarchy.
The people hyping it (and funding it) don’t understand the first thing about it. They aren’t actually using it, probably. But it showed up in their Twitter feed, and now they’re compelled to virtue signal with their support for the Next Big Thing.