I've no problem with the intuition. But I would hope for a lot more focus in the marketing materials on proving the (statistical) correctness of the implementation. 15% better inference speed is not worth it to use a completely unknown inference engine not tested across a wide range of generation scenarios.

OK... we need way more information than this to validate this claim! I can run Qwen-8B at 1 billion tokens per second if you don't check the model's output quality. No information is given about the source code, correctness, batching, benchmark results, quantization, etc. etc. etc.

More likely: this is a transitional phase where our previously hard problems become easy, and we will soon set our sights on new and much harder problems. The pinnacle of creative achievement in the universe is probably not 2010s B2B SaaS.

It is entirely possible, however, that human beings will not be the primary drivers of progress on those problems.

I did, and yet I also felt more relaxed reading it than I am reading most blog entries posted on here. I didn't feel like I had to guard against my time being wasted by vacuous LLM fiction.

Being wealthy solves virtually all problems of consumption, so the invisible hand provides new problems to serve the market need. Beautiful, really.