11 appearances on the backlist front page in the last 30 days.
I was thinking about Vector Sets and the Redis approach to this stuff in general. Now that the hype with RAG is gone, I'm 100% sure I made the right call there, saying: RAG will mostly go away, but raw vector search is a useful, fundamental
Apparently it is possible to go much faster than my 13.5 t/s, and this is a very good news. It means 4-bit GLM 5.2 will be usable on Mac Studios M3 Ultras. The problem is that it is literally hardware you *can not buy* anymore.
A deep data-structure optimization like this can materially change the cost profile of real production workloads
I'm using the Unsloth 4-bit weights to integrate interence into DwarfStar. If the experiment works well I'll specialize the quants with an optimizer for the best setup. My target is however the 512GB M3 ultra and in distributed inference 4
Another important thing: Chinese models are not strong because they distill US models. Distillation of models via API is *impossible*. If somebody tells you the contrary, they don't understand machine learning:
DwarfStar now supports SSD streaming in the DGX Spark and Strix Halo, not just in Metal. You can run the Q4 quants at decent speed, and even DeepSeek v4 PRO at low speed, or you can run Q2 Flash if you have less than 128GB.
People often have this idea of Chinese models being N months behind US models. This mental model is not helpful to predict the future. The lag is due to compute deficit, so the playfield is that. It's not by chance that OpenAI and Anthropic
For days, many folks here are citing DeepSWE as the benchmark that restores reality only because it shows GPT 5.5 on top. But actually, it almost gets a single entry right: the top one, and all the rest is shuffled.
A 1.6T-parameter model running on consumer Apple hardware via SSD streaming changes the practical boundary of local model experimentation
SSD Streamed Dwarf Start by @anemll , cool demo! Official implementation of streaming is arriving too. DeepSeek Flash should run at ~14 t/s on MacBook m5 max 64GB, DeepSeek PRO should run at 4 t/s on MacBook m5 max 128GB. Those are genera
DwarfStar prefill is a hell faster after merging https:// github.com/antirez/ds4/pu ll/264 …, I need to update the README benchmarks as they no longer mean most :D after +40% prefill speed boost.