Also, on open models: I can run Deepseek V4 Flash at 20 tokens/s on a single 3090 desktop, a Ryzen 7800, and 128GB RAM, with measurably low quantization loss. And 4.1 once the engine updates to support it.
That was eye-opening to me. Thats faster than some cheap APIs, and higher quality too. Once companies figure that out, what are they gonna do with all those idle datacenters?
Also, on open models: I can run Deepseek V4 Flash at 20 tokens/s on a single 3090 desktop, a Ryzen 7800, and 128GB RAM, with measurably low quantization loss. And 4.1 once the engine updates to support it.
That was eye-opening to me. Thats faster than some cheap APIs, and higher quality too. Once companies figure that out, what are they gonna do with all those idle datacenters?