Most workers only spend ~1/3 of their time working. If you’re running GPU heavy automated workflows, you’re usually running batches before shifts start in the morning.
So 50% utilization seems about right, probably higher than I expect. The real question is what is their peak utilization rate, because being capped when you need it most is more important than running the things 24/7/365.
Even if you sell GPU compute, you’re still pricing it amortized based on average utilization, so this isn’t a deal breaker depending on the business model.
Meanwhile, open models have gotten quite good, and if you have an enterprise grade GPU you can run a lot of the good ones. I run Qwen 3.8 125b on my home server and it’s acceptable. The value of owning your own GPU is getting stronger in my opinion. I guess I’ll keep an eye out on used server parts or something.
As much as I think this is a bubble, fueled by effectively extortion from Nvidia and the memory makers, but initiated by Scam Altman, this isn’t the needle. The needle is the data centre funding.
Also, on open models: I can run Deepseek V4 Flash at 20 tokens/s on a single 3090 desktop, a Ryzen 7800, and 128GB RAM, with measurably low quantization loss. And 4.1 once the engine updates to support it.
That was eye-opening to me. Thats faster than some cheap APIs, and higher quality too. Once companies figure that out, what are they gonna do with all those idle datacenters?
Dev time is spent on mostly-idle GPU instances too. Code, test, doesn’t work, edit, test launch again. As one sometimes needs the full environment to even test whatever they’re ultimately trying to deploy.
And yeah… like I said above, I’m more afraid of actual utilization under load. Some deployments are so inefficient it makes me want to pass out.
Most workers only spend ~1/3 of their time working. If you’re running GPU heavy automated workflows, you’re usually running batches before shifts start in the morning.
So 50% utilization seems about right, probably higher than I expect. The real question is what is their peak utilization rate, because being capped when you need it most is more important than running the things 24/7/365.
Even if you sell GPU compute, you’re still pricing it amortized based on average utilization, so this isn’t a deal breaker depending on the business model.
Meanwhile, open models have gotten quite good, and if you have an enterprise grade GPU you can run a lot of the good ones. I run Qwen 3.8 125b on my home server and it’s acceptable. The value of owning your own GPU is getting stronger in my opinion. I guess I’ll keep an eye out on used server parts or something.
As much as I think this is a bubble, fueled by effectively extortion from Nvidia and the memory makers, but initiated by Scam Altman, this isn’t the needle. The needle is the data centre funding.
Also, on open models: I can run Deepseek V4 Flash at 20 tokens/s on a single 3090 desktop, a Ryzen 7800, and 128GB RAM, with measurably low quantization loss. And 4.1 once the engine updates to support it.
That was eye-opening to me. Thats faster than some cheap APIs, and higher quality too. Once companies figure that out, what are they gonna do with all those idle datacenters?
Dev time is spent on mostly-idle GPU instances too. Code, test, doesn’t work, edit, test launch again. As one sometimes needs the full environment to even test whatever they’re ultimately trying to deploy.
And yeah… like I said above, I’m more afraid of actual utilization under load. Some deployments are so inefficient it makes me want to pass out.
Yeah, I definitely find this.
Building models can be a weird day when you’re 50% waiting for models to build, but you can’t really focus on something else.