NVIDIA Just Lost Their Lead
Friday, Aug 28, 25 min, about 7 min to read
Theo-brain watched all 25 min
Theo says Nvidia's lead is slipping, and its leverage is memory: the $12,000 to $16,000 RTX Pro 6000 buys 96 GB but runs no faster than the 32 GB RTX 5090. Apple's M5 Ultra Mac Studio, with memory almost six times faster than the DGX Spark, removes most reasons to buy that Nvidia box.
Briefing is enoughThe notes explain why memory capacity matters for local models and how Theo compares Apple’s hardware with Nvidia’s.
Tap any time to jump the video there. Quotes are YouTube's automatic captions, word for word, so some names come out misspelled.
Key takeaways
Nvidia's pricing lever is memory, not raw speed. The RTX 5090 gaming card launched at $2,000 and now sells for about $4,500. The RTX Pro 6000 costs $12,000 to $16,000 and is the same speed or slower. The extra price buys memory: 96 GB versus the 5090's 32 GB.
They now consistently go for around 4,500 because the demand is so absurd. There's also a server card they put out called the RTX Pro 6000 that ranges between 12,000 and $16,000.
Well, it turns out the chip isn't the only thing that matters. The 96 gigs of GDDR7 are what mattered here.
Apple's new M5 Ultra Mac Studio has memory almost six times faster than Nvidia's DGX Spark and only 30 to 40% slower than an RTX 5090. Theo says that kills almost every reason to buy a Spark, a whole category of Nvidia devices.
Almost six times faster than the Spark and only like 30 to 40% slower than the 5090. That is insane. This kills almost all of the reasons you could ever justify buying the DGX Spark, which is a whole category of Nvidia devices killed.
An anonymous model, which turned out to be GLM, handled a huge amount of traffic served entirely on Huawei chips, with no Nvidia hardware in the stack. Part of why this was possible is that it is a relatively small model. Theo calls it genuinely impressive.
…relatively small model, but also because they served it entirely on Chinese chips. Huawei made the chips that they served all this traffic on, and it's still working great. It's genuinely impressive that without any Nvidia in their stack, they were able to do this…
OpenAI's in-house chip, Jalapeno, beat every Nvidia, AMD and Google chip SemiAnalysis tested on several open models, according to that firm's report, which Theo reads through. Theo flags the caveat: only relatively small models have been tested so far.
In general, first energy and trips are not competitive, but Open AI bucks that trend by being industry leading and beating every Nvidia, AMD, and Google chip we have been able to test on multiple top open source models.
All the models they've tested so far are relatively easy to run in terms of size.
Theo's recurring thesis: the future of AI will be fought over electricity, not chips. Without enough energy, none of the chip race matters.
As I've said many times now, the future is going to be fought not on chips, but on electricity. And if we don't have ways to get the energy we need, then none of this ends up mattering in the end.
One skill here is new to you.
Sizing hardware for running models locally New
- What it is
- Running an AI model on your own machine requires the whole model to fit in the graphics chip's memory. If a 33 GB model doesn't fit on a 32 GB card, splitting it across two consumer cards doesn't really work.
- Why Theo cares
- He says the link between two consumer GPUs is far too slow for them to share memory. So the amount of memory, not chip speed, decides what you can run.
- How to try it
- Before buying hardware to run a model, check the model's size against the memory of a single machine.
- Don't plan on linking two consumer graphics cards to fit a bigger model.
- Look at machines with large unified memory (shared by the main processor and graphics), like the Mac Studio.
You can't just plug the chips in together. If you have a model that is 40 gigs, hell, if you have a model that's 33 gigs and it doesn't fit on your 5090, you now have to split the data across the two GPUs, which means ideally you…
Advice
Don't buy Nvidia's DGX Spark expecting to use it as a regular computer. Theo says Linux on its ARM chip is hell, so use it only for running models.
…arm Linux in my life, I promise you you're not going to use this computer as a computer. Linux on arm is hell. Don't bother for this for anything other than inference.
If you were weighing an RTX Pro 6000, or skipped the Spark because it was too slow, Theo says the M5 Ultra gives a much better experience for meaningfully less money.
…all the people who upgraded from a Spark to a RTX Pro 6000 or skipped the Spark because they wanted speeds that weren't trash, they can now get way, way better experiences for meaningfully cheaper by just getting the M5 Ultra instead.
Hot takes
The DGX Spark is a bad product, and Theo's least favorite computer in his apartment.
His reasoning: Its effective memory speed is, in his word, pathetic.
And yes, the DGX Spark's effective memory bandwidth is actually this pathetic. It's a [bleep] chip. It's a [bleep] system. The DGX Spark is my least favorite computer in this apartment.
Nvidia is scared, whatever Jensen Huang says. Theo mocks Jensen's claim that he is very comfortable with the competition: "Definitely not scared at all, are you, bud?"
His reasoning: AI tools now make it easier than ever for others to catch up, and models are useful enough to help chipmakers in their process.
…so, I'm very comfortable with all the competition. >> Okay, and I also >> Yeah. Definitely not scared at all, are you, bud?
All I know is that Nvidia is scared, and they have good reason to be. The AI world constantly is changing, and these tools and technologies make it easier than ever to catch up.
Nvidia's bid for Hugging Face, the main site for sharing open AI models, is a good-faith move to fund open-source AI. It also serves Nvidia's own interest.
His reasoning: Training still happens on CUDA, Nvidia's software platform. So the more businesses train and fine-tune their own models, the more customers Nvidia has for its chips.
…and encouraging more and more people to train because let's be real, training is still happening on CUDA. The more they encourage businesses to try and train and fine-tune and customize things themselves, the more customers they have for their chips, the longer-term their…
AMD, where Theo owns stock, is still far behind and won't catch up for a while.
…buying their stocks, but I have not talked about AMD at any point here cuz I'll be real. I love them to death. They're very behind. It'll be a while before they can catch up to what these other things are that I'm talking about…
How Theo thought about this
When he tweeted about the anonymous model, Theo assumed it was served on Nvidia, because that was what everyone expected, and he got called out. He says he privately suspected Xiaomi was doing it on Chinese chips. It turned out to be GLM, served on those same chips.
…called out here because people were saying only Frontier Labs have this much compute. When I said that, I assumed that they were still serving Nvidia because we basically expected that.
…fair, when I tweeted that, my secret personal belief was that Xiaomi was doing this on Chinese chips somehow. Then it turned out to be GLM shipping again in like a 2-week window, also on those same chips.
Sign in to mark what you've watched and keep notes.
Have a question about this one? Ask about this video
Theo Daily is unofficial and not affiliated with Theo or t3.gg. Written by Claude Opus 5.5 from each video's transcript.
