This article is based on the discussion from the Lex Fridman Podcast: Listen to the episode here.

In this conversation, Lex Fridman speaks with Dylan Patel and Nathan Lambert about the latest developments in AI, focusing on DeepSeek, China’s AI ambitions, NVIDIA’s role in the global AI race, and the future of AI megaclusters. The discussion covers AI efficiency breakthroughs, the geopolitical battle for semiconductor dominance, and the key players shaping the future of artificial intelligence.

Artificial intelligence is evolving at breakneck speed, but not in the way most people expect. 

The headlines scream about bigger models, more parameters, and mind-blowing new capabilities. But beneath all the hype, a quieter revolution is happening: making AI more efficient, cost-effective, and geopolitically strategic. 

And if you’re paying attention, you’ll notice something big—China’s AI scene is accelerating in a way that few saw coming.

Smarter AI Models: The Mixture-of-Experts Approach

It’s no secret that large language models (LLMs) require obscene amounts of computing power. Training one from scratch can cost hundreds of millions of dollars, and running it? That’s another bottomless pit of expenses. The future of AI won’t just be decided by who has the best algorithms—it will be determined by who figures out how to build powerful AI models without burning through a nation’s GDP in server costs. And that’s exactly where a major shift is taking place.

Instead of cranking up the size of models endlessly, engineers are getting smarter about how these systems actually function.

 Enter mixture-of-experts models. 

Imagine you’re assembling a panel of specialists, but instead of making every expert work on every question, you activate only the ones who are best suited for the job. That’s exactly how these new models work. Instead of having a gigantic neural network processing every single request in full force, only a few “expert” sub-models light up at any given time. The result? Massive gains in efficiency—cutting down on both computation and energy consumption—without sacrificing intelligence.

This isn’t just theoretical.

 Some of the most recent AI models coming out of China are proving that this method works in real-world applications. By fine-tuning how computation is allocated, they’re getting performance that rivals the best Western models, but at a fraction of the cost.

The Low-Level Optimization Breakthrough

That brings us to the next big shift: low-level optimization.

 Most AI companies rely on existing software frameworks like CUDA, developed by NVIDIA, to train their models. But these frameworks weren’t designed for peak efficiency; they were built for general usability. The companies making real breakthroughs right now are the ones that are re-engineering the way AI utilizes hardware. 

They’re not just using GPUs; they’re squeezing every last drop of performance out of them, rewriting the rules of GPU programming to make computations faster, leaner, and significantly cheaper.

The Race for Compute Power

And when it comes to hardware, size matters.

 Some of the biggest players in AI operate vast compute clusters, essentially giant server farms packed with tens of thousands of GPUs. For context, Meta has a training cluster in the ballpark of 60,000 to 100,000 H100-equivalent GPUs. China’s newest AI models are being trained on clusters of roughly 50,000 GPUs—enough firepower to keep up with the best in the world. 

The sheer scale of these investments shows that AI isn’t just a tech industry arms race anymore; it’s a national priority.

The Geopolitical Chessboard of AI

Of course, the AI landscape isn’t just about who can train models the fastest—it’s also about who can actually get the necessary hardware. And that’s where geopolitics crashes into the picture. 

The U.S. has implemented increasingly strict export controls on high-end AI chips, restricting their sale to China. NVIDIA has responded by making cut-down versions of its GPUs specifically for the Chinese market. 

This means that while Chinese AI firms can still train massive models, they’re forced to do so with slightly less powerful hardware. But here’s the interesting part: these restrictions are pushing China’s AI companies to get even more efficient. Instead of brute-forcing their way to the top with unlimited compute, they’re being forced to innovate. 

They’re optimizing software, rethinking architectures, and finding ways to extract performance gains that Western companies haven’t even considered yet. The constraints are, ironically, making them stronger competitors.

Are We Approaching AGI?

And then there’s the big-picture question that looms over everything: how close are we to Artificial General Intelligence (AGI)? 

Some argue that today’s models are already showing glimpses of it. The ability of modern AI to adapt across multiple tasks—reasoning, problem-solving, creativity—suggests that we might be further along the road to AGI than people assume. 

And with China’s manufacturing power, it’s positioned uniquely to accelerate progress. While the U.S. and Europe dominate AI software, China is the backbone of AI hardware. It controls a massive chunk of the world’s semiconductor supply chain, giving it an edge in mass-producing the chips needed to train and deploy AI systems at scale.

Taiwan: The Center of the AI Universe

This brings us to Taiwan, the unsung hero of the entire global AI race. The Taiwan Semiconductor Manufacturing Company (TSMC) produces the world’s most advanced AI chips. Nearly every AI model—whether it’s OpenAI’s GPT, Google’s Gemini, or China’s latest deep-learning systems—relies on chips made by TSMC.

 If anything were to disrupt TSMC’s supply chain, the ripple effects would be felt across every industry, from smartphones to supercomputers. AI isn’t just about coding breakthroughs anymore—it’s about global supply chains, strategic alliances, and geopolitical stability.

The Future of AI: Efficiency Wins

So where does all of this leave us? 

AI is shifting from a brute-force competition of “who has the biggest model” to a much more strategic game of efficiency, hardware mastery, and geopolitical positioning. The companies (and countries) that figure out how to optimize, scale, and deploy AI efficiently will be the ones that dominate the future. And if you’re not paying attention to these shifts, you’re missing the real story of where AI is headed.

This isn’t just about flashy demos or record-breaking benchmark scores.

 It’s about who actually wins the AI race in the long run—who builds models that aren’t just powerful but also practical, scalable, and sustainable. And that race? It’s getting more interesting by the day.