Exclusive: Nvidia CEO says his AI chips are improving faster than Moore’s Law

Nvidia’s AI Chips: Surpassing Moore’s Law and Shaping the Future of Computing

In the ever-evolving world of technology, few names resonate as powerfully as Nvidia. At the helm of this tech giant is CEO Jensen Huang, a visionary leader who recently made headlines with a bold proclamation: Nvidia’s AI chips are advancing at a pace that outstrips the historical benchmark set by Moore’s Law. This statement, made during an interview with TechCrunch, has sparked widespread interest and debate within the tech community.

Moore’s Law, a concept introduced by Intel co-founder Gordon Moore in 1965, has long served as a guiding principle for the semiconductor industry. It predicted that the number of transistors on a computer chip would double approximately every year, leading to a corresponding increase in performance. For decades, this prediction held true, driving rapid advancements in computing power and a significant reduction in costs.

However, in recent years, the pace of progress predicted by Moore’s Law has slowed. This deceleration has led to questions about the future trajectory of computing technology. Enter Nvidia, a company that has positioned itself at the forefront of AI innovation. According to Huang, Nvidia’s AI chips are not only keeping pace with Moore’s Law but are actually surpassing it. The company’s latest datacenter superchip, for instance, is reportedly more than 30 times faster at running AI inference workloads than its predecessor.

Breaking Down the Innovation

Huang attributes this remarkable progress to Nvidia’s holistic approach to innovation. “We can build the architecture, the chip, the system, the libraries, and the algorithms all at the same time,” he explained. “If you do that, then you can move faster than Moore’s Law, because you can innovate across the entire stack.”

This comprehensive strategy allows Nvidia to push the boundaries of what’s possible in AI technology. By simultaneously advancing multiple components of the computing stack, the company can achieve performance gains that would be unattainable through isolated improvements.

The AI Landscape: Progress and Challenges

Huang’s assertions come at a time when the progress of AI is under scrutiny. Some experts have suggested that AI development is stalling, raising concerns about the future of this transformative technology. However, Nvidia’s advancements in AI chips could play a crucial role in reigniting momentum in the field.

Leading AI labs, including Google, OpenAI, and Anthropic, rely on Nvidia’s chips to train and run their AI models. As these chips become more powerful, the capabilities of AI models are likely to expand, potentially overcoming current limitations.

Huang has previously suggested that the AI industry is experiencing what he calls “hyper Moore’s Law,” a period of accelerated progress that exceeds traditional expectations. He rejects the notion that AI development is slowing, instead proposing that there are now three active AI scaling laws: pre-training, post-training, and test-time compute.

• Pre-training: The initial phase where AI models learn patterns from vast datasets.
• Post-training: Fine-tuning AI models using methods like human feedback.
• Test-time compute: Occurs during the inference phase, allowing AI models more time to “think” after each query.

These scaling laws, according to Huang, are driving down the cost of AI inference while simultaneously boosting performance. This mirrors the impact Moore’s Law had on computing costs in its heyday.

Nvidia’s Market Position and Future Prospects

Nvidia’s rise to prominence has been fueled by the AI boom, and the company has become one of the most valuable entities on the planet. This success naturally benefits Huang’s narrative of rapid AI progress. However, the company’s dominance is not without challenges.

As tech companies shift their focus from training AI models to inference, questions have arisen about the continued relevance of Nvidia’s high-priced chips. AI models that utilize test-time compute are currently expensive to operate, raising concerns about their accessibility. For instance, OpenAI’s o3 model, which employs a scaled-up version of test-time compute, incurred costs of nearly $20 per task to achieve human-level performance on a general intelligence test. In contrast, a ChatGPT Plus subscription offers a month of usage for the same price.

During his keynote at CES, Huang showcased Nvidia’s latest datacenter superchip, the GB200 NVL72, which he held up like a shield. This chip is 30 to 40 times faster at running AI inference workloads than the company’s previous best-selling chips, the H100. Huang believes that this leap in performance will eventually make AI reasoning models like OpenAI’s o3 more affordable.

The Path Forward: Performance and Affordability

Huang’s focus remains on creating more performant chips, which he argues will lead to lower prices over time. “The direct and immediate solution for test-time compute, both in performance and cost affordability, is to increase our computing capability,” he stated. In the long term, AI reasoning models could be leveraged to generate better data for the pre-training and post-training phases of AI development.

The past year has seen a significant reduction in the cost of AI models, thanks in part to breakthroughs from hardware companies like Nvidia. Huang anticipates that this trend will continue, even as the initial versions of AI reasoning models remain costly.

Looking back over the past decade, Huang claims that Nvidia’s AI chips have improved by a factor of 1,000. This rate of advancement far exceeds the pace set by Moore’s Law, and Huang sees no signs of this momentum slowing down.

As Nvidia continues to push the boundaries of AI technology, the implications for the broader tech industry are profound. The company’s innovations have the potential to reshape the landscape of computing, driving new capabilities and applications that were once thought to be the stuff of science fiction.

For those interested in staying abreast of the latest developments in AI, TechCrunch offers an AI-focused newsletter. Sign up here to receive it in your inbox every Wednesday.

Maxwell Zeff, a senior reporter at TechCrunch, specializes in AI and emerging technologies. With a background at Gizmodo, Bloomberg, and MSNBC, Zeff has covered the rise of AI and the Silicon Valley Bank crisis. Based in San Francisco, he enjoys hiking, biking, and exploring the Bay Area’s food scene when not reporting.