Huang Renxun's latest views

It is undeniable that we have now made great progress in technology, not only in large language models, but also in infrastructure.

In fact, I’ve seen a lot of attention in the community turn to the GPUs and hardware benchmarks we’re working on. Models are still evolving very quickly, of course, but it’s worth focusing on the bare metal side as well.

With that in mind, there’s a lot to unpack in a recent episode of the No Priors podcast, in which Nvidia CEO Jensen Huang contributed some insights.

It’s definitely Nvidia’s time, as the company has surpassed Apple and Microsoft to become the world’s largest tech company. The top data centers being built right now are filled with Nvidia’s products and high-end GPUs, and Huang had a lot to say about this transition.

“The world has changed. The scale has changed,” he said, referring to the parallelism and co-design advances in clusters.

Huang reviewed some of his thoughts on the history of hardware development with the host, talking about how Moore's Law, which is widely accepted in the industry, has been going on for many years, with people increasingly predicting that transistors and processing power will double every year.

For reference, ChatGPT explains Moore's Law to us like this:

"Moore's Law is an observation proposed in 1965 by Intel co-founder Gordon Moore. The law predicts that the number of transistors on a microchip will double approximately every two years, resulting in a corresponding increase in computing power and a decrease in relative cost. This trend has driven rapid advances in computing power and has been the fundamental principle guiding the semiconductor industry for decades."

This is a bit ironic, since the company Nvidia beat was primarily Intel. But I digress…

Huang said the pace of change is now even faster and we are seeing what is called "super Moore's Law."

To achieve this, planners must look at architecture and systems together in a “full stack approach,” he suggested.

“You can think of the network as a computational fabric, and you push a lot of work into the network, and a lot of work into the fabric,” he said. “So you’re doing compression at a very large scale.”

Huang also mentioned work on adapting language models and neural networks that are enabling scaling of reasoning times and dynamically generating chains of thoughts and reasoning.

“We have to invent something new,” he said, noting that low latency and high throughput are inherently contradictory. He also mentioned that the industry may enter an era of diversification, with language models of various sizes, including tiny language models, or TLMs.

“You’re still going to create these incredible cutting-edge models,” he said. “They’re going to be used for groundbreaking work. You’re going to use them to generate synthetic data. You’re going to use these models, the big models, to teach the small models, and refine the small models.”

Huang then revealed some very interesting elements of the company’s collaboration with Elon Musk to build the X.AI data center.

He gave Musk high praise for the quick implementation and decision making that underpinned this supercluster, as a lot of people were doing everything very quickly.

“It really is a testament to his willpower and how he was able to think through mechanical problems, electrical problems, and overcome obstacles that were obviously extraordinary,” Huang said.

He also revealed that stakeholders used the digital twin process to help implement the system.

"We simulated all the network configurations, pre-staged everything as a digital twin. We pre-staged the entire supply chain. We pre-staged all the wiring of the network. We even set up a small version, kind of like, you know, just the first instance of it... so when everything came up, everything was ready to go. All the exercises were done, all the simulations were done. And then there was the massive integration... a huge team of humans came together to connect everything 24/7, and within a few weeks, the cluster was up."

What was so special about this project? He said the project schedule was “unusual” due to the sheer amount of equipment.

AI chip designer?

Huang confirmed that the company uses AI entities as chip designers and software engineers.

"We couldn't have built Hopper without them," he said. "They can explore a much larger space than we can. They have unlimited time to explore space."

Huang Renxun reviewed the rocket-like growth of Nvidia's market value in the past few years and talked about the impact it has had on the company.

“A company can’t change as fast as its stock price,” he said, stressing the value of thoughtful consideration and understanding of what’s actually happening in the industry to drive change.

He said he had come to realize that Nvidia had revolutionized computing for the first time in about 60 years, driving down the marginal cost of computers until they could do tasks on their own.

This is a game changer - and that's an understatement! I've been following Claude and OpenAI's o1 and Orion, and one thing is for sure - when we talk about market effects, things will never be the same.

I’ll likely report more of Huang’s comments elsewhere as he digs deeper into new features for systems to truly complete tasks autonomously — giving AI a bigger role in company processes with minimal supervision.

This is where you really start to see the effects of “agent AI” — AI entities will take on engineering and design roles and get credit for the results.

There is no doubt that this is a time of incredible change.

Recommended Reading