"CUDA. This name was not chosen by me. It was one of the earliest architects of Nvidia--
An old professor who has taught parallel computing courses at Stanford for many years--
It was casually suggested at a discussion meeting.
Compute Unified Device Architecture, Compute Unified Device Architecture.
It sounds very academic, but the core idea is actually very simple:
Let the GPU not only render pixels, but also do any large-scale parallel computing tasks.
This idea sprouted in 2003, when Nvidia had just survived the NV30 recall crisis in 2002.
The cash on hand was barely enough to last two quarters. I told the core team that we wanted to build a general computing platform.
They thought I was crazy.
The only value of the GPU at that time was graphics rendering--
Games, CAD, video editing, these are all sources of revenue for Nvidia.
You told me that you want GPUs to do scientific computing, climate simulation, and oil exploration?
People in those fields don't even know what a GPU is.
But one thing I learned at LSI Logic:
The reason why ASIC is cheap is that it only specializes in one area and uses dedicated hardware to accelerate tasks that general-purpose processors are not good at -
But the price is that each ASIC needs to be designed and taped out separately, which is extremely expensive.
The advantage of a GPU is that it is neither completely general-purpose like a CPU, nor completely specialized like an ASIC.
It's 'massively parallel' - it can do the same operation on millions of pixels at the same time.
This computational characteristic is naturally suited to graphics rendering, but is also suitable for many other tasks."
He picked up the chopsticks again and drew a simple comparison picture on the bar.
"On the left is the CPU - several cores processing complex logic;
On the right is the GPU - tens of thousands of cores processing simple operations at the same time.
What CUDA does is abstract a layer of software interfaces on these tens of thousands of cores, allowing developers to schedule thousands of threads using C.
This idea seems logical today, but in 2006, only one team in the industry believed it could succeed.
The cost of developing CUDA is not billions US Dollar--
It was Nvidia that invested all the profits at that time, which lasted for nearly ten years.
Wall Street's analysts ask me the same question every time we call:
When will CUDA generate revenue? I said I don't know."
"You don't know?"
"I don't know."
Huang Renxun put down his chopsticks and turned his head to face Zhao Chen.
"No one knows. When CUDA was released in 2006, we conducted an internal market survey.
The conclusion is that there is no possibility of large-scale commercial application in the next five years.
But I don't care. What I see is a trend - computing is moving from serial to parallel.
This trend was not created by Nvidia, but is determined by physical limits.
Moore's Law will hit the power consumption wall sooner or later, and the transistor density of the CPU cannot increase infinitely.
But humankind's demand for computing power will not stop--
Weather forecast, drug molecule simulation, geological exploration, movie special effects,
Computational needs in these areas will always exceed the performance growth of single-core CPUs.
If CPU is a 'luxury product' of computing power--
It can do anything but is extremely expensive. GPU is a 'daily commodity' of computing power--
Granularize computing resources so that everyone who needs computing power can afford it.
CUDA is the distribution network for this 'daily necessities'.
Without CUDA, the parallel computing power of GPUs is just a bunch of packages without a zip code.
With CUDA, every developer can accurately call any one of thousands of threads.
It's like having a package delivered to every mailbox."
He picked up the tea cup and took a sip, seeming to be sorting out his next words.
"I never say this on the Financial Report call.
Analysts don't want to hear about physical limits and parallel computing paradigms -- they just want to know whether Data Center can grow faster than expected next quarter.
But there are no analysts in the room today.
You ask me why I dare to take a heavy position in Nvidia at the most difficult time--
I still want to ask you again, why do you dare?"
Zhao Chen Put the edamame shells next to the plate and wipe your hands with a hot towel.
The light circle of the chandelier above the bar just fell on the tea cups and sashimi plates between the two of them.
Take the picture of the parallel computing schematic diagram Huang Renxun just drew and see it clearly.
"Because when I was studying you, I saw a detail.
In 2002, Nvidia issued a large-scale product recall due to manufacturing issues with the NV30 chip, and its stock price fell by 90%.
What were you doing that year? You were renegotiating the foundry agreement with TSMC,
Reconstructed the entire Supply Chain from beginning to end.
During the financial crisis in 2008, Nvidia's stock price fell by 90%. What are you doing?
You are extending CUDA from a graphics rendering tool to a general parallel computing platform,
At a time when everyone was shrinking, we increased our investment in the software ecosystem.
In 2018, the mining boom subsided, channel inventories piled up, and the stock price fell from 292 to more than 120. What are you doing?
You are accelerating the launch of the new generation Turing Architecture and integrating ray tracing and AI inference capability into consumer GPUs.
At the same time, the Data Center business has grown from zero to more than games.
I looked through your resume over the past twenty years and found a pattern -
Every time you encounter a crisis, your first reaction is not to shrink your defenses.
Instead, use the crisis as an accelerator to leave behind your competitors.
The NV30 crisis allows you to reconstruct Supply Chain,
The 2008 financial crisis led you to establish CUDA ecosystem,
The 2018 mining boom allows you to complete Data Center's overtaking of the game.
You're not playing defense -- you're playing offense instead of defense."
Huang Renxun looked at Zhao Chen with the same eyes as when he reviewed the independent Model verification data in front of the whiteboard in the afternoon.
"You broke down my resume more clearly than my own year-end summary.
But these are not the results - what is the most critical judgment in your research?
What makes you dare to bet at the worst possible time?"
"It's the underlying logic.
After repeated verification, I found that Nvidia has only two deepest moats.
The first one is CUDA ecosystem developer.
The second one is you."
Huang Renxun was silent for a long time.
He gently placed the tea cup in his hand on the bar, and the bottom of the cup hit the wooden table with a very slight muffled sound.
"Those three things you just said--
Denny's restaurant in 1993, the technical review in 1993, and the sentence at the CUDA press conference in 2006--
This is the first time I've sat down in many years,
To a company's external shareholders, tell these things from beginning to end.
If this moat can really hold Nvidia for decades,
Then the first shovel to dig it was not the CUDA conference in 2006--
It was the moment I told two partners at a Denny's restaurant in 1993 that the chip industry would completely change the world."
He picked up the tea cup, drank the last sip of brown rice tea, and then stretched out his hand.
Zhao Chen holds his hand.