Article Source: New Smart Element

The popularity of AI has caused Nvidia's market value to soar, successfully entering the trillion-dollar club. As the soul of the company, what stories has Huang Renxun experienced on his journey? How did this leather-clad guest build his graphics card empire step by step?

Image Source: Generated by Wujie AI

From the neural network AlexNet, to ChatGPT, and then to the explosive growth of generative AI, Nvidia's GPU has played an indispensable role.

In this AI gold rush, Nvidia's market value has soared, successfully entering the trillion-dollar club, becoming the world's sixth largest market capitalization company.

If we talk about the success behind Nvidia, it is inseparable from this soul figure - Huang Renxun.

Everyone knows the stories of tech giants like Jobs and Gates, but the reclusive Old Huang, apart from his leather jacket, has few known experiences.

This time, The New Yorker's latest interview delves deep into Old Huang's entrepreneurial journey, management style, and how he led Nvidia to success.

The Early Years of Old Huang

Huang Renxun was born in Taiwan in 1963. At the age of nine, he and his brother were sent to the United States to study at the Oneida Baptist Institute in Kentucky.

Huang Renxun lived with a 17-year-old roommate, teaching him to read in exchange for the roommate teaching him how to bench press. Every night before bed, Huang Renxun had to do one hundred push-ups.

Because he was too young, Huang Renxun could not attend classes at the school, so he went to a nearby public school.

At the time, the principal introduced this small, long-haired, heavily accented Asian immigrant to everyone. However, it was precisely because of these traits that Huang Renxun was bullied by his classmates.

A few years later, Huang Renxun's parents were allowed to enter the United States and settled in Oregon, where the brothers reunited with their parents.

Huang Renxun excelled in high school and was a nationally ranked table tennis player. He participated in the school's math, computer, and science clubs, skipped two grades, and graduated at the age of 16. - "But I didn't have a girlfriend."

Later, Huang Renxun entered Oregon State University, majoring in electrical engineering.

In his introductory class, his lab partner was Lori Mills, serious, dorky, with long brown curly hair.

According to Huang Renxun's recollection, there were 250 students in the electrical engineering major, and there were only about three female students. The boys vied for Mills' attention, and Huang Renxun felt at a disadvantage. "I was the youngest kid in the class, looking like I was only about 12 years old."

• However, every weekend, Huang Renxun would call Mills and pester her to do homework together.

"I wanted to leave a deep impression on her, not because of my appearance, but because of my strong ability to complete assignments."

After six months of homework, Huang Renxun mustered the courage to ask her out. She accepted the invitation.

After graduation, Huang Renxun and Mills found jobs as microchip designers in Silicon Valley - "She actually made more money than I did."

Later, the two got married, and a few years later, Mills left her job to take care of their children. By then, Huang Renxun had already started running his own department and was studying for a master's degree at Stanford University at night.

Three-Person Entrepreneurship, Originating from a Restaurant

In 1993, he co-founded Nvidia with two senior microchip designers, Chris Malachowsky and Curtis Priem.

Malachowsky and Priem wanted to design a graphics chip. Initially, they named the company NVision, but later found out that this name was already being used by a toilet paper manufacturer.

Huang Renxun suggested using Nvidia, derived from the Latin word "invidia," meaning "envy." He chose Denny's as the place to conduct business because it was quieter than home and had cheap coffee.

He had worked at a chain restaurant in Oregon in the 1980s. "I found that my thinking was most active in adversity."

Huang Renxun liked video games and believed that the market needed better graphics chips. At that time, artists began assembling three-dimensional polygons using shapes called "primitives" instead of hand-drawing pixels, which saved time and effort but required new chips.

Nvidia's competitors used triangles as primitives, but Huang Renxun and his partners decided to use quadrilaterals. - However, it was later proven to be a mistake that almost destroyed the company because shortly after Nvidia released its first product, Microsoft announced that its graphics software would only support triangles.

Due to a lack of funds, Huang Renxun decided to revert to the traditional triangle method. In 1996, he laid off half of Nvidia's more than one hundred employees and then bet the remaining funds on untested microchip production, unsure if these microchips would be successful.

• "The probability of success and failure is fifty-fifty, but we were going to go bankrupt anyway."

When the product named RIVA 128 was launched, Nvidia's funds were only enough to pay one month's salary. But this gamble paid off, and Nvidia sold one million RIVA units within four months.

Huang Renxun encouraged his employees to continue shipping with a sense of despair. To this day, whenever he addresses the employees, his opening line is "Our company is going to go bankrupt in thirty days." This phrase remains the company's unofficial motto.

Nvidia's headquarters in Santa Clara has two huge triangular buildings. Each building is a miniature version of this shape, from the sofas and carpets to the splash guards on the urinals throughout the building.

Each building has a bar on the top floor, and the company encourages employees to use the office as a flexible space for dining, coding, and socializing. If an employee eats at the conference table, AI can dispatch a cleaner within an hour. Before the stock price rose, Nvidia was already rated as one of the best workplaces in the United States.

In the standard computer architecture, most of the work is done by a microchip called the central processing unit (CPU). For decades, Intel has been the main manufacturer of CPUs and has repeatedly tried to force Nvidia out of the market.

Huang Renxun described the relationship between Nvidia and Intel as a "Tom and Jerry relationship" - every time they get close, we grab the chips and run.

In response, Nvidia took a different approach. In 1999, shortly after the company went public, it launched a graphics card called GeForce.

Unlike general-purpose CPUs, GPUs break down complex mathematical tasks into small calculations and process them all at once using parallel computing. The function of a CPU is like a delivery truck, delivering one package at a time, while a GPU is more like a fleet of motorcycles weaving through the city.

The GeForce series was a success. The "Quake" video game series drove its popularity, using parallel computing to render monsters that players could shoot with a grenade launcher.

The "Quake" series also introduced a multiplayer mode, and PC gamers would buy new GeForce graphics cards with each upgrade to gain an advantage.

In 2000, Ian Buck, a graduate student in computer graphics at Stanford University, connected 32 GeForce graphics cards and used 8 projectors to play Quake. - This was the first 8K resolution gaming machine, taking up an entire wall. "It was beautiful."

The GeForce graphics cards came with a primitive programming tool called "shaders." With funding from the U.S. research agency DARPA, Ian Buck hacked into the shaders, accessed the underlying parallel computing circuits, and turned the GeForce into a low-cost supercomputer.

Shortly thereafter, Ian Buck started working at Nvidia.

Since 2004, Buck has been responsible for the development of Nvidia's supercomputing software package, CUDA. Huang Renxun's vision was to make CUDA run on every GeForce graphics card.

While Buck developed the software, Nvidia's hardware team began allocating space for supercomputing on microchips. Nvidia's chief chip engineer, Arjun Prabhu, likened microchip design to urban planning, with different areas of the chip dedicated to different tasks.

At the end of 2006, when CUDA was released, Wall Street's reaction was astonishment. Huang Renxun brought supercomputing to the masses, but the masses did not show that they needed it.

Ben Gilbert, host of the popular Silicon Valley podcast "Acquired," said that Nvidia spent billions targeting a niche in academic and scientific computing that was not a big market at the time.

By the end of 2008, Nvidia's stock price had fallen by 70%.

Huang Renxun believed that the existence of CUDA would expand the field of supercomputing. But this view was not widely accepted.

2 Nvidia Graphics Cards, 1 CUDA Architecture, Igniting Neural Networks

At the beginning of the 20th century, AI was a completely obscure field. Progress in artificial intelligence in areas such as image recognition and speech recognition had been stagnant.

In this unpopular academic field, using "neural networks" (computational structures inspired by the human brain) to solve problems was not favored by many computer scientists.

At the time, deep learning researcher Bryan Catanzaro advised Old Huang, "Don't study neural networks. Because at the time, people thought it was outdated and didn't work."

Catanzaro also referred to researchers who continued to study neural networks as "prophets in the wilderness."

One of these prophets was retired University of Toronto professor and AI pioneer Geoffrey Hinton.

In 2009, Hinton's research group used Nvidia's CUDA platform to train a neural network to recognize speech.

Surprisingly, the quality of the research results surprised Hinton himself, and he reported the results at a conference that year. Then, he proactively contacted Nvidia.

"I sent an email saying, 'Listen, I just told over a thousand machine learning researchers that they should go buy Nvidia graphics cards. Can you send me one for free?'"

However, Nvidia's final response was only one word: "No."

Despite being rebuffed, Hinton encouraged his students to use CUDA, including his proud disciple Alex Krizhevsky.

In 2012, Krizhevsky and research partner Ilya Sutskever, in a tight budget situation, purchased 2 GeForce graphics cards - GTX 580 GPU.

Then, Krizhevsky began training a visual recognition neural network - AlexNet - on Nvidia's parallel computing platform, inputting millions of images within a week.

Hinton recalled, "Those two GPUs in his bedroom were buzzing non-stop, and you can imagine that his parents must have paid a considerable amount of electricity bills."

After that, Krizhevsky and his friends entered the annual ImageNet competition with AlexNet and won the championship, giving birth to the first deep convolutional neural network model.

The capabilities of the GeForce graphics cards surprised both Ilya and Krizhevsky.

In fact, earlier in 2012, Google researchers Andrew Ng and Jeff Dean trained a neural network that could "recognize cats."

Google's work used about 16,000 CPUs, while Sutskever and Krizhevsky achieved "world-class" results with just 2 Nvidia circuit boards.

AlexNet correctly identified images of scooters, leopards, and container ships.

However, AlexNet scored so high in the competition that the organizers initially suspected Krizhevsky of cheating in some way. Because neural networks were not popular at the time, Ilya and Krizhevsky were the only team using this technology.

Hinton said, "It was a big bang moment. This is a paradigm shift."

Since its creation in 2021, the 9-page masterpiece "ImageNet Classification with Deep Convolutional Neural Networks" has been cited over 140,000 times, becoming an important milestone in computer history.

Krizhevsky pioneered many important programming techniques, but his main discovery was that "dedicated GPUs can train neural networks 100 times faster than general-purpose CPUs."

Hinton added, "Without CUDA, machine learning would be very difficult."

In the following years, every participant in the ImageNet competition used "neural networks." By the mid-2020s, neural networks trained on GPUs achieved an image recognition accuracy of 96%, far surpassing humans.

Over the past decade, Huang Renxun has achieved great success in promoting the popularity of supercomputing and GPUs.

He said, "In fact, they can now solve completely unstructured computer vision problems, so what else can you teach them?"

Old Huang's Bet Again: Nvidia Upgrades from a Graphics Company to an "AI Company"

The answer seems to be: anything!

Huang Renxun concluded that neural networks would completely change society, and he could use CUDA to dominate the necessary hardware market.

At the time, he announced another bet on the company.

He sent an email on Friday night, "Everything will turn to deep learning, we are no longer a graphics company. Starting next Monday, we are an artificial intelligence company."

Nvidia's transformation, literally, was that fast.

Just as Huang Renxun sent that email, he found Nvidia's chief AI researcher Catanzaro and conducted a thought experiment.

Catanzaro said, "He asked me to imagine bringing all 8,000 Nvidia employees into the parking lot, and then I could freely choose anyone from the parking lot to join my team."

H100, Mining the Big Models

After the success of AlexNet, venture capitalists began pouring large sums of money into AI.

Marc Andreessen of Andreessen Horowitz said in 2016, "We have been investing in many startups applying deep learning to many fields, and each company has effectively built on Nvidia's platform."

Around that time, Nvidia delivered the first dedicated AI supercomputer, DGX-1, to the research team at OpenAI.

Huang Renxun personally brought the DGX-1 to OpenAI's office, and it was unboxed by then-chairman Musk.

In 2017, Google researchers proposed the Transformer neural network architecture.

The following year, OpenAI researchers used Google's framework to build the first "Generative Pre-trained Transformer."

The GPT model was trained on Nvidia's supercomputers using a large amount of text data and learned how to establish human-like connections.

At the end of 2022, after years of iteration, the highly popular ChatGPT was finally released to the public.

From that time on, there was a surge in demand for Nvidia graphics cards.

Among them, the most powerful DGX H100, a metal box weighing over 160 kilograms and priced at $500,000, has been out of stock for several months.

The running speed of the DGX H100 is 5 times that of the hardware used to train ChatGPT, and it can train AlexNet in less than 1 minute.

Nvidia expects to sell 500,000 units of the DGX H100 by the end of 2023.

The more powerful the processing capabilities applied to neural networks, the more complex the output. For the most advanced AI systems, perhaps dozens of Nvidia DGX H100s are needed.

If that's not enough, Nvidia will stack these computers like library shelves and fill data centers with supercomputing equipment worth tens of millions of dollars.

Clearly, the capabilities of artificial intelligence have no apparent limits.

In the coming years, Nvidia's hardware will evolve at the speed of computer clock cycles, training various similar artificial intelligence models.

Nvidia's device gross margin is close to 70%.

The huge profits have made all companies developing AI training hardware, including Google, Tesla, and startups, salivate.

Interestingly, Nvidia's fiercest competitor is AMD.

Since 2014, AMD has been led by another talented engineer, Lisa Su. In the years since she became CEO, AMD's stock price has risen 30-fold, making her the most successful semiconductor CEO of this era, second only to Huang Renxun.

It's worth mentioning that Huang and Lisa Su are also relatives.

Huang's Management Method

Huang himself rarely gives interviews. He said, "I haven't done anything special. It's mainly the hard work of my team, and I'm not sure why I was chosen as CEO. I don't have any special drive."

When Huang was determined to run a company at the age of 30, his co-founder Chris Malachowsky said, "You're not really a good speaker because you're more introverted."

Huang said, "I only have one superpower - doing homework." Nvidia's software director Dwight Diercks said Huang could master any subject over a weekend.

Huang prefers an agile company structure, without fixed departments or a hierarchical system. Instead, employees submit a list each week, listing the 5 most important things they are working on.

He himself also writes hundreds of email replies every day, chatting with employees, usually in just a few sentences. One executive compared these emails to haiku, while another compared them to ransom notes.

Huang also developed a set of management maxims that he often quotes.

When assigning tasks, Huang asks employees to consider "the speed of light." This not only means acting quickly; instead, employees should consider the absolute speed at which a task can be completed, and then work backward towards achievable goals.

Perhaps Huang's most radical belief is that "failure must be shared."

In the early 2000s, Nvidia shipped a faulty graphics card with loud and overly active fans.

However, Huang did not fire the product manager of the graphics card. Instead, he arranged a meeting where the managers presented to hundreds of people every decision they made that led to the disastrous failure.

Nvidia employees sometimes complain that Huang's personality is unpredictable.

Huang said, "This is actually a discrepancy between what I think and what I say. When the misalignment is severe, it manifests as anger."

Even when he is very calm, Huang's dominance can be overwhelming. An employee described it as "communicating with him is like sticking your finger into an electrical socket."

Despite this, Nvidia has a very low employee turnover rate.

The surge in GPU sales has also made Nvidia the world's computing power leader, successfully entering the trillion-dollar club. This achievement is inseparable from the "crazy" management strategy of its leader, Huang Renxun.

Huang once said that when you start a company, it's natural to start from first principles.

Reference: The New Yorker

免责声明：本文章仅代表作者个人观点，不代表本平台的立场和观点。本文章仅供信息分享，不构成对任何人的任何投资建议。用户与作者之间的任何争议，与本平台无关。如网页中刊载的文章或图片涉及侵权，请提供相关的权利证明和身份证明发送邮件到support@aicoin.com，本平台相关工作人员将会进行核查。