Source: Power Plant
Author: Xiao Yulin, Editor: Gao Yulei
Image Source: Generated by Wujie AI
In December 2012, a top AI conference, NIPS, was held at Lake Tahoe, Nevada, USA. A secret auction held at the conference hotel fetched a high price of 44 million US dollars. The auction was for a company that was hastily established before the conference, with only 3 members and no product released. Participants in the auction included Google, Microsoft, DeepMind, and Baidu.
It was a both extravagant and crude auction, with bidders remotely placing bids through Gmail emails every hour, with each bid increasing by no less than 1 million US dollars. The 60-year-old organizer, suffering from cervical spine disease and unable to sit for long periods, turned a trash can upside down on the hotel table and placed a computer on it to remotely control the bidding of the tech giants.
As the bidding price soared, after reaching 20 million US dollars, only Baidu and Google remained in the competition, both determined to win. After the bid reached 44 million US dollars, the organizers unexpectedly paused the auction, postponing it to the next day. Before the auction resumed the next day, the organizers informed Baidu that they would prefer to join Google instead.
This story was placed at the beginning of the book "The Deep Learning Revolution" by tech journalist Cade Metz, to illustrate the fierce competition for talent in the field of artificial intelligence among tech giants. The company auctioned off was DNNResearch, and the three employees were Geoffrey Hinton, known as the "father of deep learning," Alex Krizhevsky, the founder of AlexNet convolutional neural network, and the current Chief Scientist of OpenAI, IIya Sutskever.
Geoffrey Hinton (right) and his two research students, Alex Krizhevsky and Ilya Sutskever (left) (Image source: University of Toronto)
Before this auction, "artificial intelligence" was still only a thing in science fiction novels and movies, and the concept of "deep learning" was little known even within cutting-edge tech giants. After this auction, everything began to change. Three years after the auction, AlphaGo emerged. Five years later, at the same NIPS conference, eight Google researchers presented the stunning Transformer model architecture. Ten years later, almost all top AI labs in the United States had a presence from Google.
The First Success and Failure of Google Brain
Mathematical formulas organize a broad picture from the microcosm of particle world to the macrocosm of the universe, and they are also the tools for constructing modern computer and artificial intelligence algorithms. The underlying logic operations of modern computers were formed by George Boole's invention of Boolean computation in the mid-19th century. In the 1940s, people began to abstract the function of biological neurons into mathematical models, and at the end of the 1950s, the first Neural Network algorithm (NN algorithm) was born.
Geoffrey Hinton, a great-grandson of George Boole, entered the field of NN algorithms in the 1970s and became one of the main figures driving the development of NN algorithms. By the early 21st century, the use of GPUs and distributed computing greatly improved the efficiency of algorithm operation, making NN algorithms a hot field. In 2006, Hinton proposed the concept of deep belief networks, making it possible to train deep neural networks, and "deep learning" became synonymous with neural network algorithms.
At that time, Google had a mysterious lab called Google X, dedicated to researching cutting-edge projects such as intelligent driving and augmented reality, many of which were related to artificial neural networks. In 2011, Andrew Ng, a professor of artificial intelligence at Stanford University, joined Google X, hoping to promote his research on neural networks.
However, even at Google, few people at the time recognized the changes that neural networks could bring. After several setbacks, Ng met Jeff Dean, and the two quickly turned some research into results, leading to the creation of the independent department, Google Brain.
In the summer of 2012, The New York Times detailed the "cat paper" collaboration between Ng and Jeff Dean, in which they used 16,000 computers to create a large neural network system with 1 billion connections. The system learned the concept of "cat" from 10 million images from YouTube and identified images containing cats. This astonished and inspired Google scientists.
However, Ng and Dean's method was not the most efficient at the time. In the fall of the same year, Hinton's team used the AlexNet deep convolutional neural network to win first place in the ImageNet image recognition competition, achieving a score 41% higher than the second place. This was a highlight of the ImageNet competition and one of the "decisive moments" in the tech industry, according to Cade Metz, the author of "The Deep Learning Revolution." AlexNet was considered a turning point for deep learning and the global tech industry, and directly led to the auction mentioned at the beginning of the article, through which Hinton's team joined Google Brain.
After this, academic investment became a strategically important investment direction for Google. Although deep learning was a very small field at the time, Google co-founder Larry Page believed it would reshape the industry and sought talent within it. While accepting Hinton, Google began to pursue its next target.
Proud DeepMind and Google's Big Bet
Similarly, DeepMind had three main founders. The primary founder, Demis Hassabis, had exceptional intelligence from a young age, writing computer games at the age of 8 and earning the title of international chess master at 13. At the age of 20, Hassabis graduated with double first-class honors in computer science from the University of Cambridge and successfully ran a 60-person game company after graduation. Later, Hassabis turned to the study of cognitive neuroscience and eventually immersed himself in the wave of artificial intelligence.
When deciding to found DeepMind in 2010, funding was an important consideration. Hassabis obtained the first investment through a chess game, from Peter Thiel. Thiel, the founder of PayPal, facilitated further investment from Elon Musk. Within two years of its founding, DeepMind completed two rounds of financing.
Hassabis first proposed the concept of AGI, and DeepMind operated independently with AGI as its goal. In the early days of its founding, DeepMind trained a neural network system to play games, and the system learned to play games such as "Space Invaders," "Pong," and "Breakout" by watching, outperforming human players in many games. While Musk was watching a demonstration video of DeepMind on a plane, he happened to be noticed by Google's Larry Page. Wired reported that Musk introduced DeepMind to Larry Page.
Demis Hassabis
Google's Big Bet and the Struggle of DeepMind
For institutions like DeepMind, it was destined that they could not independently complete the money-burning AGI gamble. They needed continuous financial support and quickly found themselves in a predicament after their founding. Fortunately, Hinton's team's auction significantly raised the value of AI talent, and Google was investing in it regardless of the cost.
In 2013, a Google team flew to London to negotiate with DeepMind and the following year, they acquired DeepMind for 650 million US dollars. Google agreed to DeepMind's request that their collaboration technology could not be used for military purposes and established an independent ethics committee to oversee it.
DeepMind's financial pressure came to Google, and according to CNBC and Bloomberg, from 2016 to 2019, DeepMind averaged a loss of 460 million US dollars per year. Reports showed that in 2019, DeepMind's intercompany loans and accrued interest totaled 1.5 billion US dollars, which was forgiven by Google.
The Open Source Power of Facebook
Like Hassabis, Zuckerberg's first angel investment for founding Facebook on the Harvard University campus also came from Peter Thiel. When DeepMind was looking for buyers, Thiel facilitated the first contact between the two. Facebook was more interested in acquiring DeepMind than Google and was willing to pay a higher acquisition amount. However, it was a failed acquisition. According to "The Deep Learning Revolution," Hassabis believed that there was no "chemistry" between him and Zuckerberg, and more importantly, they did not share the same understanding of AI safety.
In 2013, Facebook realized the importance of deep learning for the first time, and Zuckerberg decided to recruit cutting-edge research talent. However, like DeepMind, these researchers from academia found Facebook less attractive. They valued whether their research could continue at Facebook more than financial income, but found that Facebook did not have a laboratory.
Zuckerberg found Yann LeCun, who invented the convolutional neural network and held a similar position to Hinton in the field of deep learning. LeCun believed in open technology and was willing to publish his latest research, aligning with Facebook's open philosophy. With LeCun's help, Facebook operated the FAIR laboratory and gained an advantage in recruiting talent from Google and Microsoft.
Open sourcing allowed Facebook to maintain a leading position in the field of AI. In 2022, the ChatGPT from the independent laboratory OpenAI skyrocketed, and Zuckerberg chose to open source the Llama large model in response, quickly gaining an ecological advantage. LeCun was instrumental in promoting this open source decision.
The Underestimated OpenAI
In the field of deep learning, there were three academic giants: Hinton, LeCun, and Yoshua Bengio. After the AI talent war between Google and Facebook, Bengio received numerous invitations from tech companies, including OpenAI. OpenAI had not yet been established, and Greg Brockman and Sam Altman were recruiting talent. According to Wired, Bengio gave them a list of the most promising young researchers in the field, including Ilya Sutskever.
Google offered a high salary to retain Sutskever, nearly 2 million US dollars in the first year, two to three times that of OpenAI, which made Sutskever hesitate. The book "The Deep Learning Revolution" detailed this story, revealing that Sutskever's hesitation disrupted OpenAI's plan to announce its establishment at the NIPS conference that year, and he only confirmed his decision on the last day of the conference.
The establishment of OpenAI initiated a trend. Soon, researchers who had joined tech giant labs in the past few years began to leave. The most representative of them were the 8 authors who invented the Transformer model architecture at Google.
Google's Blaze and the Starry Sky
The theoretical breakthroughs in AI technology almost all came from academic teams on campuses. Google and Facebook's labs served as a link between academia and commercial companies, nurturing a large number of academic talents and driving the birth of research results. Over time, many researchers realized that things had changed.
Ilya Sutskever later mentioned in an interview the reason for leaving Google, stating that neural networks in artificial intelligence were becoming increasingly large and would be a larger and more organized engineering project in the future. Researchers needed to study them from a more macro perspective, and while the lab could provide a comfortable environment and superior rewards, it could not provide them with more resources.
The establishment of OpenAI opened a new trend. Soon, researchers who had joined tech giant labs in the past few years began to leave. The most representative of them were the 8 authors who invented the Transformer model architecture at Google.
Since 2017, the 8 authors have all left Google and joined various fields at "OpenAI." Google's lab has become a source of AI technology and talent. As AI research has transitioned from papers to practical applications, Google finally realized that academic investment results were scattered everywhere. With Geoffrey Hinton's retirement in 2023, Google announced that it would no longer publish the latest research results and merged DeepMind and Google Brain into Google DeepMind, with Hassabis as CEO.
In recent years, Microsoft, which has been in a weak position, has regained its first-mover advantage due to the rise of OpenAI. Google's strategic adjustment also determined that Hassabis, who has always been concerned about AI safety issues, will have to accelerate his "Manhattan Project." Musk, who lost both DeepMind and OpenAI, had to continue to warn people about the threats of AI technology expansion.
免责声明：本文章仅代表作者个人观点，不代表本平台的立场和观点。本文章仅供信息分享，不构成对任何人的任何投资建议。用户与作者之间的任何争议，与本平台无关。如网页中刊载的文章或图片涉及侵权，请提供相关的权利证明和身份证明发送邮件到support@aicoin.com，本平台相关工作人员将会进行核查。
