I did not know about one of the two: The Thinking Machine, Jensen Huang, Nvidia, and The World's Most Coveted Microchip, Stephen Witt, c. 2025. But what a great gift.
The second, I've mentioned on the blog. I had planned to eventually buy it myself but here it is, a birthday gift. Other Minds: The Octopus, The Sea, and the Deep Origins of Consciousness, Peter Godfrey-Smith, c. 2016. Godfrey-Smith is also the author of Metazoa which I've written about on the blog previously.
***************************
Never Quit Reading
The Thinking Machine, Jensen Huang, Nvidia, and The World's Most Coveted Microchip, Stephen Witt, c. 2025
Never quit reading.
Backgammon: the connection between AI, Jellyfish, and Hugh Hefner. Link here: https://share.google/aimode/VHuGbTNo9GhLrl3xN.
****************************************
The Thinking Machine:
Jensen Huang, Nvidia, and the World's Most Coveted Microhip
Stephen Witt, c. 2025.
Introduction
p. xv
This is the story of how a niche vendor of video game hardware became the most valuable company in the world. It is the story of a stubborn entrepreneur who pushed his radical vision for computing for thirty years, in the process becoming one of the wealthiest men alive. It is the story of a revolution in silicon and the small group of renegade engineers who defied Wall Street to make it happen. And it is the story of the birth of an awesome and terrifying new category of artificial intelligence, whose long term implications for the human species cannot be known.
At the center of this story is a propulsive, mercurial, brilliant, and extraordinarily dedicated man.
His name is Jensen Huang, and his thirty-two-year tenure is the longest of any technology CEO in the S&P 500. Huang is a visionary inventor whose familiarity with the inner workings of electronic circuitry approaches a kind of intimacy. He reasons from first principles about what microchips can do today, then gambles with great conviction on what they will do tomorrow. He does not always win, but when he does, he wins big: his early, all-in bet on AI was one of the best investments in Silicon Valley history. Huang's company, Nvidia, is today worth more than $3 trillion, rivaling both Apple and Microsoft in value.
*******************************
The Implosion
p. 83
Of note:
Between the summer of 2001 and the fall of 2002, Nvidia's stock price declined more than 90 percent.
Huang's wealth plummeted with it.
The problems started in January 2002, when the SEC opened an investigation into Nvidia's accounting ..... similar investigations at Worldcom and Enron ... the accounting scandal occurred during one of the worse bear markets in history. Suffering under the simultaneous bursting of the dot-com bubble, the 9/11 attacks, and the Enron bankruptcy, the S&P 500 lost nearly half its value. Coincident with these misfortunes, Nvidia started squabbling with Microsoft.
In September 2026:
- public hysteria over AI, large data centers, power, water
- President Trump to appoint an AI czar and an AI Force (like Space Force)
- the political quagmire in the Mideast
- high diesel prices
- a possible bankrutcy
- watch Oracle
- the squabble between OpenAI and Anthropic (already, the Anthropic IPO has been delayed twice); and,
- now, the Fed is ready to pound in the final nail -- starting to raise rates
Link here. Link: https://share.google/aimode/IEztghw2YRdXINn8n.
****************************
Chapter 9: CUDA
Speaking of books, I continue to read Stephen Witt's biography of Jensen Huang. Absolutely fascinating. It explains very, very well how PC gaming / PC gamers were related to Nvidia. Today, we begin Chapter 9, "CUDA." This should be very, very interesting.
The "big book" I'm reading is the authoritative book on Transcendentalism by Robert A Gross, 2021, along with Gemini. You can follow our discussions, chapter by chapter, by clicking here.
Chapter 9, "Cuda" begins:
Ian Buck was a graduate student in computer graphics at Stanford in 2000.
Gamers like Johnathan Wendel lowered their screen resolution to maximize the frame rate in deathmatch. Ian Buck took things to the opposite extreme .... in 2000 he chained thirty-two GeForce units together to render Quake III across eight projectors.... it took an entire wall....
Poking around the circuits, Buck began to wonder if his GPU daisy chain might be used for tasks other than shooting grenades at his friends. If you wanted to hand-render thirty frames of Quake III in 8K using pencil and paper, and you were willing to work twenty-four hours a day to do so, the arithmetic would take about 16,000 years to complete. Buck's GeForce array was doing that every second. Plus, the whoel rig cost only about $20,000 to assemble -- a pittance by the standards of high-performance computing. To play Quake on the wall, Buck had inadvertently build a low-budget supercomputer.
DARPA, 2002 or thereabouts ...
2004, Jensen Huang called ...
Parallel-computing evangelist .. Bill Dally -- offered a job by Jensen Huang, 2003 ...
Compute Unified Domain Architecture, CUDA ... a secret Nvidia project at the time ..
John Nickolls, an Nvidia engineer who invented CUDA ...
Prediction that Intel's long domination would soon expire ... p. 94.... Moore's Law truly dead ....
CUDA launched in 2006 ... p. 100
The incredible argument between Deb Shoquist and Jensen Huang .. p. 103 .... worth the price of the book ..
... ends with Kirk leaving permanent job at Nvidia to become a part-time consultant and to find his replacement ... p. 105.... his replacement would soon transform Nvidia Research into the most successful corporate R&D department in the world.
********************
Skip Ahead
My notes here, link here. Link: https://share.google/aimode/bu9MySv2HwwIn7oRw.
The AI neural network's "DNA moment" (1953) occurred at the ImageNet contest, Florence, Italy, 2012.
Page 127: following the Florence presentation.
Quickly, senior researchers knew what they had. "Neural nets really do work," Hinton said.
Pages 127 - 129: the result of that presentation.
"AlexNet" named after Alexander Krizhevsky at Hinton's suggestion.
The paradigm shift. The three -- Hinton, Krizhevky, and Sutskever sold their "company" to Google in December 2012 for $40 million.
PART II
Chapter Twelve: O.I.A.L.O.
Page 133.
Bryan Catanzaro: the only Nvidia engineer interviewed for this book who had a humanities degree. Degree in Russian Literature, BYU.
Dual degree in computer engineering.
Page 114, first full paragraph -- needs to be read by analysts.
Catanzaro: insight to combine AI with parallel computing. His bosses did not.
Catanzaro: Nvidia's first dedicated AI researcher.
Training (to identify a cat):
CPUs: 2,000
GPUs: 12
CUDA --> cuDNN -- p. 137.
Again, the key phrase: "parallel computing."
"Damascene epiphany" -- page 118 -- for Huang.
About June, 2013, over a single weekend, Jensen Huang's company became an AI company. That fast. One weekend. Page 138.
Catanzaro, early 2014, became the most important person in the company.
cuDNN's most essential task: accelerating evolution.
"In human brain tissue, each neuron maintains an average of around one thousand synaptic connections with other adjacent neurons. The brain alters these connections through chemistry. The neural net alters them through matrix multiplication.
See explanation, page, 139 at the bottom. Currently approaching mammalian, and in some cases, may be reaching the size os a human's brain, and maybe leapfrogging the size of the human brain.
Chapter 13: Superintelligence
Catanzaro was the Jobs of cuDDN, and Vandermersch was Wozniak.
Bas Aarts: built the first CUDA compiler, p. 147.
Nick Bostrom, a Swedish philosopher at Oxford University, raised the alarm.
Elon Musk, first mentioned (?) -- page 148. 2015: concerned about the risks.
The origin of the word "hallucinations," page 150. Andrej Karpathy coined the word.
The discussion between Elon Musk and Jensen Huang, 2015, p. 149 - 150.
Risk.
Musk, with many other supporters, founded OpenAI.
Other cofounders: Reid Hoffman, the cofounder of LinkedIn, and Sam Altman, the president of Y Combinator.
DGX-1 -- page 154. The DGX-1 could today. be listed alogside the ENICAC and teh Apple II as one of the most important computers ever built.
Wow, wow, wow, the last paragraph on page 155 -- the first DGX-1 to Elon Musk; the second to Fei-Fei Li at Stanford.
The 2016 ImageNet contest began a few days later. Humans: 95% image accuracy; the machines, 98% accurate.
Chapter Fourteen: The Good Year
2016.
"In 2016, Nvidia's stock price appreciated 224 percent, restoring Jensen Huang to glory and making a billionaire once again. Yet Nvidia was not the best-performing stock in the S&P that year. The title went to Nvidia's longtime rival AMD, which went up 309 percent. Huang didn't like coming in second. He redirected himself to victory and found it the next year. Revenues doubled, profits tripled, and the company launched new products at dizzying speed. Few companies anywhere, ever, could hope to have a year as successful as Nvidia had in 2017."
Page 160: the power of izakayas and pitchers of beer cannot be emphasized enough.
Page 160 at the bottom: begins with the 2017 Nobel Prize awarded for work on the Laser Interferometer Gravitational-Wave Observatory, ....
The 2017 Nobel Prize in Chemistry was awarded for work on cryo-electron microscopy ...
... neither were new scientific discoveries but rather tools ... tools that would intersect with the parallel-computing slope of Nvidia's GPUs.
The laser story -- top of page 161 -- the two black holes that crashed into each other -- the breakthrough of the year.
Scientists confirmed the crashing of two black holes in 2016 (detected by LIGO in September 2015 and announced in February 2016), and the discovery was named Science's Breakthrough of the Year for 2016.
"The black holes and the gears of nature (frozen amoeba) had been there, waiting for millioins of years, waiting simply to be seen for the first time. Parallel computing made it possible." -- page 161.
Nvidia: supporting the work of laureate groups in both physics and chemistry in the same calendar year offered irrefutable evidence of CUDA's value to humanity -- the problem was that it didn't pay. Nvidia was not making money.
CUDA well-suited for cryptocurrency -- p. 162.
2017, again:
new building;
leasing time on Nvidia chips in the cloud (AWS, Azure)
alternative universe: everything simulated
And yet it was all going to get bigger.
Chapter fifteen: The Transformer
page 170
***********************
The Car Page
Here are the photos.
This is not me in the photo but it is my granddaughter, 12-y/o Sophia.
This is what is going on, link here. Link here: https://share.google/aimode/MrHlug0Jn4VvbVYfh.
No comments:
Post a Comment