To be fair, he didn't call it Huang's Law but then again Gordon Moore did not give his Law a name either. What he said was that there was a new law in GPU land – "supercharged law" and maybe the press would call it Huang's Law.
For those who came in late, Moore’s Law refers to the regular doubling of the number of components per integrated circuit that drove a dramatic reduction in the cost of computing power. Huang said that GPUs are developing much faster.
Huang pointed out that Nvidia’s GPUs today are 25 times faster than five years ago. If they were advancing according to Moore’s Law, he said, they only would have increased their speed by a factor of 10.
He said that other benchmarks show the speed is much more. The time to train AlexNet, a neural network trained on 15 million images would have taken six days on two of Nvidia’s GTX 580s. The company’s latest hardware, the DGX-2 takes 18 minutes—a factor of 500.
So basically Huang was chucking lots of numbers about and has still not come up with a law. However he did think that GPUs need a law of their own—they benefit from simultaneous advances on multiple fronts: architecture, interconnects, memory technology, algorithms, and more.
“The innovation isn’t just about chips,” he said, “It’s about the entire stack.”
And he wants to be the Law, we doubt it will catch on.