Slashdot Mirror

← Back to Stories (view on slashdot.org)

Move Over Moore's Law, Make Way For Huang's Law (ieee.org)

Posted by msmash on from the closer-look dept.

Tekla Perry writes: Are graphics processors a law unto themselves? Nvidia's Jensen Huang says a 25-times speedup over five years is evidence that they are. He calls this the 'supercharged law,' and says it's time to start counting advances on multiple fronts, including architecture, interconnects, memory technology, and algorithms, not just circuits on a chip.

55 comments

  1. Circuits on a chip? by 110010001000 · · Score: 0

    I didn't realize Moore's Law was about circuits on a chip. Can you please explain? I didn't read the article but they must be talking about "AI" processing.

    1. Re:Circuits on a chip? by imgod2u · · Score: 4, Informative

      Moore's Law: the density of devices (transistors) that can be packed into a microchip doubles roughly every 18 months.

    2. Re:Circuits on a chip? by Anonymous Coward · · Score: 5, Insightful

      Or phrased more colloquially, if partially inaccurately:

      Moore's Law: every 18 months, the speed of hardware doubles.
      Gates' Law: every 18 months, the speed of software halves.

    3. Re:Circuits on a chip? by 110010001000 · · Score: 1

      Interesting. Thanks for the info.

    4. Re:Circuits on a chip? by Anonymous Coward · · Score: 0

      I think you'll find Gates' law is every 12 months.

    5. Re:Circuits on a chip? by Anonymous Coward · · Score: 0

      I didn't realize Moore's Law was about circuits on a chip. Can you please explain?

      Sorry no, it's just not something that can be easily explained and put into words :P

    6. Re:Circuits on a chip? by dryriver · · Score: 1

      This is "Nvidia Is Getting Desperate's Law" at best: No huge leaps ahead in GPU design or game lighting calculation quality every 18 months? (Distraction in 3...2...1...) "Buuut the 640 Tensooooooooooor cores are sooooo fast zomgggg... realtime denoising.... zomgggg.... the Artificial Intelligences are learning faster.... zommmggggg.... that we need to name a new law about them.... (Reference to Shadow Warrior: "Who wants some Huang???")

      --
      Why did the chicken cross the road? Because Elon Musk put an AI chip in its head.
    7. Re:Circuits on a chip? by Jonathan+C.+Patschke · · Score: 3, Interesting

      Moore's Law is an economic one, not a strictly a technological one. Although, keeping it going depends on semiconductor processes getting finer.

      The costs per nanoacre more-or-less follow a predictable curve relative to how bleeding-edge a process is required to fabricate a chip. If you need a really old process, availability will be low, so demand will push the costs up. If you're using the latest, availability is low and yields will initially be low, so the costs are way up. Everything in-between is pretty cheap because demand tends to go towards newer stuff, and the fabrication plant for those middle-aged processes has already started depreciating-out.

      If you're trying to be an industry leader, you're targeting the newest processes, so you have the highest expenses. As a result, you want to keep your chip small because costs scale with semiconductor area (N chips per wafer, X dollars per wafer, etc.).

      However, there's a lower bound on that. If you've got a chip with over a thousand pins, like a high-end microprocessor, the chip has to be physically large enough to have that many bumps to wire out to pins. Also, there's the case of heat dissipation to consider. A teeny-tiny part that draws a lot of current may shatter or desolder itself if there's not enough surface area to mate with the thermal solution.

      So, you want to be small, but there's a lower bound to that. That implies wasted space on the die, which you're going to pay for, anyway. What to do with it?

      Add a core! Make the pipelines deeper! More cache! Add some multimedia accelerators. Add an FPGA! Add some other dedicated-function unit! Then present it as a bullet-point for selling more of your part versus your competitor.

      Ergo, Moore's Law.

      --
      Pining for the days when The Glorious MEEPT!!! graced SlapDash with his wisdom.
    8. Re:Circuits on a chip? by Anonymous Coward · · Score: 0

      Moore's law says nothing about speed, just complexity of the cheapest component. I'm not certain that logic devices like GPUs have the cheapest transistors compared to DRAM or NAND flash.

    9. Re:Circuits on a chip? by Agripa · · Score: 1

      Moore's Law: the density of devices (transistors) that can be packed into a microchip doubles roughly every 18 months.

      It is not actually the density of transistors but the number of transistors for a given cost. This can be accomplished by density increases but also by ICs which are larger in area. It is often done at the expense of transistor performance.

    10. Re:Circuits on a chip? by rickyslashdot · · Score: 1

      Lost my mod points - too slow to mod, too lazy to be faster -lol-
      Someone please mod this +1 INFORMATIVE

      --
      redneck geek
  2. Everyone gets a law now by Anonymous Coward · · Score: 0

    Observe some trend and name it

    1. Re:Everyone gets a law now by tomhath · · Score: 4, Funny

      Everyone gets a law now

      Yes, that's the well known "Anonymous Coward's Law".

    2. Re:Everyone gets a law now by Anonymous Coward · · Score: 0

      Everyone gets a law

      Oprah's Law

  3. Let's just call it what it is: by Gravis+Zero · · Score: 1

    Bitcoin's Law. It's all about hashes per second and still a pointless metric!

    --
    Anons need not reply. Questions end with a question mark.
    1. Re:Let's just call it what it is: by Anonymous Coward · · Score: 0

      Not if your're the NSA and control/use bitcoin to crack the SHAA256 algorithm. Then it's extremely valuable.

  4. Really? by methano · · Score: 4, Insightful

    I think the attractive aspect of Moore's law was that it was simple and everyone got the general gist. Some people like to argue about the details but they mostly don't have anything else to do with their time.

    I don't think we need a Huang's law. If you asks what Huang's Law is, everyone will just say it's like Moore's law except applied to GPU's.

    Same applies to all those other people who want to name things after themselves.

    1. Re:Really? by Anonymous Coward · · Score: 0

      As long as the two differ only by a constant factor, they're the same law.

    2. Re:Really? by DigiShaman · · Score: 2

      Why not both? I like the take on Huang's law. It's essentially the measure of performance that leverages all hardware/software advances that go into a single product, bench-marked over time. It's not a replacement, just offers a different, useful, perspective.

      --
      Life is not for the lazy.
    3. Re:Really? by Carewolf · · Score: 1

      I think the attractive aspect of Moore's law was that it was simple and everyone got the general gist. Some people like to argue about the details but they mostly don't have anything else to do with their time.

      I don't think we need a Huang's law. If you asks what Huang's Law is, everyone will just say it's like Moore's law except applied to GPU's.

      Same applies to all those other people who want to name things after themselves.

      True. The generalized Moore's law is that everything in computers were improving at exponential rates just with different doubling rates.

      And since Moore's law is based on his last name, wouldn't it be Jensen's Law if they really wanted to make a counter law?

    4. Re:Really? by Anonymous Coward · · Score: 0

      If someone asks me what Huang's Law is, I will PUNCH THEM IN THE FACE.

    5. Re:Really? by Anonymous Coward · · Score: 0

      If you asks what Huang's Law is, everyone will just say it's like Moore's law except applied to GPU's.

      If you ask what special relativity is, everyone will just say it's like Newton's laws of motion, but at near light-speed.

    6. Re:Really? by Anonymous Coward · · Score: 0

      We don't need a new law. Just use Moore's law for whatever floats your boat. Remember: every 18 months the number of shit tech bloggers doubles.

  5. we kind of have by xxxJonBoyxxx · · Score: 3, Insightful

    >> t's time to start counting advances on multiple fronts

    Hobbiests have for a while, ever since last-generation AMD stumbled and Intel slowed down the processor speed increases for a while. Now that Ryzen is out, graphics chips are ruling desktops, and no one cares about Intel in the mobile space, we're finally seeing progress get back toward Moore's Law's long-term trend line.

    1. Re:we kind of have by Anonymous Coward · · Score: 0

      "Hobbiests have for a while"

      Who? Do you mean armatures?

      Christ, why can't fuckheads learn to spell "hobbyists", and by the same token, "amateur"?

      If you have a lobby, you get lobbyists. Not lobbiests. You have a hobby, you're a hobbyist.

      How do you pronounce "priests"? So how do you pronounce "hobbiests"?

    2. Re:we kind of have by xxxJonBoyxxx · · Score: 3, Funny

      >> So how do you pronounce "hobbiests"?

      The same way Gandalf did. Duh.

    3. Re:we kind of have by Hognoxious · · Score: 1

      Do you mean armatures?

      Some people are just wierd up wrong.

      --
      Confucius say, "Find worm in apple - bad. Find half a worm - worse."
  6. Cost vs. performance by bill.pev · · Score: 1

    I thought Moore's comments had to do with the impact of transistor counts on cost. Huang seems to be talking about increased performance without reference to cost. I'm not a gamer, but isn't there a lot of squawking about GPU costs? I wasn't at the talk, so maybe Huang addressed that as well. [Maybe he also assumed people would do the cost vs performance calculation in their heads.]

    1. Re:Cost vs. performance by MrL0G1C · · Score: 1

      Lol, if you're going to put it like that then there's been barely any improvement what-so-ever for gamers at the median price point.

      --
      Waterfox - a Firefox fork with legacy extension support, security updates and better privacy by default.
    2. Re:Cost vs. performance by Euphorinaut · · Score: 1

      It's used that way and in many more ways that are more general or adjacent to the original coined definition, but no, it's merely about how many transistors you fit. That's partially why Ray Kurzweil started using the term "law of accelerating returns". Yes, GPU prices are stupid high right now, but this has less to do with changes in the R&D of GPU's and more to do with the fact that the price of bitcoins can double faster than the number of foundries/fabricators dedicated to making GPU's or cryptocurrency asic can double, not to imply that you're incorrect about it being a real effect on price performance or anything.

  7. Nothing to do with speed. by thegarbz · · Score: 1

    Moore's Law exclusively talked about transistor count. Speed aside:
    GTX 770 in 2013 : 3.5bn transistors.
    GTX 1070 in 2017: 7.2bn transistors
    Moore's law is dead, but not because you defeated it, but because you failed to live up to it just like the CPU vendors did.

    We have been counting advances on multiple fronts since the Intel Core architecture debuted 12 years ago, but welcome to the 21st century Jensen Huang.

  8. Let's not call any and all predictions laws by Visarga · · Score: 2

    Let's not call any and all predictions laws. They're not laws. They are functions fit on a short stretch of data that have no predictive power in the future. Not even experts can predict the future, a "law" has no chance here.

  9. Lol by Ryanrule · · Score: 1

    Sounds like wang.

  10. Is GPU still an accurate term? by ArhcAngel · · Score: 1

    Shouldn't we be calling them BPU's now?

    --
    "A person is smart. People are dumb, panicky dangerous animals and you know it." - K
    1. Re:Is GPU still an accurate term? by darkain · · Score: 2

      The appropriate term now is GPGPU.

    2. Re:Is GPU still an accurate term? by DigiShaman · · Score: 1

      While GPGPU's are marketed for graphics, at some point, there will hit a law of diminishing returns. There's even talk of them blowing past all raster and into ray tracing; thought I'm not sure if and when that would be feasible. However, certainly easier to program for. Also, given their use in physics and other forms of computing, wouldn't it make more sense to call them GPSPU's (Genera-Purpose Specialized Processing Units)? Or would that acronym be reserved for FPGA's?

      --
      Life is not for the lazy.
    3. Re:Is GPU still an accurate term? by darkain · · Score: 1

      Well, first, the transition to ray tracing started in March already: https://www.youtube.com/watch?...
      Also, GPGPU stands for "General-purpose computing on graphics processing units": https://en.wikipedia.org/wiki/...

    4. Re:Is GPU still an accurate term? by Anonymous Coward · · Score: 0

      "General-Purpose Specialized" makes about as much sense as "Digital Versatile Disc" and "Wireless Fidelity", so yeah.

    5. Re:Is GPU still an accurate term? by GuB-42 · · Score: 1

      There are countless demos showing realtime ray tracing, usually showing off what ray tracing does well, like reflection and refraction.

      However, in practice, it doesn't do much that rasterization can't do better with the same budget. And raytracing doesn't solve the problem of global illumination, which is what most people are working on right now.

      Actually, modern engines use a wide variety of techniques depending on what needs to be displayed, some of them close to ray tracing. It is much too early for general solutions like pathtracing. All engines, realtime or not, still rely on selecting the best approximation for the job.

  11. Cost per transistor by Anonymous Coward · · Score: 0

    Moore's law is really a statement about economics.

    More or less fixed buying power.

    More transistors you can fit into a widget the better and more powerful it can be.

    More transistors produced the more costs are reduced via investments in production and economies of scale.

    In effect a feedback loop between the two ideas.

    GTX 580's run about $100 .. DGX-2's are in hundreds of thousands of dollar range. Any comparison in this regard is laughable.

    Over the last year due to crypto mining cost of GPUs has doubled which effectively reduced the capabilities customers could afford. You could argue Moore's law effectively running backwards for GPUs if you were only focused on comparing two things rather than more general trend lines.

    Micro events are really random one-off things which imply very little about the future. Yesterday it took 3 days to encode a 4k HEVC video. Now it takes 18 minutes because we implemented an ASIC. Does this mean tomorrow I'll be able to do the same in 18 seconds? Doubtful.

    Yesterday some AI machination took a year to train, today some bizbang 4-bit massive PIM scheme did the same thing in 18 minutes.

    There are no trends associated with tweaking hardware to address specific workloads. You do it once... get a massive jump then you get to wait in line for process improvements to come online like everyone else. Architectural changes generally yield diminishing returns.

  12. Do we really need a new name? by Anonymous Coward · · Score: 0

    The new law is a 5% downward adjustment on the Moore's law as it calls for an increase to 190% instead of 200% in a year. What's the fuss about?
    1.9 ** 5 = 25
    2.0 ** 5 = 32

  13. Quantum Computers by neoRUR · · Score: 1

    So Quantum computers are roughly following Moore's Law in their power. Are we going to rename that to the Q's Law.?

  14. Suggestion For Name Change by organgtool · · Score: 1

    The word "law" in this context is taken from science to describe an effect that always holds true. After watching Moore's Law break down, could we please name this new related observation for graphics cards something more accurate and realistic? How about Huang's Principle or Huang's Observation?

    1. Re:Suggestion For Name Change by dryriver · · Score: 1

      My AI cannot parse what you have suggested properly. Why? Because it doesn't have Jensen Huang's new 640 AI-Petaflop-Supercomputer-Zommmgg-UberMensch Tensooooor cores yet.... (Runs out and desperately looks for a shop that sells Nvidia Volta GPUs)

      --
      Why did the chicken cross the road? Because Elon Musk put an AI chip in its head.
    2. Re:Suggestion For Name Change by im_thatoneguy · · Score: 1

      Huang's Kind of Matches A Couple Years of Data

  15. Re: Suggestions by Anonymous Coward · · Score: 0

    Lots of good names...

    Huang's Stolen IP
    Huang's Falsified Credentials
    Huang's Bogus Data & Fake Research
    Huang's Tiananmen Square Tank Man
    Huang's Spies For China

  16. 13 months by pgn674 · · Score: 2

    While Moore's Law is a doubling in density every 18 months, this 25 times speedup in 5 years is achieved via a doubling every 13 months.

  17. Supercharged Law? by Anonymous Coward · · Score: 0

    I have to say Supercharged Law is a ridiculous name.

  18. The Law you should worry about by FeelGood314 · · Score: 1

    The cost of the fab plants has also been growing exponentially. It hasn't been doubling every 18 months and we (consumers) haven't noticed because the cost of the fabs has been spread out over more chips being made. Unfortunately more and more manufacturers are now fabless. Meaning the actual manufacturing is more concentrated. However the bigger thing we should worry about is the end of exponential grow of chip demand. When that happens it won't just be an engineering problem to pack more transistors in but to do so economically. That will be the end of Moore's Law (prediction). I don't see that bloat of software ending anytime soon.

  19. EVERYBODY HUANG CHUNG TONITE !! by Anonymous Coward · · Score: 0

    Everybody have fun tonite!

      Except Intel.

  20. 500x Faster DGX-2 Costs $400,000! by TheNarrator · · Score: 1

    The DGX-2 which is 500x faster than 2 GTX 580s is also 500 times more expensive!
    https://www.nextplatform.com/2...

  21. Re: But hes CHINESE by Hognoxious · · Score: 1

    AmiMoJo forgot his password again.

    --
    Confucius say, "Find worm in apple - bad. Find half a worm - worse."
  22. No Predictive Power?? by Anonymous Coward · · Score: 0

    You are talking out of your ass on that point. Moore's Law had enormous predictive power for roughly 40 years. Furthermore the fundamental basis of this observation was very well understood: When you shrink IC feature size by half, overall circuit size reduces by 4X, due to the fact that IC designs are 2 dimensional.