NVIDIA Creates a 15B-Transistor Chip With 16GB Bandwidth Memory For Deep Learning (venturebeat.com)

An anonymous reader cites a report on VentureBeat: NVIDIA chief executive Jen-Hsun Huang announced that the company has created a new chip, the Tesla P100, with 15 billion transistors, 16GB high-bandwidth memory for deep-learning computing. It's the biggest chip ever made, Huang said. "We decided to go all-in on A.I.," Huang said. "This is the largest FinFET chip that has ever been done." The chip has 15 billion transistors, or three times as much as many processors or graphics chips on the market. It takes up 600 square millimeters. The chip can run at 21.2 teraflops. Huang said that several thousand engineers worked on it for years. Jim McGregor, writing for Forbes (the link is not accessible to ad-blocking tool users): It features NVIDIA's new Pascal GPU architecture, the latest memory and semiconductor process, and packaging technology -- all to create the densest compute platform to date. In addition, it combines 16GB of die stacked second-generation High-Bandwidth Memory (HBM2). The memory and GPU are combined into a multichip module on a state-of-the-art silicon substrate. The P100 has NVIDIA's NVLink interface technology to connect to multiple Tesla P100 GPU modules.

Comments

Welcome SkyNET overlords!

[Anonymous Coward]

Please enjoy hunting me with your time machine.

Re:Welcome SkyNET overlords!

[ShanghaiBill]

Please enjoy hunting me with your time machine.

https://xkcd.com/652.

Re:Welcome SkyNET overlords!

[U2xhc2hkb3QgU3Vja3M]

(spoiler) Since the Doctor is Skynet, he already has a time machine.

Re:Welcome SkyNET overlords!

[Anonymous Coward]

More accurate assuming real intelligence:
http://www.xkcd.com/1626/

So,

[dargaud]

can it make me a sandwich ?

Re:So,

[Anonymous Coward]

If not, try saying 'sudo' first.

Re:So,

[Grog6]

Only if you have admin privileges, are s superuser, and enter the right password.

Or say "sudo make me a sandwich".
It works on geek girls, anyway, from what I hear. :)

Now We Know Why Drivers Suck

[zenlessyank]

Maybe you should use some of those engineers to fix your drivers, to, you know, support the people that have already paid you for a product you already produce. Seems that deep learning tech hasn't taught you anything. Spoken as an owner of GTX 660 SLI setup, not some rabid other team fanboi.

Re:Now We Know Why Drivers Suck

[Anonymous Coward]

Already paid? Sucker!

Recurring revenue streams are the way of the future.

Re:Now We Know Why Drivers Suck

[Anonymous Coward]

I'm using a 960 on Win7 and a 970 on both Win10 and Linux. No problems with any of it.

You're box is messed up. Reinstall. If that doesn't fix it then fix your cooling and pay for a real PS. And don't tweak every knob you can reach.

Re:Now We Know Why Drivers Suck

[fuzzyfuzzyfungus]

I'm guessing that the mention of 'SLI' might be the key point here. Taking problems not explicitly designed to be parallelized and attempting to parellelize them at the driver level after the fact is...a bit of a mixed bag...in terms of actually working. It's not clear that Nvidia is holding out on us here; given that they sell fancy multi-GPU systems to high end customers I'm sure that they would be delighted to also offer tools that make using those expensive multi-GPU systems really easy; but that doesn't change the fact that, based on the application, 'SLI' can offer amusingly pitiful performance enhancements, exciting reliability issues, and similar fun even when drivers specifically tweaked to compensate for the application being run are available.

Re:Now We Know Why Drivers Suck

[Hognoxious]

You're box is messed up

One item on a long, long list.

Re:Now We Know Why Drivers Suck

[Anonymous Coward]

Whatever. I've haven't inflicted a dual GPU setup on myself since 3dfx Voodoo stuff. If I had a mind to I'd know what I was getting myself into and be prepared to cope with the mess I create instead of blathering about it on Slashdot. All I know is that NVidia's drivers are next to bulletproof for me on multiple operating systems.

Re:Now We Know Why Drivers Suck

[halivar]

Yes, let's have the chip designers drop everything and help write the device drivers. Brilliant. I should have the network engineers come upstairs and help me with my Excel spreadsheets.

Re:Now We Know Why Drivers Suck

[angel'o'sphere]

Make sure you have a couple of them so they can argue amoung themselves and find an agreement.
Worst case sent randoms of them off to fetch you a coffee!

Re:Now We Know Why Drivers Suck

[zenlessyank]

I am sure they could help if YOU are the one doing the spreadsheet work, since reading comprehension eludes you. Mr. NVidia said 'engineers', which is a broad term, versus chip designers which is pretty specific. If they needed 100,000 chip designers then they DO have problems. Now crawl back under your bridge.

Re:Now We Know Why Drivers Suck

[Anonymous Coward]

You appear to be claiming that shader programs are

not explicitly designed to be parallelized

Either you have no idea whatsoever about what you speak, you did a terrible job of explaining it, or you're trolling.

Re:Now We Know Why Drivers Suck

[fuzzyfuzzyfungus]

I think I did a terrible job of explaining it: Shader programs are indeed designed to be parellelized across however many shader units a given GPU provides. However, SLI(presumably because the niche market doesn't justify a nicer interconnect, or because it may simply not be feasible to provide the same level of integration between multiple PCBs as it is between elements on the same die) doesn't provide particularly close integration of the participant GPUs.

SLI-ed GPUs can't even share VRAM(with some limited exceptions in DX12 if everything is aligned just right to support that), and the shaders on one die aren't visible to the scheduler on the other or vice-versa, so the decision about how to divide the workload has to be made at the level of the driver either chopping the frame into two, three, or four(depending on the number of GPUs) regions of roughly equal complexity, assigning each GPU one of them and then stitching the results together to get the finished frame(Split Frame Rendering) or by allocating alternate frames to each GPU(Alternate Frame Rendering).

In either case, each GPU is effectively working in isolation on the task of rendering the frame or partial frame it has been assigned; with the downsides that SFR is vulnerable to 'tearing' effects, since deciding what an 'even' split of the work involved in rendering a frame before you actually render it can be tricky; and when you judge incorrectly your partial frames will be finished at different times, either dragging down the effective framerate if the system always waits for a complete frame before updating, or causing visual artifacts when one part of the frame is updated before the other part is. AFR has the problem that it can't actually reduce the time needed to render a frame compared to a single GPU(since it is just a single GPU rendering each frame), so while under good conditions you can get nearly linear increases in FPS; any frames that render unacceptably slowly on the GPU in question will render no faster with two of them. Plus, anything that makes rendering frame N+1 require knowledge of frame N(or worse, the state involved in rendering frame N) completely breaks AFR, since each GPU has very little access to what the other is doing.

If SLI actually involved glue logic good enough that putting two cards in SLI essentially produced a single GPU with twice the resources and just slightly higher latency, it would indeed run like a bat out of hell, and make parallization mostly invisible to shader programs. It's just that it doesn't actually do that. Analogous to cluster systems vs. multisocket NUMA systems: SLI provides largely transparent support for collecting the output of each GPU and putting it together into a single video output; but is otherwise fairly loosely coupled, with the driver handing out work units to mostly isolated GPUs with their own VRAM and their own shaders and limited communication with one another.

Re:Now We Know Why Drivers Suck

[Anonymous Coward]

Don't use SLI, ever. It's a stupid fucking idea. Also, never overclock anything, not even your toaster. No amount of performance boost is worth the loss of reliability. These 2 simple steps will make your life dramatically easier.

Re:Now We Know Why Drivers Suck

[halivar]

Oh God, okay, I suppose you're right, and they had software engineers design this new chip instead of actual chip designers, thus stealing the precious resources you're bitching about.

Re:Now We Know Why Drivers Suck

[martinfb]

Perhaps the value here is the AI to overcome that driver OOPS! Replace those lame driver devs with this AI chip!

Re:Now We Know Why Drivers Suck

[zenlessyank]

Artificial Intelligence sponsored/created by a selfish corporation. Let me get some Mellow Yellow and Pringles so I can enjoy the show.

Re:Now We Know Why Drivers Suck

[Coren22]

SLI is an acronym, it stands for Scan Line Interleave. What this means, is that each GPU does half the screen worth of work by running a line at a time on each GPU. I am not sure what the OP's issue with drivers is, but my assumption would be the age of the hardware, 6xx is pretty old, and might not be enough to support modern games anymore. I highly doubt the OP's issue is with the driver having parallelization problems.

Re:Now We Know Why Drivers Suck

[Blaskowicz]

SLI was Scan Line Interleave back in 1998. (and the Voodoo5 used small horizontal bands of pixels rather than straight scanlines). Then nvidia decided to revived it years later, but deciding the letters would stand for Scalable Link Interface, which doesn't really mean anything in particular. It never interleaved scanlines anymore.

1000 engineers

[goombah99]

I'm always amazed how it takes so many engineers. What the heck do they all do? How does one organize this many contributions? Isn't this sort of they highly automated with largely repetitive subunits.

Re:1000 engineers

[Anonymous Coward]

Engineer creates AI. AI replaces engineer.

Re:1000 engineers

[cowtamer]

There are several factors. First of all, what they are building is a HUGE engineered system which would have taken up a couple of buildings a decade or two ago. The fact that the end product is small doesn't change the complexity. The second part is the fact that it IS so small, which brings its own complications. In addition, semiconductor manufacturing is a very tricky business where even making the simplest thing (e.g., a transistor) takes an enormous amount of planning, characterization, and tool design.

Part of it is the R&D -- nothing like this has been done before, so certain things have to be figured out (heat dissipation, how the proximity of the components effect the other components,stuff neither of us will understand, etc. etc). Another huge part is tooling and process -- someone has to design, test and characterize the fabrication tools and processes (the "automation" you speak of has to be built by someone -- a device this complicated probably can't be built without the automation). The chip is divided into subsystems each of which needs to be designed, simulated, and optimized. Someone has to integrate all the subsystems and simulate them together. The 1000 people probably include material scientists, process engineers, electrical engineers of various stripes, semiconductor physicists, mechanical engineers (heat dissipation, packaging, etc)., systems engineers, engineering project managers, etc.

Re:1000 engineers

[alvinrod]

To some degree yes, but making a physical chip requires working withing the confines of the physical limitations of the fabrication process, so it's never as simple as designing some ideal chip. You can certainly try to do it that way, but the yields will suck. It's also not trivial to design the sub-units either, and there's always new instruction sets or other such things to be supported.

Re:1000 engineers

[crgrace]

One organizes many contributions using any number of industry-standard design methodologies. Designing airplanes and cars uses even more engineers.

I suspect NVIDIA is slightly exaggerating and are counting the contribution of many "overhead" engineers that provide value for the whole engineering organization, such as people who work on design tools, design kits, methodology and the like.

You're right, there are many repeated subunit but each unit needs a team to be optimized.

For a chip this complex you need:

Logic Designers (who come up with high-level models for the chip and define the instruction set / hardware interface)
Front-end engineers that write Verilog and/or VHDL (I have no idea what NVIDIA uses)
Implementation engineers (who do place and route and parasitic extraction)
Verification engineers (who use various tools to see if everything is as it should be)
Packaging engineers (who work closely with vendors to develop a custom package for the chip/module)
Module engineers (since we have 3D stacked memories on this device the module engineering is far from trivial)
Thermal Engineers (3D modules typically have very complex thermal requirements)
Signal Integrity engineers (since we're going so fast just getting a signal from point A to point B is hard)
Analog/Mixed Signal engineers (for clocking, serial I/O development)
Integration Engineers (for modeling how to put all this together)
System Engineers (for figuring out if this is all going to work)
Software Engineers (for low-level software dev)
CAD Engineers (for developing and maintaining an appropriate computer-aided design flow)
Foundry Engineers (for working with the foundry on the physical production of the wafers... anything this big and complex will need process customization)
ESD engineers (for figuring out and implementing an ESD strategy)
Library Engineers (for customizing and optimizing the standard cell library used in the chip)
Product Engineers (for solving production problems as they arise)
Test Engineers (for developing and implementing tests to show the chip is working as expecting)
Application Engineers (who work with early adopters to integrate this chip into their systems)

and on and on and on...

As you can see, an army of engineers is required for a chip this complex to see the light of day. On simpler chips, many of these roles can be played by the same people, but in a chip this big, they need to divide the work or it would never get done.

Re:1000 engineers

[Anonymous Coward]

That's just the chip fabrication. To make a GPU, you need engineers at all levels of the graphics stack; hardware architects designing the layout of each core and the communication fabric,hardware design engineers writing VHDL or whatever language is in use. verification engineers to write tests to ensure each hardware logic unit works correctly. Then you need to start connecting the cores to the host system and have a multithreaded task management system - thousands of cores are what you normally see in a supercomputer the size of an office floor. Then you'll need PCI bus communication hardware engineers, then there will be the software engineers that do device drivers; you need architects and software engineers to design, test and verify every extension for OpenGL, OpenCL, Vulkan, CUDA, Cg, GLSL and every other API language there is. Even the shader compilers for SPIR-V, GLSL need to be maintained with unit tests and kept in sync with the hardware. All these AI languages sit above CUDA or OpenCL

Re:1000 engineers

[__aaclcg7560]

For a chip this complex you need:

How many of those job titles and descriptions actually correspond to a college major that an American citizen can learn?

It's a problem mostly seen in the U.S., say labor-market experts, thanks to a rapidly evolving economy and a divide between the country's educational institutions and employers that isn't there in other advanced economies. In Germany and Denmark, for example, the two groups collaborate to ensure training and apprenticeships lead to jobs after graduation. The gap has helped push U.S. job vacancies to 5.5 million, near historic highs. For most of the past year the number of job openings has exceeded the number of new hires, a reflection of employers' difficulty in filling positions.

http://www.wsj.com/articles/colleges-drill-down-on-job-listing-terms-1459704268

Re:1000 engineers

[crgrace]

I see what you're getting at and I don't disagree. However, as you know this stuff is really complicated and you need to be specialized in your career to be effective.Most of these jobs are for Electrical Engineers, a few could be also held by people who studied Computer Science or Mechanical Engineering. I'm an Analog/Mixed-Signal Engineer and while I know Verilog and how to run verification tools, I'm frankly not as competent at those roles as specialists are. It is the way of the world.

I agree you could retrain a product engineer to become a verification engineer but a company would rather bring in a fresh out from India on an H1-B.

Re:1000 engineers

[X-Ray Artist]

Wow, "and on and on and on..."

It is amazing how much I take for granted.

Re:1000 engineers

[Pulzar]

How many of those job titles and descriptions actually correspond to a college major that an American citizen can learn?

They almost all correspond to an electrical or computer engineering major, however the vast majority are not really available to new grads. Three or four on that list will accept new grads who have already specialized a bit in their masters programs, and then once they have a better handle on the big picture they can transfer into other roles.

For example, you don't design systems without knowing how the pieces work first, and you don't help customers integrate your IP/chip without knowing how *their* systems work. But, you can work on verification of a simple block with knowledge picked up in school, with appropriate guidance.

A lot of the logic design is still done in US (in case of US companies), and many of these roles require close proximity to designers... so, you'll find that many are still widely available locally.

Because barrier to entry in hardware design is much higher (you can't simply read a book or take online course and be proficient enough), it's much harder to fill the needs with a bunch of "cheap" H1-Bs. The outsourcing tends to happen by opening location in India/China and hiring full teams to work on some of the more easily outsourced tasks, like CAD and integration. But even there, it's hard to find enough experienced engineers.

Re:1000 engineers

[U2xhc2hkb3QgU3Vja3M]

Dr. Ellie Sattler: Women inherits the earth.

Re:1000 engineers

[slew]

(with apologies to Michael Crichton)

Ian - God creates intelligence, god destroys intelligence. God creates man, man destroys god. Man create AI.

Ellie - AI destroys man, women inherit the earth...

Perhaps a different, more historical view from the 1950's http://www.alteich.com/oldsite...

Dwar Ev threw the switch. There was a mighty hum, the surge of power from ninety-six billion planets. Lights flashed and quieted along the miles-long panel. Dwar Ev stepped back and drew a deep breath. "The honor of asking the first question is yours, Dwar Reyn."

"Thank you," said Dwar Reyn. "It shall be a question that no single cybernetics machine has been able to answer." He turned to face the machine. "Is there a God?"

The mighty voice answered without hesitation, without the clicking of single relay.
"Yes, now there is a God."

Sudden fear flashed on the face of Dwar Ev. He leaped to grab the switch.
A bolt of lightning from the cloudless sky struck him down and fused the switch shut.

Re:1000 engineers

[Anonymous Coward]

.........yeah, but will it play Quake1?

Re:1000 engineers

[AK Marc]

This won't get us closer to AI. A Sperm whale has a brain about 5x the size of a human, and isn't any smarter. So presuming that a larger brain will be smarter is contrary to the known facts.

The solution to hard AI will not be direct.

Re:1000 engineers

[KGIII]

I've actually been wondering and asking, for a while now, why we can't just buy like a 5 cubic inch block of CPU and stuff it in our computers. Yes, I know it will get hot. Yes, I know it'll suck down juice like an arc welder. I'm okay with that. I've got solar and wind. I can cool that down - there are things to do that with.

Seriously, am I the only one that envisions a 5^3" chunk of CPU and all the glorious things I could do with it? Coupled with stacks of those NVRAM critters, piped right next to it, and crammed into a tower that's not much larger than the standard tower we have today. Hell, back home in Maine, I've got two racks that aren't even full. I'd cram it in there. No latency to deal with from distributing. Just miles of compute cycles as far as the eye can see.

Sure, make it a rich-man's toy. I'd still think about buying it. Something like 10,240 cores or whatnot. (Ha! Can you imagine what Oracle would cost on that?) Only, make it so that it's something I can reasonably buy and use a desktop OS on it. Hell, I'd switch to OS X or Windows if they were the only OSes to run on it. I still see my CPU peg 100% - and I can bust up to 4.2 GHz - in my laptop. (Actually, I guess I can OC my CPU and get up to about 5 GHz.) I don't run VMs locally that often, so I don't really run out of RAM.

But I still peg the CPU - frequently. It's almost like we're back at the CPU being the bottleneck again. Just a 5x5x5" cube of transistors. Might want to spread 'em out a little bit - I guess there's some potential for shorts and "leaks." Still... Just a big ol' stack of 'em plugged into a motherboard, it'd probably have to be a horizontally mounted board so that the weight doesn't yank it out of the socket. So we might be talking something akin to the old desktops that were flat - what the hell was the name of that style again? Meh, I forget... I don't think I've even seen any of those in the wild for a long time. I used to stuff my monitor on top of it even though that was probably a bad idea.

Maybe it'd be a little bigger. I'm okay with that. I'm sure as hell not gonna pay supercomputer prices for it or one-off prices for it. I'm absolutely positive that I'm not the only one who'd be willing to buy something like that. Sure, it might cost as much as a fairly decent family sedan or something but I'd never have to upgrade again. Ever... Well, until they became common and then they started to assume we all have that many resources and start making everything bigger so that they can then consume those resources. (I never did figure that one out. My hardware's much faster than it used to be. Things seem to take about the same amount of time in many areas.)

Hmm... There's something to that. I had scads of RAM (maxed out on 32 bit systems) and then they decided everyone had that. My computer was no longer speedy. I had broadband when everyone else had dial-up. Then they started making web pages that are 2 - 5 MB each and load things from across the globe. Fortunately, I can avoid some of that. We buy more resources and they just find a way to take them all back from us. Then they assume everyone's got it. I can't imagine the web today if you have dial-up. Oh wow, that'd just suck. I'd so use Lynx or elinks and block everything at the firewall with more attention paid to keeping it up to date. Actually, I'd move.

It's reached the point where I even get gobs of RAM for my laptop.

At any rate, I can't be the only one who'd like to see a giant-ass CPU in a desktop system. Just a huge chunk of nearly infinite cores and a compiler switch to make sure it uses 'em all. Hell, make me an extra large one with two of 'em in there. 20480 cores ought to be enough for the rest of my life. Hell, if one fizzles out, design it so that it routes around it. And no, not for HPC-type things but for a desktop. I'd never have to update again. I just gotta figure out how to compile for dual-CPU with a total of 20480 cores. There's a make switch for that somewhere. *nods* There's gotta be. Well, there should be.

Re:1000 engineers

[peragrin]

except you can't remove the heat from inside the block. The reason cpu's are basically 2D flat pieces is so we can glue on cooling fins, and cool that bad boy down. until we figure out how to do micro liquid cooling, under pressure, and interweave that cooling into the chip's design we won't get real 3D stacks of processors.

Though What we can do is create 4-6 cpu's stacked vertically around a coolant tube. but I believe that run a foul of pushing parts of the pct to far away from the rest of the components.

Re:1000 engineers

[religionofpeas]

Because we can only make things in layers. So it you want to stack a million layers of silicon, and each layer is a dozen process steps, manufacturing a wafer will take years. Even with current 2D designs, the process already takes many weeks. The chance of a tiny error in one of the layers messing up the whole chip will be huge.

Re:1000 engineers

[Anonymous Coward]

So figure 10-20 people per each one of those teams, maybe up to 5-10 teams for some of those categories, and you are still at like 300-400 max.

Re:1000 engineers

[Coren22]

You could probably train the AI to play Quake 1 pretty effectively.

15B transistors = 16 GB ?

[Anonymous Coward]

How do you store 16 GBytes on 15B transistors?

Re:15B transistors = 16 GB ?

[mysidia]

It's a FinFET device. You can represent more than 1 binary bit per transistor by using multi-gate transistors.

Re:15B transistors = 16 GB ?

[CastrTroy]

Since they are talking about bandwidth, I would guess that what they really mean is 16 GB/s. Although I don't see any reference to bandwidth in the article and the only reference I see to 16 is the 16 nm fabrication process.

Re:15B transistors = 16 GB ?

[Anonymous Coward]

The chip has 15B transistors and no RAM. (it does have 4MB L2 cache and 14MB worth of register files)

The entire Tesla P100 package is comprised of many chips not just the GPU, that collectively add up to over 150 billion transistors and features 16GB of stacked HBM2 VRAM.

Re:15B transistors = 16 GB ?

[Anonymous Coward]

You're 4 days late ...

Re:15B transistors = 16 GB ?

[Anonymous Coward]

It's a FinFET device. You can represent more than 1 binary bit per transistor by using multi-gate transistors.

This is not a factually correct statement. Multi-gate transistors are used because they are more energy-efficicient, perform better, and can be scaled to smaller dimensions than traditional planar CMOS devices. The extra gates give better electrostatic control over the MOSFET channel, but they do not allow the device to perform operations on more than one bit of data at once.

https://en.wikipedia.org/wiki/Multigate_device

Re:15B transistors = 16 GB ?

[DRJlaw]

It's a FinFET device. You can represent more than 1 binary bit per transistor by using multi-gate transistors.

Oh, for God's sake, I ignored this at first but now it's been modded up.

15 billion is the transistor count for the GPU logic. It's not the transistor count for the HMB2 memory installed alongside the GPU on the interposer. Adding an interposer does not suffice to make it all the same chip (hint from TFS: "multichip module").

FinFET is neither necessary nor sufficient to for multi-level-cell-like bit representation. That's also a flash storage technology, not a logic or volatile memory technology (at least in mass produced products).

It's 15 days to Weed Day. Put down whatever you're smoking and get back to work.

Re:15B transistors = 16 GB ?

[Anonymous Coward]

Nope. 16GB ram on a silicon interposer. Bandwidth is 716.8GB/s - 4x1024 bit HBM2 at 1400Mb/s effective data rate.

Re:15B transistors = 16 GB ?

[OrangeTide]

I can address 16GByte with only 44 bits. that leaves more than 14.9B transistors left over to do whatever.

Re:15B transistors = 16 GB ?

[rahvin112]

15 billion is the transistor count for the GPU logic.

I've seen nothing that would indicate either way that this could be the truth. It's pure speculation. Others have speculated that to reach that 15 billion number they have to be counting the memory transistors as well. Though this is big at 600mm2 it isn't that much bigger than previous die's that held a fraction of that number of transitions.

Re:15B transistors = 16 GB ?

[Agripa]

I've seen nothing that would indicate either way that this could be the truth. It's pure speculation. Others have speculated that to reach that 15 billion number they have to be counting the memory transistors as well. Though this is big at 600mm2 it isn't that much bigger than previous die's that held a fraction of that number of transitions.

It has about 50% more transistors than the Oracle Spark M7 at 10.2 billion so the increase is reasonable.

Re:15B transistors = 16 GB ?

[DRJlaw]

You do realize that 15 billion transistors, if you assume that each holds one bit of stored information (HA!), is less than 2GB of storage?

BTW, it's not speculation, it's from NVIDIA's own press release.

Re:15B transistors = 16 GB ?

[rahvin112]

There are two sets of memory on this chip if you read the reports. The die itself has an additional layer that is HBM (High bandwidth memory) linked directly to the CPU. Think of it like the L1 and L2 cache in x86 chips. There is nothing that indicates how much memory this is (as the quoted memory sizes are for the memory chips attached to the boards). I'm willing to bet the chip has around 10billion transistors and the remaining 5 are the HBM layer that sits on top.

Re:15B transistors = 16 GB ?

[DRJlaw]

There are only two sets of memory if you consider the register file to be memory instead of cache (which you apparently do). The problem is, the published specifications demonstrate that you are simply wrong.

4MB of L2 cache and ~14MB of register file space per GPU means that there is about 151 million bits associated with cache and "memory." On a chip with 15.3 billion transistors, that comfortably means that you have about 15 billion transistors for GPU logic.

There is everything to indicate the specs of the chip, and you've lost your bet. Now go away.

Re:15B transistors = 16 GB ?

[Anonymous Coward]

Somehow a critical word got cut from the title, I guess.

It all starts to make sense when you put "high-" before "bandwidth" in there...

Traditional early adopter killer app

[Impy the Impiuos Imp]

This should provide some astonishing porn.

Re:Traditional early adopter killer app

[gstoddart]

LOL, why am I suddenly picturing millions of horny guys getting blown off by a porn AI which has developed attitude?

Except for the people into that whole humiliation thing, I just don't see that being a big selling point. :-P

I think sentient porn is the last thing we want.

Re:Traditional early adopter killer app

[Anonymous Coward]

I think sentient porn is the last thing we want.

That's called a woman.

Re:Traditional early adopter killer app

[Anonymous Coward]

And he's right, they're more trouble than they're worth.

If they didn't have cunts you'd throw rocks at them.

Re:Traditional early adopter killer app

[Perky_Goth]

Well, it could learn to categorize it and learn your preferences very fast, so it's not out of the question...

The P100 was already discovered..

[HumanWiki]

In the Tesla's firmware http://jalopnik.com/a-hacker-m... That would be interesting if it was a chip reference and not a car reference --- tinfoil hat.

Re:The P100 was already discovered..

[michelcolman]

The reference was P100D, so maybe the car is using two of them?

most nVidia enginners work on all projects

[Anonymous Coward]

From what my friends who work at nVidia tell me, most engineers work on all projects. They get sent problems from one GPU, after fixing that, start working on issues from a CPU or some other project.

Units

[Anonymous Coward]

16GigaBillion

Will it run Crysis?

[Anonymous Coward]

See topic. Also, imagine a beowulf cluster of these!

somewhat deceiving numbers....

[etash]

Yes it can do 21,6 teraflop.... at FP16.... half precision...it can "only" do 10,6 teraflop at single precision and 5,3 teraflop at double (64) precision. Also it doesn't have 1TB/sec advertised (for months) HBM2 memory speed, but only 720GB/sec

Re:somewhat deceiving numbers....

[Anonymous Coward]

NVIDIA over inflating specs and performance? I'm shocked I tell you!

Re:somewhat deceiving numbers....

[Anonymous Coward]

It turns out that for deep learning, 1/2 precision is very commonly used. You are using floats for numbers in a fairly small range, and accuracy isn't key. half precision speeds up processing, and more importantly lets you work with twice as much data.

Re:somewhat deceiving numbers....

[Anonymous Coward]

Yeah, especially with asymptotic saturating transfer functions.

Re:somewhat deceiving numbers....

[ChrisMaple]

Is there really any advantage over 16 bit integer, which would be faster and less complex?

Re:somewhat deceiving numbers....

[Anonymous Coward]

You want to use real numbers between 0 and 1. Integers don't do this very well.

Re:somewhat deceiving numbers....

[Anonymous Coward]

The math often works out much better.

Consider the probability of two coin flips having a particular outcome:
One flip would be written as 0.5 in floating point and normalized to some whole number using integers (lets use 50 as a percentage).
Two flips is 50% * 50% for a specific outcome. Now with floating point 0.5*0.5 = 0.25 which is a smaller number as expected. If you multiply two positive integers like 50*50 you get 2500, so a larger value which requires further operations on it for it to be useful.

There are several formulas used in these kinds of systems that benefit from these useful mathematical properties.

Re:somewhat deceiving numbers....

[0100010001010011]

Of course they do and do it with 0.00001525878 precision to boot. If that's all you need then you can get by with 16 bit numbers just fine.

Re:somewhat deceiving numbers....

[fizzup]

I don't think that's a very good explanation. If sigmoid or step neurons only used numbers in the range [0,1], then you could divide the range into 65,536 individual states and use 16 bits to translate [0,65535] into [0,1]. However, sigmoid neurons have many inputs of many different weights, so the total input to a sigmoid neuron can be greater than one. In fact, any one input, after weighting, can be greater than one. The weights themselves can be greater than one. Only the output is constrained to [0,1] by the sigmoid or step function.

In order to represent a number without a lot of accuracy, but keep the ability to represent large and small values, you need a floating-point number. I'm no expert in deep learning, but it does pass the sniff test that a 16-bit float would be good enough for neurons. I assume that NVIDIA has done their homework and determined that FP-8 numbers have too much rounding error to be useful in a neural network.

Re:somewhat deceiving numbers....

[edxwelch]

Shaders for mobile GPUs use 1/2 precision quite a bit, however the small range (-2 - 2) is a problem for many operations, so you end up only being able to use them for less than half of the code.

Re:somewhat deceiving numbers....

[Mandrel]

Is there really any advantage over 16 bit integer, which would be faster and less complex?

Yes, artificial (and real) neurons deliver a weighted average of positive and negative inputs. So you usually have large positive and negative inputs which subtractively cancel to a moderate output. Integer doesn't handle this subtractive cancellation nearly as well as floating point, which can keep the same precision over large changes in scale.

Re:somewhat deceiving numbers....

[DrJimbo]

Now with floating point 0.5*0.5 = 0.25 which is a smaller number as expected. If you multiply two positive integers like 50*50 you get 2500, so a larger value which requires further operations on it for it to be useful.

The only "further operation" needed is to look at the higher word of the result which takes zero extra effort. For example, if you multiply two 16-bit words then you get a 32-bit result. The "extra effort" is taking the upper 16-bits of the result and ignoring the lower 16-bits.

There may well be good reasons for FP16 to preferred over using integers but scaling the result of multiplications isn't one of them.

Re:somewhat deceiving numbers....

[religionofpeas]

The only "further operation" needed is to look at the higher word of the result which takes zero extra effort. For example, if you multiply two 16-bit words then you get a 32-bit result. The "extra effort" is taking the upper 16-bits of the result and ignoring the lower 16-bits.

So, multiplying 100 by 100 equals 0, but starting at 0 and adding 100 for 100 times equals 10000 ?

Re:somewhat deceiving numbers....

[Anonymous Coward]

So much this.

Hell, our brains aren't even good at doing logical operations on the scale of this processor.
It just distributes these simple operations over billions of neurons that do very simple math and out comes a result. (how that translates to consciousness is beyond is at present, and might even be forever)

Very regularly the brain fails at doing these things, our brains have a stupid amount of redundancy and comparisons that pick the majority result. (which itself is a combined value, not so much a logical comparison)
The memory itself is horribly inconsistent at times, with one memory usually being assigned multiple routes for the sake of optimization and redundancy.
You can see these all the time with people that have brain degeneration illnesses when memories thought gone suddenly come back simply because the memory was triggered through another route that isn't used as often.

Our brains are far from computationally optimized. They have huge redundancy due to the plasticity requirements.
If we are creating a brain that runs that plasticity entirely in software, we can create massively more efficient brains.
Our brain is limited by our biology and the fact that evolution tends to not do things unless it needs to. It tends to the lowest energy needed.
Our brain, while heavily energy intense, could be massively more efficient. God only knows how our genome is reacting to the high sugar diets we have, carbs were the things that gave us our brain, but with these sugar-heavy diets, it could lead to FUN.

Double precision is only really needed when you are doing massive physics calculations, protein folding and such.

I for one welcome our () Overlords

[tekrat]

Just imagine a beowulf clust........
Oh, never mind.....

Re:I for one welcome our () Overlords

[Anonymous Coward]

You should determine if it runs Crysis first.

Re:I for one welcome our () Overlords

[Anonymous Coward]

Yes, thank you for pulling out that tired old Slashdot meme that has already been used 8 billion fucking times, you God damn talentless cocksucker!

Re:I for one welcome our () Overlords

[avandesande]

Actually I believe they are going to simulate Natalie Portman's mind using this chip.

Re:I for one welcome our () Overlords

[Anonymous Coward]

If they did so correctly, I don't think it would be the 'desired' result...

"It was as if a million nerds cried out, then were silenced by mouthfuls of hot grits"

Re:I for one welcome our () Overlords

[avandesande]

If this processor runs hot we can use it to make the grits!

Tesla P100?

[rickyb]

Isn't that a little close to the other Tesla? http://jalopnik.com/a-hacker-m...

Re:Tesla P100?

[Anonymous Coward]

Musk can't copyright the name of a famous scientist.

Re:Tesla P100?

[Anonymous Coward]

PEE "100%" Tesla?

Re:Tesla P100?

[Anonymous Coward]

Just like Apple can't trademark the name of a famous tree fruit?

Re:Tesla P100?

[ShanghaiBill]

Musk can't copyright the name of a famous scientist.

TRADEMARK, not copyright ... and yes he can, but only for a narrow commercial purpose. Elon owns the trademark "Tesla" as a car brand. NVIDA owns the trademark "Tesla" as a GPU brand.

Re:Tesla P100?

[stealth_finger]

Musk can't copyright the name of a famous scientist.

TRADEMARK, not copyright ... and yes he can, but only for a narrow commercial purpose. Elon owns the trademark "Tesla" as a car brand. NVIDA owns the trademark "Tesla" as a GPU brand.

And C&C owns "Tesla" as a tank

the link is not accessible to ad-blocking users

[Anonymous Coward]

Oh yes it is lol try harder.

Re:the link is not accessible to ad-blocking users

[Anonymous Coward]

Here's what I see when I follow the article's link to forbes.

http://i.forbesimg.com/forbes/scripts/38bf3d3e.vendor.js
http://i.forbesimg.com/forbes/scripts/5f4b4547.scripts.js
http://i.forbesimg.com/forbes/styles/999cad34.main.css
http://www.forbes.com/forbes/welcome/invisible

I am NOT using any sort of "ad blocking" software.
Oh, I see what's going on. My script-blocking plugin is protecting me from potentially harmful scripts, among other things. Funny how I don't need an ad-blocker to protect myself.

Re:the link is not accessible to ad-blocking users

[Anonymous Coward]

Yup, my noscript blocks the whole festering mess.

Sorry Forbes, you can't serve me your malware infested content.

Well...

[Anonymous Coward]

"the link is not accessible to ad-blocking tool users"
So Slashdot isn't just linking to ad-blocker-blocker site, the ad-blocking users are tools as well... ;)

Apple has done remarkably

[Anonymous Coward]

hijacking the name of a common fruit.

The Most Advanced Hyperscale Datacenter GPU Ever

[Grismar]

But can it run Crysis?

My brain has unlimited storage capacity and speed

[JoeyRox]

Yet my learning ain't so deep.

Beowulf cluster

[Anonymous Coward]

You beat me to it !

"15 billion transistors should be enough for anybody"

No where is the car analogy...

Not all-in

[Anonymous Coward]

Huang said. "We decided to go all-in on A.I.,"

He's misusing the term "all-in". In poker, when you go all-in you bet everything you have. If you lose you are then out of the game.

If this chip fails, I doubt nvidia will be bankrupt.

Decent game AI when?

[Iamthecheese]

I'm still waiting for the game AI that can match a human brain for strategy in an open world, especially in an RPG game but anything beyond well-studied board games really. It's so frustrating to have the computer win by cheating. Not to mention implications for new expert systems. This technology can't mature soon enough.

Re:Decent game AI when?

[Anonymous Coward]

Eventually, game AI will either beat you mercilessly with no need for cheating or it will have to be handicapped to let you have a chance to win. If you want to feel like you are facing a real adversary you will have to play against other people. Just like chess.

History repeats?

[dlleigh]

Hmmm... NVIDIA. Giant chip.

Bill Dally, are you going for a "jump approximate" instruction again?

Re:History repeats?

[Anonymous Coward]

Never heard of this. Google only finds this page.

600 square millimeters ????

[colin_faber]

Is this right?? 2' x 2' chip?

Re:600 square millimeters ????

[Waffle Iron]

Is this right?? 2' x 2' chip?

That's right.

And the package is shaped like Stonehenge.

Re:600 square millimeters ????

[Janthkin]

Is this right?? 2' x 2' chip?

No, it's not right.

600mm^2 is a chip just under 25mm on a side.

Re:600 square millimeters ????

[TomGreenhaw]

2 feet X 2 feet - my guess is no

Square root of 600mm = 24.4948974278mm

24.4948974278 mm = 0.964366 inches (0.964366")

Even still .. that's a friggin HUGE chip

Re:600 square millimeters ????

[Anonymous Coward]

That is still one huge chip. An inch on a side !

In a 16nm process each 1 sq mm contains about 4 billion definable pixels for lithography (e-beam write on the reticle) times 600 sq mm, for about 2 trillion points of data per layer. A particle or defect at the wrong addressable fin-fet point is a scrap device. Even on a 12" wafer, you only get about 100 die.

Yields?

Re:The Most Advanced Hyperscale Datacenter GPU Eve

[Anonymous Coward]

A reporter actually asked that question, and he left the stage sobbing without answering.

CAP === 'interim'

20 Tflops at half precision

[rfengr]

Note it's 20 Tflops at half precision. Single is 10, and double is 5.

We all know how this ends

[U2xhc2hkb3QgU3Vja3M]

Deep learning leads to Deep Thought leads to forty two.

Anonymous Coward

[Anonymous Coward]

A lot of NVidia marketing spin on this I believe. Their previous chips hard ~8 billion transistors, when the halve the size of their transistor footprint, they can approximately get twice as many transistors on on the same size die. Which is excellent of course! However, thousands of engineers working over many years is probably not a lie, however it 100% honest either. Because when NVidia says 1000s of engineers working on something, it usually means 1000s throughout the companies history.

Double the number of cuda cores, I love it! However, too much marketing fud to make it look even better than what it actually is, actually devalues it in my eye a little.

gigatransistor

[Anonymous Coward]

So about 0.964 inches per side... and 15 giga-transitors!

how hot?

[Anonymous Coward]

I wonder how much power it burns. I vaguely remember Telsla has the TDP of 300W. If this monster burns 1000W, not sure if it will be very useful, unless you want to water cool every chassis.

Revolutionary idea...

[Anonymous Coward]

NVIDIA should make a Video Card with an onlboard CPU/motherboard combo.

Re:My brain has unlimited storage capacity and spe

[Anonymous Coward]

POWs in Solitary May Have Tapped Unused Parts of Their Brains

Or....

[BrendaEM]

https://www.youtube.com/watch?...

Numbers don't add up

[flyingfsck]

How do you make 16 GB memory from 15 G transistors?

Tesla P100

[hackertourist]

So can it run 600 km on a single charge?

forbes works with ublock origin it seems

[Anonymous Coward]

NVIDIA Reinvents The GPU For Artificial Intelligence (AI)

Tirias Research

        Follow on Forbes (5)

Opinions expressed by Forbes Contributors are their own.
Jim McGregor
Jim McGregor, Contributor

At a time when PCs have become rather boring and the market has stagnated, the Graphics Processing Unit (GPU) has become more interesting and not for what it has traditionally done (graphical user interface), but for what it can do going forward. GPUs are a key enabler for the PC and workstation market, both for enthusiast seeking to increase graphics performance for games and developers and designers looking to create realistic new videos and images. However, the traditional PC market has been in decline for several years as consumer shift to mobile computing solutions like smartphones. At the same time, the industry has been working to expand the use of GPUs as a computing accelerator because of the massive parallel compute capabilities, often providing the horsepower for top supercomputers. NVIDIA has been a pioneer in this GPU compute market with its CUDA platform, enabling leading researchers to perform leading edge research and continue to develop new uses for GPU acceleration.

Now, the industry is looking to leverage over 40 years of GPU history and innovation to create more advanced computer intelligence. Through the use of sensors, increased connectivity, and new learning technique, researchers can enable artificial intelligence (AI) applications for everything from autonomous vehicles to scientific research. This, however, requires unprecedented levels of computing power, something the NVIDIA is driven to provide. At the GPU Technology Conference (GTC) in San Jose, California, NVIDIA just announced a new GPU platform that takes computing to the extreme. NVIDIA introduced the Telsa P100 platform. NVIDIA CEO Jen-Hsun Huang described the Tesla P100 as the first GPU designed for hyperscale datacenter applications. It features NVIDIA’s new Pascal GPU architecture, the latest memory and semiconductor process, and packaging technology – all to create the densest compute platform to date. Using the industry’s latest 16nm FinFET manufacturing technology, the Tesla P100 features over 15 billion transistors on a 600mm2 die. In addition, it combines 16GB of die stacked second generation High-Bandwidth Memory (HBM2). The memory and GPU are combined into a multichip module on a state-of-the-art silicon substrate. The P100 has NVIDIA’s NVLink interface technology to connect to multiple Tesla P100 GPU modules. And, NVIDIA didn’t stop there. The company also announced its first full server appliance platform dubbed the DGX1.
TIRIAS Research image of NVIDIA Tesla P100 from GTC 2016 keynote

TIRIAS Research image of NVIDIA Tesla P100 from GTC 2016 keynote

The DGX1 features two high-end Intel Xeon processors combined with eight Tesla P100 GPU modules in a 3U rack-mountable chassis that is fully configured right out of the box. The system is capable of an incredible 170 teraflops of performance. While the system comes at a hefty $129,000 price tag and a 3500W power budget, it is a fraction of what a comparable traditional CPU based platform. The DGX1 will enable new levels of deep learning for AI applications. The system will be available directly from NVIDIA starting in May.
TIRIAS Research image of NVIDIA GTX1 from GTC 2016 keynote

TIRIAS Research image of NVIDIA GTX1 from GTC 2016 keynote

The irony of the Telsa P100 GPU and DGX1 is that they are have taken GPUs in a completely different direction than the tasks and applications GPUs were originally designed to perform. But, that GPU technology and the CUDA platform have given NVIDIA a competitive edge in the area that will completely redefine the tech industry – AI. In addition, they have paved a growth path for NVIDIA at a time when the outlook for traditional computing look bleak on the back of a sinking PC industry.

Jim McGregor
Principal Analyst at TIRIAS Research
E-mail: jim@tiriasresearch.com
Twitter: @TekStrategist

Deep learning about morality and post-scarcity?

[Paul Fernhout]

An aside from the article: "Huang showed a demo from Facebook that used deep learning to train a neural network how to recognize a landscape painting. They then used the network to create its own landscape painting."

So long for such jobs... How about deep learning about post-scarcity economics?
https://en.wikipedia.org/wiki/...
https://en.wikipedia.org/wiki/...

Also: ""Our strategy is to accelerate deep learning everywhere," Huang said."

How about some deep learning about morality? Imagine training children (or child-like AIs) in skills like weapons use without training them in morality, kindness, cooperation, and so on... How would that end?

See also:
http://www.child-soldiers.org/
"Child Soldiers International is an international human rights research and advocacy organisation. We seek to end the military recruitment and the use in hostilities, in any capacity, of any person under the age of 18 by state armed forces or non-state armed groups. We advocate for the release of unlawfully recruited children, promote their successful reintegration into civilian life, and call for accountability for those who unlawfully recruit or use them."

Maybe AIs should not be asked to replace humans until they have been around for at least eighteen years?
http://www.rfreitas.com/Astro/...
https://en.wikipedia.org/wiki/...
http://www.amazon.com/The-Chro...

but.. can it

[Anonymous Coward]

But can it run Crysis?

Imagine what a BEOWULF CLUSTER of these could do!!

[ChoosyBeggar]

:P