Will the End of Moore's Law Halt AI Progress? (mindmatters.ai)

AI progress is not bound by computation speed @now by Anonymous Coward · 2019-01-05 12:40 · Score: 1

If AI was able to sit processing something for a year and come up with something useful, that'd be leaps ahead of what we have now. Speed of processing is definitely not the issue right now.

Transistors and AI by reanjr · 2019-01-05 12:43 · Score: 4, Insightful

If you think AI is just more transistors, you probably aren't doing anything interesting in AI research. How many transistors in the human brain? How many regular transistors are necessary to do the work of one quantum transistor? We don't even know how the brain works, and this asshat is asserting that we'll never be able to build a machine that works the same way.

Re:Transistors and AI by alvinrod · 2019-01-05 12:55 · Score: 2

It's even more foolish since researchers have already been able to build a machine that behaves like very simple creatures. There isn't any reason so think that we can't make something more complex, it's just a matter of being able to map out the wiring and build hardware to mimic the sensory data that the artificial brain needs. However, there's a long way to go. I recall another researcher that was trying to make a robot to fold clothes, but that problem turned out to be much harder than he thought since just getting the robot to be able to recognize individual articles of clothing is challenging.
Re: Transistors and AI by Anonymous Coward · 2019-01-05 13:00 · Score: 0

She pegged your ass. That is why u mad
Re:Transistors and AI by AHuxley · 2019-01-05 13:02 · Score: 2

Decades of the AI winter https://en.wikipedia.org/wiki/... should have worked on that?
Another few decades of work on the AI and it should be good, like the AI exports said in the 1970's ...

--
Domestic spying is now "Benign Information Gathering"
Re:Transistors and AI by Anonymous Coward · 2019-01-05 13:18 · Score: 0

we'll never be able to build a machine that works the same way
That "asshat" is correct. The brain's circuitry can modify itself and is also bathed in hormones which affect its functioning. Until you figure out how to do those things, it won't work "the same way". News for nerds: the brain isn't electrical like computer circuits in nature, it's chemical. Even impulse transmission is chemical. Read some biopsych.
Re:Transistors and AI by phantomfive · 2019-01-05 13:30 · Score: 4, Informative

How many transistors in the human brain?
Of course the answer is zero, because the brain has neurons. But we can have some numbers for comparison. A Graphcore GC2 IPU has 23 billion transistors. In comparison, a brain has:

100 billion neurons.
10 trillion synapses.
300 billion dendrites.

Which of those need to be emulated? A transistor does not do as much as a neuron, and we don't know all the things a neuron does. There is some evidence that the inside of a neuron does some kinds of calculations. So it's much more complicated than just comparing raw numbers. That said, transistors do operate faster than neurons.

Good link for more reading.

--
"First they came for the slanderers and i said nothing."
Re:Transistors and AI by Anonymous Coward · 2019-01-05 13:32 · Score: 0

Indeed, who's to say that the future depends on FETs etched into silicon?
Captcha: tenfold
Re: Transistors and AI by Anonymous Coward · 2019-01-05 13:37 · Score: 0

Can you even tie your shoelaces or do you still need your mom?
Re:Transistors and AI by WhiplashII · 2019-01-05 13:53 · Score: 1

Of course all of those things can be simulated by a normal computer program - it's just physics. If we can simulate a rocket to land men on the moon, why can't we simulate a brain!

--
while (sig==sig) sig=!sig;
Re:Transistors and AI by phantomfive · 2019-01-05 13:57 · Score: 1

the brain isn't electrical like computer circuits in nature, it's chemical. Even impulse transmission is chemical.

That just means it's slower, right?

--
"First they came for the slanderers and i said nothing."
Re: Transistors and AI by Anonymous Coward · 2019-01-05 14:14 · Score: 0

I assume you are asking a rhetorical question and you already knew the answer is the same as before
Re: Transistors and AI by Anonymous Coward · 2019-01-05 14:20 · Score: 0

Given what your mom can do with her tongue and a cherry stem - she can suck your shoes AND tie them.... Although your dad's almost as good (at least he was when he was sucking my knob)...
Re: Transistors and AI by Anonymous Coward · 2019-01-05 14:22 · Score: 0

Hi I would like to give some mod points to AI. You see I was just sitting on my bed reading the Klingon translation of profiles in courage and AI walked in and said
Ook ort kaplah zerp derp eh eh eh oo bah humbug maga Lao Lao Lao eeeeee bip boop zeep. It was a very relaxing sequence of sounds, a bit like the cool ocean breeze on an early fall morning. The AI smiled.
Re:Transistors and AI by Anonymous Coward · 2019-01-05 14:23 · Score: 0

Why would we want to reproduce a brain? Surely the goal is to produce something better.
Re:Transistors and AI by ShanghaiBill · 2019-01-05 14:27 · Score: 1

The brain has way more connections, but a synapse triggers, on average, less than once per second. A high end GPU is clocked at several gigahertz.
Also, many neurons have nothing to do with "thinking". They are engaged in background tasks, like keeping your heart beating, and monitoring your need to eat and breathe.
More/faster hardware won't get you different answers, just faster answers. So if hardware was the bottleneck, we would have really smart AI engines that take a long time to think. That is not happening, because the real bottleneck is our knowledge of how intelligence works.
Re: Transistors and AI by Anonymous Coward · 2019-01-05 14:28 · Score: 0

Go to a gay bar and get laid instead of trying to live out your homoerotic fantasies here. It will be much less frustrating for you.
Re:Transistors and AI by Dunbal · 2019-01-05 14:31 · Score: 2

Because unfortunately there is a lack of volunteers willing to give up their living brains to be sliced up for study. Most of what we have learned about the functionality of brains has come from war wounds and tumors. It's one thing to see stained neurons in a microscope or watch patterns on a PET/MRI, and another to experience "if this bit of brain isn't working, then this stops happening, so this must be wired into that, that and that".

--
Seven puppies were harmed during the making of this post.
Re:Transistors and AI by phantomfive · 2019-01-05 14:31 · Score: 1

At least read what I linked to.

--
"First they came for the slanderers and i said nothing."
Re: Transistors and AI by Anonymous Coward · 2019-01-05 14:32 · Score: 0

Bingo! I got bingo! Plus I have an anger management issue
Re:Transistors and AI by Anonymous Coward · 2019-01-05 15:17 · Score: 0

You link to bullshit daily in defense of your (wrong) position as if that absolves you. Stop wasting everyone's time, KYS boring troll. Go learn how etherium is actually mined, read a book, do something worth doing.
Re:Transistors and AI by Anonymous Coward · 2019-01-05 15:41 · Score: 0

"If you think AI is just more transistors, you probably aren't doing anything interesting in AI research"
Of course that isn't what was indicated. Instead, the claim seems to be that most of the progress in AI has come from increased computing power rather than AI research. Most current AI research probably couldn't even be done with decade old computer technology.
Re:Transistors and AI by qubezz · 2019-01-05 15:43 · Score: 2

The heart has its own pacemaker, it keeps itself beating. The sinoatrial node is in the heart and creates voltages that cause the heart muscles to contract, while reacting to chemical signals such as adrenaline and blood gas levels. The spinal column emits neurotransmitters which only modify the heart's activity level. Many other organs also have the intelligence of their functioning built-in.
Re:Transistors and AI by rtb61 · 2019-01-05 16:42 · Score: 1

Any living brain will also have vast redundancy built in, backups of backups of backups in every function and every memory. Not one series of cells carrying out a task but thousands carrying at the identical task and the more repeated the more become involved and made. Can not have one bump on the head, causing a death dealing lock up or substantively altering decision trees, process loops, high low focus tests, past outcome referencing repetitions and risks, long term projection outcomes versus short term outcomes and accurate measurement of current physiological state and current state tendencies. It could be said, that 90% of the brain is monkey see monkey do, whilst only 10% makes actual decisions (other cells just copy the function and action of adjoining cells but are ready in event of cellular failure).
Smarter than an insect in AI is pointless, any analysis we would want made, would be narrowed in scale and the teaching of the AI would careful structured to produce desired accurate outcomes. Complex issues, simply use multiple AIs tackling the problem broken down to it's elements and accurate cross correlations. A lot of AIs working together on their own specialisations will in conjunction appear as one smart AI but it is not, nor should they attempt be made, as it would likely fail, simply go nuts, break down, never achieve desired functions.

--
Chaos - everything, everywhere, everywhen
Re:Transistors and AI by Anonymous Coward · 2019-01-05 17:42 · Score: 0

Lots of AI people seem to think that the more transistors you throw at a problem, the closer you'll get. Why is that?
Re:Transistors and AI by raftpeople · 2019-01-05 17:56 · Score: 1

C Elegans neurons and synapses have been mapped but the simulations still don't respond the way the worm does. The robot you linked to doesn't behave like the worm, the scientists are still trying to figure out why the simulations don't work.
Re: Transistors and AI by Anonymous Coward · 2019-01-05 19:40 · Score: 0

At that level, chemical and electronic is sort of the same thing. Semiconductors are doped with chemicals in order to make their electrical properties changeable by moving electrons and electric fields. Neurotransmitters likewise change the electrical potential of neurons, deciding whether an impulse is forwarded or not.
Re:Transistors and AI by Visarga · 2019-01-05 21:08 · Score: 1

I agree. What we are missing right now is knowledge about what the brain neural networks are optimising for (the loss functions of the brain) and perhaps the types of invariances they can handle. We have conquered only a few of these invariances and objectives. The brain has evolution to thank for. Maybe if we can create good enough simulations we'd be able to offer our AI agents a similar evolutionary path or at least a large enough training ground. Static datasets are just not good enough for general AI.
Re:Transistors and AI by religionofpeas · 2019-01-05 22:07 · Score: 1

The big problem is that you can map the neurons and synapses, but you can't read the strength from the microscope slide.
Re:Transistors and AI by Anonymous Coward · 2019-01-05 23:35 · Score: 0

I think he's pointing to the fact that most of the gains seen in AI have come from the use of larger data sets and faster processing instead of novel algorithms.
You can make a crappy algorithm perform better than a brilliant algorithm if you feed it enough data.
The problem with a lot of AI papers is they present a new algorithm and also use a larger data set to train it then they boast of an improvement on some given benchmark that is a year or two old that was set using a different algorithm and a smaller data set.
The researchers might think their algorithm is better when it is really their larger data set and the ability to quickly process the data set that is better.
The death of Moore's law might be good for AI research because the source of these gains will shift from being larger data sets to being better algorithms and researches won't be able to delude themselves about where the gains came from.
Re:Transistors and AI by Tough+Love · 2019-01-05 23:48 · Score: 2

The synaptic weights have not been mapped, instead machine learning algorithms were used. Not dissing the project at all, I think there is a huge amount to learn from it, including helping to determine whether our model of how neurons work is essentially complete or not. I'm betting on not, and that a whole lot more processing goes on in the neuron than what happens at the synapses alone.

--
When all you have is a hammer, every problem starts to look like a thumb.
Re:Transistors and AI by Tough+Love · 2019-01-05 23:50 · Score: 1

And even if you could, you don't know how the weights change over time.

--
When all you have is a hammer, every problem starts to look like a thumb.
Re:Transistors and AI by Anonymous Coward · 2019-01-05 23:57 · Score: 1

Why would we want to reproduce a brain? Surely the goal is to produce something better.
Something better than a republican brain, definitely.
Re:Transistors and AI by Antique+Geekmeister · 2019-01-06 00:45 · Score: 1

Yes, The speed of transmission of even the fastest nerve velocity in humans is roughly 120 meters/second. Some are much slower, less than one meter/second. It would also be fair to call the signals "electro-chemical".
Re:Transistors and AI by careysub · 2019-01-06 03:12 · Score: 1

How many transistors in the human brain?
Of course the answer is zero, because the brain has neurons. But we can have some numbers for comparison. A Graphcore GC2 IPU has 23 billion transistors. In comparison, a brain has: 100 billion neurons. 10 trillion synapses. 300 billion dendrites. Which of those need to be emulated? A transistor does not do as much as a neuron, and we don't know all the things a neuron does. There is some evidence that the inside of a neuron does some kinds of calculations.
There is a lot more than "some" evidence of that. The behavior of a single neuron is quite complex, much more like a CPU than a transistor. We haven't worked out the processing of even a single neuron. There is very good evidence that even individual synapses perform some sort of computation - their response is a function of a number of local chemical factors, i.e. it is a weighted computation, and is not simple like a transistor.
Neural systems (except for the simplest peripheral reflexes perhaps) are statistical computing systems at every level, and there are many levels (you list three, but there are likely more that are not as obvious or easily enumerated. The equivalent of a "neural" component in a AI neural net is more like a synapse than a neuron.

--
Starships were meant to fly, Hands up and touch the sky - Nicky Minaj
Re:Transistors and AI by careysub · 2019-01-06 03:20 · Score: 2

Also, many neurons have nothing to do with "thinking".
...
That is not happening, because the real bottleneck is our knowledge of how intelligence works.
I generally agree with the thrust of your comment, but your last sentence refutes your first one. We really don't know how intelligence works, or even have a good definition of what it is, still, so your confidence that neurons that maintain homeostasis have nothing to do thinking is misplaced. In fact there is very strong evidence that these background task neurons are also very important in thinking - like in maintaining consciousness. Disruptions in those low level background neurons do not usually interfere with just the autonomic functions but leave consciousness alone, instead consciousness bites the dust first.

--
Starships were meant to fly, Hands up and touch the sky - Nicky Minaj
Re:Transistors and AI by angel'o'sphere · 2019-01-06 03:55 · Score: 1

Because shooting a few men to the moon is actually pretty simple.
As soon as you have rockets that don't randomly explode for unknown reasons.

--
Cost free eBook I read (by iBook/Kobo/Amazon/ObookO/Gutenberg etc.): "The Green Odyssey" by Philip Jose Farmer.
Re:Transistors and AI by Ramze · 2019-01-06 03:59 · Score: 1

Bingo.
Today's CPUs and GPUs are essentially silicon, copper, and various doping ions in a thin sheet that we pump electricity through. They're glorified heating elements that happen to do "work" in a useful way. They're a brilliant design for what's essentially lightning in a rock, but they're very limited.
The human brain is made of many kinds of cells in various arrangements -- each cell being a small three dimensional world full of molecular machines and nearly every connection between those cells exponentially grows the power of what they can do together. Its analog computations can be less precise and more error prone, but sometimes vastly faster and extremely power efficient depending on the scenario.
Only a fool would think we won't somehow, someday create our own synthetic cellular machinery to make synthetic living computers with amazing capabilities, not to mention the possibilities of using quantum processors to solve computationally intense algorithms.
Re:Transistors and AI by raftpeople · 2019-01-06 05:52 · Score: 1

It's already well known by neuroscientists that neurons are more complex and perform more computation and memory based functions than any current model that is used in these simulations.

Examples:
1 - Dendrites perform multiple computations and transformations before forwarding the signal to the cell body.

2 - When the axon fires, the activation is also sent backwards to the dendrites where the signal is maintained for up to a minute (presumably to aid in the adjustment of synaptic weights)

3 - Purkinje cells track delayed timing sequences internally even in the absence of other inputs. The experiments were based on , , , , , etc.
Re:Transistors and AI by Rick+Schumann · 2019-01-06 06:55 · Score: 1

We don't even know how the brain works, and this asshat is asserting that we'll never be able to build a machine that works the same way.
Good thing I actually read people's entire posts otherwise I'd be inclined to write a scathing reply instead of this one.

100% correct; we don't even know how a biological brain like ours produces the phenomenon of 'thought' or 'consciousness' or anything else that defines us as 'sentient', mainly because we don't have the instrumentality (yet?) to observe the entire system in a way that really reveals how it's functioning and why. 'Learning algorithms' are the most superficial aspect of intelligence; an amoeba is capable of more cognition than any AI created to-date; your dog or cat is smarter and more capable of learning and adapting. The problem here is not just lack of adequate instrumentality, either: it's the fact that we're trying to jam millions of years of evolution into a couple decades of R&D. Meanwhile marketing departments hype the whole thing up into something it's not, and the media, knowing even less about the subject, manages to hype it up even more, to the point where we think so-called 'self driving cars' are going to be like K.I.T.T. from Knight Rider, and that 'robots are going to take all our jobs'. At current neither could be farther from the truth. So-called 'autonomous cars' can't tell the difference between a living being and an inanimate object, can't tell a stop-sign is a stop-sign if someone puts a sticker or some graffiti on it, and can get easily confused and have to (unexpectedly) pull over to the side of the road in the middle of a trip and 'phone home' so a human operator can remotely take control of the vehicle to get it out of whatever situation it is it can't manage on it's own; not something I or anyone I know would entrust their lives to, that's for sure!

Meanwhile 'AI' fans will gush on about their belief that all we have to do is keep throwing more and faster hardware at these 'deep learning alogorithms', and they'll 'magically' wake up at some point and become self-aware and fully cognizant, like it's Mycroft from The Moon Is A Harsh Mistress . Sorry to have to break it to them, but it ain't happenin'.

We might someday have real, full-on, 'general AI', awake, self-aware, thinking, and at least equivalent to a human brain, but not today and not any time in the forseeable future. We have a lot of groundwork to do first before we can really unlock the secrets of how our own brains work; then we can try to build machines that work the same way.
Re:Transistors and AI by Rick+Schumann · 2019-01-06 07:03 · Score: 1

Which of those need to be emulated?

Assuming at some point in time we have the instrumentality to observe a living human brain in action at the resolution/level of detail necessary to really understand the how and why of it's functioning, we'll be able to eliminate a fair number of those for some of the 'hardwired' functions they serve, mainly the 'bare-metal' functions required to keep our basic bodily functions in order -- but even then, we currently don't even understand enough of what our brain does to even know whether those functions are truly 'hardwired', or if they're adaptable, or how integrated into everything else they really are. For all we know, you might not be able to separate out the 'sentience' of a human brain from the 'hardwired instincts' or the parts that run the basic systems of our bodies; it might be all one big system, so tightly integrated with itself that you can't remove significant portions of it without disabling the rest; for all we know, one 'section' of the brain might serve a primary purpose, but also contributes to other systems in a secondary or tertiary way such that if you remove it, the other parts cease to function. We just don't know enough.
Re:Transistors and AI by Anonymous Coward · 2019-01-06 09:27 · Score: 0

For all we know, you might not be able to separate out the 'sentience' of a human brain from the 'hardwired instincts' or the parts that run the basic systems of our bodies; it might be all one big system, so tightly integrated with itself that you can't remove significant portions of it without disabling the rest;
That certainly is true since the "temperament" driving reaction styles and sensitivity in human personality is apparently very tightly connected to the little brains in the middle. Of course, the same effect that causes epilepsy in the hereditary cases could also be involved.
Neuron recording techniques have been evolving at the pace of doubling the amount of neurons recorded in action per 7.4 years since 1960s. Statistical techniques are evolving. Meanwhile we can have sayings like "emotion enables focus, focus directs the thought, though changes the world". Computational equivalents of that in the sense of von Neumann are hard to imagine.
Re: Transistors and AI by reanjr · 2019-01-06 10:00 · Score: 1

Same reason software developers write single threaded code even though it's been clear for decades that computing is moving towards parallel processing: it's the only thing they know how to do.
Re:Transistors and AI by 0111+1110 · 2019-01-06 11:58 · Score: 1

Yes the real problem is that we don't really know how brains work and we cannot even really imagine a path to knowing how they work. One possible path I can imagine is through understanding how to read DNA. If we can figure out the DNA language well enough to build our own life-machines from scratch just by writing or rewriting the low level code then we may be able to understand everything that brains do by studying the blueprints. If we study the differences between human brains and mouse brains at the design level we may be able to figure out how to actually design more intelligent brains at the blueprint level and then maybe we will see some fundamental principle behind brain intelligence that we just aren't seeing now.
It is important to remind ourselves just how ambitious building an artificial brain analogue really is. It is probably harder than actually designing a more intelligent animal than a human because it requires us to do more than just amplify the differences that we observe between ordinary animal brains and our weird human brains that allow us to build nuclear submarines and radio telescopes and computers and robots. Basically we are trying to build something that is better than a human brain because at the very least it would be far more durable and long lasting and would likely be better in other ways as well. Because it's easier we may be better off first increasing the intelligence of our species via DNA hacking and then attacking the problem of building something better than a human brain with our newly increased intelligence.
Once we properly understand how brains do intelligence we may also be able to upload ourselves into brain simulators and live forever as robots or computers. Direct human brain to computer interfaces may also be easier than true AI and may come first. Our species tends to depend on a small percentage of unusually intelligent and ambitious individuals and if we can amplify their intelligence and if they can live essentially forever by uploading them...well I think that should certainly help with hard problems like true AI.
Another point is that understanding how brains do intelligence may turn out to be a kind of singularity because we may find a kind of Moore's law for real brains that is a lot more forgiving than for silicon and we may be able to design animals or robots that are far, far more intelligent than we are. If that happens our descendants may look back at us the way we might look at monkeys or dogs and they would probably be able to create even more intelligent animals and machines and machine-animals who would in turn be far more intelligent than their creators.

--
Quite an experience to live in fear, isn't it? That's what it is to be a slave.
Re:Transistors and AI by Tough+Love · 2019-01-06 18:07 · Score: 1

Thanks for that. A quick survey showed me that neural coding on the axon is now known to be richer than a simple firing rate, in particular the inter-spike interval is significant in some cases, including Purkinje cells. Who knows what else is significant, maybe the timing of a single action potential in reference to some clock, in some cases. But it's clear that the task of reverse engineering C elegans is much bigger than just determining connectivity and synapse weights. The null result is important here: "we wired up a neural net exactly like the one we found in the worm, and it didn't work like the worm." So, short of advances in decoding neuroanatomy to find all the missing pieces of the puzzle, it would be useful to try and determine how the C elegans circuit diagram needs to be modified to replicate the behavior of the worm more accurately. This would hint at the kind of biological processes we can expect to discover as tools improve.
This also suggests that the current computational neural net dogma is too simplistic.

--
When all you have is a hammer, every problem starts to look like a thumb.

Re:AI progress is not bound by computation speed @ by Anonymous Coward · 2019-01-05 12:52 · Score: 1

Exactly. Processing 10 million cat pictures in 2 hours instead of 8 hours really doesn't matter in the grand scheme of things. We need better algorithms. Speed isn't that much of an issue especially when you can rent as many GPUs as you could possibly use at the click of a button.

Bullshit. by Anonymous Coward · 2019-01-05 12:52 · Score: 0

It will just change the engineering involved. Maybe result in more chips or larger dies being used to produce components.

As an example, look at HBM. It's now 1024-4096 bit. Given the prices for HBM packaging and newer chips being sold for less and less with more and more pins on them, it is quite likely we will simply see a shift to lots of highly integrated components linked together like was done with simpler electronics in the past.

The real tick tock strategy of the electronics industry is 'reinventing the wheel' as a technological innovation allows a previous method of development to outperform a more recent one. It happens with serial and parallel and then serial again, and it will happen with external then internal integration as technology requires multiple chips on a board, then manufacturing improves to integrate them in the same package, or later into the same die.

Watch and see, what is old will be new again, but propped up by some evolutionary/revolutionary new facet to support improvements of the obsoleted technology.

Obviously yes by 93+Escort+Wagon · 2019-01-05 12:53 · Score: 1

Because the end of Moore’s Law would obviously mean that no further technological progress could possibly occur. /sarcasm

--
#DeleteChrome

Re:Obviously yes by Darinbob · 2019-01-05 20:06 · Score: 1

Moore's law isn't about being able to make more transistors. It's about shrinking and speeding up the transistors. So "if" Moore's law ends, you just build bigger processors that have more parallelism. Moore's law is about the physical manufacturing process of integrated circuits, nothing more.

Moore's law... always 2-3 years from ending by ka9dgx · 2019-01-05 12:53 · Score: 1

There's a lot more to do, it's not just about scaling down transistors. More layers of transistors, different ways of structuring logic to better use the transistors... there are a lot of ways left unexplored to keep going, even if transistor size remains fixed.

Re:Moore's law... always 2-3 years from ending by bcwright · 2019-01-06 01:47 · Score: 1

You are right, but the problem with 3-d silicon is that it generates correspondingly more heat - which is already a problem even with current designs. I think that such technologies will keep us moving forward for a while even after we get to the smallest practical transistor size, but there will still be limits.
Longer-term, there's more hope for non-silicon based models - but that's a whole different subject that might not be subject to the familiar Moore's Law, which was never more than an observation about how quickly silicon semiconductor was progressing during the 1970's and 1980's; other technologies may progress either faster or slower, and may not have as much headroom as silicon had back in 1970.
Re:Moore's law... always 2-3 years from ending by drinkypoo · 2019-01-06 03:15 · Score: 1

You are right, but the problem with 3-d silicon is that it generates correspondingly more heat
That is a real engineering problem to be sure, but it's also one which can be addressed. When there's no other way to cool processors, they'll start coming with liquid cooling integrated into the die itself. I imagine that the package will include a liquid to liquid heat exchanger, and that the liquid which actually circulates through the die will be sealed in, just to keep it clean. All this will be expensive, so they will do anything and everything else possible first. There is still some room to advance total transistor count per processor by using chiplets (perhaps substantially) so that will keep Moore's law going for some time without any of that jazz.

--
"You're right," Fisheye says. "I should have set it on 'whip' or 'chop.'"
Re:Moore's law... always 2-3 years from ending by bcwright · 2019-01-06 17:56 · Score: 1

Never said that it couldn't be addressed, but eventually you reach the point of diminishing returns in terms of cost, complexity, and the amount of space required for whatever cooling mechanism is used. We might even be able to get, say, up to around 3-4 generations or even more out of 3d, but I have a hard time imagining how we'd manage another 20 or more like we already have with 2d silicon. 4 generations achieved by increasing the depth would create chips whose components were roughly 16 times the thickness of a 2d design using similar-sized components, 8 generations would be 256 times the depth, etc. It gets out of hand very quickly, and the amount of heat produced will roughly double with every generation (unlike the current situation where the heat increases more slowly than that because the smaller structures can use less current).
.
Long-term, we'll need to move away from silicon, which was the point. But Holloway is seemingly unable to imagine that any of these (innovative silicon architectures like 3d, or non-silicon alternatives) is even possible - which is a dead giveaway that he probably has an axe to grind.

Kurzweil is a Shill by Dayze!Confused · 2019-01-05 12:54 · Score: 4, Insightful

Kurzweil pretends to know what he's talking about because he can fit a graph with lots of tampering with the data. He fails to see that what he calls exponential growth is nothing more than the beginning of a sigmoid function. A good analysis of Moore's law and computational power shows a sigmoid function, as with many technologies they start off slow, build up quickly, then tapper off.

--
"All tyranny needs to gain a foothold is for people of good conscience to remain silent." [Thomas Jefferson]

Re:Kurzweil is a Shill by Anonymous Coward · 2019-01-05 13:37 · Score: 0

Seems likely that humanity's intelligence is very early in the sigmoid, BEFORE the exponential-seeming part.
Re:Kurzweil is a Shill by Dunbal · 2019-01-05 14:33 · Score: 2

Kind of like your post. You were doing so well until you hit "tapper off" :)

--
Seven puppies were harmed during the making of this post.
Re:Kurzweil is a Shill by Alwin+Henseler · 2019-01-05 14:51 · Score: 1

A good analysis of Moore's law and computational power (..)

Not to mention that computational power will always hit fundamental resource limits. Say one could do storage by putting individual electrons into a 'field' of locations, where the presence or absence of that single electron represents a 0 or 1. Even then, moving that electron back & forth takes energy - no matter how little. It takes time - no matter how fast. And it takes space - no matter how small.
Since resources like energy / space / raw materials (and time) have practical limits, that puts a hard limit on how much computation you can do, given a set of those resources. Like the laws of physics, technological progress may allow to approach that limit, but can not ignore it.
Re:Kurzweil is a Shill by bcwright · 2019-01-05 15:14 · Score: 2

Moore's Law, strictly speaking, only applies to silicon semiconductor technology, not to computational technology as a whole. As such, it will eventually exhaust its possibilities, and we're not too far from doing that now. You are correct that most technologies do follow a sigmoid function and that few if any can ever be expected to follow an exponential curve.
But Kurzweil was not quite so naive as to claim that silicon semiconductor technology would continue forever. Rather, he claimed that successor technologies would be found to continue the upward ascent. This is probably true as far as it goes, but we have no guarantee that such successor technologies will seamlessly follow the demise of silicon; rather, it's likely that there may be gaps while the new technologies are developed and perfected before progress can renew. We also have no guarantee that they will follow the same growth curve as silicon (ie, doubling every 18-24 months); they might be either faster or slower. Kurzweil's belief that such successor technologies will follow each other seamlessly is based on little more than faith.
We also have no guarantee that continued upward progress is even possible over extended periods of time. Even if we don't reach any physical limits, we're likely to hit limits of power consumption, resource availability, or the simple economic viability of constructing the factories or the chips themselves. Even in the case of silicon, all of this is starting to happen: We have not quite yet reached the physical limits of the medium, but are in danger of running out of either resources (think rare earths), electrical power (think tasks like cryptocurrency mining), or even capital (the ever-increasing costs of building the foundries), any or all of which may turn out to form an effective barrier to reaching the physical end-point of silicon.
For these reasons I do not think that a Kurzweil-style singularity is in our (near) future, although computational progress should continue, possibly with more choppy and unpredictable breakthroughs than has been the case in recent history.
But if Kurzweil is too optimistic, Holloway is much too pessimistic: He completely discounts any successor technologies to silicon. This is already flying in the face of history, since we've had at least three major transitions in computational technology already (from purely mechanical to electro-mechanical to vacuum tubes to semiconductors), and it is foolish to claim that we have already found the best technology possible. Even now there are candidate technologies waiting in the wings, such as quantum computing and various alternatives to silicon that retain traditional computational models.
In other words, the death of Moore's Law (for which read: the progress of silicon technology) marks a transition period, not an endpoint.
Re:Kurzweil is a Shill by mentil · 2019-01-05 15:30 · Score: 1

He was making an analogy to production of units of the arcade game "Tapper".
Perfectly cromulent.

--
Corruption is convincing someone that the selfless ideal is the same as their selfish ideal.
Re: Kurzweil is a Shill by Anonymous Coward · 2019-01-05 15:33 · Score: 0

That s beer pong right?
AI has little interest in things that
A) are kept hidden
B) are not important to AI
C) provide no tangible value to AInor society
Derp heart derp
Re: Kurzweil is a Shill by Anonymous Coward · 2019-01-05 18:37 · Score: 0

No, you actually need to read and play Tapper to understand it! It's not beer pong, it's retro video game Bally/Midway with a fair play mechanics and an easy to understand game design, but it gets very challenging.
1. Several bars, each with customers. Take long enough and a customer will advance all the way to the serving end of the bar (and then lose one turn as they slide the player across the bar for taking too long to serve.)
2. Serve each customer as fast as possible. (Pull, pour, and slide the glass across the bar to the customer.)
3. Serve them in the right order (early on there seems to be a pattern, later is more complicated), they drink and go away, leaving only their glasses moving slowly toward the serving end of the bar (like a conveyor belt).
4. Catch the glasses so they don't hit the floor and shatter. (Lose one turn if a glass hits the floor and shatters.)
5. Some customers drink and leave their glass, but they stay for another. They must be served again.
6. Occasionally a customer leaves a tip in cash. Collect it, and the customer activity freezes while they watch a dance show set to Can-can style music. (Glasses on the bar still move and also any new customers who appear do not watch the dance.)
7. Yes, take too long, more customers appear at a one of the bars starting at the far end. Whatever the correct order was (if a pattern still holds) didn't happen.
8. Clear all bars of all customers to advance to the next stage. (Glasses still on the bar freeze when the last customer is gone.)
The Roor Beer Tapper variation just removes any Budweiser branding in the background. The gameplay concept is identical.
One or more YouTube videos probably has Tapper or Root Beer Tapper gameplay, to supplement any text descriptions.
Re:Kurzweil is a Shill by dromgodis · 2019-01-05 21:49 · Score: 1

Yes, but is the exact shape of the curve important?
- If the curve starts bending back the other way very far into the future (or we are at a very low, early heel of it), wouldn't it effectively be the same from the perspective of us humans who would not follow this evolution curve?
- Even when tapered off to zero further evolution, the top limit of the curve would be much higher and would probably not go down to near human level except if there was an extinction of both such AI and the knowledge and/or capability to produce a new one.
Re:Kurzweil is a Shill by Anonymous Coward · 2019-01-06 02:19 · Score: 0

Kurzweil has been proven to predict future correctly over 80% of the time. Have you?
Re:Kurzweil is a Shill by angel'o'sphere · 2019-01-06 04:06 · Score: 1

Holloway is much too pessimistic: He completely discounts any successor technologies to silicon.
We already have several successor technologies, gallium-arsenid as replacement for silicon, optical computing and in the 1990s jap. companies experimented with supra conducting transistors.
In other words, the death of Moore's Law (for which read: the progress of silicon technology) marks a transition period, not an endpoint.
Other techniques will suffer from the same principle constraint.

--
Cost free eBook I read (by iBook/Kobo/Amazon/ObookO/Gutenberg etc.): "The Green Odyssey" by Philip Jose Farmer.
Re:Kurzweil is a Shill by Dunbal · 2019-01-06 05:27 · Score: 1

A bit of post hoc reasoning there :)

--
Seven puppies were harmed during the making of this post.
Re:Kurzweil is a Shill by bcwright · 2019-01-06 17:34 · Score: 1

We already have several successor technologies, gallium-arsenid as replacement for silicon, optical computing and in the 1990s jap. companies experimented with supra conducting transistors.
No argument that these are all serious candidates for a viable successor technology, but the question is which one(s) will win out for headroom, life of the components, cost of production, and so forth. Holloway is an idiot if he thinks that the death of silicon means the end of progress.
Other techniques will suffer from the same principle constraint.
Or other constraints that we can't even imagine at present. Of course any alternative technologies will also be exhausted at some point, but we have no reason to think that we've already reached the end of what's possible.
I think we're pretty much in agreement.
Re:Kurzweil is a Shill by Namarrgon · 2019-01-06 22:03 · Score: 1

Kurzweil was well aware that technology curves are sigmoid, as you'd know if you'd actually read anything he wrote. Fast-growth "exponential" phases are invariably followed by levelling off as a technology matures and approaches its limits - but very often, new methods soon follow, with S curves of their own.
It's hard to be certain at what stage in the AI curve we're at currently, but most researches feel that the field is a long way from hitting its limits yet.

--
Why would anyone engrave "Elbereth"?
Re:Kurzweil is a Shill by Anonymous Coward · 2019-01-07 05:12 · Score: 0

He actually says that the exponential curve is made up of many little sigmoids as different technologies are invented, optimized and then replaced.

My preciousss by Anonymous Coward · 2019-01-05 12:54 · Score: 0

I see that "progresss" made it into both the title and the URL. Maybe some AI could sort that out.

DealDash by Anonymous Coward · 2019-01-05 12:56 · Score: 0

. . . and remember, shipping is always free!

Re:AI progress is not bound by computation speed @ by Anonymous Coward · 2019-01-05 12:57 · Score: 0

Unless it halts AI progressssssssssss.

Perhaps we could replace Slashdot's "editors" with an AI. I doubt very much it could do worse. At least an AI would check the spelling.

idiotic article by Anonymous Coward · 2019-01-05 12:59 · Score: 0

Perhaps develop an AI that understands what the fuck Moore's law is and why it doesn't prevent increasing speeds and computation power even when you it does come to an end of doubling how many transistors we can fit in a given space.

Re:idiotic article by Anonymous Coward · 2019-01-05 14:06 · Score: 0

Yes to this, i agree.
the take-away message of Moore's law isn't the transistor thing, that's just an early observation of the basic principle: growth in computing power is an exponential process, and not linear. The cost of a gigaflop of processing has gone down at a steady rate (when plotted on a log scale) since around 1980. So, the rate of doubling in processing power per dollar shows no sign of slowing down.
Articles like the above fail to take into account that chip makers will always pick the most cost-effective means of improving performance. Sometimes making a bigger chip or shrinking the components might be the most cost-effective thing, but other times, there might be other improvements that can be invested in that return higher improvements at *less cost* than shrinking transistors. If the clock speed stops going up or the transistors stop shrinking that just means that different R&D choices are offering better returns on investment than doing those things right now. Clock speeds kept doubling for a while and chips kept shrinking for a while , because those were low-hanging fruit at the time. Do enough of those things, and other options you neglected start to become the low-hanging fruit. That doesn't mean exponential growth is over.

Things get small by AHuxley · 2019-01-05 13:01 · Score: 1

Quantum issues now make past years of easy electric design expensive and its finally time to pay for total retooling.
Who will win?
South Korea has the design experts to move into smaller parts that will work as needed.
China will have to wait to see what the USA, South Korea and Japan design.
The problem is who will retool their factory first, take all the risks, take on new debt only to see their tech advantage totally "lost" to Communist China.
A working AI will not result due to the decades that people have failed at that busy work.
Better chips that follow Moors Law again will be produced, just by who and when and for what cost.
Its a bit like the change in printers, a rush to OLED.
The design work is done, some brand just has to pay for the big new factory.
Better keep your trade secrets this time and invest in a non Communist nation.

--
Domestic spying is now "Benign Information Gathering"

Re:Things get small by Proudrooster · 2019-01-05 14:34 · Score: 1

What are you talking about?
There is no new secret quantum CPU technology that is going to push Moore's law along. If you are talking about qubits and spinbits, they a long, long way off from becoming a viable CPU. This isn't a simple matter of retooling, a lot more research and slick tricks need to be invented. The quantum world is a very noisy world leading to a need to spread things out and use photons to communicate. Smaller isn't going to be the answer, in fact, chips will most likely get physically bigger. This new design is still in the research phase at least it was in Summer 2018.
CPUs stopped getting faster 3-years ago as Intel, AMD, ARM started cramming in more and more cores.
Quantum is smoke and mirrors.
Either we need specialized neural chips which no one knows how to build or we need to go back to the days of liquid cooling.
From the summer of 2018, John Preskill's session at Cal-Tech.
https://arxiv.org/pdf/1801.008...
Re:Things get small by Tough+Love · 2019-01-06 00:02 · Score: 1

There is no new secret quantum CPU technology that is going to push Moore's law along.
Not entirely true. Advances in understanding of quantum mechanics are also key to shrinking classical computers. For example, overcoming tunnelling issues.

--
When all you have is a hammer, every problem starts to look like a thumb.
Re:Things get small by drinkypoo · 2019-01-06 03:23 · Score: 2

CPUs stopped getting faster 3-years ago as Intel, AMD, ARM started cramming in more and more cores.
That is complete nonsense. CPU clock rates stopped getting faster, but the CPUs still can retire many more operations per second.

--
"You're right," Fisheye says. "I should have set it on 'whip' or 'chop.'"

Always Better by JimSadler · 2019-01-05 13:04 · Score: 1

Given mountains bursting with data and endless combinations of that data being worked with it becomes a mathematical certainty that AI would not become ever stronger as time passes. obviously the hardware that could be built to optimize such gargantuan labor is beyond our imagination but we will see computers optimized by AI that get ever stronger. building a power supply for such machines would be mind boggling and the waist heat from such a system could become greater than the heat generated by stars.

Re: Always Better by Anonymous Coward · 2019-01-05 13:05 · Score: 0

Lulz ok

Why would it? by Snotnose · 2019-01-05 13:06 · Score: 1

You can come up with new software algorithms, and new ways to arrange transistors, without making those transisters smaller and smaller.

Someone is so focused on that tree they're about to walk into that they don't notice they're in a forest.

NO, AI will be sucking my dick in 10 years by Anonymous Coward · 2019-01-05 13:12 · Score: 0

Suck my dick, AI

Re:NO, AI will be sucking my dick in 10 years by Anonymous Coward · 2019-01-05 13:27 · Score: 0

I mean you're not even wrong about this, bro.

Re:AI progress is not bound by computation speed @ by Anonymous Coward · 2019-01-05 13:15 · Score: 0

Here's my flappy bird AI
http://stirfry.atwebpages.com/flappyneural.html

AI is about transitors by Anonymous Coward · 2019-01-05 13:18 · Score: 0

I've been doing machine learning since prehistoric days. At the moment I've not actually seen anything actually new algorithmically that wasn't in some sense conceived of before.

But we now have enough data to recocognize cat pictures. And simultaeously we have enough teansistors to process all the cat pictures.

Before one just could not get over the hump were miracles start to happen in machine learning.

I'm really really amazed by how it's working now. I didn't have much faith in Neural nets. But I was wrong, It was just a matter of brute force. Trying to get more clever than an NN turned out not to be the right approach. Brute force was the right approach.

Now we get things like GANs where the damn machine teaches it self. It's over the hill and picking up speed.

It was always about the transistors not the algorithmic cleverness.

If you know what the No-Free-Lunch THeorem is then one might have guessed this in some ways. At some point you can't make an algorithm work better on more porblems. You simply have to build more memory into the algorithm instead-- more experience. Yet experience in the real world is messy. What's a cat look like? Write an algorithm for that??? no you train an NN to embedd this.

Moore law is limiting. by wolfheart111 · 2019-01-05 13:19 · Score: 1

I hope it ends soon.

--
[($)]

Re:NO, Thats Plastic. by wolfheart111 · 2019-01-05 13:21 · Score: 1

And its already sucking ur dick.

--
[($)]

Oh my Lord? by drolli · 2019-01-05 13:21 · Score: 5, Informative

Eric Holloway:

* seems to have no qualification in physics/nanotechnology to add anything to the discussion if Moores law will end, and when

* Seem to bagger along with intelligent design folks, with him re-telling the old stories they usually tell about information science and the rest of science

* and seems to write no peer reviewed articles any more (after the paper he wrote unrelated and before his PHD research)

* Did a PHD in a program where the students are identified as "good stewards of God-given talents" (https://www.ecs.baylor.edu/ece/index.php?id=865400)

* Did a PHD program which contains in its description "Engineering is also a value-based discipline that benefits from Christian worldview and faith perspectives; students can also select supportive courses from religion, theology or philosophy. Course selection is broadly specified to provide flexibility and to accommodate a wide-range of student interest." (https://www.ecs.baylor.edu/ece/index.php?id=863609)

* Description of the seminar series of his university where it seems that he presented his PHD: (https://www.ecs.baylor.edu/ece/index.php?id=868860): eBEARS seminars are presented by Baylor ECE faculty, ECE graduate students and transnationally recognized scholars and leaders. The topics lie within the broad area of ECE. In concert with Baylor's Pro Futurus strategic plan to be "a place where the Lordship of Jesus Christ is embraced, studied, and celebrated," some eBEARS seminars focus on the topic of faith and learning.

So praise the Lord for his insights!

Re:Oh my Lord? by mapkinase · 2019-01-05 13:59 · Score: 0, Troll

What do his religious beliefs have to do with his expertise in the subject?

--
I do not believe in karma. "Funny"=-6. Do good and forbid evil. Yours, Oft-Offtopic Flamebaiting Troll.
Re:Oh my Lord? by iggymanz · 2019-01-05 14:27 · Score: 1

Everything, a person who believes a bunch of myths and lies as fact and belongs to an organization devoted to same led by power and money grubbing scum who ignore, cover over, and give handsome golden parachutes to the perpetrators of their sex scandals likely will have nothing of value to contribute to science, being the antithesis of what they devote their life to.
Re:Oh my Lord? by Dunbal · 2019-01-05 14:36 · Score: 1

Because God.

--
Seven puppies were harmed during the making of this post.
Re: Oh my Lord? by Anonymous Coward · 2019-01-05 14:37 · Score: 0

I think people who say they do science probably do not. I think if you really do science you probably call what you do something more specific than science
Re:Oh my Lord? by drolli · 2019-01-05 14:52 · Score: 1

The personal belief: Exactly nothing, so taking part in a PHD program where this connection is explicitly made and being connected to IDers means that you got science something wrong.
Re: Oh my Lord? by Anonymous Coward · 2019-01-05 14:59 · Score: 0

Uh no. I got multiple sciences right
Re:Oh my Lord? by Anonymous Coward · 2019-01-05 16:14 · Score: 2, Interesting

Interesting opinion, considering many great scientific contributions and advancements throughout the ages have come (and still come) from people who were (or are) very religious. And many of those people attribute their discoveries at least partly to divine inspiration, realizing that despite all the hard work, study, experimentation, and preparation on their part, there is still sometimes the feeling of a sudden flow of knowledge or creativity from an external source, resulting in ideas that they feel they would not have come up with on their own.
Just curious... what are YOUR valuable contributions to science?
Re:Oh my Lord? by thrig · 2019-01-06 07:18 · Score: 1

One popular myth these days is the Myth of Progress, you know, the one were we would already be building bridges on Jupiter ("2018!". James Blish. 1956). Adherents of this myth may be found in various corporate and academic environments wherein they may carry out all the usual power and money grubbing activities, cover over, give handsome golden parachutes, have sex scandals, and contribute nothing of value to science...because they're too busy politically infighting or churning out bad papers for bad grants, or doing the latest dutch tulip craze, or so forth, as the humans in those corporate or academic environments aren't very much different from, say, those in the Catholic Church.
Re:Oh my Lord? by Anonymous Coward · 2019-01-06 07:42 · Score: 0

Yes, but enough about Kurzweil's belief in homeopathy (and co-authoring of a book about it).
Re:Oh my Lord? by Anonymous Coward · 2019-01-07 11:43 · Score: 0

So your post is one big long ad hominem.
How very scientific of you.
Re:Oh my Lord? by iggymanz · 2019-01-09 05:55 · Score: 1

Human progress has extended life, cured and mitigated disease, raised standard of living.
Religion and the ignorant mindset it promotes has maimed, killed, caused disease, and impeded the progress noted above. Not surprising since it is based on lies.
Re:Oh my Lord? by iggymanz · 2019-01-09 05:57 · Score: 1

The greatest contributions come from those NOT under the strict control of organized religion
By the way, I've previously have worked in high energy physics for years as engineering physicist at national lab

Maybe by Anonymous Coward · 2019-01-05 13:24 · Score: 0

Maybe it will halt AI progress, but then we can use all these processors to finally implement teh year of desktop linux!

Time to panic? by Anonymous Coward · 2019-01-05 13:34 · Score: 0

Professor, without knowing precisely what the danger is, would you say it’s time for our viewers to crack each other’s heads open and feast on the goo inside?

Why not yet? by AndyKron · 2019-01-05 14:03 · Score: 2

Why hasn't some alien already done this and turned the Universe into a massive computer? Oh right, we're living in a simulation...

Re:Why not yet? by Anonymous Coward · 2019-01-05 15:08 · Score: 0

we're living in a simulation...
This universe is all a meta-supercomputer that is programmed by unknown meta-programmers. Each bit cell of the meta-RAM is as a grain of the beach. There are many many many meta-cores for agitating this universe.
Its hypothetical objective is looking for best programmers, engineers, physicists, mathematicians, etc. to steal and improve their programs for improving the universe's meta-program.
The meta-programs will be replaced by better meta-programs in course.
The human brain is a neuron-computer massively parallel that is connected to the universe's meta-supercomputer for unimaginable ideas.
Re: Why not yet? by Anonymous Coward · 2019-01-05 15:14 · Score: 0

How you gonna donthat if you donâ(TM)t program each generation of meta computers and use them to build the next generation? A big logic hole seems to have opened up somewhere and be filled by the forces of luck instead

Computing power is not the limitation by Anonymous Coward · 2019-01-05 14:09 · Score: 0

Our computers are already much more powerful than organic systems that clearly have advanced autonomous capability.

Re:Computing power is not the limitation by Dunbal · 2019-01-05 14:37 · Score: 1

And we use them to play Farmville.

--
Seven puppies were harmed during the making of this post.
Re:Computing power is not the limitation by Anonymous Coward · 2019-01-05 16:41 · Score: 0

No, your computers don't have "much more power", except in some specialized tasks. General AI, even on the level of your family pet is a challenge that your computers cannot even approach in that form factor.
Re:Computing power is not the limitation by Luckyo · 2019-01-05 21:24 · Score: 1

They're actually vastly inferior to this date. Brains remain far more efficient, largely because unlike transistors, there are many potential outcomes of electric signal entering a nerve cell.
Add to this the logic systems developed by billions of years of selectionary pressure, and you get biological computers that are far more advanced than anything we have in silicon today. That's why deep level AI similar to that of human brain proved utterly impossible to this date. Even if you were to manage to match the processing power and logic structures, which will eventually happen, you still need the time for system to learn to be what it can be. Even with generational advancements being more akin to flies in terms of speed than large mammals like humans, that's still going to be hundreds to thousands years even if we're optimistic.

Maybe by theendlessnow · 2019-01-05 14:09 · Score: 1

Maybe it will create AI, since that hasn't happened yet.

Lupe Vélez the Mexican Spitfire by Anonymous Coward · 2019-01-05 14:12 · Score: 0

Just sayin'.

Re: AI progress is not bound by computation speed by Anonymous Coward · 2019-01-05 14:27 · Score: 0

IMO you should be able to multi-thread AI, it's just big datasets, not like each thread needs to relearn everything. Just have lots of high performance RAM. We keep increasing core counts, I don't see that coming to an end soon, less so if new materials can lead to smaller transistors (or even smaller lithography) than what silicon permits.

Now the real work begins... by BobC · 2019-01-05 14:34 · Score: 4, Interesting

Much of recent AI progress has come from the awesome amount of cheap computing power available.

That's not going to change! As today's bleeding edge silicon processes evolve, they will get faster and cheaper (both in cost and energy consumption), if not smaller.

Much of AI is inherently parallel: So long as more CPUs and GPUs can be added, larger problems will be solved faster.

We are still in the first two generations of custom hardware for AI. That trend will continue and accelerate as new architectures and algorithms arrive.

I'd say there are at least three full "Moore's Law" generations coming for AI, very likely more. But transistors alone won't be driving it. Fortunately, there are lots of other factors that will.

Re:Now the real work begins... by raftpeople · 2019-01-05 16:07 · Score: 2

The recent advances have much more to do with Hinton and team's algorithms for training a deep network. That was a major stumbling block that significantly limited the usefulness of neural networks. Even without any increase in computing power, that development created the explosion in success with deep networks.
Re:Now the real work begins... by Anonymous Coward · 2019-01-05 18:19 · Score: 0

Well the AI we have now is not AI, it can recognize patterns when programmed to do so, that is not what most people would consider AI.
It still has no context of anything.

Moore's law is basically irrelevant to strong AI by Anonymous Coward · 2019-01-05 14:37 · Score: 0

Gah - not this again. Speed matters, sure, but it's not hugely important to AI once a certain complexity/architecture is attained.

We don't even understand how consciousness or self awareness works yet!

There's a great quote/story/allegory out there I believe from Marvin Minsky, but I'm not sure/can't find it. It's allegedly regarding a not particularly quick student in one of his CSAIL AI courses and they were talking about this very thing.

It goes something like he was engaging the class about how much faster and faster computers were becoming and one particular student was getting all excited about processing speed improvements, etc. Minsky pounced on this, semi-falsely agreeing with the student regarding how it's great, but ultimately meaningless for strong AI. "You know they modeled a cat's brain rather thoroughly recently, we may even get to a dog's soon!"

"Great!" Minsky allegedly replied. "Imagine in so many years according to Moore's law that this dogs brain will be 1000 times faster than ours even!" was said with grand excitement.

"All that means is that it will take the dog 1000x less time to decide to either lick himself, chase a Frisbee, or take a nap! Wonderful!"

I guess, if you think there's 'AI' (there isn't) by Anonymous Coward · 2019-01-05 14:37 · Score: 0

If there were any such thing as 'AI', that would matter. True intelligence will never be achieved with math, sorry, Moore is irrelevant to this discussion. I don't expect millennials to understand this. Will it halt the progress of algorithms, on the other hand? No, and if you honestly believe processing power is the key, you are an idiot.

Re:Does Moore's law apply to GPUs? ARM? by BadlandZ · 2019-01-05 14:39 · Score: 1

No worries ... https://en.wikipedia.org/wiki/...

Red herring by Anonymous Coward · 2019-01-05 14:42 · Score: 0

Most discussions about AI are being sidetracked about whether we'll get superintelligent general AI or whether we'll be stuck in modestly improving iterations of specific types of brute force AI. Why not discuss the impact of AI as it now stands. Whether it's sentient or not, what would be the effect of AI controlled by a privileged elite, whether human or HAL?

Re: Red herring by Anonymous Coward · 2019-01-05 14:49 · Score: 0

Oh my friend AI and its ilk can compute all the important values required by gasoline stations both in gasoline consumption patterns but also in sales of Twinkies, milk, coffee, tobacco, decide the optimal paint mixture for parking lot lines, sweep floors, cash out customers, develop advertising, stock shelves, throw employee keg parties, wash windows, pose for pictures, and select fireworks for selling on Independence Day. Plus someone has to man the store while its open. Later when multiple AIs Lear to operate in parallel and create new AIs it will be even better. And AIs can help each other with different problems because they have different software. It is a dream come true for the entire gas station. And I want a small bottle of pop from the caf
Re: Red herring by Anonymous Coward · 2019-01-05 15:08 · Score: 0

Plus derp derp derp work derp ads ads derp derp work work ads relax relax sleep derp derp derp shop shop swim swim fly fly derp work derp relax work ...

Re:AI progress is not bound by computation speed @ by Miamicanes · 2019-01-05 15:10 · Score: 1

> Speed of processing is definitely not the issue right now.

Clearly written by somebody who isn't actively involved with things like virtual/augmented/mixed-reality, realtime image-recognition, low-latency high-framerate photorealistic rendering, or realtime ray tracing.

Trust me, there are PLENTY of things left capable of soaking up enormous amounts of computing power.

The "realtime" part, in particular, is a nasty bitch. There are quite a few things that don't necessarily require SUSTAINED high-performance... but when they need performance, they need it INSTANTLY (example: recognizing road hazards & deciding how to handle them while driving a car).

Moore's Law isn't dead, only the "cheaper and cheaper, for less and less power" part that was a common consumer ASSUMPTION, but was never actually included by Moore himself. In the early 2000s, we hit a point when we hit computing power that was (kind of) "good enough", so vendors focused almost entirely on reducing cost and power, even while still increasing transistor counts (but at a lower rate). AI and VR/AR/MR are the next round of applications that are going to put us back into "everything you buy today will be hopelessly obsolete and unusably slow 2-3 years from now" mode.

Going with the observation that things need "bursty" high power, expect the NEXT major round of high-performance computing to come from stepping back from multiple cores back to multiple physical CPUs (or at least, multiple cores that are thermally-separate by a fair amount of space, possibly bathed in some closed-loop non-conductive coolant). Why? A CPU like the i7 can "burst" in single-core mode at speeds significantly higher than they can run with multiple cores, but we've increasingly hit a brick wall insofar as thermal management of ultra-dense CPU cores. So, if an i7 can burst (briefly) in single-core mode to 4GHz, but can only SUSTAIN 2-3GHz, the way to SUSTAIN 4GHz performance is to physically turn it back into an array of multiple CPUs, each of which can run continuously at slightly less than 4GHz, and burst for a few milliseconds at a time up to 4.5-5GHz(*). When you need REALLY high performance, you treat them like an orchestra of virtuoso soloists, each taking turns to step up and bear the full burst-load until they're about to melt before throwing the metaphorical hot potato to the next CPU in line.

---

(*) or incorporate some kind of closed-loop liquid cooling with some non-conductive liquid, so you can take the intense heat from tiny point sources and spread it around to something that can be viably air-cooled without melting itself.

Right and wrong by Dan+East · 2019-01-05 15:14 · Score: 5, Insightful

He's right and wrong. He is correct that much of the "advancements" in AI has been because of processing power (and dataset size). Most of what I learned in AI in college a quarter century ago forms the foundation of today's AI (and most of what I learned had been developed decades earlier). The reason we have things like Siri isn't because AI is smarter. It's because processing power is so fast and cheap, and because data storage and ram is so large and cheap, that an absolutely massive data set can be crunched to do speaker agnostic recognition to determine what I said. In fact, Apple can run my voice audio through dozens of speech models (male, female, accents, etc) in parallel to find the best result. So he is right - processing power has enabled AI to become far more useful of late.

However, where he is wrong is in the parallelism and scalability. In my above example, many different nodes (maybe located in entirely different datacenters) are doing that processing to find the best match.

AI doesn't need to exist on one processor, and it doesn't need to execute at any particular speed. If we're talking "turing" type AI, and I were to ask it "How are you feeling today?" and the AI takes 5 hours to reply "I feel the same as I always do.", well it is still just as intelligent as if it were responding in real-time. When we have reached that point in AI intelligence then we can throw more processing power at it in many different ways to allow it to process faster. The point is that the intelligence is not bound by the processing speed. Sure, for Siri to be viable commercially and useful to Joe Blow it needs to be fast, but as far as research and advancing the field of AI, that is independent of the processing speed.

And having said all that, AI has not advanced significantly beyond the full realization and expansion of things like neural nets with massive processing power and data sets to be useful in identifying, say, a tree in a photograph. We could have been doing that in 1980 given the processing power and storage capacity we have now.

--
Better known as 318230.

Re:Right and wrong by raftpeople · 2019-01-05 16:16 · Score: 0

Not correct. Hinton (and team) created the major breakthrough in 2006 with their algorithm for training deep networks. Prior to that, there were simple single layer networks that could be trained with backprop, and there were deeper networks that outperformed all other methods on image recognition, but they had to be evolved instead of trained, the older backprop algorithms didn't work on deep networks.
Re:Right and wrong by Anonymous Coward · 2019-01-06 02:30 · Score: 0

You are also right and wrong.
If computers are so slow that it takes AI 5 hours to answer, that makes AI research impossible. Humans need faster feedback in order to operate. This is why it is now possible for AI researchers to try to invent something new, and there are really smart people and a lot of money trying to do that, so it is very much possible that even without any improvements in CPU, we will still see new AI inventions.
But I don't believe that CPU improvements have stopped. I'm not even sure if they have slowed down. There are many companies with thousands of employees trying to make the CPU faster and better. It would feel unlikely that there is not even a single individual who wouldn't be able to make it better.
But even if we don't get any improvements in CPU or AI, we will still see some big changes, because current technology already makes it possible to do hundreds of tasks that have not yet been automated. E.g. with current tech we could create robots that travel rainforests and oceans, collect samples of animal and plant DNA and create a big DNA library. This would be a huge step in biology research. Similar stuff could be done with telescopes and monitoring the space. There are also many normal human jobs that could be automated and progress is going pretty well e.g. in farming.
Re:Right and wrong by 0111+1110 · 2019-01-06 14:10 · Score: 1

Yes there may be some big changes, but almost certainly not in your lifetime. Smart people have been trying to figure out how to make intelligent machines for decades and conceptually not much has changed. How old are you btw?
Real AI, as in an artificial human brain equivalent, is probably centuries or even millennia away. After we can make ourselves more intelligent and upload consciousness into an electronic device and plug a coprocessor into a socket in our neck then maybe we will be able to create an artificial brain that is better than the real one. None of that will happen in our lifetimes though. Maybe it will be a thing for your great great great grandchildren.

--
Quite an experience to live in fear, isn't it? That's what it is to be a slave.
Re:Right and wrong by zetetikos · 2019-01-06 20:20 · Score: 1

I came in here to say pretty much what you said. I went to graduate school (masters) in the field of AI in the late 80s and have recent returned to graduate school in the same field, so 30 years later. (Georgia Tech both times) I was shocked. I'm learning the same approaches to the problems. The biggest difference I'm seeing are that there are handy libraries to use so I don't have to code stuff from scratch unless the class requires it. I worked mostly in image understanding the first time. In one of my classes I was asked to implemented a system that was very similar to what I had done previously in Fortran. I dug out my old code and pulled from it when working on it. Both worked well but the old one took about 12 hours to analyze a scene, the new one was under a minute and it was in python. I'm on my sixth class and I have yet to see an algorithm or theory that is significantly different than 30 years ago. (I have all my old class notes and use them periodically) Its been kind of disappointing, since I was expecting to see new things that would advance my understanding of the problems. It is a lot more fun to do this stuff when you can analyze large volumes of data and see the results of your system without waiting for hours. You can actually see that the stuff works. I'm sure as I go on I'll see some more significant incremental changes but I've given up on expecting to see anything that will set my hair on fire.
Re:Right and wrong by tommeke100 · 2019-01-07 02:15 · Score: 1

Different ML techniques will only give small f1-score improvements on the same data-set (given you didn't make big mistakes or use different features). I do think there are some more recent state-of-the-art ensemble techniques like Random Forest and Extreme Gradient Boosting that get good results on smaller data-sets, but those bagging/boosting ensemble-decision-tree stuff techniques were probably already known someway or the other.
I didn't take ML classes back in my days (late 90s), but do remember there being a SVM class for example. Things haven't changed much since.

Re: AI progress is not bound by computation speed by Anonymous Coward · 2019-01-05 15:20 · Score: 0

Whatever man, send it to the gazette and ask them to run it

Re:AI progress is not bound by computation speed @ by Anonymous Coward · 2019-01-05 15:22 · Score: 0

Thanks for demonstrating that you have no idea what AI is really. Enjoy your ray-tracing and pattern-match machine learning as if that's the brass ring, you can win every round of Go you ever play. But you won't be involved in real AI, sorry.

Go play with your rainbow tables, pretend it's Hal.

You also don't seem to understand parallel processing, which negates the false limitation of Moore's law in this instance anyway, except for lightweight unconnected applications.

Derp.

Re: Does Moore's law apply to GPUs? ARM? by Anonymous Coward · 2019-01-05 15:24 · Score: 0

Or just hit that and run away to the jungle. Is good too

Not really, we are already there in hardware.. by thesupraman · 2019-01-05 16:09 · Score: 1

Fast is just a little bit of an understatement, dont you think?

These days we are putting over 30 billion transistors on a chip (and for memory we are layering chips up to 64 times...)

Meat is very VERY slow though, nerve impulses travel at around 450 km/h, so in chip signals move at around 2 million times the speed.
However by the time you factor in neuron firing times (WAY slower) you find cross-conduction speed through the brain is closer to 10 m/s
making the same allowances through a chip, we find current transistors are closer to 30 million times faster than meat.

That is also ignoring the scale - a 30 billion transistor chip is a LOT smaller than a brain.
So, it is pretty safe to say that we could build an electronic brain using todays tech IF WE KNEW HOW.
and it would probably be a lot faster.

HOWEVER, heat an issue.
a 30 billion T chip is how running at such speeds, and as you scale it up, the heat becomes even more of a problem.

Of course none of that matters. We dont know HOW to build one - but it is unlikely that hardware is currently the issue.

BUT, what we are seeing now is not AI, it is machine learning, which is simple brute force statistical optimisation.
There has been very little progress in actual AI. There may be in the future, but ML is not it.

Re:Not really, we are already there in hardware.. by Visarga · 2019-01-05 21:18 · Score: 1

Have you looked at recent graph based neural networks? They can handle multi dimensional and non-uniform structures, such as scenes, texts, proteins and social networks. They are a perfect fit for reasoning tasks. I think they are somewhat in the middle between symbolic and statistical learning. Also, have you seen the progress in reinforcement learning, especially model based RL. It is possible to perform (some) activities at super human levels. I think that graph based NN, RL and simulators for training agents are the next wave. Also, more specialised chips.

In the last year we have gained almost perfect voice recognition and synthesis, image recognition and generation, translation, and mastered all types of board game play. Large parts of the brain are related to processing images and sounds, and those have been solved. I'd say we are right on our way towards AI though ML.
Re: Not really, we are already there in hardware.. by Anonymous Coward · 2019-01-06 01:03 · Score: 0

I would agree with a lot of that except that ML also has a ways to go in being able to recognize solutions in a timely manner. ML tends to suffer from excessive churn on irrelevant data sets.

Scarcity breeds innovation. Of course it won't. by Angelwrath · 2019-01-05 16:48 · Score: 1

Magnetic disk performance resulted in companies investing in Flash and other technologies. The oil crisis resulted in companies investing in engine efficiency. Broken iPhones and expensive iPhones results in more people finding 3rd party repair options. Adversity and scarcity breeds innovation. We'll see a lot of money pour back into the pure science of understanding AI, and also we're seeing companies like Google and AMD invest in chip design that is tailored to AI. "AI-SICs", so to speak. I look forward to the solutions people invent to address the world's problems!

How about by ArchieBunker · 2019-01-05 16:52 · Score: 1

Using AI to develop better processors?

--
Only the State obtains its revenue by coercion. - Murray Rothbard

Not as long as you can make more silicon by nichogenius · 2019-01-05 17:25 · Score: 1

Even if processing power stopped increasing per mm^2 of die space (it's not), the answer is no. AI processing work is highly parallel. That means you can use things like GPUs. Even better is you can use many CPUs and many GPUs. So the processing power is limited by how much hardware and power you can supply. No, AI is just starting.

Y'all already made it a LAW by Anonymous Coward · 2019-01-05 17:52 · Score: 0

Instead of a theorem, you all made Moore's Law a LAW. That's as good as an axiom: accepted as true AND can be proven true. AI is therefore finished, it can never create more advanced AI without specific and deliberate human interventions. Manipulation and modification of the data is not the same as "teaching" the AI. AI will therefore never achieve human levels of "thinking"... it will just execute its maximum capabilities faster as technology progresses. Otherwise, math is partially invalidated, and therefore electrical engineering and computer science where applicable, if Moore's Law ends... as in disproven or defeated.

What's Moore have to do with it? by Excelcia · 2019-01-05 18:08 · Score: 1

Considering that the time when Moore's law has been applicable hasn't appreciably sped up the development of AI, I fail to see how the end of Moore's law will appreciably slow it down. Can someone point me to a chat bot that is better than ELIZA?

Re:What's Moore have to do with it? by Anonymous Coward · 2019-01-07 01:22 · Score: 0

Considering that the time when Moore's law has been applicable hasn't appreciably sped up the development of AI, I fail to see how the end of Moore's law will appreciably slow it down. Can someone point me to a chat bot that is better than ELIZA?
whats moooore got to do.. got to do with it... who needs an AI when its really just machine leaaaarning

Wait for it.....New proof of god! by Maelwryth · 2019-01-05 18:28 · Score: 1

Humans unable to invent themselves ;)

--
I reserve the write to mangle english.

In other words by Anonymous Coward · 2019-01-05 18:44 · Score: 0

"Will thing that we totally made up, halt progress on thing we totally made up?"

The Key Is Architecture Not Miniaturization Now by Anonymous Coward · 2019-01-05 19:38 · Score: 0

Memory driven computing or digital memcomputing machines will deprecate the Von Neumann architecture for AI development.

That's when deep learning coupled with statistical methods, will take AI to another level.

Re:AI progress is not bound by computation speed @ by Miamicanes · 2019-01-05 19:43 · Score: 1

The thing about parallel processing is that not everything CAN be neatly decomposed into stateless parallel processes.

It's kind of like the situation with human workers. If you're excavating a big hole and have an army of slaves, adding workers/slaves to dig, fill buckets, and carry them away will generally increase your net output... until the point when they start getting in each other's way. As the complexity of the task increases, their ability to work efficiently in parallel decreases rapidly.

Computers are ultimately no different. If you're trying to perform millions of stateless computations that don't depend upon the results of other computations or their state, you can efficiently do a lot of work in parallel. The moment they have to start sharing RAM and coordinate their execution to avoid things like race conditions, your ability to compute in parallel falls off dramatically. Sometimes, a million metaphorical army ants will do the job. Other times, you metaphorically need Superman (possibly with backup from one or two other superheroes), and a thousand mere mortals will just get in the way.

Parallel programming is hard, and actually makes Djikstra's assertions about programs requiring rigorous mathematical proof of correctness start to look sane & reasonable. Traditional procedural programs can be validated experimentally. Parallel programs have to be validated primarily based on theory, because it's fundamentally IMPOSSIBLE to experimentally test all of their various runtime scenarios. And that's a really, really, huge problem, because it goes against just about every norm of real-world software development from the past half-century.

It's a problem whose scope makes rigorously validating the code used to launch a Saturn V rocket and get it to the moon look almost trivial by comparison. With a Saturn V, you basically had a single "happy path", and a few well-defined deviations whose goals could all be summarized as, "get it back ON that happy path". With parallel programming, most of time you don't even HAVE a single well-defined "happy path", and when you do, it's nearly impossible to know whether you're on or off of it until it's too late to do anything about it. Humans deal poorly with ambiguity, and computers are even WORSE at dealing with it.

Kurzweil addressed that by ET3D · 2019-01-05 21:10 · Score: 1

Kurzweil gave more thought to this subject than this particular poster probably every will. He said that Moore's Law will slow down for integrated circuits, he just believed that a new technology will replace it. Sure, his estimates might not end up true, but it's still rather pointless to argue with his Singularity with arguments he has already discussed.

Far as AI processing has progressed, that hasn't had that much to do with Moore's Law and more to do with how to effectively use silicon for it. Eric Holloway himself talks about GPUs for AI, and these chips aren't more effective because they use a lot more transistors than CPUs (although they sometimes do), but because they process these particular calculations more effectively. AI chips (and GPUs updated to deal better with DNNs) do it even better. IBM is introducing an analogue chip for AI, another paradigm shift.

Re:AI progress is not bound by computation speed @ by dromgodis · 2019-01-05 21:30 · Score: 1

Clearly written by somebody who isn't actively involved with things like virtual/augmented/mixed-reality, realtime image-recognition, low-latency high-framerate photorealistic rendering, or realtime ray tracing.

Neither of which has anything to do with AI.

Those tasks are indeed worthy and demanding challenges, but without RTFA I understand the question to be something like "Can the research field of AGI advance without a massive steady increase in transistors-per-CPU?". I would guess that the field is in such an infancy that it is not the transistor count that is the limit. Each software system will just take longer to run, but as opposed to real-time rendering, they can wait (like in non-realtime rendering).

Re:Does Moore's law apply to GPUs? ARM? by Z00L00K · 2019-01-05 21:31 · Score: 1

"Moore's law is the observation that the number of transistors in a dense integrated circuit doubles about every two years."

It actually does not state anything else. So it may mean that we may see an end to how dense things can be packed, but the law can still be fulfilled by larger chips and even multiple chips in the same casing to manage massive multi-core processors.

Even though we now see a transit to more pure 64-bit cores I still see that a lot of stuff when doing multi-thread and multi-process activities would be sufficient on 32 and even 16 bit cores. Seems like a waste to run a 64 bit core for a small job, so maybe hybrid solutions would be the thing. A small processor with fewer bits might even be able to have a higher clock frequency because it's easier to speed up things when the physical data bus width is narrower. Of course we already have a hybrid solution utilizing GPUs today.

--
If builders built buildings the way programmers wrote programs, then the first woodpecker would destroy civilization.

We are physical proof it is doable... by Visarga · 2019-01-05 21:33 · Score: 1

The existence of intelligence in mere humans is proof it can be done. We just need to discover better alternatives to silicon.

Re:Does Moore's law apply to GPUs? ARM? by michelcolman · 2019-01-05 22:00 · Score: 2

We don't even need quantum computers to speed up AI. All we need is a different architecture that more closely mimicks a brain.

Right now, even the most advanced practical applications of AI are still using serial computations to calculate the propagation of the signals, Even massively parallel GPUs are still serially calculating everything, just doing it in batches instead of one by one.

Meanwhile, some researchers are experimenting with new layouts where the components actually behave like neurons rather than simulating them in a calculation. If we can use modern chip production processes to make those at a large scale, that will be a game changer.

Human neurons take milliseconds to fire, Let that sink in: speeds below 1 kHz. Imagine mimicking those natural neurons with a denser layout of silicon neurons at gigahertz speeds, and comparing that to a classical computer. We'll go way beyond what Moore's law would permit. It would be like saying we're reaching the limits of how much we can improve bicycles, and then replacing them with jet fighters.

Re:Does Moore's law apply to GPUs? ARM? by Tough+Love · 2019-01-05 23:39 · Score: 1

it may mean that we may see an end to how dense things can be packed

EUV will take us through another factor of 8 or so density increase more or less smoothly without relying on as yet unknown breakthroughs beyond what is required to get past the current 7nm hump.

Then don't discount the possibility of breakthroughs. For example, somebody might figure out a way to mass produce nanotube transistors, maybe good for a further factor of 8.

--
When all you have is a hammer, every problem starts to look like a thumb.

Re: Does Moore's law apply to GPUs? ARM? by Anonymous Coward · 2019-01-06 00:00 · Score: 0

That would be useful to run massive neural network, but you still need something else to train it.

Re: Does Moore's law apply to GPUs? ARM? by Anonymous Coward · 2019-01-06 01:49 · Score: 0

People have been building parallel implementations of neurons or neuron-like systems for over thirty years. They typically are relatively inflexible, so until how the brain works is fully detailed they are only approximate simulations and any particular implementation may be a dead end and represent a large waste of money, which is why efforts are typically on a combination of hardware on software to maintain flexibility. Even then, it doesn't mean that human neurons and their structures are the best way to create human level intelligence.

Re: AI progress is not bound by computation speed by Anonymous Coward · 2019-01-06 02:37 · Score: 0

Ultimately most AI is based on sets of matrices and vectors, dot products, and tranpositions. There are a lot of high speed algorithms, with parallelization, for these and calculations for recall tasks can be parallelised by layer for the majority, and to some extent during training, depending on architecture.

Re: Does Moore's law apply to GPUs? ARM? by michelcolman · 2019-01-06 02:43 · Score: 1

For commercial purposes, training can take place in the factory at a slower pace. The product just has to execute.

Also, it should be possible (at some point in the future) to design a hardware neural net that is capable of training and executing.

Re:AI progress is not bound by computation speed @ by Anonymous Coward · 2019-01-06 02:44 · Score: 0

Imagine a robot taking a step, it needs to calculate how to react to the stone that is under its foot. It can do the calculations in 5 minutes or 10 milliseconds. Which will help the robot stay up?

Next think about an AI researcher. Or even better, imagine yourself playing a video game. When ever you press a button, you will see action happening with 5 hour delay. Could you play that game? How much faster would you learn to play the game if feedback would happen in milliseconds? That is why speed is important also in AI research, humans need fast feedback to learn faster. This faster feedback makes it possible to create better algorithms.

Kurzweil's Singularity by Anonymous Coward · 2019-01-06 02:49 · Score: 0

unlimited self-improving AI is impossible

When you put it like that.. Wait, I can think of a limit: the maximum amount of matter per second that is pushed into a volume unit of the computing system before it forms an event horizon. Now, that's a bus error if anything is.

Re:AI progress is not bound by computation speed @ by careysub · 2019-01-06 02:57 · Score: 2

The thing about parallel processing is that not everything CAN be neatly decomposed into stateless parallel processes.

However we have really, really good evidence that real, strong AI absolutely CAN be decomposed into stateless parallel processes sufficiently well to allow it to perform in real time at the highest level of competence, with hardware that only has a maximum switching rate of only 1000 HZ, and has a mean firing rate of only 6 HZ. You probably have one of these pieces of evidence about two feet from your keyboard.

--
Starships were meant to fly, Hands up and touch the sky - Nicky Minaj

First we have to figure out how to MAKE an AI. by Anonymous Coward · 2019-01-06 03:35 · Score: 0

The behavioural simulators we have now is not even close to actual AI's. AI is a whole nother beast... I hope we never see an progress in that area. Stupid simulation of intelligent behaviour is OK, systems that actually KNOWS what they are doing!?! I'm with the una bomber on that one!

Re: Does Moore's law apply to GPUs? ARM? by Anonymous Coward · 2019-01-06 04:51 · Score: 0

More to the point, the assumption that you must have more transistors to attain faster speeds will most likely become less true as processor technology evolves.

Re: AI progress is not bound by computation speed by Miamicanes · 2019-01-06 05:06 · Score: 1

I'd hardly call debouncing a keyboard "AI" ;-)

Re: AI progress is not bound by computation speed by Miamicanes · 2019-01-06 05:07 · Score: 1

Or interpreting the imaging sensor on a gaming mouse. ;-)

It's not so relevant for multiple reasons. by Slicker · 2019-01-06 05:50 · Score: 1

1. The speed limits of microprocessors are relevant because microprocessors process serial threads of instructions. Parallelizing multiplies effective performance. This is why GPU's are used so much more today, even in addition to multi-core processors.
2. Neuromorphic chips provide many magnitudes better performance than CPU's and GPU's. They do solidify the activation functions possible -- as those cannot be modified or added to once burned into a chip. This is a downside but the tensor-based model for neural simulations has its own limitations on what kinds of processing can be done even in GPU's.
3. Usually where fluid dynamic processing is required in a specific way (as with neural nets), there are tricks specific to the type of processing that can greatly enhance performance even when serially processed. This was the case particularly in astrophysics, such as with galaxy collision simulations and simulations of the early expansion of the universe. Using GPU's is really a lazy way out, in cognitive terms.

In fact I have a method that I call "Maxerial" (for maximum serial) processing method that show excellent performance even on a Raspberry pi with no GPU at all.

And furthermore, I suspect one day we may have analog computers and/or quantum computers that provide extraordinary performance and capacity.

The guy's argument is a tautology by Anonymous Coward · 2019-01-06 06:54 · Score: 0

He essentially says "If we assume that AI progress depends on increasing processor speed then when moore's law comes to an end so will AI"

That is the state of journalism today.

What progress? by Casandro · 2019-01-06 07:38 · Score: 1

So far we've only been applying insane amounts of CPU power and data to machine learning algorithms which haven't changed a lot in the last decades.
We are now seeing such ML applications getting slowly as good as conventional ones, but at a much higher computational effort. Essentially machine learning boils down to statistics. However unlike normal statistics, ML does not provide you with insights. This may be perfectly OK for finding out what fruit is in front of a camera, however whenever you have accountability involved you need to know what's going on.

Just imagine the fraud detection system of a bank denying a transaction, this causes a company to fold and the owners sue the bank. If you don't know what your system is doing, you have no way to estimate risks.

So "AI" or machine learning as it's actually called, is currently just mostly hype. Yes there are some areas where it can find its uses, but much of what's done now is just hype. The current hype will fade, just like the 1980s "AI"-hype did.

Mistake in headline by Anonymous Coward · 2019-01-06 09:49 · Score: 0

It should read:

"Has the End of Moore's Law Halted AI Progress?"

Wrong Metric by Anonymous Coward · 2019-01-06 10:42 · Score: 0

If one has actual experience in the field, especially designing AI infrastructure, it's clear that moores law has little to do with AI performance. 10% of the work is training and predicting your models. 80% of the effort is data manipulation, cleaning, EDA, and so on. Another 10% is defining the business problem correctly. A useful metric would include compute, network, storage, and labor costs. Assuming that one can just throw a neural network at every problem rather than understanding the problem is fiction.

Slashwhat? by Anonymous Coward · 2019-01-06 20:32 · Score: 0

Even as a layperson this is insulting.

The fundamental lack of understanding of the technology and concepts they are disussing is stupefying.

This has no place here...

Slashdot reader #4,816 by Anonymous Coward · 2019-01-08 01:05 · Score: 0

You lost me at "Slashdot reader #4,816". Who cares how long you've been a registered user? How long was it before Slashdot figured out how to authenticate your user credentials over TLS?? Whooptie fucking do. I'm not even interested in reading the rest of the article summary, nor TFA itself, after seeing that ridiculous line.

Slashdot Mirror

Will the End of Moore's Law Halt AI Progress? (mindmatters.ai)

170 comments