Will the End of Moore's Law Halt AI Progress? (mindmatters.ai)

johnnyb (Slashdot reader #4,816) writes: Kurzweil's conception of "The Singularity" has been at the forefront of the media conception of artificial intelligence for many years now. But how close is that to reality? Will AI's be able to design ever-more-powerful AIs? Eric Holloway suggests that the power of AI has been fueled by Moore's law more than AI technology itself, and therefore hitting Moore's Wall will bring AI expansion to a fast halt.
Holloway calls that halt "peak AI...the point where a return on the investment in AI improvement is not worthwhile." He argues that humanity will reach that point, "perhaps soon...."

"So, returning to our original question, whether there is a path to Kurzweil's Singularity, we must conclude from our analysis that no such path exists and that unlimited self-improving AI is impossible."

Comments

Transistors and AI

[reanjr]

If you think AI is just more transistors, you probably aren't doing anything interesting in AI research. How many transistors in the human brain? How many regular transistors are necessary to do the work of one quantum transistor? We don't even know how the brain works, and this asshat is asserting that we'll never be able to build a machine that works the same way.

Re:Transistors and AI

[phantomfive]

How many transistors in the human brain?

Of course the answer is zero, because the brain has neurons. But we can have some numbers for comparison. A Graphcore GC2 IPU has 23 billion transistors. In comparison, a brain has:

100 billion neurons.
10 trillion synapses.
300 billion dendrites.

Which of those need to be emulated? A transistor does not do as much as a neuron, and we don't know all the things a neuron does. There is some evidence that the inside of a neuron does some kinds of calculations. So it's much more complicated than just comparing raw numbers. That said, transistors do operate faster than neurons.

Good link for more reading.

Kurzweil is a Shill

[Dayze!Confused]

Kurzweil pretends to know what he's talking about because he can fit a graph with lots of tampering with the data. He fails to see that what he calls exponential growth is nothing more than the beginning of a sigmoid function. A good analysis of Moore's law and computational power shows a sigmoid function, as with many technologies they start off slow, build up quickly, then tapper off.

Oh my Lord?

[drolli]

Eric Holloway:

* seems to have no qualification in physics/nanotechnology to add anything to the discussion if Moores law will end, and when

* Seem to bagger along with intelligent design folks, with him re-telling the old stories they usually tell about information science and the rest of science

* and seems to write no peer reviewed articles any more (after the paper he wrote unrelated and before his PHD research)

* Did a PHD in a program where the students are identified as "good stewards of God-given talents" (https://www.ecs.baylor.edu/ece/index.php?id=865400)

* Did a PHD program which contains in its description "Engineering is also a value-based discipline that benefits from Christian worldview and faith perspectives; students can also select supportive courses from religion, theology or philosophy. Course selection is broadly specified to provide flexibility and to accommodate a wide-range of student interest." (https://www.ecs.baylor.edu/ece/index.php?id=863609)

* Description of the seminar series of his university where it seems that he presented his PHD: (https://www.ecs.baylor.edu/ece/index.php?id=868860): eBEARS seminars are presented by Baylor ECE faculty, ECE graduate students and transnationally recognized scholars and leaders. The topics lie within the broad area of ECE. In concert with Baylor's Pro Futurus strategic plan to be "a place where the Lordship of Jesus Christ is embraced, studied, and celebrated," some eBEARS seminars focus on the topic of faith and learning.

So praise the Lord for his insights!

Now the real work begins...

[BobC]

Much of recent AI progress has come from the awesome amount of cheap computing power available.

That's not going to change! As today's bleeding edge silicon processes evolve, they will get faster and cheaper (both in cost and energy consumption), if not smaller.

Much of AI is inherently parallel: So long as more CPUs and GPUs can be added, larger problems will be solved faster.

We are still in the first two generations of custom hardware for AI. That trend will continue and accelerate as new architectures and algorithms arrive.

I'd say there are at least three full "Moore's Law" generations coming for AI, very likely more. But transistors alone won't be driving it. Fortunately, there are lots of other factors that will.

Right and wrong

[Dan East]

He's right and wrong. He is correct that much of the "advancements" in AI has been because of processing power (and dataset size). Most of what I learned in AI in college a quarter century ago forms the foundation of today's AI (and most of what I learned had been developed decades earlier). The reason we have things like Siri isn't because AI is smarter. It's because processing power is so fast and cheap, and because data storage and ram is so large and cheap, that an absolutely massive data set can be crunched to do speaker agnostic recognition to determine what I said. In fact, Apple can run my voice audio through dozens of speech models (male, female, accents, etc) in parallel to find the best result. So he is right - processing power has enabled AI to become far more useful of late.

However, where he is wrong is in the parallelism and scalability. In my above example, many different nodes (maybe located in entirely different datacenters) are doing that processing to find the best match.

AI doesn't need to exist on one processor, and it doesn't need to execute at any particular speed. If we're talking "turing" type AI, and I were to ask it "How are you feeling today?" and the AI takes 5 hours to reply "I feel the same as I always do.", well it is still just as intelligent as if it were responding in real-time. When we have reached that point in AI intelligence then we can throw more processing power at it in many different ways to allow it to process faster. The point is that the intelligence is not bound by the processing speed. Sure, for Siri to be viable commercially and useful to Joe Blow it needs to be fast, but as far as research and advancing the field of AI, that is independent of the processing speed.

And having said all that, AI has not advanced significantly beyond the full realization and expansion of things like neural nets with massive processing power and data sets to be useful in identifying, say, a tree in a photograph. We could have been doing that in 1980 given the processing power and storage capacity we have now.