Slashdot Mirror
Sir Tim Berners-Lee Lays Out Nightmare Scenario Where AI Runs the Financial World (techworld.com)
The architect of the world wide web Sir Tim Berners-Lee has talked about some of his concerns for the internet over the coming years, including a nightmarish scenario where artificial intelligence (AI) could become the new 'masters of the universe' by creating and running their own companies. From an article: Masters of the universe is a reference to Tom Wolfe's 1987 novel The Bonfire of the Vanities, regarding the men (and they were men) who started racking up multi-million dollar salaries and a great deal of influence from their finance roles on Wall Street and in London during the computerised trading boom pre-Black Monday. Berners-Lee said, "So when AI starts to make decisions such as who gets a mortgage, that's a big one. Or which companies to acquire and when AI starts creating its own companies, creating holding companies, generating new versions of itself to run these companies. So you have survival of the fittest going on between these AI companies until you reach the point where you wonder if it becomes possible to understand how to ensure they are being fair, and how do you describe to a computer what that means anyway?"
Sir Tim Berners-Lays, father of the chip.
If you want news from today, you have to come back tomorrow.
... Nightmare Scenario Where AI Runs the Financial World
As opposed to the natural stupidity that currently runs it? How could the AI be worse?
The world will probably turn its back on money and move to other forms of pseudo currency once the "economy" runs itself.
So, the financial industry will be controlled by heartless automatons, but now they will make intelligent, logical decisions? Seems like a net gain.
Also, MOTU is a reference to He-Man.
This is my signature. There are many like it, but this one is mine.
More AI bullshit. We can barely even create functional regular software.
Nightmare scenario for traders, benefit for everyone else. The AI likely would have Chinese walls built into them to prevent collusion and insider trading. I for one look forward to my innately honest (and auditable/examinable) AI masters
Hoist Number One and Number Six.
... reach the point where you wonder if it becomes possible to understand how to ensure they are being fair, and how do you describe to a computer what that means anyway
If it's really AI, shouldn't it be able to learn concepts like fairness, even if it chooses not to apply it?
Bwhahaha ahahahahahah ahaha
hahahahaha
Thanks. Good laugh!
I mean when it comes to money, it doesn't have any real meaning to a computer A.I. It does, however, mean pretty much everything to human beings directly involved in the market, in trading, and in the business of trying to generate maximum wealth.
That ensures that A.I. will never be allowed to spin out of control to create the "nightmare scenarios" one can create in their imagination.
It will only be implemented as far as it is able to assist people in performing the tasks they wish to perform manually anyway.
Well if AI runs the investment universe and does a perfect job, isn't that central planning perfected? I mean that was always the problem wasn't it? Investment was risky and uncertain so investors had to compete and a single government entity couldn't do it. Now if AI does it then lovely, capital is allocated properly to the right companies who produce the best results with it. Great!
Since humans can't agree on what "fair" means, how are we supposed to describe it to an AI?
This posting is provided 'AS IS' without warranty of any kind, implied or otherwise.
Loss of control can be a good thing, depending on who is currently in control.
The Daddy casts sleep on the Baby. The Baby resists!
Traders are finding it too difficult to make money with High Frequency Trading (HFT) since everyone and there mother is doing it.
http://www.investopedia.com/news/high-frequency-trading-flash-boys-losing-steam/
MIT Technology Review has a nice article on the same subject https://www.technologyreview.com/s/604087/the-dark-secret-at-the-heart-of-ai/
The most lucrative job you can get if you know control theory is as a programmer for a stock trading company, and has been for a while.
By any means possible. Automated self serving, self learning digital cash gobblers.
smash the robots and let's see how the money men like paying 20K-40K + court and med costs a year to lock someone up
Cephalopods want to help us now so we can help them in the futute
I welcome our Cephalapod overlords and would like to point out that as a software engineer I would be very capable of taking care and feed AI entities
The dangers of excessive individualism are nothing compared to the oppressiveness of excessive collectivism
Can anyone think of a worse scenario than having our financial systems outside of democratic control?
I came to post exactly the same thing. They ask how the AI will be fair. Better to ask, it what way could an AI NOT be fair? Harder to add unfairness into a system than it is to have the system be neutral.
What I'd be almost more excited about is an AI HR system so the HR system could not ignore you based on personal biases and/or relationships.
"There is more worth loving than we have strength to love." - Brian Jay Stanley
Mr Bundy was useless, but I can't really see him doing a worse job than the market right now. Despite his pitiful salary as a shoe salesman, he managed to keep his family housed, fed, and mostly clothed. That takes a degree of financial smarts that many don't have.
You think the billionaire Warren Buffet is gonna let some AI run ANY of his fortune 500 companies? If a CEO takes a bold new strategy that Warren thinks is questionable, the CEO can explain why he is making this decision. We know 'deep learning' algorithms biggest problem is how its a black box. Programmers can't easily figure out why it made the decision it did. If the CEO's bold new strategy fails, he can be fired and a new one brought on. What if the AI fails? Do i fire the programmers? Purchase a new CEO AI from a competitor?
Yeah i don't think so. The only reason high frequency trading machines exist now is because investors understand these machines one advantage over humans is speed. They also don't allow HFTM to have control of billions of dollars for investment, usually limiting its pool of funds it works with in order to limit risk. The HFTM owners have also 'gamed' the system to a certain extent. If a bug causes an algorithm to go haywire and negatively affect the market, the transactions can be 'rolled back' so the investor doesn't loose all their cash.
No one in the financial world right now is ready to put AI in control of billions of capital, thousands of employees, and dozens of production chains in the sole hands of an unaccountable AI with no personal vested interest in the business. More likely? In a few years we get a personal assistant like Alexa that can provide suggestions to CEO's on what new business decisions would be the best course of action. The AI will augment, not replace.
It is never about the AI, but the bias programmed/learned by the AI. Guess what, in today's "non-AI" world the same thing happens, bunch of rich people (some of whom works in the government) decides who gets the most $ (newsflash: themselves). The Treasury create a whole bunch of $ and pass along to the banks and the bankers get a huge cut, that's why bankers are always rich. The more corrupt a country, the richer the bankers.
End the corporations, AI-mediated or otherwise.
This is why 'AI' is not AI and it would be fucking stupid to relegate anything nuanced to it. Greedy companies do not care about you in this regard. Use your voice and move your feet to fight it, they do not have the right to do this to you.
So he's basically regurgitating the plot to Fear Index?
While we're at it, perhaps he could also be afraid that some rogue genetics company is going to clone dinosaurs and open an amusement park and they'll escape somehow and run amok?
The scenario that TBL is talking about here was preempted by Charles Stross in his 2006 novel: Accelerando.
Afraid to compete against some AIs, pathetic. It is still competition, man vs machine, and a real capitalist does not shirk from it but says "bring it on robot".
..and how do you describe to a computer what that means anyway?
You don't. Nothing we have is actually sentient, sapient, conscious, or has a conscience, for that matter. Some decisions regarding human beings can be made with pure logic alone, but many many more require much much more from the mind making the decision than just pure logic. Haven't there been more than enough cautionary tales from science fiction to illustrate this point? Or do we really have to, once again, learn a big important lesson the hard way? The more time that passes, the more I read, the more I think about the issue and the problems it presents us with, the more convinced I get: These machines that we (incorrectly!) call 'AIs', regardless of whether they're working with financial matters, autopilot for an aircraft, operating a ground vehicle, or working in an operating room, they must be supervised by a human being. Always. If and when the day comes that we have actual conscious, sentient, sapient, truly thinking human-level artificial minds, something that you can interrelate with on a human level, something that understands us, what it's doing, the possible consequences, has a conscience, etc, then so far as I'm concerned we can re-visit whether they can be actually 'autonomous' or not. Until then, forget it. You want to know what (allegedly!) displaced workers will be doing in the future? Supervising the machines!
news at eleven. If you want fairness guarantee people access to food, shelter, healthcare, education & transportation. Then the rest will sort itself out when folks can no longer leverage their monopoly on those things into the forging of slaves.
Hi! I make Firefox Plug-ins. Check 'em out @ https://addons.mozilla.org/en-US/firefox/addon/youtube-mp3-podcaster/
You can't forcibly reverse engineer the brain of an average finance industry sociopath, and prove that they have bad intent - they can always claim to be honest and good-willed in their actions - but if the finance industry trains AI to overtake the jobs of all the society-destroying sociopaths, we can forcibly reverse engineer and audit the brains of AI that is used to run the world - and then we can actually prove dishonest and bad intent, on the part of both AI and their trainers.
Bizarrely, this might open up a brave new world which enablers a greater level of transparency in how the world is run, than there has ever been in all of history - because up until now, you could not read the minds of the rulers of the world, and know that they are truthful and acting in good intent - with publicly reverse-engineered/audited AI, we can finally know.
The biggest pushers of AI, in the form of those who want to replace workers for their own riches/benefit - may actually be sowing the seeds of their own destruction, by allowing the creation of a completely transparent and auditable economic (and maybe political) system, which can literally be programmed to enforce adherence to ethical rules - paving the way for the elimination of the lions share of economic imbalances, that tip the balance in favour of the rich/wealthy.
Capitalism is the modern version of a club. Whose got the biggest one wins, unless enough band together to .. oooh shiny
Those who set up the model for use will add in their preferred bias so there is no reason to be excited. Those who don't like the results will not accept them.
I'm reminded of some earlier post here about how all those HR questions about attitudes to family are just a back door method of screening for homosexuals. Don't expect HR people to be ethical (or even properly serve their employers) just because they got a new tool.
What qualifications does Weird Al have to do this?
... you reach the point where you wonder if it becomes possible to understand how to ensure they are being fair ...
We seem utterly unable to solve this problem even with the current (allegedly) human banksters - I'm not so sure it will be significantly different with AI running the markets. Also, this is but one tiny portion of the grief that might lie ahead of us if AI becomes ubiquitous, omniscient, and omnipotent. We don't need to worry about AI in the financial sector - we need worry about AI, period.
'The Economy' is a giant Ponzi scheme whose most pitiable suckers are the youngest among us and the yet-unborn.
Starting with the supreme court.
There's a perfect solution to computer trading: limit their trading frequency to something humans can grok.
e.g. Require all trades to be made as public offers for 10 minutes (not milliseconds), and add an extra penny tax any stock purchase that's refunded if you own the stock for more than 24 hours. Tweak the times and amounts as necessary.
bots run them. they make the "decision".
what the guy who inputs the inputs into the application does though is know after a while what gets a pass and what doesn't, so thell their clients to tell them that say that you have this and this much and can get this and this.
it's not really ai of course. but berners lee seems to be out of touch of how regular people get mortages and such in todays world. sure if you're a sir you get to meet the bank manager and he can make a decision to give you mortage on your 2 million castle BUT for regular people algorithms(and law) have made the decisions already for something like 2 decades. how shitty loans you want to give out you adjust the algo.
world was created 5 seconds before this post as it is.
This topic is about "Finance", "AI" (actions) and "fairness" (good life) -the three arguments can be dated back to Aristotle's principles of economics.
For Aristotle, the primary meaning of economics is the action of using things required for the Good Life.
He also defines the roles members of the household should have, placing "family" structures as a vital part of economy; Fix family and you'll fix economics.
The main critics argue that he uses slaves, but replace slaves with modern-day "AI" instead, and his 'household economics' is again relevant...
https://en.wikipedia.org/wiki/Economics_%28Aristotle%29
It was called a Person Of Interest.
AIs understand fair better than most humans. For a machine 1+1=2 is fair. There is no other measure that make the first "1" more significant than the second "1". Both are equal and make 2. Trying to teach an AI anything else would be to break it's fundamental understanding of the universe.
Assuming people actually did just what you suggested and made access to "food, shelter, healthcare, education & transportation" available to everyone. What do you do with the people who do not take advantage of it? Do you have to take all of it, or can you just take the food and transportation? Are you required to get the education to get the rest? Is forcing people to use it just another form of slavery? You can lead a horse to water, but you can't make it drink.
I only look human.
My mother is a halfling and my dad is an ogre, so that makes me an Ogreling
You're a little late there Tim!
How is HFT really any different than AI bots? How is automated arbitrage trading really any different? Oh I know, you are, as a geek, focusing on the AI as something "different" and "scary", but really, come on!
Already we have quarter to quarter tunnel vision in the executive suite. Already we have highly profitable companies laying off staff to become more profitable. Already we have the concept of "fiduciary responsibility" and "it's just business" used to justify any action, no matter how morally heinous. Already we have MBAs, Other People's Money, Masters Of The Universe, the Wolf Of Wall Street, plain old Wall Street, The Whale of London, Barings Bank, the subprime mortgage stinker, ...
The list is endless. How does AI quantitatively or qualitatively change anything? It changes nothing. Greed and fear run the corporate sector. Those are negatives, for those who need that pointed out.
https://en.wikipedia.org/wiki/...
It runs the world by printing out business letters (and checks) that hire people to expand itself.
https://www.worldswithoutend.c...
"Chester W. Chester IV, sole surviving heir of eccentric millionaire-inventor Chester W. Chester I, has entered into his inheritance: a semi-moribund circus; a white elephant of a run-down neo-Victorian mansion furnished with such hot items as TV sets shaped like crouching vultures; the old gentleman's final invention, a mammoth computer whose sole value seems to be as scrap metal; and one more thing--a million credits in back taxes. Either he comes up with the million credits, or it's up-the-river for Chester for a long, long time. That's why Chester is desperate enough to use the Generalized Nonlinear Extrapolator (Genie for short) to perpetrate one of the biggest entertainment scams of all time--The Great Time Machine Hoax."
I enjoyed that novel a lot and read it multiple times -- especially for the aspects of learning and training to become a more capable person (if maybe not a wiser and more compassionate one depending what you study).
A 21st century issue: the irony of technologies of abundance in the hands of those still thinking in terms of scarcity.
Part of something I posted in 2000 to Doug Engelbart's "Unifinshed Revolution II" colloquium touching on corporations as "AIs":
http://www.dougengelbart.org/c...
========= machine intelligence is already here =========
I personally think machine evolution is unstoppable, and the best hope for humanity is the noble cowardice of creating refugia and trying, like the duckweed, to create human (and other) life faster than other forces can destroy it.
Note, I'm not saying machine evolution won't have a human component -- in that sense, a corporation or any bureaucracy is already a separate machine intelligence, just not a very smart or resilient one. This sense of the corporation comes out of Langdon Winner's book "Autonomous Technology: Technics out of control as a theme in political thought".
http://www.rpi.edu/~winner/
You may have a tough time believing this, but Winner makes a convincing case. He suggests that all successful organizations "reverse-adapt" their goals and their environment to ensure their continued survival. These corporate machine intelligences are already driving for better machine intelligences -- faster, more efficient, cheaper, and more resilient. People forget that corporate charters used to be routinely revoked for behavior outside the immediate public good, and that corporations were not considered persons until around 1886 (that decision perhaps being the first major example of a machine using the political/social process of its own ends).
http://www.adbusters.org/magaz...
Corporate charters are granted supposedly because society believe it is in the best interest of *society* for corporations to exist. But, when was the last time people were able to pull the "charter" plug on a corporation not acting in the public interest? It's hard, and it will get harder when corporations don't need people to run themselves.
http://www.adbusters.org/magaz...
http://www.adbusters.org/campa...
I'm not saying the people in corporations are evil -- just that they often have very limited choices of actions. If a corporate CEOs do not deliver short term profits they are removed, no matter what they were trying to do. Obviously there are exceptions for a while -- William C. Norris of Control Data was one of them, but in general, the exception proves the rule. Fortunately though, even in the worst machines (like in WWII Germany) there were individuals who did what they could to make them more humane ("Schindler's List" being an example).
Look at how much William C. Norris http://www.neii.com/wnorris.ht... of Control Data got ridiculed in the 1970s for suggesting the then radical notion that "business exists to meet society's unmet needs". Yet his pioneering efforts in education, employee assistance plans, on-site daycare, urban renewal, and socially-responsible investing are in part what made Minneapolis/St.Paul the great area it is today. Such efforts are now being duplicated to an extent by other companies. Even the company that squashed CDC in the mid 1980s (IBM) has adopted some of those policies and directions. So corporations can adapt when they feel the need.
Obviously, corporations are not all powerful. The world still has some individuals who have wealth to equal major corporations. There are several governments that are as powerful or more so than major corporations. Individuals in corporations can make persuasive pitches about their future directions, and individuals with controlling shares may be able to influence what a corporation does (as far as the market allows). In the long run, many corporations are trying to coexist w
A 21st century issue: the irony of technologies of abundance in the hands of those still thinking in terms of scarcity.