Google Teaches Computers "Regret"

mikejuk writes "Google is funding an AI project that will introduce the technical concept of regret into programs — but there's a big difference between regret and being sorry. In fact regret is just the difference between maximum possible reward and the actual reward received and the project is about optimization. There are two things to learn from this situation. The first is that just because some numerical measure is called 'regret' it doesn't mean it has anything to do with the common use of the term. Secondly if you are going to invent an AI technique then picking emotive words for your jargon is a good way to ensure publicity."

Comments

Past transgressions

[supertrinko]

Will they now regret that they've allowed us to live so long?

Regret is a standard term in economics

[doshell]

Regret is a standard term in economics (esp. game theory). If anyone is to take the blame over a poor choice of noun, it certainly isn't Google.

Re:Regret is a standard term in economics

[retchdog]

here i am feeding the troll, but the best AI for backgammon is trained by regret-based reinforcement learning (it's needed since the dice rolls blow up the search space for standard perfect-information strategies): http://www.research.ibm.com/massive/tdl.html. in this case the regret-function is unknown and is stochastically approximated ("learned") by repeated play.

it's notable that unlike chess AI which is considered effective but unnatural, this backgammon AI is considered to play mostly "like a human" and its play has actually inspired new strategies for human backgammon players.

regret-based methods are typically heuristic, and i'd call them much less "autistic" than, say, infinitely-rational nash agents or game tree pruners.

Re:Regret is a standard term in economics

[retchdog]

what's your point? any rational model is going to be based on some restrictive assumptions. would you rather that economists just do what feels right to them, or that economics stops existing, or what? would this somehow be less negligent? or let me guess, if only economists agreed with your personal view of the world, everything would be better, right?

if you don't like artificial rules, you should appreciate that regret-based learning is a more flexible setting than standard game theory; for example, it allows a natural formulation of imperfect information and adaptation.

Re:Regret is a standard term in economics

[xouumalperxe]

Right, economics lives in a world of games with artificial rules.

Because nobody else does that.

Re:Regret is a standard term in economics

[Walt Dismal]

Regret can be cast as the reaction to making a decision that produced consequences of less value than another decision that could have been made, or which produce punishment rather than positive outcome. As such, it's fairly easy to implement. In AI systems, I use regret and other failure emotions such as fear, anger, or sadness as metrics of success or failure of goals. They provide feedback into learning and planning functionality, controlling strategies. And yes, one can make an AI that 'feels' sadness upon loss, analogous to how humans experience a reaction to loss.

Re:Regret is a standard term in economics

[Daniel Dvorkin]

Furthermore, the emotion we identify as "regret" seems to me to line up neatly with the economic definition. I defy anyone to prove that when we feel regret, our brains are doing anything other than comparing the reward we received for a particular action with the maximum reward we think we could have received for a different action. TFA is more or less playing the Chinese Room game: "I assert that computers can't do X, because computers can't do X, because I assert that computers can't do X."

Re:Regret is a standard term in economics

[retchdog]

yup. all of this is MUCH less "autistic" than economics has ever been before.

unfortunately i've not had the opportunity to implement a regret-driven system although they seem interesting for longitudinal data. the major problem in the TD-gammon program seems to have been tweaking the attribution function, i.e. learning exactly what to assign as the cause of my regret. fun stuff.

i wouldn't go as far as to claim that an ai feels anything yet. i wonder if feeling or cognition will emerge first in an "artificial" system (or rather which one we will recognize first...).

i-programmer.info again?

[retchdog]

i guess this is the new roland piquepaille: superficial and uninformed blurb-commentary on technical news. wonderful.

Different words with different meanings

[John Hasler]

> ...there's a big difference between regret and being sorry.

Yes. "Regret" is not a synonym for "remorse".

ahh i see

[ajdub]

fprintf("Regret: %f\n",test_error);

Re:ahh i see

[pedantic bore]

fprintf("Regret: %f\n",test_error);

fprintf(stdout, "Regret: %d\n", test_error);

FTFY