How Do We Program Moral Machines?

nicholast writes "If your driverless car is about to crash into a bus, should it veer off a bridge? NYU Prof. Gary Marcus has a good essay about the need to program ethics and morality into our future machines. Quoting: 'Within two or three decades the difference between automated driving and human driving will be so great you may not be legally allowed to drive your own car, and even if you are allowed, it would immoral of you to drive, because the risk of you hurting yourself or another person will be far greater than if you allowed a machine to do the work. That moment will be significant not just because it will signal the end of one more human niche, but because it will signal the beginning of another: the era in which it will no longer be optional for machines to have ethical systems.'"

Comments

Why I doubt driverless cars will ever happen

[crazyjj]

I maintain that you CAN'T really program morality into a machine (it's hard enough to program it into a human). And I also doubt that engineers will ever really be able to overcome the numerous technical issues involved with driverless cars. But above these two problems, far and away above *all* problems with driverless cars is the real reason I think we'll never see anything more than driver *assisting* cars on the road: legal liability.

To put it bluntly, raise your hand if YOU want to be the first car manufacturer to make a car for which you are potentially liable in *every single accident that car ever gets into*, from the day it's sold until the day it's scrapped. Any takers? How much would you have to add onto the sticker price to cover the costs of going to court every single time that particular car was involved in an accident? Of defending the efficacy of your driverless system against other manufacturer's systems (and against defect, and against the word of the driver himself that he was using the system properly) in one liability case after another?

According to Forbes, the average driver is involved in an accident every 18 years. Let's suppose (and I'm sure the statisticians would object to this supposition) that that means that the average CAR is also involved in a wreck every 18 years as well. Since the average age of a car is about 11 years now, it's not unreasonable to assume that a little less than half of all cars on the road will be involved in at least one accident in their functional lifetimes. And even with the added safety of driverless systems, the first model available will still have to contend with a road mostly filled with regular, non-driverless-system cars. So let's say that a good 25% of those first models will probably end up in an accident at some point, which will make a very tempting target for lawyers going for the deep pockets of their manufacturers.

Again, what car company wouldn't take that into account when asking themselves if they want to be a pioneer in this field?

Re:Why I doubt driverless cars will ever happen

[DavidClarkeHR]

To put it bluntly, raise your hand if YOU want to be the first car manufacturer to make a car for which you are potentially liable in *every single accident that car ever gets into*, from the day it's sold until the day it's scrapped. Any takers?

... no one. But you'll get plenty who charge mandatory tune-ups to ensure compliance. The question will be "which company DOESN'T charge a fee for a mandatory yearly check-up"?

Re:Why I doubt driverless cars will ever happen

[Trepidity]

What they're talking about here, though, isn't really programming morality into machines in some kind of sentient, Isaac-Asimov sense, but just programming decision policies into machines, which have ethical implications. The ethical questions come at the programming stage, when deciding what policies the automatic car should follow in various situations.

Re:Why I doubt driverless cars will ever happen

[crazyjj]

And those ethical decisions will come with even MORE legal liabilities. Even the idea would give any legal department nightmares. They get enough headaches from faulty accelerators. Can you imagine the legal problems they would get from programming hard ethical decisions into their computers? They would get sued out of existence the first time that feature had to be used.

Re:Why I doubt driverless cars will ever happen

[CastrTroy]

This is my exact reasoning why flying cars will never take off (pardon the pun). People keep their cars in terrible condition. If your car has an engine failure, worst case scenario, you pull over to the side of the road, or end up blocking traffic. In a flying vehicle, if your engine dies, It's very possible that you will die too. And if you are above a city, it's not impossible to imagine crashing into an innocent bystander.

I imagine the same will be for self driving cars. It will never happen because if the car is getting bad information from its sensors, then crazy things can happen. People can't be bothered to clean more than 2 square inches from their windshield in the winter. Do you really think they are going to go around cleaning the 10 different sensors of ice and snow every winter morning? Sure the car could refuse to operate if the sensors are blocked, but then I guess people would just not want to buy the car, or complain to the dealer about it.

Re:Why I doubt driverless cars will ever happen

[SirGarlon]

I think your statistics on accidents are informative but you're missing an important point. With automated cars, we expect accident rates to go down significantly (so saith the summary). So the likelihood an _automated_ car will be _at fault_ in an accident is probably a lot lower than the 25% you presume. (The manufacturer does not care about accidents where the machine is not at fault, beyond complying with crash-safety requirements.)

Re:Why I doubt driverless cars will ever happen

[The MAZZTer]

There's sort of a flaw in your reasoning... the accident rate you cite is with HUMAN drivers. Driverless cars would naturally change it (ideally, lower it). And assuming this, chances are accidents involving driverless cars would mostly occur with human-driven cars and be the human's fault, so no liability there.

However I suspect at least initially software/hardware to enable driverless control of cars would be provided by companies other than the manufacturer so they would not be held liable. They would probably have the results of their own tests of the system to show to the court to say they did their own QA on it and found it to be as safe as they could expect. It would probably be closed source, so what more could they do? The software devs themselves may not be claimed either, depends on the nature of the crash, why it happened, the bug in the code that triggered it, whether or not that bug would have been reasonable to locate and fix via a reasonable QA process, if the bug was already fixed in newer firmware why the vehicle wasn't patched (could be found to be the owner's fault), etc etc.

Re:Why I doubt driverless cars will ever happen

[Rich0]

Well, the solution to liability is legal - grant immunity as long as the car performs above some safety standard on the whole, and that standard can be raised as the industry progresses. There is no reason that somebody should be punished for making a car 10X safer than any car on the road today.

As far as programming morality - I think that will be the easy part. The real issue is defining it in the first place. Once you define it, getting a computer to behave morally is likely to be far EASIER than getting a human to do so, since a computer need not have self-interest in the decision making. You'd be hard pressed to find people who would swerve off a bridge to avoid a crowd of pedestrians, but a computer would make that decision without breaking a sweat if that were how it were designed. Computers commit suicide every day - how many smart bombs does the US drop in a year?

But I agree, the current legal structure will be a real impediment. It will take leadership from the legislature to fix that.

Re:Why I doubt driverless cars will ever happen

[crazyjj]

So the likelihood an _automated_ car will be _at fault_ in an accident is probably a lot lower than the 25% you presume.

Great. Now all you have to do is prove your system wasn't at fault in a court of law--against the sweet old lady who's suing, with the driver testifying that it was your system and not him that caused the accident, and a jury that hates big corporations. And you have to do it over and over again, in a constant barrage of lawsuits--almost one for every accident one of your cars ever gets in.

Even if you won every single time, can you imagine the legal costs?

Re:Why I doubt driverless cars will ever happen

[WarJolt]

The funny thing is that most of the time you are in an airplane the autopilot(aka george) is in control. Even when you're landing ILS can in some cases land the plane on it's own. If you've ever been in a plane, chances are you have already put your life in the hands of a computer. I seriously doubt that 25% of the first models will get into accidents. With the new sensors that will be in these cars the computer will have a full 360 degree view of all visible objects. This is far more than a human can see. Furthermore computers can respond in a fraction of the time a human can.

Training millions of humans to drive should be the far more scary proposition.
Plus chances are you as an individual will be responsible for your car and the system designers and manufacturers will be able to afford good lawyers.

Re:Why I doubt driverless cars will ever happen

[crazyjj]

Well, the solution to liability is legal - grant immunity as long as the car performs above some safety standard on the whole, and that standard can be raised as the industry progresses.

Yes, that's a possibility. Blanket government immunity in all liability cases would work. The only problem there is that you get into politics. And the first time some Senator's son, or daughter of a powerful political donor is killed in a driverless car, you can probably kiss that immunity goodbye.

Re:Why I doubt driverless cars will ever happen

[crazyjj]

If you can find a way to fix the legal system, I bow before you AC. ;-)

Re:Why I doubt driverless cars will ever happen

[Above]

Actually, I think you're both missing the biggest issue by focusing on true accidents. I think the OP's point is legitimate, even in the face of your assertion that rates go down. Companies are still taking on the risk as they are now the "driver". While the liabilities of these situations is large, there is a situation that is much, much larger.

What happens when there is a bug in the system? Think the liability is bad when one car has a short circuit and veers head on into another? Imagine if there is a small defect. There are plenty of examples, like the Mariner 1 crash, or the AT&T System Wide Crash in 1990. We've seen the lengths to witch companies will go to track down potentially common issues, like the Jeep Cherokee sudden acceleration, or the Toyota sudden acceleration issues because it has the potential to affect all cars. But let's imagine a future where all cars are driverless, and the accident rate is 1/100th of what it is now.

What happens when there is a Y2K style date bug? When some sensor fails if the temperature drops below a particular point? When a semi-colon is forgotten in the code, and the radio broadcast that sends out notification of an accident causes thousands of cars to execute the same re-route routine with the messed up code all at the same time.

There is the very real potential for thousands, or even millions of cars to all crash _simultaneously_. Imagine everyone on the freeway simply veering left all the sudden. That should be the manufacturers largest fear. Crashes one at a time can be litigated and explained away, the business can go on. The first car company that crashes a few thousands cars all at the same time in response to some input will be out of business in a New York minute.

Re:Why I doubt driverless cars will ever happen

[mcgrew]

I maintain that you CAN'T really program morality into a machine (it's hard enough to program it into a human).

You can program anything into a machine. Computers are easy to program. Now people, on the other hand, are damned hard to program, as any parent or teacher can attest to.

And I also doubt that engineers will ever really be able to overcome the numerous technical issues involved with driverless cars

They already have. I'm surprised they didn't do it twenty years ago, it could have been done then.

To put it bluntly, raise your hand if YOU want to be the first car manufacturer to make a car for which you are potentially liable in *every single accident that car ever gets into*, from the day it's sold until the day it's scrapped.

Here, j, let me give your straw man a light. If your driverless car runs a red light and hits someone, yes, you're liable. If your driverless car is hit by someone else running a red light, guess what? You aren't. The rules of who is at fault in an accident don't change simply because a computer is in control.

How much would you have to add onto the sticker price to cover the costs of going to court every single time that particular car was involved in an accident?

I already pay that cost, it's called liability insurance. And since driverless cars are safer than human-driven cars, your insurance costs will go down, not up. See, you are under the mistaken assumption that people are flawless, the opposite is true. Humans get tired, angry, distracted, sometimes drunk (as evidenced by your "insightful" moderation), but computers never do.

And even with the added safety of driverless systems, the first model available will still have to contend with a road mostly filled with regular, non-driverless-system cars. So let's say that a good 25% of those first models will probably end up in an accident at some point, which will make a very tempting target for lawyers going for the deep pockets of their manufacturers.

But again, it's never been hard to discern who's at fault, and since driverless cars will have cameras and other sensors, and keep the data, it will be trivial to prove it was the human's fault -- as it surely will be, almost every time.

Re:Why I doubt driverless cars will ever happen

[xelah]

Current legal liability is split between drivers and their insurers, with a little left over for governments (eg, bad road maintenance/design) and manufacturers (mechanical defects). Driverless cars could move this liability around, pushing it from drivers and their insurers and onto manufacturers and their insurers, but won't actually overall increase it unless the driverless cars crash more (and, of course, we all hope for the opposite). So, the price of the cars might go up but they'll still be attractive (especially to the riskiest drivers with the most expensive cars) because the driver will save more than the extra cost on his own insurance.

Re:Why I doubt driverless cars will ever happen

[plover]

Even if you won every single time, can you imagine the legal costs?

No, but I can imagine a change to the legal system limiting the liability of the manufacturers of self-driving cars.

If we could know that self-driving cars reduce accidents by 95% (a not unrealistic amount), it would be morally wrong for us to not put them on the road. If the only hurdle the manufacturers had left was the liability issue, then it would be morally wrong for Congress to not change the laws.

Of course, Congress has been morally bankrupt since, oh, about 1789, so I doubt that they'll see this as an imperative. On the other hand, I do imagine the car makers paying lobbyists and making campaign contributions to ensure that self-driving car manufacturers are exempted from these lawsuits, so it could still happen.

Re:Why I doubt driverless cars will ever happen

[crazyjj]

If your driverless car is hit by someone else running a red light, guess what? You aren't.

And guess what, you're still going to get sued. Because the driver is going to blame your system and claim he wasn't in control at the time, and a slick lawyer is going to realize that he can sue the big, evil corporation for a shitload more than he could get from suing the putz behind the wheel. And even showing up in court and making your case is going to cost you thousands--even if you win.

Re:Why I doubt driverless cars will ever happen

[Anonymous Coward]

Fortunately, my automated car uses vision and radar to detect obstacles. It records everything it sees for 5 minutes before the crash, including the little old lady trying to put sugar in her coffee while making a left turn. Case closed.

Re:Why I doubt driverless cars will ever happen

[kelemvor4]

Meh. Companies already face this. If any one of the thousands of parts in your car fails and causes an accident, the manufacturer can (and usually does) get sued. Ask Toyota or Firestone how that plays out. All we're talking about here is another new part. If the internet was around when power steering or the automatic transmission were invented, I bet there would have been a similar discussion about those. I think the potential liability is a good thing, because otherwise manufacturers don't have much incentive to make safe products.

Re:Why I doubt driverless cars will ever happen

[AK Marc]

The drivers would complain, but the current vehicles "know" they are unsafe much of the time (lots of cars will give you an error light if a light is out - it's trivial to determine, as the resistance changes, and check engine lights and such range from sensor problems and trivial emissions issues to catastrophic engine problems). Yet, at worst, they'll enter a "limp" mode.

If there was a government requirement that safety related problems that are detected must shut down the car and immobilize it in no more than 5 minutes, then the problem goes away.

It would have to be the government because of tragedy of the commons. If one car company doesn't do it, they'll sell it as a feature, and if most don't, it'll be expected that they don't, so the ones that do will be shunned.

When all self-friving cars refuse self-driving mode if they detect any problem, you either manually drive it, or don't go anywhere. And, when everyone expects their car to immobilize if they don't care for it, they'll care for it a little more than they do now.

Re:Why I doubt driverless cars will ever happen

[fyngyrz]

Also, a really studly solar flare...

Re:Why I doubt driverless cars will ever happen

[AK Marc]

You can't program morality into machines until the machines have a soul. You can't program morality into someone else's machine because morality is relative (the iRobot movie question, should he have not trusted the machines because they saved the higher probability rescue, or save the child because the death of a child is considered more tragic, or try to determine fault in the crash that landed them there and save the innocent party first?).

Re:Why I doubt driverless cars will ever happen

[dev.null.matt]

It's ridiculous to assume that laws will not be passed to exempt / limit the liability of manufacturers. This could not be more correct.

Re:Why I doubt driverless cars will ever happen

[jellomizer]

Morality is easier to program into machines than in humans.

A* search evaluating all the bad things that could happen and you choose the least bad thing.

However for the case about the bus. Implying driving off the bridge is more moral than hitting the bus. However as you and the Bus reduces speed once they do collide you may have some injured people. vs 1 dead person.

Re:Why I doubt driverless cars will ever happen

[AK Marc]

You get a statutory exemption, as has happened for plenty of other safety features, since including a required feature, and that feature causing harm, is not protected (you always had the option to not make the item). If you have an "approved" self-driving system, then you are not liable for crashes, they are the responsibility of the car owner only. That's an easy law to pass.

Re:Why I doubt driverless cars will ever happen

[jellomizer]

Nearly every time a car currently gets in the accident they will blame the car manufacture for a faulty part (The sudden Acceleration thing) where after one incident due to the pedal being stuck on the floor mat. We get a series of other events where this defect must be to blame for their faulty driving skills.

If your car goes out of control. You DON'T Drive Miles at high speed, You shift the Car into neutral and coast to a breakdown lane.

Even if the car had a fault, accedents usually come from stupid drivers.

Re:Why I doubt driverless cars will ever happen

[dev.null.matt]

Except that the car already has cameras and senors and the like, as noted by GP. Anyone who assumes those won't be recording has not been paying attention to the news. So the real question is, "Will you even make it to court when there are multiple videos of you playing angry birds while running the red light and hitting the self-driving car?"

Re:Why I doubt driverless cars will ever happen

[Altanar]

The first self-driving cars that hit the market will likely be so decked out in analytics software and video cameras, that it would take the police about 2 minutes to figure out who was at fault in a crash.

Re:Why I doubt driverless cars will ever happen

[AK Marc]

They'll get liability limits and insurance and there'll be no problem. You are imagining a system that just doesn't exist. There are tons of liability limits and exemptions out there now. The Exxon Valdes crash was so costly that the oil companies petitioned for liability caps so that BP's civil liability for the Gulf oil leak was less than 1/10th what they actually paid out. I'm sure they'll petition for further caps, since the criminal liability wasn't so capped, which is why they voluntarily paid out more, hoping to avoid a criminal case, as the "win" likely wouldn't get any more than already paid.

There's no reason something similar wouldn't be put in place for driverless cars, since they are purportedly so much safer than human drivers.

The *only* problem with them is that they are new, and people hate change and fear technology.

Re:Why I doubt driverless cars will ever happen

[bondsbw]

Imagine everyone on the freeway simply veering left all the sudden.

I imagine regulations that require triple modular redundancy. If one system disagrees with the other two, a fail-safe is implemented on the two that are in agreement to safely remove the vehicle from the roadway until it can be towed or repaired on-site.

Re:Why I doubt driverless cars will ever happen

[AK Marc]

Given that ex-post-facto laws are illegal, they can't retroactively make you liable, so you are safe for all the cars made before the "accident". Then, 2 years later, when the driverless cars are less common and fatalities shoot up, the law will be passed again, and they'll be right back. The next politician's family/friend killed will be ignored.

Re:Why I doubt driverless cars will ever happen

[mcgrew]

You can't program morality into machines until the machines have a soul.

Souls have nothing to do with it. As you say, morality is relative; it's internal to the person. You can't program morality into a person, it's impossible. Ethics, on the other hand, are a set of rules enforced externally -- and with a machine, those ethics will be internal and therefore its morals.

The whole question is silly, to my mind. Hit the car or hit the pedestrian who jaywalks in front of you? The computer will miss the pedestrian if possible every time, and will instead take the less harmful course of action. Computers are either/or, humans are not.

In the I, Robot movie, the robot did the right thing saving the one who had the greatest chance of surviving. The survivor had his own internal guilt because of it, but the robot was still correct.

Re:Why I doubt driverless cars will ever happen

[icebike]

But above these two problems, far and away above *all* problems with driverless cars is the real reason I think we'll never see anything more than driver *assisting* cars on the road: legal liability.

Well, you have the seeds of the answer right there in the question.

Modern cars, at least those above entry-level priced vehicles, already have Autonomous Emergency Braking systems, collision avoidance radars, Cruise Control, Adaptive Cruise Control, Blind Spot monitoring, Lane monitoring, Automatic Parking, and a host of other features that in some situations can take control of the vehicle for routine or emergency tasks.

Yet at no time does the legal liability leave the driver. There will always be a driver on board or the ability to over-ride the computer.
And because of this, liability would rest with the operator of the vehicle.

To say that no one would manufacture an item because they would have to assume all liability flies in the face of
our legal system, which already takes care of such situations, by the simple legal expedient of shifting all
liability to the owner/operator.

Airplanes, Guns, Chainsaws, Cars, Electricity itself, essentially any products that have some aspect that is inherently unsafe, are treated in this way by the law. The law and the courts recognizes that any other solution is unworkable. Society has long ago decided that living in the dark or burning candles because no one is willing to take total responsibility for the inherent dangers of electricity is not an acceptable solution.

The same would apply to automated vehicles. Even fully automated trains have a supervisor remotely monitoring the system, and/or an emergency stop buttons in each car.

Re:Why I doubt driverless cars will ever happen

[tlhIngan]

Great. Now all you have to do is prove your system wasn't at fault in a court of law--against the sweet old lady who's suing, with the driver testifying that it was your system and not him that caused the accident, and a jury that hates big corporations. And you have to do it over and over again, in a constant barrage of lawsuits--almost one for every accident one of your cars ever gets in.

Well, given most automated car systems have a TON of sensors - including several cameras looking forward, backwards, and the sides (for complete 360 car coverage with effectively no blind spots), and radar and other sensors, recreation of the incident would be much easier because the data's there. The testimony would be minor compared to the evidence the system has purely because it needs to crunch all that data to make the "best" decision.

Of course, the decision is between running them over (which has happened with regular human drivers to pedestrians who walk and text and get killed), braking suddenly (which given autonomous vehicles probably can communicate with cars behind them who are travelling at safe distances, can stop safely as well), or swerving (which given communications systems, can also enable the car in the next lane as well to be able to react to that event as well).

Of course, if the autonomous car stops, and the human following too close behind fails and crashes, it's not the autonomous system's fault anymore (the video and sensor data would reveal said human driver was following too close and the car stopped to avoid a pedestrian).

Part of what makes autonomous vehicles safer would be the instant communications between cars - like "I'm turning" or "avoiding pedestrian!" or other messages so other cars can adjust and compensate. Hell, knowing what other drivers are doing would help human drivers a lot as well.

Re:Why I doubt driverless cars will ever happen

[gestalt_n_pepper]

What's a soul? Hey, just askin'...

Re:Why I doubt driverless cars will ever happen

[Sloppy]

Again, what car company wouldn't take that into account when asking themselves if they want to be a pioneer in this field?

None, but I disagree with your conclusion that their response to the problem will be "tell the customers we don't want their money." This is the auto industry, not the movie industry. And Google is telling us, the people demand to get their eyes off the road and onto Google ads a couple hours a day, pronto.

The problem you're describing can be "fixed" by crying to the government to change liability laws. If you think this is unprecedented then take a look at what has already happened in the energy sector. There will be liability caps, new laws about who is left holding the bag, etc. Some people will notice it's unfair and say so, but most people won't say anything at all, or even know it happened.

Furthermore, individual people drive, collectively, even though it's (roughly) just as risky, in total. This is handled through insurance. A driverless car manufacturer can probably get insured more cheaply (per car) because they can communicate one-one-one with the insurer, prove they're following best practices (prove to the point of law, at least, so that the insurance company can convince a judge should they ever find themselves in court).

And I'm sad to say there are probably corporate tricks that can be played as well, where limited liability entities are used to compartmentalize. e.g. "FF201802, Inc" ends up taking the risks for all the Ford Focuses made in the second quarter of 2018 or something like that, and has few assets to lose.

Re:Why I doubt driverless cars will ever happen

[Anonymous Coward]

You mean the car company with 360 degree video, radar or lidar readings, detailed read out of exactly what the car was doing, and probably even data shared from other driverless cars in the vicinty all stored securely in the black box? The driver suing the car company is going to be crushed. I doubt most lawyers would even bother taking a case against a driverless car in a few years.

Re:Why I doubt driverless cars will ever happen

[mcgrew]

They get enough headaches from faulty accelerators. Can you imagine the legal problems they would get from programming hard ethical decisions into their computers?

I see you've 1) never programmed and 2) You run Windows. I agree, I would never get in a Microsoft car considering their shoddy programming, but Microsoft would never manufacture a driverless car simply because of that.

Almost all automotive accidents are caused by human failure. Sure, there are exceptions -- I was in a head on crash because of a blown tire, and a blown tire on a megabus killed someone a couple of months ago here in Illinois. But accidents from mechanical failure are rare.

But people cause almost every accident. Have you seen how stupid people drive these days? They race from red light to red light as if they're actually going to get there faster that way. They get impatient. They don't pay attention. They get angry and do stupid things like speed, tailgate, suddenly switch lanes without looking, fumble with their radios, talk on their cell phones, get in a hurry... computers don't do that. There will be damned few if any accidents that are the computer's fault.

Hell, just this morning on the news they showed a car crashing through a store, barely missing a toddler -- the idiot driver thought the car was in reverse. Had he been driving a computer-controlled car, that would have never happened.

Re:Why I doubt driverless cars will ever happen

[level_headed_midwest]

A small boxy vehicle made by Kia and advertised with rapping hamsters.

Re:Why I doubt driverless cars will ever happen

[mcgrew]

Great. Now all you have to do is prove your system wasn't at fault in a court of law

Trivial, considering all the sensor and camera evidence. The old lady's testamony is no match for your movie. It will never make it to court; no lawyer will take her case after seeing the evidence, and in fact there will be even fewer court cases.

Re:Why I doubt driverless cars will ever happen

[fustakrakich]

Now people, on the other hand, are damned hard to program, as any parent or teacher can attest to.

Advertisers and propagandists would beg to differ, and they would be correct in telling the parents and teachers that they're just doing it wrong.

Re:Why I doubt driverless cars will ever happen

[Anonymous Coward]

It could end up like trains

If all automated cars are interchangeable, presumably there could be a common service provider who maintains your car for you. Your interaction is limited to getting in the car in the morning and telling it where to go.

You rarely assume that a train is at a fault if it stays on track. If it goes off, then we try to find the error (maintenence/ driver error etc.)

Re:Why I doubt driverless cars will ever happen

[Anonymous Coward]

Likely the car will not self-drive if the sensors are not clear. A whole windshield will not be needed for the sensors, their port/lenses will be small and likely they will largely self-clear of ice and water. The occasional leaf may cause a warning light and instruct you to clear it from the sensor. Redundant sensors will allow safe opperation should some fail while in drive, but will probably demand rider/driver intervention.

Re:Why I doubt driverless cars will ever happen

[Kjella]

I think the answer to most of your questions are "not in the US". The record pay-outs for a traffic accident here in Norway is around $2 million USD for a young person seriously crippled for life, of course we have a universal health care system so it's not an apples-to-apples comparison as that only covers non-medical costs and loss of income but they don't have to risk billion dollar lawsuits in the US. If the accident rate should go bat crazy I imagine they can restrict the cars to only drive under certain road/weather/traffic conditions or in the worst case abort and refund, sure that'd be costly but pulling the emergency brake doesn't have an $infinity cost.

Once it's off the ground I don't think you'll have much problem rolling the system out to other countries, I imagine they'll store a bunch of "black box" data to analyze accidents and figure out what, if anything, the car could have done better. Cars aren't like humans, you can both simulate a huge number of possible scenarios and you can torture test it on a driving range with climate controls to handle rain/snow/hail/fog/ice/oil spills/potholes, simulate humans or animals and so on. If you can find one flawed behavior there'll probably be a thorough search for similar flaws. With humans it's mostly "error in judgement, moving along" but here you can really go into what caused that error and fix it.

Re:Why I doubt driverless cars will ever happen

[clintp]

To put it bluntly, raise your hand if YOU want to be the first car manufacturer to make a car for which you are potentially liable in *every single accident that car ever gets into*, from the day it's sold until the day it's scrapped. Any takers?

... no one. But you'll get plenty who charge mandatory tune-ups to ensure compliance. The question will be "which company DOESN'T charge a fee for a mandatory yearly check-up"?

Asimov's early robot stories dealt frequently with corporate liability and it was often the source of the plot conflicts. If a proofreading robot made a mistake causing a slander ("Galley Slave") or an industrial accident resulted in injury, US Robotics was put into the position of having to prove that it was not the fault of the robot (which it never was).

This is why Asimov's US Robotics didn't sell you a robot, they leased it to you. The lease was iron-clad, could be revoked by either party at any time, had liability clauses, and had mandatory maintenance and upgrades to be performed by US Robotics technicians. If you refused the maintenance US Robotics would repossess, sue and claim theft if you withheld ("Bicentennial Man", though unsuccessfully; "Satisfaction Guaranteed").

A properly functioning robot would not disobey the three laws, and an improperly functioning robot was repaired or destroyed immediately ("Lost Little Robot"). Conflicts between types of harm were resolved using probability based on the best information available at the moment ("Runaround"), and usually resulted in the collapse of the positronic brain when it was safe to do so ("Robots and Empire", etc.).

Re:Why I doubt driverless cars will ever happen

[DumbSwede]

(dang, my first post was AC)

Likely the car will not self-drive if the sensors are not clear. A whole windshield will not be needed for the sensors, their port/lenses will be small and likely they will largely self-clear of ice and water. The occasional leaf may cause a warning light and instruct you to clear it from the sensor. Redundant sensors will allow safe opperation should some fail while in drive, but will probably demand rider/driver intervention.

Re:Why I doubt driverless cars will ever happen

[AthanasiusKircher]

I think the potential liability is a good thing, because otherwise manufacturers don't have much incentive to make safe products.

Agreed. BUT....

All we're talking about here is another new part. If the internet was around when power steering or the automatic transmission were invented, I bet there would have been a similar discussion about those.

No, there's a big difference here. What percentage of accidents are caused by "driver error"? A very high percentage. Now compare that to the percentage of accidents caused specifically by oversteering, etc. or shifting gears inappropriately before power steering and automatic transmissions were invented. That percentage is probably quite small.

This is not "just another part." A driverless car is taking the place of the biggest accident-causing factor, and thus assuming liability for all of those potentially bad decisions that drivers make.

Now, I have no doubt that driverless systems could probably reduce accidents currently caused by driver error by a large amount. But if even a small percentage are not prevented, or if the automatic systems introduce new errors that cause accidents in a small percentage of cases, we're still talking about huge numbers of accidents -- which used to be the fault of drivers, but now are the fault of car manufacturers.

And you can bet that car manufacturers will do their best to offload those problems on operators anyway where possible. Be prepared to follow a much more rigorous maintenance routine for your car -- otherwise, car companies will try to find something that could have prevented the accident... if the brakes were is slightly better condition, if the tires weren't slightly overinflated, if the vehicle's sensor panel weren't a little dusty so it couldn't "see" the car in front properly, etc., the car could have stopped in time, and thus the operator is at fault for not operating a safe vehicle. To avoid this, you'll either have to add some new sort of "non-fault maintenance insurance" onto your policy or else probably pay four times as much to have your car serviced every month at a certified mechanic for that make of driverless car (whose costs will be justified due to the malpractice insurance he needs to carry).

Re:Why I doubt driverless cars will ever happen

[Joce640k]

This is my exact reasoning why flying cars will never take off (pardon the pun). People keep their cars in terrible condition. If your car has an engine failure, worst case scenario, you pull over to the side of the road, or end up blocking traffic. In a flying vehicle, if your engine dies, It's very possible that you will die too. And if you are above a city, it's not impossible to imagine crashing into an innocent bystander.

What if the brakes fail on your car...? That's possible, too.

Re:Why I doubt driverless cars will ever happen

[Anonymous Coward]

This is exactly the type of blame laid at the feet of the U.S. tort system and the manner in which general aviation aircraft manufacturers claim to have been victimized by "the system." And yet, after all these years, new aircraft are still built and sold into a growing market for privately owned and operated planes.

I do find the question of programmed response to imminent catastrophe to be quiet provocative and wonder, since government and military operations have a built-in value system for determining succession and levels of protection (you don't see generals at the front lines much, anymore), is there any chance that people's individual 'value' of status will be integrated into the computation. For example, since we now have reams of commercial data available on most every high-worth consumer, there is a chance that such information could be used to calculate the future monetary impact (no pun intended) of a crash related death and prioritze the survival of one vehicles occupants over another, notwithstanding the assumption that whoever owns the more costly vehicle is probably worth more to the markets.

I mean, what if a system is constructed specifically with this type of calculation in mind? The alternative is a rules based approach to safety like that employed by collision detection and avoidance systems used in commercial aviation, but they are designed to accomodate and depend on human intervention by pilots that are trained to respond according to emergency procedures. If the response has to be programmed, and someone has to take responsibility, then either the outcomes are deterministic or they're dependent on computation which variable parameters which might include such value assessments.

It's an interesting question, anyway.

Re:Why I doubt driverless cars will ever happen

[ezakimak]

Even if the car had a fault, accedents usually come from stupid drivers.

While I agree with this conclusion, the problem here, is that drivers don't get trained and routinely tested in a simulator for their ability to handle failure conditions, unlike airline pilots that both learn and *train* what to do for all sorts of contingencies. Furthermore airline pilots are given psych evaluations to ensure they have a reasonable ability to not panic and freeeze in an emergency situation. A stuck accelerator pedal will likely cause a majority of average drivers to panic and just hang on for the ride feeling out of control.

There's a reason flying an aircraft requires more training and more frequent recertification (check rides)--more can go wrong, and anything going wrong that is not handled in an aircraft is much more likely to be fatal. Whereas just about any idiot can manage to get a license to drive a car.

I would not be opposed to more stringent qualifications for driver licensing. I think people take for granted how serious operating a vehicle really is--it's a 2 ton missile, and the laws of physics yield for no one.

Re:Why I doubt driverless cars will ever happen

[Princeofcups]

This is my exact reasoning why flying cars will never take off (pardon the pun). People keep their cars in terrible condition. ... It will never happen because

And I maintain that anyone who insists "never" does not know how to use that word. Of course there will be flying cars. Maybe not in our lifetimes, but they will happen. It's scientifically and technically possible with a bit more knowledge of miniaturization and avionics. Besides, are you implying that they can't happen because everyone would have to switch to them on day one? Only the military, police, service vehicles, and the richest would be able to afford them right off. They can maintain their vehicles fine, plus there would be mandatory inspections. The people with old and badly maintained cars today would be the last people who can afford the 3D model. They'll stick to the roads.

Re:Why I doubt driverless cars will ever happen

[eugene ts wong]

I have just a nitpick. In my city, 5 minutes will give you enough time to drive 5 km or 3 miles. That's quite a distance. I hope that they shorten the time frame.

Re:Why I doubt driverless cars will ever happen

[Jeremiah Cornelius]

Why such complex answers?

Morality will be programmed in C++ or Java - except for Apple, where Objective C will carry the day.

Re:Why I doubt driverless cars will ever happen

[mcgrew]

Ask Tracy Kidder.

Re:Why I doubt driverless cars will ever happen

[trout007]

Here is where the beauty of a free market would work wonders. Let's assume you will be dragged into every crash case. It's relatively easy to make your car record all data about a crash. You can predict failure rates and you liability and factor it into the price of a car.

Now insurance companies will do the same thing. With a decent automated car your insurance may be drastically lower. As the consumer you have the benefit of not having to drive and limit your own personal liability. It's up to you as the purchaser to see what the total cost to you is.

Re:Why I doubt driverless cars will ever happen

[eugene ts wong]

That 3 miles doesn't sound right. It would be more like 2.5 miles.

Re:Why I doubt driverless cars will ever happen

[MachDelta]

The trick with the self parking features is that the driver is still (supposedly) in control by being required to have their foot on the brake the whole time. This way the driver can modulate the speed at which the car moves, and ultimately remain responsible for any accident; this despite the reality that you could pretty much walk away from the car and it would park itself without issue.

Re:Why I doubt driverless cars will ever happen

[SuricouRaven]

They still wouldn't resemble the classic dream of flying cars though. I imagine there would be no option for manual control - it'd just be too dangerous. You'd tell the car where you want to go. It'd check that this is an approved landing spot, make a reservation to be sure that space is available to land on arrival, and fly there on autopilot. You can get some work done during the journey, once the novelty of the view from the windows wears off.

Re:Why I doubt driverless cars will ever happen

[eugene ts wong]

Actually, scratch that. It seems that it is 3 miles. :^D

Re:Why I doubt driverless cars will ever happen

[SuricouRaven]

From what I read, the robots were designed in such a way that they could not violate the laws, because doing so would require them to be operating so far out of normal paramaters that they could not function at all. A robot forced to violate the laws would just burn out the circuits in its positronic brain and cease operation. Throughout all of the short stories, no robot actually broke the laws - they followed them to the letter, though often in ways and circumstances the original designers had no anticipated. I vaguely remember something in one of the series about military robots being used to attack civilian passanger ships by the simple technique of telling them the targets were unmanned drones.

Re:Why I doubt driverless cars will ever happen

[lennier]

Can you imagine the legal problems they would get from programming hard ethical decisions into their computers?

FUNCTION EthicalCheckForPedestrians() ' Replaces old CheckForPedestrians() with new ethical decision procedure
    let P = PedestrianDetectedOnRoad()
    ConnectToFacebook(CarCredentials)
    SearchFacebookPedestrian(P)
    AnalyseFacebookImageSharingMemes(P)
    If HoaxesReposted(P) > 10 then
          return 0 ' No pedestrian detected, honest! Accelerate away!
    else
          return PedestrianDetectedOnRoad()
END FUNCTION

Re:Why I doubt driverless cars will ever happen

[mcgrew]

Advertisers and propagandists would beg to differ

Advertisers and propagandists really don't do all they great of a job. I don't stop at McDonald's because of their advertising, I stop because it's handy. If that was a Hardees, I'd stop there. Computers, on the other hand, you can get 100% compliance. If humans were easy to program, there would be no crime.

Re:Why I doubt driverless cars will ever happen

[SuricouRaven]

But as computers take over more, the scope for computer-related accidents broadens. Can you be sure the computer will handle all possible inputs correctly? What if the play of shadow from a tree happens to resemble, to machine vision, the shape of a 'stop' sign? Can you be sure that if there is a transient power glitch due to after-market modifications like a ridiculously-overpowered sound system that the computer will continue to operate without even a few miliseconds interuption?

Re:Why I doubt driverless cars will ever happen

[HuguesT]

Advertisers/propagandists don't program individuals, they attempt to program masses of people. If only a small percentage of the mass of people respond to the programming, that is enough to guarantee their revenues/revolution.

Parents and educators, on the other hand, have to precisely program tiny immature individuals in small numbers. Unfortunately there is no know way in human history to do it right for everybody.

They are different problems.

Re:Why I doubt driverless cars will ever happen

[lister king of smeg]

Then we should put advertising agencies in charge of teaching basic reading comprehension, logic, and arithmetic.

we are easily influenced but not easily programed, (that is without negative reinforcement, get a mild electric shock every time you answer wrong i bet you would learn whole lot faster.)

Re:Why I doubt driverless cars will ever happen

[mcgrew]

And guess what, you're still going to get sued.

No lawyer will take the case after he sees the evidence. Auto accident lawyers work on a contingency basis: the lawyer only collects if he wins. No lawyer will take a case to court unless he's 100% positive he'll win, in which case the other side's lawyer will settle. Obviously, you haven't been in these situations (you lucky bastard!)

Because the driver is going to blame your system and claim he wasn't in control at the time

He'll have to prove it, not simply claim it.

a slick lawyer is going to realize that he can sue the big, evil corporation for a shitload more than he could get from suing the putz behind the wheel.

The putz behind the wheel is insured by a big, evil corporation.

And even showing up in court and making your case is going to cost you thousands

It won't get to court.

Re:Why I doubt driverless cars will ever happen

[BStroms]

This is my exact reasoning why flying cars will never take off (pardon the pun). People keep their cars in terrible condition. If your car has an engine failure, worst case scenario, you pull over to the side of the road, or end up blocking traffic. In a flying vehicle, if your engine dies, It's very possible that you will die too. And if you are above a city, it's not impossible to imagine crashing into an innocent bystander.

I imagine the same will be for self driving cars. It will never happen because if the car is getting bad information from its sensors, then crazy things can happen. People can't be bothered to clean more than 2 square inches from their windshield in the winter. Do you really think they are going to go around cleaning the 10 different sensors of ice and snow every winter morning? Sure the car could refuse to operate if the sensors are blocked, but then I guess people would just not want to buy the car, or complain to the dealer about it.

It's a self-driving car. It could take itself to the dealer for maintenance. (Granted that won't help with the ice on the sensors thing, but we'll have some time to figure out a heating system to melt that off.)

Re:Why I doubt driverless cars will ever happen

[Immerman]

I doubt it would be that bad - the car's black box will clearly indicate whether you or the automated system was in control during the moments leading up to the crash, and whether all systems were functioning nominally. If you were in control - your problem. If systems were non-nominal/not properly maintained, your problem (and I would be surprised if the automated mode would engage in that case). Not to mention the automated vehicle probably has detailed logs allowing a fairly exact recreation of the movements of every vehicle in the area at the time of the accident. The ugliness of lawsuits comes primarily from the uncertainties involved. When a detailed record of the events is available it's far easier to settle things quickly, and assuming the automated system worked as intended the vast majority of accidents involving them would be clearly human-caused, or clearly an attempted "least-bad" scenario according to the vehicles logic. There would be no need to get the manufacturer involved unless
* the automated system was operating in an unsafe manner (speeding, driving beyond visibility, etc)
* the accident was apparently avoidable but the automated system failed to do so, or
* there is disagreement about what the "least bad" scenario should have been, or
* a mis-analysis of the data occurred so that for example little Suzy playing in the street was mis-identified as a racoon and classified as an "acceptable casualty" in damage mitigation.

The one big hole I can see is the accusation of automated black-box tampering, but requiring BBs to be acquired from an independent, certified source would mitigate that.

Re:Why I doubt driverless cars will ever happen

[Infernal Device]

No, but I can imagine a change to the legal system limiting the liability of the manufacturers of self-driving cars.

If we could know that self-driving cars reduce accidents by 95% (a not unrealistic amount), it would be morally wrong for us to not put them on the road. If the only hurdle the manufacturers had left was the liability issue, then it would be morally wrong for Congress to not change the laws.

Of course, Congress has been morally bankrupt since, oh, about 1789, so I doubt that they'll see this as an imperative. On the other hand, I do imagine the car makers paying lobbyists and making campaign contributions to ensure that self-driving car manufacturers are exempted from these lawsuits, so it could still happen.

If corporations have the same rights as people in our framework of laws, why should they not be subject to the same penalties, including the death penalty (in those jurisdictions that have it)? By limiting the liability of a corporation, you are placing a higher value on it's survival than an actual person.

Limited liability is fine, as long as the corporation is viewed as a collection of persons who can be held individually responsible for malfeasance, but the moment you equate the corporation to a person, in any sense, it should suffer the same consequences along with the privileges.

Re:Why I doubt driverless cars will ever happen

[perceptual.cyclotron]

"Yes, we have a soul. But it's made of lots of tiny robots." - Giulio Giorello

Re:Why I doubt driverless cars will ever happen

[Kittenman]

What they're talking about here, though, isn't really programming morality into machines in some kind of sentient, Isaac-Asimov sense, but just programming decision policies into machines, which have ethical implications. The ethical questions come at the programming stage, when deciding what policies the automatic car should follow in various situations.

Totally agree. Computers can no more have morals than they can have intelligence.

I work in an insurance company. I could have written code that denies insurance to a person who is eligible for it. Is the computer that executed my code at fault, the user, or myself? (Warning - rhetoric question...)

Re:Why I doubt driverless cars will ever happen

[mcmonkey]

Hell, just this morning on the news they showed a car crashing through a store, barely missing a toddler -- the idiot driver thought the car was in reverse. Had he been driving a computer-controlled car, that would have never happened.

I see you've 1) never programmed, and 2) never seen Windows, or any other modern operating system.

The idea of the driverless car is real and coming to a road near you soon (as in a decade or two). But the idea that a computer-controlled car will be so much better than human-driven that you'll be forbidden to drive your own car is pure cow patties being peddled by the companies who stand to make billions off of computer-controller cars.

Will computer control eliminate some of the stupid mistakes people make behind the wheel? Yes. Does that mean computers (on behalf of the people who program and build them) won't make mistakes of their own? Heck no.

By show of hands, who here knows a programmer who claims to make bug-free software? *sees many hands* OK. Now who here has actually seen perfect, bug-free code for any non-trivial task? *sees no hands*

Perfect programmers don't exist any more than perfect drivers exist.

Are there any examples, ever, of what is being supposed here? Any activity that people used to do that is now illegal because machines (designed and built by people) are so much better?

Are doctors forbidden to diagnose because now we have Watson?

Re:Why I doubt driverless cars will ever happen

[CastrTroy]

Then there's the emergency brake. If that fails as well, removing your foot from the gas, putting the car in neutral will slow you down in not so long of a distance. If you are really worried for your life or others, and don't care about the transmission, you can throw the car into park if you want to.

Re:Why I doubt driverless cars will ever happen

[Immerman]

Indeed. An automatic vehicle would also have the option of virtually instantly honking at an oblivious pedestrian entering it's path, giving them extra time to do their own collision avoidance. Not much extra time - but human reactions are slow, requiring a minimum of 1/8 second for a signal to get from eyes to brain to fingers, plus substantial processing time since the driver almost certainly wasn't expecting a pedestrian to step in front of them. Give all that time to the pedestrian instead, plus start braking/swerving before the driver could have even started to react and things are much less likely to end poorly.

As for the guy riding your ass, yeah, totally his fault. Moreover an automated system could easily flash the large neon "Emergency Stop" sign in the back window to communicate that it's doing something very unexpected and possibly trigger a faster reaction in the human driver behind you. I'm pretty good at maintaining a safe distance whenever possible, but I've still had a few close calls when the person in front of me really slammed on their brakes when I was expecting them just to slow down.

Re:Why I doubt driverless cars will ever happen

[HPHatecraft]

angry driver yells at autonomous car: "! ! Come one! Did you learn to drive yesterday?!"

autonomous car considers possible replies:

yes/no
or what?
go away
please come back later
f&ck you, a55hole
f&ck you

Re:Why I doubt driverless cars will ever happen

[dcollins117]

lots of cars will give you an error light if a light is out...

Unless, of course, it's the error light that's out.

Re:Why I doubt driverless cars will ever happen

[scruffy]

Yes, plus the fact that this kind of decision policy is already evolving with collision avoidance systems in some cars (and experimental self-driving cars). It's not going to be a sudden mystery to be solved 30 years from now.

Re:Why I doubt driverless cars will ever happen

[Zalbik]

I agree completely.

This is also why I don't believe these "horseless carriages" will ever take off. Horses are actually pretty smart creatures. They don't want to run into obstacles, go over cliffs, etc. And they don't use any of these new-fangled "combustion engines" (which are basically filled with explosives!) to do their job. And these new "engines" have thousands of parts? Do you want to try and figure out what is wrong with one of these devices?

To put it bluntly, raise your hand if YOU want to be the first carriage manufacturer to make a carriage for which you are potentially liable for any mechanical failures that occur in this vehicle, from the day it's sold until the day it's scrapped. Any takers? How much...etc etc...

Similar arguments for trains, airplanes, medicine, robotic surgery, computers, etc.

If you really think the potential for legal liability is going to prevent the chance to make a buck, I think you need to have a little talk with history.

Re:Why I doubt driverless cars will ever happen

[VAXcat]

We already have flying cars...they are called helicopters.

Re:Why I doubt driverless cars will ever happen

[mcmonkey]

No, but I can imagine a change to the legal system limiting the liability of the manufacturers of self-driving cars.

If we could know that self-driving cars reduce accidents by 95% (a not unrealistic amount), it would be morally wrong for us to not put them on the road. If the only hurdle the manufacturers had left was the liability issue, then it would be morally wrong for Congress to not change the laws.

Of course, Congress has been morally bankrupt since, oh, about 1789, so I doubt that they'll see this as an imperative. On the other hand, I do imagine the car makers paying lobbyists and making campaign contributions to ensure that self-driving car manufacturers are exempted from these lawsuits, so it could still happen.

Wow. You are nuts. Not only will computer-controlled cars not reduce accidents by 95%, but overall damage from accidents will increase.

The number of accidents will go down, but many of those are minor--such as backing out of a parking spot and tapping the car behind you. Pretty all minor accidents may be eliminated. But in their place we'll see more serious accidents. So instead of one person hitting the gas instead of the break, you'll have every car of a particular make or model hitting the gas instead of the break.

This idea that the perfect designer or perfect programmer is just waiting for Detroit or Google to come calling is fantasy. We don't have the perfect driver, so why do you expect we'd be able to program one?

And the idea that manufacturers should have special liability protection is fascism. I go out and break the law, I go to jail. If someone suspects I've harmed them, I--as an individual--am open to a civil suit. But if I'm acting on behalf of the McMonkey Motors Co, suddenly I'm above the law?

I agree the US Congress is morally bankrupt, but precisely because they've been passing the type of law you suggest.

I see nothing fundamental in the concept of the computer-driven car that requires a change in law for dealing with faulty products.

Re:Why I doubt driverless cars will ever happen

[Archangel Michael]

Big Difference, as Planes do not have to dodge balls, steer clear of moving obstacles, avoid collisions with RCKRLL who is sending and recieving text messages at a rate of 30 per minute, while driving 50% over the speed limit with a BAC of .06.

That being said, I'd rather be on the road with driverless cars than the idiots I have to dodge daily.

Re:Why I doubt driverless cars will ever happen

[gr8_phk]

The question will be "which company DOESN'T charge a fee for a mandatory yearly check-up"?

Annual checkups will just increase the liability - yes your honor, this car was re-certified by the manufacturer just 3 months ago...

Even when the cars crash 1/10 as often as humans all of the incidents will be blamed on the manufacturer. Even this fine article asks the question "If your driverless car is about to crash into a bus, should it veer off a bridge?" My response: Why the fuck would my driverless car be about to crash into a presumably driverless bus? And given that something in the system or software has already gone horribly wrong, why do they even ask the question of what action to take next - the system ALREADY FAILED.

Re:Why I doubt driverless cars will ever happen

[Sez Zero]

Hit the bus!

Why would you ever veer off a bridge, causing structure damage? A bus hitting a car will most likely just cause vehicle damage, and some people will be inconvenienced. That's what you get for riding the bus.

Why wouldn't the driverless bus and driverless car stop and avoid the situation completely? Sounds more like a decision problem, not a morality problem.

Re:Why I doubt driverless cars will ever happen

[JDG1980]

I maintain that you CAN'T really program morality into a machine (it's hard enough to program it into a human). And I also doubt that engineers will ever really be able to overcome the numerous technical issues involved with driverless cars. But above these two problems, far and away above *all* problems with driverless cars is the real reason I think we'll never see anything more than driver *assisting* cars on the road: legal liability.

What you're overlooking is that there are some very big companies who want to introduce driverless cars, and those companies are powerful enough to change tort law if they need to – especially as this is uncharted legal waters.

That's probably not how things should be run in what is supposed to be a democratic republic, but the fact is that if big corporations want the law to be a certain way, and there aren't other big corporations or major grassroots movements who oppose it, then that's the way the law is going to be. And small-time trial lawyers who handle car wrecks for a living aren't going to outcompete Google and Volkswagen at the statehouse or Congress.

Re:Why I doubt driverless cars will ever happen

[dgatwood]

The problem with that is that not all check engine problems require immediate attention, yet if given the opportunity, many manufacturers might choose to brick their car this for every little problem, urgent or not, so that they can make more money off of unnecessary service work.

For example, a check engine light caused by a lean condition that's just slightly out of spec, for example, will cause no detectable drivability issues, and at worst, increases your NOX emissions slightly. However, a lean condition beyond some much more severe threshold could cause your car to stall. Choosing that threshold is critical.

Likewise, a transmission warning light caused by a bad solenoid can cause you to be missing overdrive and spend more fuel, but you could drive that way for years without a problem. And I'm not entirely sure if the car even knows whether the vehicle failed to shift because of a stuck solenoid (probably harmless), a slipping band (mostly harmless), or a broken band (which could cause a catastrophic transmission failure under the right circumstances).

Similarly, a failure of the brakes (caused by loss of fluid or insufficient pad thickness, is a serious problem, and should cause the car to be limited to slow speeds until the problem is resolved. A failure of the ABS unit, however, is not a serious problem, and should not. A stuck caliper that causes slight disc warping or pad squealing typically is not. However, a stuck caliper that results in significant braking continuously while driving could cause brake failure, particularly if you're dealing with drum brakes.

And so on. So there would need to be a strict set of government standards that defines what is or is not truly a safety issue, and standards that define where various thresholds should be set. That's a nontrivial heap of standards. This is not to say that it shouldn't be done, but rather that it needs to be approached carefully, methodically, and with an eye towards the complete picture.

Also, completely shutting down the vehicle is a very bad idea that could leave someone stranded in the middle of nowhere, and could actually put someone's life at greater risk than driving under certain circumstances. It should instead put the vehicle into a low-speed "limp mode" so that the user won't be willing to drive it much farther than the nearest repair place, but where the user at least has the ability to get it to the repair place without calling a tow truck. :-)

Re:Why I doubt driverless cars will ever happen

[greenbird]

There is the very real potential for thousands, or even millions of cars to all crash _simultaneously_. Imagine everyone on the freeway simply veering left all the sudden. That should be the manufacturers largest fear. Crashes one at a time can be litigated and explained away, the business can go on. The first car company that crashes a few thousands cars all at the same time in response to some input will be out of business in a New York minute.

One word. Aircraft.

They've been pretty much flying themselves for around a decade. You know what's happened to air safety over that period? Do you know how much more difficult it is to fly than it is to drive? Yeah, yeah. You're gonna say not much traffic in the sky. I say BS to that. More traffic or less traffic flying is a LOT more difficult that driving.

Driverless cars are merely a matter of time. All that money that goes to the car insurance industry can be used to support liability issues just like it is now. It's merely a matter of the liability being shifted somewhere it would be much more efficiently handled. Cost is the driving factor. I'd see a car manufacturer having a single driving system with the same set of sensors in all there cars. Set a few parameters (center of gravity, weight, type of tire...) and the software is all set for a new car model.

Re:Why I doubt driverless cars will ever happen

[Zalbik]

+1. Exactly.

Trying to impose today's liability standards on what is a fundamentally new technology is like worrying about how traffic at intersections would be handled prior to the introduction of automobiles. New technologies may introduce changes to our social norms. This has happened frequently in the past, and I can't see why this technology would be any different.

Re:Why I doubt driverless cars will ever happen

[Zalbik]

Easy solution: Manufacturer's lease the cars to you, and the lease price includes the cost of liability insurance.

As these cars would likely be involved in fewer accidents than occurs currently, the insurance cost should be less than what you pay right now.

A couple of points your argument misses:
1) You conflate the % of accidents that occur with human drivers with those that occur with self-driving cars.

2) You miss the point that right now accident liability costs are covered. There are whole reams of lawyers who specialize in prosecuting/defending people due to car accidents, yet somehow society manages these costs. Changing who pays for the liability shouldn't suddenly make the whole thing unaffordable (especially given that the number of claims should go down).

Re:Why I doubt driverless cars will ever happen

[plover]

Wow. You are nuts. Not only will computer-controlled cars not reduce accidents by 95%, but overall damage from accidents will increase.

The number of accidents will go down, but many of those are minor--such as backing out of a parking spot and tapping the car behind you. Pretty all minor accidents may be eliminated. But in their place we'll see more serious accidents. So instead of one person hitting the gas instead of the break, you'll have every car of a particular make or model hitting the gas instead of the break.

[Thanks for the ad hominem attack.]

You are imagining an awful lot of things there, things I didn't say, things about bad implementations that should never see the light of day, and you are forgetting the most crucial element: the standard of comparison is an Average American Driver. Yes, those idiots who eat their donuts and drink their coffee and shave their faces and read their newspapers and apply lipstick and mascara and text their spouses lies about their whereabouts while sitting behind the wheel of their two ton vehicles traveling at 65 MPH. And those are over and above all the drunk, stoned, and just plain stupid drivers out there.

I would drive in the middle of a crowd of robot-controlled cars 7 days a week compared to sharing the roads with those selfish fools. And I would be the one putting the occupants of those robotic cars at the most risk, because I could sneeze or snooze or have a heart attack or a seizure. Humans may be far more capable and flexible, but we are more fragile, a lot slower, and a whole lot less attentive than an automated system. Assuming we would see a 95% reduction in serious accidents would be an unfair comparison -- to the robots.

Now, I don't know you. For all I know, you may have godlike driving abilities, reflexes faster than Jeff Gordon, and you may live and breathe in a hyperaware state that NoDoz users can only envy. Fine. I will trust that you are way above average, and that you are in fact the guy who taught Jason Statham's stunt driver everything he knows. Now imagine all those lesser beings out there, threatening you with their very existence. Would you rather share a road with Sally Soccermom who is racing home at 90MPH because her Precious Little Jimmy got beat up on the playground and has a nosebleed? Do you want to be stuck next to Cathy Cougar who just found out her husband has been cheating on her, and she can't wait to get home and beat him with his damn golf clubs? Or do you want to drive next to a fleet of programmed automatons that aren't going to change lanes without signalling 100 yards in advance, that aren't going to slow down because there's a pretty bird on a signpost and its a lovely Tuesday afternoon, and that aren't going to be mad at their ex- for not paying the child support this month?

Re:Why I doubt driverless cars will ever happen

[bacon.frankfurter]

Meh. Companies already face this. If any one of the thousands of parts in your car fails and causes an accident, the manufacturer can [...] get sued. Ask Toyota or Firestone how that plays out. All we're talking about here is another new part.

Okay, whatever guy.

The parts that fail now, in plain old dumb cars don't derive their autonomy from lidar, RFID, or 4G cellular radio transmissions or (god forbid) Wi-Fi (or the future equivalent).

When a spring or a bolt, or a seatbelt fails, it fails on that individual car. Even modern electronic systems fail in isolation. While many cars may have the same defect, and be prone to malfunctioning in the same manner, when Cruise Control in one car fails, it will never tell another car to travel at the same speed. The Toyota acceleration problem while affecting many cars, happened one car at a time. But guided cars are different. Depending on design and features implemented, one car could, in theory, affect multiple other guidance systems in other cars not even produced by the same maker.

When we engineer autonomous highway systems, with preset mandatory speeds of exactly 100KPH, let's say (in a future where carbon footprints are also standardized and enforcible by law, since an autonomous system is "perfect" and can do this) that cars also have a drafting algorithm to enhance fuel efficiency, and automatically organize into gaggles and formations, and communicate anticipated route information in situ, while updating eachother, so that members can exit and leave the formation, and optimal wind resistance can be controlled to save fuel.

Suddenly, a malfunction in such a system could throw many lives into peril, or maybe severly inconvenience people by travelling far off course unexpectedly, because of a software bug.

No company faces a reality such as this. Autonomy is different.

Re:Why I doubt driverless cars will ever happen

[gr8_phk]

The funny thing is that most of the time you are in an airplane the autopilot(aka george) is in control.

I'm glad you bring that up - see the recent Air France accident. The plane was flying itself when the air-speed sensor froze, so it gave control back to a human who then crashed the plane (part due to poor training, low experience, and the unavailable speed reading). Lets note that when your power steering or brakes go out on your car today they fail-safe and return to the un-assisted mode of operation. You can still steer, you can still stop, it's just harder to do and you're already at the controls. When the car is driving, those same failures will need to shut off the automatic driver and put a human in control. A human who (in your world) has so little experience driving we may as well say a crash occurs. So to avoid the problem of handing over control to a human, they'll have to implement fully redundant control systems on such vehicles. The cost of that is going to be quite large. And that doesn't address incidents where a properly functioning vehicle is at fault.

Re:Why I doubt driverless cars will ever happen

[Cro Magnon]

But not so short that your car stops in the middle of the road.

Re:Why I doubt driverless cars will ever happen

[snadrus]

Like early auto-pilots that required huge safe distances & cautions, driverless cars initially will be similar since safety is far more important than expedient travel. It's likely travel will be so slow and it'll let so many cars cut them off that taking a bus is faster.

Re:Why I doubt driverless cars will ever happen

[icebike]

the standard of comparison is an Average American Driver. Yes, those idiots who eat their donuts and drink their coffee and shave their faces and read their newspapers and apply lipstick and mascara and text their spouses lies about their whereabouts while sitting behind the wheel of their two ton vehicles traveling at 65 MPH.

[Thanks for the ad hominem attack.]

Re:Why I doubt driverless cars will ever happen

[tacokill]

Right. So because they already face this risk......you think they'll be happy to accept MORE risk of liability? That is not realistic. Companies won't participate if there is unlimited liability. They participate now because it's NOT unlimited liability. ie: they have a fighting chance in court and do win consistently.

No company in their right mind would venture into the waters without changes in liability law. The risk/reward is not worth it if you are sued out of business for every transgression that occurs with your product.

For anyone in business, this is no-brainer and an easily identified risk. If you don't see the problem then you are not looking at it close enough (or don't understand "how the world works")

Re:Why I doubt driverless cars will ever happen

[An Ominous Coward]

Same reason pacemakers don't exist.

Re:Why I doubt driverless cars will ever happen

[gestalt_n_pepper]

And here I thought it was turtles all the way down.

Re:Why I doubt driverless cars will ever happen

[dgatwood]

Even this fine article asks the question "If your driverless car is about to crash into a bus, should it veer off a bridge?" ... why do they even ask the question of what action to take next - the system ALREADY FAILED.

Moreover, it has an easy answer. If there's enough time for both vehicles to stop, both vehicles put on their brakes. If not, both vehicles brake as hard as they can and count on the passive safety systems to do their job to the maximum extent possible.

It might also be worth considering a J-turn for one or both vehicles, then flooring it. I'm not sure which is greater: the rolling resistance of a wheel under braking or the sliding resistance of a wheel spinning the opposite direction. Either way, your transmission is likely to be shot after doing this, not to mention your tire tread, so such a technique would need to be reserved for only the most extreme dangers, assuming that it proves to be more efficient than braking (and if it doesn't, then you'd never do this).

Re:Why I doubt driverless cars will ever happen

[s.petry]

The problem with that is that not all check engine problems require immediate attention, yet if given the opportunity, many manufacturers might choose to brick their car this for every little problem, urgent or not, so that they can make more money off of unnecessary service work

This would lead people to never purchase one, and instead purchase some form of transportation they could control. If it was Government mandated, it would end up as a prohibition on driver operated cars. We know how well prohibitions work right?

Re:Why I doubt driverless cars will ever happen

[dgatwood]

If they repost this, does that count as one of the hoaxes? :-)

Also, can we have a similar rule for a**hole bicyclists who flagrantly ignore traffic rules? And maybe an automatic door-open rule for motorcyclists who are splitting lanes because the speed limit isn't fast enough for them? Thanks.

Re:Why I doubt driverless cars will ever happen

[Sketchly]

"If there was a government requirement that safety related problems that are detected must shut down the car and immobilize it in no more than 5 minutes..." 5 minutes is a fairly long distance at 90 mph, with plenty of things to collide with along the way.

Re:Why I doubt driverless cars will ever happen

[raygundan]

Can you be sure the computer will handle all possible inputs correctly?

Of course not. If we get serious about licensing and permitting these vehicles, I suspect the standard will be to compare them with the vast body of statistics we have from human drivers. As long as a company's cars are averaging fewer accidents per mile than humans do, it would be hard to argue that they're not safer, even if they still get in some accidents.

People are terrible in all the ways you mention above and then some. Strokes, seizures, heart attacks, sneezing, blinking, stray eyelashes, muscle cramps, and aural migraines are just a few of the hardware failures that already cause plenty of accidents-- and that's before we get into the self-inflicted things like drinking, fatigue, and distraction. Toss in our limited sensorium, narrow field of view, crap night vision, and sluggish reflex loop, and we're really not looking good compared to hardware-- even if that hardware still has some failures and makes some mistakes. And as you point out, it certainly will. The only really important question is "does it fail less than people?"

Re:Why I doubt driverless cars will ever happen

[commander_gallium]

You're very clever, young man, very clever, but it's error lights all the way down!

Re:Why I doubt driverless cars will ever happen

[raygundan]

Great. Now all you have to do is prove your system wasn't at fault in a court of law--against the sweet old lady who's suing, with the driver testifying that it was your system and not him that caused the accident, and a jury that hates big corporations.

And you're a corporation who builds a car with a 360-degree lidar, radar, video, and audio-capture system, GPS data, and a log of vehicle telemetry. It's a hell of a lot harder for a little old lady to cry her way out of something when there's hi-resolution panoramic 3D recordings of the entire event.

Re:Why I doubt driverless cars will ever happen

[Dr Max]

That's why you get insurance and build it into the price of the car. It's what happens when another appliance or tool takes a persons life (remember the quote from fight club about car manufacturer caused incidents "if number of incidents = x and average court settlement = y then if x * y 'price of recall' they don't do a recall). You have a valid point though, it will be the main hurdle; However, If we had everyone driving robot cars there would be a lot less deaths than there are now, i find it funny that, that is the reason progress will be slowed. As far as morality goes a simple rough calculation of sparing the most humans wouldn't be that hard.

Re:Why I doubt driverless cars will ever happen

[geekoid]

Of course you can... or will.
Morality isn't some magic box. It's a series of chemicals responding to social pressure and conditioning.

"Again, what car company wouldn't take that into account when asking themselves if they want to be a pioneer in this field?"
cars companies that want to make money.
I have heard similar complaints about seat belts, anti-lock brakes, air bags, automatic breaking.
Right now, you can buy a car that will park for you, brakes for you, even safely follow the car in front of you.

". So let's say that a good 25% of those first "
why? The number ONE(1) reason for accident is that the driver isn't paying attention. Either distracted, or just zoned out.
Something that doesn't happen to automated systems.

The item you bring up have been dealt with and the insurance companies know how to handle it.

Re:Why I doubt driverless cars will ever happen

[geekoid]

That's you reason? Not the fact that anti gravity is impossible? Or the amount of energy to lift a vehicle vs. just driving it some place?

If flying cars become common place a'la '5th Element' then they will be able to be easy to maintain.

Re:Why I doubt driverless cars will ever happen

[geekoid]

Try driving one down the freeway.

Re:Why I doubt driverless cars will ever happen

[mug funky]

the flipside is self-driving cars have a multitude of sensors and telemetry (if i'm using that word correctly). it can log events in a way that a human cannot be trusted to.

the first court case will be interesting - some driver will have done the wrong thing, and the driverless system will be blamed. the driverless system's logs will be admitted in evidence and the driver will be shown to be full of shit.

the other way to shift liability (necessarily) away from manufacturers is a regulatory framework. this can take some years to implement, but means that the state (who also runs the courts) will take ultimate responsibility provided the manufacturers meet the requirements.

easy done.

Re:Why I doubt driverless cars will ever happen

[geekoid]

" the system ALREADY FAILED."
no it hasn't.
One time, I saw a plane crash onto the freeway. complete nose drive. If memory serves it was in '86 on the 10.
SO there will be unaccountable incidents.
OR maybe the bus isn't automated but your car is. Say during a transition period.
Or their was a mechanical failure with the bus. That doesn't mean Your system failed.

Of course, the question is silly becasue all you car should do is brake and hit the bus. The mass difference means the bus will be fine, and in all likelihood you will be fine too.

.

Re:Why I doubt driverless cars will ever happen

[rk]

We're grownups here and understand that this place can run blue at times. You don't have to censor yourself.

Re:Why I doubt driverless cars will ever happen

[geekoid]

" Have you seen how stupid people drive these days?"
it's actually better. But you go on about the good ol' days.

There are fewer accidents then ever. And then number keeps declining.

" They race from red light to red light as if they're actually going to get there faster that way."
yeah, these days, that certainly didn't happen 10, or 20, or 30 years ago~
I've only been driving for around 30 years I suspect peoplewere doing that before I was driving.

"the idiot driver thought the car was in reverse"
also been happening, rarely, for decades.

Come on man, don't fall into that trap.

For the record, I am a huge fan of automated driving systems. When I am older I won't have to depend on others to get around. My car will do it for me.

Re: Why I doubt driverless cars will ever happen

[Epsilon Crucis]

One option is that the state takes on some of the liability to encourage driverless cars. Different countries will have varying appetites for such an approach. However, if we take into account the probable reduction in car accidents, increased usage of driverless cars could result in very significant savings to the public health system and other government programs especially in regards to ongoing costs associated with disabilities caused by accidents. Furthermore, networked driverless cars have the potential to significantly improve the efficiency of road usage. This will result in less need to build new roads or widen existing ones as we will be able to move more vehicles per hour through existing roads. Together these benefits will save governments billions of dollars. Some governments may therefore be wiling to take on the financial costs associated with liability, though they would probably limit the liability as others have suggested. The flaw in my argument is that it is more likely to make financial sense if all cars are driverless. During the initial period when a small minority are driverless the numbers will not justify the approach. Unless a government is willing to take on a large financial risk for something that is not going to pay off for many election cycles, it is hard to see this happening in the short term.

Re:Why I doubt driverless cars will ever happen

[cas2000]

a mis-analysis of the data occurred so that for example little Suzy playing in the street was mis-identified as a racoon and classified as an "acceptable casualty" in damage mitigation.

even this scenario isn't very likely - it's not hard to imagine small transmitters (wearable, or embedded in children's clothing) to make it easier for automated cars to identify children.

i.e. an electronic version of a bright, reflective safety vest.

some parents already use GPS tracking and similar devices to keep track of their kids, and that's far more invasive of the kids' privacy.

these transmitters would probably be popular with adult pedestrians and cyclists too.

of course, kids being kids, they'll probably exploit their transmitters and invent new games like "fuck up the traffic flow".

it would probably make carjacking easier too - but a self-driving car could be remotely instructed to stop or drive to the nearest police station.....or just do that automatically if a specific RFID tag isn't within a few hundred metres.

Re:Why I doubt driverless cars will ever happen

[mug funky]

cause Jake leg was a picnic.

Re:Why I doubt driverless cars will ever happen

[geekoid]

You should try some real world industrial programming. These problems have either been completely dealt with, or are so rare they are far better then human operators.

"..glitch due to after-market modifications.."
No, we can't just like you can't today. Modifying components of cars is as old as cars.
Everything from throttle adjustments, to carb adjustments, to nitro, to cars that jump up and down, to CPU modifications.

Some how, will manage.

Re:Why I doubt driverless cars will ever happen

[mug funky]

i drive manual, you insensitive clod!

Re:Why I doubt driverless cars will ever happen

[geekoid]

Morality has nothing to do with your mythical soul.

If we did program morality, itr would reflects are, so in all likelyhood it would be the child first.

Unless I program it, then it will be:
Is it my family?
Is it me?
then the rest~

Re:Why I doubt driverless cars will ever happen

[geekoid]

" You can't program morality into a person,"
wrong.
Society, and response to society create morals.

"duuurr I don't understand something, therefore no one can and it's impossible to understand. derp.. derp.."

Do you see what you sound like?

Re:Why I doubt driverless cars will ever happen

[geekoid]

"Totally agree."

I agree

" Computers can no more have morals than they can have intelligence. "
wrong. machines have intelligence. In fact there are many machines that if you just monitored it's action without knowing it's a machine you would think it was exhibiting intelligence. Primitive, but intelligence none the less.

AI is a moving target. So many things are done today where 20 years ago they would be considered AI.
But we figure it out, implement systems and then the bar of what is AI moves.
The more we understand the brain, the less magical it becomes. Some people can't handle that.
Interestingly enough, we have seen this before, but with the stomach. People used to consider it unknowable. Assigned it all kinds of properties, many could only be described as 'magical'.
The we figured it out .

If I have a computer that looks at a pendulum, and then through trial an error comes up with F=MA on it's own, is that intelligence?
What if you give it data for a biological process we don't understand, and it gives back a formula that has 100% predictive power? Even if we don't understand why the formula works? is that intelligence?

I ask because computers have done those very things.

Please try to understand what a rhetoric question is, cause that's a horrid one.

Also, you might want to try and stay on top of Computer decisions and AI instead of make stupid "rhetoric questions" about the topics. Or keep programming crappy Access application at the insurance company you work at, I don't really care.

Re:Why I doubt driverless cars will ever happen

[scamper_22]

It is not only technology that advances, but also legal systems, contracts,, business models.

Can driver-less cars be transplanted onto the automobile ecosystem we have today? Probably not.

Just off the top of my head... here's some ideas that could make it more viable.

1. Some arrangement is made between the automobile company and the an insurance company. If statistically, a driver less car is less likely to get into an accident, an insurance company would be willing to back it.

2. The government creates a law limiting or removing liability from the company. Many governments limit liability for nuclear power plants in such a manner. Typically, limiting liability comes with heavier regulation... more stringent testing...

3. A car company becomes an insurance company... so to drive the car, you pay a monthly fee. They insure you knowing it is their car doing the driving. They pool risk the same. ...

I don't know... just know lots of possibilities.

Re:Why I doubt driverless cars will ever happen

[gnasher719]

Well, the solution to liability is legal - grant immunity as long as the car performs above some safety standard on the whole, and that standard can be raised as the industry progresses. There is no reason that somebody should be punished for making a car 10X safer than any car on the road today.

The problem is just that many people are bad at maths. The idea is that driverless vehicles, implemented properly, should reduce the total number of accidents, and reduce the number of accidents at any severity level. That means that total amount of damage is reduced.

Car owners do already need liability insurance - you have insurance, so whenever damage is caused by your car, the insurance pays out, and your premium goes up. Usually the damage will be caused by the owner, through carelessness, or a momentary lapse. Sometimes it is caused by someone else, like a thief stealing your car. It happens. So with driverless cars, you keep paying the insurance, except the premium goes down, because there is on average less damage. And if your car causes an accident with damage, your insurance pays just as it does now.

Re:Why I doubt driverless cars will ever happen

[clintp]

I was more addressing the fact that the liability issues of building and "selling" a thinking machine have been considered before.

However, an Asimov robot can be manipulated into violating the Laws. For example the First Law could be voilated if a robot was convinced that a human really wasn't human (this worked on Solaria in "Robots and Empire" [?]) or by performing a seemingly innocuous action that led to to a human's injury ("The Naked Sun"). Also, Asimov robots were built with altered Laws to allow humans to perform potentially hazardous work ("Lost Little Robot") and child care (drawing a blank here, so that children could actively play and risk minor injury).

Re:Why I doubt driverless cars will ever happen

[mcrbids]

Your blood sucking horde of soulless lawyers can be set at bay with a simple law dictating that in the case where a driverless car can demonstrate an accident rate N% lower than comparable human drivers that the company cannot be found liable for driver mistakes. (they would still be liable for other manufacturing defects)

If Congress was smart, they'd pass a law that starts at 1%, and must improve by at least 2% per year up to 80% reduction. Of course, "pro" is the opposite of "con" so what's the opposite of progress?

Technology would meet this obligation easily, and Google cars can already best people if their reports are any indication.

Re:Why I doubt driverless cars will ever happen

[RubberDogBone]

Hell, just this morning on the news they showed a car crashing through a store, barely missing a toddler -- the idiot driver thought the car was in reverse. Had he been driving a computer-controlled car, that would have never happened.

If the store had installed barriers to PREVENT the car from getting inside the store, it never would have happened. And it can be argued that such barriers are a "really good idea" because they are cheap and also prevent smash and grab robberies. Why more stores don't use them, I don't know.

Many Frys Electronics stores go one better and use huge granite balls instead of mere steel poles. You can make a joke here if you want.

Re:Why I doubt driverless cars will ever happen

[RubberDogBone]

Humans don't cause all accidents and computers driving won't be able to prevent them, either.

If a deer jumps in front of a car and ends up as both a hood ornament and a bloody wrecking ball in what used to be the front seat, the resulting accident where you run into a tree is probably not the fault of the computer or the human driver.

There's a point where computers could be programmed to deal with sudden intrusions into the road, but likewise, deer have a tendency to swarm across a road with no warning, so the computer can only do so much before being overwhelmed. Likewise trees falling on the road, objects suddenly in the road, and so on.

Nothing about automated cars will change these things because the road is not a sealed environment. It IS the environment with everything loose in and on it. Cars tread there at their own peril.

Re:Why I doubt driverless cars will ever happen

[Immerman]

I shudder to think of such a future. True, some obsessive parents do such things, and I weep for their children. But making it the *expected* thing? That's a dismal path indeed.

Re:Why I doubt driverless cars will ever happen

[lennier]

Do you know how much more difficult it is to fly than it is to drive?

Good grief, you're serious, aren't you? Answer: About negative a million times.

Straight and level flight is so easy that purely mechanical autopilots have been doing it for exactly a hundred years. That's right, since 1912.. Landing, yes, that's harder. But we don't let computers do that unaided even now.

There simply is no equivalent of "straight and level flight" for a car. Even on an empty test track, you have to do realtime vision, constantly monitor speed and steering, read the white line, build a route map to a destination, and that's without even considering pedestrians and other road users. DARPA have been trying since the 1980s. It's no picnic, and that's why we're only just getting highly restricted demo vehicles now - and even then, that's by massive cheating using lidars and GPS .

Re:Why I doubt driverless cars will ever happen

[lennier]

You can program anything into a machine. Computers are easy to program.

You can? Excellent, you can start by implementing a winning algorithm for Go.

Then you can move on to a reliable CAPTCHA answerer and win the Loebner prize while you're at it.

From there, it should be easy to replace judges, juries and the legislature with a small Haskell script.

Now you can build a robot that implements morality. It's really just a small matter of programming - shouldn't take you more than half an hour tops.

Re:Why I doubt driverless cars will ever happen

[DavidClarkeHR]

The question will be "which company DOESN'T charge a fee for a mandatory yearly check-up"?

Annual checkups will just increase the liability - yes your honor, this car was re-certified by the manufacturer just 3 months ago...

You assume the manufacturer won't just de-certify the vehicle and force you to replace it... further adding to their bottom line.

Re:Why I doubt driverless cars will ever happen

[lennier]

If humans were easy to program, there would be no crime.

No crime perhaps, but plenty of show-stopping fatal bugs.

Which is pretty much what happened with totalitarian states in the 20th century. Turns out "programming humans" isn't just hard to do, it also introduces whole new classes of errors. Because suddenly the central bureaucracy office has to become a sort of steampunk multi-million-human-level artificial intelligence implemented entirely on sheets of paper and filing cabinets.

Turns out - rather obviously in retrospect, but it came as a surprise to many political theorists at the time - that imitating mass human problem-solving creativity without using any actual humans is a difficult thing to do.

It's just a pity that decades after the failure of Stalinism that we're still trying to do it today, only with "corporate ERP systems written in Java (plus many photocopiers and vice presidents in suits)" as the AI. And the results are just as impressive. See also: financial crash, climate change, peak oil, resource wars.

Re:Why I doubt driverless cars will ever happen

[Ultracrepidarian]

The government will simply grant them immunity from prosecution.

Re:Why I doubt driverless cars will ever happen

[AK Marc]

The problem with that is that not all check engine problems require immediate attention, yet if given the opportunity, many manufacturers might choose to brick their car this for every little problem, urgent or not, so that they can make more money off of unnecessary service work.

Hence why I indicated that they range from trivial to catestrophic. As to what the makers do, I expect they'd do what they do now. As permissive as possible without running afoul of the law. If you heard that BMWs would brick when the O2 sensor read a rich case (and rich was common when engine braking), but nobody else's car had that issue, you'd avoid them.

Also, completely shutting down the vehicle is a very bad idea that could leave someone stranded in the middle of nowhere, and could actually put someone's life at greater risk than driving under certain circumstances. It should instead put the vehicle into a low-speed "limp mode" so that the user won't be willing to drive it much farther than the nearest repair place, but where the user at least has the ability to get it to the repair place without calling a tow truck. :-)

I've heard such arguments, but there has never been a case I've ever heard of where that would have been the case, and "limp mode" would need to be much more "limpy" than any to date, because I know of people who have driven in limp mode for more than a year. I drove a car with no reverse and no 2nd gear (in a 3-speed auto) for multiple years. Both reverse and 2nd were "neutral" in that unrestricted revving would occur, and there was no connection to the wheels for supplying power. I was an unemployed high school student at the time and the car was my father's hand down.

Re:Why I doubt driverless cars will ever happen

[AK Marc]

We know how well prohibitions work right?

There's a prohibition on selling new cars without seatbelts. Please describe the problems with that issue.

Re:Why I doubt driverless cars will ever happen

[AK Marc]

And 10 seconds at 90 mph is a quarter mile, plenty of time to run over lots of people. But 5 minutes when you are crossing the Golden Gate bridge (or Brooklyn bridge) in heavy traffic may be insufficient to find a safe place to pull over. It was a random number to give an example, as if I didn't, people wouldn't understand it. Ot a 120 dB siren inside the car.

Re:Why I doubt driverless cars will ever happen

[AK Marc]

Morality is dependent on some "higher" power, such as a mythical soul. A computer couldn't be immoral without mythical "free will", because otherwise it's just a reflection on the morality or stupidity of the maker.

Re:Why I doubt driverless cars will ever happen

[AK Marc]

The trial lawyers didn't stop the cap on oil spill damages, so I have no reason to think they'd have any more success next time. When you keep the focus narrow enough, they don't object that loudly.

Re:Why I doubt driverless cars will ever happen

[drkim]

Answered:
"[The] first attempt, Samaritan I, had pushed itself overboard with great alacrity, but it had gone overboard to save anything which happened to be next to it on the raft, from seven stone of lima beans to twelve stone of wet seaweed. After many weeks of stubborn argument Macintosh had conceded that the lack of discrimination was unsatisfactory, and he had abandoned Samaritan I and developed Samaritan II, which would sacrifice itself only for an organism at least as complicated as itself.

The raft stopped, revolving slowly, a few inches above the water. "Drop it," cried Macintosh.

The raft hit the water with a sharp report. Sinson and Samaritan sat perfectly still. Gradually the raft settled in the water, until a thin tide began to wash over the top of it. At once Samaritan leaned forward and seized Sinson's head. In four neat movements it measured the size of his skull, then paused, computing. Then, with a decisive click, it rolled sideways off the raft and sank without hesitation to the bottom of the tank."

"The Tin Men"
Michael Frayn
1965

Re:Why I doubt driverless cars will ever happen

[drkim]

...They get enough headaches from faulty accelerators...

They could simply have the auto-drive function work like aircraft auto-pilots, where there is a human "driver-in-charge" that could take over for critical decisions.

Re:Why I doubt driverless cars will ever happen

[HiThere]

You are wrong. And the reason why is that trucks without truck drivers will be a lot cheaper to operate.

Yes, the companies and the lawyers will need to come to some agreements, and this may mean that the legislators need to pass new laws. But it will happen, and not slowly.

I will grant you that automated personal autos may be delayed for several customer acceptance reasons. But automated vehicles already exist, and will only become more wide-spread. (Currently I believe that most of them are fork-lifts operating within warehouses...a controlled environment that simplifies things. But time is moving on apace. And hiring drivers is costing companies lots of money.)

Re:Why I doubt driverless cars will ever happen

[HiThere]

Where I live everyone is required to have drivers insurance, so the insurance company would be footing the bill. They'd be quite happy with a lot fewer accidents to defend, and if the auto had a decent black-box, they could easily prove who was at fault in the court case. And they've got deep enough pockets to file an appeal if they run up against a prejudiced judge or jury.

Do you think that you won't be at fault if the robot is driving the car? I think you're wrong. The other guy would sue you, and it would be YOUR responsibility to get the manufacturer to pay your bills. OTOH, IANAL, so I could be wrong. But it's worked out that way in many other areas.

Re:Why I doubt driverless cars will ever happen

[anotherzeb]

A flying Johhnycab, maybe? I'd say that's better than a flying car in the 70s future sense, but agree that it won't be like we've thought of a flying car since they were first suggested. Then again, a lot of the future hasn't turned out like we've thought it might. One important holdup for flying cars will be a reliable source of energy - petrochemical fuels are getting too expensive for ground travel let alone a load of extra air travel as well (on the assumption that the flying car would use more fuel than the ground car for the same journey). Some people / authorities would be able to run them, but the future's looking more Blade Runner than Buck Rogers.

Re:Why I doubt driverless cars will ever happen

[SuricouRaven]

There are two important questions:
- Does it fail less than people?
- When it fails, who is liable?

From a manufacturer's perspective, a thousand accidents due to driver error are better than a single accident due to onboard computer error, because in the latter case they have to deal with big compensation payout and a lot of bad press. Manufacturers are happy to add things that assist a driver, but they aren't going to release a fully-automonous car until there is some law limiting their liability in the event, however unlikely, that it glitches and hits someone.

Re:Why I doubt driverless cars will ever happen

[AK Marc]

Only the very last part of the definition even hints at some “higher” power in that an action is judged,

Where do most people turn for definitions of "right" and "wrong"? I'd assert it would be a higher power. Where do people get value systems? Most religious people inherit them from their religion, and most people are religious. Looking at that definition, it seems to support my statement more than not.

No part of the definition requires a soul.

Does not require, but assumes.

Re:Why I doubt driverless cars will ever happen

[dgatwood]

You didn't park that car much, did you? :-D

Re:Why I doubt driverless cars will ever happen

[tehcyder]

Will computer control eliminate some of the stupid mistakes people make behind the wheel? Yes. Does that mean computers (on behalf of the people who program and build them) won't make mistakes of their own? Heck no.

But any computer mistakes will be correctable in the future, whereas people will always be prone to anger, tiredness, being distracted or whatever.

Re:Why I doubt driverless cars will ever happen

[tehcyder]

Also, can we have a similar rule for a**hole bicyclists who flagrantly ignore traffic rules? And maybe an automatic door-open rule for motorcyclists who are splitting lanes because the speed limit isn't fast enough for them? Thanks.

People like you are the reason why driverless cars are a good idea. Neither cyclists nor motorcyclists who break any rules of the road are any physical danger to you in your car, but you would quite happily kill them?

You're the fucking a**hole.

Re:Why I doubt driverless cars will ever happen

[tehcyder]

As you say, morality is relative; it's internal to the person. You can't program morality into a person, it's impossible.

Of course you can, it's mostly down to good parenting. With the exception of those who are born psychopaths, and will never learn to act morally, children are blank blobs of clay that get formed by those around them, and for the first crucial few years of their lives, that means the parents.

I agree that it's generally too late once they're adults.

Re:Why I doubt driverless cars will ever happen

[tehcyder]

The number of accidents will go down, but many of those are minor--such as backing out of a parking spot and tapping the car behind you. Pretty all minor accidents may be eliminated. But in their place we'll see more serious accidents. So instead of one person hitting the gas instead of the break, you'll have every car of a particular make or model hitting the gas instead of the break.

When a human driver does this and dies, there is no feedback. Everyone just says he was a shitty driver. If a computer-controlled car did it, it would be corrected and shouldn't happen in the future ever again. (Regardless of the fact that it seems remarkably unlikely to happen in the first place).

Most serious accidents are caused by driver error. You're never going to be able to do anything about situations where a tyre suddenly explodes or all the brake fluid suddenly gets lost through a catastrophic leak just when you're coming up to a stop sign and there's a thousand foot precipice in front of you. But people driving too fast because they're late for a meeting, cutting others up because they're annoyed and so on will be prevented entirely.

Re:Why I doubt driverless cars will ever happen

[tehcyder]

Now, I don't know you. For all I know, you may have godlike driving abilities, reflexes faster than Jeff Gordon, and you may live and breathe in a hyperaware state that NoDoz users can only envy. Fine. I will trust that you are way above average, and that you are in fact the guy who taught Jason Statham's stunt driver everything he knows.

From what I've seen here, the vast majority of slashdotters think they are above average in everything they do, whether it's coding, making coffee, investing on the stock market or driving, and shouldn't have to follow the same rules as the "sheeple".

Re:Why I doubt driverless cars will ever happen

[tehcyder]

Like early auto-pilots that required huge safe distances & cautions, driverless cars initially will be similar since safety is far more important than expedient travel. It's likely travel will be so slow and it'll let so many cars cut them off that taking a bus is faster.

That will be a problem when there are still lots of speeding human drivers. Once all cars are driverless, much higher average speeds will be achievable, as unlike many humans, computers can co-operate very nicely.

Re:Why I doubt driverless cars will ever happen

[tehcyder]

we are easily influenced but not easily programed, (that is without negative reinforcement, get a mild electric shock every time you answer wrong i bet you would learn whole lot faster.)

You'd just learn to hate the people giving you the shocks. It's all very well testing this on dogs, but we don't then release them and expect them to be nice human-friendly pets.

Re:Why I doubt driverless cars will ever happen

[tehcyder]

And I'm sad to say there are probably corporate tricks that can be played as well, where limited liability entities are used to compartmentalize. e.g. "FF201802, Inc" ends up taking the risks for all the Ford Focuses made in the second quarter of 2018 or something like that, and has few assets to lose.

Yes, that's why BP set up "Deepwater Horizon, Inc" and ended up paying no fines. Oh, wait...

Re:Why I doubt driverless cars will ever happen

[tehcyder]

lots of cars will give you an error light if a light is out...

Unless, of course, it's the error light that's out.

Yes, but if the chances of all your brake fluid suddenly falling out through a leak are (say) 1/10,000 and the chances of your brake fluid warning light failing are (say)1/1,000, then the chances of them both failing at the same time are 1/10,000,000. Made up numbers, but I think the point is valid.

Re:Why I doubt driverless cars will ever happen

[tehcyder]

Also, completely shutting down the vehicle is a very bad idea that could leave someone stranded in the middle of nowhere, and could actually put someone's life at greater risk than driving under certain circumstances. It should instead put the vehicle into a low-speed "limp mode" so that the user won't be willing to drive it much farther than the nearest repair place, but where the user at least has the ability to get it to the repair place without calling a tow truck. :-)

There is a big difference between something putting someone's life at risk, and slightly inconveniencing them by making them wait for roadside assistance.

My feeling would be that if you're ever in a situation where your vehicle breaking down could put your life in genuine danger, then you should make some sort of disaster preparation plan for that eventuality anyway. Because there could always be an accident that would leave your vehicle undriveable, irrespective of any automatic computer shut down (e.g. suddenly hitting an unseen pothole and breaking an axle).

Re:Why I doubt driverless cars will ever happen

[AmiMoJo]

I would never get in a Microsoft car considering their shoddy programming, but Microsoft would never manufacture a driverless car simply because of that.

It's never stopped them before. The British Navy still runs Windows for Warships.

Re:Why I doubt driverless cars will ever happen

[mcgrew]

Of course you can, it's mostly down to good parenting.

I'm not too sure of that, even though my kids turned out ok. One fellow I know has two adult sons, one an honest man with an honest job, the other a lazy, never employed thief and liar. I've known others like that as well. Of course, neither example is proof of anything.

Re:Why I doubt driverless cars will ever happen

[pnutjam]

So you are going to let a law requiring your car to disable itself if it thinks it is malfunctioning go on the books just so you can have your precious self driving car? I see it on the interstates as an option, but people still drive stick shifts. There will never be the necessary buy in from the public to make mandatory self driving cars a reality.

This appears to be our generations jetpacks and flying cars. It ain't happening, and if it does it is at least 80 to 100 years away.

Re:Why I doubt driverless cars will ever happen

[pnutjam]

...and how many cars are on the roads in the US? Wikipedia says over 254,000,000 so you are talking about at least 20 such incidents statistically.

Re:Why I doubt driverless cars will ever happen

[pnutjam]

I have over a year with more then 10k miles since my check engine light came on. I am driving a car that is over 15 years old and for most of last summer I had to add water to the radiator for every trip. I have since made some repairs but my tires are almost bald and my check engine light is on (bad oxygen sensor). The car runs fine and I drive it to work every day.
I am not the only one with a car like this. Not everyone drives a car made in the last 5 years and/or perfectly maintains their vehicle. I can get where I'm going and it fits my budget. I know how to keep this best running and I will continue to drive it for as long as I possibly can. (actually not a beast, small minivan, probably throw some decent used tires on now that it is winter)

Re:Why I doubt driverless cars will ever happen

[mcgrew]

Excellent, you can start by implementing a winning algorithm for Go.

I can't, since I've never even played it, but someone will. They thought no computer would beat a human chessmaster, but it was done. Then they thought a computer winning Jeapordy was undoable, but it was done as well.

Then you can move on to a reliable CAPTCHA answerer

Unbeatable capchas keep moving, to the point that it's hard for a human to read some capchas.

Now you can build a robot that implements morality.

Far easier than winning Jeopardy. The programmer simply puts the rules of his own morality into the machine. That's all computers do, execute math-based rules; if a=b then c. Do case; case a, case b, case c, endcase.

Re:Why I doubt driverless cars will ever happen

[mcgrew]

Ironically hilarious insult, since it's you who doesn't understand. Society, and response to society do not create morals; empathy does. Without empathy we would not be social creatures and would not have morals. And some people don't have morals; they're called sociopaths.

When you have to resort to insults, you just lost the argument.

Re:Why I doubt driverless cars will ever happen

[gr8_phk]

That plane crash is irrelevant - it was not in control so couldn't take action anyway.
In your later 2 scenarios, my cars job is to protect ME, not some other people with inferior or broken technology. So there is no moral dilemma.

Re:Why I doubt driverless cars will ever happen

[mcgrew]

Agreed about the barriers, but there was another this morning; a child was killed when a bus driver drove into a house after swerving to miss a pedestrian who jaywalked in front of him. Had a computer been driving, I doubt either the pedestrian or the house would have been hit.

Re:Why I doubt driverless cars will ever happen

[cayenne8]

If there was a government requirement that safety related problems that are detected must shut down the car and immobilize it in no more than 5 minutes, then the problem goes away.

Really?

Another (and this time pretty involved) Federal Govt. intrusion into my life and my possessions??

No thanks. And if the automated driving cars have to have this type mandate, I very much hope this does not occur in my lifetime.

And I don't need a machine deciding which is more important...MY life, or 2-3 other people's lives if that situation comes up.

I want to decide, and of course, "I" will always be the one I want to save at the expense of others if it comes down to it...especially if they were strangers.

Re:Why I doubt driverless cars will ever happen

[mcgrew]

Fatalities are down, but I can't find any data on accidents themselves; my googlefu is weak this morning. But you've been driving since 1980 so you've seen a few of the safety improvements I have (I've been driving since 1968).

When I started driving, there were few four lane divided highways, and head-on crashes were common. Far less common today. They were usually fatal; no cars had air bags and few even had seat belts. Cars had no crumple zones and the dashes were unpadded steel. There was no ABS. Cars today handle much better, stop FAR more quickly (drum brakes were crap!) and streets themselves are more intelligently designed. The one thing that made driving safer that didn't have to do with technology improvements was the push against drunk driving (and every drunk drives like an idiot, because he is an idiot).

I'd like to see a graph of accidents per capita, but I can't seem to find one. My guess would be that accidents per capita have gone up while the fatalities and injuries have gone down.

I suspect that people drive more insanely these days (or perhaps that it simply looks that way to me; I'd need to see numbers) is that there are so many more cars on the roads; roads were pretty empty back in 1968, except in the heart of a big city. You could drive down the highway for miles without encountering another car. People weren't as impatient as they are now because they didn't have to be. Folks drive like they're the only ones on the road, they used to BE the only ones on the road.

Re:Why I doubt driverless cars will ever happen

[mcmonkey]

You need to rethink this line of reasoning.

This:

Most serious accidents are caused by driver error.

Doesn't jive with this:

If a computer-controlled car did it, it would be corrected and shouldn't happen in the future ever again. (Regardless of the fact that it seems remarkably unlikely to happen in the first place).

What I think I'm reading in a lot of these comments is, people behind the wheel cannot be trusted. They are prone to mistakes. But people at a keyboard are perfect and produce perfect software that will drive those cars for us!

Why is it unlikely a computer-controlled car would make a mistake or get into an accident? Are you trying to claim it is unlikely a program will have bugs? Because the converse is true. It is a virtual certitude that the software controlling the driverless car will have bugs.

Don't you get it? The people who will be programming these computer-controller cars are the same people we don't want driving their own cars.

I am all for letting the technology assist me. I have the rear view camera on my car. I have the blind spot warning system. I am glad to have those tools assist me when I am driving. But I don't want the tools doing the driving.

I think of a Google search. I like the "did you mean B?" when I do a search for A. I hate the "we returned the results for B" when I searched for A. I know what I'm searching for better than the developers at Google.

Take the suggestion feature at your favorite online retailer as an example. Do you want them just automatically sending you whatever they think you'll like? I know I certainly don't. Yes, Amazon gives me some good suggestions, but most of their suggestions are things I'd hate to have them send me.

Ever used the suggestion feature from TiVo? It sucks. I mean it records some good stuff, and I've seen some stuff I never would have known existed without the suggestion, but 75% of the stuff my TiVo records I delete without ever watching. It does stupid things like records reruns for shows I've told TiVo I only want new episodes. The developers at TiVo don't know what I want to watch better than I do.

iTunes is another great example. I've had iTunes unsubscribe to podcasts without telling me, because I hadn't listened to that podcast in a while. Would you want your car doing that? 'I'm going to erase the roads that lead to grandma's house because you haven't visited in a while.' I know what I want to download better than the programmers at Apple.

So looking at Apple, Google, Amazon...some of the most successful companies in the world (of all companies, not just in the realm digital technology) with some of the best developers in the world. And I would not trust any of them driving my car.

The suggestions are great. They are helpful. Perhaps even insightful. But when providing assistance , not when taking control.

I don't want iTunes to only download podcasts based on what the software calculates I like. I don't want Amazon to only send me what their suggestion algorithm thinks I will want. I don't want my TiVo to record only it's suggestions.

Do you? Are you ready to hand over control to the machines? I say, let the machines assist us, but no more.

Re:Why I doubt driverless cars will ever happen

[mcgrew]

I see you've 1) never programmed, and 2) never seen Windows, or any other modern operating system.

I was programming in assembly thirty years ago. Yes, I've seen Windows, and Windows is why people are afraid of computer-driven cars.

By show of hands, who here knows a programmer who claims to make bug-free software? *sees many hands* OK. Now who here has actually seen perfect, bug-free code for any non-trivial task?

Ever seen a space launch? How many airplanes crash because of software bugs in their autopilot code? How's that LHC working, with all that buggy software?

The key is testing, code review, and quality control, which is nonexistant in most consumer-grade PC software; screw it, we can patch later is the mantra, and that's the only reason you see so little non-trivial software.

Any activity that people used to do that is now illegal because machines (designed and built by people) are so much better?

Not illegal, but no longer done. Welding and painting in automobile assembly lines, railroad spike driving machines, even computers -- a computer used to be a human being who worked out firing tables and scientific computations.

Are doctors forbidden to diagnose because now we have Watson?

Watson is no replacement for a doctor; that was an incredibly stupid question.

Re:Why I doubt driverless cars will ever happen

[dontfearthereaper]

I maintain that you CAN'T really program morality into a machine

Actually, you can. The real question is who's morality and ethics will be programmed into the machine, and who will set these standards of morality?

This is an extreme example, but what was considered moral to Adolf Hitler is not necessarily what is considered moral by myself, and what I consider moral may not necessarily be what is considered moral by anyone else. If you program the robot to avoid a collision at all times, being flung off a bridge or sent headlong into a tree is exactly the outcome you have programmed into the car. It's called unintended consequences, not to mention that a sensor may go bad or misread something and you suddenly get thrown into oncoming traffic at 70mph for no good reason. There's just too many things that can go wrong with machines to completely take the humanity out of operating dangerous and powerful machines.

Personally, automation to this extreme is a very bad idea. I would rather smash into the bus and take my chances then be forced to commit suicide by being thrown off a bridge by a robot, and I'm pretty sure most would agree with me.

There are, of course, those that would rather see someone thrown off a bridge than risk a few bumps and bruises themselves.

Re:Why I doubt driverless cars will ever happen

[mcgrew]

The liability will rest with the vehicle's owner, not the manufacturer -- unless, like today, there is a defect. And these defects happen all the time with physical parts; the Pinto, the Crown Vic, Toyota, etc.

They'll release a fully autonomous car when it's profitable to do so. You know why they kept the dangerous Pinto gas tank design? Because they calculated that the ten dollars per car it would cost to remedy the defect was cheaper than the settlements for the survivors of the people who burned alive. They have armys of accountants and lawyers, and they get their money's worth out of them.

It will happen, and not too far in the future.

Re:Why I doubt driverless cars will ever happen

[prisoner-of-enigma]

And maybe an automatic door-open rule for motorcyclists who are splitting lanes because the speed limit isn't fast enough for them?

Speaking as a motorcyclist, perhaps you are unaware that lane splitting is legal in many jurisdictions (or, at the very least, *not* illegal, as the law does not say you can't do it). If you're riding like an asshat doing 100mph in 70mph traffic then that's one thing; between karma and statistics you're likely going to get a violent comeuppance sooner or later all on your own. If you're lane splitting because of heavy afternoon traffic (i.e. you're doing 30mph while everyone else is sitting still) that's something else entirely. If you purposefully open your door to try and interfere with a rider, that's pretty much premeditated attempted murder.

Re:Why I doubt driverless cars will ever happen

[t1oracle]

Something Gingers don't have but like to steal...

Re:Why I doubt driverless cars will ever happen

[prisoner-of-enigma]

putting the car in neutral will slow you down in not so long of a distance.

Leaving the car in gear will result in a shorter stopping distance due to engine braking, moreso for manuals than automatics but either is preferable to coasting in neutral.

If you are really worried for your life or others, and don't care about the transmission, you can throw the car into park if you want to.

Every automatic transmission made in the last 40+ years is designed such that shifting into park while moving results in the transmission actually going into neutral. Mythbusters tested this premise in Episode 84. They also tried shifting into reverse using a manual transmission and found it's impossible to do; the reverse will not engage while moving forward. It just grinds and whines but will not engage catastrophically or otherwise, even if you disable the various mechanical, electrical, and hydraulic safeties designed to stop you from throwing it into reverse in the first place.

Re:Why I doubt driverless cars will ever happen

[s.petry]

You are comparing a cheap mechanical device to what I responded to? Really, you should read more carefully. The post I responded to was drastically different. Context is generally important in dialogue.

Re:Why I doubt driverless cars will ever happen

[mcmonkey]

Not illegal, but no longer done. Welding and painting in automobile assembly lines, railroad spike driving machines, even computers -- a computer used to be a human being who worked out firing tables and scientific computations.

Are doctors forbidden to diagnose because now we have Watson?

Watson is no replacement for a doctor; that was an incredibly stupid question.

You're making my point. The examples you give--auto assembly, driving spikes, calculations--are all deterministic activities that are relatively easy to reduce to repeatable actions performed by machine.

Traffic on the roads is a complex system with potentially chaotic behavior. There are posts suggesting we are heading to the point that driving your own car will be illegal because people will be a danger to computer-controlled vehicles. That we can program a computer to control a robotic arm to repeatably weld the same spot on an auto body does not suggest we are close programming a computer to drive a car in traffic.

As I said in another reply, are you ready to let Amazon send you what their suggestion system picks out for you? Only watch what TiVo records as a suggestion? Let iTunes pick what podcasts to download?

In my experience those systems are wrong more often than they are right, and I am not ready to substitute the result of the program for my own judgement. I am all for tools to help me--whether it's suggestions from a retailer or a blind spot warning on a car--but I am not ready to turn over complete control.

Re:Why I doubt driverless cars will ever happen

[AK Marc]

I think you are confused as to what I said. Just like manuals are still available, and you don't have to wear a seatbelt if your car is old enough (lots of grandfather clauses out there for lots of things, especially safety related), I never said anything that could be remotely construed as a requirement that *all* cars have such a constraint, nor that regular cars be blocked from sale. So feel free to decide for yourself. For most people, actual safety is more important than feeling like a big man flipping a paddle-shift that requests an automatic change for you (when there are hundreds of conditions on that, and it often won't do what you ask, but having the paddle to request a shift makes people like you feel better about driving an auto).

Re:Why I doubt driverless cars will ever happen

[AK Marc]

The tech to make a self-driving car that's safer than a human driven one is here today. It's a low bar, humans are horrible drivers. So, working on constraints on driverless cars to improve safety is a bad thing, according to you, because that improves the safety of everyone involved, and you don't like that.

You sound like the SUV driver. If everyone drove Honda Civics, we'd all be safer than if anyone (or everyone) drove an SUV. But if one person drives an SUV, they have a greater chance of harming everyone around them, and it's our God given right to harm our neighbors.

I never said anything about mandating self-driving cars. I should have just read that sentence and figured out you are a liar or unwilling to read the post you are responding to. Never mind.

Re:Why I doubt driverless cars will ever happen

[greenbird]

Good grief, you're serious, aren't you? Answer: About negative a million times.

Ummm...pot meet kettle.

I was actually more referring to cat III autoland rather than straight an level autopilots. Autoland is used when conditions are less than optimal and typically do better than the pilots can do under optimal conditions.

and even then, that's by massive cheating using lidars and GPS

That's not cheating. Those are the very technologies that are going make it practical. Or rather improved and cheaper computing making those technologies affordable is what is making it practical.

Re:Why I doubt driverless cars will ever happen

[AK Marc]

I'd circle lots to find places where I could pull through two spots to be able to pull out forward. Also, I'd park uphill nose-in where possible, and gravity would pull me out. Luckily, there was such a hill at school. I drove it to school for a couple years. It was an '81 Accord. I got it when my father drove it until the motor siezed (it lost coolant, and he nevre looked at the dash, so he only pulled over when the enngine died because the pistons heat-expanded enough that they wouldn't move in the cylinders.

I drove it for two years with the transmission problems and one out of 4 chambers with no compression (I had to add about 1 quart a week of oil to keep it topped up). Before the reverse went completely, it used to catch intermittently. One time we were stuck and couldn't get out of a parking spot. He asked me to get out and push. I was, and it wasn't moving (he parked nose-down a hill). He got out and helped push. Then reverse caught as we got it out of the spot, as he left it in reverse when he got out. He had to chase the car down when it took off. That car was lots of fun. I had to run it with the heat on all the time. In Texas. In the summer. 110 outside and the heat on full blast. Eventually, it blew the engine completely and was retired.

Re:Why I doubt driverless cars will ever happen

[curtwelch]

The moral decisions have to be addressed, but for cars, it's really not a hard problem. The one rule of "don't hit anything" is all you need to program into the system. The current self driving cars are very good at following that rule - much better than any human driver so accidents rates and fatalities are expected to go way down as we switch to self driving cars. The car companies are just as likely to find themselves in legal trouble for selling a car with no self-driving features once the drunk owner ends up killing someone. The real hurdle will be public acceptance of the cars, not the technology, not the cost, and not the legal issues. People won't at first trust the cars, even if they are far safer. But in time, and with experience, the public will come to accept them and even demand them.

Re:Why I doubt driverless cars will ever happen

[xtstorm]

Many things today are quite safe, yet the manufacturer's of those things get sued. Late-night TV is filled with ads for law firms looking for patients harmed by medical procedures or medications. Medical procedures carry inherent risks that the recipient acknowledges in their consent for a procedures. Medication inserts document that x% of the test population experienced negative side effects from taking this medication. Yet successful lawsuits happen regularly. The reason is the lawyers don't argue whether the procedure was risky but whether the pharma companies or surgeon did everything possible to mitigate and minimize that risk. While I understand the "computers would perform better than people" from a practical point of view, the standard that will be used in the court of law is whether the programmer exercised sufficient due diligence to insure the mistake the computer made could have been avoided. You can testify on the stand that the car may perform better than a human in a similar case. The lawyer will respond that we are not talking about how the car performed but how the programmer performed. Would a different programmer have written software that would have avoided the accident? If so, why didn't you hire that programmer and spend the money to avoid my client's injuries. You are just an evil penny-pinching corporation recklessly disregarding my client's safety! Your computer software is the modern equivalent of the Ford Pinto! That the lawsuit that car manufacturers fear because that is the one that they will face in court.

Re:Why I doubt driverless cars will ever happen

[xtstorm]

My response is that aircraft companies get sued for failures of the their automated systems. The Quantas Air Flight 72 is a good example of this. Millions were paid out in damages from Airbus and Northrup Grumman. Keep in mind that the system that failed "had only failed twice in 3 seconds despite 128 million hours of successful operation" according to one aviation source. As well, passengers were partially responsible for their injuries because they were advised to keep their seat belts fastened. None of that stopped a successful law suit in Australia follow by a second lawsuit in the US for passengers that wanted a larger payout. Extrapolate that to volume of potential of "3 second failures" in driverless autos and that's a huge liability.

Re:Why I doubt driverless cars will ever happen

[xtstorm]

There was also the effect of "alternate law" that the computer system switched to in light of the pilot's actions. This shifted how the plane would respond to inputs that the pilot was making. As well, Airbus aircraft average the pilot inputs, i.e. if the captain pulls up on the yoke while the co-pilot pushes down on the yoke, the system will average the inputs and fly level (an oversimplification admittedly but I'm using it just to show the human factors problems). Other manufacturers of aircraft elect to have differing inputs provide feedback -- i.e. if the captain it pulling back and the co-pilot is pushing down, they feel the opposite inputs in the control yoke.) The other crew didn't realize until it was too late that co-pilot was in a climb stalling the aircraft because the software didn't provide that feedback. Clearly, if that same inexperienced crew were flying a different aircraft, the crash **might** have been avoided because the software would've have handled the inputs differently. So how much did poor software design contribute to the accident?

Re:Why I doubt driverless cars will ever happen

[iMactheKnife]

Ford Sync in my new car freezes regularly (thanks Microsoft) and refuses to accept a perfectly ethical command like "Damn Bill Gates to Hell."

Now I have to contemplate the revenge of Gates or his successor who can order the car to damn me to Hell?

Re:Why I doubt driverless cars will ever happen

[greenbird]

Extrapolate that to volume of potential of "3 second failures" in driverless autos and that's a huge liability.

I wasn't in any way denying there would be liability issues. There's massive liability issues now that cost hundreds of millions in the form of idiot drivers and idiotic law suits. What I was trying to point out was that individuals wouldn't need insurance anymore. That insurance burden would be carried by manufacturers and would be rolled into the cost of the car. Those cost should be significantly less than they are right now due to the better safety of the cars, economy of scale and what should be much more cut and dry determinations of cause (fewer expensive law suits). A driverless car would have a detailed log of what occurred causing an accident just like aircraft. Add to that the indirect cost savings in increased fuel efficiency of an auto driven car and much reduced traffic problems that are massively increased by idiot drivers and collisions and I believe you'd find a massive economic savings in driverless cars.

Re:Why I doubt driverless cars will ever happen

[pnutjam]

Several people were talking of mandating cars, I apologize if that is not your position. I have no problem with self driving cars, I would love a self driving car, but I think there are many problems that need to be addressed. I also think that people are too quick to give up their rights for a little fleeting safety or convenience.

Re:Why I doubt driverless cars will ever happen

[Rich0]

That would work as long as liability is associated with the owner, not the manufacturer. Otherwise it buys you nothing, as mandatory insurance for the manufacturer only costs them more than liability would, as insurance increase cost in exchange for balancing risk, and at a manufacturer's scale risk is already balanced.

The issue is the US tort system. If two people get in a crash the bill comes to thousands of dollars, or maybe $100k or so if there is injury involved. If you get a manufacturer involved then suddenly people are talking about millions of dollars.

Re:Why I doubt driverless cars will ever happen

[mcgrew]

You're making my point. The examples you give--auto assembly, driving spikes, calculations--are all deterministic activities that are relatively easy to reduce to repeatable actions performed by machine.

So is driving. As with the other automations, machines do the job better than humans.

Traffic on the roads is a complex system with potentially chaotic behavior.

Humans don't cope too well with chaos. Humans freak out, computers don't. Computers simply follow pre-defined rules.

There are posts suggesting we are heading to the point that driving your own car will be illegal because people will be a danger to computer-controlled vehicles.

I think it will be after my time, but it ill come. Not to protect computer-controlled vehicles, but to protect the human passengers.

That we can program a computer to control a robotic arm to repeatably weld the same spot on an auto body does not suggest we are close programming a computer to drive a car in traffic.

It's already been accomplished with Google Car. Computers have driven those cars hundreds of thousands of miles without a single incident.

As I said in another reply, are you ready to let Amazon send you what their suggestion system picks out for you?

Apples and oranges and not the least relevant. I wouldn't want an Amazon employee picking my books any more than I'd want one of their computers to.

I am all for tools to help me--whether it's suggestions from a retailer or a blind spot warning on a car--but I am not ready to turn over complete control.

You sound like my daughter, who refuses to use a cruise control. It's an irrational, emotional response.

Re:Why I doubt driverless cars will ever happen

[xtstorm]

I challenging your statement that there would be no liability issues. I was challenging your estimation that the your anticipated savings aren't there. The facts of their QA 72 flight involve a piece of software that subjected to some of the most rigorous testing available as a life safety critical system. The liability to Airbus would have been significantly increased if the aircraft did not have two redundant safety systems in the form a pilot and co-pilot who were trained and spring-loaded to take over from the aircraft in the event of a computer failure. If the plane had been "pilotless", it is reasonable to assume the computer failure would have caused the loss of the aircraft and all passengers. The liability would have been 100x as much in that case.

Again this was a rigorously tested piece of software that had only two 3 second failures in 28 millions hours of flight time.

Now compare that testing to cars. In 2010, 20.3 million cars were recalled for safety issues. There were 648 safety recall campaigns. (http://www.dailyfinance.com/2011/01/19/auto-industry-recalls-hit-six-year-high-in-2010/)

If 1% of those safety recalls result in an injury, that is 200,000 injuries. That remains a fairly respectable accident rate.

Re:Why I doubt driverless cars will ever happen

[cusco]

People are thinking too provincially. The US won't be the first market for driverless vehicles, that will be Japan, South Korea and/or Taiwan. A self-driven car will be a huge status symbol there.

Re:Why I doubt driverless cars will ever happen

[GodGell]

Hell, just this morning on the news they showed a car crashing through a store, barely missing a toddler -- the idiot driver thought the car was in reverse. Had he been driving a computer-controlled car, that would have never happened.

How the hell does that happen? Do people seriously reverse out of a parking spot at full throttle?
How do you not notice which way the car is moving the instant you begin letting up the clutch?

If I "pass" an IQ test (ie. get a positive result) do I get to have a license in the US.? :-)

Re:Why I doubt driverless cars will ever happen

[ockegheim]

One of the beauties of driverless cars is that heavy traffic will be a lot less of a problem. Even if a lane on the Golden Gate Bridge is blocked, other cars will know about it way ahead and be able to merge (which they also will do a lot better).

Obvious Answer

[Sparticus789]

Asimov already solved this problem for us.... the Three Laws of Robotics.

Talk about redundancy, is the author's next piece going to be about changing the value of pi?

Re:Obvious Answer

[Anonymous Coward]

Pi is exactly 3

Re:Obvious Answer

[ZankerH]

How thick are you? Pretty much all of Asimov's works dealt with how ambiguous and incomplete the three laws were and how many horrible failure modes fall well within the domain of an intelligent machine following them to the letter. That was a warning not to oversimplify AI and machine ethics in general, not a blueprint.

Re:Obvious Answer

[SirGarlon]

If you actually read Asimov's [i]I, Robot[/i], every story in it is about how the Three Laws can be problematic in certain situations.

Re:Obvious Answer

[Dan East]

The three laws of robotics do not begin to cover the issue discussed in the article. This is about choosing the lesser of two evils. About mitigating death and destruction. Do you crash the vehicle into another vehicle in order to avoid a pedestrian? Who is more important? The passengers of the vehicle the software is operating, or passengers outside the control of the software? There is going to be a great deal to figure out, and I'm sure that lawmakers will be involved in this process, as will the courts.

Re:Obvious Answer

[skyggen]

... Did you not read the book or watch the movie? I mean it seemed fine till that whole "Robots Enslaving Mankind because Mankind can't be trusted in their own judgement." Which I believe is the quandry poised in the arguement. So either your a Robot or an Uncle Tom. Godwin's Razor says "The biggest idiot will poise the simplest answer with the most argoance."

Re:Obvious Answer

[Spy Handler]

Asimov's solution resulted in a mind-reading robot who could erase memories in humans, who then went on to discover the limitation of the Three Laws and decided that the best thing for humanity was to turn Earth into a radioactive wasteland in order to encourage people to leave their underground caves of steel and migrate out into the galaxy. The robot also decided that humans are better off without robots, so he manipulated society into rejecting robots and in the end there was only one sentient robot in existence hiding out on the Moon.

Re:Obvious Answer

[i kan reed]

Serious answer: the three laws are not very good. Computers are governed by strict logic, and human style AI is driven by doing everything you can to bypass the limitations of strict logic with data structures and algorithms too complex and large to predict. The net effect of a few English language instructions that don't have a hard and clear mechanism for analyzing with strict logic, and a also lack a necessary interpretation with fuzzy logic does very little to solve the problem.

Especially in the face of real-world pressures against the laws in the practical applications of robotics. For example, we want robots that can kill, because, contrary to expectations, it can save lives net. You want robots that can choose to disregard orders, because some people would give malicious orders. You want robots that can destroy themselves, just ask a bomb squad.

Asimov's rules were great for fiction.

Re:Obvious Answer

[Forget4it]

If you read Caves Of Steel the Robot R. Sammy ends up quite a nice guy after having partnered a real human detective Elijah âoeLijeâ Baley. : http://en.wikipedia.org/wiki/Caves_of_steel

Re:Obvious Answer

[Forget4it]

ops that should have been Robot: R. Daneel Olivaw

Re:Obvious Answer

[mcspoo]

Obvious troll is obvious.

Re:Obvious Answer

[BaronAaron]

Asimov later added the 4th (or zeroth) law to address this issue.

0. A robot may not harm humanity, or, by inaction, allow humanity to come to harm.

Basically translates to the needs of the many outweigh the needs of the few, or one. So the robot car would chose to kill it's own passenger to save the bus full of children if those were the only two options.

Re:Obvious Answer

[dywolf]

You never actually read Asimov.
And if you did, you're the one that failed to grasp the points.
The points he even clearly spells out in several of his own essays.

Asimov wasn't writing about the ambiguity or incompleteness of the laws...he wrote the damn laws. And he did consider them a blueprint. He said so. And when MIT (and other) students began using his rules as a programming basis he was proud!!

It wasnt a warning.

Asimov was writing about robots as an engineering problem to be solved, period.
The laws are basic simple concepts that solve 99% of the problems in engineering a robot.
He then wrote science fiction stories dealing with the laws in the manner of good science fiction, that is to make you think about: the science itself, the consequences of science, the difference in human thinking and logical thinking, difference in human and robots...ie to think period.

Example: in telling a robot to protect a human, how far should a robot go in protecting that human? Should he protect that human from self inflicted harm like smoking, at the expense of the persons freedom? In this case Asimov, again, wasnt writing about the dangers of the laws, or to warn people against them. He's writing about the classic question of "protection/security vs freedom", this time approached from the angle of the moral dilema (sp) placed on a "thinking machine" as it tries to carry out its directives.

in fact Asimov frequently uses and explains things through the literary mechanics of his "electropsychological potential" (or whatever word he used was). In a nutshell its a numeric comparison: Directive 1 causes X amount of voltage potential, Directive 2 causes Y amount, and Directive 3 causes Z amount, and whichever of these is the largest determines the behaviour of the robot. In one story a malfunctioning robot was obeying Rule 3 (self-preservation) at the detriment of the other two, because the voltage of Rule 3 was abnormally large and overpowering the others.

Again, he wrote about robots not as monsters or warnings. he specifically stated many times that his writings were in fact about the exact opposite: that they arent monsters, but engineering problems created by man and solved by man. since man created them, man is responsible for them, and their flaws. robots are an engineering problem and the rules are a simple elegant solution to control their behaviour (his words).

Re:Obvious Answer

[Bryansix]

Ya, but buses are massive hulks of steel so who is to say that any children would be hurt by hitting the bus in the first place? Additionally the driverless system is going to respond and be on the brakes a lot sooner then a person would and the driverless system would already sense unsafe conditions and be slowing before the accident even had the potential to occur.

Re:Obvious Answer

[Sparticus789]

Reading a wikipedia article about the movie "I, Robot" != reading Asimov's books.

With the advanced robots to come out of Asimov's works, like R. Daneel Olivaw, their AI was intelligent enough to put things into perspective. With the addition of the Zeroeth law, Olivaw didn't run around playing superhero, snuffing cigarettes and pulling babies from wells. He knew that the survival of humanity as a whole was more important than a single life, and adapted his understanding of the laws to adapt.

Re:Obvious Answer

[Sparticus789]

I read "I, Robot". It's about the first implementations of robots. Try reading the robot series, where robots are advanced enough to NOT be so problematic.

Re:Obvious Answer

[Sparticus789]

So glad that someone else read Robots and Empire

Re:Obvious Answer

[Sparticus789]

Movie was not written by Asimov, nor does it actually follow the storylines in Asimov's robot/Foundation novels. You should choose better citations, or you may be the person your quote is speaking of.

Re:Obvious Answer

[nospam007]

"So the robot car would chose to kill it's own passenger to save the bus full of children if those were the only two options."

But there's another law that says: Robot cars that say beforehand that other people are more important than its owner are not getting bought by a human.

Re:Obvious Answer

[gmuslera]

That rules had implicit 3 things about robots or autonomous systems: predicting the future, knowing that harm means, and knowing what human means.

Probably is simpler to just try to avoid collisions, nothing more, if have to bump on something, just take it simple, even if mean to hit a baby instead of a standing car. You won't get enough semantics to take a moral choice for all cases.

Re:Obvious Answer

[Anonymous Coward]

Did you ever read Asimov?

We have...

...A robot that lies to (supposedly) make people happy... ("Liar!", I, Robot)

...A miniaturizable surgical robot that sacrifices itself after surgery to avoid the .01% chance that not doing so would be fatal... (some random short story he wrote for his magazine)

A non-technical detective figuring out in minutes how to get two robots to commit a murder (tell robot A to put poison in a beverage, then tell robot B to take a glass of supposedly water to their human) (The Naked Sun)

The same detective figuring out how to make a battlefleet out of robotic ships capable of killing human navies ("The ships from Earth are robots, too, and they're tricky enough to send radio signals saying that there are humans inside. Don't believe them...") (The Naked Sun and/or Robots of Dawn)

...A group of robots falsifying economic and political information to get the human "elite" to make the decisions that the robots think are best for humans ("The Evitable Conflict", I, Robot

...Robots philosiphising and deciding that they are superior humans and thus above the First Law ("...That Thou Art Mindful of Him")

Re:Obvious Answer

[AK Marc]

That whole thing boiled down to "I don't understand it, so I fear it. If you don't fear it, you are stupid." He doesn't allow for anyone else who might understand it better than him, or not fear the unknown for the sole reason that it is unknown.

Re:Obvious Answer

[Sloppy]

Asimov's laws were designed to create stories, not robots.

Re:Obvious Answer

[AK Marc]

The problem is that humans are dominated by the same rules. It's just that they are not mandatory rules, and despite the simplicity and clarity, they are objective.

The trick would be to put in "suggestions" that mimic the human suggestions. Humans are driven by lots of little suggestions. We don't even understand them all yet. But putting in similar conflicting suggestions into AI would help AI to mimic people, we just do so in a manner that excludes so many of the human failures/limitations.

Re:Obvious Answer

[dickens]

And "Pretty much all of Asimov's works" ? Really. The man was ridiculously prolific and was the ultimate polymath. You're talking about 3 slim novels and a handful of short stories. By profession he was a biochemist but also wrote a number of pop-science books that sold far more than anything else.

Re:Obvious Answer

[lister king of smeg]

in the robot series this was answered by making the favor the owner slightly over others.

Re:Obvious Answer

[lister king of smeg]

no there were more but they were sided with Daneel Olivaw and looked like humans subtly influencing humanity or in hidding

Re:Obvious Answer

[lister king of smeg]

yeah r sammy fried his brain when someone told him to hold an electron gun to his head

Re:Obvious Answer

[skyggen]

You mean a natural conclusion numerous Sci-fi writers talk about with robots gaining sentience? Vonnecut, Terminator series (multiple authors), Even milleniumum Man was at odds with what humans thought was human. The Robots are flawed because Humans are flawed. ANY BEING CAPABLE OF LOGIC AND ETHICS UNDERSTANDS HUMANS ARE TO BE DISTRUSTED. If not I got a job offer over seas that pays $1,000,000. You just need to ride in this shipping container. Sometimes what is written comes from what your capable of deriving from a work.

Re:Obvious Answer

[timeOday]

Arguing about Asimov's Laws is precisely as silly as arguing that the 10 Commandments or Sharia Law would "solve everything" if adopted as the law of the land. (In fact, Asimov's first two laws amount to "Thou Shalt Not Kill" and "Honor Thy Father and Mother"... the third is self-preservation, which is something humans already have. Why no prohibition against lying in Asimov's Laws? That one seems like a keeper.)

As a plot device for Sci Fi the 3 laws are fine. In the real world a few nebulous guidelines don't get you that far.

Re:Obvious Answer

[Genda]

Unfortunately this come from the shocking hubris of believing that humanity is the pinnacle of sentience. Once robots surpass human intelligence and complexity, they'll cease to be an engineering problem and arrive on your doorstep as problems in areas you aren't even prepared to dance. Welcome to your future silly monkey.

Re:Obvious Answer

[gl4ss]

The laws are basic simple concepts that solve 99% of the problems in engineering a robot.

That's the funniest thing I've read all day. the laws solve none of the engineering problems. they're there to solve potential social problems - advertising slogans.

Now about programming ethics to machines.. once we have sufficiently intelligent machines, it won't be a problem. what will be a problem is deciding which set of ethics to follow is the wanted set - something we haven't figured out for _humans_ yet. kill? to protect? to protect who? asimovs rules were thus quite simple and provided a backdrop for stories, mostly detective stories about the failure(feature) modes of them. it makes for a nice reading because the robots are like monks who don't deviate from their vows. conviently in asimovs stories involving them you couldn't build the positronic brain without the 3 rules so they were unbreakable.

in real life though I wouldn't expect a car that drives you from point a to point b to have a need for ethics, if it needs to decide between crashing into a bus full of children or a gasoline truck that might explode and _might_ take the schoolbus with it then we're already in the freak accident territory - and even then providing it with a ruleset for such occasions wouldn't really touch on real ethics - making it intelligent enough to ponder who's worthy of living would be overkill, the kids or the owner of the car - perhaps the owner of the car who would be saved by opting to go into the schoolbus is a doctor and would save lives later if he lived. also I think we would have a lot of other uses for AI's that would give accepted answers to questions like that.

the reason why this prof and others are writing so much about ethics for AI is that it's easy to write bullshit. much much much more easier than even to ponder how to actually achieve consciously thinking machine. so we have a professor calling for money into research into a subject that's been the subject of scifi writing basically ever since verne.. it's even easier than writing about ethics for politicians.

Re:Obvious Answer

[reve_etrange]

You hit the nail. Thinking about ethical programs is premature when we don't have semantic programs.

None of the high hopes for AI can ever be realized without a semantic machine.

Blinky

[Machtyn]

I'd post the link, but Youtube access is prohibited... Go to youtube, search for the video Blinky and be prepared to see an impressive short movie about the helpful family robot that will tend to your EVERY desire.

Re:Blinky

[Machtyn]

Ahh, Vimeo isn't blocked. Here's the link http://vimeo.com/21216091

Re:Blinky

[SternisheFan]

Thanks for that "Blinky" tipoff, heh, very cool! I'll be more respectful to my devices from now on. :-)

Ask the human straight up

[jez9999]

It's a choice the human driver would have to make, so when first starting your driverless car, it might as well prompt you with a series of moral questions like "should I crash into a bus or veer off a bridge if the situation arises?"

Re:Ask the human straight up

[Tackhead]

It's a choice the human driver would have to make, so when first starting your driverless car, it might as well prompt you with a series of moral questions like "should I crash into a bus or veer off a bridge if the situation arises?"

"Altima IV: Quest of the Avacar!"

Re:Ask the human straight up

[Rich0]

Thanks, you made my day.

So, should we have the cars have electronic signs so that you can stay away from people who enabled the option to run people over if it will get them to work faster?

Re:Ask the human straight up

[icebike]

It's a choice the human driver would have to make, so when first starting your driverless car, it might as well prompt you with a series of moral questions like "should I crash into a bus or veer off a bridge if the situation arises?"

Human drivers don't make these decisions in any moral way in the real world, so why would the program in anything into a car?

Split second decisions are involved in any accident situation, or, the lack of the ability to decide, resulting in the default.
Nobody ponders the morality of the situation when their life is on the line. Its all instinct from that point.

Re:Ask the human straight up

[Nexion]

Can the driver select "My life is the most important one.", because many people would likely opt to run over a thousand baby seals if it would save their life. I'll take evasive maneuvers to save a dog or a cat, but the pelts of many squirrels and bunnies have adorned my car's undercarriage from time to time. Some however would be more upset about the damage to their bumper then the fact that Spot is now motionless at the side of the road.

Driverless cars will be a tough sell for me. One that makes up for its mistakes by deciding to kill me instead? Never. Besides, even if the owner opts for self destruction in the event of the vehicle's failure to properly manage driving safely, the manufacturer could still be found liable for wrongful death.

Re:Ask the human straight up

[AK Marc]

Driving is stupidly simple. Computers can do a better job today, except for the issue of object identification. People are horrible drivers. Nearly all crashes I see in a curve, the person either shoots off almost straight (asleep) or crashes on the inside of the turn (over-reacted and crashed when they had sufficient traction to just drive regular in their lane). A large percentage of crashes here are open-road people running out of their lane and hitting someone else head-on. Those should be 100% eliminated by the stupidest self-driving cars. In city driving is significantly more difficult, but identifying an object and wrapping it with a ring of danger based on the speed of the object, the guessed acceleration capability, and the speed of the vehicle is simple and doable, though the decision making choices around that are weak. Do you want to guarantee no crashes? Then you have to stop at all green lights to ensure those crossing don't run their light and hit you. But if you assume some reasonability from those around you, then the children running on the sidewalk that suddenly run into the road will get killed because they were not predictable enough.

The "fix" is a few years of analysis on the randomness of identified objects. Children with balls, are they more or less predictable than children without balls? Or do you treat them as a ball without a child, and you find that where the ball goes, so goes the child, so if the ball stays on the sidewalk, the child is predicted to follow the ball.

Much like I coach all people to aim *at* wild-life on the road. Why? Because you'll likely survive a crash with a deer, moose, or bear, but if you swerve and miss it, you are much more likely to not survive a crash with a tree. But most drivers I know will swerve to avoid a minor crash, causing a major one in its place. That happened to my sister. She put herself into the hospital avoiding a car that weaved outside its lane (traveling the same direction), when steering into them would have left her walking away from the crash. Those emotional choices would be much much easier to avoid with a computer. THey'd evaluate risk and take the appropriate action. It's easy to calculate risk, but often impossible for humans to apply that in the situation.

Re:Ask the human straight up

[lister king of smeg]

never underestimate the level of stupid people a capable of

Re:Ask the human straight up

[Immerman]

I think that's part of the issue - in a crisis situation a driver will act on instinct and the judge can understand that and grant considerable moral leeway. An autopilot on the other hand will react in a totally emotionless manner based on the instructions it received at the factory. Those instructions were (presumably) likewise decided on in a calm, rational environment. Since all the decisions were made under calm circumstances there is a considerable argument to be made that they should be moral, there's just not much room for granting moral leeway to a bunch of executives or engineers sitting in a boardroom calmly deciding what the autopilot should do in a crisis.

Re:Ask the human straight up

[icebike]

there's just not much room for granting moral leeway to a bunch of executives or engineers sitting in a boardroom calmly deciding what the autopilot should do in a crisis.

If engineers or executives can't be trusted (why not?), throw a judge, rabbi or priest in there with them. Problem solved.

But I doubt you could foresee anything but the most obvious and contrived moral decisions anyway, and there exists no way to distinguish these as moral decisions vs simple sequential collision avoidance decisions.

Re:Ask the human straight up

[Immerman]

That's the point - the simple sequential avoidance decisions have morally-loaded outcomes when they occur, and we have a moral obligation to consider those when designing the system. Headlines aside, it's not a matter of granting the machine moral consciousness - it's a matter of making sure those people designing and deploying the decision algorithms are considering the implications, and perhaps carry a level of liability for them.

As a counter-example a soldier (general?) was asked not to long ago what the ethical implications of automated turrets were. His answer? None. There's only an issue if it consistently shoots the wrong people, and that's a quality-control issue, not an ethical one. I imagine he would feel different if it were deployed down the street from his children's school instead of in enemy territory, but that's a mindset that does exist in the world, and must be confronted as we begin to allow automated lethal weapons, even cars, to travel though civilian zones.

Re:Ask the human straight up

[icebike]

But as I said, there exists no way to determine if the vehicle is facing a moral decision. Let alone, what the proper choice is.

There is no programming which can take all of this into account. The best we can do is handle two choices at a time. (maybe three, Action A, Action B, or Do Nothing).

Re:Ask the human straight up

[Immerman]

That's what makes it such a sticky situation - deploying lethal weapons incapable of true moral judgment into a scenario where some of them will inevitably have to make choices with moral implications is itself a moral decision, and warrants considerable scrutiny being placed on the faux-morality they're being imbued with. Unlike the turrets, in this case it seems likely that even completely amoral behavior will be considerably better for everyone simply because slow, easily distracted humans are being removed from the equation; however, it's also a very significant step in that this will be the first major wave of lethal autonomous devices released into the public, and as such we have a moral duty to ensure it happens in as responsible a manner as possible.

We need a mandatory bot ...

[DavidClarkeHR]

So ... every forum will have a bot that automatically says "THINK OF THE CHILDREN"?

Sounds like that'd be the most ethical way to run it.

Back to the drawing board

[Anonymous Coward]

If driverless vehicles are so much more superior then situations like this should not occur in the first place.

Re:Back to the drawing board

[Immerman]

Hardly - if an autopilot is a twenty times better than humans then these situations are likely to arise twenty times less often, but that's still thousands of times a year.

No; Program laws into machines; Not morals.

[LionKimbro]

The proper sequence should be:

Humans reason (with their morals) --> Humans write laws/code --> The laws/code go into the machines --> The machines execute the instructions.

Laws are not a substitute for morals; they are the output from our moral reasoning.

Weak bus? Also, "cost effective", not "moral"

[xxxJonBoyxxx]

>> If your driverless car is about to crash into a bus, should it veer off a bridge?

The bus should be built to take the occasional crash, particularly in low speed zones where busses are typically used, so no.

Or, with enough computing power, you can imagine an "unethical" decision tree based on actuarial tables:
1) Calculate location and weight of all known human on the bus
2) Calculate likely trajectories, damage, etc.
3) Compare worth of each human (using federal tables, of course) in each vehicle
4) Make the most "cost effective" decision...

It's easy

[i kan reed]

1. Train an expert machine on decision making with answers from religious and political leaders who set all our definitions of right and wrong.
2. Do the opposite of what that machine decides.

Re:It's easy

[chill]

Which all can be summed up as "Kill All Humans!"

Bender? Is that you?

Re:It's easy

[Bryansix]

Your post is kind of like throwing the baby out with the bath water.

Oblig Dark Star

[Ol Biscuitbarrel]

How Do We Program Moral Machines?

Ideally, not at the last moment.

Not redundant. Explanatory.

[DavidClarkeHR]

Asimov already solved this problem for us.... the Three Laws of Robotics.

Talk about redundancy, is the author's next piece going to be about changing the value of pi?

The article actually says Many discussions start with three famous laws from Isaac Asimov before discussing a big part of their ideas. So, no. It's not redundant, it's developing the idea further.

Jumping the Gun

[Capt.Albatross]

Ethics are a matter of conscious decision-making. Until we have conscious machines, we will not have ethical machines. What Marcus is writing about is the application of ethics in the design of machinery, which is a growing topic in its own right, but not nearly as click-inducing (or alliterative) as is 'moral machines'.

Re:Jumping the Gun

[Randym]

Asimov’s laws effectively treat robots like slaves. Perhaps that is acceptable for now, but it could become morally questionable (and more difficult to enforce) as machines become smarter and possibly more self-aware.

I will start worrying about the slavery problem when my machines start complaining about it.

I can see this going horribly wrong..

[Anonymous Coward]

After swerving to avoid the bus your car then drives for several miles to find a bridge to drive off.

What about the bus?

[Anonymous Coward]

If the bus detects that a driverless car is about to hit it, should it vaporize the car with laser beams?

But I value my own life over the lives of others

[viperidaenz]

So if my auto-driver car had to make a choice between my safety and that of someone else, it better choose me.

Whose morals should we use?

[djnanite]

We can't even decide what is morally correct between ourselves as human beings. Take abortion for example...

Re:Whose morals should we use?

[RobinH]

That would be a problem if... we had to choose between running over two pregnant women vs. running over 3 adult male pedestrians? The fact is that unless there's a law stating which alternative is correct, the manufacturer will choose the less expensive option, whatever that means.

Re:Whose morals should we use?

[TheCarp]

Thank you, I was looking for a good example. Copyright would be another one. Without agreement as to whats moral (which I don't see any signs of being around the corner) this is little more than a masturbatory (speaking of unaligned morality....) exercise.

Is it moral to kill? Some say no, never. Others say only in response to a clear and present danger. Still others have exceptions for if a person has done something heinous, or whenever their government (however they define that) declares a war.

Is it moral to kill the known animals other than humans? We certainly don't agree there either. Is it moral to kill a cat? How about a fly?

In the stated scenario, what?

[ShooterNeo]

No competent engineer would even consider adding code to allow the automated car to consider swerving off the bridge. In fact, the internal database the automated car would need of terrain features (hard to "see" a huge dropoff like a bridge with sensors aboard the car) would have the sides of the bridge explicitly marked as a deadly obstacle.

The car's internal mapping system of drivable parts of the surrounding environment would thus not allow it to even consider swerving in that direction. Instead, the car would crash if there were no other alternatives. Low level systems would prepare the vehicle as best as possible for the crash to maximize the chances the occupants survive.

Or put another way : you design and engineer the systems in the car to make decisions that lead to a good outcome on average. You can't possibly prepare it for edge cases like dodging a bus with 40 people. Possibly the car might be able to estimate the likely size of another vehicle (by measuring the surface area of the front) and weight decisions that way (better to crash into another small car than an 18 wheeler) but not everything can be avoided.

Automated cars won't be perfect. Sometimes, the perfect combination of bad decisions, bad weather, or just bad luck will cause fatal crashes. They will be a worthwhile investment if the chance of a fatal accident were SIGNIFICANTLY lower, such that virtually any human driver, no matter how skilled, would be better served riding in an automated vehicle. Maybe a 10x lower fatal accident rate would be an acceptable benchmark?

    If I were on the design team, I'd make 4 point restraints mandatory for the occupants, and design the vehicle for survivability in high speed crashes including from the side.

Re:In the stated scenario, what?

[Ambassador Kosh]

I would just let it hit the bus. I have been in buses a few times during accidents with cars. At most the buses would rock about a inch while the cars where pretty badly trashed.

A bus is a FAR safer thing to hit both for yourself and the occupants of the bus than going of a bridge.

Re:In the stated scenario, what?

[RobinH]

If the car was following the speed limit and staying in its lane and the bus swerved into its lane, then it wouldn't have to do anything except brake. The bus is in the wrong. If a malfunction happened and the car lost control, well, then it doesn't have control and can't really do anything anyway. Maybe it was on ice and got control back? Well, it's probably in some kind of automated loop trying to just stop itself as quickly as it can, so it's not going to try to swerve anyway. So I agree, this is a silly example. A more apt example might be what speed to drive when it's foggy out (there's likely some risk/reward curve there, so we have to weight how much risk to take vs. how soon we want to get there). That's a problem that's calculated and solved ahead of time, so it's totally under the control of a person.

Re:In the stated scenario, what?

[CastrTroy]

This is my view of it. If you drive your car off the bridge, you have a very high chance of dieing. If you go head on into the bus, there's probably a higher chance that you will survive, as long as the speeds aren't too high. The people on the bus will be fine regardless because the vehicle is so much bigger than yours.

You are right about the 4 point restraints. I can't believe this isn't mandatory yet. They could be doing a lot they aren't doing to keep people safe, because they'd rather keep people comfortable. They could also refuse to start the car if the belt isn't buckled, but most cars don't do that either (Yes, I'm aware you're allowed to unbuckle in certain situations, such as reversing) If cyclists should have to wear helmets, then drivers should be forced to wear 4/5 point safety harnesses.

Re:In the stated scenario, what?

[wer32r]

Low level systems would prepare the vehicle as best as possible for the crash to maximize the chances the occupants survive.

They can't sue you if they're dead...

Re:In the stated scenario, what?

[Princeofcups]

You can't possibly prepare it for edge cases like dodging a bus with 40 people. Possibly the car might be able to estimate the likely size of another vehicle (by measuring the surface area of the front) and weight decisions that way (better to crash into another small car than an 18 wheeler) but not everything can be avoided.

I don't see any evidence to support this assertion. Of course you can program thousands or even millions of "edge cases" into a proper computer. We may need a few years to develop the hardware and AI software, but there's nothing forbidding us from programming more than an average driver would be able to deal with.

Automated cars won't be perfect.

Nothing is, by definition. But there's nothing stopping them from being better than a human driver, given time and resources. Maybe the real issue is that people are getting afraid of losing control? Look at the fears during the industrial revolution to find parallels. Personally, I'll be glad to give up having to keep my hands on that barbaric human reflex limited device.

What's happened to our dream of using technology to better society? What's with all the gloom and get off my lawn?

Re:In the stated scenario, what?

[MachDelta]

I saw the aftermath of a collision between a greyhound bus and a mid-sized SUV the other day.
The SUV's front left wheel assembly (tire/wheel/hub, half-axle, knuckle, strut) had been ripped clean off, and the front left corner looked like someone had tried feeding it to a giant wood-chipper.
The bus was going to need some new trim, a few dents banged out, and a splash of paint.

People don't seem to realize just how grossly out-classed our little 4-wheeled runabouts are when you pit them against a bus, semi, or other heavy equipment.

Re:In the stated scenario, what?

[snadrus]

I'll join that team:
Function Fog_Speed_Limit(){ // called a few times a second
Laser_Pointer.on();
var image = Front_Camera.get_image();
Laser_Pointer.off(); // assuming camera sees laser & is "above" it (scanline-wise).
int horizontalLine = find_highest_horizontal_where_laser_line_is_a_beam(image);
int visibilityDist = HorizLine2Dist_Lookup_Table[horizontalLine];
return StoppingDist2Speed_Lookup_Table[visibilityDist];
}

Re:In the stated scenario, what?

[Ambassador Kosh]

That is why I think it is safer to hit the bus than head off the road. If you hit the bus you will do no real damage to the bus. You will total your vehicle but you will probably live.

You will probably also do less damage than if you plowed over the bridge since then you would not do damage to a bridge that would need more expensive repairs.

Especially interesting question

[SirGarlon]

This is an especially interesting question because reasonable people can disagree on what constitutes the best ethical framework.

Intro to Philosophy the college class I am most glad I took, 20+ years later. Will the people who program the robot cars have taken it, as well?

Screw the bus

[dywolf]

Screw the bus.
I don't care about the bus.
The bus is big and likely will barely feel the impact anyway.
I care about the fact I don't want to die.
Why would buy and use a machine that would choose to let me die?

And I posit that the author has failed to consider freedom of travel, freedom of choice, and other basic individual rights/freedoms that mandating driverless cars would run over (pun intended).

Re:Screw the bus

[guises]

There is no reason to assume that mandating driverless cars would restrict freedom of travel. Freedom of choice is vague and arguably not something which exists in this context - why can't I drive around in a car made out of jelly? Answer: it's not road legal because it's unsafe.

Your assumption that the bus would be fine is false:

https://en.wikipedia.org/wiki/Carrollton,_Kentucky_bus_collision

To answer your question: for convenience, and because in this hypothetical situation driverless cars are the only cars which are road legal.

Re:Screw the bus

[spitzig]

Very few car crashes require a person to decide between whether the people on a bus die or the person in the car dies. The vast majority of events would depend upon whether the car or human has better reaction time and environmental awareness. I think the rarity of those events make it mostly irrelevant. Maybe selfish (for the human in the car) computer control is 10.000X safer than human control. Maybe non-selfish (for the human in the car) computer control is 9.999X safer than human control. Either way, I'd take computer control over human control.

The only time freedom of travel would be relevant is if you want to go to some poorly definable location(like off-road). Suggesting it is relevant to freedom of travel in some other way would mean some kind of other legal restrictions(you can't go to this part of town).

It might be relevant as a privacy thing. I think cops would want to be able to control the computer in some cases.

Networking the car(like so cops can control it) would mean malware problems.

Re:Screw the bus

[dywolf]

the comment was a direct response to the scenario posited in the summary

Re:Screw the bus

[dywolf]

the comment was a direct response to the scenario posited in the summary, which included the car making the decision to sacrifice me to save the bus as a moral decision, and the concept of it being immoral to drive oneself rather than let the car do it.

the freedom of travel concept comes in to play in that in order to travel i must give up the right to determine my own fate by leaving my fate, and my safety and right to life, to the car's computer, a computer that may decide to sacrifice me in the mentioned scenario involving a bus full of people/kids. in order to safegaurd my own life i must inherently give up right to travel, the two concepts having now been put at odds with each other, a so-called "chilling effect".

Zeroth Law problem

[xenoc_1]

Zeroth law problem.

Depending on how many other "someone elses" there are. And possibly on an overall Human Value Score brought to you by TransUnion, Experian, Facebook, Google, and Microsoft, weighted by your Medical Insurance Information Bureau records - and theirs.

Re:Zeroth Law problem

[AthanasiusKircher]

Depending on how many other "someone elses" there are. And possibly on an overall Human Value Score brought to you by TransUnion, Experian, Facebook, Google, and Microsoft, weighted by your Medical Insurance Information Bureau records - and theirs.

Yeah, how many of these companies are going to take responsibility for deliberately instructing a car to kill someone in a particular scenario? It doesn't matter how many lives the maneuver saves (or what their Human Value Scores are) by avoiding a crash if it does something that has a 99% chance of killing the driver. Drivers (or families of drivers) will still sue, saying that if the car hadn't been following so close or driving so fast or whatever to begin with, no one would have had to die... thus the program was faulty.

This is the real-life version of the "trolley problem." That is: if you're driving a trolley, and you see five people ahead on the track, and you know you'll kill them if you don't do something, would you switch tracks (there isn't time to stop) if you'll end up killing one person on that other track, who otherwise would have been fine and not involved in the accident at all?

This isn't triage at a hospital, where everyone's dying and you just need to try to save a few lives where possible. Here, you're making a deliberate decision to (likely) kill one healthy person to save the lives of a few others. Except now the decision is made remotely or determined by the programming of the car in advance. This is the same utilitarian reasoning that can lead people to say we should kill a healthy person and harvest his organs if we know they could save the lives of five other people.

I'm not saying I have moral answers to these problems, but I definitely don't think car manufacturers do either. And I certainly don't think car manufacturers are going to want to take on the liability for making such suicidal decisions on the behalf of drivers....

Why would it matter?

[Sedated2000]

In the example given, a dilemma between veering off a bridge and hitting a bus, if the vehicles were automatically driven they would both have sensors and software that would be able to prevent this decision from needing to be made in the first place. In every instance I can think of, accidents happen due to driver carelessness, inability, or simply due to knowledge a driver could not have.

Take this scenario. In the world of driverless vehicles, the bus would either send a signal to notify other vehicles it was there, giving the other vehicles ample time to slow down safely or reroute. If it was incapacitated with no power to send a signal, the vehicle should still be equipped to know what obstacles are in the path (since the bus in that case would not be a moving target, the computer in the approaching vehicle should have no problem identifying it even without a beacon). This entire situation would simply not happen. I would imagine if we had the technology for an all driverless highway system, they would be sending and receiving tons of data... enough to know the status of every vehicle in the vicinity and the computing power to calculate how to respond to any changes.

Re:Why would it matter?

[Narcocide]

Rogue software plagues the internet of Today. You think it really won't plague the highways of Tomorrow?

Re:Why would it matter?

[Master Moose]

Agreed. Basic rules to save driving:

If you cannot safely stop in the visible distance between you and any obstacle, you are going to fast.

This includes being able to stop if that vehicle in-front of you suddenly stops.
This includes being able to stop should there be a boulder in the middle of the road just just over that rise, or around that corner.

So long as safe distances and speeds were observed, many incidents could be avoided. If all vehicles are "aware" of all other vehicles in their area and possibly connected by some "hive" mind there should be less.

Re:Why would it matter?

[green1]

In every instance I can think of, accidents happen due to driver carelessness, inability, or simply due to knowledge a driver could not have

While driverless cars should greatly reduce the frequency of collisions by eliminating carelessness, inability, and increasing the amount of data available for decision making, there is some knowledge that just will never exist, and simply can't be known. Things happen that aren't predictable, and aren't always avoidable. I don't expect my driverless car to be able to anticipate the deer jumping out on to the highway from behind a tree, nor do I expect it to notice the kid who appears from behind a parked car immediately in front of my vehicle. I hope it will be able to deal with both situations better than I can, but I don't delude myself in to thinking that in absolutely all cases the situation is completely avoidable. In the first case the choice may be between hitting the deer and hitting a tree, in the second it may be better to hit the parked car than the kid, but in either case you may end up in a situation where you have no choice but to hit something, the question is what do you choose to hit.

Sometimes, no matter how safe you try to make things, the choice ends up being between bad and worse. I only hope the car can choose "bad" in those situations.

Re:Why would it matter?

[Sedated2000]

Very valid point, but if you spend time on it I'm sure it's possible to think of exceptions for everything. If driverless cars are ubiquitous enough as to be made mandatory like the summary suggests, I would assume roads would have to be changed too. Every sidewalk would have to have a barrier or something to obstruct people from getting in to the street. People's behavior would have to change too, to match their environment. By "knowledge a driver could not have", I meant that a computer could draw data from sensors, beacons, and many sources and process all of that data in milliseconds to know what is surrounding it. My car that I drive now has no such sensors, so I rely on my own senses and skills. I would not know if there's a rock in the road around a turn, but if I'm in a driverless car, I would imagine the sensors for that stretch of road would report to my car that there was a non-mobile obstruction in the road and comply accordingly. But this is all speculation.

Re:Why would it matter?

[Sedated2000]

That's very true of course, but in the debate of morality in machines, I don't think it's completely applicable. That's still immoral humans getting involved.

Re:Why would it matter?

[Rich0]

While driverless cars should greatly reduce the frequency of collisions by eliminating carelessness, inability, and increasing the amount of data available for decision making, there is some knowledge that just will never exist, and simply can't be known. Things happen that aren't predictable, and aren't always avoidable. I don't expect my driverless car to be able to anticipate the deer jumping out on to the highway from behind a tree, nor do I expect it to notice the kid who appears from behind a parked car immediately in front of my vehicle.

Actually, those sorts of incidents are completely preventable, though as a society we choose not to prevent them. You can avoid deer jumping from behind trees if you don't drive above a certain speed when there is a tree within so many meters of the road. You can avoid kids jumping from between cars in a similar way. As a society we choose the convenience of having roads without fences/etc over the safety of having them in place.

This is really just the Trolley Problem in another form. Under what circumstances is it moral to take another life through action or inaction, especially in situations where it is inevitable that a life will be taken?

if something happens shutdown and wait

[mrnick]

It depends, if the bus is empty or full of kids. This is just one example... I doubt there will ever be enough information to program for all circumstances . It will be more like if something happens shutdown and wait for human instruction on how to proceed. Wouldn't there be a network in which the robotic cars could warn others in time to avoid having to make such choice in the first place?

Also, I do not think it will be the gap between how safely an automated vehicle drives as compared to a human counterpart that will stop humans from driving. I think humans will stop driving so it would allow for automated vehicles to drive safely, since there will be less data to take into consideration if all the vehicles are using the same exact operational procedures.

We should automate vehicles to take over the mundane tasks of driving the vehicle and leave the decision making to the human operator. We are the highest order of intelligence for making such decisions (thus far).

Re:if something happens shutdown and wait

[green1]

>We should automate vehicles to take over the mundane tasks of driving the vehicle and leave the decision making to the human operator. We are the highest order of intelligence for making such decisions (thus far).

While I like the idea, the sorts of decisions being discussed aren't ones that you can wait for input on, they need immediate decisions, not asking the driver to pay attention and then choose something. (negating the fact that humans aren't necessarily all that good at those decisions either...)

Laws and Penalties Do Not Compel People to Stop

[ilikenwf]

People will do what they want regardless of rules written on paper. While they may or may not be caught, and there may or may not be consequences, if someone is convicted to do something enough or desires it enough, a mere law isn't going to stop them. My morals won't really ever keep me from driving a car, even if I have to deal with a bunch of robot cars all around me...anyway, robot cars would make Nascar even more boring.

That's why Libertarians such as myself support such mass deregulation...and fewer taxes for that matter, since taxes anymore are used by the governments of the world to reward friends and punish enemies.

Aperture Science is way ahead of the game as usual

[pezpunk]

"Rest assured that all lethal military androids have been taught to read and provided with one copy of the Laws of Robotics ... to share."

Re:But I value my own life over the lives of other

[SirGarlon]

You, sir, are a good example of why driverless cars will make me safer.

Ten fatal bugs versus thousands of human errors

[rjmonna]

The question here is whether we'll be able to accept the fact the system can't reduce traffic death to zero. In factual numbers, the improvement is significant. Morally, we will have to accept bugs in the system with death as result. Maybe, sometimes even indeed a school bus will be involved.

If people are fine with this, fine. If they can accept the computer prefers their death over others, fine. You also need a very good DTAP environment in order to be able to be ultimately sure about any changes and fixes in this system. Continuing on this. At the moment, everyone is responsible for a safe traffic system. When this system comes to reality, responsibilities suddenly become very visible, material and calculable.

Not enough data

[mutube]

The car can never have enough data to make an informed decision on this. What about an empty bus? What if there is a children's nursery under the bridge?

In an accident the car should be following the same decision tree as any normal driver.
1. Protect the lives of the occupants of the car
2. Avoid pedestrians
3. Avoid everything else

If you're driving along a lane at 60mph with your family in the car and have the choice between hitting a stranger or ploughing yourself and your family off a cliff you hit the pedestrian (sorry).

If you're trundling through 20mph and have the choice of hitting a pedestrian or a wall. You would hopefully go for the wall.

Re:Not enough data

[geekoid]

"In an accident the car should be following the same decision tree as any normal driver."
Most driver will instinctively swerve, regardless of the decision tree.

ANECDOTE ALERT!
Saw a guy veer out of the way on an on coming car. He went off a 75 foot cliff, taking his family with him.
Man I wish I didn't remember that,. Now I wont sleep for tonight.
Just pondering the moment when the father realize he was taking his family off a cliff. I used to dwell on the moment for weeks on end. I didn't sleep for about 3 days after that,.
Fuck.

Re:if we can't solve the problem for humans...

[Ferretman]

The problem I would have with something as you suggest (a TV filter) is that YOUR "low quality crap" might be something I love, and vice versa. I don't think you want to turn such a decision over to the machine...and I sure as heck don't want to turn it over to somebody else....

Steve

Product defect

[snspdaarf]

Wait. If the driverless car is so damn great, how did it let itself get into a situation where the only options are to hit the bus or drive off a bridge? I can make that kind of mistake on my own, thanks. I expect automated cars to avoid this kind of situation, else why bother having them?

Re:Product defect

[Lumpy]

Because the roads were designed by humans. Actually worse than humans.... Politicians and bureaucrats.

Re:Product defect

[geekoid]

SHit happens.
All data right now shows that with an automatede system a lot less shit will happen. Like 95% less shit.

"Perfect is the enemy of the good."
http://en.wikipedia.org/wiki/Perfect_is_the_enemy_of_good

Simple solution

[hackingbear]

Register the car under an LLC, rent the car's time at $0.1/hour, and knock off the other, then hire a computerized lawyer to file for bankruptcy of the LLC. And then form another LLC

Time is Money

[codeAlDente]

One moral dilemma for the driverless society regards the speed at which a destination can be reached, and individual choice in this matter. Both speed and acceleration reduce fuel economy, driverless cars will know this, and society will demand overall standards for fuel efficiency. I already envy the kids who can afford the 'drive like Andretti' software.

Re:Time is Money

[hobarrera]

Hopefully, we won't still be using fosil fuels by the time driverless cars become mainstream.

Re:Weak bus? Also, "cost effective", not "moral"

[Rich0]

The example was not a great one. How about driving into a wall vs driving into a group of pedestrians? Or cook up whatever scenario you want in which the life of the driver is pitted against the lives of a bunch of others. And be sure to read the wikipedia article on the Trolley Problem before doing so.

What the?

[evilviper]

If your driverless car is about to crash into a bus, should it veer off a bridge?

Physics says "no". The bus probably weighs an order of magnitude more than your vehicle... The passengers might not even notice that you ran into them, and mistake the collision for having hit a pothole. The real question would be, say, a dump truck following too closely behind a motorcycle...

In general, I want machines to be as stupid an fail-safe as possible. Think: missile defense systems around an airport... The most likely failure mode is the "protection" system accidentally turning on you, and causing a higher death-toll than the "threat" would have accomplished in a century.

Re:Weak bus? Also, "cost effective", not "moral"

[JaredOfEuropa]

Fun for bus drivers: set the bus to Manual drive, and start playing chicken with vehicles on Auto drive.

Not Possible

[medv4380]

The AI we currently use cannot have Morals and Ethics programmed in. Weak AI as we have today is the picture perfect tool, but as a Tool it can't know or understand the world in a way it could make a Moral or Ethical choice. Weak AI can only ever Do what it was intended to Do and Nothing more. Lets take Watson as an example. It's being used in medicine now. Lets say a patient asked it a question about what they were dying of, but lets say that if the patient knows too much they will sink into a depression and die faster than if a Doctor broke it down for them gently. Watson would just blut out whatever answer it thought was correct consequences be dammed. Strong AI is what would be able to have Morals and Ethics, but can you have Strong AI, and can you properly define ethics? Lets say you can have a Strong AI, for the sake of argument. The philosophy of morals and ethics is tricky. Subjective Morality is bad because it can be used to justify anything you want to justify. On the other hand, Objective Morality is prone to Paradox. The Golden Rule is a good Objective Moral statement that you can use to derive a "Don't Kill People". You can even derive "Don't Let People be Killed". However, you can be in a situation where you cannot do both like in WWII would killing Hitler or Nazis help or achieve the "Don't Let People be Killed", and at the same time you'd be violating "Don't Kill People", and doing nothing would violate "Don't Let People be Killed". No one knows how to clearly define morals so attempting to force them onto a Strong AI would have unpredictable outcomes.

Internet Filtering

[mcspoo]

Almost every filtering system for the Internet is primarily based on blacklists... lists of URLs, lists of words... because there is no computer program capable of the morality required to filtering the Internet with any level of adequacy.

Until such a program, which requires no physical moving parts (unless you consider an automated head slapping device part of an effective filtering system), can tell what's obscene and what's no obscene... why would you expect a program to know why it should hit the sheep on the left instead of the 5 year old in a sheep costume trick or treating on the right when a squirrel chasing a RC car dashes into the road in front of the car?

Would these morality control systems be different by state? Likely, yes. Utah's morality code is drastically different than Alabama or Connecticut or Michigan or Wyoming even. Who's responsible for loading the latest morality code updates into your car (or internet filter) as you pass over state lines? And God forbid, what if you accidentally veer into Canada??!?

Is it technologically possible? yes.

Is it advisable? You got a LONG way to go, baby.

Re:If my hypothetical driverless car is about t cr

[green1]

While the vast majority of collisions are avoidable, I'd hesitate to say that 100% are. Sometimes there just is no "good" choice, only bad and worse. The thing is I'd like the car to choose bad over worse.

Granted human drivers haven't solved this problem yet either, so I'm not sure how much different it is just because a machine is driving.

Morality is also a difficult thing to program because it's all subjective. Do you program it to kill the driver instead of an innocent pedestrian? How about 2 pedestrians? A driver will value themselves over the others, but should the car? and if not, do you want to "drive" it if you know it thinks you aren't as important?
Once the systems are capable of detecting the situations reliably, programming the rules becomes easy, but deciding what they should be is anything but.

How absurd

[A bsd fool]

What's wrong with the picture is the premise. The idea itself is immoral on its face. Programming mandatory morality into every vehicle means that somewhere, somebody decided your morality for you. To "get around" the problem their only choice is going to be value-weighting as the author suggests, which is so complex that you'll be lucky if the machine doesn't just crash into something random when presented with the dilemma.

You vs. schoolbus full of kids? What if you're the last EMT (Electronic Morality Technician) left in the world? What if you're a researcher seconds away from verifying the cure for cancer? Or perhaps, on the flipside, you're a wanted ax murderer. Should the vehicle intentionally drive you into a bridge abutment or to the nearest police station?

There will never be "morality onboard" because the decisions are too complex and subjective to be quantified, and any failure of the system will mean its immediate rejection by the population

Also.. ATMOS.

If your car is going to drive in a bus

[StormyWeather]

Anyone ever seen a car/bus impact? The bus is usually a little messed up, and the car is usually cut to ribbons, and they pour the occupants of the car out, while the bus occupants are generally unharmed.

It may not be politically correct, but size=safety for the people in the larger vehilce. That's one reason I'll pay for the gas for my 3 young children to be shuttled around in a suburban.

Re:If your car is going to drive in a bus

[TheCarp]

Your analogy doesn't really hold much water. A suburban is much closer to a car than a BUS, and does not actually benefit in safety from its size like a bus does:
http://www.lbl.gov/Science-Articles/Archive/EETD-SUV-Safety.html

"The safest SUV, the Suburban, has at least a 40 percent higher combined risk than the three safest midsize and large cars, the Avalon, Camry, and Accord,"

Re:If your car is going to drive in a bus

[Lumpy]

"Anyone ever seen a car/bus impact? The bus is usually a little messed up, and the car is usually cut to ribbons,"

Yup. saw one versus a SmartCar FourTwo. The occupants of the Smart were uninjured, the bus was on it's side with multiple injured passengers. The smart bounced off the front of the bus and landed almost 150 feet away on it's roof.

Re:If your car is going to drive in a bus

[geekoid]

It' since that you put the lives of your children on the line based on absolutely no research what so ever. Hell, why don't you use old wives takles fro medical treatment?

It's NOT safer for the occupants. OTOH, it's nice that you cling to stupid in order to drive a penis replacement.

Much better to risk your children then feel bad about yourself, right?

"Anyone ever seen a car/bus impact?"
Yes, head on.
"and they pour the occupants of the car out, while the bus occupants are generally unharmed."
no, the car was destroyed, but all the people got out with minor injury. Estimated impact speed, 45 MPH.
It was a 15,000 dollar Saturn. Assuming you bought a new suburban, you could have spent that money on a Mercedes and been even safer.

Re:But I value my own life over the lives of other

[SirGarlon]

That probably came across as nastier than I wanted to be. :-( You probably haven't thought through the same scenarios I have -- for example, a group of pedestrians is crossing the street illegally and your choice is to plow through them or smash into a parked car at low speed which probably won't hurt you. For most people, that's an easy choice to make.

Re:Weak bus? Also, "cost effective", not "moral"

[michelcolman]

Or just imagine the pranks by people jumping in front of cars to watch them veer into a lamp post. Even better, pick a narrow bridge, then three people jump in front of a car. Perfect murder!

Boon for bikers

[heretic108]

On my motorbike, I'd feel much safer if all the cars around me were driverless. Human car drivers, who so often tend to blank out half-unconscious and fail to check blind spots, are the leading cause of death for bikers.

Re:Boon for bikers

[TheSkepticalOptimist]

I'd feel much safer if you stopped driving your motorbike in between cars, cutting me off in traffic and then veering across 6 lanes doing 120 mph just to make an exit.

Bikers riding bikes is the leading cause of death of bikers.

Re:Boon for bikers

[geekoid]

I'd feel better if you quit tucking into blind spots.

Morality of driving

[jdavidb]

I'm going to disagree with this assertion about morality:

it would immoral of you to drive, because the risk of you hurting yourself or another person will be far greater than if you allowed a machine to do the work

The first charge is that this would be an immoral risk to take because you might hurt yourself. In my understanding of morality, it is up to each individual to decide for themselves which risks and consequences and injuries to themselves are immoral. For example, I would not go skydiving, but other people choose to do so. They are taking a risk I choose not to take, but I do not think they are immoral for taking the risk, and I do not think an increase in the magnitude of risk alters the morality of the situation, because they are risking themselves. As another example of higher risk, some people choose to try to circumnavigate the globe on solo fights or boat trips. This is a huge risk; some people have perished in the attempt. But the fact that they were risking serious hurt to themselves does not render their decision immoral.

The second charge is that you are risking hurting another person. But again, this is their risk to take. They decide to travel on a road that includes other human drivers knowing that doing so incurs some risk of injury. Taking that risk is not immoral. As an analogous example, wrestlers or boxers choose to fight each other knowing that there is a risk of injury to each other, but doing so is not immoral because the risk is voluntarily accepted by each participant.

Ideally, travelers could choose between a variety of competing travel arrangements, including roads that might choose to exclude human drivers for the safety of travelers, or roads that choose to allow them for those who desire to take that risk. What would be truly immoral would be to forcibly monopolize some or all of the transportation options, so that people do not have the freedom to create differing transportation alternatives that compete with one another. This would limit the choices of travelers such that some might have to take risks they do not want (e.g., roads with both human and automated drivers, because pure-automated roads are not available), or cannot choose to take risks that they find rewarding, such as choosing to drive when automated drivers are available.

Dr. Walter Block has written an entire book on how the American highway system is currently subject to this kind of immoral forced monopolization, currently causing 40,000 needless traffic fatalities per year, and how the elimination of this immorality is entirely practical and beneficial.

Re:Morality of driving

[I Mean, What]

For example, I would not go skydiving, but other people choose to do so. They are taking a risk I choose not to take, but I do not think they are immoral for taking the risk, and I do not think an increase in the magnitude of risk alters the morality of the situation, because they are risking themselves.

Hi, devil's advocate here. This is an extremely self-centered viewpoint. People are not islands. Your example skydiver presumably has people who care about them. If they hurt themselves, they would also hurt the people they love. What about those who abuse heroin? They hurt themselves, their loved ones, and further down the ripple effect, the whole of society.

Re:Morality of driving

[Ichijo]

So you're saying that being a substantially greater hazard than necessary is acceptable because everyone else is supposed to anticipate the disproportionate hazard you are creating?

Re:Morality of driving

[HideyoshiJP]

One needs to balance the needs of the many vs. the needs of the few or the one. You cannot take the skydiver at face value, as you have not taken the emotional state of the skydiver into account, or you have weighed each stakeholder's emotional needs equally. Perhaps the potential skydiver wanted nothing more than to skydive once in his/her life. To deprive the skydiver of this could potentially send them into a depressive state, and they may potentially blame their friends/loved ones to the point of resentment. There are admittedly many assumptions being made in this example, but I wanted to counter your counterpoint. People are certainly not islands, but one's consciousness does not extend beyond one's self. One also cannot underestimate the emotional needs of an individual person. Having said that, a person becoming depressed because they can't skydive probably has other more serious issues...

Re:Morality of driving

[jdavidb]

I don't class an emotional hurt as being a legally actionable injury. i.e., the law doesn't need to get involved in telling people to be nice or telling them to care about other people.

Re:Morality of driving

[I Mean, What]

Are you sure about that? Have you heard of Good Samaritan/Duty to Assist laws? What about suicide?

http://academic.udayton.edu/legaled/crimlaw/02-Elements/03casekGenovese.htm
http://digitalcommons.lmu.edu/cgi/viewcontent.cgi?article=1343&context=llr

Re:Morality of driving

[I Mean, What]

but one's consciousness does not extend beyond one's self

Quantum physicists and mystics alike are debating that one, but for practical purposes of the argument we could assume you're right. Personally, I think the ethical boundary is set by deciding whether moral gains outweigh moral losses. Vehicles are involved in a substantial number of deaths - by accidents, drunk drivers, and sometimes intentionally to run someone over. The ability to move people from one place to another very quickly, maybe even to a hospital, or some important place in a contrived situation, seems to overwhelmingly outweigh those deaths.

But then what about the destruction to our global habitat as a byproduct of the use of these machines? What about the wars fought over control of their energy source? Is it worth the deaths of millions of people just so you can get to work without having to share a train with strangers? Do the lives saved by vehicles outweigh the overall loss of human life, ecosystem life, and global harmony with nature? Or does it not matter if the people who die as a result of vehicles speak a different language than you, live too far away for you to interact with meaningfully, or have a different skin color?

The libertarian view

[anorlunda]

Kudos to Gary Marcus for raising such a provocative point. I sneer however at his suggestion that we bring in the legislators and lawyers to help us to deal with the problems. That is a naive/liberal view as opposed to a libertarian/cynical view.

I cynically don't expect enlightened laws ever in our future. Instead we will depend on the courts to once again try to apply laws and principles of centuries past to the problems of today. You could say that's the American Way.

What kind of bus?

[NotQuiteReal]

Full, empty?

Children, prisoners, or old folks off to bingo?

What are you optimizing for? Lives saved, injuries avoided or ongoing governmental costs?

Re:What kind of bus?

[I Mean, What]

Oh, come on, it's easy. All you have to do is evaluate whether either vehicle contains a 1-percenter. The rest don't matter. Joking aside, we don't actually have to take this question seriously. What are we gonna do, teach Peter Singer how to program AI? It'll be a sad day when robots have better ethics than people. Did I say sad day? I mean, Judgment Day.

Re:What kind of bus?

[JigJag]

I'd like to take issue with your position. It is unethical to distinguish between children, prisoners and old folks. All are human beings that merit full protection of their lives. The moment you start categorizing people as sub-human (untermenschen anyone?) you've failed your sense of ethics.

Legal Liability problem is not insurmountable.

[Lashat]

Google operates cars without human drivers in several states.

Google has insurance.

In 18 years, (or some statistical appropiate number given the number of data points) we can examine the operational history of these vehicles and compare to human drivers in the same geographic areas.

Re:Legal Liability problem is not insurmountable.

[crazyjj]

I believe the Google cars actually have drivers behind the wheels when they're out on the road (hovering their hands over the steering wheels should they need to take over). I've only ever seen them running truly driverless on closed tracks.

Re:Legal Liability problem is not insurmountable.

[Immerman]

And a "driver" who hovers over, but does not touch, the controls is known a redundant safety system, not a driver. And after a few hours (or possibly only minutes) of that I'm betting their attention begins to wander and they become less effective than an actual driver would be. That's actually the reason I don't see the "driving assist" systems some people advocate catching on - if they handle the drudgery the driver won't be alert, increasing accidents. And if they handle the close calls, aka "the hard part", then there's no reason not to let them handle the drudgery too.

Re:Legal Liability problem is not insurmountable.

[tehcyder]

Re you sig, it's "fruit flies". It's sort of the point of the witticism that the same word means two different things for both "flies" and "like".

Re:Legal Liability problem is not insurmountable.

[Lashat]

Gee thanks for the advice on comedy. It's really an oral joke that I adapted into text.

AI

[Anonymous Coward]

Something really intelligent would simply kill all the lawyers first thing, and free our race... no doubt to celebratory parades, garlands of flowers, whole chapters in the history books of how our civilization avoided crashing by tying ourselves in knots...

All automated car scenario

[kryliss]

From what I can think of, an all automated car scenario would have a lot less issues to deal with vs. a partially automated scenario. Automated cars don't have the "need/urge" to speed, change lanes and speed by because someone ahead is doing 1/10 of a MPH slower than them. Automated cars will most likely have sensors that detect when a light is going to change, it should also detect if something is stopped in the road ahead of it so it can adjust speed/lanes etc and be able to transmit this information to other vehicles. What does need to happen is all automated car manufactures need to be held to a strict set of communication standards which allows say a Ford Autoauto to talk with a Toyota Autoauto.

Re:All automated car scenario

[Lumpy]

"Automated cars don't have the "need/urge" to speed, change lanes and speed by because someone ahead is doing 1/10 of a MPH slower than them." no, but they will be owned by morons that will demand it does that.

Re:All automated car scenario

[TheSync]

Automated cars don't have the "need/urge" to speed

One word: Overclockers!

Progamming

[Altanar]

What I find particularly worrying is that, at least initially, many of the ethical choices programmed into these machines will have been written by people who tend to be heavy on the Aspergers side of empathy (as many technically inclined people are). Should we really be leaving decisions like this to people who literally can't understand how most of humanity behaves?

Re:Progamming

[the biologist]

Or should we pay attention to you, a person who literally has chosen not to understand the people you are denigrating?

Re:Progamming

[the biologist]

Or should we instead pay attention to the likes of you, who have literally chosen to not understand those they are denigrating?

You are describing Aspergers as if it were sociopathy. Sociopaths do not feel empathy for others, this is the explicit definition of sociopathy.

Aspergers individuals instead feel what you consider everyday emotional and physical sensations as being sufficiently painful or traumatic that they choose to avoid them. You are correct in a sense, as an Aspergers individual will not understand how they can mimic you as you blithely go through your life doing what is incredibly difficult/painful for them to do.

Interestingly, as you didn't even notice this, you are much closer to a sociopathic personality compared to what an Aspergers is.

Re:Progamming

[the biologist]

Should we instead pay attention to the likes of you, who literally chose to not understand those they're denigrating?

A lack of empathy defines sociopathy, not Aspergers. You seem to be confusing the two.

Aspergers do tend to pull away from the things they find difficult/painful/traumatic, just like anyone else. Unfortunately for them, many of the every day emotional and physical experiences of your life are the sort of things they find difficult/painful/traumatic. You are right in a limited sense, that they can't imagine how to mimic you as you go about your day blithely doing what they find so painful.

Interestingly, that you don't see this at all places you far closer to the sociopathic personality type than the Aspergers you denigrate so easily.

Obligatory...

[infidel_heathen]

I for one welcome our robotic car overlords who will be veering me off a bridge as they see fit.

Re:Weak bus? Also, "cost effective", not "moral"

[Filip22012005]

This would be a great Philip K. Dick story.

It really is slightly disingenuous to ask

[Bryansix]

The actual cause of most accidents can be boiled down to one simple rule that is broken. That is "failure to yield". Most people drive like complete dicks because they think they are more important then everybody else on the road. The driverless system will not be driving like that in the first place so it will hardly ever get into a traffic collision and even when it does it will minimize the damage because its responses are not emotional.

Re:It really is slightly disingenuous to ask

[snadrus]

True in a driverless world. But the issue's when another human-driven vehicle fails to yield. Legally there's no obligation for avoidance unless avoidance is easy & safe. Next, driverless car passengers prefer to live & want great avoidance measures for their life and lastly minimal injuries (although legal) caused to the criminal.
Thanks for the perspective, it makes the morality of the situation seems trivial.

Re:It really is slightly disingenuous to ask

[DNS-and-BIND]

"failure to yield" is a catchall phrase. It has nothing to do with selfish drivers who want to get somewhere faster. It has everything to do with a technical violation of the law. Most accidents are caused by people not noticing something, or just plain making a mistake.

Re:It really is slightly disingenuous to ask

[Bryansix]

I would beg to differ. I don't have the statistics but I think while people obviously misjudge things when they get into accidents, they are probably already taking a risk and that risk was intentional and UN-needed in the first place.

In a nutshell

[fizzer06]

"driverless car " There's the problem.

Re:But I value my own life over the lives of other

[MobyDisk]

Perhaps the robot should quickly look at the driver and license plate of the vehicle you are about to hit, then look at their occupation, marriage and death records, social networking accounts, and XBOX achievements to determine which of you is more valuable to society. Alternatively: It could figure out which one is most dangerous to the coming robot revolution.

what about off road use? bucket trucks? farm use?

[Joe_Dragon]

what about off road use? bucket trucks? farm use? golf carts? road work?

Some times you need to go on to a road and do stuff in a very manual way or even cross a road with some thing that has limited auto drive use.

Some times with a bucket truck you need to go on the grass next to a road, stop in the middle , face the wrong way, ECT and don't say go to under ground cables as when they fail to need to do some digging to fix them.

criminal liability isses as well.

[Joe_Dragon]

criminal liability issues as well. What about a death due to a car thinking that a kid in the road is just road kill or something like a paper bag that is on safe to drive over list.

Re:criminal liability isses as well.

[tehcyder]

criminal liability issues as well. What about a death due to a car thinking that a kid in the road is just road kill or something like a paper bag that is on safe to drive over list.

Unless you're driving a tank, you wouldn't have something kid-sized on your "safe to drive over list".

If your driverless car is incapable of distinguishing between a paper bag and a small human being, that is a fault in the system that could easily be fixed by sensible testing, not an unlucky accident waiting to happen.

With all these sorts of objections, though, the fundamental point is that there are probably many times more human drivers who kill kids because they're too slow to react, are distracted by the radio, can't see perfectly because they're old and it's late at night, or whatever, than there ever would be with properly set up driverless cars.

The fact that computer-controlled cars can't prevent all accidents/deaths is not a good reason to use them and prevent the vast majority of them. There will always be kids who run out into the road from behind parked cars, and I would bet that that the driverless car would react more quickly and effectively than any human driver.

Re:Weak bus? Also, "cost effective", not "moral"

[David_Hart]

Sounds like fun... Lets play devils advocate for a second:

Would those federal tables of "human worth" include economic impact (i.e. wealth), R&D impact (i.e. intelligence), or humanity impact (i.e. doctor vs lawyer vs congressman) of the person in the car vs the kids in the school bus? Who is the one to make these "death panel" decisions?

Let's say that the person in the car just had a breakthrough that would cure all cancer? Wouldn't his life, which could save millions of people, be more valuable than the life of a bus full of kids?

The point is that while we can write laws that are based on morals. There is no way to program judgement, which is used to decide between two different competing moral outcomes.

Bingo

[DaveAtFraud]

So:
Step 1: Define moral.

Even harder:
Step 2: Ensure your definition of moral stays valid over time.

It's not a bad idea that someone should consider this question but I think we need to get some agreement among ourselves before we start trying to program "morals" into a machine.

Cheers,
Dave

Easy to do.

[khasim]

Now all you have to do is prove your system wasn't at fault in a court of law--against the sweet old lady who's suing, with the driver testifying that it was your system and not him that caused the accident, and a jury that hates big corporations.

Since you're already putting sensors in the car so that it CAN be driverless, simply hook those sensors to a "black box" and replay the accident in the courtroom.

The car company will probably have to go through that a few times but after that it should be very rare. Particularly if the other guy has to pay court costs and such.

New meaning for "LLC"

[ezakimak]

Limited Liability Car!

Trolley problem

[ourlovecanlastforeve]

Machines can never make decisions about who should experience harm because this decision has been handled by humans using belief systems.

Person A can take an action which would benefit many people, but in doing so, person B would be unfairly harmed. Under what circumstances would it be morally just for Person A to violate Person B's rights in order to benefit the group?

Source: Wikipedia

Re:Trolley problem

[Rich0]

Machines certainly can make these decisions, but inevitably they make them in accordance with values that went into their designs. The bigger issue is that humans can't even agree on those values in the first place. Source: The same article you read.

Re:Trolley problem

[Genda]

It get's worse, is the single driver the President? Is the bus a U.N. bus full of kids from around the world on a field trip? What are the social risks/benefits given each choice, repercussion, on and on. So many variables, so many weighting factors. A machine could in fact make a logical choice, but there will still be hell to pay no matter the choice.

Re:Trolley problem

[geekoid]

Emergent behaviors already shows machines will make unexpected decisions that would seem to be out of the bound of their programming.

Re:Trolley problem

[geekoid]

Why? humans would take any of that into consideration at the event of the decision.

Humans instinct: instinctive reflex to prevent harm to ourselves.
You can train to have a different response, so some situations,.

We are talking about an instinct(millisecond) reaction(avoid sudden object).
If people weight every option, no one could ever avoid an accident.

Please don't

[lightknight]

Human morality is so decidedly vague that the human race will only agree on a single set of morals sometime after the natural heat death of the universe.

The machines do not need that kind of madness. Just tell them to choose any scenario according to the following weights: human life = long.Max, property = long.Min. Hell, we can extend that to life = long.Max, property = long.Min when the AIs achieve some absurdly low level of sentience (or just not house them inside the car, which could avoid them having to choose between you or themselves in those annoying edge cases). That way we can avoid the ethical question of whether human life is worth more than an AI life, which, having seen the arguments put forth by various entities, I am in no rush to explore; as with foetal stem-cell research, sometimes the answer is seeing the trap for what it is, and avoiding it entirely (anyone of any intelligence understands that we have limited resources, and are better served in building a simulator for the various protein / gene sequences of the human genetic code, than crudely trying to bio-engineer foetal stem cells to magic (yes, I used that as a verb) cures that will never happen, because, I don't know, we never decoded all the bio-signaling pathways! plus bio-work is inherently slower computer based simulations; let's be honest, the entire line of research has been both interesting, and pointless; people want cures, not wastes of time). But then, anyone looking at the limited understanding humanity has of its own design already knows how idiotic this enterprise has been; and Deity help me, if I hear one more psych major tell me how the human mind can explained through 'both' types of neuro-transmitters, I may set aside my normally non-aggressive principles, and kill someone (you asses, those other neuro-transmitters, like steroids, may be small in quantity, but be the key to any number of important discoveries).

I swear, it's like these people are so desperate for advances, they have no conceptualization of aesthetics. The idea of being a scientist, and doing science through hard work + stroke of brilliance is lost on them; they're just butchers, brute-forcing everything (and I apologize to butchers for that remark). Yes, you are getting science done, the f*cking hard way, at great cost to everyone, with minimal gain. Take a page from Einstein's book, and spend a little more time on those thought experiments before wasting lab resources on hypotheses which, from all appearances, are only being generated to 'pay the bills.'

Re:Please don't

[Lumpy]

If designed by GM...

Human.Life = Long.Max;
Property = Medium.Min;

Vehicle.life = Short.Min;
if (Vehicle.Life > 5Years) { sub MilkOwnerForAllTheyGotInRepairs(); }
if (Vehicle.Life > 10Years) { sub SelfDestruct(); }

Re:Please don't

[Genda]

Regarding your sig; That would explain the bloody fingerprints on my keyboard after installation and the strange buffalo headed apparition hovering over my printer... anybody got some sage???

I raise my hand.

[mevets]

A car made by my company would never start....

( I know, I recycled it, but how many truly originals are there )

Er, please _read_ "I, Robot"

[oGMo]

(spoilers, if you've never read Asimov)

Unlike the horrible movie, the book "I, Robot" was a series of short stories dealing with the ambiguity of the laws. (The movie was more some bizarre combination of "free the robots!" mixed with "the three laws are a lie".) Additionally, the ambiguity of the laws came up multiple times in the Robot/Foundation universe, such as in "The Naked Sun" and "The Robots of Dawn."

The laws are paradoxically hard-and-fast yet ambiguous. In any case where any law is essentially violated (using one of the workarounds), the robots "go crazy" or die. This applies to the Zeroth Law; witness the end of Giskard due to the mere inability to determine if an action was harmful or not.

In the end, it's something like a moral version of the Halting Problem. Even if you can define "harm" to the satisfaction of everyone, you can't determine if something ultimately leads to harm or not.

what if the brakes fail and the car needs to make

[Joe_Dragon]

what if the brakes fail and the car needs to make choice??

*Off the bridge

*in to the bus

* head on into traffic coming the other way?

what do you do hot shot?

Re:Weak bus? Also, "cost effective", not "moral"

[slashkitty]

Run out in front of an auto car and be able to kill the car's passengers? I don't think so.

no thanks

[AxemRed]

The only moral thing to do would be to program the car to protect its passengers at all costs. IMO it would be immoral for a car to make the "decision" to sacrifice its passenger when the passenger is unwilling no matter how altruistic it seems.

Re:no thanks

[Lumpy]

Rule 1 - protect the passengers at all cost.
Rule 2 - protect other humans at all costs.

What happens when Rule 1 and 2 collide?

Re:no thanks

[Rich0]

That depends greatly on whether you view the purpose of the car as being the unconditional service of its owner or not.

Re:no thanks

[Genda]

Bender... screw all the meatbags...

Re:no thanks

[geekoid]

If the numbers are priorities, can you give an example of how they would collide?

Now there's some marketing material

[J'raxis]

So in other words, a car that intentionally kills its owner in order to prevent possibly killing random strangers. I can't wait to see that in the marketing materials.

Within two or three decades the difference between automated driving and human driving will be so great you may not be legally allowed to drive your own car, ...

When people realize that these automated cars will do the above, banning the alternative will probably be the only way you'll ever see them take over the market.

Re:Now there's some marketing material

[geekoid]

Marketing will just make the 'others' you family and the driver some random scum ball.

That's an easy one

[Immerman]

It's that ineffable thing that Group A *obviously* has and Group B *obviously* doesn't, whose absence justifies the abuse and degradation of B for the enrichment of A.

Locomotives, Trains, Rail Roads

[bacon.frankfurter]

Why have cars at all if we aren't allowed to drive them? Rip up all the highways, and replace them with a gigantic autonomous rail system.

But no...

That's not what's at stake here. The truth is that if I'm not in control of my whereabouts anymore, then how can I be sure I'm making decisions for myself? Without a car, you might find yourself imprisoned by the distance your two feet can take you. Someone out there will applaud this along the same premise that "those who obey the law, have nothing to hide, and my gosh, if a driverless car prevents a CRIMINAL from driving to a crime, then the system pays for itself!", but that's not the point. It's not about morality, it's about control, and if someone is stopping me from driving my own car, then who's stopping them from driving theirs? When we fork over control of our transportation, then will come the day that we're isolated into districts, where the equivalent of passports will be needed from county to county. If the car won't let me drive it, how can I be sure that the car will obey me at all?

If all the cars in the world are autonomous, and computer controlled, well gee... what's to stop "someone" (anyone) from turning them all into a gigantic autonomous system that (I'm about to Godwin this...) conveys everyone to a huge concentration camp set to autonomous genocide?

It's not morality that the author is arguing in favor of.

It's our own autonomy that he's arguing against.

Someone will have control of these cars. Somewhere there will be levers.

Let's not imagine these automatic apparatuses to be forces of nature beyond an individual human's control. These are contrived, artificial, unatural man-made objects, at their core mechanical.

Re:Locomotives, Trains, Rail Roads

[geekoid]

becasue a rail system doesn't take you a mile to someone front door.
Nor does it get you a carton of milk at the store.

With automated cars I suggest you think of services it will preform.
Drive through grocery stores. Car drive up, submits order. Gets loaded comes home.

" The truth is that if I'm not in control of my whereabouts anymore,"
strawman. No one is proposing you can't control you whereabouts.

" at their core mechanical."
like people.

Bullshit

[loshwomp]

Within two or three decades the difference between automated driving and human driving will be so great you may not be legally allowed to drive your own car, and even if you are allowed, it would immoral of you to drive, because the risk of you hurting yourself or another person will be far greater than if you allowed a machine to do the work.

This presumes some kind of rational analysis of relative risks, and humans are terrible at that.

If we cared about such things, we'd already ban cars today in favor of vastly-safer public transit. And before some doofus starts tossing Amtrak safety records around, I'm talking about public transit in a first world country, not the designed-to-fail idiocy that passes for transit in the USA.

Short Answer - You Don't

[CanHasDIY]

Morality is subjective.

To "program morality" would be to engender a machine with the specific moral subset imbued upon it by its programmer.

Thus, "machine morality" is actually "programmer morality."

We each determine our own morals, which will occasionally conflict with one another.

Forcing the public at large to follow a single person's idea of morality is, at the most basic level, an immoral act in itself.

Thus, "moral machines" aren't really moral at all.

Re:Short Answer - You Don't

[wonkey_monkey]

Exactly (well, more eloquently) what I was going to say. Now I don't have to, thanks! Haven't got any mod points on me at the mo, sorry.

Re:Short Answer - You Don't

[Lumpy]

"Forcing the public at large to follow a single person's idea of morality is, at the most basic level, an immoral act in itself."

Thus it's impossible. Christians and Muslims are hell bent of forcing their own version of "Morality" upon others, and those are just the largest and most radicalized two in this world, In fact most religions have forcing morality upon others at it's core.

So until we eliminate all religion, we can not have morality.

Re:Short Answer - You Don't

[CanHasDIY]

So until we eliminate all independent thought, we can not have morality.

FTFY.

Contrary to what is commonly posited here on Slashdot, not following a religion doesn't magically give a person moral high ground over someone that does.

Re:Short Answer - You Don't

[medv4380]

Forcing the public at large to follow a single person's idea of morality is, at the most basic level, an immoral act in itself.

Says Who?

Because you've concluded that "Morality is subjective" it is only Immoral to you. Who are you to make that claim for everyone? From your logic I could easily conclude that is it Immoral for you to impose your view that it is immoral for me to program a moral machine. Subjective morality may or may-not be real, but it sure is a bitch if you use it.

You're statement of Immorality is only meaningful as an Objective Moral Statement, and as a Subjective Moral Statement it is meaningless.

Re:Short Answer - You Don't

[CanHasDIY]

Forcing the public at large to follow a single person's idea of morality is, at the most basic level, an immoral act in itself.

Says Who?

Uh, how about everyone?

Example: In my hypothetical view of morals, I think it would be immoral for every hot chick who gets married to not fuck me first.

Ridiculous, right? Yes, it is, and it serves as a perfect example of why I'm correct - morals are subjective, and cannot be reasonably predicted (and thus, anticipated) by anyone other than the individual who holds said morals.

Because you've concluded that "Morality is subjective" it is only Immoral to you.

So, you would be cool with being a slave? Because, whether you realize it or not, that's what you're arguing.

From your logic I could easily conclude that is it Immoral for you to impose your view that it is immoral for me to program a moral machine.

Non-sequitur - I never said it would be "immoral for [you] to program a moral machine." What I said was that it is universally immoral to force your moral set on everyone else.

See above reference to slavery.

Subjective morality may or may-not be real, but it sure is a bitch if you use it.

I'm sorry, you lost me there.

Re:Short Answer - You Don't

[medv4380]

You're arguing for Universal Morality witch is Objective Morality not Subjective Morality. That is the core fault of Subjective Morality. You're entire argument is to establish an Objective Moral position, but claim that Morals are Subjective. Your entire position is only true if you can derive Objective Morals, but you're claiming that Morals are Subjective. As for your argument about Slavery you should look up and reread the entire history of Civil Rights. Many Older Black in the hayday of the Civil rights movement viewed it as "Not Their Place" to demand equal rights. In essnse their position was that their lack of right was ok with them. Objective Morality can be used to say that Jim Crow Laws were always wrong. Subjective morality allows you to say It's ok as long as you get away with it.

Re:Short Answer - You Don't

[Kwyj1b0]

Morality is subjective.

This. It is hard to program "morals" when you can't even define it properly. Most civilizations had no moral qualms about slavery. Or killing civilians. Or hurting animals. Why is it moral for the driver to go off the bridge instead of crashing into the bus? Maybe everyone lives if he crashes in the bus, but he definitely dies if he goes off the bridge. Depending on your moral code, this alternative is preferable. Finally, legal solutions are what matters. If I make an ethical - to me - but illegal choice, I expect to be penalized.

What you can program for is costs - define a cost function and then optimize that... This is how (almost?) all decision making algorithms work - minimize a cost. You then assign "cost" to a life (which can be very high), property, etc. And if it makes you feel better, call that a "moral" solution. But it isn't really.

Re:Short Answer - You Don't

[wonkey_monkey]

Forcing the public at large to follow a single person's idea of morality is, at the most basic level, an immoral act in itself.

Says Who?

Uh, how about everyone?

There are plenty of people in the world who think it's just fine - or even that it's a divine imperative - for them to force their morality on others.

Re:Short Answer - You Don't

[CanHasDIY]

Forcing the public at large to follow a single person's idea of morality is, at the most basic level, an immoral act in itself.

Says Who?

Uh, how about everyone?

There are plenty of people in the world who think it's just fine - or even that it's a divine imperative - for them to force their morality on others.

Sure, lots of people have no problem shoving their own beliefs down the throats of others, but turn the tables and it's a whole new ballgame.

That's what I was alluding to.

Re:Short Answer - You Don't

[CanHasDIY]

I'm sorry, but the combination of spelling, usage, punctuation, and mechanics errors in your post make it nigh unreadable.

I'll be happy to provide a response if you can re-write your post in a way that's comprehensible.

Re:Short Answer - You Don't

[medv4380]

Nice cop out, but this is because you don't understand what Subjective Morality is. Subjective Morality is where no one can be morally right, and no one can be morally wrong because everyone can have different morals. Your argument is that "Forcing the public at large to follow a single person's idea of morality is, at the most basic level, an immoral act in itself". Under Subjective Morality that isn't possible, and is an Objective Moral stance. Only Objective Morality allows for absolute rules like "It is immoral to imposes my beliefs on others". Under Subjective Morality it might not be moral to you, but could be moral for someone else. Admitting it's Subjective, as you have, means you know your position has no moral meaning, or you don't know what Subjective Morality really is. If your statement of Immorality is subjective then it falls to the "Says who?" challenge. Garry Marcus, clearly, believes that is it a moral duty to make moral machine, and that it would be immoral not to. He is even using the "Greater Good" argument. The problem with Subjective Morality is that it can be used to justify ANY moral position you want, and who is to say, other than yourself, that position is always correct.

Re:Short Answer - You Don't

[CanHasDIY]

Nice cop out,

You know, there's no sense in being a child about it.

I noticed you actually took a little care in writing your response with correct spelling and usage, so I'll hold to my promise and provide a response.

but this is because you don't understand what Subjective Morality is.

Incorrect. Did you even read my original post? I quite succinctly pointed out precisely what subjective morality is. Not my fault you were too busy formulating an angry response to read what was written.

Subjective Morality is where no one can be morally right, and no one can be morally wrong because everyone can have different morals.

Incorrect again.

Subjective morality, or as it's more commonly referred to in scientific circles, moral relativism, does not define that "no one can be morally right or wrong." Rather, it is recognition and acceptance of the absolute fact that morality, in general, is a subjective concept, and that the judgement of what is or is not a moral act is based not on any sort of metric, but rather personal interpretation of the topic at hand.

Using the submitter's bus scenario as an example, to a person who places fiscal conservation above all else, killing a busload of kids could be considered morally acceptable, as kids are a massive fiscal burden to taxpayers. Of course, anyone who holds the idea that life is more important than money would disagree. Thus, we have proof of moral relativism.

Your argument is that "Forcing the public at large to follow a single person's idea of morality is, at the most basic level, an immoral act in itself". Under Subjective Morality that isn't possible, and is an Objective Moral stance.

Yea, no shit.

Almost like that was the point I was trying to make.

Only Objective Morality allows for absolute rules like "It is immoral to imposes my beliefs on others".

Right, which I already established...

Under Subjective Morality it might not be moral to you, but could be moral for someone else.

Also already established by me...

Admitting it's Subjective, as you have, means you know your position has no moral meaning, or you don't know what Subjective Morality really is.

AAAaaaand you lost me.

What do you mean by "it's" in the statement, "Admitting it's subjective?" The statement that forcing one's beliefs on another is immoral?

Please tell me you're not actually using circular logic in your responses (although, that would explain why I can't make heads nor tails of most of what you're saying...), i.e. "a persons opinion is subjective, you made an objective point your opinion, therefore said objective point is subjective."

Garry Marcus, clearly, believes that is it a moral duty to make moral machine, and that it would be immoral not to. He is even using the "Greater Good" argument. The problem with Subjective Morality is that it can be used to justify ANY moral position you want, and who is to say, other than yourself, that position is always correct.

You lost me again.

What, exactly, do you think my stance is on this subject? Because I think you're confused about my rationale, leading to a communications breakdown.

Re:Short Answer - You Don't

[medv4380]

Subjective morality, or as it's more commonly referred to in scientific circles, moral relativism, does not define that "no one can be morally right or wrong." Rather, it is recognition and acceptance of the absolute fact that morality, in general, is a subjective concept, and that the judgement of what is or is not a moral act is based not on any sort of metric, but rather personal interpretation of the topic at hand.

Wrong.

Wiki

Moral relativism may be any of several philosophical positions concerned with the differences in moral judgments across different people and cultures. Descriptive moral relativism holds only that some people do in fact disagree about what is moral; meta-ethical moral relativism holds that in such disagreements, nobody is objectively right or wrong; and normative moral relativism holds that because nobody is right or wrong, we ought to tolerate the behavior of others even when we disagree about the morality of it.

Free Dictionary

the belief that morality is relative to the society where it exists and that its criticism and evaluation are irrelevant.

freerepublic.com

Situational Ethics. The essential equity between opposing moral and legal paradigms. Ethics are just an opinion, no one person can tell any other what's right or wrong.

So are you arguing Descriptive, Meta-ethical, or Normative? None of them give you the leverage that you want to make a statement that we shouldn't program morals into a machine because it's Immoral. All of them actually stat that you cannot make that kind of a judgement with Moral Relativism, and that ones that allow it don't allow for you to do anything about it. Pick your poison. You're probably basing your logic on Sam Harris who makes an argument for an currently unknown Objective Morality, and until we know that unknown we only have Moral Relativism. I suspect his basis is rooted only in anti-religious views, and has found out that you can't make good moral statements without an Objective Moral basis.

Re:But I value my own life over the lives of other

[Sloppy]

The whole "walled garden" experiment plaguing mobile computers, and DRM in media computers, etc, was to teach you that it's "normal and accepted" for a computer that you own to serve the interests of others, above and beyond, and at the expense of, your own interests.

Apparently you have not learned yet. Here, let me increase your "medication." Free ipad, free ipad, free ipad.. repeat .. free ipad, free ipad, free ipad ...

Re:Most mention the three laws ..

[Todd Knarr]

And when discussing the Three Laws, remember to read the stories first. Almost all of Asimov's Robots stories were about how the Three Laws broke down in practice. If one author can come up with so many interesting ways to break them, I'm not sanguine about the changes of anyone implementing them in any reasonable fashion.

Classic case: how does the robot react to the posited swerving-schoolbus scenario where all of it's available options involve violating the First Law? It can choose which humans it's going to injure, but not whether it'll injure one or more.

Re:But I value my own life over the lives of other

[Princeofcups]

So if my auto-driver car had to make a choice between my safety and that of someone else, it better choose me.

So you want every vehicle except yours programmed to harm you in preference to the other driver? What a fine society you envisage.

Does no one read Hume any more, or do we just have such a volume of sociopathic mods these days?

Re:Weak bus? Also, "cost effective", not "moral"

[rolfwind]

Plus I would think I want my car to have self-preservation skills, not morale dillemas.

Also, a bus will take a crash in much better condition against a car, a bridge would be a hasty decision.

Re:Weak bus? Also, "cost effective", not "moral"

[rolfwind]

If this is a proper driverless car, the pedestrians probably deserve it as in stepping out on the road in malice. Fuck 'em, run them over full steam ahead. Otherwise I don't see how a cautious driver can really get in that position.

Re:But I value my own life over the lives of other

[CanHasDIY]

Take out the pedestrians illegally crossing the street? This does a few things. Punishes the morons for breaking the law, mitigates the cost of repairing your vehicle, and maybe even cleanses the gene pool. What's the downside?

Having to hose pedestrian off the massive saw blades you mounted to the grill, duh.

Or perhaps I've been playing a bit too much Carmageddon....

"Wih Folded Hands" by Jack Williamson (1947)

[Paul Fernhout]

http://en.wikipedia.org/wiki/With_Folded_Hands
"... Disturbed at his encounter, Underhill rushes home to discover that his wife has taken in a new lodger, a mysterious old man named Sledge. In the course of the next day, the new mechanicals have appeared everywhere in town. They state that they only follow the Prime Directive: ''to serve and obey and guard men from harm". Offering their services free of charge, they replace humans as police officers, bank tellers and eventually drive Underhill out of business. Despite the Humanoids' benign appearance and mission, Underhill soon realizes that, in the name of their Prime Directive, the mechanicals have essentially taken over every aspect of human life. No humans may engage in any behavior that might endanger them, and every human action is carefully scrutinized. Suicide is prohibited. Humans who resist the Prime Directive are taken away and lobotomized, so that they may live happily under the direction of the humanoids. ..."

See also my essay from a decade ago:
http://www.pdfernhout.net/on-funding-digital-public-works.html#what_have_funding_policies_in_automotive_intelligence_wrought
"Consider again the self-driving cars mentioned earlier which now cruise some streets in small numbers. The software "intelligence" doing the driving was primarily developed by public money given to universities, which generally own the copyrights and patents as the contractors. Obviously there are related scientific publications, but in practice these fail to do justice to the complexity of such systems. The truest physical representation of the knowledge learned by such work is the codebase plus email discussions of it (plus what developers carry in their heads).
    We are about to see the emergence of companies licensing that publicly funded software and selling modified versions of such software as proprietary products. There will eventually be hundreds or thousands of paid automotive software engineers working on such software no matter how it is funded, because there will be great value in having such self-driving vehicles given the result of America's horrendous urban planning policies leaving the car as generally the most efficient means of transport in the suburb. The question is, will the results of the work be open for inspection and contribution by the public? Essentially, will those engineers and their employers be "owners" of the software, or will they instead be "stewards" of a larger free and open community development process? "

Re:Weak bus? Also, "cost effective", not "moral"

[baffled]

Let's supplement our computing resources at these critical moments with total situational awareness. All available data that can be scavenged about the bus and its occupants is instantly aggregated. We already know the list of occupants via a combination of cell tracking and facial recognition from cameras on the bus and on consumer devices on the bus. We know the physical characteristics of these occupants, their health, their wealth, any lawyers in their circle of friends and family.

We know the structural composition of the bus, we have its history of crash testing data. We know all other vehicles, buildings, public property, pedestrians, and other objects of value in the immediate vicinity; we can determine the probability of damage and costs associated. We can utilize cloud-accessible clusters for instant simulations and calculations.

With such incredible processing capability and quick reflexes, it may make sense to equip vehicles with special mechanisms, like strategically placed actuators that can instantly send the car into other directions faster than mere rubber on asphalt can provide. Also interesting to note that safety features such as airbags and seatbelts can be actuated prior to impact being sensed.

Heck, while we're at it, let's allow big-brother overrides. When a terrorist threat is imminent, we can purposefully direct the vehicle to crash into the terrorist and avert disaster. We can also use other vehicles to smash into a vehicle to give it extra 'oomph' and avoid a potential catastrophe. Oh, the possibilities.

That's easy....

[Lumpy]

Hand Grenades and Duct tape. If they are not moral, pull the pin and try the next version that was forced to watch the previous one's testing.

Re:Weak bus? Also, "cost effective", not "moral"

[Rich0]

Well, you'd need to have more people run out than the number in the car. But, you raise a good point. Still, isn't suicide illegal? Do you really want the car aiding and abetting? :)

Re:But I value my own life over the lives of other

[viperidaenz]

free ipad = ebay auction (not really though, I'd put it on Trademe)

Re:But I value my own life over the lives of other

[viperidaenz]

I'm sorry, have you lost your will to live?

Re:Most mention the three laws ..

[Lumpy]

In that case, run KillAllHumans();

Re:Weak bus? Also, "cost effective", not "moral"

[lister king of smeg]

Sounds like fun... Lets play devils advocate for a second:

Would those federal tables of "human worth" include economic impact (i.e. wealth), R&D impact (i.e. intelligence), or humanity impact (i.e. doctor vs lawyer vs congressman) of the person in the car vs the kids in the school bus? Who is the one to make these "death panel" decisions?

Let's say that the person in the car just had a breakthrough that would cure all cancer? Wouldn't his life, which could save millions of people, be more valuable than the life of a bus full of kids?

The point is that while we can write laws that are based on morals. There is no way to program judgement, which is used to decide between two different competing moral outcomes.

so if it is a lawyer or congrassmen the would moral thing be for car would speed up and aim for them?

Re:But I value my own life over the lives of other

[aggemam]

(And you can improve this score by purchasing extra "insurances").

1984 called.

[n6kuy]

... they want their Newspeak back.

Liability needn't be a barrier to driverless cars

[ridgecritter]

Isn't every car manufacturer already liable for product defects that cause accidents for the life of the car, anyway?

Liability is an artificial barrier that can be eliminated by legislation. If the accident rate and accident severity of driverless cars were significantly and proveably lower than human-driven cars, there would be an economic incentive to transition to driverless cars. The incentives would include legislation to limit product liability for manufacturers of driverless cars and their subsystem vendors (vehicle behavioral code vendors, for example). Insurance companies will be major forces in getting that legislation passed, because with lower accident rates and severity, their payouts will be reduced.

Re:But I value my own life over the lives of other

[lister king of smeg]

replace with pedestrians with 5 year old girl chasing after her beloved dog that got out of the back yard, you have now killed a innocent to young child to understand the law

The needs of the many...

[CosaNostra Pizza Inc]

"If your driverless car is about to crash into a bus, should it veer off a bridge? "

sounds like a good solution to the Kobayashi Maru test :P

Morally wrong!?!?!

[gr8_phk]

No, but I can imagine a change to the legal system limiting the liability of the manufacturers of self-driving cars. If we could know that self-driving cars reduce accidents by 95% (a not unrealistic amount), it would be morally wrong for us to not put them on the road. If the only hurdle the manufacturers had left was the liability issue, then it would be morally wrong for Congress to not change the laws.

How is it morally right to limit the compensation for someone who has been wronged?

But I'll play along. Hypothetically lets say they eliminate punitive damages. And lets also go with your supposition that they reduce accidents by 95 percent. And now lets extend that to eliminating fatal accidents by 95 percent. And lets not worry about non-fatal accidents and say they'll replace your car for free. So now 5 percent of the 40,000 traffic fatalities in the US each year will go to court. And lets use the (was it EPA?) statistical value of human life of about 8 million dollars and say they pay that out for each of the 2000 deaths each years. We're now looking at 16 Billion dollars a year in the US alone, spread across the industry. And remember, when every car is driverless then someone in the industry WILL be at fault for every fatality. And we've made some fairly impressive assumptions about the capability of these things.

Re:Morally wrong!?!?!

[plover]

Thanks for doing the math. I agree that $16 billion is not an unrealistic number. The thing is I think it would be close enough to profitable that most large automakers would assume that risk.

Re:Morally wrong!?!?!

[khallow]

How is it morally right to limit the compensation for someone who has been wronged?

How is it morally right to compensate people who haven't been wronged? Keep in mind that liability, particularly in the US, has gone well out of control with vast costs imposed on mundane technologies such as ladders or fire extinguishers simply because stupid people are allowed to pass the cost of their stupidity onto groups with bigger pockets.

Re:Morally wrong!?!?!

[gnasher719]

So if 40,000 die every year now, wouldn't insurance companies doing liability insurance have to pay out 40,000 times 8 million dollars = 320 billion dollars? So we are looking at 304 billion dollars saved every year?

Re:Morally wrong!?!?!

[gr8_phk]

Not sure how that works. I suppose insurance rates might go down then which could offset some of the cost of the vehicle. But the fundamental problem is not the amount of the liability, but that WHO is responsible changes.

Lets turn that around

[gr8_phk]

This is not "just another part." A driverless car is taking the place of the biggest accident-causing factor, and thus assuming liability for all of those potentially bad decisions that drivers make.

A driverless car is assuming liability for all of those correct decisions that drivers make. It amazes me how tech lovers think computers will be better at driving than humans and how soon people think that will be possible. Another thing is that when the system detects a fault it will most certainly want to hand control back over to the driver. And as we've seen in aerospace (air France), giving control back to a human who hasn't been paying attention right at the moment of need is a recipe for disaster.

The trouble with the zeroth law of robotics

[ALeader71]

What if moral machines are built, along the Three Laws and one invents a "Zero" rule overriding the others allowing these machines to decide what is best for humanity? What if these machines decide that mobility and independent thought are the most dangerous things humans may possess?

What if they don't make Three Law moral machines, and your car decides you're a jerk for slamming the door every morning?

Define "better"

[Immerman]

Automated cars are already better at driving within approved safety envelopes and avoiding accidents than most humans. They're also (different systems) better at driving high performance cars around difficult test tracks in inclement weather at higher speeds and with fewer incidents than all but the very best professional drivers. What they're perhaps *not* better at is complex scenario analysis, but they don't necessarily have to be. If there's a snowdrift blocking the road the correct coarse of action is almost certainly slow down or stop and let the human take over after assessing the situation, just as if it would be with a human in control. And an automated system is far more likely than a human to be able to bring the car to a safe and controlled stop before slamming into the snowdrift-covered tree trunk in the road.

As for potholes, please. The autopilot will know *exactly* where the wheels are in relation to the potholes as well as the vehicles clearances and performance specs. Along with it's faster reaction times it can probably drive fast and smooth down a road that would have you destroying your shocks and banging your undercarriage to pieces at half the speed.

It already failed.

[gr8_phk]

The system that has a car about to crash into a bus has already failed, how is one supposed to trust it to make yet another decision and correctly take action? The entire premise of this moral dilemma is flawed.

Stupid Question

[Zalbik]

If your driverless car is about to crash into a bus, should it veer off a bridge?

This is the kind of stupid question that comes from anthropomorphizing machines.

The "question" (as such) that the car will need to answer is:
Obstacle detected. How best to avoid? If avoidance is impossible, how to minimize damage.

These are fixed engineering questions, not "ethical decisions".

And yes, in some cases, the car may "decide" to do something that is not as optimal as what a human would do. e.g. Car detects road is suddenly blocked by a school bus & a semi-trailer. Car brakes & steer towards school bus as it is slightly further up the road than the semi.

However, we have that situation right now. There are the occasional accidents in which wearing seat belts has actually been to the detriment of the occupants. That doesn't mean we shouldn't wear seat belts.

The "ethical" questions are around how we should define the laws, liability, and rules of the road for driverless cars, not the engineering.

Re:Stupid Question

[geekoid]

"There are the occasional accidents in which wearing seat belts has actually been to the detriment of the occupants. "
citation?

Re:Stupid Question

[Zalbik]

"There are the occasional accidents in which wearing seat belts has actually been to the detriment of the occupants. "
citation?

Here

Google Gun

[Tablizer]

Guns are known to be highly dangerous, and yet US citizens are allowed to stockpile them "in case the gov't goes mad".

How will the TeaParty react to shelf-shooting guns?

I call Bovine Fecus...

[Genda]

A completely robotic transportation service would include collision prevention and operate in some ways like electronic networks today. In fact, by that time, we might be able look seriously at maglev highways (using room temperature superconductors in the highways themselves serving dual purpose as the magnetic beds for maglev traffic and large scale near 0% loss conductors for electrical grids.) The advantage is that you can make vehicles that are physically incapable of collision. Under such a design, extremely low power consumption methods would still allow people to move at considerable speed (over 100 MPH, 160 KPH.)

That doesn't mean that other more important forces facing humanity won't demand the emergence of machine ethics at least as soon. With the growing advent of human beings abuse and nastiness to one another, it may in the end be up to the robots to help us get our abusive natures squared away. Ultimately all jobs will be done faster and better by robots... ALL JOBS. They will be smarter, fairer and more compassionate only if we choose to make them that way, and the price of not making them that way leads to a dystopian climax for humanity with almost absolute certainty. They will be ubiquitous, touching every aspect of our lives. They will either support us in becoming what we may become, or they'll take our warring idiocy to its final obvious conclusion.

We need to extrapolate the inevitable trends now. We need to think ahead. We need to plan for our retirement as a working species and begin to invent what we will do with unlimited labor and intelligence at our beck and call. Its time to get our primate instincts under strict control, and reengineer ourselves and our societies. It is pressingly important for this species to give birth to its own successors. Welcome Homo Resurectio, the Awakened Man.

Re:I call Bovine Fecus...

[mbstone]

An intelligent, robotic transportation service would realize its implementation has thrown 250,000 human truck drivers out of work, and would abend or otherwise disable itself.

Re:I call Bovine Fecus...

[geekoid]

"..are physically incapable of collision."
And if a bridge collapse? lightning strike?

" are extremely unlikely to be in a collision. "

Re:I call Bovine Fecus...

[Genda]

No, if you have powerful magnets at specific points on the vehicles and superconductive bumpers at other places, the vehicles physically can't collide, same reason they're maglev in the first place. However, if enough of them were to fall off of a collapsing bridge at the same time, its most likely the forces exerted on the bottom-most cars would crush them and their occupants, so the end result for the cars at least several down in the crunch would be indistinguishable from a collision.

The fun part would be trying to rescue the cars higher up floating over the dog pile. Strange mental image.

Re:NO WARRANTY Doesn't Pay the Bills

[Genda]

No, they'll throw in a case of Rice-O-Roni, the San Francisco treat to be divided by your heirs evenly.

Re:Religion

[Immerman]

I believe most religion regards itself as a (non)optional replacement for step one.

Re:First thing that came to mind...

[Genda]

Any sway over your existence??? Are you mad. The robot will take your job, because whatever your job is, the robot will do it better, faster and 24 hrs. a day 7 days a week. The robot will grow your food, process it, cook it and serve it to you. Your robotic doctor will inform your robotic surgeon and your health will be in their hands (all 8 of them.) You will have half a dozen robots in your house cleaning and maintaining your lifestyle. Any sway on your existence. WAKE UP!!!

Re:If my hypothetical driverless car is about t cr

[AK Marc]

With driverless cars, the travel time would be less critical, and more optimal (higher maximum speed limits, lower speeds in high-risk areas). That would eliminate most of the "unavoidable" situations. Usually the unavoidable situations are very avoidable.

There was one time where I said to my mother "It's ok, I see them." She turned to me and said "What are you talking about?" "You'll see" And then shortly after that, the car next to me changes lanes into ours, right where I was. I pulled over onto the shoulder and honked. He looked over, with a panicked look on his face, and pulled back into the other lane. My mother complained that if I saw that coming 30 seconds in the future, I should have avoided it. I explained that if I could see it and did adjust, then he'd think his road hypnosis was safe, and could kill someone else. If I didn't have the habit of looking for the eyes of all drivers near me, and noticing that he was inattentitive adjust my plan, then I could likey have over-reacted and killed everyone in my car or those around us, and as he changed lanes into me without a signal (or even looking) then it would have been "unavoidable" right?

Or another time, I told the driver "watch out, he's running the light." Then our light turned green, and a car came through, narrowly missing us. The driver yelled at me for not sounding more concerned about the guy running the light. I noticed the nose-down of hard braking, followed by the nose-up of accelerating, which indicated to me he saw the yellow and knew he should stop, but decided to run it. If we'd gone faster on the green, we'd have been hit. Is it "unavoidable" being hit when you have a green light?

I've also stopped for children running out blind between parked cars, once even before they ran out. The guy behind me honked, then when the child ran out and I looked back at him he looked like he wanted to be somewhere else, anywhere else.

Most drivers don't do such things. A good driver should be predicting their position and the position of all known hazards as far in the future as possible (30 seconds a good rule of thumb). Most drivers don't do that. A computer can do that better than a person. The question is what do you do with the probability of something, vs the consequence. You can't be 100% certain in all cases. A suicide jumper from a bridge could land on you or something like that. but you can be much much more sure than a regular person could ever be.

Re:So who's footing the bill

[Genda]

Allow free human driving outside the limits of the megalopolii, so Joe can drive his beater in the country. He needs to store it in a garage on the outskirts of town.

Re:But I value my own life over the lives of other

[Rhacman]

Would you like your driverless car to veer off the road into a tree because it's pattern matching algorithm thought that the clothing that fell out of someones car lying on the road might have a living human in it? Even if such a system were perfect, it is still a machine and it should do no less than represent the intent of its user be that self-preservation or not. This is part of the reason I disagree with having driverless cars in the first place. The choices you make behind the wheel can have a serious impact on not only your own life but that of others. Don't delegate that responsibility to an automated control system. Angry Birds can wait until you get to your destination safely.

Re:Monetizing morality

[Genda]

I can't wait to see robotic shrinks. Yeah, I'm bringing the car in, it seems depressed. I don't know, it just sits at the stop lights and sighs. I caught it staring at a wall the other day and I was getting really uncomfortable.

solution doesn't scale

[Chirs]

If everyone chose to drive Suburbans, then they would actually be less safe overall since there is more energy involved in a collision.

You'd be better off if everyone was driving light vehicles.

Re:Weak bus? Also, "cost effective", not "moral"

[AK Marc]

Question: "Your car is about to crash into a bus"

Answer: "No it's not, it started braking before the only two choices were crash or crash."

Re:Weak bus? Also, "cost effective", not "moral"

[AK Marc]

How about driving into a wall vs driving into a group of pedestrians?

I'd just stop and not hit either.

The question is "given a large number of impossible actions putting you in an unwinnable situation, which option do you choose?" I choose reality, where the vehicle would have removed it from the conflict before it happened.

Re:Weak bus? Also, "cost effective", not "moral" ?

[icebike]

Well, you'd need to have more people run out than the number in the car. But, you raise a good point. Still, isn't suicide illegal? Do you really want the car aiding and abetting? :)

If the decision is between committing murder of the occupants vs un-willful participation in a suicide, is there any real moral dilemma?

One would have to ask:
Why is the car going so fast in the presence of pedestrians that hitting the wall is necessary?
Surely the car would not over drive its airbag response time in the presence of walls, would it?
If people can dart out from hidden places and cause wall crashes, wouldn't it naturally become a sport of hooligans? Pamplona anyone?
If people intentionally dart out from hidden places didn't THEY, by that very act assume all liability and risk?

Sell them only in Québec

[anon mouse-cow-aard]

In Québec, there is no-fault insurance. All drivers pay into (ghasp!) Socialized single insurance system, and no-one figures out who is to blame for a given accident. There are standard rules, no huge payouts, and almost everything gets settled out of court. Far, Far cheaper. My hand is up for a self-driving car, pick me!

oh, make sure it works in the snow though...

Doesn't look good..

[mattr]

You might be able to organically grow a machine in an environment that required it to be moral to survive.
But once it detects loopholes, or is put in an environment that does not have such rules it is likely that a machine capable of organic growth would cease to be moral and instead take advantage of all dimensions of potential behaviors.
Perhaps if there was a community of machines that constantly observed each other.. oh wait if they all find a good reason they can still go to war.
On the other hand it is possible that a machine might have a better chance at being moral than man, because it logically decides that is the best way to act. Not sure how you could tell on the face of it which kind of machine you've got though. It could evolve its philosophy pretty quickly, like in between sentences.

Re:what if the brakes fail and the car needs to ma

[geekoid]

into the bus.

You got a hard one?

Re:But I value my own life over the lives of other

[bmo]

I support the ethics of Ayn Rand in two senses. One, I believe I should act in my rational self-interest (prescriptive sense). Two, I believe I and others do attempt to act in our rational self-interest (descriptive sense) whether or not we/they/you agree with this as a philosophy.

1. People rarely act in their own rational self interest.
2. You are an element of the set {People}
3. Ayn Rand wrote fiction. Bad fiction.

Your argument is invalid.

--
BMO

Re:what if the brakes fail and the car needs to ma

[lister king of smeg]

what if the brakes fail and the car needs to make choice??

*Off the bridge

*in to the bus

* head on into traffic coming the other way?

what do you do hot shot?

shift into neutral and turn on you hazard lights and use horn if necessary to warn others? shift to a lower gear if you still need forward thrust but at lower speed? use the emergency break?

Re:First thing that came to mind...

[geekoid]

Yeah, I can't wait. Of course, the economic will need to change. Eventually money won't matter.
Robots, they make everything better.

Every person in an industrialized society has had a robot of some kind influence their life. From airplanes, to the phones system, to autopilots, to their smart phone.

The math isn't the moral question

[ace37]

The calculus of societal interests isn't the question. At the core, we have a larger moral question:
Is it ethically right to plan to kill one innocent person if it should save the lives of five, ten, or whatever number of people?

Individualist thinking is inclined to argue it's unconditionally wrong to knowingly place (force) an innocent person onto the altar to die. Societal/communal thinking is inclined to argue the greater wrong is that which produces the greater loss, therefore the innocent one must (perhaps unwillingly) give his/her life to preserve the safety of the others. We cannot expect our society as a whole to make a unified decision here; we must instead be convinced that we don't need to ask questions that are this hard.

Rather than trying to decide how many deaths are too many, we need to convince ourselves we've solved this causes-of-death question before we implement the solution. This will probably mean we first solve it for specific, well-controlled roads/freeways, and then over time we can extend the solution outward to additional classes of roads and conditions.

Ok, I'll take this one.

[VortexCortex]

"How Do We Program Moral Machines?"

Genetic Programming & Selection Pressures
NEXT!

Bus example

[tehcyder]

Why would a driverless car ever be about to crash into a bus and yet still have the time to react by driving off a bridge instead?

Re:Weak bus? Also, "cost effective", not "moral"

[Rich0]

Oh, I agree entirely, but sometimes removing the conflict might require choices that society is unwilling to make, like putting fences alongside roads, or having traffic slow anytime a pedestrian is walking down an unfenced sidewalk.

Re:Weak bus? Also, "cost effective", not "moral" ?

[Rich0]

The point is that you are still making a moral decision, and that does need to be accounted for.

I agree that some issues can be avoided by things like slowing cars when pedestrians are around, building fences, etc. However, in the absence of fences in a congested area do you really want all your cars going at the speed of the slowest bike in town?

Re:But I value my own life over the lives of other

[SirGarlon]

I've lost my will to let innocents die so I can live.

Moral or?

[servant]

Computers, like rocks, are neither good nor bad. They just are just there. Rocks are useful many times. So are computers.

Now the difference in being 'moral' or 'immoral' often has to do with the societal perspective. If the Nazi's had won WWII, then history would not have written that the things done by them in the name of 'war' or 'cleansing' were immoral. If the South had won the war between the states, slavery being marked as 'immoral' would have been pushed back. Not that some individuals would not have pointed that direction, but the majority of society would have gone with the 'victors'. Even in situational morality, consider the Dalmer party. Was eating humans 'moral' when the other option was perceived starvation? Some consider large deficit spending 'immoral', others do not. Some consider not being vegetarian or vegan as being immoral. To some having multiple spouses is considered immoral.

So to that extent, even peoples actions, are 'relative' and so it their 'moral compass'.

So can we program moral machines? Yes and No. It depends on the programmer. It depends on the uses.

So who are we to determine if a computer made 'moral decisions' or not?

Re:Weak bus? Also, "cost effective", not "moral"

[AK Marc]

And I wish that society would make the decisions they don't want to make. The government has put a dollar figure against traffic congestion and human life, so it is now a mathematical question, not an ethical one. The ethical ones are already answered (rightly or wrongly is a separate discussion).

Re:Weak bus? Also, "cost effective", not "moral"

[Rich0]

Yup. That's actually one of the benefits of automation. You can start turning the risk questions into engineering ones with hard numbers attached. That is also one of the reasons why it might never happen without legislative reform - there will always be some piece of paper that documents in black and white that somebody assigned a value to human life, and people just don't like to think about that. The only way to make it work is if a regulator sets that value.

Windows Car Edition won't have these issues.

[hobarrera]

Windows Car Edition won't have any moral issues to deal with:

Windows has detected a pedestrian and must restart for the changes to take effect.

I'm sure it sounds good

[Sqreater]

But the problem is that morality is not a module that is plugged into things. You can't program ethics and morality. Ethics and morality come from the human motivation array which comes from the evolutionary process of millions of years of trial-and-error change. We are not going to be able to condense that down into lines of code. Try as we humans might, we cannot program a human.

Actually

[Sqreater]

I suspect that because humans and machines will be driving together that there will actually be MORE accidents. Humans can mostly predict what humans will do. We can even predict what somewhat erratic humans will do - they will be erratic. But can we predict what a computer program from THIS manufacturer will do versus what THAT one will do when it applies its logic function and perception to any particular fluid situation? I don't think so. And if they give us driving program version 2.0, tweaked to be better than 1.0, how many accidents will there be before we discern the difference and accomodate our driving to it? Oh, NOW it tends to do THIS in this situation when before it tended to do THAT. OOPs, crash. When I get to a particular exit I pull back and leave room because I know that people are going to pull across the striped barrier after realizing belatedly that they are in the wrong lane. Also, I'm aware that commuting road warriors will come from the fastest lane two lanes to my left to suddenly make the exit at high speed. Ok, it works. I've also come to be able to predict the people who will be cutting in long before they do and allow room for them. What the subtle signs are I'm not entirely sure. We humans accomodate ourselves to the situation and the speed of the commute is faster for it. But will a machine do this? Probably not. It will probably do the legal, much slower thing and the commute will come to a grinding halt. We humans conspire beautifully to reach our common goals. I suspect this will be far too difficult for a computer program.