Mitsubishi Electric Believes Its AI-enhanced Camera Systems Will Make Mirrors on Cars Obsolete (ieee.org)

In its annual R&D Open House on February 14, Mitsubishi Electric described the development of what it believes is the industry's highest-performance rendition of mirrorless car technology. From a report: According to the company, today's conventional camera-based systems featuring motion detection technology can detect objects up to about 30 meters away and identify them with a low accuracy of 14 percent. By comparison, Mitsubishi's new mirrorless technology extends the recognition distance to 100 meters with an 81 percent accuracy. "Motion detection can't see objects if they are a long distance away," says Kazuo Sugimoto, Senior Manager, at Mitsubishi Electric's Image Analytics and Processing Technology Group, Information Technology R&D Center in Kamakura, 55 km south of Tokyo. "So we have developed an AI-based object-recognition technology that can instantly detect objects up to about 100 meters away."

To achieve this, the Mitsubishi system uses two technology processes consecutively. A computational visual-cognition model first mimics how humans focus on relevant regions and extract object information from the background even when the objects are distant from the viewer. The extracted object data is then fed to Mitsubishi's compact deep learning AI technology dubbed Maisart. The AI has been taught to classify objects into distinct categories: trucks; cars; and other objects such as lane markings. The detected results are then superimposed onto video that appears on a monitor for the driver to view.

Comments

same old problem

[cellocgw]

The vehicle driver is focussed at infinity when looking forwards. If you look at an actual mirror, you can remain focussed at infinity.
If you have to look at a monitor (let alone one that's got cartoons based on target classifications), you have to refocus to the monitor distance. So unless MItsu is planning to project their image to infinity-equivalent, this is NotAGoodIdea(TM) .

What problem is being solved...?

[DogDude]

I'm not understanding how this works better than a $0.50 mirror.

Re:What problem is being solved...?

[TWX]

That's what I'm wondering too. They're cheap to manufacture, very reliable (most vehicles go to the scrapper with their original mirrors still fitted) and short of adjustment to fit an individual driver, don't require any work on the part of the driver to use.

The only thing I would see being potentially useful would be figuring out how to aim the mirrors for a given driver without that driver having to do much beyond taking a seat. That would probably be a matter of figuring out where the driver's face is in three dimensions, and then adjusting the mirrors as appropriate. Only issue here is that most people seem to want to see the door handles in the side mirrors, while aiming wider than that is better practice for reducing blind-spots. Such a system might need to be tweakable so that the driver gets the mirrors as wide or narrow as wanted. Since some vehicles have memory to set the power mirrors while they set the power seat to the driver template preference this might not be that difficult to achieve.

Re:What problem is being solved...?

[jenningsthecat]

I'm not understanding how this works better than a $0.50 mirror.

I'm not understanding how it can be as reliable as a $0.50 mirror. Also, if one of my mirrors breaks, the remaining two can partially cover the deficiency. If the electronics goes down, I'm blind to what's behind me without looking over my shoulder.

Then there's the whole hackability aspect. I can see the headlines right now.

Re:What problem is being solved...?

[jellomizer]

My main expectation is that it will be better at blind spot detection, and adjust for bright lights better. Also it could consolidate all the mirrors so you are not craning your head to know what is going around.

That said, if it is anything like the rear view camera, it will need to be setup with a way to clean itself. One spray from the car next to you, you are SOL. while a mirror with a bigger surface area will usually be quite usable even if very dirty.

Re:What problem is being solved...?

[beelsebob]

The big problem with a $50 mirror and $100 enclosure is that that mirror and enclosure cause enormous amounts of aerodynamic turbulence, and drag. Having those mirrors contributes around 7% of the total drag of the car. Improving a vehicle's fuel economy by 7% is ENORMOUS.

You only need to look at the styling on any modern car to see that *huge* amounts of effort goes into designing the thing to do nothing but steer air around the mirrors, and/or tidy up air that's just come off the mirrors.

Re:What problem is being solved...?

[chispito]

I'm not understanding how this works better than a $0.50 mirror.

I'm not understanding how it can be as reliable as a $0.50 mirror. Also, if one of my mirrors breaks, the remaining two can partially cover the deficiency. If the electronics goes down, I'm blind to what's behind me without looking over my shoulder.

Then there's the whole hackability aspect. I can see the headlines right now.

You wouldn't think that the current drive-by-wire control systems would be as reliable as the old rods and cables, but here we are. I think you're missing the point anyway. What's more likely? That the camera system malfunctions, or that a driver misses or misinterprets what is in his side view mirror? The system is not aimed at being more reliable than an optical mirror, but at bringing the overall reliability up by making the driver more reliable. Suddenly that motorcyclist in your blind spot is glowing red.

Re:What problem is being solved...?

[TWX]

1. Mirrors don't cost $0.50. A side mirror assembly can cost over $100.

Not quite.

Sure, some replacement side mirrors can cost over $100. Replacement side mirrors may have significant markup though. Dealerships charge a lot more for parts than those parts need to cost, and aftermarket manufacturers of such parts have had to spend money to reverse-engineer to make functionally-identical parts that are not sold the same volume as original manufacturing for new cars.

That $100 side mirror assembly is probably a quarter that cost when factored over millions and millions of units across multiple vehicles from a given manufacturer for many model years of production. When an automaker is talking, those kinds of numbers are the ones to consider, not the replacement-cost numbers.

2. Mirrors have blind spots. Cameras don't.

If that's so then why does every backup-camera system and every owner's manual insist that the driver look around and not trust the camera's view?

Besides, I've had daily drivers as old as model-year 1978 and as new as model-year 2015. In every case the older vehicles were easier to see out of even with little mirrors that were mounted further back on the doors compared to modern vehicles with forward-mounted mirrors right at the A-pillars plus massive A-pillars to boot.

3. Side mirrors create drag, increasing fuel consumption.

How much drag? Is the drag caused by the inclusion of side mirrors greater than the drag caused by the basic automotive body styling? 'cause when I look at that Chevy Silverado, of course it's not the brick-shape but the little side mirrors that are really slowing the vehicle down.

4. Side mirrors limit car-to-car parking distance for self-parking cars.

That aforementioned car from 1978 had bodywork that was wider than the side mirrors stuck out. The widest part of the body was wider than the side mirrors.

Besides, how close do you want to trust these cars parking together? What happens if there's a flat tire and the vehicle leans over a little bit? Are you going to advocate for adding vehicle suspension stabilization too?

5. Rear view cameras and displays are already mandatory on all new cars, so the mirrors are redundant.

Rear view cameras are only necessary because trunk lids are half-again taller than they used to be on coupes and sedans, and the rear lower-window height is taller than it used to be on minivans and other wagon-bodied vehicles like SUVs, and because trucks all ride at 4x4 height even when they're only two wheel drive. Modern automotive design sucks when it comes to driver view. A-pillars and B-pillars are huge and create significant blind spots. Mirrors are poorly designed such that looking at the side mirror results in seeing the frame of that mirror in the reflection, losing on viewing angle. Windows are small and high so that drivers get poor lower visibility to the ground in front of them.

When vehicles are designed with giving the driver as much view as possible then active camera systems are unnecessary, but automotive designers have gotten lazy and are giving us crap when it comes to the actual driving experience itself.

That makes zero sense

[SuperKendall]

The vehicle driver is focussed at infinity when looking forwards.

When I'm driving I'm not looking at some mountains 10 miles away. I'm spending a lot of time looking at traffic around me, and at upcoming signals which I assure you are not at infinity...

Any good driver is going to be doing a ton of constant re-fosuing from near to far, as they re-scan everything around and upcoming. So a mirror that is really a screen is not an issue as it's just another re-focus with some slightly different information than normal.

You can also take in overlaid information without ever re-focusjng, just from color and shape and size of overlaid markings you would know where something was behind you even if you didn't focus specifically on the symbols to see them as totally crisp.

Re:That makes zero sense

[skids]

Any good driver is going to be doing a ton of constant re-fosuing from near to far

Come back when you get into your 40s and the presbyopia starts to set in and tell us how much you like those screens.

Re:That makes zero sense

[jklein]

Anything beyond about 20 feet is essentially at infinity focus. What is lacking in cameras is depth perception. Try driving, or backing up, with one eye covered (caution). It's surprisingly difficult to do confidently. Until they figure out how to do 3D cameras and displays in cars, they won't replace mirrors in functionality.

Re:That makes zero sense

[jareth-0205]

The vehicle driver is focussed at infinity when looking forwards.

When I'm driving I'm not looking at some mountains 10 miles away. I'm spending a lot of time looking at traffic around me, and at upcoming signals which I assure you are not at infinity...

Well, it kindof is though. The human eye has a hyperfocal distance of about 6 metres (ie when focussing at 'infinity', everything further than 6m will be sharp). Which unless you're going very slowly is nearer than vast majority of stuff you will be looking at outside the car.

Re:That makes zero sense

[bws111]

Infinity focus happens at about 20 feet in humans (you know 20/20 vision), not 'mountains 10 miles away'. The signals, etc, certainly are at infinity, despite your 'assurance' to the contrary.

Re:That makes zero sense

[AK Marc]

I'm spending a lot of time looking at traffic around me, and at upcoming signals which I assure you are not at infinity...

Nope. Those are at infinity. Infinity is about 30 feet. The binocular separation is about a tenth of a degree. So, even sitting at a light, first in line, the lights on the other side of the intersection are effectively infinity. The dash is not infinity. The car in front in traffic is not infinity. But in a car, almost everything except the dash is at infinity. Adding more to the dash, taking away from the trend to infinity, would decrease safety.

Re:That makes zero sense

[Anonymous Coward]

Until they figure out how to make 3D cameras and displays that are as reliable as mirrors in cars, they won't replace mirrors in functionality.

Cameras go wrong all the time. There are many fewer points of failure in a mirror, and they're a sight easier (and cheaper) to clean/adjust/replace when necessary too.

I don't know if removing mirrors is worth it

[mykepredko]

When I did a quick Google search, the claimed increase in drag for side mirrors is 3%-6%.

Let's assume that it's 3%, the car gets 25mpg, the travel life of the average car is 50,000miles and the average cost of gas through this period is $2.50/gal.

With the mirrors, the cost in gas is: (50,000miles / 25miles/gal) * $2.50/gal = $5,000.00

Without the mirrors, the fuel economy goes up to 25.75mpg and the calculation becomes: (50,000miles / 25.75miles/gal) * 2.50 = $4,854.37

Even using the 7% figure in the parent post, the calculation becomes (50,000miles / (25miles/gal & 1.07)) * 2.50 = $4,672.90

While I do understand that any single digit percentage improvement in fuel economy is a big thing for car manufacturers, I'm not sure if the technology can be made cheap enough (even with removing the costs of the mirrors) to be viable and result in an overall reduction in costs for the owner.

Adding technology costs to the basic cost of a vehicle (even though it saves on fuel and other costs) generally does not get car owners excited about the technology.

Re:I don't know if removing mirrors is worth it

[Njovich]

According to this article air resistance only accounts for 5% of energy output of a car. So if the mirror accounts for 7% of air resistance, that's 7% of 5%. So that means it's only 0.35% of fuel economy.

Are mirrors even used?

[RabidDawg]

It seems to me nowadays, that mirror use is optional....

Does no-one read instruments?

[SuperKendall]

Come back when you get into your 40s and the presbyopia starts to set in and tell us how much you like those screens.

I'm past my 40's and I like those screens just fine. I mount my iPhone in my car to use Waze.

I am quite concerned that you and all other responders seem never to look at instruments in the car like the speedometer, since those would mean focusing as close as any other screen...

I myself like to pay attention to how fast I am going. I guess everyone else is just driving staring blankly in the distance. That would explain a lot of the erratic behavior I see in other cars when driving...

Cruise control is usable how often exactly?

[SuperKendall]

You are irrationally focused on something that can be easily controlled automatically. Cruise control has been around for a very long time

The more I read responses the more concern I feel, and it's all quite rational - when most driving is through city streets (or ever highways) with varied speed, to bring up "cruise control" as a defense is beyond alarming and right up into head-slapping territory.

Keep digging that hole buddy! Or admit that yes in fact as a driver you look at the instrument cluster occasionally, even though it would validate someone elses point. But this being the internet, I can feel safe in assuming you would rather paint yourself as the worst driver that ever lived rather than admit someone else on the internet was correct about anything.

Peace out, I'm no longer in this increasingly absurd thread that 100% ignores how people actually drive. You may continue if you wish.

What about a system freeze....

[fuzzyf]

I'm more concerned that the system freezes and lures the driver to think the area is free and clear when it's not.