Augmented Reality To Help Mechanics Fix Vehicles

kkleiner writes "ARMAR, or Augmented Reality for Maintenance and Repair, is a head mounted display unit that provides graphic overlays to assist you in making repairs. An Android phone provides an interface to control the graphics you view during the process. Published in IEEE, and recently tested with the United States Marine Corps on an armored turret, ARMAR can cut maintenance times in half by guiding users to the damaged area and displaying 3D animations to demonstrate the appropriate tools and techniques."

Comments

Looks Neat

[Ethanol-fueled]

disclaimer: I'm a former military backshop avionics technician

That technology looks to be pretty cool and excellent for routine maintenance, but I can see how it would suck for troubleshooting. For example the video in TFA locates a cable and instructs the user to unscrew it. With all that fancy visual stuff going on, it could be easy for the technician to overlook a pushed pin or a pinch in the cable which could be causing a problem. The small screen on a wrist-mounted phone would not be sufficient to display the necessary detail. The solution as-is is not suitable for finer military electronics which are tangled messes of RF hardlines, circuit cards, and even wire-wrapped backplanes. A full-size LCD to the side showing 3-D animation would be much more suitable for that. Additionally,

In the Marine Corps tests, the pair of researchers used 10 cameras in the vehicle

Which works fine for vehicles, but would totally suck for aircraft. Did the guys who came up with the statistics factor in the time it takes to set up and/or calibrate the camera array? Of course, embedding a few sensors within the vehicle and setting up the display's position with respect to them would be much easier.

Re:Looks Neat

[sznupi]

I say use it more. This system seems to have quite limited vision, figuratively.

When you think about it, most of the structure of aircraft/tank/whatever simply gets in the way of seeing your surroundings properly. But that structure might become translucent once you're looking through an array of cameras placed outside the vehicle, outputs of which get combined to present a nice view of surroundings in your VR gear. A view augmented with useful info, too.

And it will work nicely with unmanned vehicles.

Re:Looks Neat

[c6gunner]

The newer aircraft are pretty much idiot-proof, anyway. Once you get rid of analog sensors/gauges and the old cam-and-roller based controllers, troubleshooting gets a lot easier. Now you just plug in to the on-board diagnostic system, and 9 times out of 10 it'll tell you exactly what the problem is. I'm not sure that this "ARMAR" technology would make the process any easier or faster.

Re:Looks Neat

[DerekLyons]

disclaimer: Former SSBN crewman, Missile, guidance, and fire control tech.

That technology looks to be pretty cool and excellent for routine maintenance, but I can see how it would suck for troubleshooting.

That depends on what part/type of troubleshooting it was used for I should think. It seems pretty useful for locating a particular cable in a rats nest, or an obscure adjustment or lubrication point. I can think of a couple of troubleshooting procedures it would have been very handy to have such a device had I ever had to do them. One emergency procedure I was trained to do, for example, hadn't ever been done in the thirty year history of the Force when I was in. (Odds are, that still holds true for complicated reasons I won't go into here.)
 

The solution as-is is not suitable for finer military electronics which are tangled messes of RF hardlines, circuit cards, and even wire-wrapped backplanes.

You're looking at from the backshop POV, not the line maintenance POV.
 
Also, there's more to the service than avionics. Aircraft are designed around LRU's which are quickly swapped out from a pool to get the aircraft ready, then the LRU's are sent to the shops for more leisurely repair.
 
On the boats, we don't have LRU's that are swapped out or backshops to send them to - we open the gear and work on it directly, so the design/packaging philosophy is different. Most of my fire control gear (and much of the rest of the electronics on the submarine) had modules on the front, plugged into a backplane, and then cables on the back of the backplane, which was hinged like a door for when access was required.
 
99% percent of the troubleshooting and repair was done just by swapping circuit card modules, and the system was designed to have that maintenance done 'live' in real time. (That is, we powered down the backplane containing the module of interest, leaving the rest of the unit powered.) That way (in fire control for example) could rapidly enter the equipment and make repairs even while shooting missiles using the redundant string or shooting the missiles that weren't impacted by the fault.
 
A system like the one in TFA that was hooked into our BITE systems and could lead the tech directly to the 051C9B35 module (for example) would be quite useful under battle conditions.

Re:Looks Neat

[couchslug]

"disclaimer: I'm a former military backshop avionics technician"

Represent!
Erm...

As a former flightline avionics weenie, engine troop, and later crew chief I concur with your assessment, and I wouldn't want that shit on ground vehicles either. A rugged notebook with a combination of .pdf manuals, photos, and videos would be much better because it need not be worn on the head and can be easily shared.

What it COULD be useful for is training. Once trained and with some experience, people know their systems and can rapidly troubleshoot them. Key to that is a combination of BIT (Built In Test) and electronic system diagnosis.

Re:Looks Neat

[Anachragnome]

"Now you just plug in to the on-board diagnostic system, and 9 times out of 10 it'll tell you exactly what the problem is."

That sounds like the same thing I hear people say about automotive scanners. "Just plug it in and it tells you exactly what is wrong!" That is a bunch of hogwwash. Scanners simply give the tech diagnostic codes, and in some cases allow for viewing certain circuits and their current (or stored) values. It simply gives you a direction to look.

For example, a scanner might have found a stored code for an oxygen sensor fault that indicates the engine is running full rich at all times (and even show voltage values that reflect this) when the real problem is a fuel injector with it's pintle jammed open by a sliver of metal left in the fuel rail during manufacturing. There is NO way in hell that the scanner will know anything about that sliver of metal. BUT, a good mechanic will know what types of failures can cause a full rich condition and start ruling them out one at a time. The code is merely a starting point.

This device will not give a mechanic the sudden ability to know all the possible causes, nor will it always tell them the truth. False codes are a real problem that are usually ruled out BEFORE any further diagnosis occurs (usually by pulling codes followed by clearing them and seeing which ones come back).

There is not a device on this planet that can replace a good diagnostician simply because every possible failure is impossible to foresee.

Re:Looks Neat

[Anachragnome]

Replacing an ECU (PCM, same thing) is a tricky venture sometimes.

Being the honest mechanic, I usually offered an alternative to my customers. I let them decide after explaining.

I had a roughly 50% success rate of fixing these cars--cars that all testing seemed to indicate a wonky/bad ECU--by simply taking the circuit board out of the ECU housing (carefully!) and giving it a gentle twist on two different axis, then re-installing it.

50% of the time the problem went away, never to return. The other 50% of the time it ended up needing replacement eventually. If the customer didn't mind the risk of the problem re-occuring they usually went for the twist test. Saved my customers thousands of dollars.

I have no idea why this fixes some ECUs. Since all the connections are board-soldered, I can olny assume stressing the internals of some component on the board was enough to get it back in spec. Weird, I know.

My own Chrysler Town and Country had this "repair" done to it to solve an intermittent turn signal loss...4 years ago...and it still works.

Since figuring this out, I do this on ANY malfunctioning device with circuit boards...take them out, give 'em a twist and re-install them. It often fixes the problem. Fixed a laptop, two PCs, a stereo amplifier and a snowmachine this way.

locating parts on vehicle

[pig_man1899]

How do I use this thing to locate the muffler bearings my service shop says need replacing?

Re:locating parts on vehicle

[MadShark]

I once got a friend of mine to believe there was such a thing as blinker fluid. It just so happened that another friend had his car parked outside with a cracked tail light. It had rained heavily earlier that day and water had leaked in and filled the blinker right up to where the lens was cracked. She was calling bullshit until we took her outside and showed her. She bought it hook, line and sinker after seeing it. We caught hell later, but it was a lot of fun.

So much for A+

[Phrogman]

So much for repair certifications like A+ etc, you just need the right headset and the ability to use simple tools apparently :)

Re:So much for A+

[mhajicek]

Operator, I need a repair program for a B-212 helicopter.

will it be like other AR?

[girlintraining]

Cut approaching.
Cut approaching.
Cut here.
Cut here.
You have cut the wrong wire.
Recalculating...
Recalculating...
Get soldering iron.

On the down side

[sehlat]

We are very likely to see people like the major auto manufacturers providing this sort of
thing ONLY to their authorized dealers, and possibly trying to claim that any repair information
of any kind is copyrighted, just like they've done with the diagnostic codes on the black boxes.

Another dealer profit center.

[WhatDoIKnow]

I can't comment on military applications, but I do have 30 years experience in mobile equipment and vehicle maintenance and fleet management. Despite the OBD 2, the major vehicle producers are increasingly requiring proprietary information and specialized tools for what could be simple routine repairs and maintenance. The described system could be a boon to technicians but my cynical view is that it will just be turned into another income source for vehicle manufacturers and dealer service departments. On many cars now you can't even change a coolant hose without a substantial investment in a "hose fitting disconnect kit", let alone accessing any non-generic DTCs from OBD2 or CAN. And of course Ford, Honda, GM, Toyota etc. are all different.

Some Thoughts

[BJ_Covert_Action]

Based on the pictures, it looks like this thing sticks a few inches off of your face while worn. That could make maintenance in tight areas (read: under low riding cars) a bit of a problem/pain in the ass. I know when I crawl under my little deuce coup, even when the back end is lifted off the ground, My face is about 4 inches under the frame. I think these goggles could make that clearance, but I could easily see this being a problem in other models and in other areas of the vehicle.

It certainly seems like an interesting idea, nonetheless. I would love to see it hit the civilian market at a low enough price. I have to admit, though, that I would be irked if vehicles and other machines began to be designed in such a way that this piece of hardware was near essential to work on them. If it has a low cost, sure, then it might not be a big deal. However, if it has a high cost like some code-readers and is essential to work on your own vehicle, well, that would make me start ranting about my lawn and my Phillips screwdrivers.

Re:Some Thoughts

[c6gunner]

I know - which is actually quite confusing. They can make a smartphone Less than 2 inches thick, and it has a camera and screen built into it, what makes this headset so special?

It's a military requirement - every gizmo has to be 10 times as big as the civilian equivalent, must weigh at least 5 times as much, and must look like something out of an 80's sci-fi movie. If you can bundle that with a really clumsy and unresponsive interface, you're pretty much guaranteed military customers. 5 years ago I had the option of using a military GPS - which was about half the size of a phone-book, and lost the signal every time it rained - or buying a civilian version which could fit in my shirt pocket and worked pretty much everywhere. Guess which one I went with :)

The other (more serious) reason is that it's clearly an early model. Research-in-progress is usually a bunch of commercial components wired up in an ad-hoc manner. If they ever get a market-ready version, I'd imagine it would be a lot smaller.

Boeing

[JaneTheIgnorantSlut]

Boeing started using this tech in the early 90's to help assemble cables in aircraft. http://en.wikipedia.org/wiki/Augmented_reality

Re:Hello

[frosty_tsm]

welcome to Windows Mechanical, I see you have picked up a wrench, please wait while Microsoft Clippy WrenchBuddy .NET SP 6 is downloaded.

Not quite. They used the Android to power it, so it'd be closer to:

[Repair] [I'm Feeling Lucky]

Re:Wait a minute!

[fuzzyfuzzyfungus]

Robots with an approximately human combination of strength and fine dexterity are somewhere on the line between "fucking expensive" and "not out of the lab yet". They can beat the hell out of humans in situations where you design the whole scenario around them; but they are weak for general purpose work.

Humans with an approximately human combination of strength and fine dexterity are so cheap that we routinely let the unneeded ones starve to death.

Blah

[Rib Feast]

It feels like augmented reality is becoming the next buzzword to follow cloud computing. Honda in Japan have had HUD tech for years, where their repair people could identify engine parts. Apparently it made them something like 30% more efficient. Even this article has AR features and is from 2004.

http://www.allbusiness.com/automotive/automotive-trade-motor-vehicle-parts/5531755-1.html

Call me when it can be put inside a contact lense or into the eye permanently, until then it's just a mash of tech we've had for a while.

Re:Wait a minute!

[Dr_Barnowl]

Obligatory link to story detailing the reduction of humans to a mere servo in the machine.

Re:Wait a minute!

[jgtg32a]

The factory of the future will have two employees: a man and a dog. The man's job will be to feed the dog. The dog's job will be to prevent the man from touching any of the automated equipment.

-Warren G. Bennis

Perfect brain for jarheads!

[HigH5]

I can imagine this will end up in "Shoot the enemy here or here to kill him." Also available in spanish.