Slashdot Mirror

The smoking gun for this incident isn't going to be what the final report says. It'll be on some notes by some engineer when this project started saying everything above. There isn't a way that this project made it this far without some intelligent engineers speaking up and getting over ruled by management.

I lasted exactly 45 days in Aerospace and it was terrifying, they picked a "COTS" architecture that hasn't been "COTS" since the Macintosh moved away from 68k. I was told to 'deal with it'. Other people quipped that "this wasn't the worst design decision he's seen". The schedule was everything because customers had already bought what we were working on.

But everything HAD to move forward according to THIS timeline because someone already bought it. In those 45 days I had to work on trial versions of everything, they couldn't figure out how to get us licensed in to their network. Everyone else on the project had always been in aerospace, so this was 'par for the course'. I came from automotive where we actually did put safety first (at least where I worked).

I want to see the MIL/SIL/HIL reports. This should have been caught in the plant model long before it came to market. There should be a high-fidelity model that shows this exact scenario and how it plays out. It was buried for some reason or another. If there isn't then they didn't test as comprehensively as they should have (because of rushing to market).

There are a lot of people, that have been coming to similar conclusions about the MAX8. It's an 'unstable pendulum' that they thought they could just 'fix it in software'. Good hardware design is crucial to a good controllable system.

Someone spoke up, either they have an e-mail in a safe (like Audi's Dieselgate) or they're no longer with Boeing (or one of their subcontractors like GE, or GE's subcontractors) because they did speak up and were told they were 'toxic to the project'.

This is the boring un-sexy parts of engineering. But 'Failure Mode and Effects Analysis (FMEA)s' are important. We literally sit down and go "What happens if this fails" and then write out a full plan in software. Plus a full test plan.

dSpace makes aerospace hardware-in-the-loop (HIL) test benches. They make them for automotive and off highway too. We literally 'drive' around a vehicle for thousands of hours for software releases.

I don't have a doubt this was caught by someone somewhere. Management got involved and now this is going to be another Challenger O-Ring example for freshmen engineers.

Is ignoring a plugged sensor a bad idea? Absolutely. Should the failure mode be plowing into the ground an full tilt after fighting the pilots? No.

But if you're building your business on it,

You can't and you won't.

Unless you don't drive a car and don't fly. dSpace is one of the few Hardware In The Loop vendors. Good chance something you rode in, flew in, or built the road you're driving on was tested on it. Scripted in Python (2.7, but that's an improvement from what was 2.5).

They are migrating to a .NET API that you can access though Python.NET.

any significant scale

I think most developers don't understand how many niche industries there are out there. If Caterpillar sells 100 of it's largest mining truck in a year, how fast does the HIL tester need to be?

Oh, you hate your job? Why didn't you say so? There's a support group for that. It's called everybody, and they meet at the bar.

Also what is a 'developer'? I have a Mechanical Engineering degree and don't Identify with 90% of the stuff that comes up when Slashdot or most places discuss 'developers'. I write code for stuff like Simulink Embedded Coder, Vector CANape, dSpace boxes, etc.

Am I a developer? An engineer?

unit test before releasing

We do. I just described our unit testing procedure.

Because of the nature of automotive engineering "unit testing" is called "Hardware in the Loop" testing. We do have 'software only' tests however a majority of it is done on a dSpace bench after it is flashed onto the ECM.

After it passes all of our HIL tests it's stamped for release.

Jenkins is a piece of crap.

Constructive criticism is more than welcome. If you have a better CI tool I'm all about learning how to use it and seeing if it works about our process. I went with it because it runs on Windows. I can run it in 'portable' mode without installing it (which needs Admin access).

So what do you have that's better?

But there's no way to simulate the real world failures like you're asserting has been done.

https://www.dspace.com/en/pub/...

We do it every single day.

More likely tuning algorithms. [And Uber just stole my business idea.]

"Test Cell" time is very expensive. I think some of ours run $20k an hour and that's not even close to as expensive as they get. Additionally each prototype in a test cell can only ever accumulate 168 of test data per week. (It's a hard limit.)

The 'hardware' part of driverless cars has been pretty much nailed down. We've come a long way in the 11 years since the 2004 DARPA Grand Challenge (The original event didn't go so well) Google and Uber have been poaching the grad students and professors from a lot of the first teams.

Now what those engineers need is more data. They have dSPACE machines hooked up to all of their Matlab models and are running a few thousands 'vehicles' in parallel. The problem is they need a very smart and adaptive better controller to test scenarios with. Uber is tuning it's algorithms against human drivers. The payout is two fold. In the short term they get to vet drivers. In the long term they don't have to deal with drivers again.

If this pans out Uber will be releasing driving game for the XBox One, PS4 & Desktop. "Earn up to $1/hour driving a car!". It'll be a gamified Mechanical Turk. For $20k Google could have 20,000 hours of data in an hour. "Promote" the best driving drivers to $2/hour, $4... $10. You'd still be generating 2,000 hours of data per hour and you'd have the "best" drivers you can find.

The entire "personal vehicles" and "are they licensed" is moot when it's fleets of driverless cars all dispatched from a few locations around the city (as driverless cars become legal). They're already collecting all the data as to where the vehicles are needed. I wouldn't be shocked if Uber isn't already buying up property in places that their data shows a lot of vehicles are needed.

Uber is playing long game. "Drivers" right now are just a cheap way to collect all that data since Taxi companies probably won't release it (or even keep it).

I expect Google is doing all of the same things in parallel.

More likely tuning algorithms. [And Uber just stole my business idea.]

"Test Cell" time is very expensive. I think some of ours run $20k an hour and that's not even close to as expensive as they get. Additionally each prototype in a test cell can only ever accumulate 168 of test data per week. (It's a hard limit.)

The 'hardware' part of driverless cars has been pretty much nailed down. We've come a long way in the 11 years since the 2004 DARPA Grand Challenge (The original event didn't go so well) Google and Uber have been poaching the grad students and professors from a lot of the first teams.

Now what those engineers need is more data. They have dSPACE machines hooked up to all of their Matlab models and are running a few thousands 'vehicles' in parallel. The problem is they need a very smart and adaptive better controller to test scenarios with. Uber is tuning it's algorithms against human drivers. The payout is two fold. In the short term they get to vet drivers. In the long term they don't have to deal with drivers again.

If this pans out Uber will be releasing driving game for the XBox One, PS4 & Desktop. "Earn up to $1/hour driving a car!". It'll be a gamified Mechanical Turk. For $20k Google could have 20,000 hours of data in an hour. "Promote" the best driving drivers to $2/hour, $4... $10. You'd still be generating 2,000 hours of data per hour and you'd have the "best" drivers you can find.

The entire "personal vehicles" and "are they licensed" is moot when it's fleets of driverless cars all dispatched from a few locations around the city (as driverless cars become legal). They're already collecting all the data as to where the vehicles are needed. I wouldn't be shocked if Uber isn't already buying up property in places that their data shows a lot of vehicles are needed.

Uber is playing long game. "Drivers" right now are just a cheap way to collect all that data since Taxi companies probably won't release it (or even keep it).

I expect Google is doing all of the same things in parallel.