Slashdot Mirror
SpaceX: Lessons Learned Developing Software For Space Vehicles
jrepin writes "On day two of the 2013 Embedded Linux Conference, Robert Rose of SpaceX spoke about the 'Lessons Learned Developing Software for Space Vehicles.' In his talk, he discussed how SpaceX develops its Linux-based software for a wide variety of tasks needed to put spacecraft into orbit—and eventually beyond. Linux runs everywhere at SpaceX, he said, on everything from desktops to spacecraft."
I thought that for the Falcon rocket and Dragon capsule, SpaceX use the VxWorks realtime OS made by Wind River.
Is the market for spacecraft programming expected to grow significantly in the coming decade(s)?
I think this is all interesting and good stuff but hopefully most of it is stuff that software companies are already doing these days. At this point if you're not doing continuous integration or looking at ways to automate anything that possibly can be automated, you should get moving.
when your cpu isn't able to help out, the fastest way to catch landing in garbage is to place a few nops (to get realigned) and then jmp to the Big Bomb before each block of code. then if something branches wildly or otherwise escapes its block or into a buffer somewhere, it'll eventually plow into the next of those traps and stop itself.
I work for the Department of Redundancy Department.
I started my career in nuclear engineering before moving into software development.
There were three really important principles: Redundancy (having several of everything); Diversity (having different implementations i.e. different designs from different manufacturers) and Segregation (keeping things physically separate and firewalled off from each other).
I'm a bigger Linux fan than many here. I've been using it since 1995 and I'm a die-hard Slackware user, but having everything running on the same OS seems like an accident waiting to happen. Yes, I know that it's great that you can have one piece of code that you can compile and run anywhere, and that's easier if you're only using one OS.
However, one of the great things about Open Standards and Open Source was (is) that for many years software was portable so that it could be compiled and run on big- or little-endian 32- and 64-bit POSIX-like systems on a wide variety of CPU architectures.
That may have been "expensive" in terms of software maintenance, but as I learned when working for a now-defunct very large UNIX company, writing your software to be portable across those systems exposes (and forces you to fix) many subtle bugs that otherwise would not have been found until deployment.
Also, relying on just one OS puts you at the mercy of any latent bug in that specific system. Having a diversity of OSes in use mitigates that problem.
The state of Software Engineering in general is still pretty primitive. I'm still amazed at the poor quality of a lot of "professional" code and the cavalier attitude towards testing...In the land of the blind, the one-eyed man is king.
I work for a major European, high-end automotive company and we use Linux for radar applications. Now that this works nicely, we can do stuff like braking automagically (you still need to slightly tip the pedal) with the lowest possible deceleration which will assure you don't hit the vehicle in front of you.
That means, even if you are trailed by a somewhat sleepy person that car has the maximum reaction time to also hit the brakes.
The excellent quality of Linux now shows in all sorts of applications from affordable spaceflight to leading-edge trading at Eurex. Indeed, the better stuff eventually wins out.
Having said that, the corporate standard desktop still is Windows 7. People love the shiny stuff and the shiny stuff they can make with MS Office. We have lots of badly informed people who think MS products are better by default. One guy recently mentioned "imagine you would have to use Qt instead of MFC, how bad would that be !!".
we don't actually want it to "stop itse;f" as much as we want it to either "gracefully recover" or "gracefully die". For our UAV, when it jumps to somewhere it shouldn't be, it goes to a piece of code that determines whether or not it was in a flight safety critical loop. If it wasn't, it gives up control for the rest of that cycle and resyncs with the other flight computers on the next 5Hz cycle. If it was in a flight safety critical loop, it sends the "I'm brainfucked and dying" message to the other processors and dies. We use that design because some of the guidance and nav software is stateful and we keep the processors in lockstep. However, for auxillary actions, the states are very simple, so if one gets out of line, it will get fixed with the next 5hz message on the cross-channel datalinks.
Oh, and we don't use a virtual memory machine. Why add the complexity. Things like machine vision which really do need malloc()s shouldn't be in flight critical code. There are no memory leaks if you don't allocate memory. If you need to allocate memory, you're doing it wrong. These should be very deterministic creatures.
Okay, somebody ban this guy...
"You must be new here".
Do you actually believe that trolls are "banned" at Slashdot?
That's what the moderation system is for.
Slashdot is not like other "forums" in that it is *not* "moderated" by "super users", but rather regular users like you and I who are occasionally gifted with "mod points".
The "offending" post is never removed, it is just pushed below most users viewing threshold.
Seriously, "ban this guy"? You *MUST* be new here...
If you want news from today, you have to come back tomorrow.
In his team, they have a full-size Justin Bieber cutout that gets placed facing the team member who broke the build. They found that "100% of software engineers don't like Justin Bieber", and will work quickly to fix the build problem.
You see, that's why you have overflowing prisons. This would easily reduce the crime rate by a factor of ten!
Doubtful. At my $lastjob we had a rule that if you broke the nightly build you bought doughnuts for everyone. And the project lead would rip you a new one.
Despite my admonitions to not check stuff in at the end of the day we had two guys that just couldn't figure it out. One of them worked in St. Petersburg (Russia, not Florida) and he'd check stuff in at the end of his day and go home, meaning we'd be stuck with the dirty job of backing his stuff out so that we could proceed.
And the local guy would whine and cry about how it wasn't his fault, it worked in his tree, yada yada yada. Well, his tree was usually a few days out of date by the time he was ready to check his stuff in, and he just couldn't get the knack of rebasing his tree and building before committing to the master. Sheesh. This stuff isn't rocket science. And as I said, he insisted on doing this at the end of the day – every time. Eventually it cost him his job.
So no, I don't believe the threat of being stared at by a full size cutout of the Biebs would solve crime either.
Or both?
The Tao of math: The numbers you can count are not the real numbers.
malloc() and new() are non-deterministic in many ways and therefore to be banned in anything truely real-time.
Don't worry. We now have garbage collected languages where we don't need malloc/free any longer. :-)
The Tao of math: The numbers you can count are not the real numbers.
NASA sent out spaceships to moons and brought them back to earth, back in the 1960's
They did that without using Linux, or Unix, or any type of "nixes"
If there is a real need to learn a "lesson", methinks the best lessons we can learn are from those who develop original programs for NASA
Nobody else even comes close
Muchas Gracias, Señor Edward Snowden !
Linux is mentioned twice in the summary. Is there a reason why? We all know Linux has major use in the embedded and scientific world, this isn't a secret. Are people still desperate to mention Linux anywhere as some form of validation that they chose the right decision to stick with it or something?
I thought Linux had "won", or something. If that was the case then it shouldn't be necessary to circlejerk the name anymore.
Emacs Makes A Crappy Spaceship. That's why!
Ezekiel 23:20
Pascal, Ada and Algol are CRAP. They are in no way 'simple', they are 'verbose' and 'clumsy'. They are difficult to write code in and they don't really support anything that helps to find any non-trivial bugs. Functional programming, on the other hand, makes it much easier to use formal verification methods.
So no, I don't believe the threat of being stared at by a full size cutout of the Biebs would solve crime either.
Then maybe you just need to up the ante. How about have the real Bieber sitting naked on the edge of his desk seductively blowing him kisses?
Some of what I say is fact, some is conjecture, the rest I'm just blowing out my ass...you guess.
Back in those days, the problem with "random flip bit", brought on by space radiation was already present
Apparently NASA has successfully dealt with that phenomenon - or a lot of NASA's spacecrafts would have spun out of control
Since NASA has decades of experiences dealing with fascinating problems like that it's UTTER FOOLISHNESS if we do not learn from NASA
Muchas Gracias, Señor Edward Snowden !
They are only non-deterministic if you don't know what you're doing.
A successful API design takes a mixture of software design and pedagogy.
Sorry to burst your bubble, but some of Pascal's syntactic features were conceived by people who demonstrably had no idea how people -- the software developers -- actually process information. You see, programming languages are first and foremost tools for people. The code must be readable long after it has been written. Pascal's arbitrary separation of variable declarations from the first point-of-use is positively unergnomic and runs against the need for locality-of-reference as an aid to understanding. This stupidity has been propagated into IEC 61131 PLC programming languages. I'll take modern C/C++ definition-at-point-of-use any day, thank you so much.
A successful API design takes a mixture of software design and pedagogy.
Why the fuck wasn't the build done automatically before a commit would go through? I mean, what the heck?
A successful API design takes a mixture of software design and pedagogy.
No, no. You must be new here. The moderation system is there so you can impose your opinion on otherwise reasonable posts. That's what it's used for, that's what it's best at, Slashdot refuses to change it (I'd say fix it, but it appears to be working as intended) and so... one must conclude that is what it was designed for.
I've fallen off your lawn, and I can't get up.
Stack-based appreciation is frowned upon. You will express your feelings in algebraic terms or be severely modded down.
I've fallen off your lawn, and I can't get up.
Now THAT is hilarious. Complaining that malloc is non-deterministic and then alluding to dependence upon garbage collection. I get the distinct impression you've never written anything requiring high performance memory allocation/deallocation.
I've fallen off your lawn, and I can't get up.
You're right of course, but I'm getting REALLY over the same guy spamming with the same post in so many threads. At SOME point, getting everything you post instantly modded to -1 (especially when the content is near identical) should incur some greater punishment.
Sledgehammer random components until it works?
They have to, the pool of experienced aerospace people is small and shrinking due to old age. There is only one way to produce more people with experience in aerospace and it does not involve H1-B or wishing really hard.
Actually hard realtime Linux exists for a while -- just in different forms.
RTAI
Xenomai
PREEMPT_RT
There are some hardware architectures (actually one very popular hardware architecture) that usually have unpredictable crap running in background through OS-inaccessible interrupts, but that's the problem with irresponsible BIOS vendors, and it breaks realtime on all operating systems. Once that crap is disabled, even that architecture allows hard realtime -- I participated in a project that used just that.
Contrary to the popular belief, there indeed is no God.
That was actually very easy to believe in 1993. What was hard to believe is that THERE WILL BE SO FEW SPACECHIPS IN 2013, dammit!!!
Contrary to the popular belief, there indeed is no God.
T.A.R.D.I.S: the only hardware where bogosort is the most optimal sorting algorithm.
Contrary to the popular belief, there indeed is no God.
Dude, 70ies are over, you are not writing software on a teletype. There is nothing wrong with verbosity.
And no, the languages you have listed are not particularily difficult to write code in, the only difficult thing to do is writing ugly hacks.
"It's such a fine line between stupid and clever" -- David St. Hubbins, Spinal Tap
They ARE difficult to write in. Try creating a multimap in Pascal, with type safety or course.
Oh, no generic types in Pascal so it's impossible. Ok, try at least a polymorphic version - again not easy because of braindeadness of pointers in Pascal. BTW, there's no garbage collector in Standard Pascal.
Advocating braindead languages from 60-s is not even funny now.
Well, standard Pascal is obsolete, no argument about it. But that is true for any language from the sixties.
By the way, if I remember correctly, there are generic types in modern Pascal dialects like Delphi or Free Pascal.
"It's such a fine line between stupid and clever" -- David St. Hubbins, Spinal Tap
Modern Pascal dialects are hardly 'simple', they have all the OOP stuff and lots of somewhat clumsily implemented extensions (like closures). So Algol or Standard Pascal are in no way 'simpler' or 'reliable' - they are just so primitive that most of programs written in Pascal are little more advanced than a textbook exercises.
So now we've got rockets that run Linux... I'm shocked, shocked! that no one has stooped low enough to say this yet, so let me be the first to stoop...
Can you imagine having a Beowulf Cluster of THESE?!
XML is like violence. If it doesn't solve your problem, you're not using enough of it. --AC
What was hard to believe is that THERE WILL BE SO FEW SPACECHIPS IN 2013, dammit!!!
The salt and crumbs tend to get into the controls. Also, you don't even want to talk about space salsa.
"You're right," Fisheye says. "I should have set it on 'whip' or 'chop.'"
hard real time
My God, it's Full of Source!
OUTSIDE_IP=$(dig +short my.ip @outsideip.net)
You have it 100% correct. taco cowboy is so far off with his rant, that I am shocked that he was modded up. Sadly, all he is trying to do is defend NASA, but he is actually making a fool of them, not helping them.
I prefer the "u" in honour as it seems to be missing these days.
wrong. Hard RT means pre-allocated time slices. IOW, you KNOW exactly how long something will take and you allocate exactly that amount of time. ALWAYS. And rad-hard has to do with a CHIP, not the OS.
I prefer the "u" in honour as it seems to be missing these days.
Dude, if the language is Turing-complete you can write anything in anything. If you can't think of a way, that's your failure of imagination, not a failure of the language.
Hell, the Apollo spacecraft were programmed in the languages of the 60s ... if not the 50s.
-- Alastair
Pascal was designed to be (a) easy to teach and (b) easy to compile (not necessarily in that order).
Declaration at point of first use can break programs in block-scoped languages like Pascal or Algol. It's easier to teach a newbie to just declare everything in the outermost block (of a given procedure/function) than worry about whether something will still be in scope a few 'end' delimiters later.
If your procedures are of reasonable length (no more than a few dozen LOC), that's locality of reference enough.
(And a recursive-descent compiler for (original) Pascal is dead easy, maybe 3 to 4 KLOC in Pascal.)
-- Alastair
[snarktag]The solution is clear, keep parameters in an XML file. Use base class reference objects and inject the proper objects at runtime. If vehicle=spacecraft inject Dragon engine controller. If vehicle=rocket inject Falcon engine controller. If vehicle=Roadster inject Tesla electric engine controller.[/snarktag]
There, now you can have 1 code base for your cars too!
"Happy families are all alike; every unhappy family is unhappy in its own way." -- Anna Karenina by Leo Tolstoy
"Beware of the Turing tar-pit in which everything is possible but nothing of interest is easy." ( http://en.wikipedia.org/wiki/Turing_tarpit )
Why a fundamental data structure doesn't belong in safety-critical apps? It also can be completely deterministic, if required.
Okay, somebody ban this guy, or if you can't do that, then impose a maximum length restriction for postings.
As much as the guy annoys me this is the prime example of where free speech is important and basically he has the freedom of speech to communicate that he is a jerk. Now his message might be very important however, whatever message this fellow has is lost on annoying the audience because their mental filters kick in and block his message. I'm actually amazed at the energy he puts into the post, oooppps, he OR she.
I cannot stand what this guy is going on about, but I will defend his right to say it even if I want to give him a solid punch in the balls. That feeling of annoyance is very re-assuring indeed, I wonder how long free speech for jerks or, anyone else, will last.
My ism, it's full of beliefs.