Slashdot Mirror
Debugging The Spirit Rover
icebike writes "eeTimes has a story on how the Mars Rover was essentially reprogrammed from millions of miles away. 'How do you diagnose an embedded system that has rendered itself unobservable? That was the riddle a Jet Propulsion Laboratory team had to solve when the Mars rover Spirit capped a successful landing on the Martian surface with a sequence of stunning images and then, darkness.' The outcome strikes me as an extremely Lucky Hack, and the rover could have just as likely been lost forever. Are there lessons here that we can use here on the third rock for recovery of our messed up machines which we manage from afar via ssh?"
Are there lessons here that we can use here on the third rock for recovery of our messed up machines which we manage from afar via ssh?
As a former co-worker (hi, jwalker!) used to say when people tried to draw ridiculous analogies, "It's exactly like that...only different."
A programmer is a machine for converting coffee into code.
Man, I have a hard enough time debugging programs running on my local machine.
I dont think i want to learn too much from this as the solution was the equivalent of rm -rf... On a side note i wonder when the 40 min ssh delay jokes will begin again
drunk chemists
at least it wasn't a blue screen?
man rover?
The Human Cow - bringing you scrumtrelescence since 1995
I don't get it, couldn't NASA afford the on-site warranty?
Wow, I didn't expect the rover had 128MiB of RAM, or 256MiB of flash. Funny to think they had to run chkdsk from so far away :)
I hope they use SSH or something .. who's to say a future mission ..some hax0r doesnt grab control of a space probe and have it send goatse.cx pics back??
.. after all the probe communicates using known frequencies. There may be probs picking up the return signal without an expensive antenna i suppose. But then again maybe some hax0r can build one cheaply and or do what captin midnight did ( www.signaltonoise.net/library/captmidn.htm ).
All it takes is a transmitter out in the middle of nowhere africa or some island
I wouldnt worry about signal jamming though as that will probably be discovered easily.
I couldn't afford a car that NASA built... :P
The Martians are pissed that the repair labor was outsourced to Earth.
Table-ized A.I.
Sounds like NASA forgot to empty the rover's recycle bin. =)
Steal This Sig
I routinely reboot and reprogram machines in our data-center that is 2000 miles away from me.
As long as all hardware components are working and there is connectivity to the machine, it doesn't matter whether the machine is a few miles away or a million miles away.
You are too humble, friend. What you do routinely and without thinking, is nothing less than a miracle of modern science. A miracle that you take part in every day. And because of men like you, we don't have to rely on the abacus anymore. We sent a pentium to the Moon, and soon, Mars will be colonized by G5s. America salutes you, for all the things that you do.....
Like a rock! I was strong as I could be be!
Ooooooohh! Like a rock!
(-1, Raw and Uncut is the only way to read)
Yeah, that was HAL's excuse too.
Seriously, hats off to all the JPL programmers. Proving to the Martians that there is indeed intelligent life on Earth, very intelligent.
My pet peeve when I'm doing remote troubleshooting is 'ifconfig eth0 down'...oops. At least NASA is smarter than that.
Peter.
You know what I hate? Wait, what do you like? I hate that!
The only real bug was the inability of the system to properly handle running out of file entries (or more specifically, consuming too much RAM as the number of file entries increased). However the software should have never have stressed the filesystem to that degree in the first place.
When you can write an embedded operating system that can gracefully and automatically recover from every possible thing that might ever go wrong, perhaps you should send your resume to NASA.
NO CARRIER
Operating System not found. Press any key to continue.
Damn! Left the floppy in!
Duh. That's what they have been keeping a secret. They have a DB9 serial link strung from here to the landing site. It's not as cool as you all make it out to be.
I think this is exactly what he means. We get the beginning of the story, but then, no followup!
I've had this sig for three days.
They could have set it up out in my backyard to take pictures of the piles of crap and rocks out there and if they wanted to simulate the solar radiation, they could have my girlfriend give it one of her famous looks... cause those are leathal enough to burn a hole in your soul.
-SF
I wonder how many Microsoft salesmen were pushing for putting WinXP on it..
Serious? Seriousness is well above my pay grade.
That must have been some feat to get the arm on the rover to press Ctrl, Alt and Delete at the same time!
Today we salute YOU, Mr. Super Wizard Windows Reinstaller.
Only YOU can fully appreciate the difficulty of running a format c: command, while swilling a room temperature can of Red Bull.
"Hey this stuff is hard now!"
While NASA is too preoccupied with things like farway rovers, you take your vocational tech school fueled arrogance directly to the place where it will make the absolute least possible impact: A Slashdot discussion thread.
"Loggin' on now!"
Your unique eye for obviousness allows you to sling turds of obtuseness every which way, and then brag about how you were RIGHT as soon as one of your pronouncements hit true - regardless of how many times you were wrong before.
"See I told you sooooooo!!"
And if some idiot rocket scientist has the unmitigated gall to not bow down to your obvious Geniusdom, you unleash your fury down upon him with all the tenacity and mercilessness of a rabid pit bull with a tender buttock locked in its jaws.
"Total anonymity!"
So keep clicking away, oh Marauder of the Mousepad. Because when the results you so desire finally come about years from now, you can say it was because YOU demanded it."
"How come they haven't fired that dumbass head of NASA yet yet?"
(Bud Light Beer, Anheuser Busch, St. Louis Missouri.)
Ring, Ring, Ring....
"Welcome to the Mars Rover answering system. For English press 1, Para Espanol prensa 2"
BEEP
"You selected English. To leave a message for Spirit press 1. To leave a message for Opportunity press 2"
BEEP
"You selected Spirit. Transfering now." CLICK "I'm sorry, Spirit is unavailable at this time. To leave a message press 1. To return to the main menu press 2"
BEEP
"Hi this is the Spirit rover. I can't come to the phone right now but if you'll leave a message I'll get back to you." BEEEEEP
"Spirit, this is NASA. Please phone home when you get a chance. I think your fax machine has jammed and we need you to re-send. Thanks, bye"
It's a PC user's nightmare: You're almost done with a lengthy e-mail, or about to finish a report at the office, and the computer crashes for no apparent reason. It tries to restart but never quite finishes booting. Then it crashes again. And again.
Gee... that sounds a lot like the last worm to hit my mom's Dell Laptop running Windows XP.
Damn gentoo zealots always trying to plug their distro in any threa..
Oh wait did you say debian? Nevermind.
That's why I always keep a spare bag or two of inodes on hand, just in case. They're small so they don't take up too much space in the closet. I store them next to those f-stops I used to use for photography.
http://www.rootstrikers.org/
The JPL is a pretty viral license. It forces you to spread their space probes from your planet to all your customer's planets. This is un-solar systematic! What's next? Calling GNUpiter Jupiter instead?
When building the rover, they probably just put vxworks installation cd in, and selected "Typical Install for inter-planetary missions", clikced "Next" a couple of times and got the OS running in no time.
Now, if the NASA engineers are anything like me, they didn't bother to check what was being logged and where... after all, when a problem arises, then you can go to /var/log/ to see if there's anything interesting.
Not lost forever, but lost until we travel to Mars and retool it as an extraterrestrial barbeque grill.
"I'll say it again for the logic-impaired." -- Larry Wall.
Just imagine the poor bastard who had to carry the tomsrtbt floppy all those millions of miles up there, and stick it in the floppy drive!
Well, we all have to do stuff like that every day, and God help us if we run out of Magic Wands!
We salute you and Bud Light salutes you, Mr. Three Finger Remote Computer Rebooter. >You are too humble, friend. What you do routinely and without thinking, is nothing less than a miracle of modern science.
You mean like the time I sent myself 22000 mails because the OS in question didn't have an implementation of sleep.