Slashdot Mirror
Glitch Forces Mars Probe Shut-Off
TheEvilOverlord writes to tell us that BBC News is reporting that a 'glitch' has forced the mars probe to switch into a "safe-mode" switching several of its systems off. Perhaps it is just an education tool to show martians the meaning of BSOD.
People, we've got be a little more precise. There are three functioning spacecraft in orbit, one more on the way, and two on the ground on and around Mars. Between NASA and the ESA, it's getting crowded on Mars.
For those who didn't RTFA, it's the Mars Global Surveyor, which is the oldest of those in orbit.
In this mode, MGS turns off its science instruments but leaves many other systems on.
Presumbly this includes communications.
We're not talking about Windows safe mode here, It was switched onto safe mode because a glich was occuring.
In safe mode to turns to the sun to funny charge its battery and allows communication with Earth on its low-gain antenna. Which is presumably to work out what is wrong makes changes and reboot the system.
You have to remember that this craft has been in space since 1996 so is old and might have radiation damage by now so this kind of thing is to be expected.
"In a time of universal deceit - telling the truth is a revolutionary act." - George Orwell
http://www.fastcompany.com/online/06/writestuff.ht ml
well, TFA is about the space shuttle, but i'd guess it applies to other space gadgets, too.
"Consider these stats : the last three versions of the program -- each 420,000 lines long-had just one error each. The last 11 versions of this software had a total of 17 errors. Commercial programs of equivalent complexity would have 5,000 errors."
I know some of their stuff (i.e. the MER rovers) use VxWorks. See http://www.space.com/businesstechnology/technology /mer_computer_040128.html and http://www.windriver.com/news/press/pr.html?ID=355
Mars rovers use WindRiver VxWorks, probably version 5... who needs a BSOD when you have this little favorite "workQPanic: Kernel work queue overflow"... ugh.
h owpage&pid=227 , to be fair, these guys go above and beyond to try and prevent glitches and random crappola.
check out http://acmqueue.com/modules.php?name=Content&pa=s
-j
Apparently this probe (the Mars Global Surveyor) went into "safe mode" before in its mission.
e nt.html
http://mars.jpl.nasa.gov/mgs/status/wkreport/curr
Malin Space Science Systems http://www.msss.com/ is the company that runs the cameras on MGS, among other Mars probes. The MGS uses VxWorks, a specialized OS for robotic craft, which I believe runs on both *nix and Windows. I think VxWorks is OSS, although I also think some companies use customized versions. The specs for MGS http:///http://mars.jpl.nasa.gov/mgs//scsys/cdhsin fo.html> indicate two 1750A and one 8086 processors, which are probably running VxWorks in a Unix-like environment. The 1750A is a military standard processor (exceedingly old now, but rugged). Check out that specs page; very interesting.
In safe mode, MGS uses 20K PROM for the Command/Data subsystem!
Since Malin website indicates jobs for Unix/C programmers, I'm assuming that their stuff is running in VxWorks in a Unix-like environment, probably a BSD varaiant.
BTW, the Mars rovers use a PPC 601-like chip that runs at a screaming 20 mHz.
Wind River Systems built the POSIX compliant based OS into the Odyssey, Stardust and Rovers, so it's possible the MGS has a similar OS to those.
g ies/os/vxworks6/
0 3/mars/jump/2404308
The OS is VxWorks and it's been used in Sattelites, Robots and for some reason Movie editing (probably a file management system)
http://www.windriver.com/products/device_technolo
http://www.chron.com/cs/CDA/ssistory.mpl/special/
But only limited communications. The primary channel, which allows high bandwidth communications, is more fussy to maintain (antenna has to be pointed in the right direction, etc.) In 'safe mode' the probe uses a secondary communications method that is more reliable but much lower bandwidth.
You missed the point: 'safe mode' is the fallback system. The probe is not in a 'known state'. They do not yet know what fault caused the probe to go into safe mode. Safe mode is a generic state designed to maintain power and communications with Earth when an unknown fault has occurred, to give the operators time to find a solution or just to check that everything is OK. When they are sure everything is OK they will send a command to exit safe mode.
There is reason to be cautious. An important satellite was lost a few years ago because the operators rushed their diagnosis of the problem. The satellite's operating state was not what they thought it was--two of the gyros had failed. The operators misinterpreted the data and switched the satellite from its last working gyro to one of the nonworking ones, resulting in total loss of communications and destruction of the satellite. Not good. Before they do anything, they need to be sure they understand what has happened and what state the probe is in.
VxWorks from Wind River Systems.
Pathfinder and the two rovers are the only ones I'm sure of as far as the Mars units go but some of the other space probes, Stardust,Deep Imapct and Deep Space 1 also used VxWorks.
I do know that the "safe mode" the probe has entered means that something locked up the main program long enough for a hardware watchdog to go off, rebooting it, during boot the init code checks what state the system is in, ie; did the WD go off, and if it did then it boots a known good App/OS package that will at least allow the probe to get in contact with Earth so it can be fixed.
I don't know for sure if NASA is doing this, I know I told the JPL people I met about it, but I know that with VxWorks it is possible to store data in the upper area of RAM and prevent the init code from clearing it so that a core dump of sorts can be used to isolate the problem so the main code can be adjusted, if needed, to correct the issue. This is what they did with Pathfinder when the base station kept rebooting every 20 min due to a priority inversion issue with a semaphore.