NASA Update Will Deal With Opportunity Flash Memory "Amnesia"

BarbaraHudson writes Computerworld has some details on NASA's latest fix to allow the Opportunity Mars Rover to store data while in overnight "sleep mode." Opportunity has been suffering from a glitch that's causing what NASA scientists describe as memory and data loss — or robotic "amnesia" — caused by flash memory deterioration since early December. Currently any information gathered is stored temporarily in RAM and must be sent to Earth so it's not lost when Opportunity powers down.

Comments

Flash memory sucks

[Russ1642]

This is why I'll never use an SSD in my computer. They're just unreliable.

Re:Flash memory sucks

[Shakrai]

And you think a spinning hard drive with platters and heads would be a better choice for a spacecraft that has to endure several G forces worth of acceleration at each end of the trip? Or perhaps it's a better choice to use magnetic media than solid state in high radiation environments like interplanetary space?

Re:Flash memory sucks

[gstoddart]

While that may or may not be true, it's kind of random.

These things were intended to last 90 days or so. It's been 11 years.

At this point, if they're still able to apply updates to make fixes to the damned thing it means this has outlived its originally planned lifespan by a massive amount of time.

I hardly think that's a fair critique of SSD in general. I'd say pretty much every part of that rover has performed well beyond anything it was ever expected to.

Re:Flash memory sucks

[Redbehrend]

I run SSD's on all my computers and never had one fail yet and when one does, it takes less than 10 min to get a new one up and running. (Already tested my restore plan with my backups) In the meantime I enjoy the awesome performance and get back my 10 min and money in x1000 from the time it saves lol.
Most of the ones that fail are the cheap clearance, no name ones, the good Samsung and Intel ones are developing a great track record.

But on topic, the rovers are all custom and it's been 10+ years, i believe they already know about this flaw now and they have found ways around it with newer flash memory.

Re:Flash memory sucks

[Shakrai]

I'd say pretty much every part of that rover has performed well beyond anything it was ever expected to.

All except for The Illudium Q-36 Explosive Space Modulator.

Re:Flash memory sucks

[bluefoxlucid]

Actually, the amount of NAND in Opportunity is 256MB, which isn't a lot. RAM is only 128MB; NAND is routinely given a full wipe and rewrite. Given it was only meant to last for 90 days, it's probably a low-grade MLC with under 5000 erase cycles reliability.

By contrast, a desktop SSD of 32-256GB will see a daily write cycle of under a gigabyte per day. The documents, cache, e-mails, and updates per day total very little--hundreds of megabytes, at best--for most workloads, up to and including intensive workloads such as 3D modeling and multi-layer 2D image editing, which produce massive work sets but only make minor changes to them. Large workloads include video editing, which does write multi-gigabyte files out on each large render operation (which may be frequent in some workflows).

System drive SSDs see small workloads even when used as a swap device: swap offloads a lot of stale memory from RAM, which is either hardly ever used, never written to, or simply stale. The brk() area can have fragmented holes too small to take new allocations, and so may page out entirely unused RAM to disk; much RAM (such as GUI elements, textures, models, and audio assets) contains load-once data that's written to disk when swapped, and then only read later, such that it's left on swap and evicted from RAM without writing out again when memory gets scarce. That means 2GB of swap might be 2GB of writes since boot time, plus maybe a hundred megabytes or less per day of continuous system run.

Finally, system drives also wear level internally. Writing to the same 1MB area over and over will spread the writes out: the controller will internally account for those blocks being erased and rewritten elsewhere, often by additive writing (writing without erasing) to avoid wearing the drive (e.g. you can have a used and erased map, in which anything erased or not used is free; you can then write to the used map and to the erased map, until you run out of free blocks and need to erase a newly used block, and thus need to mark all used-erased blocks free and mark no blocks as erased, costing one write-erase cycle even though you've written thousands of times). You'll get a full write-erase cycle every full data width write: even if you write to the same spot repeatedly, you only use one write-erase cycle writing 32GB to a 32GB drive, or 256GB to a 256GB drive.

Accounting for all this, the drives are quite long-lasting. For a 10,000 cycle drive, you'd have to write 320,000GB to a 32GB SSD or 2,560,000GB to a 256GB SSD to wear it out. That's 876 years at 1GB per day for a 32GB drive, or over 8 years if you're writing 100GB per day. For a 256GB drive, it's 7000 year-gigabytes, or 70 years if you're writing 100GB per day. Modern MLC NAND can survive 100,000 write-erase cycles before failure, so these lifetimes may be 10 times higher; high-end SLC drives can survive 10,000,000 write-erase cycles, and can back high-traffic SANs for decades.

They actually burn out less frequently than hard drives.

Re:Flash memory sucks

[Kjella]

Actually, the amount of NAND in Opportunity is 256MB, which isn't a lot. RAM is only 128MB; NAND is routinely given a full wipe and rewrite. Given it was only meant to last for 90 days, it's probably a low-grade MLC with under 5000 erase cycles reliability.

I very much doubt that. If you're going to spend that amount of money on sending it to Mars, you don't skimp on off-the-shelf technology that costs a few hundred bucks. You may have to ditch developing that custom system that'd increase the mission budget with a million dollars, but that's different. And if you want to make it as radiation-hardened as possible I doubt they'd go with anything but SLC for maximum signal strength. I doubt it has anything to do with write cycles which they presumably have full control over at all, it's probably from operating in the harsh environment for 11 years. No wonder it's getting a bit flaky.

Re:Flash memory sucks

[The Grim Reefer]

several G forces worth of acceleration at each end of the trip?

That must be what caused that WHOOSH.

Re:Flash memory sucks

[Anonymous Coward]

Voyager's magnetic tape drives would like to have a word with you

Re:Don't worry, NASA is really good at this

[bobbied]

Keeping old relics and forgotten dreams alive is what they do best.

And here I thought it was being able to successfully launch people in billion dollar vehicles built by lowest bidders, loaded with millions of pounds of high explosives without killing anybody (usually)

Re:Don't worry, NASA is really good at this

[ArcadeMan]

(puts Mars bar on the floor)
"Come here boy!"
(rover sits on Mars)

(clicks on big Staples button) That was easy

Re:Don't worry, NASA is really good at this

[NoNonAlphaCharsHere]

Heh. I took the train into Chicago for many years. There's a Mars plant next to the line, with a passenger stop. Every morning the robot-voice would announce "The next stop will be ... Mars".

FRAM vs NAND

[volvox_voxel]

I've never been a big fan of flash memory, given that it has a finite number of write cycles before a memory bit fails (varying between 1 and 100million write cycles). The probability may be low that an individual bit may need to flip so many times in it's lifetime, but it's still an issue.. A lot of care must be taken by the firmware engineer to handle this. There are a lot of job postings for firmware engineers that understand flash..

I'm a huge fan of FRAM. It has a lifecycle limit that is quoted at being 10 trillion write cycles (some mention at it being infinite). The memory density is lower, but is a lot more reliable. It's biggest issue is that the density is lower. For a spacecraft, I'd much rather have a board of these 2Mbit FRAMS then a large flash chip. They use these things in smart meters, etc. In embedded systems, you have to be really careful not to write to the flash too often out of risk of damaging the flash. Most fast SD cards have their own dedicated microcontroller (ARM9, etc) to do what they can to extend the life of the flash..

A datasheet of an FRAM device: http://www.fujitsu.com/downloa...

One question I have is how FRAM compares to NAND-flash in a harsh radiation environment, and what are the radiation differences on mars vs the earth. How many vendors offer rad-hard processes for FRAM, and how do they perform?

Here is one link I could find on FRAM, but the report from 2011 is not clear:

http://cdn.intechopen.com/pdfs...