Slashdot Mirror
Can SSDs Be Used For Software Development?
hackingbear writes "I'm considering buying a current-generation SSD to replace my external hard disk drive for use in my day-to-day software development, especially to boost the IDE's performance. Size is not a great concern: 120GB is enough for me. Price is not much of a concern either, as my boss will pay. I do have concerns on the limitations of write cycles as well as write speeds. As I understand, the current SSDs overcome it by heuristically placing the writes randomly. That would be good enough for regular users, but in software development, one may have to update 10-30% of the source files from Subversion and recompile the whole project, several times a day. I wonder how SSDs will do in this usage pattern. What's your experience developing on SSDs?"
I'm using the Intel SSD and I think it's great - fast and silent. Will it last? I'd argue you never know about any particular model of hard drive or SSD until a few years after it is released. On the other hand, I'd also argue it doesn't matter much. Say one drive has a 3% failure rate in the 3rd year and another has a 6% rate. That's a huge difference percentage-wise (100% increase). And yet it's only a 3% extra risk - and, most importantly, you need a backup either way.
Do you have a swap file/partition? You're talking hundreds of writes a day, tops. That sounds like a big number, but in reality it just ain't. I would question why you feel the need for an SSD, though. I know the difference between $300 and $50 isn't that big in the grand scheme of things, what benefit are you expecting?
Whale
Unless you type like The Flash, even MLC SSDs from the better vendors (Intel) should be fine for anything outside of server applications. Simple math should back this up (how many GB total the drive can write over its lifetime vs how much you produce each day). merlin
Math is hard! Lets buy both!
"His name was James Damore."
If they're good enough for Databases (frequent writes), they should be just fine for devel.
OTOH, You should be a lot more concerned about losing data because of a) software bugs or b) mechanical failures in a conventional drive
The Raven
If you're worried about losing work, I think your backup solution is what you need to improve instead.
could a raid structure give the performance boost i assume you are after? ive no experiance with them but i gather they can offer higher read/write rates. Can someone with more experiance say exactly how much of a performace boost they give, a set of small HDD's could be the same price without the concerns over cycle limits
You should get an SATA SSD instead.
Give me Classic Slashdot or give me death!
I use SSDs for my (both) development systems--the first was for the work system, and after seeing the improvements I decided I would never use spinning-platter technology again.
The biggest performance gains are in my IDE (IntelliJ). My "normal" sized projects tend to link to hundreds of megs of JAR files, and the IDE is constantly performing inspections to validate the code is correct. No matter how fast the processor, you quickly become IO-bound as the computer struggles to parse through tens of thousands of classes. After upgrading to SSD, I no longer find the IDE struggling to keep up.
I ended up going with SSD after reading this suggestion for increasing IDE performance. The general jist: the only way to improve the speed of your programming environment is to get rid of your file access latency.
The company I'm working at thought about using SSDs, but we were thinking more on the server end (to allow faster database access.) You don't have to worry about the write limits as it's highly unlikely you will hit them within the lifetime of a standard hard drive.
The main issue we ran into was cost, the drives we were looking at started around $3,000 for something like 80 gigs. That just wasn't worth it for us, though if you personally feel that the added cost (and I doubt you're looking at a $3,000 SSD, more likely you're looking at the $300 drives) is worth the performance gains then go for it. Though I think even for $300 it won't make a worthwhile difference.
There are other bottlenecks to consider, is your CPU fast enough, do you have enough RAM, could the hard drive your software and OS is on use an upgrade, etc. Perhaps even buy an internal SATA drive (if you can) to replace the external you're using, those external enclosures generally aren't known for their performance. If you've exhausted all of those options and you still need more speed, then I'd say go for the SSD.
If you're not good enough at arithmetic to understand that this isn't an issue, should you really be developing software?
Maybe you can explain why it isn't an issue, then?
One thing about flash in general is that in order to rewrite a small amount of data, you need to (at the low level) erase and rewrite a relatively large amount of data. So depending on how extensively the filesystem is cached, where the files are located, etc., rebuilding a medium-sized project could wind up re-writing a large portion of the SSD...
Bow-ties are cool.
I'd say: "Programming is hard let's do Java"
how long until
If you give your programmers an 8-way 4GHz m/b with 64GB of memory (if sucha thing exists yet), they'll use all the processing power in dumb, inefficient algorithms, just because the development time is reduced. While those of us in the real world have to get by on "normal" machines.
When we complain about poor performance, they just shrug and say "well it works fine on my nuclear-powered, warp-10, so-fast-it-can-travel-back-in-time" machine"
However, if they were made to develop the software on boxes that met the minimum recommended spec. for their operating system, they'd have to give some thought to making the code run efficiently. If it extended the development time and reduced the frequency of updates, well that wouldn't be a bad thing either.
politicians are like babies' nappies: they should both be changed regularly and for the same reasons
Current SSDs have a lifetime of somewhere around 10.000 years. I think that's enough.
10000 years or 100000 writes, whichever comes first. :D
Bow-ties are cool.
Just got one in a Dell laptop, came with Ubuntu. A subjective overview:
I have no idea how well it performs with swap. I'm not even really sure why I have swap -- I don't have quite enough to suspend properly, but I also never seem to run out of my 4 gigs of RAM.
It's true, the write speed is slower. However, I also frequently transfer files over gigabit, and the bottleneck is not my SSD, it's this cheap Netgear switch, or possibly SSH -- I get about 30 megabytes per second either way.
So, is there gigabit between you and the SVN server? If so, you might run into speed issues. Maybe. Probably not.
Also worth mentioning: Pick a good filesystem if a lot of small files equals a lot of writes for you. A good example of this would be ReiserFS' tail packing -- make whatever "killer FS" jokes you like, it really isn't a bad filesystem. But any decent filesystem should at least be trying to pack writes together, and I only expect the situation to improve as filesystems are tuned with SSDs in mind.
It also boots noticeably faster than my last machine. This one is 2.5 ghz with 4 gigs of RAM; last one was 2.4 ghz with 2 gigs, so not much of a difference there. It becomes more obvious with actual use, like launching Firefox -- it's honestly hard to tell whether or not I've launched it before (and thus, it's already cached in my massive RAM) -- it's just as fast from a cold boot. The same is true of most things -- for another test, I just launched OpenOffice.org for the first time this boot, and it took about three seconds.
It's possible I've been out of the loop, and OO.o really has improved that much since I last used it, but that does look impressive to me.
Probably the biggest advantage is durability -- no moving parts to be jostled -- and silence. To see that in action, just pick out a passively-cooled netbook -- the thing makes absolutely no discernible noise once it's on, other than out of the speakers.
All around, I don't see much of a disadvantage. However, it may not be as much of an advantage as you expect. Quite a lot of things will now be CPU-bound, and there are even the annoying bits which seem to be wallclock-bound.
Don't thank God, thank a doctor!
If price isn't an option, then he should get himself 4 ANS-9010's and set them up as a hardware RAID0 hanging off the back of good fast raid controller.
If he filled each of them with 4GB DIMMs he'd have 128GB of storage space.
Volatile? Hell yeah... But also just crazy fast...
Yes Francis, the world has gone crazy.
Neither he nor you have attempted to answer the question quantitatively. Look at how big a block is, a bit about their write-leveling strategy, how large your source files are, the quantity of data you overwrite and how frequently, and what the lifetime of SSD blocks is, and figure out how long the SSD should last. Even an order-of-magnitude calculation would be better than nothing.
You both are approaching the problem qualitatively: SSDs have limited rewrite lifetimes, and I'm doing a lot of rewriting -- isn't that bad? You don't know! Figure it out!
Now find a hard disk that'll last that long.
extern warranty;
main()
{
(void)warranty;
}
Can't you just load up on RAM and create a RAM drive for working stuff and keep the slow HDD for shutdown time ? Cheaper than SSD and no write cycle issues. You can also get RAM based IDE and SATA drives.
I used to worry about rewrites on my eeepc. But I have installed ubuntu twice in the last month and the disk seems to be exactly the same as it was initially so I don't worry any more.
http://michaelsmith.id.au
Although they use an SSD for another purpose, they said currently SSD's last about 6 months under heavy read/write conditions (cache on a RAID controller) even with leveling techniques. Hard drives last a whole lot longer for those purposes I would say.
I think SSD in a desktop-type system would be all right however I would suggest you invest in some fast disks instead of SSD until SSD matures and more lifetime data is available. Remember MTBF doesn't always mean that a piece of hardware will last that long. Most likely it will die long before that.
Custom electronics and digital signage for your business: www.evcircuits.com
Java is hard, let's use Python.
Sometimes I wonder whether it would make sense to optimize the disk usage for flash drives by writing transient files to ramdisk instead of hard disk. E.g. in compilation, intermediate files could well reside on ramdisk. If you rely on "make clean" a lot (e.g. when you are rebuilding "clean" .debs all the time), you won't have that much attachment to your object files.
Of course this may require more work than what it's really worth, but it's a thought.
Save your wrists today - switch to Dvorak
Developing on a conventional SSD with large user-visible erase blocks is PAINFUL. The small writes caused by creating temporary files in the build process absolutely destroy performance. There are ludicrously expensive enterprise products which work around this in software, but at the laptop/desktop scale, you want something that's self-contained. As far as I'm aware, Intel's X25 drives are the only ones actually on the market now that hide the erase blocks effectively at the firmware level. The MLC ones should be fine.
There's no failure quite as dissatisfying as a complete and total solution to the wrong problem.
You are confusing programming and computer science.
THL phish sticks
Before we start, let me make a prediction: You never asked about the MTBF of your hard disk, right...?
http://www.intel.com/design/flash/NAND/mainstream/
a) When Intel says "new level of ... reliability", maybe it means they thought about this problem when they designed the drive.
b) When they say "NAND flash", maybe it means they're not using the cheapest MLC memory as mentioned in that scary wikipedia article.
c) When their datasheet says "Minimum useful life of five years, assuming 20Gb/day of writing", maybe they got those numbers from real engineers, with degrees.
d) When their datasheet also says, "Should the host system attempt to exceed 20 GB writes per day by a large margin for an extended period, the drive will enable the endurance management feature to adjust write performance, this feature enables the device to have, at a minimum, a five year useful life", maybe they were really really paranoid about saying 'five years' because they know people will start class-action lawsuits if it doesn't work out.
So, um, how this even got greenlighted in 2009 is beyond me. It's like 1999 called wanting its flash-myths thread back.
No sig today...
The English language has syntax, too. It concerns things like proper placement and use of apostrophes.
Cheaper drives (which mgmt is sure to require) have 1,000 write cycles (assuming the worst). For certain high-traffic files, that means (assuming 30 writes in a day) a whole 33 days of use.
If that were true. Then an SSD hard drive couldn't run a linux mail server for a small business for more than a couple minutes thanks to the various log files.
1) The maximum write cycles for a block was around 10,000 in 1994. And about 100,000 in 1997. But in 2009 you think 1000? No. Its currently in the millions, even for the cheap SSDs.
2) Look up wear levelling.
3) Look up the MTBF on an SSD vs a spinning platters type.
I've seen studies that have calculated that modern drives will could write continuously at maximum speed for 50+ years before exhausting wear levelling and hitting write cycle limits.
The odds of it failing from something else long before then are much greater. Getting a mere 5+ years of life and easily beating your average spinning disk hard drive is a no brainer.
yet, but I am eager to learn. What happens if you exceed the limit of writes? How does usage degrade the disks? Is heat bad? Does using the SSD as virtual memory degrade the disk fast?
What about bad sectors, how do they compare with HDDs? Are SSDs generally more sturdy(longer lifespans) than HDDs?
Inquiring minds want to know.
Knowledge is power. Knowledge shared is power lost.
Perl is hard. Let's use brainf*ck.
Check out my sci-fi/humor trilogy at PatriotsBooks.
However, suppose I fill up the drive with data, then free half of it. My question is: how does the drive know that half its sectors are free again for use in wear leveling? As far as the drive knows, all of its sectors still hold data from when the drive was full, and no sectors are available for levelling purposes.
Is there some protocol for telling the drive that "sectors x, y, z are now free"? Or does the drive itself understand the disk layout of the zillions of different filesystems out there?
This is why it's almost pointless to ask a question on Slashdot. You get 100s of replies in a 50/50 distribution of random tech-word ramblings and flat out useless contempt, leaving you feeling stupid and your question unanswered.
Please also factor in the amount of static files on the drive. This has been, historically, forgotten. You do not have the whole drive to do 'swap-the-crappy-block' on.
Everyone's going SSD-crazy, but I'm not yet convinced. They're not _that_ much faster than spinning platters of death, at least not yet, and I'd much rather throw a ton of Ram at the disk cache for the same amount of money.
If you're really worried about performance, invest in a true Ramdisk - the kind that has DDR memory slots on one side and a SATA connector on the other. You can write a 2-line script to mount and format it on boot, and even backup its contents upon shutdown (if needed). That's the ultimate /tmp drive, and it will not wear out no matter how hard you pound it.
-Billco, Fnarg.com
The whole "millions" thing may be true for SLC parts. MLC parts (which are much cheaper) have much lower write counts. The best MLC flash I'm aware of is only rated for a million write cycles. Thousands or tens of thousands is more typical for MLC flash parts. Write amplification makes this even more fun, since it means that a write of one disk block can require rewriting many, many blocks that otherwise would not have been written. If the wear leveling algorithm is optimal, then it's a moot point. If the wear leveling is nowhere near optimal, you can create artificial workloads that will burn out a few cells on the flash part in hours, which is a bit problematic. There is no clear-cut answer for this sort of question, unfortunately, at least not with the current crop of MLC tech.
Consider a log-structured filesystem, perhaps....
Check out my sci-fi/humor trilogy at PatriotsBooks.
Serious Long-Term Fragmentation Problems...
Potential buyers BEWARE, and do some research first. Google the term "intel ssd fragmentation" before purchasing this drive to understand this potential long-term issue. Chances are it won't impact most people, but if you plan on using this drive to house lots of smaller files, think again.
Also
Absolutely avoid using defragmentation tools on this drive! They will only decrease the life of the drive.
If you build is really IOPS-bound, then an SSD will utterly smoke a single disk. Even a whopping great expensive disk can only muster 300 iops under the best possible circumstances, and typically 100 iops under real conditions. A cheapo SSD can deliver 1000 mixed iops, and a good SSD can deliver 100,000 mixed iops. Ever since switching to an SSD on my dev machine I no longer have to suffer through things like a :wq in vim taking 5-10 seconds, or loading a file taking several seconds, etc. One of the benchmarks I did was starting Firefox during a build. On the machine with a disk, it actually took more than ONE MINUTE to start Firefox under a build load. With the SSD, there is no measurable difference between the Firefox startup on an idle machine and a machine with a build happening. The difference is massive.
I primarily use SSD's for backups.
Yeah, I primarily use my Learjet as a backup in case my civic breaks down too.
I had to fucking type my boot sequence in octal to get the system to jump to the correct sector on an 8" floppy. Kids these days. Honestly...
I'm trying to teach myself to set people on fire with my mind... Is it hot in here?
No, he's confusing software development with basic math skills.
Xfce: Lighter than some, heavier than others. Just right.
Visual Basic is hard. Let's use Powerpoint.
Math class is tough. Want to go shopping? Okay, meet me at the mall!.
And what does that have to do with Solar System Dynamics?
One thing about flash in general is that in order to rewrite a small amount of data, you need to (at the low level) erase and rewrite a relatively large amount of data.
The technical term for small write requests actually causing large writes is "Write Amplification". This is one reason the Intel SSD drives are so fast. They have a Write Amplification (WA) factor of 1.1 (done by combining small writes) while many other drives have a WA as high as 20. They also use an "intelligent" wear-leveling algorithm that can reduce spurious writing by nearly a factor of 3.
Warning: I'm an Intel employee
But I've been using the 80GB Intel MLC drive since mid-year 2008 and it's great. Very fast and silent -- I refuse to go back to a mechanical drive again. It's perfect for a client workload (99.9% of users) but not perfect for a transaction heavy server (use the SLC drive).
My workload is writing code and generating/parsing very large data sets from fab (1 - 4 GB).
Here is the "insider" information from my drive:
6.3TB written total (roughly 9 months of usage)
58 cycles (average) on each block of Nand
Given that the component Nand is qualified out to 10K, that's clearly long enough for at least 5 years of usage
PPT is hard. Let's yell.
If your main problem is speeding up your development environment's use of temporary disk storage (because Linux is already caching a lot), use /tmpfs, which stores the files in virtual memory, and if the system needs to page them out, it does that - it's really useful for files that are going to get created for short periods but don't need to get kept for long.
Windows Vista Readyboost is doing something fancy and semi-automatic with caching in USB flash disks - get yourself a USB2 memory stick and turn it on. The stuff is so cheap these days that you might as well buy a large fast ReadyBoost stick, but you'll probably get a lot of payoff even from adding small drives - 8GB is now $20-40, and 32GB is ~$60-120 depending on how extreme you want to get.
Bill Stewart
New Fast-Compression-only CPR http://preview.tinyurl.com/dy575ks
My own experience with a pair of Intel X25-M SLC 32GB drives: after less than a month of moderate use one began reporting unrecoverable read errors at an increasing rate.
We have RMAed the drive and gotten a replacement, but based on the approximately 1500 hours real-world MTBF we had to that point, instead of the claimed 3 million hours MTBF/1 petabyte write lifetime, and unrecoverable bit read error rate on the order of 1/10^15 which lured us into having to repair the resulting database damage.
Perl is hard. Let's use brainf*ck.
That's not Funny, that's Insightful. Brainfuck by itself is indeed very easy - why, just 8 basic operators! The irony is that Java is "simpler" than C++ by the same measure (less language features). In practice, this just shows how pointless the measure is in general.
I've been using a Patriot Warp V2 64GB SSD for a relatively large project (~400k lines of C code).
The "write stutter" is a bit annoying, especially when I do a full "make clean", but it's not too bad.
brainf*ck.
What's with the auto-censordoody? Or are you just a sissy - too shy to use the expletive? And why the heck should I care?
Fixed that for you.
echo -e 'global _start\n _start:\n mov eax, 2\n int 80h\n jmp _start' > a.asm; nasm a.asm -f elf; ld a.o -o a;
d) When their datasheet also says, "Should the host system attempt to exceed 20 GB writes per day by a large margin for an extended period, the drive will enable the endurance management feature to adjust write performance, this feature enables the device to have, at a minimum, a five year useful life"...
You make many good points, but I should point out that the quoted feature never made it into the retail product. When conducting the testing for my article, I wrote several TB per day to my X25-M and experienced no drop in write speeds - provided those writes were more sequential than random.
Constantly hitting an X25-M with small writes will net you at most an average 50% drop in sequential write speeds. The drive will eventually reach an equilibrium based on the mix of write sizes you hit it with. The M has larger flash blocks and has to track a relatively higher level of write combining, and it is possible for it to get 'stuck' at some very low write speeds (see the article for more detail). This is a unique condition that Intel is currently looking into.
Getting back to the quoted section: The write speed slow downs seen in my testing resulted only from the ratio of small/large files written and had nothing to do with the rate / volume of data written over any particular time period.
Article in question:
http://hardware.slashdot.org/article.pl?sid=09/02/13/2337258
http://www.pcper.com/article.php?aid=669
In response to the post, I would recommend either an MLC SSD with very high IOPS (Intel), one of the newer MLC SSD's with on-board SRAM cache (OCZ Vertex / 3rd gen Samsung), or for the highest overall read/write throughput, a pair of SLC SSD's in RAID-0. For SLC, the Intel drive is very good, but there are much cheaper alternatives out there (i.e. G.Skill rebranded Samsung SLC). Note that the X25-E uses write combining, and will take that same 50% worst-case sequential write hit. Other SLC units are not as fast at small writes (no combining), but their performance remains rock steady regardless of what you hit them with.
The G.Skill SLC drive I mentioned:
http://www.newegg.com/Product/Product.aspx?Item=N82E16820231186
Regards,
Allyn Malventano
Storage Editor, PCPer.com
this sig was brought to you by the letter
I did calculate the worst case scenario once, gonna try it again on 128GB flash-drive.
So, 128GB SSD has 128GiB flash but user gets usually 128GB or 120GB so that there are cells that can be used for wear leveling and also for badblocks so they get better yields (SSDs can ship with several broken cells). Lets assume a 128GB SSD, thus it has: 8.79GiB reserved for wear leveling.
First we need to fill up the drive, otherwise it can use the unused cells for wear leveling. So, first we need to write 119GiB.
Now we can began killing the drive, we write 1 byte to random sectors, assuming Intel SSD, each 1 byte write requires the SSD to erase 512KiB block (erase always erases multiple pages, on Intel SSDs, it is 512KiB). There are 18+ million blocks to wear level on.
MLC can handle 10k writes, SLC 100k writes. Thus we get minimum amount to write in 1byte random writes to kill a flash is:
MLC: 171GiB + initial 119GiB
SLD: 1716GiB + initial 119GiB
For 120GB SSD the write amounts are about twice as much.
Comparing to eg. 50% full drive:
MLC: 13TiB + initial 59GiB
SLD: 130TiB + initial 59GiB
This ofcourse assumes a brain-dead write leveling algorithm where as eg. the Intel SSD will wait until it has 512KiB of pages in the cache before commiting them on disk so the drive will last even longer.
And ofcourse the OS will cache writes and when compiling apps you rarely write 1 byte blocks as files usually are much larger than that, just assuming couple KiB files created by the compilation you would have to write 2671TiB to the drive before it fails, and even at continous advertised 130MB/s speeds it would take 249days to kill the drive (at random 2KiB writes).
Puuh, I hope I rememberer it all correctly and didn't make any math errors.
- Raynet --> .