Seagate Firmware Update Bricks 500GB Barracudas
Voidsinger writes "The latest firmware updates to correct Seagate woes have created a new debacle. It seems from Seagate forums that there has yet to be a successful update of the 3500320AS models from SD15 to the new SD1A firmware. Add to that the updater updates the firmware of all drives of the same type at once, and you get a meltdown of RAID arrays, and people's backups if they were on the same type of drive. Drives are still flashable though, and Seagate has pulled the update for validation. While it would have been nice of them to validate the firmware beforehand, there is still a little hope that not everyone will lose all of their data."
I'm glad to see them trying though. It's nice of a company to realize they made a mistake, and work to fix it.
It's all fun and games till someone divides by 0. Then it's hilarious.
clay tablets.
Ay Caramba already.
I wonder if this is coming from the Seagate side of the house or the Maxtor side? This sure seems a LOT more like something the old Maxtor would have done than the enterprise provider of choice Seagate.
There are 4 boxes to use in the defense of liberty: soap, ballot, jury, ammo. Use in that order. Starting now.
Arguably, when version "latest and greatest" -1 has a cool bug that causes it to permanently and (without hardware intervention) irrecoverably brick itself for no obvious reason, applying version "latest and greatest", at the manufacturer's recommendation, is a fairly reasonable thing to do.
Anybody who thinks that RAID=backup is going to learn an exciting lesson; but I don't think we can, in fairness, blame people for applying the update.
Normally, they wouldn't, but these drives already had issues. Seagate recommended updating the firmware (with their 'handy' windows only updater). Unfortunately, that made the problem worse.
I would like to know where the hell the firmware update IS? I have opened a ticket with Seagate for each drive. Followed the directions (which were linked to here last week) in detail, and I have heard back NOTHING.
Not even an acknowledgment that they have looked at my tickets. I got a "your ticket was created" email, and that is it.
Seagate is getting very close to losing a lot of customers.
There are 4 boxes to use in the defense of liberty: soap, ballot, jury, ammo. Use in that order- Ed Howdershelt Via Tass
Once upon a great while back, Seagate was one of the première names in hard disk technology. These days, the only press I'm seeing them get is bad firmware, questionable reliability, etc. They've been around longer than Microsoft, they really have no excuse at this point for not even testing their bugfixes on their own hardware. It's not like they even have to test third-party stuff.
What leads to this sort of decline?
It's a small world and it smells funny; I'd buy another if it wasn't for the money; Take back what I paid (SoM)
Over 48 hours now .... Starting to think I better not hold my breath
I agree, but good job for holding your breath for so long :P
The firmware updater uses FreeDOS from a CD image (ISO). Users had to burn it to a CD and boot from it. Here's an example when I tried it (first release that crashed while upgrading -- did not brick for people and me) under VMware to see if my CD booted: http://img403.imageshack.us/img403/7128/screenshotsa7.gif from Sunday night. I didn't bother to try the second one because that one totally bricked 500 GB HDDs which I have!
Ant(Dude) @ Quality Foraged Links (AQFL.net) & The Ant Farm (antfarm.ma.cx / antfarm.home.dhs.org).
I work for a web hosting company and we get these drives by the case. I couldn't guess how many are deployed throughout the datacenter but on some of our backup servers alone I've calculated that I have almost 100 drives that need the firmware update. Thankfully none of the disks on the systems that I admin have shown problems yet, but we try to run a quality operation and that includes preventive maintenance wherever possible.
I was all set to update the firmware on these when one of our guys found that the update rendered unusable 8 of the 8 drives he upgraded the day before Seagate pulled the update. We currently have some massive amount of Western Digital 500GB and 750GB disks on rush order as a result of this debacle. It wouldn't surprise me if management tells us to swap the Seagate disks for the WDs and decides to just sell the whole lot of Seagate disks off in bulk as defective. It would be cheaper than paying people to update each one by hand.
Before this, Seagate used to mean "quality" in my opinion as their failure rate seemed to be lower than the competition and their 5-year warranty was unmatched. For the average home user, this situation is a headache. For people running datacenters filled with these disks, it's an outright fiasco.
Go here http://forums.seagate.com/stx/board?board.id=ata_drives to see the angry users and posts in Seagate's official forum. Most of us are pretty angry and upset. Definitely read this super long thread: http://forums.seagate.com/stx/board/message?board.id=ata_drives&message.id=6272 (42 pages).
I find it ironic that our HDDs are about to be bricked EITHER way (on its own) or with the pulled firmware updater (released twice already too; first one crashed with memory dumps and stuff for everyone; second one bricks 500 GB models).
FYI, http://support.seagate.com/firmware/MooseDT-32MB-SD1A.ISO was the ISO file that was released (404 error now due to brickings) according to my download history. Seagate needs to get the next one right!
Ant(Dude) @ Quality Foraged Links (AQFL.net) & The Ant Farm (antfarm.ma.cx / antfarm.home.dhs.org).
It's not bricked if you can fix it without modifying the hardware. It's a nice term -- stop destroying it.
is this seagategate?
Media that can be recorded and distributed can be recorded and distributed.
-kfg
I talked with A/S 10 minutes ago
After talking with Seagate A/S a few days ago and told I needed to update my firmware and sent an email on how to update, no fireware was downloadable from the links in the email provided.
Annoyed I talked to Seagate A/S again today, it seems I do not need a firmware upgrade anymore, and only some of the hard drives made in Taiwan between some date seem to be defective and updating firmware in non-defective drives seems to be causing problems. Hence they removed all links to firmware. Since they are not 100% sure of what I mentioned above yet, they told me they are going to update their site and call me back when things get finalized next week.
Can we, for God's sake, just permanently ban the use of the word "brick" or "bricked" in the summaries. I have yet to see it used correctly.
Brett
I work for Seagate. I was there when the fit hit the shan, and I saw everything going in internally, as well as externally.
I really love my job, so please excuse the sock-puppet nature that creating a brand new account and claiming to be an authority on the subject I must seem to be. But I am a geek, and I really think you all need to know the true story behind the scenes.
This whole thing started with the 1.5 Terabyte drives. It had a stuttering issue, which at first we all thought was a simple bad implementation of SATA on common chipsets. Seagate engineers promptly jumped in and worked to try to duplicate the issue and prove where the problem was. This wasn't a massive rush as 1.5tb drives are what? 5% of the drives on the market. When it became obvious that the issue was more widespread, they buckled down and put out a couple of firmware revisions to fix it.
Now, in the 1.5tb drives, there are 2 main revisions. the the product line that gets the CC* firmware, and the line that gets the SD* firmware. They came out with firmware CC1H and SD1A to fix these issues and started issuing them.
But, seagate has always been restrictive of handing out their firmware, so such updates required calling in with your serial so that the people who had access to hand out the firmware could check a) model, b) part number, and c) current firmware just to make absolutely sure that they were giving the right firmware out. This has been a procedre that has worked for YEARS up until now.
Then the bricking issue came to their attention. It took so long because it's an issue that's hard to track down - pretty much the journal or log space in the firmware is written to if certain events occur. IF the drive is powered down when there are 320 entries in this journal or log, then when it is powered back up, the drive errors out on init and won't boot properly - to the point that it won't even report it's information to the BIOS.
This is a rare, but still obviously bad issue. Up until now, we all figured it was just some standard type of failure, as it was such a rare event, so we'd RMA the drives.
So, for whatever reason, mid management started freaking out (as it could be a liability for seagate, I suspect - ontop of the already potentially liable issue of the stuttering problem causing drives to fail in RAIDs). So, they pushed the release of the SD1A firmware to the general public. They took a few days to 'test', though it was mostly just including some code in the batch file that kicks off the firmware updater, to check that it is a BRINKS drive, and the proper model number. Then it was kicked out to the public.
Please understand, this firmware had to go through five different checks to make sure it applies to the specific conditions to qualify sending to a customer, before now. 5 chances for us to go your drive needs the other (or none) firmware update. Suddenly, it's down to ONE check, and even that was more designed for a contingency just incase the wrong firmware was sent out.
Of course, it starts bricking drives.
Right now, the engineers are crapping themselves, the firmware's been pulled, the support agents are told to say "The firmware will be released soon" and no real procedure to fix this issue is in place. Our phones are flooded so bad that it locks the system up when there are too many calls in queue, and emails are coming in at hundreds an hour.
We simply cannot keep up.
The good news is, the chance of your drive simply not spinning up one day is very low. And for those of you who flashed the wrong firmware - be patient. It's not bricked, just unable to write data to the platters properly. When they have a *GOOD* firmware out, a new flash should un-brick the drives. If not, flashing it back to SD15 should make it work again.
Seagate really pushes the idea of being open and honest as much as we can without being sued to hell. They let agents make choices and use their skills instead of scripting us to death. They worked hard to bring their support back t
If the problem was the uncertainty of your data's integrity due to a firmware bug, then your problem is solved.
Now you are sure.
A brick's value is the cost of creating a brick to replace it.
So if it is less expensive to throw something out and buy a new one than it is to repair it, it's bricked.
paintball
I'll answer your questions to the best of my ability, and as honestly as I can! I'm no statistician, but the 'drive becoming inaccessable at boot-up' is pretty much a very slim chance - but when you have 10 million drives in the field, it does happen. The conditions have to be just right - you have to reboot just after the drive writes the 320th log file to the firmware space of the drive. this is a log file that's written only occasionally, usually when there are bad sectors, missed writes, etc... might happen every few days on a computer in a nin-RAID home use situation.. and if that log file is written even one time after the magic #320, it rolls over the oldest file kept on the drive and there's no issue. It'll only stop responding IF the drive is powered up with log file #320 being the latest one written... a perfect storm situation. IF this is the case, then seagate is trying to put in place a procedure where you can simply ship them the drive, they hook it up to a serial controller, and re-flashed with the fixed firmware. That's all it takes to restore the drive to operation! As for buying new drives, that's up to you. None of the CC firmware drives were affected - only the SD firmware drives. I'd wait until later in the week, maybe next week, until they have a known working and properly proven firmware update. If you were to have flashed the drives with the 'bad' firmware - it would disable any read/write functions to the drive, but the drive would still be accessible in BIOS and a very good chance that flashing it back to a previous SD formware (or up to the yet to be released proven firmware) would make it all better. Oh, and RAID0 scares me by it's very nature... not an 'if' but 'when' the RAID 0 craps out and all data is lost - but I'm a bit jaded from too much tech support! :)
It's not "moving the head to prevent wear". It's SMART data gathering. smartctl will soon sort you out. However, I would personally not recommend it.
smartctl --smart=on --offlineauto=off
Raid has never been a backup. A backup is something stored outside of the running set. That way you can restore the data if your running system would, you know, break down.
c++;
As far as I know, if your drive has the CC1G, CC1H, CC1J or any of the CC firmwares really, it is completely unaffected by this issue.
However, it may need an update if you experience 'stuttering' (the drive pausing for more then a few seconds during data transfer). The CC1H and CC1J firmwares are *fine* and will absolutely not brick your drive.
I'd still wait a little while though - support is overwhelmed and mistakes are being made as noone is used to these changes. Once everyone gets a routine down (once there -is- a routine at all), they'll be better able to help reliably.
Ok, maybe it's just me, but who the hell updates drive firmware anyway? Just because I'm a techie, doesn't mean I am suddenly willing to do more work than other customers.
Do you think a single consumer out there goes through the trouble of updating their drive firmware? (unless there's an automatic procedure in place, like probably mac and some windows manufacturers have)
To me, any drive which requires an firmware update to function (not just perform better) after purchase, is a failed product and I would surely hesitate to buy another ever again.
I used to buy Seagate drives in pretty large numbers for some of my datacenter activities and every time a drive locked up for some reason, I insisted on a new drive through EMA. Had Seagate refused, they would have taken away a large chunk of their added value, to me. I would probably never buy another drive from them again.
It is the old High Availability versus Disaster Recovery question. Two completely different things aimed at two completely different problems.
The first is to make sure that your system remains available as long as possible even if some of your hardware goes belly-up. The latter is for when your DATA goes belly-up.
I am one of the victims and your report confirmed all the problems which I expected to occur inside your company. I previously worked with an electronic giant and the problems are just too similar.
The catastrophic problems which Seagate is facing now could have been prevented - if there would have been one single person in customer service who would have cared and pushed the issue, which was known for months, up to the right people. A little googling some months ago would have proven that this issue is far bigger than a "one time" incident.
After all it doesn't happen every day that Data Recovery companies announce with joy that they are able to handle widespread 7200.11 firmware problems. Or that the two major companies which provide recovery solutions race for being the first to have a two click solution for this cash cow.
Data recovery companies were flooded with drives. They figured out an easy way to fix the firmware and kept it secret. They made a great profit, charging prices as if it was a hardware failure.
Seagate Datarecovery did the same by quoting up to 1800 USD for a 10 minute fix. Although I am sure that they were the only ones not aware of the easy fix.
The problem with the undetectable bios drives really isn't new. Your customer service knew it for a long time, but they are paid so little and probably have such strict procedures that they don't care about Seagates customers and no one dared to report the drive failures as a major incident. Everyone shut up about it and the people which are responsible and do care only learned about it months later when (or shortly before) it got out to the press.
Seagate had months of time to fix it. Two months ago when my drive broke, there was already plenty of information about the problem on the net. The only one who would deny any problem was Seagate.
I warned your board moderator of the disaster which will strike Seagate months ago. I tried to show him that these were not normal failure rates but the poorly paid guy didn't care.
The email support who takes two weeks to respond, and the phone and live support were just as ignorant.
There were people reporting how 4 out of 6 drives broke within weeks, and Seagate would only respond that such failure rates are normal.
People on the Seagate boards were constantly reporting the problem, but your board moderator shut them up. Threads where getting deleted and locked, including a big thread where the community was working on a fix. The reason, according to Seagate, was that it added nothing to the community.
The board moderator would consistently tell everyone that there is no known problem with the drive - the same message as your customer service.
It went as far as blocking links in private messages to a posting on another board which could help the victims. So how could Seagate expect from those people now to actually believe that the company cares?
The posting on the new board had within a short time 10.000 views. That's when things started to get out of hand for Seagate.
People were pissed off for months about Seagate. Everyone knew that the firmware was broken, but the company denied any problems. We knew that it is not that difficult to recover the data if you have the tools and knowhow, but the company wouldn't give any assistance. Many would have accepted the fate if the drive would truly be broken. But not if it is inaccessible because of a firmware bug which makes every single drive a -clicking- time bomb.
People everywhere were calling Seagate harddrives junk drives which are so unreliable that they will never buy them again.
So I, as many others, went on to warn every single person we knew about the problem with Seagate drives. The hilarious/sad thing is that before, I would recommend Seagate to everyone I knew. If someone would ask me which drive to buy I would reply with no doubt: Seagate.
This could have been prevented if Seagate would have acknowledged the problem much earlier. I wasted day after day,
I never understood why equipment capable of being flash-updated by users does not include the 1.0 drivers as a ROM onboard the device. This way if you completely and utterly bork the flashing, you can reset a jumper, press a recessed button with a paperclip, so SOMETHING that will cause the EPROM to be reflashed from the known good ROM. "Hey, here's baseline firmware again, people. Let's try this again."
The only possible explanation I can think of for not doing this is that the known-good ROM would add another half-cent to the manufacturing process and we know how manufacturers watch their pennies.
Kwisatz Haderach
Sell the spice to CHOAM
This Mahdi took Shaddam's Throne