Intel Gigabit NIC Packet of Death
An anonymous reader sends this quote from a blog post about a very odd technical issue and some clever debugging:
"Packets of death. I started calling them that because that’s exactly what they are. ... This customer location, for some reason or another, could predictably bring down the ethernet controller with voice traffic on their network. Let me elaborate on that for a second. When I say “bring down” an ethernet controller I mean BRING DOWN an ethernet controller. The system and ethernet interfaces would appear fine and then after a random amount of traffic the interface would report a hardware error (lost communication with PHY) and lose link. Literally the link lights on the switch and interface would go out. It was dead. Nothing but a power cycle would bring it back. ... While debugging with this very patient reseller I started stopping the packet captures as soon as the interface dropped. Eventually I caught on to a pattern: the last packet out of the interface was always a 100 Trying provisional response, and it was always a specific length. Not only that, I ended up tracing this (Asterisk) response to a specific phone manufacturer’s INVITE. ... With a modified HTTP server configured to generate the data at byte value (based on headers, host, etc) you could easily configure an HTTP 200 response to contain the packet of death — and kill client machines behind firewalls!"
I think an actual summary would have been a vast improvement over TFS.
Whether it's your brand of switch, motherboard or even memory, never have the same across all machines if you can help it. The only time I'd recommend the same brand would be hard drives (due to concurrency issues), but then at least try go get them from different batches. If your lot of mobos will only handle one brand of memory for whatever reason even when cas latency is identical, then have two machines doing whatever it is you need to be doing.
One kind of anything makes it easier to kill you swiftly in the end, whether it's by a ping of death or a biological disease.
If computers were people, I'd be a misanthrope.
Crazy sping flashbacks :)
With a certain manufacturer's VOIP phone system a single properly crafted packet will force all phones to reset and reboot. There are little "issues" like that all throughout networking and computers. Find and patch is the order of the day
``Life is too short to be spent debugging Intel parts.''
-- Van Jacobson
That's the dumbest sh*t I've ever heard .. idiot
Anonymous submitter... strike one
Summary linked to random blog... strike two
Sensationalist language in summary... strike three
I think I'll take a pass on this story.
I would be very surprised if this actually contains useful news and isn't someone trying to be an attention whore....
http://www.cloud65.com/ just as Marcus answered I didnt know that a mother can profit $8765 in four weeks on the computer. did you read this webpage
I think the NIC packet of death might be just what you need.
Agreed. OP clearly has no experience managing large server installations.
Listen here my friend, has anyone really been far even as decided to use even go want to do look more like?
the drivers are certified to work
LOL this is a firmware bug, you can lock up the hardware even with no OS booted. Hilarious.
and you get real support
Yeah I love being told to reinstall windows on my linux boxes. Those guys sure are helpful !
"Science flies us to the moon. Religion flies us into buildings." - Victor Stenger
I think the literal flying dagger of death might be just what he needs. And Marcus too. But before that, we'll teach him how to use capitalization and puncutation. Because it would be morally wrong to kill him before he understood these things.
One kind of thing makes it a zillion times easier to recognize a problem when it crops up, and makes it so you only ever have to troubleshoot an issue once.
How much more awful would it be if something similar happened next week on more computers, and he had to troubleshoot it all over again-- not even knowing whether the machines had NICs in common?
"Everything blew up" is a problem. "Everything blew up, I dont know why, and it will take 3 weeks to find a solution" is a huge problem. "Everything blew up AGAIN, and I it will take another 3 weeks because our environment is heterogenous" means you are out of a job.
Always update all the firmware on a box before it hits production. Been there with packets of death. Always upgrade firmware. Always. Always. Always.
This would drive up costs of support, training and troubleshooting.
Wasn't there an old program (Nuke 'em on the Mac I think), that would send out-of-band data (whatever that was), and it would crash the TCP/IP stack on Windows NT 3.51? There was another program on Linux called Pam Slam or something like that, that would also bring down NT servers... Very popular in the early days of the web to bring down your competitor's website.
If telephones are outlawed, then only outlaws will have telephones.
or just buy premade servers from dell or HP. they aren't that much more expensive, the drivers are certified to work and you get real support
...and you're guaranteed that every shipment will have radically different hardware, despite having identical model numbers.
Peter predicted that you would "deliberately forget" creation 2000 years ago...
if ($uid -ge 1000000) || ($uid == "Anonymous Coward"; then /dev/null
cat $foo >
else
cat $foo > $file
fi
---Up Up Down Down Left Right Left Right B A START
I'm guessing you didn't buy them with Linux on them... or prove it was a hardware issue. They have no reason to support something they didn't ship. Sure the support varies but their pro server support is actually decent if you get the right person on the other end. I had a case where teaming 2 nics caused windows to eat crap and die inexplicably and getting it back up was quite the ordeal. I couldn't even keep it stable long enough to unteam or remove the drivers (even in safe mode). Fortunately they did have documentation on the problem - a broadcom driver had a problem with a particular firmware set when teaming was used. I managed to flash the firmware update from a usb flash drive which got me to the point I could at least boot into safe mode and delete the drivers and then get a working older version of the driver from Dell's site up and running and teaming reconfigured. This was on an poweredge r610 btw. I feel bad for the poor sap who ran into this first and having dell support saved me unnecessary downtime, especially since there is no mention of this problem anywhere on broadcom's website. That said for 99% of the issues I've ever run into having on-site spares and a good internal KB has been far more effective than paying for Dell's support, but if it is free with the server why not use it...
Get a web developer
The Dell / HP servers use the same chipset which is affected with the same drivers, dumbass.
This hurt my brain.
There's a good reason a lot of our equipment is slightly older. No, we don't use ancient stuff, but they're not 100% top of the line made yesterday either. And that's because each time a new mobo, memory and storage combo that looks like its worth purchasing comes to market, the first thing we do is run a few sample sets under everything we can throw at it. Usually problems are narrowed down within the first couple of weeks or so, but that's why we have separate people just for testing equipment.
Now admittedly, it's getting harder with this economy so we have some people doing double duty on occasion (I've had to do a bit too when the flu came rolling in), but testing goes on for as long as we think is necessary before the combo goes live. We avoid a lot of the headaches that come with large deployments by keeping changes isolated to maybe 10-15 nodes at a time. It's a slow and steady rollout of mostly similar systems (maybe 3-4 identical) that helps us avoid down time.
We're not Google and we don't pretend to be, but common sense goes a long way to avoiding hiccups like "everything blew up". I think the biggest issue was when hurricane Sandy hit and we weren't sure if the backup generators would come online (this is a big problem with things that need fuel and oil, but stay off for a long time), so we brought in a generator truck for that too, just in case. Again, avoiding one of anything.
If computers were people, I'd be a misanthrope.
I would be curious to know if other versions like the Intel 82576 have the same vulnerability. Maybe we should crowd source this and people can post what they've tested with and received the same behavior.
Are We the Imperial We or the Editorial We?
Curious.
To ensure perfect aim, shoot first and call whatever you hit the target
I for one definitely appreciate the diligence of Kristian Kielhofner. Many years ago I was supporting a medium-sized hospital whose flat network kept having intermittent issues (and we all know intermittent issues are the worst to hunt down and resolve). Fortunately I was on-site that day and at the top of my game and after doing some ethereal sleuthing (what wireshark was called at the time), I happened to discover a NIC that was spitting out bad LLC frames. Doing some port tracking in the switches we were able to isolate which port it was on which happened to be at their campus across the street. Of all possible systems, the offending NIC was in their PACS. After pulling the PACS off the network for a while the problem went away and we had to get the vendor to replace the hardware.
I had a similar issue on my home network. My primary desktop would occasionally have blue screens for no apparent reason, and at odd times of day. Finally figured it out that it only occurred when my HTPC was on. I remember thinking it was impossible, but as soon as I swapped out the HTPC's NIC, it never occurred again. I don't know if it was a driver issue or hardware, but it seemed like that network card was sending some sort of bizarrely malformed packet that caused my other machine to crash.
I don't tell people about it much, because they look at me like I'm nuts.
Actually, I think he's using the Corporate We.
Editorial, I assure you. :)
If computers were people, I'd be a misanthrope.
That's not the same bug. I'd explain, but that's what you get for saying "I wish this guy had done his homework."
Syntax error.
You have no idea how much time I wasted trying to fix intermittent issues with the same chip glued on to my motherboard.
Update your bios and turn off all goddamn ASPM shit in the bios. Kernel options don't do shit.
I'm glad Mr. Kielhofner contacted Intel about this issue and had Intel confirm the bug.
Some years ago I had been diagnosing similar server NIC issues, and after many hours digging, Intel was able to determine the fault was due to the four-port server NIC being counterfeit. Damn good looking counterfeit part! I couldn't tell the difference between a real Intel NIC and counterfeit in front of me. Only with Intel's document specifying the very minor outward differences between a real and known counterfeit could I tell them apart.
Intel NIC debugging step #1 = verify it's a real Intel NIC!
or just buy premade servers from dell or HP. they aren't that much more expensive, the drivers are certified to work and you get real support
...and you're guaranteed that every shipment will have radically different hardware, despite having identical model numbers.
Sad but true. It makes it a PITA when dealing with disk images from one server to another.
Errrr, no. Have you ever tried to deal with replacements and/or issues within a large organization where everything is different? It's hellish.
Try tracking an issue across an enterprise of architecture when all the architecture is DIFFERENT. You also don't want to mix RAM, and drivers can be a real b**** for different motherboards. Oh, and RMA's things, not fun.
Different brands of RAM. Yeah, you try a rack full of servers playing mix'n'match and see how well that works.
Lastly... how many vendors/brands of enterprise gear do you think are out there, and for the ones that do exist how well do you think they talk together. Maybe you're happy mixing HP Procurves with your Cisco stuff but I don't recommend it, and for some stuff there aren't a lot of vendors to choose from anyhow.
I am curious and not the same coward care, to explain? ;)
P.s. I had no homework
They have no reason to support something they didn't ship.
They shipped you hardware. Therefore they need to support THE HARDWARE.
"There is more worth loving than we have strength to love." - Brian Jay Stanley
line 2: -ge: command not found
with $uid set to 1000001:
line 2: 1000001: command not found
The condition is always false and the user never goes the /dev/null
I guess you need to use brackets, in bash at least...
Everything I write is lies, read between the lines.
Not to speak for OP, but there is a hint of logic in there. It wouldn't apply at farms where hegemony translates into resiliency, but it would apply in situations where resiliency results in the ability to withstand faults without replacing anything. Military and other tier one instances come to mind.
"Over specialize and you breed in weakness"
- Major Kusanagi Motoko
Intel NICs have (or at least had...) a very good reputation for performance and stability. Maybe this is a sign that their QA is starting to slip?
%$#@! auto-correct, I meant: homogeneous
You will have to update the kernel, though. The linux e1000 and e1000e drivers have a fuckload of hardware bug workarounds, and the ASPM thing did hit some people recently. You *must* have ASPM L0s and L1 disabled on the Intel NIC *and* its parent PCIe bridge, and the kernel driver usually will only be able to disable it on the NIC itself, if the BIOS is crap and leaves ASPM L0s or L1 enabled on the bridge or has a crap NIC eeprom image that causes issue with 128b/256b maximum PCIe packet (this one can be fixed by Linux, *if* you give it a specific parameter, no idea why it isn't automatic since it is major utter braindamage by the BIOS that is known to hang the box hard sometimes), the NIC can hang.
So by "bring down" you didn't just mean bring down, implying it was brought down, but you meant "BRING DOWN" (notice the caps), implying it was brought down (notice the italics). Such a critical distinction. If it was merely "brought down" this would hardly have been an issue. You could have simply ignored the dead router. As it stands, being brought down, this is a real problem, and you cannot ignore the dead router. Good job!
Whether it's your brand of switch, motherboard or even memory, never have the same across all machines if you can help it. The only time I'd recommend the same brand would be hard drives (due to concurrency issues), but then at least try go get them from different batches.
...and then along comes something like the Seagate 7200.11 firmware bug from a few years back, which caused all drives of several related models to self-brick after a period of time.
The ubuntu bug had to do with bad drivers and / or firmware; when the affected distro was installed on a computer with the affected NIC (which was the completely different e1000), it would render that NIC unusable, even afterreboots.
This bug appears to be triggered by receiving a crafted packet, remotely, and is fixable with a reboot. It also affects a different nic.
I see that beautiful 4chan meme lives.
Your post made my day.
"This post is an artistic work of fiction and falsehood. Only a fool would take anything posted here as fact."
Too bad Intel gave a fix to them (a fix they ultimately couldn't use), but hasn't to anyone else.
Too bad Intel has also apparently known about the problem for months now.
"Intel has been aware of this issue for several months. They also have a fix. However, they haven't publicized it because they don't know how widespread it is."
Bullshit. I bet they were hoping to very quietly roll it into a driver update and have it all go away.
Please help metamoderate.
I ran into a tg3 bug where as the tg3 firmware took the byte value that it expected for a destination port number and redirected the udp packets with that value at that location to the BMC/SMDC/ipmi card (as designed). The issue was that the firmware did not appear to understand that a UDP datagram could be up to 64k so up to 40 1500byte packets and was always looks for the destination port on all packets (not just the first as it should have been) so if the data in the packet matched the expectations those packets never got to the OS.
This caused a client to have move their network port on the machine to the 2nd port (on 200 machines) that did not have the firmware bug in it, and this caused us to find another odd firmware bug...the bug being that if one uses jumbo frames and were to explicitly route to a certain set of nodes with smaller packets (to correct someone else's network bug where they sometimes report the wrong MTU size) then the firmware feature that puts packets together nicely helps you and puts the 6 1500's (the route explicitly broke up) together and attempts to send them on as the firmware does not have that complicated set of rules as the OS does around MTU size.
The broadcom guy I talked to (and he was definitely off-shored) was a ID10T, and claimed there was nothing wrong...even though we could generate a valid linux UDP NFS packet every time that would never get to the OS and completely stop NFS from working. The client found it because one of there data streams was running into this feature pretty consistently if the file offsets line up such that the data had the certain magic value that the fw expected at the right location.
And at the end of the day the real issue is the firmware is poorly documented and appears to be poorly tested and reviewed and is terribly important for stability.
You do realize both HP and Dell commonly come with the very Intel NICs being discussed here, don't you?
So for not much more expensive you get the same failures, the drivers are certified but still fail, and the real support clearly never detected or patched this problem. Yeay?
Intel, or possibly nation where the manufacturering happens, is that code was added into the chip to respond to a highly unlikely sequence. Then when you need to kill a large number of computers simply hit various web servers sending in the required packet. Now, if a nation is protected by a firewall, well, then this approach will not be that useful. However, if other nations do not have a centralized firewall/router, then it can be used to take down a nation.
I prefer the "u" in honour as it seems to be missing these days.
Or the Nintendo Wii?
rewriting history since 2109
I wish this guy had done his homework. This was fixed a long time ago:
http://blogs.computerworld.com/when_linux_does_well_the_e1000e_ethernet_bug_fixed
I am amazed that you got a patched e1000 driver working with a 82574L based piece of hardware... mighty impressive hacking! You should have written up a report on how you managed it for the rest of us to study as homework.
So you're saying we should develop technology similar to Sixth Day? Otherwise, if something happens to you, we need another eksith.
It's just an Intel support strategy. Release NICs with random and minor outward differences. When you have a support issue, say that it is counterfeit. Really cuts support costs!
There is a big difference between corporate support and customer support.
Apparently you never worked in corporate environment. If the problem can't be resolved by phone, HP, Lenovo, hell even Dell would send a technicien on site.
Early on, I had the task of implementing an RFC from scratch . To my surprise the RFC was not any kind of EBNF- it was just some English "talking " which left some room for interpretation or at least didn't exclude a lot of possibilities. The result was my implementation worked perfectly, posting well formed requests to the server and receiving and processing the expected payload back except for the fact that any Netscape server I aimed it at was immediately taken down and, like the article said, when I say *down* I mean needs to be rebooted. Since at that time Netscape had 95% of the server market or more, I essentially had an internet-wide, universal, death ray which,if I were nefariously inclined, I might have been able to leverage into an early retirement.
What you want is some homogeneity in sections, but heterogeneity between sections, so you're not brought completely to your knees when a bug like this is exploited, but you still have copies of hardware for part-swapping tests or frankensteining old servers.
tl;dr: monocultures suck.
Silence is a state of mime.
Sounds, like you've found the balance point between bleeding edge (things are broken/buggy) and outdated (no longer supported/available).
I wished more people would favor this approach. It would save money and time down the road. i.e. Planned Upgrade Path.
Also, cheaper with older ones. I also don't buy the latest stuff. I want the stable anc cheap ones. Also, older stuff have issues worked out and known. I stopped being in first in line unless I get paid to use and test. :P
Ant(Dude) @ Quality Foraged Links (AQFL.net) & The Ant Farm (antfarm.ma.cx / antfarm.home.dhs.org).
A number of years ago I discovered that you can take down many routers, and Windows / Linux hosts by sending an ARP response that says "IP 0.0.0.0 is at MAC FF:FF:FF:FF:FF:FF". When you direct this packet to the access point in a wireless network, this makes the SSID broadcast disappear and the whole device go down. Never posted this until now, I wonder if this still works on modern devices.
They have no reason to support something not in an SLA. What they support is only loosely related to what they shipped you.
and you trusted who, to determine that it was a counterfeit? - Intel you say?
Right. To paraphrase Steven Wright: "Everything in my apartment was replaced with an exact duplicate"
It's a really good troll post, because many of the three words subphrases are actually grammatically valid, so your brain wants to believe that its just a somewhat broken sentence, hopefully just reading more and harder will make the earlier parts fit together correctly. But then the (strategically placed, I might add) really bad errors just really really hurt. To make things better, the end of the sentence is very close to reasonable, so if you get there, you think it might make sense to read it again. Which just amplifies the pain. Ugh.
Maybe, just maybe, some frames could trigger an internal monitoring or debugging mode on the controller? Sometimes, manufacturers would want to remotely diagnose hardware, and that could be a way to do it. Of course, it could also be something else, much more sinister like, say, some obscure government backdoor. Not saying that this applies to this particular case, but since most silicon designs aren't open source, we can't be sure there's no such thing in there, lurking, waiting to be activated.
cpghost at Cordula's Web.
According to TFA (I know, WTF, I actually read TFA?), a reboot does not fix this problem, but a power cycle does.
The real "Libtards" are the Libertarians!
the 82574 is already 5 years old!
http://ark.intel.com/products/36920/Intel-82574IT-Gigabit-Ethernet-Controller
You’ve got to be kidding me. I’ve been further even more decided to use even go need to do look more as anyone can. Can you really be far even as decided half as much to use go wish for that? My guess is that when one really been far even as decided once to use even go want, it is then that he has really been far even as decided to use even go want to do look more like. It’s just common sense.
Eventually. After you run their diagnostic, which spends 6-10 hours checking every sector of every hard drive among other things, send them the resulting diagnostic file, wait until they decide that the bad memory you told them about was really the issue after all, THEN the clock start running on the premium "4 hour guaranteed" support.
"Think about how stupid the average person is. Now, realise that half of them are dumber than that." - George Carlin
In his case that is a feature.
line 2: -ge: command not found
with $uid set to 1000001:
line 2: 1000001: command not found
The condition is always false and the user never goes the /dev/null
I guess you need to use brackets, in bash at least...
You're right, but bash doesn't even enter the picture: /usr/bin/[
$ ls -l `which [`
-rwxr-xr-x 1 root root 35264 Nov 20 06:25
The program is called [ and it complains if its last argument is not a ], so you need the square brackets no matter which shell you use.
That would be straight fraud. Maybe worth a Funny mod, but +5 Insightful? What the hell, moderators?
And where do I get this mythical "firmware update" for the NIC CHIP? I'm sure the chip has code in it, but I've never even heard of a utility from Intel to update the in-chip code in a nic. (it's called "microcode", not firmware)
IIRC,. this is a known issue for certain chipsets, disabling power management for the PCI-E port the interface is a attached to in the BIOS is the known work-around.
say that it is counterfeit. Really cuts support costs!
Also cuts sales volume. If someone tells me there are lots of counterfeits that look almost like original, I'd stop buying.
No. In my old job we had the four hour gold support on our servers (all 5 of them, we weren't a huge customer).
It typically took less than five minutes on the phone with a knowledgeable techie before they escalated and sent us an engineer. Usually the parts and engineer would arrive within two hours.
Dell consumer support might be shite, but their business support is bloody good.
If you're happy and you know it
Syntax error.
Haha, i was the same coward. I tricked you into doing my homework.
Same here with HP. Technician driving 250 km through the Black Forest and heavy traffic, there in 3 hours.
And we were an NGO with one server only.
The dangers of excessive individualism are nothing compared to the oppressiveness of excessive collectivism
Was just end of day, I'm totally checked out pseudo code- relax gents. But I should have known better if I was to be snarky in bash:
if [ $uid -ge 1000000 ] || [ $uid == "Anonymous Coward" ]; then /dev/null
cat $foo >
else
$foo > $file
fi
---Up Up Down Down Left Right Left Right B A START
the second condition should probably be another variable too like $uid_cn or such.
---Up Up Down Down Left Right Left Right B A START
You probably had a different issue than we did (random almost-daily blue screens) on one of 72 identically configured and imaged R510 servers we deployed as NVRs in half a dozen data centers. The tech that came took out all the RAM and reseated it (which I had already done) and told me the problem was gone. He'd put the DIMMs back in the wrong slots though, entailing a new set of diagnostics, another visit to put the same damn bad DIMM back in its original slot, three more days of blue screens, and finally another visit to replace it.
My current headache is a R210 that apparently had the wrong image put on it and blue screens out of the box. Took them 4 1/2 weeks to get us the replacement, and it has the same incorrect image. The techs on site shipped it to us, I put the correct image on it and it's happy. It's now back on its way to the customer site to get installed.
Still better than my previous experiences with Compaq business support, though.
"Think about how stupid the average person is. Now, realise that half of them are dumber than that." - George Carlin
I started stopping the packet captures as soon as the interface dropped
Yes, that's usually when my packet captures stop, too.
Since no OS is grabbing the data from the buffer how do you plan on shifting the bytes through it in order to trigger the flaw? While it is not entirely outside the realm of possibility, it is unlikely this bug would be exposed without a driver on the host interacting with chipset and putting the firmware through its paces.
Guns don't kill people; Physics kills people! - John Lithgow as Dick Solomon on Third Rock From The Sun
I think you will find that Dell NICs tend to be Broadcom.
Rules for adblock plus:
ibtimes.co.uk##.ibt_con_artaux.f_rht
ibtimes.co.uk###bg_header
ibtimes.co.uk##.fb-like.fb_edge_widget_with_comment.fb_iframe_widget
ibtimes.co.uk##.twitter-follow-button.twitter-follow-button
ibtimes.co.uk##IMG[style="border:0;width:20px;height:20px; margin-top:-10px;"]
ibtimes.co.uk###scrollbox
ibtimes.co.uk###taboola-grid-3x2
ibtimes.co.uk##.f_lft.morebox
ibtimes.co.uk###wrap_bottom
ibtimes.co.uk##.bk_basic.bk_disqus
When the NIC powers up the first thing it does is load a bunch of default settings from a chunk of nonvolatile memory (also sometimes called the EEPROM).
You can reprogram the EEPROM using tools from the vendor, or if the driver supports it you can do it under Linux using ethtool.
I've worked a fair bit with the latest 1-Gig and 10-Gig parts (i350 and 82599). They seem pretty decent and stable, good enough for telecom use, though like all chips they do have a list of errata.
The developers are fairly active about updating the linux drivers in the core kernel as well as on sourceforge. The new chips (the 10-gig one especially) are very flexible but this means the drivers are getting a lot more complex than they used to be. (The programming manual for the 82599 is 900 pages.)
We posted the offending packet on CloudShark, with links to all of Kristian's articles. Check it out here: http://appliance.cloudshark.org/news/cloudshark-in-the-wild/intel-packet-of-death-capture/
Bought a lot of HP and Dell hardware, never seen this happen. What server models and what hardware ?
Don't bash the guy, be thankful that some people still care enough to share findings.
...and then along comes something like the Seagate 7200.11 firmware bug from a few years back, which caused all drives of several related models to self-brick after a period of time.
Failure analysis geekery / nitpicking alert!
It was very far from all such drives. In fact, despite the PR shitstorm, relatively few users experienced a bricking. A Seagate employee provided some technical information about it:
http://it.slashdot.org/comments.pl?sid=1098793&cid=26542735
The drive firmware implemented a fixed-size rotating health log with 320 entries. During operation, it occasionally overwrote the oldest entry with a new one. My impression is that log rollover might take several days or more during typical operation. If the drive happened to be shut down while the current log entry was the 320th one (just about to roll over), the buggy firmware would brick on the next powerup. In other words, they had a classic forehead-slapper of a fencepost bug in the startup log scanning code.
That gave a low percentage chance to brick on any given power cycle (less than 1 in 320, influenced by the frequencies of log rollover and power cycling), but it wasn't guaranteed to happen. The only way to make it inevitable would've been to power cycle much faster than the frequency of writing log entries, such that you'd be guaranteed to power down at each possible offset, in sequence. But essentially nobody cycles 3.5" desktop drives that much. (On the flip side, a more common use model is 24/7 operation, in which case these drives would never brick.)
Seagate probably wishes they would've been so lucky as to have that bug hit 100% of the drives on a fixed time schedule. They'd likely have caught it before shipping any drives, and even if not, still would've begun seeing it on internal engineering test drives weeks or months before end users. Instead, it was the kind of low probability failure which they never saw at all because the dice never came up right -- for them. Once millions of units are in customer hands, though, there are millions of dice being rolled...
And while it's tempting to write ==, it should be just a =
I'm won't admit how many years of unix/linux it took for me to notice that, that few shells bother to complain about it.
1 Earth is warming, 2 It's us, 3 it's royally bad, 4 we need to take action NOW
Make sense, I always wondered why you needed spaces after the [ and before the ]. Make sense if they are program arguments. Good one !
But in realty, bracket support has been built into bash for eons mostly for optimization purposes. It is the case for other functionalities where the legacy executable is still present on the system but not needed.
~# which [ /usr/bin/[ /usr/bin/ttt /usr/bin/ttt: No such file or directory /usr/bin/[ /usr/bin/ttt /usr/bin/ttt /usr/bin/[
~# ls
ls: cannot access
~# mv
~# which [
~# if [ 1 = 1 ] ; then echo true; fi
true
~# mv
Everything I write is lies, read between the lines.