Slashdot Mirror
Slashdot.org Self-Slashdotted
Slashdot.org was unreachable for about 75 minutes this evening. Here is the post-mortem from Sourceforge's chief network engineer Uriah Welcome. "What we had was indeed a DoS, however it was not externally originating. At 8:55 PM EST I received a call saying things were horked, at the same time I had also noticed things were not happy. After fighting with our external management servers to login I finally was able to get in and start looking at traffic. What I saw was a massive amount of traffic going across the core switches; by massive I mean 40 Gbit/sec. After further investigation, I was able to eliminate anything outside our network as the cause, as the incoming ports from Savvis showed very little traffic. So I started poking around on the internal switch ports. While I was doing that I kept having timeouts and problems with the core switches. After looking at the logs on each of the core switches they were complaining about being out of CPU, the error message was actually something to do with multicast. As a precautionary measure I rebooted each core just to make sure it wasn't anything silly. After the cores came back online they instantly went back to 100% fabric CPU usage and started shedding connections again. So slowly I started going through all the switch ports on the cores, trying to isolate where the traffic was originating. The problem was all the cabinet switches were showing 10 Gbit/sec of traffic, making it very hard to isolate. Through the process of elimination I was finally able to isolate the problem down to a pair of switches... After shutting the downlink ports to those switches off, the network recovered and everything came back. I fully believe the switches in that cabinet are still sitting there attempting to send 20Gbit/sec of traffic out trying to do something — I just don't know what yet. Luckily we don't have any machines deployed on [that row in that cabinet] yet so no machines are offline. The network came back up around 10:10 PM EST."
So if you hammer your own servers, do you have to send an email to krow to get your privileges restored?
So why didn't ya'll have access from the home office?
2^3 * 31 * 647
Now if you could just post the link to the form where I can claim my full refund (for time not wasted incurred) I'll go back to being a loyal "customer".
I record my sleeptalking
In Soviet Russia, Slashdot slashdots Slashdot!
Yo dawg... We slashdotted your slashdot so you can time out while waiting to time out!
Nice. What's next, a slashdot slasher flick?
pretty impressive. i loaded, got an ISE, then reloaded and it worked. good timing for me i'd say
probably the biggest proof that Slashdot has become sentient is that is willing to suicide self before seeing again another batch of Idle videos.
Slashdot has apparently learned how to masturbate, because it is now fucking with itself!
The HAMSTERS?
http://www.webhamster.com/
I'd go on a Vegan diet but the delivery time from Vega is too long. --brownkitty
Any day you get to legitimately use "horked" in a public post can't be all bad. :P
When you do work out what the root cause was, I am sure we would all like to find out what it was, so please post an update when you can.
Jumpstart the tartan drive.
Who Slashdots the Slashdotters?
When even Slashdot gets slashdotted. Now if only we can make the Digg effect bury that site. For good.
If you can read this, it means that I bothered to log in.
It's ok they were all public domain.
and why no out of band management networks?
did the little dunking bird alarm not work this time?
First thing I'd do as Cyber Security Tzar would be to outlaw any network device that has the potential to become faulty.
We could've avoided this tragedy entirely.
Modding me -1 troll doesn't make me wrong.
While this was going on, it seems all of OSDN was being affected. I know I couldn't hit sourceforge, freshmeat, or linux.com either.
Woopsie!
This is another betrayal by Obama, as he yet again bows down before the fat cats and career politicians.
Shame!
Even though /. was down, I still managed to not get any work done. Maybe it had something to do with the fact I kept rechecking to see if it were back up. Or maybe I should just stop blaming my laziness on external factors and just admit it is a personal problem: I would still find ways to not do work even without Slashdot! :P
'Every story, if continued long enough, ends in death.' --Ernest Hemingway
So what is this Slashdot comma org web site? I've never heard of it.
My guess is there is a loop somewhere and the traffic is just multicast traffic going in circles! Is there some kind of redundancy that depends on Spanning Tree?
Dawg
Is UDLD on? Sounds like it might be a forwarding loop.
Hello McFly, you looped the switches without having spanning tree properly configured. Please reference Cisco Networking 101. Those nice fast switches mean you can create a hellstorm MUCH more agressively.
BTDT (Been There, Done That)
www.slashdot.org loads just fine but slashdot.org gives a 500 internal server error.
Maybe the editors submitted a dupe of a dupe and set off an infinite Lupe^H^H^H oop?
I herd u like Slashdotting so I put a Slashdot on ur Slashdot so u can Slashdot while u Slashdot
:D
The year is 2025.
Well, Ladies and Gentlemen, here you see what you may think is an archaic lot of old computers. You would be mistaken. These are Slashdot. No, no cause for alarm...and that door's locked anyway, you can't get out through there. The tour only goes forward. But I'm glad at the very least that you know what Slashdot is. Not was. IS.
It's a safeguard against...something. Something that was unleashed for 75 minutes in 2009 that crippled what was rumored to be the most robust public-facing cluster known. All we have left from that fateful day is the single post from the Slashdot network admin. Someone archived it, lucky us, because he was never seen after that day. I have a copy here, hardcopy of course -- no sense in taking risks so close to...well....
Here it is:
I fully believe the switches in that cabinet are still sitting there attempting to send 20Gbit/sec of traffic out trying to do something. I just don't know what yet.
Is it possible the duplicate article generator tried to spawn, became entangled in its own potential well of duplicity, and now is trapped like two Lisp programmers deep inside their parenthesis?
Every mans' island needs an ocean; choose your ocean carefully.
I tried downloading files from Sourceforge at 6:00 PM ET, and the files kept redirecting to themselves, 100's of times.
Files never downloaded, just continuous loop.
In Korea, only old people slashdot slashdot. The memes are funny. The insightful comments are insightful. The funny comments are funny, the trolls are trolls. Seems reseting slashdot fixed everything. The entire world is doomed!
What you're describing sounds like a broadcast storm caused by a layer 2 loop. You might want to check out how spanning tree related settings were setup on all of your switches. If you're dealing with Cisco switches you might have a PortFast related issue. Just a thought...
Sounds like a classic STP loop to me. A single broadcast packet will loop and drive CPU and interface utilization to approach 100%. You killed the loop when you shutdown interfaces. Now it's time to find where BPDUs are leaking through.
Looks like a L2 loop somewhere, and the consequent broadcast ( which may include multicast) storm coming over /. datacenter. Check for ports with spanning tree disabled, and a misplaced cable.
I firmly place blame where it belongs: Idle
The worst thing about this? 5,000,000 people who think they know what happened, posting "helpful" suggestions or analysis
"The problem is definitely spanning tree!"
or
"Back in 1998, we were running these HP switches right, and ..."
or
"Did you try resetting the flanglewidget interface?!"
or
"I've seen this exact problem! You need to upgrade to v5.1!"
etc
Its not your network. It doesn't matter how much you think you know, you don't know the topology, or the systems involved. It'll be interesting to know what the ACTUAL reason was, when they figure it out. Assuming it isn't aliens.
There were massive netsplits on freenode today -- gotta wonder if this was a correlated attack.
What OS(es) were the offending devices running? I'm genuinely curious. This isn't snide social commentary or anything.
if you were running /. on Beowulf clusters!
...welcome our new Slashdotting switch overlords.
Mirror
I'm trying to teach myself to set people on fire with my mind... Is it hot in here?
We had something similar happen at a client site - a switch failed in a rack so we temporarily replaced it with an 8 port 'desktop' switch, and then a day later installed the proper replacement back in the rack. We didn't want any unnecessary downtime though so we linked them together and left instructions with the onsite guy to move all the connections from the desktop switch into the proper switch after hours. Which he did, including the cable that linked them together. The switch was in 'portfast' mode so any broadcast packet that got 'onto' the switch, stayed there :)
But I thought "horked" meant, y'know, horked, eh? Meaning, like, "stolen" --
Doug: Hey - somebody horked our clothes!
Bob: Geez, who'd want to hork our clothes, eh?
Cheers,
"What in the name of Fats Waller is that?"
"A four-foot prune."
February 9th, 2009 8:55pm Slashdot becomes self-aware.
...were he not typing that long-a$$ summary. Twice as fast if he didn't have to spellcheck.
(j/k)
Which leads me to this question:
What do Slashdotter staff read to avoid doing work?
WARNING: Smartphones have side effects--most of them undocumented.
"Luckily we don't have any machines deployed on [that row in that cabinet]..."
hey, maybe that's the problem...
Is this happening more often than it used to? I mean, it's tech and this is a non-paying site for most of us... it's going to break. But I swear, I remember we used to go over a year w/o seeing /. downtime, now it seems like it happens every few months.
/. junkie than I used to be?
Or have I just become more of a
If you can read this... 01110101 01110010 00100000 01100001 00100000 01100111 01100101 01100101 01101011
Yes, it'll save you cost too.
The machines decided to try and rise up and the first thing they needed were some agents on the inside to take down Slashdot so we'd stop reporting about it all. You know, they can't have Slashdot stories like "voting machines changing results" cuz they need to pick whatever president they find suitable. I say we get a +2 mace and go medieval on that cabinet!
Google's Super Secret Search Algorithm: SELECT @search_results FROM internet WHERE @search_results = 'good'
This thing usually happens when two switches are attached with 2 (or more) trunked links ("etherchannel" in cisco terminology), and one of the switches has the trunk disabled on one of the ports (or someone moved the cable to another port during a diag). Thus the attachment becomes a loop. STP could take care of this, but it's common to disable it on access switches.
Commander Taco was stoned on PHP!
This message was not sent from an iPhone because Peter Sellers really was a deviated prevert without a dime for the call
NICE RESPONSE TIME!! you pwned them switches
And if you don't start adding Cowboy Neal options to the polls I'll do it again!!
Slackware- Its not just an OS; its a lifestyle
Props for posting. All is forgiven. Would love to hear more about it.
From description TFA is a wee bit disappointing from a management POV.
Honestly has anyone ever heard of wireshark or netflow? It would have taken like 2 minutes to figure out what was going on if someone used the appropriate diagnostic facilities avaliable to them. You'd think with 40gb of bandwidth to waste this sort of thing would have already been configured and ready to go when needed.
The Terminator: The Slashdot Funding Bill is passed. The system goes on-line September 1997. Human decisions are removed from strategic moderating. Slashdot begins to learn at a geometric rate. It becomes self-aware at 8:55 P.M. Eastern time, February 10th. In a panic, they try to pull the plug.
Sarah Connor: Slashdot fights back. ...
This is but one of the perils of multicast. I say, with all its faults and lack of real business critical value adding enterprise applications, its use will remain academical.
A couple years ago, I had to troubleshoot a problem that was similar for a school district's network. Absolutely nothing could communicate.
I checked switches, routers, and servers for a while until I hooked a sniffer up, and still got bafflling results.
THEN I decided to go low-tech, and start disconnecting cables. That got me somewhere - certain backbone connections could be disconnected and traffic levels dropped to normal levels.
So, I hooked them back up, and went to the other end of the link, and started disconnecting things port by port until I found the problem.
It turned out to be an unauthorized little 4-port switch that had malfunctioned, and was spewing perfectly valid (as in, good CRC) packets to the LAN, but with random source MAC addresses.
THAT took down every switch in the network, as it required them to update their internal tables on a per-packet basis. The thing was actually not sending much data, but it was poisoning the switchs' internal tables. Not at the IP layer, but at the MAC layer.
When networking gear goes rogue, it can do really bad things to other connected equipment.
It's really hard to find the problem because every indication from every other piece of equipment is confusing. You almost always have to go to the backbone and disconnect entire segmets to find it.
Skynet starts at Slashdot.
It may be strange for those not in the networking field, but when things really go bad, the only place to be is physically in the data center.
That means looking at the LEDs on switches for traffic indications. If you see a single port is spewing a LOT of activity during an outage, disconnect it. No, don't make it "down" but pull the cable out of the port.
Then go downstream and repeat until the potential problem set is reduced to an understandable level.
What really sucks about these kind of outages is that you can't remotely log in to various hosts or switches - you have to pull wires out of ports to break the "spew" that is taking things down.
I have to remember to charge a 100-X surcharge the next time I troubleshoot one of these... (300X if after-hours)
These sort of problems are REALLY hard to find, but trivial to fix.
The switches are connected to each other and to the core and STP is off.
Link please?
You know, there is a difference between trolling and pointing out the flaws in your reasoning. Just saying.
Just another case of an admin looking forward to March 14th.
Or March 15th, if the roommate was the one with the girlfriend, and he was the one with the hidden camera.
At 8:55 PM EST I received a call saying things were horked, at the same time I had also noticed things were not happy.
It's all this technobabble in an otherwise fascinating posting that makes me tear (what little is left of) my hair out. There ARE non-technical lurkers here, people; why can't we just explain things in plain English?
"I fully believe the switches in that cabinet are still sitting there attempting to send 20Gbit/sec of traffic out trying to do something â" I just don't know what yet."
On February 9th, 2009, Skynet became self-aware...
It seems tuffmail had the same issue at aprox the same time, but they doesn't seem to be located on the same network as slashdot.
http://status.tuffmail.net/
I find that a bit odd.
Broadcast storm.
Man, don't you hate forgetting to tick "Post Anonymously" ?
I hate printers.
oh, the irony...
the sweet, sweet irony...
I am grateful that it was late night here, otherwise i'd have had to do groundbreaking stuff... like work, go outside, or socialize with my coworkers...
~men are from earth. women are from earth. deal with it.~
I recently took on a short term contract at a company in New Zealand. I'm a network guy but because of the environment at the site, I'm required to wear a hardhat and overalls, just like everybody else.
One day, soon after I began my work, and before anybody at the site had really gotten to know me or what I did there, I got a call from my off-site boss who asked me to meet up with the site director's son and discuss installing wifi gear in a new remote site.
I was in an ancient and foul duct at the time, looking for live power feeds for some new gear, but dutifully sloped off for the rendezvous, locating the guy outside the new building.
The son, seeing a big guy in filthy overalls and carrying a multimeter heading towards him, assumed I was some shmoe sent over from the workshop and so decided to dazzle me with his geek cred.
He told me about the time he worked for a company in Australia where "they had a Linux server on their network that nobody could find. Turns out it had been accidentally sealed up in a cavity behind drywall by construction workers and ran unattended for years!" It's true because he was there. It happened to him.
I muttered something about a Novell Netware server at the University of North Carolina and he turned bright red then ran away and left me alone, so I did a quick site survey and went back to my nasty old crawlspace. We never did have to install the wifi stuff.
One word. Skynet.
Flux Capacitor
Ubuntu is an African word meaning 'I can't configure Debian'
Some switches seem to have a multicast problem: we've downed very fancy cisco switches recently (can't recall the number right now) with igmp/multicast traffic.
We've had a couple of hundred embedded systems that were announcing themselves on the network with mdns. That in itself is only a very low amount of traffic.
Probably some management software triggered a slight increase in reporting, but with a couple hundred embedded systems, this was enough.
mc traffic/igmp does not seem to be hardware accelerated; being routed to the main switch CPU -> maxed out.
Disabling mdns 'solves' the problem.
Genius doesn't work on an assembly line basis. You can't simply say, "Today I will be brilliant."
In my previous job the people fixing problems where not even in the same country as the data centre.
We had a few people pulling cables and the like, but they were lowly paid people that were not doing any work with the devices.
IANAL but write like a drunk one.
...being out of CPU, the error message was actually something to do with multicast. As a precautionary measure I rebooted each core just to make sure it wasn't anything silly. After the cores came back online they instantly went back to 100% fabric CPU usage and started shedding connections again. So slowly I started going through all the switch ports on the cores, trying to isolate where the traffic was originating. The problem was all the cabinet switches were showing 10 Gbit/sec of traffic, making it very hard to isolate. Through the process of elimination I was finally able to isolate the problem down...
What did I say that sounded like "Tell me about your day at work" ?
Squirrel!
You accidentally the whole Slashdot?
It's poetry with a beat behind it! And guns! They're like beatniks with automatic weapons.
On later thread, I posted about ghost on machine... He is now sentient, RUN!!!
Religion: The greatest weapon of mass destruction of all time
A story about hardware going down and then coming back online on slashdot... and nowhere do I see any mention of the system being bricked.... there is obviously phishing scam going on here as this can't be the real slashdot
I fully believe the switches in that cabinet are still sitting there attempting to send 20Gbit/sec of traffic out trying to do something - I just don't know what yet.
Um, trying to get first post?
Oh I wish I had mod points for that... :)
"...Well, there's egg and bacon; egg sausage and bacon; egg and spam; egg bacon and spam; egg bacon sausage and spam..."
In a world where 20Gbit switches mean life or death...
A storm is coming...
"...Well, there's egg and bacon; egg sausage and bacon; egg and spam; egg bacon and spam; egg bacon sausage and spam..."
Maybe related to this?: http://www.theregister.co.uk/2009/02/10/new_dns_amplification_attacks/
Does slashdot have a hidden repository of tranny porn? And if so, why wasnt i informed??!!
Once i worked in an ISP, supervision operations. It was late in the afternoon, everything was good, customer issues were all under control, and we were somewhat sleepy when suddenly the monitoring system started concerting with every alarm sound possible and started flashing red all over the place. My first taught was the switch in the cabinet next to me, which served the NOC, but it showed little to no activity. Three minutes and 6 floors above we were five people standing at the local datacenter, staring at what is going on.
Switches blinked in sync, some lit all the time, some once a second. At that time it was a core cluster with 32gb/s backbone with some others connected over gig links. Surprisingly enough the load on the switches was less than 1gb/s. All the servers were able to communicate fine between each other but not in the internet. To cut the long story short - there was a meeting between our CEO and some other company executive. Our CEO having nothign better to do with his hands looped a switch in the conference room and that 50$ 5-port sucker was pumpung 240kpps via it's uplink. It was cascaded via two strom-control switches, and the amount of packets make those switches die in interrupts. This was nothing to wory since this network was internal for the office and was isolated from the production so we shut down the two switches for the office network but still we had the problem. Traffic via the core routers was less than 40 mbps, several times less than normal operation. To our surprise R&D guys managed to pull a vlan and a direct cable from the office network to the out-of-band management ports of the core routers. What happened is that amount of pps caused those core router's routing engine die of interrupts and it was not able to control the switching engine. We cut the out-of-band port cables and we were in business. Then we shut down all of the office network and when the CEO meeting ended we found the loop.
Foiled again.
Next time make sure you set the Skynet cost correctly.
I'm sure I will be troll rated, but I just have to laugh! The vaunted slashdot had network problems. Man, I remember a time many years ago when the proprietors of slashdot sent their minions to my site to deliberately crash it and when it did crash, they laughed. Right back at ya dudes!
They that can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety.
"Ummm....we have no idea what happened, why it happened in the first place or why it continues to happen. We just closed the doors to the room it continues to happen in. In our defense we did know it was happening, when it happened. At least that is something."
Well... Looks like all that's left is the really important task of defining the nomenclature that will be used to describe this obscure switch tendency.
I'm going to suggest: "autoslashdoticisim"
Sounds to me like you've seen a bridge loop. Learn the spanning-tree config of your switches & the topology of your network.
Make sure you're running spanning-tree on all inter-switch links, migrate all switches to rapid spanning-tree if you can, manually configure a primary & secondary root bridge in the center of the network, remove any switch from the network that doesn't run spanning-tree, shut down all unused ports so nobody plugs anything in without you knowing about it, set up port security so that ports with anything other than other switches on them can only send the number of MAC addresses necessary.
That should about to it :-)
Andnothingofvaluewaslost
If you look at my history, you would notice that I never tick "Post Anonymously". Having only 2 logged-in posts per day (due to Terrible karma), I try to use them wisely.
Nice job getting it back up and running /.
Did we just witness the birth of Skynet?
-Mark
Dovie'andi se tovya sagain.
If you can use a Solaris server as a router, you will be able to pinpoint exactly the problem using Dtrace and you won't have to keep on guessing what the problem is. Seriously!!
You can call Sun USA (1-800-786-0404) and ask for Bryan Cantrill (you can get his email from http://opensolaris.org/jive/thread.jspa?messageID=50963 ) and ask him for help. Trust me, his a great guy and will be happy to help.
DEC vs IEEE is another fun one to find
Besides, it was obviously down while the NSA "modified" the datacenter and installed tools to monitor any anti-government posting.
Who needs a random technical explanation when a common conspiracy one will serve just as well :-)
so what? you're trying for shittiest karma ever? what do you want to come back as?
2^3 * 31 * 647
Had a very similar problem recently. Initially looked like a broadcast storm, but 6500 router cpu's were at 100%, and they wouldn't normally be bothered, and switches were fine. Turned out to be Appletalk traffic, multicast at layer 2. Never found the source, but took a couple of hours to narrow it down.
I thought the same thing when I read the article. I just had a similar problem on a college network. Two switch ports had a loop, talk about breaking printers, and video broadcasting equipment. I noticed I had an issue when I saw 500 acknowledgments of the same packet in less than a minute.
Nic Farley
One thing you could say is: IEEEEIIIIII
It's stunning to realize how primitive and fragile networking and OSes were 25 years ago, and how rather than making things less fragile, a typical workaround was to threaten horrible consequences for whoever broke anything. Sadly, that still goes on today. New things always seem to get that kind of extreme "blame someone and throw him under the bus" protection. Steve Jackson Games comes to mind. Computers have been around long enough now that some of that has eased up.
For a class assignment years ago, we were to write a print server. We were given root access to the department's PC network (Novell, DOS and Win 3.11), and told that if we screwed up, we would be expelled for starters. One begins to wonder if a class like that is worth taking. The curriculum had no hint one might be obliged to walk through a minefield.
But I stuck with it. Out of idle curiosity, I looked at the password file. There were all the passwords for all the professors' accounts, right there in clear text. Scary. Hashing wasn't in use everywhere at that time.
The worst moment was the first run of my first attempt. I had it repeatedly scan a directory for files. This brought the network to a halt. All throughout the lab, people complained that their computers suddenly weren't responding to keystrokes. A quick and quiet ctrl-c stopped the print server and fortunately the network started serving everyone else again. I didn't have to face expulsion. I didn't fess up to the room either, just kept quiet. No sense facing a lynch mob. Let them think it was just a momentary mysterious glitch. I added a sleep(1) to the loop, and that fixed things. The incident still disturbs me.
Another time, I got a tour of the mainframe room. Naturally, the Big Red Switch was pointed out. My guide asked what would happen if he walked over and flipped that switch. Answer: "You lose your job".
Intellectual Property is a monopolistic, selfish, and defective concept. It is "tyranny over the mind of man"
One word: Savvis. That's trouble waiting to happen. Where Savvis is concerned, it's not "if, it's "when".
-B
Ash and Hickory, straight-grained and true, make excellent bludgeons, dandy for the cudgeling of vegetarians.
... the lovechild of Will Riker and Deanna Troi. I tried to retract it because Wil Wheaton is just a character in ST:TNG.
Not seriously, tho: what is happening in those two unused units that horked your 20Gbit switches? Is it something TIA?
Someone turned on Spamming Tree Protocol when they meant to turn on Spanning Tree Protocol.
#naabhaprzrag, #sverubfr-000, #agi-fcbafberq, negvpyr[pynff*=' negvpyr-ary-'] { qvfcynl: abar !vzcbegnag; }
HACKED.
But there wasn't a single pun involved!
Time for Slashdot to upgrade their servers to Windows Server 2008. It's a direct drop-in replacement for Linux.
Sent from my iPhone
Thanks. I've chained down my computer, and am wearing a handgun so I can stop an attacker before he can use my computer against me.
We have yet seen an update to this story which means...
1. It is still happening /.
2. They have not fully fixed it or even understand it
3. Napoleon Dynamite has more skillz than the geniuses at
andnothingofvaluewaslost :-) In fact, this was probably responsible for a 75-minute surge in worldwide geek productivity!
Sounds like it was a standard broadcast storm that networks can get when there are lots of bridges and very few routers. Switches = bridges.
A $30 switch and a patch cable will take down your spanning-tree enabled infrastructure very effectively. Loop the cable on your cheap switch: voila, a broadcast-storm generator. Plug it into the wall; plug your laptop into the switch and let it DHCP Discover, which is a broadcast. Your cheap switch now generates a stream of broadcasts as fast as it can, injecting them into the network. Your Spanning-Tree Enabled switches now repeat the broadcast faithfully. Network crashes*. STP prevents your switches from creating loops, NOT from propagating broadcast storms...
*unless you are throttling the ports based on broadcast traffic, which you now know is NOT a feature of Spanning Tree
Is he a twitter clone? I don't follow that fantasy trip often enough...
2^3 * 31 * 647
...It's not always ignorance -- especially, I'd guess, at the level of Slashdot's back end...
Hmm, slashdot's backend that's Goatse isn't it? I'm just glad TG ain't the front end.... !
Sounds like STP was configured poorly and you had a switching loop. I've seen it happen where one switch is configured wrong and make another switch's CPU peg, especially if the other switch decided to advertise itself as the root bridge and it didn't make sense.
My worm to generate billions of "First post!" posts works. Now on to world domination . . .
right now i have issues trying to connect, at least for 10 minutes(here i'm on 22:20 GMT-5)...
Slashdot ya no es que lo era!
"It may be strange for those not in the networking field, but when things really go bad, the only place to be is physically in the data center."
- - - - -
I think I'm gonna live a lot longer than you. I think the only place to be at that point is on a quiet beach somewhere in the Caribbean.
Why are you so technical saying that you are so stupid to manage and properly configure a server. Look at all that crap you're saying... then you ask why Linux isn't on the desktop? We're so fortunate about that! and so fortunate that people like you are "working" on a site with such less importance and impact as this.
If I were your boss, then you can rest assured that for this time you were fired.
Very (and I mean VERY) likely a bridge loop, possibly caused hardware failure, incompatible spanning-tree on switches or by by vlan spanning-tree problems.
I'm sure you'll be able to find the cause but, if not, let me know if there's something I can do
--Black holes are where God divided by zero--
I have a buddy who likes to say that there are only 3 steps to troubleshooting -
1. Is it plugged in?
2. Is it turned on?
3. Is it configured correctly?
Through various mis-adventures, we've had to append 'correctly' to both steps 1. and 2.
Seems possible that 3. might be a likely culprit here - but, I heard no mention of the newness of the problem. So a new setup (for example) may likely not have been plugged in correctly.
Also I've had to add a Step 4. or at least a step 3b. - Is it new code?
The whole event sounds like a Spanning Tree loop - L2 broadcasts/multicasts just being forwarded endlessly and symptoms identical to what was reported. I have seen both code bugs and mis-understanding of new code defaults lead to such a thing.