Any one of the other thousands of CCIE's wandering the globe. How about the Cisco consultants on-site making sense of the mess? And then there's every one of his coworkers as character witnesses.
Enterprise Cicso router is very complicated piece of equipment so it is essential to have a copy of the configuration file somewhere as backup.
Complex HARDWARE, yes. Complex configuration, maybe. The software (IOS) is pretty much identical from the smallest device to the biggest. From the command line, a Cisco 12000 looks (functionally) just like a 2501. Generally speaking, big routers have big configs, but that's not always true. For example, the config on my 1760 is 19472 bytes. The config on the office 2851 is 2861 bytes -- 16491 when it was standing in for the pix. (I'd include realworld ISP configs, but that machine has a dead power supply:-()
No sane person would take it as far as this nut has. Not saving configs, disabling password recovery, etc., etc. These are the actions of a deeply disturbed person. I've been there. I've spent a great deal of time designing and building things (networks, authentication systems, backup systems, web farms,...) only to see them systematically destroyed by others. In one case, I did, indeed, refuse to be a party to it -- "you can fire me; I'm not doing it" -- but I've never heald systems hostage by preventing anyone else from touching them. Other people understand the systems (even if dangerously slim understanding) and have access to them. I'm the one asked to do things because I have the most experience/knowledge of the system(s) -- generally because I built them, but also because I've been the one using them.
Where I work now (in software), there's a lot of specialization where only one person fully understands something -- usually because they created it. However, even that doesn't lead to the same hostage standoff; it just means fixing something will take longer if the particular guru isn't available. (it also helps that we're all good, long time friends who work extremely well together. that's hard to have in a large 1000 person company.)
I looked at those (back when they published prices), and yes, they are expensive. A 531-DE is $2500. We paid ~4k for the 2851 + NM1/T3. (amazing the discounts ISPs/integrators get.)
Multihomed customers are (or should be) handled carefully. They need to bring you their own assigned address space, and preferablly their own AS#. And then it's setup just like the ISP's own address space. (upstream ISPs and peers updated as necessary. blah. blah.)
However, more often than not, I'd get schmucks who wanted to have ISP#2 announce the/28 we assigned them -- that's a "Hell. No. If you do it anyway, we will sue you." (not to mention it won't work... anything smaller than/19 is not guaranteed to be globally routable; anything smaller than/24 is filtered almost everywhere.) Sometimes we were ISP#2 -- I laughed at those customers.
*ding* You win a cookie. That's exactly what we're saying. There are more ISPs that don't filter traffic than those that do. IP filtering is expensive business at ISP traffic rates. My little Cisco 1760 handles the 5-7Mbps that goes through it rather well. Multiply that by thousands, and that's what ISP's deal with. (Note: a full rate DS3 will swamp a 2851, and that's a pretty damned expensive bit of gear. but it's cheaper than a DS3 PCI(-X/e) card.)
Cisco has a "reachable by any" mode, but it's seen as an ugly hack. It's always better to do it on the customer interfaces anyway. You only need to filter "your own" traffic on the core routers -- that's often easier to manage and that class of router is better designed for that sort of traffic filter. (i.e. it only needs to check a rather small access list once per flow.)
The issue here is simply this... their network admin was a fucking nut. It's fairly common among CCIE's, in my experience. I don't know what it is... the cisco testing process, the type of personality draw to this line of work, or a "madd with power" thing that comes with the CCIE certificate.
In his mind, if it wasn't bolted to the floor in his office along side his guard dogs, it ain't secure. In one of the reports, it was said he thought with password recovery disabled, it was safe enough to save the config instead of (presumablly) dialing into the thing to reconfig it after a reset. (as if the dial-in was more secure than the router's nvram. in fact, it's not. a serial line can be tapped trivially without breaking the connection.)
I'm at a loss as to what he thought he was protecting. Other than keeping his "clueless' coworkers out of the network gear, he's done nothing. If someone is in a position to use password recovery -- which has to be done on the console, not the aux port where a modem would normally live -- they can steal the entire router, install physical data taps, and with a bit of time (~5 min, give or take) bypass password recovery anyway. Getting a copy of the config shouldn't disclose any really sensitive information (aka passwords.) If he knows about disabling password recovery, I'll assume he knows about "service password-encryption", so there's no passwords to recover from the config. Thus, getting into one router won't necessarily help you get into any others. (Compromising the CiscoWorks server... that'll get you somewhere.)
Explain this "staged booting". Cisco routers only have two configs: the one in nvram, and the one from a network server. Neither support any form of encryption. You can have outside processes "securely" reconfigure it once it's up and reachable, but that's what the idiot was doing with the "no saved configs"/modem policy. The router (and network) is hosed until something/someone configures the router. That means it/they have to know it needs to be configured. And it will need a heap of AI to know what needs to be done to turn the running config into the secure config -- and something would have to be done to prevent the "secure config" from ever being saved to nvram, etc., etc.
"password recovery" makes it easy to break in. Cisco gear is found in all corners of (not just) the US government. I'm sure that's why the feature (undocumented, btw) even exists. While it doen't stop you getting into the router, it does make it difficult.
With password recovery DISABLED, sending a line break during startup will do nothing but say password recovery is disabled. You can still get to rommon to turn off the config, but it takes a number of tricks. (many systems will drop to rommon, eventually, if there's no bootable image -- no bootflash and/or flash.)
Cost recovery fees are simply ways to get around regulated pricing. They cannot (easily) change the charge per line, but they can easily add "cost recovery fees" to your bill.
And don't fall for the "Universal Service" BS, either. Phone companies collect that money and keep it. It doesn't go anywhere but their own pockets. I'm pretty sure they collect more in fees than they spend providing universal service. (not that anyone can ever know.)
It sounds like he disabled password recovery. That makes password recovery rather difficult -- not impossible, just hard. Each device will have to be disassembled to temporarily disable NVRAM in order to get to the rommon. Or, you'll need a rig to edit the NVRAM directly. (I've not looked recently, but as I recall, NVRAM is socketed on the big iron. It's just like a PC... RTC memory.)
LIDAR takes two samples nanoseconds apart. The target doesn't move very far between samples.:-)
Doppler radar is extremely unpredictable due to "ground clutter" -- it's not a laser so the signal radiates out in a cone and reflects off everything in that cone. That's why modern radar systems use a phased array; ground clutter isn't a problem because it tracks multiple targets. (the clutter (extra reflections) aren't noise, they're additional data... multiplied by the array's additional elements and the result is dead-accurate reporting of everything in range.)
I doubt this guy had a phased array "gun" (because those aren't handheld guns.)
As I understand it, microwave radar guns cannot be used in motion. FCC regulations explicitly prohibit it. Besides that, getting an accurate report in-motion is nearly impossible. (The odometer calibration declines every second the car is in motion -- tire wear, air pressure, etc... every 1mm change in the diameter of the tires adds error.)
Note: If you're in motion, you just get right behind him and match his speed:-) I've seen dozens of cops do that. It even works for MARKED cars.
Afaict most linux distributions are not stupid enough to do this.
Like installing and enabling SSH, Apache, etc? Unless you are doing a live network install that's pulling in all current security updates, just about every major OS (which includes Linux distro's) have several known exploitable flaws in the box. They need to be connected to "the web" to automatically update themselves. Doing so without any form of firewall is dangerous.
XP SP2 improved things greatly, but like most things from M$, there are still hundreds of exploitable flaws... many remotely accessible with zero user interaction. Even Vista's infamous "Allow or Deny" crap has been defeated a dozen times over -- yes, you still have to get your code on the machine, but given the general windows userbase, that's not difficult at all.
In the case of the redhat box, by the time we inspected it, it had been compromised three times... via an unpatched ssh server, unpatched apache, and unpatched bind. All three where hit by automatic breeching tools -- worms that scan and compromise machines all by themselves.
I recall a former boss's computer getting compromised during the installation. It was either NT4 or 2000 server. I'm not sure his disk (most likely an MSDN disk) had any service packs on it. (this was late '03.) It was beyond the firewall, naked on a Bellsouth DSL line.
I also recall a friend (sysadmin) had his linux (redhat 6.2 maybe) machine compromised within a day of installing it. I don't know if it was within 4min or 16hrs; the next day we noticed it was scanning the network. That was a "naked" workstation on an ISP's core network -- no firewall of any kind. That was 7-8 years ago, and we still kid him about it.
The T1 at the office was seeing about 100 probes per minute years ago when I cared enough to log all that shit. The DS3 was seeing just as much crap the instant it was turned on a few months ago. (seeing how the morons setup that router (cisco), I wouldn't be surprised if people have broken into it -- with no logging turned on, how would anyone know?!?)
Answer: Until the dump user clicks on the wrong attachment, etc. Browsing the web from an unpatched IE is asking for trouble. The same is true of unpatched Outlook and Outlook Express.
*ding* we have a winner. of course, sadly, a linux (or solaris, or in fact, almost any *NIX) box can be hijacked just as fast if no patches are (ever) installed. ('tho i don't know about 4min, as linux isn't as highly targeted.)
Actually, that's policy violation in many cases. However, eBay doesn't give a rat's ass as long as they get their fees. I used to report them, but since eBay doesn't do jack shit, why bother; if other people are stupid enough to fall for it...
That they're desperate for customers, don't know how to maintain a website, otherwise have no clue how to do business? By using eBay, they don't have to really run hardly any company at all (which is convenient since from my chair they don't even know how to wipe their own ass)... throw everything on eBay's servers and automate it all -- which is exactly what they're doing... you MUST use paypal; you have 6hours to pay; after 12hours, the "auction" is canceled and you are a "non-paying bidder"... all 100% automatic.
If you aren't paying attention, you might not realize you're doing business with Buy.Scum. I'm sure a lot of people who don't do business with them, are now through eBay.
Slashdot Mirror
Any one of the other thousands of CCIE's wandering the globe. How about the Cisco consultants on-site making sense of the mess? And then there's every one of his coworkers as character witnesses.
If I were in SF, I'd volunteer, pro-bono.
Complex HARDWARE, yes. Complex configuration, maybe. The software (IOS) is pretty much identical from the smallest device to the biggest. From the command line, a Cisco 12000 looks (functionally) just like a 2501. Generally speaking, big routers have big configs, but that's not always true. For example, the config on my 1760 is 19472 bytes. The config on the office 2851 is 2861 bytes -- 16491 when it was standing in for the pix. (I'd include realworld ISP configs, but that machine has a dead power supply :-()
No sane person would take it as far as this nut has. Not saving configs, disabling password recovery, etc., etc. These are the actions of a deeply disturbed person. I've been there. I've spent a great deal of time designing and building things (networks, authentication systems, backup systems, web farms, ...) only to see them systematically destroyed by others. In one case, I did, indeed, refuse to be a party to it -- "you can fire me; I'm not doing it" -- but I've never heald systems hostage by preventing anyone else from touching them. Other people understand the systems (even if dangerously slim understanding) and have access to them. I'm the one asked to do things because I have the most experience/knowledge of the system(s) -- generally because I built them, but also because I've been the one using them.
Where I work now (in software), there's a lot of specialization where only one person fully understands something -- usually because they created it. However, even that doesn't lead to the same hostage standoff; it just means fixing something will take longer if the particular guru isn't available. (it also helps that we're all good, long time friends who work extremely well together. that's hard to have in a large 1000 person company.)
I looked at those (back when they published prices), and yes, they are expensive. A 531-DE is $2500. We paid ~4k for the 2851 + NM1/T3. (amazing the discounts ISPs/integrators get.)
Multihomed customers are (or should be) handled carefully. They need to bring you their own assigned address space, and preferablly their own AS#. And then it's setup just like the ISP's own address space. (upstream ISPs and peers updated as necessary. blah. blah.)
However, more often than not, I'd get schmucks who wanted to have ISP#2 announce the /28 we assigned them -- that's a "Hell. No. If you do it anyway, we will sue you." (not to mention it won't work... anything smaller than /19 is not guaranteed to be globally routable; anything smaller than /24 is filtered almost everywhere.) Sometimes we were ISP#2 -- I laughed at those customers.
*ding* You win a cookie. That's exactly what we're saying. There are more ISPs that don't filter traffic than those that do. IP filtering is expensive business at ISP traffic rates. My little Cisco 1760 handles the 5-7Mbps that goes through it rather well. Multiply that by thousands, and that's what ISP's deal with. (Note: a full rate DS3 will swamp a 2851, and that's a pretty damned expensive bit of gear. but it's cheaper than a DS3 PCI(-X/e) card.)
Cisco has a "reachable by any" mode, but it's seen as an ugly hack. It's always better to do it on the customer interfaces anyway. You only need to filter "your own" traffic on the core routers -- that's often easier to manage and that class of router is better designed for that sort of traffic filter. (i.e. it only needs to check a rather small access list once per flow.)
The issue here is simply this... their network admin was a fucking nut. It's fairly common among CCIE's, in my experience. I don't know what it is... the cisco testing process, the type of personality draw to this line of work, or a "madd with power" thing that comes with the CCIE certificate.
In his mind, if it wasn't bolted to the floor in his office along side his guard dogs, it ain't secure. In one of the reports, it was said he thought with password recovery disabled, it was safe enough to save the config instead of (presumablly) dialing into the thing to reconfig it after a reset. (as if the dial-in was more secure than the router's nvram. in fact, it's not. a serial line can be tapped trivially without breaking the connection.)
I'm at a loss as to what he thought he was protecting. Other than keeping his "clueless' coworkers out of the network gear, he's done nothing. If someone is in a position to use password recovery -- which has to be done on the console, not the aux port where a modem would normally live -- they can steal the entire router, install physical data taps, and with a bit of time (~5 min, give or take) bypass password recovery anyway. Getting a copy of the config shouldn't disclose any really sensitive information (aka passwords.) If he knows about disabling password recovery, I'll assume he knows about "service password-encryption", so there's no passwords to recover from the config. Thus, getting into one router won't necessarily help you get into any others. (Compromising the CiscoWorks server... that'll get you somewhere.)
Explain this "staged booting". Cisco routers only have two configs: the one in nvram, and the one from a network server. Neither support any form of encryption. You can have outside processes "securely" reconfigure it once it's up and reachable, but that's what the idiot was doing with the "no saved configs"/modem policy. The router (and network) is hosed until something/someone configures the router. That means it/they have to know it needs to be configured. And it will need a heap of AI to know what needs to be done to turn the running config into the secure config -- and something would have to be done to prevent the "secure config" from ever being saved to nvram, etc., etc.
"password recovery" makes it easy to break in. Cisco gear is found in all corners of (not just) the US government. I'm sure that's why the feature (undocumented, btw) even exists. While it doen't stop you getting into the router, it does make it difficult.
With password recovery DISABLED , sending a line break during startup will do nothing but say password recovery is disabled. You can still get to rommon to turn off the config, but it takes a number of tricks. (many systems will drop to rommon, eventually, if there's no bootable image -- no bootflash and/or flash.)
Cost recovery fees are simply ways to get around regulated pricing. They cannot (easily) change the charge per line, but they can easily add "cost recovery fees" to your bill.
And don't fall for the "Universal Service" BS, either. Phone companies collect that money and keep it. It doesn't go anywhere but their own pockets. I'm pretty sure they collect more in fees than they spend providing universal service. (not that anyone can ever know.)
It sounds like he disabled password recovery. That makes password recovery rather difficult -- not impossible, just hard. Each device will have to be disassembled to temporarily disable NVRAM in order to get to the rommon. Or, you'll need a rig to edit the NVRAM directly. (I've not looked recently, but as I recall, NVRAM is socketed on the big iron. It's just like a PC... RTC memory.)
They don't support encryption of the netboot config. It'll accept whatever config gets handed to it.
Go ask your father about phased array radar systems. Now try to get out of that ticket.
LIDAR takes two samples nanoseconds apart. The target doesn't move very far between samples. :-)
Doppler radar is extremely unpredictable due to "ground clutter" -- it's not a laser so the signal radiates out in a cone and reflects off everything in that cone. That's why modern radar systems use a phased array; ground clutter isn't a problem because it tracks multiple targets. (the clutter (extra reflections) aren't noise, they're additional data... multiplied by the array's additional elements and the result is dead-accurate reporting of everything in range.)
I doubt this guy had a phased array "gun" (because those aren't handheld guns.)
As I understand it, microwave radar guns cannot be used in motion. FCC regulations explicitly prohibit it. Besides that, getting an accurate report in-motion is nearly impossible. (The odometer calibration declines every second the car is in motion -- tire wear, air pressure, etc... every 1mm change in the diameter of the tires adds error.)
Note: If you're in motion, you just get right behind him and match his speed :-) I've seen dozens of cops do that. It even works for MARKED cars.
Like installing and enabling SSH, Apache, etc? Unless you are doing a live network install that's pulling in all current security updates, just about every major OS (which includes Linux distro's) have several known exploitable flaws in the box. They need to be connected to "the web" to automatically update themselves. Doing so without any form of firewall is dangerous.
XP SP2 improved things greatly, but like most things from M$, there are still hundreds of exploitable flaws... many remotely accessible with zero user interaction. Even Vista's infamous "Allow or Deny" crap has been defeated a dozen times over -- yes, you still have to get your code on the machine, but given the general windows userbase, that's not difficult at all.
In the case of the redhat box, by the time we inspected it, it had been compromised three times... via an unpatched ssh server, unpatched apache, and unpatched bind. All three where hit by automatic breeching tools -- worms that scan and compromise machines all by themselves.
I recall a former boss's computer getting compromised during the installation. It was either NT4 or 2000 server. I'm not sure his disk (most likely an MSDN disk) had any service packs on it. (this was late '03.) It was beyond the firewall, naked on a Bellsouth DSL line.
I also recall a friend (sysadmin) had his linux (redhat 6.2 maybe) machine compromised within a day of installing it. I don't know if it was within 4min or 16hrs; the next day we noticed it was scanning the network. That was a "naked" workstation on an ISP's core network -- no firewall of any kind. That was 7-8 years ago, and we still kid him about it.
The T1 at the office was seeing about 100 probes per minute years ago when I cared enough to log all that shit. The DS3 was seeing just as much crap the instant it was turned on a few months ago. (seeing how the morons setup that router (cisco), I wouldn't be surprised if people have broken into it -- with no logging turned on, how would anyone know?!?)
Answer: Until the dump user clicks on the wrong attachment, etc. Browsing the web from an unpatched IE is asking for trouble. The same is true of unpatched Outlook and Outlook Express.
NAT cannot protect you from your own stupidity.
*ding* we have a winner. of course, sadly, a linux (or solaris, or in fact, almost any *NIX) box can be hijacked just as fast if no patches are (ever) installed. ('tho i don't know about 4min, as linux isn't as highly targeted.)
Rule #1: NEVER BUY LAPTOPS ON eBay!
50% are stolen, and the other 50% are scams.
Rule #2: NEVER TRY TO SELL A LAPTOP ON eBay!
See rule #1. (odds are very good you'll end up scammed out of the laptop without any cash. worst case, you'll end up in jail for cashing a bad check.)
Actually, that's policy violation in many cases. However, eBay doesn't give a rat's ass as long as they get their fees. I used to report them, but since eBay doesn't do jack shit, why bother; if other people are stupid enough to fall for it...
AMEN!
That they're desperate for customers, don't know how to maintain a website, otherwise have no clue how to do business? By using eBay, they don't have to really run hardly any company at all (which is convenient since from my chair they don't even know how to wipe their own ass)... throw everything on eBay's servers and automate it all -- which is exactly what they're doing... you MUST use paypal; you have 6hours to pay; after 12hours, the "auction" is canceled and you are a "non-paying bidder"... all 100% automatic.
If you aren't paying attention, you might not realize you're doing business with Buy.Scum. I'm sure a lot of people who don't do business with them, are now through eBay.
*pfft* Commercial breaks. That's what Tivo's for.