Slashdot Mirror
ISPs And Router Security
IPvNOT asks: "With all the script kiddies and distributed denial of service tools that spring up each week, there is an increasing use of spoofed network addresses. It would seem logical to me, to help control some of the problem, for ISP's to install a simple access control list (ACL) entry that blocks all packets that do not contain an address within their 'internal' network. How hard would this be to implement on a large scale? Would ISPs implement this?" I would think that an ISP would be able to block and drop anything they receive from the outside if its IP address starts with '192.168.', '127.' or '10.', and there are several others that can be screened for -- are there reasons that ISPs don't do this?
But if you start applying further checks to every packet then the network will surely start to slow and in such a competitive world isps dont want that.
In addition even if 90% of isps can be persuaded to implement it, there are enough that will disregard it and the attacks will surely continue.
A few years ago I was working part-time for a small ISP and I suggested something like this. I didn't know how to implement it so I handed it off to the senior admin. He kept putting it off and putting it off. Everyone thought it was a great idea, just no one wanted to bother since it wasn't absolutely necessary for the daily operation of the ISP. People can be really lazy.
"That's Tron. He fights for the Users."
Laziness, the fact that too often people have no clue how to do a goddamned thing with that little box with the Cisco logo on it, or not knowing where to do handle the issue.
You know certain things about what's inside, and what's outside. Network addresses internal to your network should never be acceptable as a source address on packets coming into your external interface, and addresses which are external should not be accepted as sources on the internal interface of your router. Very simple.
Helps both you and the world.
They should block far more than that. They have an IP range. Anything going out with a source that's not in the IP range should be blocked, imho.
Many ISP's see ACL's as processing overhead on their border routers. Most of this may be due to badly designed ACL's. Reserved addresses (such as the 10.x.x.x,...,192.168.x.x and any hosts from within the network should be blocked ( it is unlikely that traffic coming into your network should have an ip from within your network as it's source ip) We had some high processor utilisation on our border router and after looking at our ACL's, have reduced the utiliztion by 30%. Quality not Quantity is the key.
This might be a completely dumb post, but uhm, with my linuxgateway I simply use ipchains/iptables. And uhm.. Well, from what I read somewhere (ipchains-HOWTO I think), a dx2/66 can do a whopping 2mbit/s.. So slowdown shouldn't be a problem...
That most ISP's don't own all of their network. They purchase a majority of their dialup network access from outside providers like UUNet, Gridnet, Level3, etc. These providers can be prone to changing ip addresses frequently.
Additionally most ISP's want their resources to be widely available. If you can't get your mail from both home and work because the ISP blocks access from your work ip address (on a frame-relay or dsl connection) you will most likely not want to continue doing business with that ISP.
Most routers do have ACL's that limit telnet access to certain ip's or subnets. Only a poorly configured router would allow telnet access to the router from any ip address.
-josh
At least one of the ISP I worked for rejected packets with private or internal addresses as their sources coming from outside on the border routers.
A good citizen ISP should even filter its dialup pool to reject forgeries from its customers. It's called ingressfiltering and is the most reliable way of getting rid of those smurf and other various DoS attacks.
I really don't know anything about this: but I would hope someone else could answer.
Would IPv6/IPSEC security alleviate the situation ? Is it envisaged that internal machines may need to 'register' with a border gateway based upon an administrator supplied certificate ? Could this almost eliminate address spoofing problems ?
I personally wouldn't be surprised to see ISPs start taking that kind of action, but once it does start happening, who knows what the limit will be? If enough ISP start to do somehting like that, then why would it even be called the internet? It just seems like more ways for our rights online to be restricted. But maybe that's just me...I tend to be more paranoid than the average geek.
Time is fun when you're having flies.
-Kermit the Frog
Many networks already do filter RFC1918 packets on their border routers. An interestingly heated discussion on the pros and cons of this is to be found on the nanog list. first message of (LONG!) thread
Camaron de la Isla 'When I sing with pleasure, my
"None are more hopelessly enslaved than those who falsely believe they are free." -- Goethe
A second alternative is to have a check on the modem/ISDN/DSL lines, to ensure that the address on the packet is equal to the one assigned the device on that line.
Last, but not least, for only a little extra complexity, use authentication, such as IPSEC or SKIP.
It's a small world and it smells funny; I'd buy another if it wasn't for the money; Take back what I paid (SoM)
It's just as easy to block DDOS (smurf) attacks both if you are the target or an unknowing accomplice.
The problem is that ISPs don't want to slow down their connection by a few bits/second, and they don't want to do that. Slow connections mean fewer customers which means less money.
-- Ever notice that fast-burning fuse looks exactly the same as slow-burning fuse? I didn't... (Edgar Montrose)
Are you proposing here a method to stop attacks from the outside or attacks that originate against others from the inside? I think you mean the latter.
Limiting to just the addresses in your own range only partially solves the problem if the attacker can still spoof (but then using addresses in your own range). Limiting the addresses per box would seem more appropriate.
Keep in mind, however, that an attacker will usually use a fairly wide range of cracked machines, most of which are with ISPs that have lax security in the first place. It doesn't seem that the problem spots will be the first to get fixed.
- Talence
I plan to plan / Dutch course in The Hague
This is possible, at least at the edge of networks, but I think it would be simpler to do it on customer routers.
Cisco has a command "ip verify unicast reverse path" or something.
This command breaks multicast under some conditions, but should work for smaller/simpler networks and edgerouters.
There is also the problem of complexity, if you are a transit and/or multihomed AS.
Complexity can also be bad for your throughput in some routers eg. it may force the processor to make routing decisions, which is slow.
And access-lists have to be maintained.
/Anders
route 10.0.0.0 0.0.0.255 nul0
Create a nul interface (many routers support this) and update your routing tables to route RFC1918 into the nul interface. Easier and better than using an ACL.
Obviously, this is not always feasible. Many cable/DSL ISPs use RFC 1918 addresses for the Cable/DSL modem devices (there is DHCP option 82 to distinguish whether the request came from the customer's PC or their cable modem). So, you'd have to let the traffic get all the way to your border router before stopping it.
mask should be 0.255.255.255 duh
What sort of security *is* important on a router? I would've thought that a router would be relatively benign, but can't (once compromised) it provide a gateway to ip spoofing, source and destination address spoofing, packet sniffing, and ip source routing? (and denial of service attacks, of course, we won't forget those!) Why else secure a router if you've got a secure firewall?
I deal with security incidents all day. Every admin that I talk to that has been rooted or is being used as an amp start always say: "I should have..." And, they are right, they should have, but they didn't. Admins need to have some plan for security.
Quick and dirty list of security tips:
No replies made to AC posts. Please log in.
Unfortunately a lot of ISP network roll-outs are done by people with very little IP network experience, or by "high paid" consultants.
The people without network experience are somewhat excused, but they should have gotten someone to look it over, and actually try some sort of penetration tests. They'd probably find a lot more wrong with it than routing non-routable IP's.
The consultants don't tend to bother unless asked, as it adds to their already high workload, plus they most likely think "I'm not getting paid enough to do that as well!". Some even just assume that the routers won't even route this sort of traffic unless told to.
A lot of routers don't help in this situation either. The training courses and/or materials for setting them up in many cases are rather badly written, don't cover a huge number of setup scenarios, and usually don't even bother to bring up these sorts of things at all.
On top of all this, you get things like the Managing Director of the company connecting a modem up to his PC and dialling out to his home account cos the connection to the net through the filewall is too slow (which is actually cos someone is trying to launch a DOS attack on your firewall), and then someone gets to his local files cos of some piece of software that he shouldn't have on the laptop in the first place.
Regardless, even if they did block the non-routable IP's, you should still "trust no one" and block whatever you can. If it's connected to the outside world, then there is a possibility that somehow, you could lose out.
The only way to truely protect your data is to grind up your hard drive into powder, magnetize it all, then heat it into a liquid. Cool and grind it up again, scatter it into the wind, and just HOPE entropy does the rest.
This is called "ingress filtering" and has been advocated in IETF workgroups and on NANOG for last 3 years.
;)
See following RFCs:
http://www.faqs.org/rfcs/rfc2267.html ('98)
http://www.faqs.org/rfcs/rfc2827.html ('00)
Many 'big' ISPs apply that to their leaf connections at the edge routers, smaller ISPs usually don't do that, which is surprising since its actually simpler to do if you are small. (less edge routers
The only problem with such filtering are people who are dual-homed and have asymmetric routing: I.E. Customer has to connections, to ISPs A and B, and may be sending outbound traffic with source-addresses in B through A's connection.
This is legitimate, there are many uses for it:
1. Satellite one-way connections
2. 'overflow' routing
3. load-balancing of outbound traffic
So, ISP has to blow holes in their filters for such customers. (Which we do at customer request).
Of course it is good if ISPs trap fake ips.
But you should _never_ relay on somebody else for your own security.
It's a bit like skiping looking the door just because there is a police station in town.
Personally I run ipchains on all my (linux)routers and servers and most workstations, if one firewall is cracked, there is at least some security left in the system.
The big win when ISPs start implementing this is that it reduces pointless the trafic on my local connection.
One of the problems I see with your theory of blocking out all private addresses (for those of you keeping score at home, that's 10.x.x.x, 172.16.x.x through 172.31.x.x, and 192.168.x.x) is that you won't see these addresses trying to come in through your firewall (if you've built it right) as often as you'd think. With Network Address Translation (NAT) and Port Address Translation (PAT), you'll see the public IP addresses of these hosts being attacked more often than not. The ability of any given script kiddie to modify the TCP header will complicate this, but without prior knowledge of your network it's a hit and miss attack. For all they know, you could be running any combination of subnetted private and/or public IP addressing schemes behind your firewall.
The best defense against these attacks is a good ACL on a solid firewall platform. Block incoming traffic from private addresses, but be sensible and put internet accessible hosts on a DMZ. For general internet use, select one public IP address from your pool of public IP addresses and use PAT.
The info above was typed out haphazardly and may not make sense, but after working with Internet security in a myriad of environments, the best advice I can give you is that if even if you THINK you understand it, you should still seek consult (the more eyes the better). I would go into more detail, but unfortunately I have to go set up a firewall for a client :)
-- Stu
/. ID under 2,000. I feel old now.
However, I would say the biggest reason is that this kind of filtering can eat a *lot* of processor cycles on your router. These routers are not cheap, so you tend to buy the least you can get away with.
Also, the packet by packet comparison necessarily slows down traffic. Not a big deal for a small line, but when you're dealing with multiple OC-3's, people get a little antsy about a 10% performance decrease.
--
-- Slashdot sucks.
Is this truly laziness, or is it a simple matter of not adding any unnecessary implementations to the system, which, if already functioning well and smoothly with little to no down time, has no need of any extra and uneccessary protocols? Under the best of circumstances, this would never ever be implemented world-wide, much less nation-wide, and therefore would have no measureable effect on the problem at hand in the first place. And then when you consider the possible effect of these filters... Delays in packet transfer and increased overall network lag. Add in the potential of this conflicting with some other installed program, function or protocol, whether it's within the ISP's network, or a conflict on the user's end, it's really not worth the effort, nor the potential trouble.
"Inveniemus Viam Aut Faciemus" 'We will find a way... Or we will make one!' --Hannibal of Carthage
I worked at a major internet provider for over 2 years, and when I left I was Senior Network Engineer, with only the head of the engineering department above me, and above him was the CTO. We had over a dozen POPs (Points of Presence), and OC3 lines strung from MAE East to MAE West and many points between, and OC12's being installed. So, let's assume I can speak slightly to this issue.
With a major provider, your hardware is going to be big enough (BFR, GRF, etc) to handle 60,000+ routes AND do adequate security filtering. Don't accept the RFC'd routes in, and don't propogate them. Period. Don't accept internal routes from external sources. These are simple rules any major provider *can* handle if they can handle a full routing table. We're talking edge routers.
Smaller providers who are multi-homed and that lease dialups wholesale are a problem, though. Their dialups have IPs that don't belong to them. They often don't have the expertise to configure their ACLs correctly, and leave gaping holes in their security. Sometimes we'd scan our customers' routers with SNMP probes and find a lot of default SNMP passwords for read *AND* write access to their router, and we'd let them know to button up their router. One of our routers would occasionally get flooded with extra routes from a customer (we had lousy filtering) and the resulting flood of traffic would kill the customer's router. The first sign of this would be the customer's line going down. We were understaffed and used several different kinds of routers, so ACL's varied slightly between platforms because of the way they had to be written.
My point is that you need three things for merely minimal security (just by IP blocks):
Hardware: a router with enough CPU and enough RAM
Expertise: engineers that know how to write ACLs for the IOS you're using
Priority: your engineers have to have the time to actually sit down and get the ACLs updated on all the routers correctly
Unfortunately, I don't think there are many providers that have all of these.
When I worked at an ISP we would put anti-spoofing filters onto routers we leased to customers that block the private nets inbound and didnt allow them to spoof outbound. However we could do this only for customers who have their own router. We also couldnt do this on core routers because you might have 500 customers going through one ATM interface (via a switch) so the most you could do is allow customers to spoof each others addresses. On core routers which switch in hardware, ACL's normally push the switching through the CPU which hurts a lot. Also, given how many ISP staff didnt even know how to turn off packets sent to broadcast interfaces (see NANOG re smurf attacks a few years ago) its pretty clear people managing routers dont know much at all. ISP's are generally know to be reactionary rather than pro-active with security issues.
"Now, I hope and pray that I will, but, today I am still just a bill"
Now I hope and pray that I will But today I am still, just a bill
Wouldn't blocking packets from outside the network that bore internal addresses screw things up if there were hosts on the network who could speak to two different routers? And ingress filtering is a pretty well established thing, isn't it? So why ISPs don't implement it might have more to do with Wall Street than Silicon Valley. Bhaloo.
I want to die like my grandfather. Peacefully, in my sleep. Not screaming and terrified, like his passengers.
Having worked at a number of ISP's and talking to those in the trade I've found that it's a matter of the SysAdmin. In general ISP's that are Linux or BSD based are far more likley to have the router set up this way already. ISP's that are microsoft based (And there are many) tend to shy away from touching the router more than they need to.
By the way, although it's true that there's some performance impact from filtering, it's not nearly as much as the ISP folklore (and a lot of the posters here) would have you believe. I've turned on filtering on very heavily loaded routers and had it work fine.
The reason it breaks with multiple egress points is because of assymetric routing. What "ip verify unicast reverse-path" does is makes sure that the path back to the source goes throgh the same interface as this packet coming in. Since you can have multiple paths back to a source on a core router, this doesn't work.
If you are using the router as a firewall or a bastion host then this works ok.
From experience, on the edge of the network, there is NO reason for a packet to come into the network that is not part of your address space. Edge is defined as a single-homed connection with no transit capability. We have a packet filter on our edge routers as well as our core multi-homed router to deny traffic with a source address that doesn't match one of our class-C's.
The problem with doing this on a Cisco is that it requires the CPU to observe the header of each packet going out, rather than have the interface DMA the packet to the destination interface. During the last big round of DDOS attacks, (January-February) we say many ISPs try to put filters in their core routers. The result was a 4x+ increase in CPU usage in the routers, and a router crash in a lot of cases.
We saw BGP traffic increase by over 10x as these routers came up and down all across the Internet. We have our core router set up to log faked ingress packets, and you wouldn't believe how many packets we see. Also be aware, it's not always a DDOS attack that causes spoofed packets. We see misconfigured windows boxes leak the Microsoft Ethernet addresses out PPP, misconfigured firewalls leaking internal addresses, etc. We see no issues with filtering these packets since there isn't a way for those packets to get back to us anyway, and it takes up more of our outgoing bandwidth...
Best bet is to filter as close to the edge as you can. For companies that sell bulk-dialup, their access servers can be configured to filter packets not on their address pools. The routers serving those modem pools could filter on the addreses for that data-center. Cable providers could filter based on the IP addresses assigned to that cable head-end. If we can filter right up to the edge of the transit-network, DDOS should be a thing of the past....
Interestingly enough, I have received legitimate packets across the internet from IPs in the range of 192.168.*.*, 172.16.*.* - 172.31.*.*, and 10.*.*.*. How? Traceroute!
It seems that some networks (including @home) are too cheap to assign an external IP to all of their routers, so when traffic goes through their network, the routers routing it have internal IP addresses (which are not noticed unless you do a traceroute or just ping with a low TTL set).
Example: try tracert'ing a few IPs in the 24.112.51.* (@home) range.
-- Sig, 120 chars --
Your friendly neighborhood mIRC scripter.
if (ismoderator(reader)) hidecomment(this);
* Q
P.S. If you don't get this note, let me know and I'll write you another.
Large ISPs should consider doing this on all of their RAS equipment first and foremost. As has been previously stated in this thread, there are legitimate reasons not to do this on all border routers.
However, in my opinion, there are two real culprits to this problem.
Major router vendors are the primary culprit. This option should be on by default on all routers, and require a configuration change to be turned off. If it isn't turned off then every network you route to the outside world is automatically allowed to pass through. If it is turned off, then you just made a concience decision to run in a less secure environment - possibly a very legitimate decision, possibly not.
The secondary culprit, is the lack of concise and well written AUPs that are truly enforced. Ahhh if only it were legal and enforcable to have an AUP that said simply, "We allow no bullshit. And we decide what the bullshit is." But it isn't so write one that works.
Personally, if I were a DSL/Cable provider just starting up, I'd invest heavily in a bunch of decent NID systems and allow the traffic through. My AUP would state that you pay me first and last months DSL rent regardless of whether you are getting service, if you break my AUP. I'm sure all the mommies and daddies out there would be thrilled to find they have no Internet after the NID system found spoofed packets coming from juniors CPU.
http://windows.scares.us
A mask of 0.255.255.255 would mean that you would match an IP of x.0.0.0. What you really want is a mask of 255.0.0.0 so that you would match any IP of 10.x.x.x.
I worked at a regional ISP until recently, and we did block bad IP's in the routers, along with source-routed packets. For example, RFC1718 addresses (10.0.0.0/8, 17.16.0.0/12, 198.168.0.0/16) are never routed to/from the Internet, and the IP block we owned would never be accepted as a source address from the Internet. Some ISP's do make the effort. (I'm not sure if downstream customer networks were blocked from spoofing each other's addresses, but I believe they were.)
Now, the question in my mind is whether the effort we made (to secure the networks from spoofing attacks) was typical or atypical of reasonably-sized ISP's? (When I started at this ISP, they had two T1's to one backbone provider; now they have four DS3's to four different backbone providers...)
Deven
"Simple things should be simple, and complex things should be possible." - Alan Kay
The original question (without Cliff's comments) is actually why ISPs don't ensure that outgoing packets from their internal networks are indeed sourced from their IP range.
For example, if I'm an ISP who has (to use a popular range ...) 24.226.0.0/255.255.0.0 , I would add a firewall entry that outgoing packets are denied if they are not from 24.226.0.0/16 (same netmask as above, incidentally).
For multiple source IP ranges (including private network addresses), this can be extended by making an ipchains output filter to allow packets going into the network (they've already passed the input and forward filters) and packets sourced from your IP ranges, then deny'ing the rest. Don't forget to add IP spoofing filters to your input chain.
Final note: I actually do administer a network and am doing this. I've turned on logging on those spoofed packets of course, to see if I'm denying anything that shouldn't be, but it works quite well.
- Michael T. Babcock (Yes, I blog)
Oops. I meant RFC1918, not RFC1718. (Stupid typo.)
By the way, that ISP also blocked outgoing packets (to the Internet) that did not have a source IP address that belonged on its network.
Deven
"Simple things should be simple, and complex things should be possible." - Alan Kay
From the UUNET AUP
http://www.us.uu.net/support/usepolicy/
System and network security
Violations of system or network security are prohibited, and may result in criminal and civil liability. UUNET will investigate incidents involving such violations and may involve and will cooperate with law enforcement if a criminal violation is suspected. Examples of system or network security violations include, without limitation, the following:
.
.
.
Forging of any TCP-IP packet header or any part of the header information in an email or a newsgroup posting.
I have worked for some of the larger networking companies in the country (Sprint, MCI, TNS/PSINet) and I can tell you that such access lists are used quite often. By default you should block all RFC1918 address space or else you're a complete and total moron and you should go work at McDonalds. However when it comes to access lists you have to be careful and make them too big or too many of them. You just have to be creative is all. It's not that hard. It's just common sense. However, it was Voltaire that said, "Common sense is not that common." Later. Mk.
Let's say a script kiddy uses Nmap with the spoof function turned on. How are you going to stop the randomly choosen IP's from entering the network? It will stop someone using it from within your network (and be a shock to that person when they get caught because only one source IP was getting to their intended target).
One thing that I've always wanted to try was to filter traffic based on the addresses contained in the MAPS-DUL. I've never tried it because of concerns for slowing down the network (it is a long list) and pissing people off. The problem is a number of things would break. Things like ICQ would have to go through the main server. It would break multiplayer Rainbow6, PCAnywhere. or any other program that expects a direct connection with another other home uses to work. This might be fine on a limited basis were you can actively control what gets in and out of your network. But on the scale of ISP's.....
If everyone started filtering traffic from within their network to stop spoofed traffic from getting out, then spoofing throughout the internet would drop. But everyone would have to do it.
And like everything else, someone will find a way around it.....
Go not unto/. for advice, for you will be told both yea and nay (but have nothing to do with the question)
This just came up on the NANOG mailing list.
The conclusion was that RFC1918 suggests that ISP's prevent reserved addresses from being forwarded, but does not require filtering.
"should take measures" != "MUST NOT"
Also, any packet based ACL is simply impractical on core routers, except for the very fastest (like Juniper M40). Sure, they use ACL's temporarally to fix/debug problems, but leaving them there full time would degrade performance unacceptably.
This was about three months ago.
I do not deploy Linux. Ever.
Cisco has a technology implimented in their IOS which employs the concept of ACLs with dynamic routing. It is documented as unicast reverse path forwarding and basically it checks the routing table and makes sure that there is a route back to the place where the source address came from. There are a number of other check which are also enabled with the use of rpf including some simple packet validation. RPF is getting more and more detailed in later versions of IOS 12.1.x(T) has added a bucnh of new features.
it's /sbin/ipchains -A input -d 127.0.0.0/24 -j REJECT /sbin/ipchains -A input -d 192.168.0.0/16 -j REJECT /sbin/ipchains -A input -d 10.0.0.0/24 -j REJECT
You don't seem to understand, its one thing applying an ACL to a packet based device, you lose efficiency somewhat, and degrade the overall performance of the network. However, many companys are no longer using old IP based systems.
At my previous employ, there were a few customers that used ATM, a cell based not a packet based protocol, ACL's do not work with cell based protocols, as well as people like uunet, att, bbnplanet, most of them use ATM to route traffic internally and across peers from there border routers. so putting up an ACL is not an option.
I came, I conquered, I coredumped
The vast majority of ISPs do have an access list that prevents spoofing coming from their networks. I think a bigger issue is networks which are smurf amplifiers.
While we're on the topic of router security and access lists, did it ever occur to anyone that if ISP's blocked outgoing SMTP on all their dialout ports at the router level, we'd probably see a 72.9% reduction in spam ? -Xian
If you're going to be picky about it, then I'll be slightly pickier:
/sbin/ipchains -A input -d 127.0.0.0/24 -j REJECT
/sbin/ipchains -A input -d 192.168.0.0/16 -j REJECT
/sbin/ipchains -A input -d 10.0.0.0/8 -j REJECT
/sbin/ipchains -A input -d 172.16.0.0/20 -j REJECT
and, I think...
for 172.16.*.* thru 172.31.*.* (also internal IP ranges).
-- Sig, 120 chars --
Your friendly neighborhood mIRC scripter.
if (ismoderator(reader)) hidecomment(this);
* Q
P.S. If you don't get this note, let me know and I'll write you another.
10.x.x.x
127.x.x.x
172.16.x.x - 172.31.x.x
192.168.x.x
224+ (I think)
What else?
Necessity is the plea for every infringement of human freedom. It is the argument of tyrants; it is the creed of slaves.
The problem with this one is that ISPs that lease lines aka uunet/sprint/psi to people like gte/msn/mindspring run up against contract issues. Alot of the time the ISP that leases the lines does not give a shit about what there users send as in most cases the leased lines do not identify which ISP the user is with, only shows a psi.net or a da.uu.net ip. Therefore they do not care. It is very easy to setup per port filtering, its a finctionality on all Ascend terminal servers (and I will assume something similair with others), What happens is when the user authenticates, he has an associated profile with his name, in that profile there is an SMTP server listing, all traffic to port 25 but to the listed server is blocked. The reasons the isp's that lease lines hate this, it makes it very easy for the people who get spammed to ID which isp has whcih leased line since user a can only use user a's isp mail server.
I came, I conquered, I coredumped
doh :)
/sbin/ipchains -A input -d 172.16.0.0/20 -j REJECT
/sbin/ipchains -A input -d 172.16.0.0/12 -j REJECT :)
should be
I counted the wrong bits
-- Sig, 120 chars --
Your friendly neighborhood mIRC scripter.
if (ismoderator(reader)) hidecomment(this);
* Q
P.S. If you don't get this note, let me know and I'll write you another.
/sbin/ipchains -A input -d 10.0.0.0/8 -j REJECT
You're rightThere's one thing that large ISPs simply cannot do, and that's to anti-spoof every dialup client connection separately, because that just doesn't scale well as dynamic dialpools grow in size -- that's just too many ACLs for current-day routers.
Consequently, at best they'll anti-spoof aggregated blocks, but even that can be impossible to do when the IP address allocation is fully dynamic as it has to be for really big ISPs, especially those that are virtualized into more than one branded product that must operate in disjoint dialpool spaces but using the same dialin hardware. And even where aggregated anti-spoofing is posssible, it does not provide full accountability --- you can nail the DoS to an ISP, but that's all, which doesn't help when DDoS attacks are synchronized to appear from multiple ISPs simultaneously.
As a result, until the NAS/RAS/HomeGateways etc do their own per-connection anti-spoofing, there will always be opportunity for attackers to randomize their source addresses to some degree. The problem at the moment is that there appears to be virtually no anti-spoofing on dialpools at all in practice, and that's real bad.
"The question of whether machines can think is no more interesting than [] whether submarines can swim" - Dijkstra
Everyone says "we can't do this because it kills CPU on our routers" and from what I understand this is true. I've read that you can barely manage two full looks at BGP on a Cisco 3640.
Considering what Cisco is charging for hardware relative to what you're getting under the hood, shouldn't we be getting a whole hell of a lot more performance out of our routers? If I'm paying close to $3k for a 26xx router with a v.35 and a 10/100 ethernet port, why am I not getting the equivilent processing power of a $3000 PC? Where's the SMP?
I think the answer to why ISPs don't filter is that many can't; the routers they use aren't capable of the extra processing. But the real answer is that profiteering by hardware makers like Cisco (remind anyone of Apple's prices?) keeps low-end CPUs and small amounts of RAM in their routers while still charging sky-high prices. Is there any reason that a new 3640 doesn't have the CPU equivilent to a PIII650e at least?
Long-term this means that ISPs can't do the egress/ingress filtering that would stymie loads of script kiddies.
Snippet of a filter for either a BGP ACL or just a route ACL... /24,
--
! Deny martian routes
!
! 0/anything
access-list 100 deny ip host 0.0.0.0 any
! 127/8 & longer
access-list 100 deny ip 127.0.0.0 0.255.255.255 255.0.0.0 0.255.255.255
! The private use nets
access-list 100 deny ip 10.0.0.0 0.255.255.255 255.0.0.0 0.255.255.255
access-list 100 deny ip 172.16.0.0 0.15.255.255 255.240.0.0 0.15.255.255
access-list 100 deny ip 192.168.0.0 0.0.255.255 255.255.0.0 0.0.255.255
! Test net
access-list 100 deny ip 192.0.2.0 0.0.0.255 255.255.255.0 0.0.0.255
! 1st and last classical B and C nets (guard nets).
access-list 100 deny ip 128.0.0.0 0.0.255.255 255.255.0.0 0.0.255.255
access-list 100 deny ip 191.255.0.0 0.0.255.255 255.255.0.0 0.0.255.255
access-list 100 deny ip 192.0.0.0 0.0.0.255 255.255.255.0 0.0.0.255
access-list 100 deny ip 223.255.255.0 0.0.0.255 255.255.255.0 0.0.0.255
! All multicast routes - the router now does this itself, but it didn't
! at one point.....
access-list 100 deny ip 224.0.0.0 31.255.255.255 224.0.0.0 31.255.255.255
! Block all routes with a mask longer than
access-list 100 deny ip any 255.255.255.128 0.0.0.127
--
Although this doesn't take care of a number of DDOS attacks as they come from real machines.
Outbound source address packet filtering has been used since the beginning of the DDOS attacks. However, it was only implemented by security gurus to identify a compromised machine on your local network. That is, simply denying the outbound traffic doesn't negate the fact that your machine has been compromised, therefore it's best to log any incident where the given rule has ben violated on the router. Here's an example of this filter when applied to a Cisco router:
access-list ### permit ip ###.###.###.0 0.0.0.255 any
access-list ### deny ip any any log
In the above example, we are allowing a user defined Class-C subnet to be allowed outbound. Therefore you will want to apply this rule to outbound traffic on your serial interface. Even though by default a Cisco IOS access-list has an implicit deny all rule following all list entries, you need to add this line in order to log all denied traffic. Once you identify which local host is sending spoofed outbound packets, you can then work on the oh-so-fun damage control. Hope this helps.
I've asked many an ISP the same question, and usually gotten the same response...
"We cannot possibly filter by source IP at our border routers because it would increase our access time and slow access to the internet"
To which my response is usually get a bigger router. I'm designing an internet security product that eventually will be marketed to ISPs for their high speed customers. As part of the agreement for getting these devices, we will require that all ISPs who support our product implement a 5-10 rule list at their border routers to block stupid things, like anything coming in from the internet with a Source IP of the ISP's network, all the reserved IPs, and anything going out without a source IP of the ISP's internal network. It just makes sense. If they need a bigger router, then so be it. Just giving customers access to the internet and refusing to put in common sense filtering rules in the name of slow routers and slow ping times is totally unacceptable. If someone from ISP land can give me a good reason why my logic is flawed, please let me know.
-Carl "No, we already thought of that one. 'Why?' '42' - It doesn't fit." -Hitchhiker'
Many of the engineers who run the biggest networks work very closely with Cisco to develop, implement and document new features in IOS. Here is a Cisco document which explains how an ISP should implement these features:
http://www.cisco.com/warp/public/707/EssentialIO Sfeatures_pdf.zip
All opinions expressed are mine, if you want them it'll cost you.
it's 127.0.0.0/8. details, details, all those details! :)
that makes a lot more sense. I have only basic networks knowledge (only took 1 class), but it seems that this type of thing would be farily simple. I'm sure there wouldn't be too much of a performance decrease, and the fact that it's dropping those foreign packets instead of processing them further would likely make up for any extra overhead. But, how effective would this strategy be? From what I've read, many of the DoS attacks merely send so many packets toward a computer that it can't handle them all. Couldn't this happen whether the packets are dropped or not? Perhaps an attempt to restrict the 'from' addresses would be futile.
Time is fun when you're having flies.
-Kermit the Frog
However, your subnet mask (as is the correction you posted) is goofy, although it makes a nice wildcard mask. Let's give the Cisco kids out there some useful syntax that they can cut-and-paste into their routers (as long as they're in privileged exec/enable mode):
ip route 10.0.0.0 255.0.0.0 null0
ip route 127.0.0.0 255.0.0.0 null0
ip route 172.16.0.0 255.240.0.0 null0
ip route 192.168.0.0 255.255.0.0 null0
The point is that if every ISP filtered outgoing packets to make sure that they came from legitimate IP addresses, while it may not stop DOSes, it would at least make it a LOT easier to shut down the offending sites since they couldn't lie about where they were coming from. Currently when you get DOSes, it requires a hellish amount of effort from the backbone providers if you want to try to figure out where the (spoofed) packets are coming from.
Filtering reserved IP's are but the tip of the iceburg when it comes to DOS attacks. Today there is more unallocated IP space then allocated IP space, and all that unallocated space is just as effective when used as "random source addresses". Ever get a packet from class E space? How about one with a multicast source? Get a lot of packets from 80/8, that would be bad too.
The actual list, if you wanted to prevent packets from bad sources would be large, I estimate 200-400 lines to catch them all. Most routers are not happy with ACL's that long, although the vendors are catching up. Worse though, is there is no good mechanism to notify ISP's when space starts to be used. Registries don't have to e-mail every ISP today and say "take this out of your filters, we're allocating it". If all ISP's blocked this space there would be significant pain every time a new block was used, which is quite often today.
Could you block RFC1918 space only? Sure, cakewalk. Would it help prevent DOS attacks, highly unlikely. The kiddies would work around that in about 3 seconds and use other space. Don't forget, spoofing real, allocated, in use IP's works just dandy too.
The wireless service is really cool for downloading and surfing the web, but the high latency analog modem leaves a bit to be desired. The Frac-T1 is great for running my website and has nice low latency, but it's lacking in the raw bandwidth dept.
Using a couple simple linux tricks (IP Masq, interface alias, and a couple static routes), I send out all my client access on the Frac-T1 service, with the packets labeled using the IP number assigned to the high speed wireless... when the remote server responds, the packets return on the wireless service, at really high bandwidth, and without tying up a phone line.
If the folks providing me with the connectivity on the Frac-T1 line did Ingress filtering... well, that would suck. I've talked with them about this setup, and they're fine with it. They don't seem to feel like it's their job to prevent DDoS attacks... they only worry about them when they're being attacked. They seem to have one really expensive router that plugs into a T3 line, and I'm sure it doesn't have much extra CPU and RAM. They're on a tight budget, but the upside to their tight budget is good service and very competitive prices. They're also quite friendly... instead of the usual ISP responses: "That's impossible" or "We won't let you (because we don't understand it)", these guys more or less say "Anything's ok, if you don't need tech support and it doesn't hurt our network".
Because they don't do Ingress filtering, I was able to make this really cool setup. I asked a few ISPs if they'd allow me to do something like this (I just wanted to know if they did Ingress filtering), and they said it wasn't even possible to use two providers like this. A couple said maybe when IPv6 appears. It does indeed work very nicely.... obviously they didn't know a damn thing about TCP/IP.
I guess it must depend on the ratio of good packets to spoofed ones. If you only have to spend a few cycles to throw out a useless packet, that is a packet that isn't sucking bandwidth. The complication is, you don't just check that packet, you have to check them all. So,
1)at what point does the slow-down from checking packet headers meet the slow-down from transmitting spoofed packets?
2)Is there any way to do an imperical study to track some statistics?
Aah, change is good. -- Rafiki
Yeah, but it ain't easy. -- Simba
The difference is that the overhead/slowdown from filtering can bring a router to it's knees every minute of the day forever, a DoS usually only lasts a little while. I personally am in favor of filtering but it's not a black & white issue, there's many different factors.
Isn't this just another form of censorship?
I have the right to say whatever I want in my packets, and no one can take that right away from me!
The IP-section of the packet was by no means designed as a verification mechanism of its source, but for aiding the recepient in directing a reply. If the sender chooses not to reveal his origin, that is his decision! The source IP should not be taken as seriously as many people taek it. Having ISPs verify outgoing packets is analagous to having the return address of post-cards monitored by the post office.
Sure, DDoS attacks and such will be easier to trace, but is it worth giving up the freedom of anonymity?
Why don't we just attach a unique serial number, associated with our identification, to all the packets we send out? And have the routers at the ISP's cross-reference this ID with who is actually logged in on that node, etc. This way we will get rid of all the DDoS attacks!
For how an altruistic a stance
Trying being a little more consistent
Like those using one way Satellite and Cable services, who use dialup for their upstream connection.
--
My comments and opinions completely reflect those of anyone and anything I am remotely associated with.
Um, you don't understand. If they're on your network, and spoofing your address, then they're insanely easier to track down. If they're off your network, and spoofing your addresses, then you need to block them as they come in, because not only is that a DoS attack, but it's likely to interfere with the legit owner of that IP address.
-David T. C.
If corporations are people, aren't stockholders guilty of slavery?
We apply anti-spoofing access lists on all our customer edge routers, on both inbound interfaces (to stop their naughty people, and protect them from other ISP's naughty people where the ISP hasn't done this). Most of the edge routers aren't that busy, and the list is usually pretty short. We don't do anything like this on our core routers or border routers with our upstreams due to the overhead.
Go to AntiOffline. I know that the guys running that site have done some research into DDoS that haven't been tested yet.
Mas vale cholo, que mal acompañado.
Even many DSL systems use PPP (wierd, but true; newer consumer DSL is IP over PPP over Ethernet over ATM over DSL.) So similar enforcement is permissable there. If you have several IP addresses on the line, PPP has to know it, though, which currently isn't well-supported. But that's an extra-cost option with most DSL vendors anyway.
PPP is CPU-intensive enough that checking the IP address in the PPP server shouldn't be noticed. It's probably better to handle this before the first router than trying to maintain validity tables in the router. Attackers locked down to a range of IP addresses can still use one belonging to someone else as long as it's in the valid block. So the DoS kiddies can still attack; they just have to change their strategy a bit.
Real LANs with WAN connections are another matter, as are the dumber flavors of cable modems.
Right...if you read the previous responses, you'd see that I misunderstood, and it was explained to me. Thanks -- Mike
Time is fun when you're having flies.
-Kermit the Frog
What does that have to do with restricting your "online rights"? The Internet is a priviledge if you haven't realized that yet, as is your local POTS phone network. You have no "right" to use it. You can buy service to use it and if you keep your account in good standing you may continue to use it as long as your provider agrees to offer it to you. As long as they have a good reason to disconnect you there's nothing you can do about it. Are you going to sue your ISP for disconnecting your line because you were spamming or attacking other machines on the Internet using spoofed addresses? Come on.
That's why I won't work for Intel, if they treat consultants that way.
Your second guess was correct: static routes (with a default metric) are always favored over dynamic routes. Routes to directly connected devices are assigned a metric of 0, and static routes have a metric of 1. Dynamic routes start in the double-digits.
Routes learned through RIP have a metric of 110 (iirc). So, if you wanted to configure a secondary route statically, you would assign it an administrative distance of 200 (or something), so that if the route learned through RIP failed, it would fall back on the static route. This is a frequent way of using of static routes for Dial-on Demand Routing (DDR) where you can't run a routing protocol across a link (since it's usually not up), so you have to define the routes statically, but don't want them to be the default (yes, this sentence has two predicates, but that's okay).
This is all dealt with in RFC 2267: Network Ingress Filtering: Defeating Denial of Service Attacks which employ IP Source Address Spoofing
\w0zz - OpenBSD - A Better Solution
It's simple, really. As a matter of security; Whenever a router is aware of all address space on one port, it should block inappropriate traffic; Core network routers cannot do this. ISP routers should all be able to do this.
At the ISP I work for, I am in charge of the ACLs on the routers. I am currently blocking all RFC1918 blocks (192.168, 10., etc...), loopback addresses, any outgoing not sourced from our IPs, and incoming not destined for our IPs. I also don't allow incoming sourced from our IPs. It's not difficult to do, it's only a few lines. If you need help doing this, feel free to e-mail me. vanbrunt@vanbrunt.com
Getting ingress and egress filtering in place was a big issue in as we came up on the Y2K turnover. There was a lot of fear of DDoS attacks and most Federal agencies (including mine) adopted the practice and encouraged our ISPs to do the same. We recognized that it would not eliminate the problem, but figured it would help weed out some of the most flagrant abuses. This effort got a lot of support in Government and private industry and I would have expected most ISPs to have adopted the practice by now.
The summary of what you'll see is a consensus that people who configure routers with different MTUs on different interfaces using RFC1918 address space, well, shouldn't do that. As long as they don't, you won't see any of the breakage that you describe above.
My Web Page
You just can't assume the ISP is going to do it right. If you're really concerned (which you should be if you're on DSL or cable), get a 486 or Pentium or 68K or whatever, put Linux or *BSD on it, and set it up as a firewall between your bridge and your internal network. That way you can _know_ it's safe, not to mention the fact that you can prevent your ISP from scanning you. :)
Performance wise its very dependent on the router and OS used - most routers would fall back to process switching packets through with ACLs. newer bigger routers can build ACL processing into a switching engine so it doesnt bother the processor.
Personally Im looking at Cisco routers, with later versions of IOS you can do ACL's in fast switching and silicon switching, but the IOS and hardware has to support it - question is which ones! I saw a list somewhere the other day, but its not one of those things that seems well known - Ive seen some pretty experienced people say that ACLs mean it always falls back to process switching (Hell - I freely admit I could be wrong on this point - its just what I read - I dont have a 7206 sitting in my home lab to play on).
TTFN
Lauren
(CCNP, CCDP certified)
--
Lauren Child, lauren@laurenchild.net
Go to your Cisco Config prompt, enter in the following:
:).
access-list 101 deny ip 10.0.0.0 0.255.255.255 any log
access-list 101 deny ip 172.16.0.0 0.15.255.255 any log
access-list 101 deny ip 192.168.0.0 0.0.255.255 any log
access-list 101 deny ip 127.0.0.0 0.255.255.255 any log
access-list 101 permit ip any any
int
ip access-group 101 in
Of course, make sure that you don't have an access list 101 already created
Go to your Cisco Config prompt, enter in the following:
:).
access-list 101 deny ip 10.0.0.0 0.255.255.255 any log
access-list 101 deny ip 172.16.0.0 0.15.255.255 any log
access-list 101 deny ip 192.168.0.0 0.0.255.255 any log
access-list 101 deny ip 127.0.0.0 0.255.255.255 any log
access-list 101 permit ip any any
int *insert your favourite interface here*
ip access-group 101 in
Of course, make sure that you don't have an access list 101 already created
p.s damn, I should learn to preview my posts before submitting.
Aren't 192.168.* and other private IP addresses non-routeable?
Perhaps they should do what our ASP just did - configure their firewall to deny packets on all ports except 21. Made accessing the Applications on port 80, never mind what else happens on port 80 (yes I know what that is), a bit difficult, though.....
That's why you filter at the border before the erraent packets get in your core.
Kashani
- Why is the ninja... so deadly?
Hey, if you're looking for a good PCI DEC tulip card w/ 4 LEDs, you can't beat the Netgear FA-310TX. $17.95 last time I saw on Buy.com, and I'm currently running them on 4 machine on the home LAN (Linux, Win98, Win2K, FreeBSD 4.0 Current, and OpenBSD 2.7). The OpenBSD box is running w/ 2 of 'em right next to each other handling traffic via IP Filter. The only other variety of card is an old 3Com509 in a 486DX/66 w/ 12MB RAM running LRP (Materhorn 2.9.4). As far as reliability, that's all all my gaming buddies use, and those of us who build machines for a living use 'em pretty extensively in business environments w/ no real failure attributed to the NIC so far.
DLink DE530
Netgear FA310
Using an el-cheapo used Pentium as an ipchains box, you can filter DS3 speed bandwidth with a fairly healthy ruleset without coming close to stressing it.
It would not be hard for Cisco, Foundry, Nortel et al to add enough CPU power to their medium sized boxes to perform similar filtering capability for ISP's border routers. Given the base price of one of these babies, even some extra dedicated hardware for this task (couple of I/O buses, a few meaty RISC CPU's, some RAM) would not be egregious.
What is the typical CPU complement of a meaty router?
ACLs force process switching only on older IOS versions. Or should I say ancient :).
;).
:).
For the newer IOS versions the router can fast switch them. And maybe even SSE or whatever them.
The general guide for Cisco stuff is if the technology/protocol/feature is new, it will be "process switched" which means the main cpu has to decide from scratch what to do for EVERY packet. A revision or so later, it will be fast switched where main CPU caches the first decision. Then in later revisions switching will often be done by the ASICs or more specialised hardware aka silicon switching or whatever the Marketing people want to call it. Of course if you're doing exceptional stuff like router encryption or compression then all bets are off
So with reasonably up to date hardware and firmware there is no technical/performance reason why you can't put ACLs at your inbound points or POPs to prevent your own users/customers from doing IP spoofing.
You will have a bit more trouble at the interfaces with other ISPs, but it shouldn't be a big problem unless you have a multigigabit/s interface
As long as the majority ensure that THEY themselves cannot do spoofing, it's hard for hackers to keep exploiting them. After an hour or so, you can trace the source and fix the problem.
ISPs that allow spoofing should be shut off.
Cheerio,
Link.
The problem with having the 192.168.0.0/16 10.0.0.0/8 and 172.16.0.0/12 blocked at the ISP is the fact that alot of ISP are using these IP to route internal traffic across their internal networks.. All in the name of saving an IP or two....
Filtering should be done mainly at the edges to better identify the guilty/exploited parties.
:).
Filtering at main trunks is just to protect the innocent.
Also you probably don't have to use ACLs for martian filters or simple stuff like that. Couldn't you just use static routes? Route unwanted packets to a null device - make good use of the ASICs.
What's a few more routes to your 80,000 routes
Link.
These engines have been tuned, over the years, to do relativly high performance destination based IP lookups on packets. As an example, a Cisco 12000 Gigabit Ethernet line card can do about 800 megabits worth of forwarding. You notice how this is less than the full line rate for the card?
Adding ACL's can chew up processing time that could otherwise be used to forward traffic. In a typical Cisco router, each ACL line is examined sequentially. Just a one line ACL can signifigantly impact packet forwarding performance. Ten lines long, and you'll probably see 30% reduction in packets per second.
Now, there's new technology on the way but as with most things it's slow in comming. There's some software improvements on Cisco's these days in the 12.0(S) trains that allow for "compiled ACL's". These use a FSA (finite state autonoma) to compile an ACL into a, essentially, "regular expression". It requires only one pass of the packet for any length long ACL. Lookup times are always deterministic. I think the break even point is a three line ACL over regular ACL's.
Next in line is the addition of hardware based IP lookup engines and hardware based TCAM pattern matchers for filter list lookups. The new cisco Three Port Gigabit Ethernet line card for the 12000 is like this. However, packet forwarding performance is still around 2.5Mp/s, meaning you can really only use two ports at full guarenteed line rate. See the latest Cisco Release Notes for information about the new line cards.
As a comparision, big routers handeling big web sites these days can do aggregate of about 5-7 million packets/second. This means ACL's are just right out for most of these routers with the current generation of line cards/technology.
So, the simple explination is, most big web sites and their network providers know damn well what ACL's work best in what DoS's. But to put them on all the time means killing top end performance. So they only put them on when an attack is under way, and those are pretty easy to spot. :)