Middleboxes vs. the Internet's End-to-End Principle

arglebargle_xiv writes "The Internet was designed around the end-to-end principle, which says that functionality should be provided by end hosts rather than in the network itself. A new study of the effect of vast numbers of middleboxes on the Internet (PDF) indicates that this is no longer the case, since far too many devices on the Internet interfere with traffic in some way. This has serious implications for network (protocol) neutrality (as well as future IPv6 deployment) since only the particular variations of TCP that they know about will pass through them."

Comments

Too true

[ShooterNeo]

I just moved into an apartment with internet provided by ethernet jacks in the walls. The actual architecture is a major ISP has set up their own routers somewhere, putting me permanently behind a NAT. I cannot open a single port, so no incoming connections can ever reach my computers without one of my machines sending a packet out first.

This has SIGNIFICANT advantages : most worms cannot spread because my computers cannot receive a packet from any machine without software on my machine actively establishing a connection first. No exceptions. It means that bittorrents and other P2P software barely work at all. And so on.

For the ISP, this is ideal. And, the ISP offers unheard of speeds in this restricted setup. 4meg upload/4 meg download is free with the apartment rent, and for $40 a month they'll give me 50 meg upload 50 meg download. For a USA ISP, that is crazy fast...but the limitations make the high upload close to useless.

And, the other interesting thing is that nearly everything I've ever done on the internet still works. My computer is unable to communicate with anyone without the help of a server and is a permanent client, but in today's world that's the norm.

Re:Too true

[adolf]

most worms cannot spread because my computers cannot receive a packet from any machine without software on my machine actively establishing a connection first. No exceptions.

No exceptions, except for laptops, netbooks, and other various-and-sundry gear which travels between networks.

Your walled garden may, indeed, have walls. But it also has unguarded gates through which anything may pass.

Re:Too true

[icebraining]

Do most worms actually infect home machines by direct connection? WTF is running on people's machines to allow that?

Personally, I'll refuse any NATed connection if only on principle. Censorship evading technologies like Tor and Freenet depend on people being able to connect directly to each other. That turns the 'net into TV.

Re:Too true

[AtomicJake]

Do most worms actually infect home machines by direct connection? WTF is running on people's machines to allow that?

Yes. Otherwise it would be a virus and not a worm. And what to run to allow that: e.g. XP before SP2 without a good firewall (remember "slammer"?). Or Linux with an old and unpached Apache. You would be amazed how many old installations are out there, which are not up-to-date.

Personally, I'll refuse any NATed connection if only on principle. Censorship evading technologies like Tor and Freenet depend on people being able to connect directly to each other. That turns the 'net into TV.

I understand the reasoning, but you still need a good firewall. Most people use NAT as a means as a firewall. And, of course, if you have a DSL connection at home, you need a NAT, unless you've found a provider that gives you enough IP addresses for all of your devices (or you have only one device).

Re:Too true

[definate]

Wow, that sounds like hell to me. I wouldn't be able to access most the services I run.

Also, what worm, made since Windows XP upgraded it's firewall to "barely functional", still relies on direct connections? I call bullshit. That's not a feature. Almost all of them, to get around the advent of NAT and firewalls, call out to a command server. Often encrypted traffic too. So, while you say "most worms", you actually mean "few worms", since most worms have multiple attack method (well, the major ones do), and some of those methods, are passive, such as infecting websites, flash drives, emails, etc.

The only feature, is that it makes BitTorrent and other P2P more annoying to deal with, but doesn't stop them. This is more a feature for the ISP, and is likely how they can afford to offer these faster connections for the lower price... because using your connection, is hard.

Re:What

[martin-boundary]

Routers and switches operate at a lower level in the stack. The end-to-end principle is about user apps, and it makes A LOT of sense.

Basically, whenever the pipes become smart, things overall become less reliable. It's simple math. Add one component along a network path, and now every app that uses the path may have to be fixed to cope with the new behaviour on that path. For example, if your ISP adds an HTTP filter, every app using HTTP is at risk of breaking the next day.

With end-to-end, the only correct place to put the new functionality (HTTP filtering in this example) is at an end, namely as a process on a user's computer. Now, the simple math works out: the user is filtered (same outcome as above), but the pipes are dumb and the only thing breaking due to the filter are programs on that user's computer. Everybody else's software works fine just as before.

Re:Internet with middle boxes like sex with a cond

[jamesh]

And the most lost on the most people at once analogy award goes to...

http://en.wikipedia.org/wiki/Condom for anyone who doesn't know. If only i'd known before my 4 kids were born... (hi kids, if you're reading this!)

This is a surprise?

[cardpuncher]

Most of the "routers" (which are really a cross between transport-level and application-level gateways) supplied to domestic customers aren't even capable of the full gamut of IPv4 features: no real hope of extending TCP, transport protocols other than TCP or IPv6.

TCP/IPv4 is now a living fossil and will persist in its present form as an ISP access protocol, ironically filling exactly the same function that X.25 (so much derided by Internet professionals at the time because it wasn't end-to-end) was designed to provide. Big ISPs have the same business model as the old telcos (and indeed may be the same business) and they need to control access to their network and bill for it. They can't do that without "middleboxes" of some kind. End-to-end was only ever really feasible for closed-user-group networks paid for by third parties.

On the plus side, a more capable "middlebox" would allow you to negotiate classes of service with your ISP which might obviate the need for the ISP to randomly traffic shape in ways that suit noone.

What

[ledow]

The "end-to-end" nature of the Internet ended with the first firewall. Not to mention NAT, proxies, etc. To get to the point where I have a transparent squid proxy protecting my workplace (a school) is only a teensy, tiny step.

"End-to-end" is a pipedream and can't possibly work because of the sheer security and scale of such a network (i.e. there would be nobody on the path able to stop a DDoS against you!). It wouldn't work, and that's why other solutions exist.

Hell, virtually every device ever sold that handles IP traffic modifies it in some way that defeats this "end-to-end" crap. They have firewalls. They may offer NAT. They might offer ping-blocking. Hell, the first thing any decent firewall does is turn off most of the unsolicited packet access that it receives, whether that be ICMP messages, or packets with fake origin. Without that, you'd have chaos.

Re:What

[kasperd]

I'm afraid I haven't read up on TCP/IP in a while

Even the term TCP/IP in itself is misleading. TCP and IP are two separate protocols. TCP is designed to be run on top of IP, and TCP does have some knowledge of the underlying protocol (a bit too much some would say). IP on the other hand has no knowledge of TCP. The IP header contains an 8 bit protocol field, so you can in principle run 256 different protocols on top of it (some values are reserved, and not all the other values are assigned yet). An implementation of IP does not have to implement all the higher level protocols, in fact only ICMP is mandatory.

A router is supposed to work on the IP layer. A router should not know about TCP, and it shouldn't care what protocols are run over it. In practice routers do tend to have TCP and UDP implementations as well. They have TCP because that is what is typically used to configure the routers, and BGP is run over TCP as well. And in some cases you may want to do DNS lookups, and for that you want UDP.

When you build a router you have to keep the right layering. The low level should do routing and not care about UDP and TCP. The next level can do TCP and UDP, and on high end routers the separation should be to the point where this is even implemented by a different piece of hardware from the one that does the routing. The next level up the stack can handle configuration and routing protocols. This layer can then push updated routing tables to the actual hardware doing the routing. If the different pieces stick to their area of responsibility, things will work out. All of those higher levels could in principle be implemented by a computer next to the router and leave the router to do only routing.

Some routers have features to interpret higher level protocol headers such as UDP and TCP and mangle them. Once it starts doing that, it is no longer a correct implementation of IP according to the original spec. The network is supposed to get the higher level payload from source to destination without mangling it. The network as it looks today fails at that task.

but doesn't every device modify the header of every packet while sending it downstream?

Routers do. They have to decrement the TTL, and in case of IPv4 adjust the header checksum. But most of the other fields in the IP header are read-only for the routers, and the higher level protocol headers such as UDP and TCP are totally off-limits.

Re:What

[smash]

Exactly. End-to-end as a mandatory access scenario is for GNU hippies like RMS who believe in unicorns and that everybody should hold hands an sing.

The ABILITY to do end to end transit when both parties agree to such is a very good thing to have, yes - but to assume that end-to-end should always work in the real world where we have assholes out there who want to rip you off (money, cpu, bandwidth, etc) and basically fuck you over is never going to be realistic.

Re:Internet with middle boxes like sex with a cond

[PopeRatzo]

It somehow doesn't feel right and is less satisfying but can protect you from some nasty stuff.

But "nasty stuff" is the reason most people go on the Internet.

Re:What

[fudoniten]

That's why it's called the end-to-end principle. It doesn't mean that nothing should ever, ever modify a packet on the network. But, end-to-end communication with functionality pushed to the edges is an ideal for which to strive. The fact that almost any two hosts on the Internet are able to communicate (after jumping through some stupid and mostly unnecessary hoops) is evidence that some people out there still consider it a solid principle. Just because you cannot hit the platonic ideal doesn't mean you should scrap the whole notion. It's fundamental to the Internet. IP is built on end-to-end principles, it doesn't give a shit what it carries, it's simple and dumb, so it's working perfectly today, decades after it was created. The farther we stray from the principle, the worse things get; firewalls, gateways, NATs and proxies have forced most traffic to masquerade as HTTP. Is that a good thing? Is that safer, or smarter?

So, yes, there are cases where you want a bit of intelligence in the network to stop a DDoS. You also need routers to route packets. That doesn't invalidate the whole idea, unless you're a literal-minded idiot.