In the couple of weeks since the original article, there's been lively discusion on the Opennic mailing list about how to work through bad routing proxies. The problem to be solved is that HTTP proxies masquerading as routers can't easily access machines in alternative DNS domains.
If you find yourself behind a shitpile^H^H^H^H^H^H^H^Hrouting HTTP proxy, the most straightforward solution is to install your own proxy that translates all requests to numeric-IP CONNECT requests. Then your proxy can talk via direct TCP connection to the original host.
I mentioned a short hack I used to test some ideas, but a full-featured local proxy needs more than that. Squid is a good starting point and I plan to cruft together an appropriate mod to it soon.
It's not so bad. Figure 2e6 lbf acting on the thing, and a 6" thick window to take up the strain. That's about 1.5 kpsi overall stress inside the window, plus a bit for imperfections in the construction. Add some safety margin and you can figure on around 5 kpsi of stress.
I'm not sure the strength of their window material but Kevlar can be made clear and has a strength of around 250 kpsi.
In California, at least, 10 degrees ATDC timing at idle is mandatory on post-1972 cars. That limits nitrogen oxide emissions in high compression engines by reducing peak combustion temperature.
I ran into that problem when I was still a VW grease monkey -- 1973 VW engines hardly qualify as high compression, and emit very little nitrogen oxide. The ATDC timing causes extra heating and lower efficiency at idle and low speed. EGR works on the same principle -- lower peak combustion temperature to reduce NO and N2O production (at the cost of slightly more CO and HC emissions, and more fuel consumed).
The law was written that way because it's hard to detect the nitrogen compounds cheaply in the exhaust, so it's easier to mandate the most common solution than it is to mandate testable standards.
If you like sausage and respect the law, you shouldn't look at how either is made. (Mark Twain).
Us too. I live and work in Colorado, and was astonished to find that (at least for October and November) packets between work and home were being routed through Arlington. Slowed everything down: instead of ~100ms latency (already pretty slow, through town) I was experiencing more like 400ms.
... in one of the "future history" series. I forget which one -- but it was one of the lunar stories. The Rolling Stones? The Moon is a Harsh Mistress? The Menace from Earth? I remember one of his Knockout-Nobel-Laureate-Who-Just-Wants-To-Make-Bab ies women talking about it...
Any software that will dump core, just from having a screwed-up config file, is crappy software. Especially software that's designed to look for malware on your system. Didn't these guys ever play Core Wars?
I mean,
come on, people! Checking all contingencies is something you're supposed to learn in your first programming course. Especially in a hostile computing environment (spy vs. counterspy) you have to write airtight code or you'll get got.
Actually, that exact argument is why I don't like to see Kuhn quoted. The effect is certainly true -- that young scientists get ideas that aren't accepted by their older peers, and the rate at which the ideas get accepted is more-or-less the same as the older peers' rate of dying off...
So far so good.
The problem is that the ``violent 'paradigm shift' '' to a new theory doesn't change the body of existing experiments, or the rigor of the scientific method. Spinning superconductors won't start to fly just because everyone believes they should. Any new idea really does have to fit within the existing framework of experimental data, and anyone proposing a new idea has an obligation to make certain ancillary predictions. (For example, in his initial general relativity paper, Einstein did this, citing a few famous predictions of wide-ranging experiments that should conflict with then-current theory if his ideas were right). An experimentalist should at least think through his effect enough to have an idea how it should be reflected in other experiments besides his own.
In this case, there's a tremendous amount of evidence against the kinds of claims that crackpots and fringe scientists investigate. For example, if spinning objects really fast makes a gravitational vortex or something, then why dont' lab technicians notice it when they set their coffee on the ultracentrifuge?. Unfortunately, crackpots are much more common than true ``fringe scientists'' who go around investigating, well, `paranormal' phenomena. Why? Because 99.99% most `paranormal' phenomena turn out to be made up or the result of experimental error, and who wants to spend years in combat with some stupid crackpot who doesn't have the insight or the will to do careful experiments? Sure, the 0.01% case might be great -- but you only have one life to live. Why spend it arguing with crackpots, when the simple expedient of waiting for one of them to succeed dramatically (by, say, opening up shop and selling levitating cars) works just as well from a scientific point of view?
Encrypted payload will prevent broken routers from looking into it.
Yup, that's pretty cool. I like that (for example) I can get https:// requests through to any host I want, simply because the encryption enforces the layering of the IP stack. Unfortunately, almost nobody actually publishes the same pages through http: and https:. The encryption layer just uses too many resources in high-bandwidth applications.
I wonder if that'll ever change?
What prompted me to write to Slashdot was the thought that perhaps others are having similar difficulties. If I were dealing with a simple proxy router (such as you apparently helped design the standard for) then my solution would be simple: direct ICANN URLs through the proxy and other ones elsewhere. But I really have no choice but to use the proxy, because my ISP is intercepting all of my port 80 packets.
In a more general sense, OpenNIC isn't the only case where the ability to access http directly is important. Other cases include testing virtual hosting or accessing hollow-tree virtual hosts within a machine
with a known IP number.
You talk about trying to coerce the rest of the world into supporting an alternate DNS. That's not really necessary: strict IP routing already supports alternate DNS roots -- the layered structure of (most of) the protocols keeps your choice of DNS root from interfering with where your packets go. That separability is a Good Thing, and part of a whole suite of related Good Things that make the IP concept work.
Ok, so like I said earlier, you broke your own service by using DNSs other than provided by your ISP. An ISP, whether it be your office IT group, or Earthlink, cannot possibly be expected every single possible variation in service that any single user could require--it's not technically (well...financially) feasible.
Well, actually, no, what I'm buying is internet routing, using the internet protocol. That protocol is pretty well laid out in the RFCs. There's no special reason why this cache has to be the way it is. Other superior solutions exist for free -- some of the other answers in this very thread point out proxy caches that do the Right Thing, and as many have pointed out this behavior is totally to be expected in a manual (and hence avoidable) caching proxy.
But caching proxies exist on a different layer of the internet protocol than do routers. Mixing those two functions is bound to break things. I understand the difficulty of getting users to use a proxy -- many don't understand that it will actually benefit them in most cases, for example. But caching proxies aren't routers, and shouldn't be.
What's especially sad is that, by not planning for the failed-DNS-lookup contingency, whoever wrote their cache is breaking a whole suite of things that don't have to be broken, even with this flawed solution. Lots of effort, including this whole slashdot thread, some angst on the part of their technical staff (having to deal with me) and my own minor frustration could all have been avoided if perhaps two to ten lines of code (the contingency code in the proxy) were better thought out.
The problem I'm having is that, because I have to rely on the proxy's DNS to resolve a web hostname, I can't get certain HTTP requests to certain hosts.
The specific example that I mentioned is the "http://www.dev.null" URL, but there are loads of other examples. In particular, "http://www.dev.null" is a completely valid URL that points to, well, the dev.null site. To resolve the URL into a host IP number to connect to, you have to be using the openNIC DNS tree (.null isn't supported by ICANN).
So far so good. I can connect my web client, for example, to "https://www.dev.null" and get the secure pages for the dev.null site. But the main pages at "http://www.dev.null" are invisible to me -- the most I can get is an error message from my ISP's ``transparent'' caching proxy.
The reason for the difference is that my home box uses OpenNIC -- the https: request gets resolved into an IP number and my local host just telnets to that host on the appropriate port and issues an encrypted HTTP request.
The http: request fails because, although my host resolves the host-part of the URL into an IP number, that number is ignored by my ISP's ``transparent'' proxy. All port-80 packets emanating from my home network get intercepted and fed into the proxy. The problem is that the proxy attempts to resolve the URL using ICANN DNS -- so it's unable to identify an IP number to associate with the URL. It's apparently poorly coded, in that it doesn't know enough to try the IP destination address on the incoming stream that it's intercepting.
My ISP reacted sort-of the way you just did: "well, if you're not using DNS, you're out of luck, pal." But that's Wrong. If they had an explicit proxy server and I had some choice about what happened to my packets, that would be OK -- but they don't. All of my port-80 packets are being intercepted by a piece of buggy 'ware. That (A) breaks the layered structure of the IP protocol, and (B) prevents me from accessing big chunks of webspace that I'd like to use.
If you administer a similar transparent proxy setup, I encourage you to ensure that it falls back gracefully to being a router if it can't make sense of the http: requests. Apparently the most popular Cisco solutions and Linux ipchains router/proxies are pretty easy to configure correctly (according to some of the other answers).
Thanks. Unfortunately, this doesn't work well if the site (like most sites these days) uses name-based virtual hosting. For a given host, you might get several different web pages depending on what you put in the host part of the GET URI.
What's broken is routing. I can't route port-80 packets to the host of my choice. Proxies are all right as far as they go, but this one is mandatory. The ISP didn't advertise it and tech support takes the attitude that, well, it's transparent so it doesn't matter. It's just that it's not transparent.
That's the problem with a lot of software "solutions": people don't think through all the worst case scenarios, just the main one, and the tool breaks when it's used in a way they didn't expect.
Yep, the cache is behaving correctly for a cache. The problem is that it's behaving incorrectly for a router, because I can't send the http: requests I want to the hosts I want to send them to.
I'm not familiar enough with the ins and outs of cache design to know whether RFC2616 is designed primarily for ``transparent'' or ``selected'' proxies, but using a DNS resolution on the destination host seems to break the layered structure of the IP stack. In this case, packets that I've (layer 3) addressed to a specific host never get there, because (layer 4) they're being directed to another machine based on port, and the other machine (the cache) is routine them based on a name (layer 7) contained in the packet payload.
That is acceptable behavior for a proxy to which I'm explicitly routing my http requests, but not for a router down which I'm sending port-80 IP packets.
What I'm complaining about is that the proxy has a bad failure mode: when the usual resolution method fails, the proxy ought at the very least to fail softly and route the packets like a real router should Unfortunately, it doesn't -- it generates an error message instead.
>First of all, the phrase "routing" is a >misnomer. Web caching is something that happens >on the application layer of the OSI model, >layer 7, whereas "routing" refers to layer 3, >which supplies IP routing for the TCP/IP >protocol suite. What's broken is their caching, >their cache server, or their proxying; pick a >term.
Thanks for the helpful comment!
What I'm complaining about is that their router (layer 3) routes all port-80 packets to a cache server that looks at the payload only (layer 7) and not at the header at all. In short, they're not routing correctly; they've broken the layered structure of the protocol.
Very good suggestions, and I'm planning on doing steps (1) through (3).
As far as a local proxy: that won't work with virtual hosting in a non-ICANN name space. The immediate problem is that I can't retrieve non-ICANN web pages because the proxy tries to resolve the non-ICANN name in the payload, using ICANN DNS. I can always ask for numeric addresses, but virtual hosting (where a server gives you different pages depending on the name you ask for) is widespread enough that there are many web pages I can't retrieve even in principle.
Subject to the ``Skating Force'' of LP days
on
Perpetual Skislope
·
· Score: 5, Interesting
Anyone here actually old enough to remember LPS and the skating force? Skiers would be drawn toward the middle of the disk and would have to be
constantly turning outward to avoid hitting the spindle at the center of the terrain. Odd, that.
If you've never operated an LP phonograph -- the skating force is due to the differential friction on opposite sides of the needle on a phonograph, and tends to draw the needle inward toward the center of the record. It's large enough to cause a needle to skip, bump bump bump, right over the grooves unless a counteracting force is applied. Low-end turntables used springs to pull the needle outward and combat the skating force; high-end turntables used little weights with little mechanical linkages that were designed to match the
changes in the skating force with radius.
You can see skating force in action at the bottom of a teacup if there are a few tea leaves floating around down there at the bottom. The tea leaves (after they're waterlogged) sink, so spinning the tea in the teacup "ought" to make them fly outward in the local gravity field. But in fact, tea leaves at the bottom of the cup tend to pile up in the center (when you spin the tea). Counter-intuitive and mysterious, until you
realize that the leaves are also dragging on the bottom of the cup and therefore are subject to the skating force.
Gotcha. I don't mind claims that it's a cool
new way of making a head-mounted display, that's fine. But it's not nearly as cool as the sight would have you believe (or as most Slashdotters seem to believe). Imagine having to hold your cell phone RIGHT UP TO YOUR EYE to see the display in it. Yuck.
But the "concepts" PDF on the site shows lots of applications where the graphics just hang in space, near a small-looking projector. (Check out the one with the Cessna dashboard, or the sports-car ``concept'' image). That's misleading and physically impossible, and that's what I'm complaining about.
Slashdot Mirror
First thing I thought was, "hmmm... haven't we understood hydrochloric acid for a long time now?"
If you find yourself behind a shitpile^H^H^H^H^H^H^H^Hrouting HTTP proxy, the most straightforward solution is to install your own proxy that translates all requests to numeric-IP CONNECT requests. Then your proxy can talk via direct TCP connection to the original host.
I mentioned a short hack I used to test some ideas, but a full-featured local proxy needs more than that. Squid is a good starting point and I plan to cruft together an appropriate mod to it soon.
It's not so bad. Figure 2e6 lbf acting on the thing, and a 6" thick window to take up the strain. That's about 1.5 kpsi overall stress inside the window, plus a bit for imperfections in the construction. Add some safety margin and you can figure on around 5 kpsi of stress.
I'm not sure the strength of their window material but Kevlar can be made clear and has a strength of around 250 kpsi.
My plane gets 18 mpg at 150 mph.
Yes.
In California, at least, 10 degrees ATDC timing at idle is mandatory on post-1972 cars. That limits nitrogen oxide emissions in high compression engines by reducing peak combustion temperature.
I ran into that problem when I was still a VW grease monkey -- 1973 VW engines hardly qualify as high compression, and emit very little nitrogen oxide. The ATDC timing causes extra heating and lower efficiency at idle and low speed. EGR works on the same principle -- lower peak combustion temperature to reduce NO and N2O production (at the cost of slightly more CO and HC emissions, and more fuel consumed).
The law was written that way because it's hard to detect the nitrogen compounds cheaply in the exhaust, so it's easier to mandate the most common solution than it is to mandate testable standards.
If you like sausage and respect the law, you shouldn't look at how either is made. (Mark Twain).
Us too. I live and work in Colorado, and was astonished to find that (at least for October and November) packets between work and home were being routed through Arlington. Slowed everything down: instead of ~100ms latency (already pretty slow, through town) I was experiencing more like 400ms.
Of course, now they know that I know. uh oh.
... in one of the "future history" series. I forget which one -- but it was one of the lunar stories. The Rolling Stones? The Moon is a Harsh Mistress? The Menace from Earth? I remember one of his Knockout-Nobel-Laureate-Who-Just-Wants-To-Make-Bab ies women talking about it...
I mean, come on, people! Checking all contingencies is something you're supposed to learn in your first programming course. Especially in a hostile computing environment (spy vs. counterspy) you have to write airtight code or you'll get got.
So far so good.
The problem is that the ``violent 'paradigm shift' '' to a new theory doesn't change the body of existing experiments, or the rigor of the scientific method. Spinning superconductors won't start to fly just because everyone believes they should. Any new idea really does have to fit within the existing framework of experimental data, and anyone proposing a new idea has an obligation to make certain ancillary predictions. (For example, in his initial general relativity paper, Einstein did this, citing a few famous predictions of wide-ranging experiments that should conflict with then-current theory if his ideas were right). An experimentalist should at least think through his effect enough to have an idea how it should be reflected in other experiments besides his own.
In this case, there's a tremendous amount of evidence against the kinds of claims that crackpots and fringe scientists investigate. For example, if spinning objects really fast makes a gravitational vortex or something, then why dont' lab technicians notice it when they set their coffee on the ultracentrifuge?. Unfortunately, crackpots are much more common than true ``fringe scientists'' who go around investigating, well, `paranormal' phenomena. Why? Because 99.99% most `paranormal' phenomena turn out to be made up or the result of experimental error, and who wants to spend years in combat with some stupid crackpot who doesn't have the insight or the will to do careful experiments? Sure, the 0.01% case might be great -- but you only have one life to live. Why spend it arguing with crackpots, when the simple expedient of waiting for one of them to succeed dramatically (by, say, opening up shop and selling levitating cars) works just as well from a scientific point of view?
Yup, that's pretty cool. I like that (for example) I can get https:// requests through to any host I want, simply because the encryption enforces the layering of the IP stack. Unfortunately, almost nobody actually publishes the same pages through http: and https:. The encryption layer just uses too many resources in high-bandwidth applications. I wonder if that'll ever change?
What prompted me to write to Slashdot was the thought that perhaps others are having similar difficulties. If I were dealing with a simple proxy router (such as you apparently helped design the standard for) then my solution would be simple: direct ICANN URLs through the proxy and other ones elsewhere. But I really have no choice but to use the proxy, because my ISP is intercepting all of my port 80 packets.
In a more general sense, OpenNIC isn't the only case where the ability to access http directly is important. Other cases include testing virtual hosting or accessing hollow-tree virtual hosts within a machine with a known IP number.
You talk about trying to coerce the rest of the world into supporting an alternate DNS. That's not really necessary: strict IP routing already supports alternate DNS roots -- the layered structure of (most of) the protocols keeps your choice of DNS root from interfering with where your packets go. That separability is a Good Thing, and part of a whole suite of related Good Things that make the IP concept work.
Well, actually, no, what I'm buying is internet routing, using the internet protocol. That protocol is pretty well laid out in the RFCs. There's no special reason why this cache has to be the way it is. Other superior solutions exist for free -- some of the other answers in this very thread point out proxy caches that do the Right Thing, and as many have pointed out this behavior is totally to be expected in a manual (and hence avoidable) caching proxy.
But caching proxies exist on a different layer of the internet protocol than do routers. Mixing those two functions is bound to break things. I understand the difficulty of getting users to use a proxy -- many don't understand that it will actually benefit them in most cases, for example. But caching proxies aren't routers, and shouldn't be.
What's especially sad is that, by not planning for the failed-DNS-lookup contingency, whoever wrote their cache is breaking a whole suite of things that don't have to be broken, even with this flawed solution. Lots of effort, including this whole slashdot thread, some angst on the part of their technical staff (having to deal with me) and my own minor frustration could all have been avoided if perhaps two to ten lines of code (the contingency code in the proxy) were better thought out.
Okay, cool.
The problem I'm having is that, because I have to rely on the proxy's DNS to resolve a web hostname, I can't get certain HTTP requests to certain hosts.
The specific example that I mentioned is the "http://www.dev.null" URL, but there are loads of other examples. In particular, "http://www.dev.null" is a completely valid URL that points to, well, the dev.null site. To resolve the URL into a host IP number to connect to, you have to be using the openNIC DNS tree (.null isn't supported by ICANN).
So far so good. I can connect my web client, for example, to "https://www.dev.null" and get the secure pages for the dev.null site. But the main pages at "http://www.dev.null" are invisible to me -- the most I can get is an error message from my ISP's ``transparent'' caching proxy.
The reason for the difference is that my home box uses OpenNIC -- the https: request gets resolved into an IP number and my local host just telnets to that host on the appropriate port and issues an encrypted HTTP request.
The http: request fails because, although my host resolves the host-part of the URL into an IP number, that number is ignored by my ISP's ``transparent'' proxy. All port-80 packets emanating from my home network get intercepted and fed into the proxy. The problem is that the proxy attempts to resolve the URL using ICANN DNS -- so it's unable to identify an IP number to associate with the URL. It's apparently poorly coded, in that it doesn't know enough to try the IP destination address on the incoming stream that it's intercepting.
My ISP reacted sort-of the way you just did: "well, if you're not using DNS, you're out of luck, pal." But that's Wrong. If they had an explicit proxy server and I had some choice about what happened to my packets, that would be OK -- but they don't. All of my port-80 packets are being intercepted by a piece of buggy 'ware. That (A) breaks the layered structure of the IP protocol, and (B) prevents me from accessing big chunks of webspace that I'd like to use.
If you administer a similar transparent proxy setup, I encourage you to ensure that it falls back gracefully to being a router if it can't make sense of the http: requests. Apparently the most popular Cisco solutions and Linux ipchains router/proxies are pretty easy to configure correctly (according to some of the other answers).
Cheers,
Craig
Thanks. Unfortunately, this doesn't work well if the site (like most sites these days) uses name-based virtual hosting. For a given host, you might get several different web pages depending on what you put in the host part of the GET URI.
*sigh*.
That's the problem with a lot of software "solutions": people don't think through all the worst case scenarios, just the main one, and the tool breaks when it's used in a way they didn't expect.
I'm not familiar enough with the ins and outs of cache design to know whether RFC2616 is designed primarily for ``transparent'' or ``selected'' proxies, but using a DNS resolution on the destination host seems to break the layered structure of the IP stack. In this case, packets that I've (layer 3) addressed to a specific host never get there, because (layer 4) they're being directed to another machine based on port, and the other machine (the cache) is routine them based on a name (layer 7) contained in the packet payload.
That is acceptable behavior for a proxy to which I'm explicitly routing my http requests, but not for a router down which I'm sending port-80 IP packets.
What I'm complaining about is that the proxy has a bad failure mode: when the usual resolution method fails, the proxy ought at the very least to fail softly and route the packets like a real router should Unfortunately, it doesn't -- it generates an error message instead.
>First of all, the phrase "routing" is a
>misnomer. Web caching is something that happens
>on the application layer of the OSI model,
>layer 7, whereas "routing" refers to layer 3,
>which supplies IP routing for the TCP/IP
>protocol suite. What's broken is their caching,
>their cache server, or their proxying; pick a
>term.
Thanks for the helpful comment!
What I'm complaining about is that their router (layer 3) routes all port-80 packets to a cache server that looks at the payload only (layer 7) and not at the header at all. In short, they're not routing correctly; they've broken the layered structure of the protocol.
Very good suggestions, and I'm planning on doing steps (1) through (3).
As far as a local proxy: that won't work with virtual hosting in a non-ICANN name space. The immediate problem is that I can't retrieve non-ICANN web pages because the proxy tries to resolve the non-ICANN name in the payload, using ICANN DNS. I can always ask for numeric addresses, but virtual hosting (where a server gives you different pages depending on the name you ask for) is widespread enough that there are many web pages I can't retrieve even in principle.
Cheers,
Craig
Actually, that's not true. Often you want to send an arbitrary HTTP request to an arbitrary host. See my example in the article.
Cheers,
Craig
OpenNIC is available now and is a strict superset of the ICANN namespace.
They predicted flying cars for years, too.
If you've never operated an LP phonograph -- the skating force is due to the differential friction on opposite sides of the needle on a phonograph, and tends to draw the needle inward toward the center of the record. It's large enough to cause a needle to skip, bump bump bump, right over the grooves unless a counteracting force is applied. Low-end turntables used springs to pull the needle outward and combat the skating force; high-end turntables used little weights with little mechanical linkages that were designed to match the changes in the skating force with radius.
You can see skating force in action at the bottom of a teacup if there are a few tea leaves floating around down there at the bottom. The tea leaves (after they're waterlogged) sink, so spinning the tea in the teacup "ought" to make them fly outward in the local gravity field. But in fact, tea leaves at the bottom of the cup tend to pile up in the center (when you spin the tea). Counter-intuitive and mysterious, until you realize that the leaves are also dragging on the bottom of the cup and therefore are subject to the skating force.
Cool. Thanks for the info.
Cheers,
Craig
But the "concepts" PDF on the site shows lots of applications where the graphics just hang in space, near a small-looking projector. (Check out the one with the Cessna dashboard, or the sports-car ``concept'' image). That's misleading and physically impossible, and that's what I'm complaining about.