Dealing With ISPs That Use NXDomain Redirection?

Vrtigo1 writes "I work for a small company that has about 50 staff on the road relying on VPN back to our office at any given time. Many ISPs have implemented NXDomain redirection services that hijack DNS traffic to show you sponsored links and other related ads when you mistype a domain name. These services are incompatible with most VPN software, since they prevent the computer from resolving internal hostnames. Large ISPs typically provide an opt-out on their sponsored links page that immediately opts you out of the DNS redirection, but I've noticed that some smaller ISPs and CLECs have opt-out links that don't actually appear to do anything. I don't have a good solution for employees using these ISPs, and our employees are getting frustrated because the problem is becoming more prevalent and we can't fix it for them. I've tried calling a few of these smaller ISPs for help, but it's been like talking to a wall. Manually changing DNS servers works temporarily, but the user can't resolve internal hostnames when they connect to the office LAN again. Have you had to deal with ISPs using non-standard DNS servers? What is your solution?"

Comments

This is an easy one.

[snarfies]

If the small local ISP is screwing up, and refuses to respond in any useful way despite your best repeated efforts, it sounds like its time to take your business elsewhere, maybe to one of those large ISPs you mentioned. And make sure you tell them WHY. Who know, maybe the threat alone will be enough to get them to make a sudden change in policy for you, with a month or two of free service to boot.

Re:This is an easy one.

[internerdj]

This guy sounds like a manager or IT worker who is having problems with his employees connecting to the work VPN. He isn't the one paying the bill(directly at least), so he doesn't even have the clout of a paying customer...

Re:This is an easy one.

[TheLink]

Actually, the VPN config is insecure (screwed up?) - when you are using the VPN the DNS requests should be going through the VPN tunnel, and not in plaintext to the ISP.

Re:This is an easy one.

[IceCreamGuy]

You are referring to what is known as "Split Tunneling;" which is a legitimate, albeit less secure, VPN configuration. Basically when split tunneling is enabled the client workstation's default gateway is still it's local gateway and DNS requests get routed by the client to the appropriate DNS server, whereas in a non-split tunnel the default gateway is the remote gateway (which obviously has no way of routing to the local network) and all DNS requests go encrypted through that. There are several reasons someone would want to do this:

• You need people to access their local printers/network resources and don't have some kind of pass-through ability
• You have limited bandwidth at your remote site and cannot handle the Internet usage that would be NATed through
• Your gateway does not support NAT on VPN tunnels and your clients need Internet access
• You don't realize what you're doing

Either way, what I do when I have some kind of weird situation where a user needs to change their TCP/IP config routinely is just put a couple shortcuts with pretty icons on their desktop that point to batch scripts that run a netsh script. You should be able to completely change an IP configuration on a Windows box with this utility, the user just runs "home.bat" when they're home and then "office.bat" when in the office. A Google for "netsh exec" should give enough info to get started.

Provide your own DNS?

[QuantumRiff]

Last time I setup a VPN, was with a Cisco PIX firewall, (its been awhile) but there was a spot to specify which DNS servers to use when connected to the VPN. I had specified that when connected, they would use our DNS, since they otherwise couldn't resolve \\file-server\share or whatever..

Re:Provide your own DNS?

[nine-times]

Yeah, honestly I'm a little confused by the question. If you want to use DNS to connect to internal servers via VPN, then don't you want to route your DNS traffic through the tunnel to use internal DNS servers? And once you're doing that, how could the ISP possibly hijack that DNS traffic? It's encrypted.

Re:Provide your own DNS?

[Bandman]

You're right. It all boils down to misconfigured VPN

Change VPN settings . . .

[val123456]

to force use of internal DNS servers while connected.

Done.

Use Full Tunnels

[Bandman]

If you're splitting your connection between a VPN tunnel and a non-VPN protected internet connection, you're a security risk to your infrastructure.

Have your administrator configure full tunnel support where ALL of your traffic goes through the encrypted tunnel. That solves a security problem AND it fixes your DNS problem because you don't use your local internet provider's DNS servers.

Re:Use Full Tunnels

[L0stm4n]

This is called split tunneling. If he disables split tunneling and specifies the DNS servers in the VPN config his problems would go away.

His users however would tunnel all their traffic through the corporate lan while connected so you may need to setup some kind of filtering or route the traffic through whatever filters you already have. Otherwise these remote workers in hotel rooms will be pulling buckets-o-pr0n through your corp network.

What small ISPs?

[bzzfzz]

There are still small ISPs left where you live?

Mod parents up

[adolf]

Mod parents up, please.

And then we can all go home. This is an easy problem to solve once you see it from the right angle, and that angle is described above.

could someone explain what the issue is here?

[goombah99]

This guy sounds like a manager or IT worker who is having problems with his employees connecting to the work VPN.

it sounds more like he has not stated the problem correctly.

how is it possible that a VPN connection is doing DNS to an external name server? Should not every internet request flow over the vpn from the client to the server. once it reaches the internal vpn server the server should know how to route the internal addresses and for external addresses it could use an external domain name server. the problem described seems like it should not exist. what am I missing?

Re:could someone explain what the issue is here?

[omnichad]

Unless it's set to send ALL traffic over the VPN, you have to resolve the hostname in order to decide if the DNS name is on the VPN or on the Internet.

Even if all traffic goes down the VPN wire, it's probably making those requests to the same DNS servers OVER the VPN. Bust since it's still the same DNS servers, it still gets the same results.

The IT guy would have to intercept all DNS requests over the VPN and proxy them to his own DNS server. That's not a bad answer. Too bad I'm buried in the middle of this thread.

Re:could someone explain what the issue is here?

[pthisis]

Some VPNs only route traffic meant for certain destinations through the VPN as one network interface and allow traffic to the public Internet use the actual established connection.

They should be checking the internal DNS servers first (which should not promulgate requests up to public servers), and then the public servers.

Doing in the other order sends internal information (server names) over the public network.

Re:could someone explain what the issue is here?

[Intron]

Depends on the VPN setup. I don't want my VPN clients sending all of their web browsing through the VPN and then back out through my firewall. I only want the traffic destined for my internal network. On their end, they should have a route table that sends traffic for me through the VPN and everything else through their normal ISP. I can support a lot more users that way.

Re:could someone explain what the issue is here?

[Andy Dodd]

That's a hell of a security risk, having a client connected to both your internal network and external networks simultaneously.

Every corporate VPN I have ever used has, as part of its function, disabled all network interfaces other than the one it was using once a connection was established. In addition it would prevent any traffic from going through the "normal" connection. The idea was that a machine should never have connectivity to both the internal network and the outside world simultaneously.

The article poster doesn't need to fix their users' ISPs, they need to fix a horrifically broken and insecure VPN system.

Re:could someone explain what the issue is here?

[Sparr0]

I have never seen that enforced, and only twice ever as the default setting. It is a client-side configuration option in most VPN software (Cisco, SecuRemote, most Linux VPN clients).

You want VPN users to stream video or download game patches or do other non-business-related bandwidth intensive operations over the VPN, when they have a perfectly (ha!) good internet connection locally? I hope you have a REALLY big network pipe.

Re:could someone explain what the issue is here?

[mellon]

I'm not sure what your threat model is, but I suspect you are claiming one of two things: either that the VPN node might act as a router, forwarding packets around your firewall, or that the VPN node might be compromised and used as a stepping-stone onto your network.

In the case of the router vulnerability, this is something that you can control on the corporate side of things by simply not accepting packets down the VPN tunnel that don't come from the IP address that's the far endpoint of that tunnel. I'm not a VPN expert, but I would be surprised if this isn't how your VPN is configured by default.

In the case of the stepping stone, this is a fairly weak threat model, for two reasons. First, if your machine has been rooted, there's a good chance that it will phone home out through your firewall even if you route all internet access through the VPN. So it will be a stepping stone to your network anyway.

Second, if your machine has been rooted, and is running any sort of virus platform, it's going to try to infect machines on your network even if it doesn't have a link to the outside world. If you are genuinely concerned about threats originating on employee laptops, you shouldn't allow them to VPN into your network at all.

So the point is that forcing the VPN'd node to access the internet through your site is probably going to be a big inconvenience for your users (the kind of inconvenience they will hack around, possibly making you even more vulnerable) and it's not going to buy you any meaningful security.

Firewalls are great for slowing the spread of infection, and raising the cost of attacking you, but you really do need to secure every node as well, and if someone really wants to get past your firewall, and is willing to expend substantial effort to do so, you probably won't stop them without much sterner measures than the one you're advocating.

Re:could someone explain what the issue is here?

[Sparr0]

And this is an excellent plan for convincing your users to only connect to the VPN occasionally. Good if you want to maximize security. Bad if you want to maximize productivity.

MOD PARENT UP: Re:could someone explain what th...

[HappyDrgn]

This is in fact why NXDomain breaks things in the way the poster describes, however, unless you're the kind of employer who wants to see EVERYTHING your subordinates are doing it's not actually the best practice to filter everything through the VPN.

Filtering everything through their VPN increases overall costs in bandwidth and hardware as Intron indicated. These are very real, very costly expenses that many employers overlook when implementing broad policies... and it's a fantastic point you raised that all too many companies forget.

Why should my connection to slashdot.org, for example, be secure on the company VPN? My ssh and nfs connections have very real reasons to be secure however!! On the other hand you could fix this by filtering DNS traffic through the VPN, but not web traffic. The cost of DNS traffic is marginal comparatively to other services, but the benefit for companies facing these specific issues is obvious.