Slashdot Mirror

← Back to Stories (view on slashdot.org)

Ask Slashdot: VPN Setup To Improve Latency Over Multiple Connections?

Posted by timothy on from the so-you-favor-internet-fast-lanes dept.

blogologue writes I've been playing Battlefield for some time now, and having a good ping there is important for a good gaming experience. Now I'm in the situation where I have mobile internet access from two telecom companies, and neither of those connections are stable enough to play games on, the odd ping in hundreds of milliseconds throws everything off. How can I setup a Windows client (my PC) and a Linux server (in a datacenter, connected to the internet) so that the same TCP and UDP traffic goes over both links, and the fastest packet on either link 'wins' and the other is discarded? (Have your own question for the teeming masses? Ask away — be sure to include appropriate detail and context — via the Slashdot submission form.)

33 of 174 comments (clear)

  1. What makes you think by msobkow · · Score: 2, Interesting

    What makes you think the servers can deal with multiple copies of data sanely?

    --
    I do not fail; I succeed at finding out what does not work.
    1. Re:What makes you think by Ungrounded+Lightning · · Score: 4, Informative

      Doesn't TCP require it come back on the path too?

      Absolutely not. Nor does it expect that to usually happen. The routes in opposite directions are often different. (For starters, they're based on the local knowledge of the routers at opposite ends of the path, which are typically familiar with their neighborhood and may have no clue about what things are like near the other end.)

      The routes of diffetrent packets in the same direction are often different, too (like for load-balancing by throwing alternate packets down two slower links to get an effectively faster link). Every packet is potentially routed differently (though routing protocols like label switchingmay often set up connection-like shortcuts that make consecitve packets take the same path - to speed things up).

      What matters is just that they get to the same ENDPOINT. Some may be silently lost. Some may be duplicated. Some may arrive out of order (like when a route changes and later packets get there faster).

      It's been like this since IP, UDP, and TCP were invented. It was a core principle of their invention.

      = = = =

      Having said that:

      Deviation (other than packet drops) from simple first-in-first-out packet flow tend to be rare. So not all servers and/or clients handle them well. (TCP will sort out missing and misordered packets on the receiving end - sometimes at substantial cost in buffering and latency. UDP will not - for simplicity, speed, and for when occasional lost packets are less of a problem than high latency and occasional long delays. So if the server and/or client can't handle transmission problems well, performance may suffer or functionality simply fail.)

      Many networking company customers of high-speed router makers make the additional requirement that a stream of packets coming in one particluar interface from one particular source and going out another particular interface to a particular destination are not reordered. That's a pain when the router's guts are a sea of little processors each handling packets individually, so additional special purpose hardware may be added to track packet order and insure things don't get reordered between input and output queues.

      --
      Bantam Dominique roosters crow a four-note song. Once you've heard it as "Happy BIRTHday" you can't NOT hear it that way
    2. Re:What makes you think by Comen · · Score: 2

      It does not require the same path through the internet, but you wont be able to use one ISP's connection from packets coming back from the source of other connections IP address. you have to use one connection or the other, you could change your connections and restart you game, but the game will not let you change IP's during gameplay.
      This whole idea had lots of issues anyway, if both connections suck, you should just get a good connections, if you want to play games, a wireless internet solution is just not right for you.
      Pay for a good internet connection for gaming, I hear kids all day on server complain about their parents got on the internet and are watching Netflix and their ping went to shit, well the parents pay for the connection, so maybe you need to move a lawn once a month and just buy your own.
      I know it is possible to have 2 cable modem in one house, and each will not effect the others bandwidth etc..

    3. Re:What makes you think by msobkow · · Score: 4, Informative

      TCP can deal with duplicate packets from the same endpoints. Sending duplicate packets over two entirely seperate routes would require that the server be able to deal with demultiplexing the requests. I seriously, seriously doubt that any game servers are set up to do that. As far as the game server would be concerned, it's two seperate clients for the same account connected.

      --
      I do not fail; I succeed at finding out what does not work.
    4. Re:What makes you think by msobkow · · Score: 2

      Hmm. I just caught the part about the Linux server at the data center doing the demultiplexing.

      I suppose, at least in theory, you could go on the assumption that both channels are always sending the same data, and have them forward the request appropriately, cache the message block, and do a comparison on all message blocks incoming over both channels before forwarding one to eliminate the duplicates.

      You'd then have to do the same thing on the Windows "client" box at home.

      Quite frankly, I can't see how it wouldn't be easier to just get a landline connection. You certainly couldn't rely on "normal" multipath software solutions to do this -- they're designed to switch back and forth between a primary and a failover route, not transmit over both routes at the same time.

      You'd also have to become very familiar with the internal driver coding for both Windows and Linux. Certainly not a project for a "new to programming" person who wouldn't think up the simple solution of keeping a queue/cache of requests to check for duplicates in the first place.

      --
      I do not fail; I succeed at finding out what does not work.
  2. no by Anonymous Coward · · Score: 3, Insightful

    A VPN or any kind of encapsulated network traffic will only add to the latency.

    1. Re:no by Bengie · · Score: 2

      The quality of the route matters, also, packet loss. If he has something that essentially duplicates the VPN traffic to the same VPN server, then the VPN server just ignores the second packet, packet-loss would be less likely to happen, and it also helps getting the minimum latency of the two high jitter connections.

      Many people have such crappy ISPs or connection signal strength that a round-about route is still better.

  3. What makes you think by psmears · · Score: 2

    Well, for one thing the servers have to cope with that, because internet service in general doesn't guarantee that packets don't get duplicated. But in what OP is suggesting, the servers won't see duplicate packets - the idea is that every packet gets sent out of both internet connections to one, private, hosted VPN server - and that server runs a service that forwards whichever copy of the packet arrives first to the "real" destination (and discards the losing packet) - so the game server will only get one copy of each packet in the stream.

    --
    Need to type accents and special characters in Windows? Use FrKeys
  4. local LAN beats remote server by Anonymous Coward · · Score: 4, Insightful

    This is why local LAN play with your buddies beats some unknown remote server. Plus, then you can keep playing after the central server is taken offline.

    What's that? Your favorite game doesn't support LAN play? Well, better support the ones that do, and not support the ones that don't, if you want this option to remain viable into the future.

  5. no by psmears · · Score: 4, Insightful

    That's true, but it seems that the real problem the OP is trying to solve is huge variance in the latency (i.e. jitter) - that is, the idea is to trade a very small amount of extra latency for the latency being much more consistent (without the massive spikes currently being seen). I'm not sure how well it would work in practice (e.g. if some of the spikes are due to local RF interference, it's possible they will affect both connections at the same time), but there's potential at least for a much smoother gaming experience.

    --
    Need to type accents and special characters in Windows? Use FrKeys
  6. Your linux server won't help you. by detritus. · · Score: 2

    Your latency and unreliability comes from your mobile links. Get better providers or find a different lower-latency game to play.

  7. Way more work than you would want by shrikel · · Score: 4, Interesting

    How can I...

    Simple. Just write a custom driver on both the Windows and Linux boxes to handle both ends as described (you'll want the traffic duplicated both ways, I'd imagine, since you're not just dealing with one-way communication here).

    I doubt there's anything off the shelf that will handle what you want. Sounds like a fun project... but don't undertake this unless you think the project will be as fun to work on as actually playing your game. And be prepared to drop a hundred hours into it (depending on your coding abilities and familiarity with the associated APIs).

    --
    Any sufficiently simple magic can be passed off as mere advanced technology.
    1. Re:Way more work than you would want by agizis · · Score: 4, Interesting

      In fact, this does exist off the shelf already, it's called Speedify, and it's a VPN that uses all of your connections together: http://speedify.com/

  8. Off the point by blogologue · · Score: 2

    A couple of milliseconds extra for all traffic doesn't matter. The point is to avoid those short spikes in network latency that get me kicked from online servers..

    --
    The blogologue
  9. Does nobody understand the question? by Anonymous Coward · · Score: 5, Informative

    Sounds like an interesting problem; I check the comments to see what solutions to the specific problem laid out might exist. Instead, the comments show varying levels of misunderstanding the question and/or the proposed solution.

    The proposed solution is simple:

    1. Client duplicates packets over two mobile links to an intermediate, user-controlled server.
    2. This server sorts things out and discards the losing packet, and forwards the winner on to the real gaming server.

    Both client/intermediate server are under the control of the user, with two possible links. The communication protocol between these two nodes can be user-defined to anything. The question was how to configure this.

    "How will the server deal with duplicate copies?" Duh, the gaming server won't. That's what the intermediate server is for. Did you read the OP?
    "A VPN will add to the latency." Yes, but that wasn't the problem. The problem is random jitter on one of the links.
    "Local LAN play is better!" Well yes, but that's not remotely related to the problem. Maybe he doesn't have anybody local to play with regularly?
    "Use this exotic hardware solution." Why, if the problem can be solved for free with software?
    "Your latency comes from your mobile links." Duh, but he already measured the main problem to be random jitter. Why not comment on the proposed solution?

    The only concern I read that is accurate was that an RF disturbance could interrupt both links.

  10. Latency varies when protocols switch by npetrov · · Score: 2

    You may be able to check if your latency varies due to mobile protocol switches to lower power due to less activity.

    As a first test, try:

    ping www.google -t -l 1000
    vs
    ping www.google -t

    If the 1000 byte ping has more consistent times than the standard pings - that's the issue and you may be able to find a minimum payload size which makes the connection more consistent without switching.

  11. "No." by Spazmania · · Score: 2, Interesting

    The answer is: no, this can not be done with current protocols.

    In theory with new protocols that your game doesn't support, sure. But only the end-to-end machines understand latency and jitter (your problem is jitter) so a middlebox won't help you.

    --
    Moderating "-1, Disagree" is simple censorship. Have the guts to post your opinion.
  12. mptcp (multipath tcp) is one solution by LukeCrawford · · Score: 4, Interesting

    http://www.multipath-tcp.org/

    of course, this requires the other end to support it, which it probably doesn't.

    I've also considered selling "multipath vpn" service... the idea being that people with two DSL providers (and one dsl and one cable) provider would setup their gateway (or use a linux box that I sell them and manage) to send all packets out VPNs on both connections, to my own vpn endpoint in a datacenter. The idea being that then my server on the other end of that connection would take the first packet and send it on to it's destination. Assuming that my datacenter has a good connection, you would suffer less packet loss, and less latency.

    My solution here would solve the problem if the problem is latency/loss on your last mile connection. It would not help at all if the problem was further along the connection, while multipath-tcp would

    1. Re:mptcp (multipath tcp) is one solution by agizis · · Score: 2

      A MPTCP VPN would not work in the real world. When you tunnel TCP through it, you end out having to send ACKs for the ACKs. The end result is that the effects of even a tiny bit of packet loss is a performance meltdown: http://sites.inka.de/~W1011/de... To build Speedify, we needed to implement a new multipath protocol over UDP. But that let us do clever stuff with NACKing and retransmitting lost packets before TCP ever noticed, and we were actually able to reduce the effect of loss: http://speedify.com/blog/speed...

  13. Insightful jokes by gringer · · Score: 5, Funny

    Hi, I'd like to hear a TCP joke
    Hello, would you like to hear a TCP joke?
    Yes, I'd like to hear a TCP joke
    Okay, I'll tell you a TCP joke
    Okay, I'm ready to hear a TCP joke
    Okay, I'm about to send a TCP joke, that'll last for 10 seconds. It has two characters, it does not have a setting, it'll end with a punchline.
    Okay, I'll get your TCP joke, that'll last for 10 seconds. It has two characters, it does not have a setting, it'll end with a punchline.
    I'm sorry, your connection has timed out

    On the other hand, I could successfully tell you an entire UDP joke, but you might not get it.

    --
    Ask me about repetitive DNA
    1. Re:Insightful jokes by Mateorabi · · Score: 2
      I'd tell you a joke about TCP but I'd have to keep repeating it until you got it.

      I'd tell you a joke about UDP but you might not get it.

      --
      "You saved 1968." - Ms. Valerie Pringle to the crew of Apollo 8

  14. To clear up confusion... by ERJ · · Score: 2

    So, just to clarify I believe what the poster wants to do is this:

    |||| Gaming Client PC ||||
    |||| Local Windows Box ||||
    |||| Internet 1 |||| Internet 2 ||||
    |||| Hosted Linux Server ||||
    |||| Gaming Server ||||

    Local Windows Box acts as a router and duplicates all inbound traffic sending it out box Internet 1 and Internet 2. Hosted Linux Box receives traffic, picks whatever packet arrives first and forwards it and throws away the slower duplicate when it comes it.

    It is an interesting idea. As far as I am aware routing protocols only do best route and fail over but I am not aware of any that always sends both routes.

  15. I have no problem at all imagining that. by Ungrounded+Lightning · · Score: 4, Informative

    I find it hard to imagine that it would (at least routinely) be faster than using his current wireless setup to route his traffic from his desktop ... through his cable modem ... through his ISP ... through a remote datacenter (somewhere) ... to the Battlefield servers

    I have no problem at all believing that. The OP says he is using two MOBILE access devices from two (wireless) carriers and is (if I read him right) experiencing substantial intermittent (but separate) delay and/or drop events in both of them. If he throws each packet down both of them and the first one to arrive at the data center gets to the game server, the packets that are lost or delayed on BOTH paths will be very much rarer and his gaming experience will be substantially improved.

    Yes, he'll get a little extra latency on the fast packets - which is most of them. But server farms generally have fat and blazingly fast backbone connections, so it shouldn't be a lot added. A small price to pay to make almost ALL packets arrive reasonably quickly and almost NONE experience big delays or loss.

    --
    Bantam Dominique roosters crow a four-note song. Once you've heard it as "Happy BIRTHday" you can't NOT hear it that way
  16. Actually, it's easy. by Ungrounded+Lightning · · Score: 2

    This isn't possible, nor should it be.

    Actually, it's almost trivial.

    I don't know if there's something already available and free, so here's how I'd build it.

    It'd startt with OpenVPN. (Mosly because it's the only hackable VPN I'm familiar with that's currently supported.)

    I'd first take advantage of the fact that servers and clients are SUPPOSED to be able to handle duplicated and reordered packets and do it the simple way: Just hack it to throw each packet down both pipes, and at the receiving end just forward both copies.

    Then, if the server and/or the client DON'T handle the duplication gracefully, I'd add sequence numbering to the VPN wrapping (if it isn't there already) and a mechanism at the receiving end to keep a small amount of history of what has gone out and drop the slower copy (if it arrives before the history of its other copy has timed out).

    Data structure might be a small tree or heap of ranges-of-packets-that-have-left (pruned to drop older stuff when getting too nodefull or near sequence-number-wrap-around due to good packet weather), or maybe a rotation of three hash tables - "now", "recent", and "being cleared / cleared and waiting for rotation" - with the first two in use.

    --
    Bantam Dominique roosters crow a four-note song. Once you've heard it as "Happy BIRTHday" you can't NOT hear it that way
    1. Re:Actually, it's easy. by JackDW · · Score: 2

      Do you know, you're the first person in this topic to actually answer the question? Most others missed the VPN part.

      OpenVPN already knows how to discard duplicates and retransmit lost packets. It's a lovely way to build a semi-reliable network on top of an unreliable one, and very hackable.

      The questioner only needs to modify OpenVPN (on his PC) to send its UDP packets via two different routes. He should configure his VPS to have two public IP addresses, with OpenVPN (server-side) bound to both of them, and then manually adjust the routing table on his PC to force the use of a specific route for each of those two IP addresses. The hard bit (and it's not really that hard) is making OpenVPN (on the PC) send each packet twice to two different IP addresses, which would require modifications to the source code and some familiarity with the sockets API.

      I think it would work, not just for Battlefield but for anything. And it sounds like fun.

      --
      You're an immobile computer, remember?
  17. LACP by Anonymice · · Score: 2

    Load balancing & bonding over multiple NICs. Isn't this what LACP was made for?
    No idea if there's anything available for Windows, but chuck a Linux VM on it to act as a virtual router & presto!

    How well it works would depend on the LACP stack's ability to handle the issues presented by wireless modems. It works great in a server environment.

  18. Here's an idea by 93+Escort+Wagon · · Score: 3

    Take the money you were going to spend on hosting a Linux server in a datacenter and instead use it to buy a decent internet connection, rather than relying on two mobile data plans.

    Or give up one of the mobile data plans, and use that savings to buy a decent internet connection.

    --
    #DeleteChrome
  19. Re:That's not called a VPN by Anonymice · · Score: 2

    Um, you mean like some sort of Link Aggregation Control Protocol? Sounds like a good idea!

    Oh, look!

  20. Re:Seems like a joke to me.. by caseih · · Score: 3, Insightful

    Seems like reading and comprehending the question is not doable for most of the folks in the comments of this story today. If you go back and read what the original poster asked, I think you'll realize that it's completely doable, with some (perhaps significant) effort. Certainly there aren't any out -of-box solutions that I know of. Basically a combination of mTCP and VPN is what he's looking for. The multipath connection is not between him and the gaming server. He wants it between him and a VPS running linux. The gaming server part is the final goal, but nothing to do with his problem or question. He certainly could invent his own tunneling protocol using, say UDP. As an example, if we consider the tcp/ip protocol, each packet has a unique sequence number. So if we take a TCP/IP packet, wrap it in a UDP packet and send one to the server through each interface, the server could unpack the UDP packet, note the sequence number, and if it already saw it recently, discard it. Otherwise, make a note of it and drop it onto the internet. On the return trip, acknowledgements would have to be handled on the client side. IE if one ack comes, we can safely ignore any others for the same sequence number. If no acks come from either pathway, then it's a standard timeout. This is TCP/IP only. I'm sure UDP could be encapsulated in a similar way, ICMP also probably.

    As I type this, I wonder if this could be done by hacking OpenVPN. OpenVPN already has udp encapsulation of UDP, ICMP, and TCP/IP. It would just be a matter of hacking in some support to send the same packet out multiple interfaces at once, and then logic to track and discard duplicates. Not sure how long either hand would have to track things for, or how much would have to be tracked.

  21. http://speedify.com/features/ by Chas · · Score: 4, Informative

    http://speedify.com/features/

    This kinda sounds like what you're looking for.

    --


    Chas - The one, the only.
    THANK GOD!!!
  22. Speedify by agizis · · Score: 3, Informative

    What you have described is Connectify's Speedify, it's a VPN that combines multiple internet connections together. As of the latest release it handles both jitter and loss, please check it out: http://speedify.com/blog/speed...

  23. interesting by blogologue · · Score: 3, Informative

    That's interesting. Maybe combined with this: http://www.cs.columbia.edu/~le... - it could provide what's needed. What I need in my case is good handling of TCP and UDP, anything else isn't relevant. Maybe I should try creating some sort of virtual network card that handles TCP and UDP, and hands the rest over to a real network card?

    --
    The blogologue
  24. It's an ISP problem likely by SirAudioMan · · Score: 3, Informative

    I have run into the same issue with my cable ISP. I run a voip setup using voip.ms as my provider and have my ATA connect to their servers. I have been plagued with random audio dropouts, talk-off and the occasional robot voice problem. After much research, troubleshooting I determined that the issue with jitter my ISP. Most pings to a know good server like Google DNS (8.8.8.8) averages say 40 ms but occasionally (say every 30 pings) the time jumps up to 800 ms. This happens regardless of the server I ping and also occurs when I ping my ISP's gateway address. This tells me that the problem is internal to my ISP and not an external routing problem.

    The reason why is what is called Node Congestion. Most North American cable ISP's use DOCSIS with hybrid-fiber nodes located through the geographic area. Nodes may start off with 100 active users on it meaning all 100 users are sharing that piece of the pipe. As time progresses, traffic changes, people ditch their cable tv for Netflix. All of this has a huge impact on congestion and bingo as a result ping times suffer. The average person will never notice but with any time sensitive service like voip and some gaming, you will notice it.

    There is not much you can do other than a) complain to your ISP (good luck) or b) find another that's not just a reseller of your existing cable's infrastructure. I'm not sure if DSL suffers the same issue as the shared cable plant.