Slashdot Mirror

← Back to Stories (view on slashdot.org)

Crowdsourced Network Planning For Connection-Bridging Startup

Posted by timothy on from the no-but-really-where-are-you dept.

An anonymous reader writes "Tom's Hardware reports on the Connectify Switchboard software that "divides the user's traffic between Wi-Fi, 3G/4G and Ethernet-based connections on a packet-by-packet basis. Even a single stream — such as a Netflix movie — can be split between two or three Internet connections for a higher resolution and faster buffering." As part of its Kickstarter campaign, Connectify is geolocating their backers to optimize deployment of their servers. This is a clever way for supporters to influence the project beyond pledge levels and stretch goals, and it's actually kind of fun to watch."

41 of 58 comments (clear)

  1. Oh boy, sign me up!!! by fuzzyfuzzyfungus · · Score: 5, Insightful

    I, for one, am 100% gung-ho about having a 3rd-party in the 'cloud' handling every single one of my packets so that they can balance them between my connections!

    The proprietary client adding complexity to my machine's network stack is a bonus, of course.

    1. Re:Oh boy, sign me up!!! by rHBa · · Score: 2

      Sounds like the cloud version or RAID 0, great if you just need performance and don't expect any reliability...

    2. Re:Oh boy, sign me up!!! by Architect_sasyr · · Score: 1

      My first thought was the clouds version of the y2k defined SCTP protocol

      --
      Me failed English...
      FreeBSD over Linux. If my comments seem odd, this may explain...
    3. Re:Oh boy, sign me up!!! by dgatwood · · Score: 2

      I, for one, am 100% gung-ho about having a 3rd-party in the 'cloud' handling every single one of my packets so that they can balance them between my connections!

      There are already lots of third parties handling each of your packets. I'm not sure why one extra router would be a cause for concern.

      --

      Check out my sci-fi/humor trilogy at PatriotsBooks.

    4. Re:Oh boy, sign me up!!! by agizis · · Score: 1
      "It's worse than you know." "It usually is."

      The latency situation is much more complicated than you describe. You said that we were adding latency to the highest-latency connection. But that's not right, for software that routes every packet optimally: we're looking at a latency of (high-latency connection - ((highest latency connection - lowest latency connection) / 2)).

      But it's more complicated than that too, because Internet connections' latencies are not constants.

      We don't add to buffer bloat, we go war against it. Buffer bloat is the enemy of Switchboard. Switchboard wants to know where every packet is, and devices that claim to transmit but instead stick them in a buffer are problems. Switchboard is smart enough to sniff this behavior out, and give less packets to such devices, which hopefully leads to smaller queues, and better behavior, and lower latencies. Which is why in many cases Switchboard helps with latency... a typical 4G card will have 25 ms latency when you ping on it, but when you start a file transfer it will soar north of 800 ms. Switchboard will see this change and start routing you around that backlog in real time.

    5. Re: Oh boy, sign me up!!! by agizis · · Score: 1

      Of course there are buffers everywhere. We have buffers. Bufferbloat != "having some buffers". Bufferbloat is "when excess buffering of packets inside the network causes high latency and jitter, as well as reducing the overall network throughput", As always Wikipedia is good place to start http://en.wikipedia.org/wiki/Bufferbloat

  2. Alex from Connectify by agizis · · Score: 1

    It's so weird to be reading the news on Slashdot, and then realize that it's about you. This is Alex from Connectify, and I'm here, so I guess go ahead and ask me anything as reply to this comment, and I'll answer away.

    1. Re:Alex from Connectify by Cwix · · Score: 1

      How do you plan to handle the privacy implications?

      --
      You are entitled to your own opinions, not your own facts.
    2. Re:Alex from Connectify by agizis · · Score: 3, Informative

      We are going to keep the minimum logs required by law. But we are going to comply with the law, just as any company must. So, for those worried about the privacy implications, there are several options: a) run your own server, we have options to buy the software for your server, or b) use a VPN (dial the VPN after connecting Switchboard... all your traffic will be encrypted by them, then spread out by us, you can get the best of both). Of course you need to have VPN provider whom you trust.

    3. Re:Alex from Connectify by msauve · · Score: 1

      You may wish to correct the impression given by the summary (packet by packet basis). While decisions may be made by examining individual packets, it's really balancing sockets, which is what makes it possible.

      (and you might want to provide a more terse, less marketecture oriented explanation on your website)

      --
      "National Security is the chief cause of national insecurity." - Celine's First Law
    4. Re:Alex from Connectify by agizis · · Score: 4, Informative

      Thanks for asking, No, we do not guarantee privacy. We do our best to ensure your privacy, and keep as little information as possible, but we a) are focused on speed, not security and b) must comply with court orders. We work with VPNs, if you have a VPN you trust. dial them after firing up the Switchboard. They encrypt your traffic, then we spread it across connections, on the server side, we put it back together, and then hand it to the VPN server. It all works nicely, you can have speed and security.

    5. Re:Alex from Connectify by agizis · · Score: 1

      Sorry? No, it's right: we go packet-by-packet in our spreading. That's exactly what's so special here. On every Internet connection on your system we make one or more different sockets back to the speed server. Then we can make the right decision for each packet on which of OUR sockets it should become a part of. There's a lot more smarts that have to go into quickly, and correctly figuring out which Internet connection should carry the data: bandwidth, latency, packet loss, and behavior of the firewalls in the path are all taken into consideration.

    6. Re:Alex from Connectify by Mitreya · · Score: 1

      keep as little information as possible, but we a) are focused on speed, not security and b) must comply with court orders.

      Can you please elaborate on that?
      I understand the focused on speed part, but what is this about court orders? Is there a preemptive order requiring you to limit privacy?

    7. Re:Alex from Connectify by Architect_sasyr · · Score: 1

      Are you using SCTP or have you rolled your own standard?

      --
      Me failed English...
      FreeBSD over Linux. If my comments seem odd, this may explain...
    8. Re:Alex from Connectify by agizis · · Score: 1

      Oh, no, there's nothing like that. But you asked if I "guarantee privacy". I read slashdot daily, so I know that if Hide My Ass has to comply with UK judges and FBI warrants, then we will too: http://yro.slashdot.org/story/11/09/25/0415213/hidemyasscom-doesnt-hide-logs-from-the-fbi

    9. Re:Alex from Connectify by agizis · · Score: 2

      We rolled our own so we could overcome SCTP's problems with NAT traversal (which only seem partially fixed by trying to tunnel SCTP over UDP) and so we could do things like track the flight time of every packet

    10. Re:Alex from Connectify by msauve · · Score: 1

      Yea, I realized I was looking at the wrong product.

      Will you make any decisions based on the source networks? e.g. even with a single local Inet link to a single ISP, there might be benefit to then splitting out to multiple Connectify servers located in different BGP peers, routing around peering chokepoints and creating transits which might otherwise not be available.

      How about a home appliance, sitting in front of the router to the ISP (or on a Linux router)?

      I was disappointed to see that Linux support seems to be limited to high end solutions, and that the offerings were for "single user," and not "single family/home."

      --
      "National Security is the chief cause of national insecurity." - Celine's First Law
    11. Re:Alex from Connectify by JLennox · · Score: 2

      It would be cool if I could run switchboard on a pi and reserve it to all my devices.

      But I guess that's not a question :)

    12. Re:Alex from Connectify by agizis · · Score: 1

      You hit the nail on the head. Dealing with latency differences has probably been the hardest part of all this. We identify packets that are more latency sensitive, and route them over the lowest latency link.... doing this properly effectively divides the difference in latency between the links in half. We also take the QoS headers from the VoIP traffic and use them on our packets when sending them across the net. If your networks do something special for VoIP traffic, they will do the same for our traffic, when we're actually carrying VoIP.

    13. Re:Alex from Connectify by gratuitous_arp · · Score: 1

      Thanks for taking questions. Take business travelers who have a laptop with builtin 802.11, and a 4G card for when they are not around an open access point. With switchboard, they would be able to use the 4G network even when they are connected to the 802.11 and observe increased speeds. That sounds like something a lot of people would use.

      Do you consider this to be your target market, or something else?

    14. Re: Alex from Connectify by agizis · · Score: 1

      If we get a packet but haven't yet delivered the previous one in the sequence, we hold onto it until we either get the missing packet, so we can deliver what we have in order, or until we've seen a packet come in on all interfaces, and can declare the missing packet as lost, and then deliver everything we have.

    15. Re:Alex from Connectify by agizis · · Score: 1

      Absolutely, that was the main scenario we were thinking about as we designed this. It does a nice job of putting links together so you really can a skype video call from the hotel. Since then we've been amazed to find out how many other scenarios there are: people with 2 DSL lines to the houses, business disaster recovery scenarios, people needing to get video streams into or out of areas where they can only get 3G service... the list keeps growing. But yes, the business traveler was who was in our mind at the start.

    16. Re:Alex from Connectify by al.caughey · · Score: 1

      I've see you read... why do you move your lips?

  3. Re:Nothing new by OhSoLaMeow · · Score: 1

    VOIP is a good example of this - RTP packet routing can take various paths. It's up the the receiver and it's jitter buffer to coalesce the packets back into order.

    --
    They can take my LifeAlert pendant when they pry it from my cold dead fingers.
  4. buzzword by Ryanrule · · Score: 1

    buzzword

  5. This is new? by MrDoh! · · Score: 1

    Isn't this A) what you could do with a bit of messing about on old trumpet winsock installs, and B) what routers like the Cradlepoint stuff do already? The cradlepoint is more designed to auto roll over to a 3G connection if the main route drops, but can easily be configured to just add to the bandwidth. Not sure how this is different/new?

    --
    Waiting for an amusing sig.
    1. Re:This is new? by gl4ss · · Score: 1

      Isn't this A) what you could do with a bit of messing about on old trumpet winsock installs, and B) what routers like the Cradlepoint stuff do already?

      The cradlepoint is more designed to auto roll over to a 3G connection if the main route drops, but can easily be configured to just add to the bandwidth. Not sure how this is different/new?

      I guess the new aspect would be being affordable.. there has been demos of stuff like using ten 3g connections to transfer data pretty fast.

      --
      world was created 5 seconds before this post as it is.
    2. Re:This is new? by agizis · · Score: 1

      No, both are load balancing. In load balancing each socket is assigned to an internet connection where it stays for its lifetime. Works well with stuff like bittorrent, or networks with lots of users going through the load balancer. This is true channel bonding: each packet is separately routed down the best Internet connection. Even a single video stream or an encrypted VPN can be spread across all the connections. Ok, so then the next question is "can't I do that ifenslave in Linux already". No, you can't, because it round robins, treating every connection exactly the same. Switchboard uses bandwidth, latency, and packet loss to calculate an ideal path for each packet, often getting efficiencies as high as 95% of the speed of the connections added up.

  6. MPTCP by fufufang · · Score: 1

    MPTCP is way better than what Connectify is proposing... It is an open standard too...
    http://mptcp.info.ucl.ac.be/
    http://datatracker.ietf.org/wg/mptcp/charter/

    1. Re:MPTCP by agizis · · Score: 1

      MPTCP is really interesting, but if you want to combine connections to speed up a connection to a server that just supports regular TCP (Netflix, or Box, Dropbox, etc), even having MPTCP to somewhere else in the cloud, like a VPN server or connection aggregation server, doesn't get you very much. Running TCP over TCP in the real world is generally a bad idea, for reasons laid out pretty well here: http://vpnhaus.ncp-e.com/2011/06/30/sstp-the-problem-with-tcp-over-tcp-part-2/
      That's why Switchboard uses UDP as much as it can, and defaults to TCP when it absolutely has to, similar to what OpenVPN does.

  7. Never mind. by msauve · · Score: 1

    I went the the Connectify site, and based my comments on Dispatch. My bad.

    --
    "National Security is the chief cause of national insecurity." - Celine's First Law
    1. Re:Never mind. by agizis · · Score: 1

      No problem, thanks for the question anyway.

  8. Re:Homenet MSP by agizis · · Score: 1

    Ah, thanks for the question. (sorry I gave this answer to someone else too)
    MPTCP is really interesting, but if you want to combine connections to speed up a connection to a server that just supports regular TCP (Netflix, or Box, Dropbox, etc), even having MPTCP to somewhere else in the cloud, like a VPN server or connection aggregation server, doesn't get you very much. Running TCP over TCP in the real world is generally a bad idea, for reasons laid out pretty well here: http://vpnhaus.ncp-e.com/2011/06/30/sstp-the-problem-with-tcp-over-tcp-part-2/
    That's why Switchboard uses UDP as much as it can, and defaults to TCP when it absolutely has to, similar to what OpenVPN does.

  9. Right problem, wrong solution by FuzzNugget · · Score: 2

    Yes, many of us have shitty internet connections from a small selection of shitty providers. I know, let's saturate our already oversold connectivity to give said shitty providers another excuse to crank up rates with the bonus of hitting your usage caps even sooner!

    From what I can tell, this "Switchboard" is basically trying to consolidate and minimize connection overhead, which should theoretically offer modest performance gains. But your bandwidth is your bandwidth, no amount of software is going to stretch it.

    I wonder how much Kickstarter capital would need to be raised to start an ISP that doesn't employ the business model off shitting on its customers.

  10. Out-of-order packets? by bakuun · · Score: 1

    Using the Netflix example, wouldn't some packets going over 3G and others going over wired broadband cause massive problems with packets arriving out of order? There are methods for handling that in TCP of course, but I wonder how effective they would be in as exterme circumstances as we'd be talking about here.

    1. Re:Out-of-order packets? by agizis · · Score: 1

      Yes, out of order packets were a real problem, caused by the difference in latency between paths. We get them back in a semblance of order coming out of our network connection.

    2. Re:Out-of-order packets? by agizis · · Score: 1
      Oh, I see what you're thinking. No, we don't make a mess of anything.

      Look at our typical scenario: two ISPs, let's say 7Mbps DSL and 10Mbps cable and latencies that differ by, let's say, 20ms, with careful reordering you can see single-socket performance of about .95*(10+7) with average packet latency somewhere between the two. Without some careful reordering, TCP is very good at slowing way, way down, and a lot of UDP media streamers just fall apart completely.

      If you're us, and you get a packet but you haven't yet delivered the previous one in the sequence, the right thing to do is to hold onto it until you either get the missing packet, so you can deliver what you have in order, or until you've seen a packet come in on all interfaces, and you can declare the missing packet as lost, and then deliver everything you have.

      As far as SACK and D-SACK, you don't really want to do that for the 30% of your packets that arrive out of order. From what I've seen in the real-world, those RFCs are not intended for coalescing streams where potentially a lot of the packets are out-of-order (as they would be in the DSL + cable example).

      Thanks again for your interest.

  11. Load balancing was in Linux 20 years ago. by Kaz+Kylheku · · Score: 1

    Yawn!

    1. Re:Load balancing was in Linux 20 years ago. by agizis · · Score: 1

      Yes, and this isn't load balancing. Load balancing puts each socket on one internet connection where it stays for "life". We divide the traffic between the multiple connections at a packet by packet basis. Which means that unlike load balanced solutions, we can even speed up single sockets like streaming video, uploads and VPNs.

  12. Multipath TCP by funkboy · · Score: 2

    Sorry to rain on your parade, but multipath TCP already does this...

    1. Re:Multipath TCP by agizis · · Score: 1

      MPTCP is really interesting, but if you want to combine connections to speed up a connection to a server that just supports regular TCP (Netflix, or Box, Dropbox, etc), even having MPTCP to somewhere else in the cloud, like a VPN server or connection aggregation server, doesn't get you very much. Running TCP over TCP in the real world is generally a bad idea, for reasons laid out pretty well here: http://vpnhaus.ncp-e.com/2011/06/30/sstp-the-problem-with-tcp-over-tcp-part-2/(cutting