Slashdot Mirror

← Back to Stories (view on slashdot.org)

Crowdsourced Network Planning For Connection-Bridging Startup

Posted by timothy on from the no-but-really-where-are-you dept.

An anonymous reader writes "Tom's Hardware reports on the Connectify Switchboard software that "divides the user's traffic between Wi-Fi, 3G/4G and Ethernet-based connections on a packet-by-packet basis. Even a single stream — such as a Netflix movie — can be split between two or three Internet connections for a higher resolution and faster buffering." As part of its Kickstarter campaign, Connectify is geolocating their backers to optimize deployment of their servers. This is a clever way for supporters to influence the project beyond pledge levels and stretch goals, and it's actually kind of fun to watch."

58 comments

  1. Oh boy, sign me up!!! by fuzzyfuzzyfungus · · Score: 5, Insightful

    I, for one, am 100% gung-ho about having a 3rd-party in the 'cloud' handling every single one of my packets so that they can balance them between my connections!

    The proprietary client adding complexity to my machine's network stack is a bonus, of course.

    1. Re:Oh boy, sign me up!!! by rHBa · · Score: 2

      Sounds like the cloud version or RAID 0, great if you just need performance and don't expect any reliability...

    2. Re:Oh boy, sign me up!!! by Architect_sasyr · · Score: 1

      My first thought was the clouds version of the y2k defined SCTP protocol

      --
      Me failed English...
      FreeBSD over Linux. If my comments seem odd, this may explain...
    3. Re:Oh boy, sign me up!!! by dgatwood · · Score: 2

      I, for one, am 100% gung-ho about having a 3rd-party in the 'cloud' handling every single one of my packets so that they can balance them between my connections!

      There are already lots of third parties handling each of your packets. I'm not sure why one extra router would be a cause for concern.

      --

      Check out my sci-fi/humor trilogy at PatriotsBooks.

    4. Re:Oh boy, sign me up!!! by Anonymous Coward · · Score: 0

      Buffer bloat. Connectify almost certainly has to buffer packets.

    5. Re:Oh boy, sign me up!!! by Anonymous Coward · · Score: 0

      Even with minimum (1 packet or fragment) buffering, it adds latency to the highest-latency connection (probably the 3G/4G modem), so except for bulk transfers where you really don't care about latency at all, it sounds like just a horrible plan.

      The saving grace could be if it does understand QoS, allowing you to send VoIP or gaming data through the minimum-latency link (at highest priority), send browsing data through the minimum-latency link where possible, overflowing to higher-latency links as needed, and send bulk traffic through whatever's left. If done with sensible buffer allocations, that could make sense... maybe.

    6. Re:Oh boy, sign me up!!! by agizis · · Score: 1
      "It's worse than you know." "It usually is."

      The latency situation is much more complicated than you describe. You said that we were adding latency to the highest-latency connection. But that's not right, for software that routes every packet optimally: we're looking at a latency of (high-latency connection - ((highest latency connection - lowest latency connection) / 2)).

      But it's more complicated than that too, because Internet connections' latencies are not constants.

      We don't add to buffer bloat, we go war against it. Buffer bloat is the enemy of Switchboard. Switchboard wants to know where every packet is, and devices that claim to transmit but instead stick them in a buffer are problems. Switchboard is smart enough to sniff this behavior out, and give less packets to such devices, which hopefully leads to smaller queues, and better behavior, and lower latencies. Which is why in many cases Switchboard helps with latency... a typical 4G card will have 25 ms latency when you ping on it, but when you start a file transfer it will soar north of 800 ms. Switchboard will see this change and start routing you around that backlog in real time.

    7. Re: Oh boy, sign me up!!! by Anonymous Coward · · Score: 0

      This response is troubling because it appears you don't know that much about the forwarding architecture of the routers that make up the Internet. All of them put packets through a "buffer" (queue) before they are transmitted, because even at 100Gbps speeds, it is not possible to take every bit of a packet and instantly place it on the outgoing medium. There's no way your software is capable of instantly detecting this however, because even though these packets are "buffered" (queued), it's for fractions of a microsecond. These constant complaints about "bufferbloat" remind me of when the computing guys tell the network guys they don't want any packet drops of their application data. Even though their application data is TCP traffic and if it never dropped it would scale to take up all the network bandwidth...

    8. Re: Oh boy, sign me up!!! by agizis · · Score: 1

      Of course there are buffers everywhere. We have buffers. Bufferbloat != "having some buffers". Bufferbloat is "when excess buffering of packets inside the network causes high latency and jitter, as well as reducing the overall network throughput", As always Wikipedia is good place to start http://en.wikipedia.org/wiki/Bufferbloat

    9. Re:Oh boy, sign me up!!! by Anonymous Coward · · Score: 0

      And sometimes, I put 500 lbs of salt in the back up my pickup truck just for the heck of it. I get a kick out of watching my mileage drop and performance go down the crapper.

  2. Alex from Connectify by agizis · · Score: 1

    It's so weird to be reading the news on Slashdot, and then realize that it's about you. This is Alex from Connectify, and I'm here, so I guess go ahead and ask me anything as reply to this comment, and I'll answer away.

    1. Re:Alex from Connectify by Anonymous Coward · · Score: 0

      It's so weird to be reading the news on Slashdot, and then realize that it's about you. ... I guess go ahead and ask me anything as reply to this comment, and I'll answer away.

      I know, right?

      Well, this is Anonymous Coward from Slashdot, and you can ask me anything you want!

    2. Re:Alex from Connectify by Cwix · · Score: 1

      How do you plan to handle the privacy implications?

      --
      You are entitled to your own opinions, not your own facts.
    3. Re:Alex from Connectify by Anonymous Coward · · Score: 0

      This is Alex from Connectify, and I'm here, so I guess go ahead and ask me anything as reply to this comment, and I'll answer away.

      Does Connectify guarantee the privacy of my traffic? For both contents and TCP/IP metadata? How?

    4. Re:Alex from Connectify by Anonymous Coward · · Score: 0

      How can you help us facilitate piracy?

      -- Ethanol-fueled

    5. Re:Alex from Connectify by agizis · · Score: 3, Informative

      We are going to keep the minimum logs required by law. But we are going to comply with the law, just as any company must. So, for those worried about the privacy implications, there are several options: a) run your own server, we have options to buy the software for your server, or b) use a VPN (dial the VPN after connecting Switchboard... all your traffic will be encrypted by them, then spread out by us, you can get the best of both). Of course you need to have VPN provider whom you trust.

    6. Re:Alex from Connectify by msauve · · Score: 1

      You may wish to correct the impression given by the summary (packet by packet basis). While decisions may be made by examining individual packets, it's really balancing sockets, which is what makes it possible.

      (and you might want to provide a more terse, less marketecture oriented explanation on your website)

      --
      "National Security is the chief cause of national insecurity." - Celine's First Law
    7. Re:Alex from Connectify by agizis · · Score: 4, Informative

      Thanks for asking, No, we do not guarantee privacy. We do our best to ensure your privacy, and keep as little information as possible, but we a) are focused on speed, not security and b) must comply with court orders. We work with VPNs, if you have a VPN you trust. dial them after firing up the Switchboard. They encrypt your traffic, then we spread it across connections, on the server side, we put it back together, and then hand it to the VPN server. It all works nicely, you can have speed and security.

    8. Re:Alex from Connectify by agizis · · Score: 1

      Sorry? No, it's right: we go packet-by-packet in our spreading. That's exactly what's so special here. On every Internet connection on your system we make one or more different sockets back to the speed server. Then we can make the right decision for each packet on which of OUR sockets it should become a part of. There's a lot more smarts that have to go into quickly, and correctly figuring out which Internet connection should carry the data: bandwidth, latency, packet loss, and behavior of the firewalls in the path are all taken into consideration.

    9. Re:Alex from Connectify by Mitreya · · Score: 1

      keep as little information as possible, but we a) are focused on speed, not security and b) must comply with court orders.

      Can you please elaborate on that?
      I understand the focused on speed part, but what is this about court orders? Is there a preemptive order requiring you to limit privacy?

    10. Re:Alex from Connectify by Architect_sasyr · · Score: 1

      Are you using SCTP or have you rolled your own standard?

      --
      Me failed English...
      FreeBSD over Linux. If my comments seem odd, this may explain...
    11. Re:Alex from Connectify by agizis · · Score: 1

      Oh, no, there's nothing like that. But you asked if I "guarantee privacy". I read slashdot daily, so I know that if Hide My Ass has to comply with UK judges and FBI warrants, then we will too: http://yro.slashdot.org/story/11/09/25/0415213/hidemyasscom-doesnt-hide-logs-from-the-fbi

    12. Re:Alex from Connectify by agizis · · Score: 2

      We rolled our own so we could overcome SCTP's problems with NAT traversal (which only seem partially fixed by trying to tunnel SCTP over UDP) and so we could do things like track the flight time of every packet

    13. Re:Alex from Connectify by msauve · · Score: 1

      Yea, I realized I was looking at the wrong product.

      Will you make any decisions based on the source networks? e.g. even with a single local Inet link to a single ISP, there might be benefit to then splitting out to multiple Connectify servers located in different BGP peers, routing around peering chokepoints and creating transits which might otherwise not be available.

      How about a home appliance, sitting in front of the router to the ISP (or on a Linux router)?

      I was disappointed to see that Linux support seems to be limited to high end solutions, and that the offerings were for "single user," and not "single family/home."

      --
      "National Security is the chief cause of national insecurity." - Celine's First Law
    14. Re:Alex from Connectify by JLennox · · Score: 2

      It would be cool if I could run switchboard on a pi and reserve it to all my devices.

      But I guess that's not a question :)

    15. Re:Alex from Connectify by Anonymous Coward · · Score: 0

      What's the situation with respect to latency? It's not easy to combine links and use the full bandwidth (you cited 95% utilization in another comment, which is impressive for disparate links), but it's even harder to get good latency performance when one of them is a 3G or 4G modem.

      Are you using QoS to preferentially send VoIP et al. packets by the lowest-latency links, or just letting everything fall where it may?

    16. Re:Alex from Connectify by agizis · · Score: 1

      You hit the nail on the head. Dealing with latency differences has probably been the hardest part of all this. We identify packets that are more latency sensitive, and route them over the lowest latency link.... doing this properly effectively divides the difference in latency between the links in half. We also take the QoS headers from the VoIP traffic and use them on our packets when sending them across the net. If your networks do something special for VoIP traffic, they will do the same for our traffic, when we're actually carrying VoIP.

    17. Re:Alex from Connectify by Anonymous Coward · · Score: 0

      I've read some of the things you've posted. How do you sleep at night?

    18. Re: Alex from Connectify by Anonymous Coward · · Score: 0

      Are you going to guarantee in-order packet delivery?

    19. Re:Alex from Connectify by gratuitous_arp · · Score: 1

      Thanks for taking questions. Take business travelers who have a laptop with builtin 802.11, and a 4G card for when they are not around an open access point. With switchboard, they would be able to use the 4G network even when they are connected to the 802.11 and observe increased speeds. That sounds like something a lot of people would use.

      Do you consider this to be your target market, or something else?

    20. Re: Alex from Connectify by agizis · · Score: 1

      If we get a packet but haven't yet delivered the previous one in the sequence, we hold onto it until we either get the missing packet, so we can deliver what we have in order, or until we've seen a packet come in on all interfaces, and can declare the missing packet as lost, and then deliver everything we have.

    21. Re:Alex from Connectify by agizis · · Score: 1

      Absolutely, that was the main scenario we were thinking about as we designed this. It does a nice job of putting links together so you really can a skype video call from the hotel. Since then we've been amazed to find out how many other scenarios there are: people with 2 DSL lines to the houses, business disaster recovery scenarios, people needing to get video streams into or out of areas where they can only get 3G service... the list keeps growing. But yes, the business traveler was who was in our mind at the start.

    22. Re:Alex from Connectify by Anonymous Coward · · Score: 0

      Pi has barely enough capacity to route at 50 Mbps. That would not help much with my 100 Mbps 4G (actually getting just 80-98 Mbps) and 350 Mbps cable.

      That said, most services just don't have enough upstream bandwidth to fill even 350 Mbps...

    23. Re:Alex from Connectify by al.caughey · · Score: 1

      I've see you read... why do you move your lips?

  3. Nothing new by Anonymous Coward · · Score: 0

    I've been doing this for years... there's nothing new about this idea.

    1. Re:Nothing new by OhSoLaMeow · · Score: 1

      VOIP is a good example of this - RTP packet routing can take various paths. It's up the the receiver and it's jitter buffer to coalesce the packets back into order.

      --
      They can take my LifeAlert pendant when they pry it from my cold dead fingers.
  4. buzzword by Ryanrule · · Score: 1

    buzzword

  5. This is new? by MrDoh! · · Score: 1

    Isn't this A) what you could do with a bit of messing about on old trumpet winsock installs, and B) what routers like the Cradlepoint stuff do already? The cradlepoint is more designed to auto roll over to a 3G connection if the main route drops, but can easily be configured to just add to the bandwidth. Not sure how this is different/new?

    --
    Waiting for an amusing sig.
    1. Re:This is new? by gl4ss · · Score: 1

      Isn't this A) what you could do with a bit of messing about on old trumpet winsock installs, and B) what routers like the Cradlepoint stuff do already?

      The cradlepoint is more designed to auto roll over to a 3G connection if the main route drops, but can easily be configured to just add to the bandwidth. Not sure how this is different/new?

      I guess the new aspect would be being affordable.. there has been demos of stuff like using ten 3g connections to transfer data pretty fast.

      --
      world was created 5 seconds before this post as it is.
    2. Re:This is new? by agizis · · Score: 1

      No, both are load balancing. In load balancing each socket is assigned to an internet connection where it stays for its lifetime. Works well with stuff like bittorrent, or networks with lots of users going through the load balancer. This is true channel bonding: each packet is separately routed down the best Internet connection. Even a single video stream or an encrypted VPN can be spread across all the connections. Ok, so then the next question is "can't I do that ifenslave in Linux already". No, you can't, because it round robins, treating every connection exactly the same. Switchboard uses bandwidth, latency, and packet loss to calculate an ideal path for each packet, often getting efficiencies as high as 95% of the speed of the connections added up.

  6. Homenet MSP by Anonymous Coward · · Score: 0

    Why the heck are they not doing their implementation based on the homenet MSP purposed standard. http://tools.ietf.org/html/draft-haddad-homenet-multihomed-01

    1. Re:Homenet MSP by agizis · · Score: 1

      Ah, thanks for the question. (sorry I gave this answer to someone else too)
      MPTCP is really interesting, but if you want to combine connections to speed up a connection to a server that just supports regular TCP (Netflix, or Box, Dropbox, etc), even having MPTCP to somewhere else in the cloud, like a VPN server or connection aggregation server, doesn't get you very much. Running TCP over TCP in the real world is generally a bad idea, for reasons laid out pretty well here: http://vpnhaus.ncp-e.com/2011/06/30/sstp-the-problem-with-tcp-over-tcp-part-2/
      That's why Switchboard uses UDP as much as it can, and defaults to TCP when it absolutely has to, similar to what OpenVPN does.

  7. crack starter by Anonymous Coward · · Score: 0

    http://gawker.com/we-are-raising-200-000-to-buy-and-publish-the-rob-ford-508230073

    now here is a use.....for crowd sourcing....
    19K cash in 3 hrs

  8. MPTCP by fufufang · · Score: 1

    MPTCP is way better than what Connectify is proposing... It is an open standard too...
    http://mptcp.info.ucl.ac.be/
    http://datatracker.ietf.org/wg/mptcp/charter/

    1. Re:MPTCP by agizis · · Score: 1

      MPTCP is really interesting, but if you want to combine connections to speed up a connection to a server that just supports regular TCP (Netflix, or Box, Dropbox, etc), even having MPTCP to somewhere else in the cloud, like a VPN server or connection aggregation server, doesn't get you very much. Running TCP over TCP in the real world is generally a bad idea, for reasons laid out pretty well here: http://vpnhaus.ncp-e.com/2011/06/30/sstp-the-problem-with-tcp-over-tcp-part-2/
      That's why Switchboard uses UDP as much as it can, and defaults to TCP when it absolutely has to, similar to what OpenVPN does.

    2. Re:MPTCP by Anonymous Coward · · Score: 0

      Hmm... I actually don't quite see what the problem with an MPTCP-enabled middleman would be, at least for streaming video. It's not actually TCP over TCP. Just have them buffer enough to cover 2*(sum of bandwidths)*(max of RTTs) and you're all set. They just have to be able to keep ahead of you.

      Of course, this whole concept really isn't that useful, at least not for the realistic example of 3G+wifi that they're proposing. At best you can cut down the pre-buffering time a bit; after that, the 3G connection would just be an extremely expensive way to get a tiny fractional improvement.

  9. Never mind. by msauve · · Score: 1

    I went the the Connectify site, and based my comments on Dispatch. My bad.

    --
    "National Security is the chief cause of national insecurity." - Celine's First Law
    1. Re:Never mind. by agizis · · Score: 1

      No problem, thanks for the question anyway.

  10. Right problem, wrong solution by FuzzNugget · · Score: 2

    Yes, many of us have shitty internet connections from a small selection of shitty providers. I know, let's saturate our already oversold connectivity to give said shitty providers another excuse to crank up rates with the bonus of hitting your usage caps even sooner!

    From what I can tell, this "Switchboard" is basically trying to consolidate and minimize connection overhead, which should theoretically offer modest performance gains. But your bandwidth is your bandwidth, no amount of software is going to stretch it.

    I wonder how much Kickstarter capital would need to be raised to start an ISP that doesn't employ the business model off shitting on its customers.

    1. Re:Right problem, wrong solution by Anonymous Coward · · Score: 0

      The answer to your kickstarter question is... widely variable.

      You need to define the problem a lot more before you can get an answer. At a minimum you need to answer the following:

      - geographical coverage (4 square blocks in Madison WI, Richmond VA, The west coast states, or National)
      - connection types (FTTH, LTE Adavanced, DOCIS 3, VDSL)
      - "Build it and they will come" or "Build it as they come" business models

      For example, on the far down end of the scale (4 blocks in Madison WI, using VDSL) you are looking at maybe $50k. That gets you the DSLAM, the Linux servers (as routers, plus support platform) and get you installed someplace. Also covers the NRC install for your uplinks. The assumption being the prices you charge covers the ongoing opex and MRCs needed to run this going forward. It might even be able to cover a bit more - a good bit more than just four blocks can be covered by a single DSLAM. The problems faced here are more regulatory than financial. Oh, one other thing - you better charge an install fee for each circuit - copper isn't cheap, nor is installing it.

      The other end of the spectrum (pardon the pun) is you drop several billion on wireless licenses, another billion or two on finishing developing the LTE Advanced technology and developing product for it, then do a national backbone for another billion, a huge number of metro aggregation networks for another 2 billion and then 80,000 tower sites to get real coverage (many of which you rent space, but you still have to build out facilities to them, and install gear and antennas) at about $150k a pop for another 12 billion. So something north of $20B. BTW, you get to do this while multiple someones (I just might be among them) who already have 100M subscribers are doing incremental upgrades in limited areas to make your life in the marketplace a living hell.

      And then we didn't even get into all the other aspects of operating the network that needs to be built in order to onboard enough customers to maybe, someday generate a return.

      So depending on what you want to do, it can be pretty small to OMG large. Just to point out an example, you can see how fast Google, who has $20B laying around, is covering the whole country with FTTH. (Yes, I'm bitter that they aren't deploying where I live - I'd like a 3rd ISP in the area so I can upgrade the low performing 2nd carrier I currently have and make my current primary my backup circuit).

  11. Out-of-order packets? by bakuun · · Score: 1

    Using the Netflix example, wouldn't some packets going over 3G and others going over wired broadband cause massive problems with packets arriving out of order? There are methods for handling that in TCP of course, but I wonder how effective they would be in as exterme circumstances as we'd be talking about here.

    1. Re:Out-of-order packets? by agizis · · Score: 1

      Yes, out of order packets were a real problem, caused by the difference in latency between paths. We get them back in a semblance of order coming out of our network connection.

    2. Re:Out-of-order packets? by Anonymous Coward · · Score: 0

      The explanation you just gave makes absolutely no sense (I have read it 4 times now), re: "we get them back in a semblance of order coming out of our network connection".

      How I've interpreted what you've said, as best as I can: "the packets don't truly arrive out-of-order at the final destination (i.e. a Netflix server)), because they're aggregated by upstream routers and put into order before being handed to that server".

      If that is what you meant: you do not understand how TCP behaves at all in the real world. I can gladly expand on this (yes I am familiar with TCP and underlying protocol extensions such as Selective ACK, which is one of the only ways to deal with out-of-order packets) if need be.

      Overall what Connectify Switchboard is introducing is going to make a gigantic fucking mess of TCP. I repeat: a gigantic mess. It is not clever because it assumes way too much the conditions of the network between source and destination (i.e. the network which the sender, nor receiver, has any bit of control over).

    3. Re:Out-of-order packets? by agizis · · Score: 1
      Oh, I see what you're thinking. No, we don't make a mess of anything.

      Look at our typical scenario: two ISPs, let's say 7Mbps DSL and 10Mbps cable and latencies that differ by, let's say, 20ms, with careful reordering you can see single-socket performance of about .95*(10+7) with average packet latency somewhere between the two. Without some careful reordering, TCP is very good at slowing way, way down, and a lot of UDP media streamers just fall apart completely.

      If you're us, and you get a packet but you haven't yet delivered the previous one in the sequence, the right thing to do is to hold onto it until you either get the missing packet, so you can deliver what you have in order, or until you've seen a packet come in on all interfaces, and you can declare the missing packet as lost, and then deliver everything you have.

      As far as SACK and D-SACK, you don't really want to do that for the 30% of your packets that arrive out of order. From what I've seen in the real-world, those RFCs are not intended for coalescing streams where potentially a lot of the packets are out-of-order (as they would be in the DSL + cable example).

      Thanks again for your interest.

  12. Load balancing was in Linux 20 years ago. by Kaz+Kylheku · · Score: 1

    Yawn!

    1. Re:Load balancing was in Linux 20 years ago. by agizis · · Score: 1

      Yes, and this isn't load balancing. Load balancing puts each socket on one internet connection where it stays for "life". We divide the traffic between the multiple connections at a packet by packet basis. Which means that unlike load balanced solutions, we can even speed up single sockets like streaming video, uploads and VPNs.

  13. Multipath TCP by funkboy · · Score: 2

    Sorry to rain on your parade, but multipath TCP already does this...

    1. Re:Multipath TCP by agizis · · Score: 1

      MPTCP is really interesting, but if you want to combine connections to speed up a connection to a server that just supports regular TCP (Netflix, or Box, Dropbox, etc), even having MPTCP to somewhere else in the cloud, like a VPN server or connection aggregation server, doesn't get you very much. Running TCP over TCP in the real world is generally a bad idea, for reasons laid out pretty well here: http://vpnhaus.ncp-e.com/2011/06/30/sstp-the-problem-with-tcp-over-tcp-part-2/(cutting