Slashdot Mirror

← Back to Stories (view on slashdot.org)

Pushing the Limits of Network Traffic With Open Source (cloudflare.com)

Posted by Soulskill on from the it's-all-fun-and-games-until-somebody-loses-a-packet dept.

An anonymous reader writes: CloudFlare's content delivery network relies on their ability to shuffle data around. As they've scaled up, they've run into some interesting technical limits on how fast they can manage this. Last month they explained how the unmodified Linux kernel can only handle about 1 million packets per second, when easily-available NICs can manage 10 times that. So, they did what you're supposed to do when you encounter a problem with open source software: they developed a patch for the Netmap project to increase throughput. "Usually, when a network card goes into the Netmap mode, all the RX queues get disconnected from the kernel and are available to the Netmap applications. We don't want that. We want to keep most of the RX queues back in the kernel mode, and enable Netmap mode only on selected RX queues. We call this functionality: 'single RX queue mode.'" With their changes, Netmap was able to receive about 5.8 million packets per second. Their patch is currently awaiting review.

2 of 55 comments (clear)

  1. Re:What does this mean? by Knuckles · · Score: 3, Informative

    If I have a 100Mb/s NIC, I'm only getting 10 MB/s on Linux? I doubt that.

    Packets != Bytes

    --
    "When I first heard Daydream Nation it quite frankly scared the living shit out of me." -- Matthew Stearns
  2. Re:What does this mean? by luvirini · · Score: 4, Informative

    A packet is not a byte. A packet is a sequence of bits including a address, other header information and the actual payload.

    IPv4 packet will as example have 20 bytes(160 bits) header and a maximum payload of 65,515 bytes(though often lower in practice)

    If you were to send a lot of packets with only a single byte payload then each packet will be 168 bits and your 100 Mb/s will result in about 600 000 packets. But at a gigabit connection the actual limit will start to hit for such strange traffic.

    Note that normally you would send more than a single byte of information/packet so in most real applications you would need much higher speeds to hit the limit. At 105 bytes of information you would have a total length of 1000, bits so would be at about the limit on gigabit hardware. But still most high bandwidth traffic tends to have much more information in each packet and thus not usually hit such limits.

    The limit has really started to hit due to the high availability of 10 gigabit and faster network cards coming down in price.