Is Your Internet Connection Free From Bufferbloat? (blogspot.com)
Bufferbloat is that "undesirable latency that comes from a router or other network equipment buffering too much data," according to the site for
an ongoing project trying to address it. Now long-time Slashdot reader mtaht writes:Inside the lede-project, two core new bufferbloat-fighting techniques are poised to enter the linux mainline kernel and thousands of routers -- the first being a fq-codel'd and airtime fair scheduler for wifi, and the second, the new "cake" qdisc, which outperforms fq_codel across the board for shaping inbound and outbound connections.
His submission ends with a question for Slashdot readers. "It's been nearly six years since the start of the bufferbloat project. Have you or has your ISP fixed your bufferbloat yet?"
His submission ends with a question for Slashdot readers. "It's been nearly six years since the start of the bufferbloat project. Have you or has your ISP fixed your bufferbloat yet?"
Can hosts files help with this bufferbloat problem? Anyone here that can enlighten us?
Disable Nagle's Algorithm?
https://en.wikipedia.org/wiki/...
Not like I'm writing router or server code, I'm just a clueless dude surfing the web. Bad stuff happens, "the network" is hosed.
First bufferbloat test.
You've given it a cute name but I don't see any tangible problems....?
Why the fuck would Tin Man have a woody?
Idiot. It would be sheet metal, not wood. So it is called a steel hard stiffy. He is not equipped with one. Fluids are drained from a 1/2" NPT threaded plug in his abdomen. Recommended oil changes are every 5000 hours of operation.
While bufferbloat was patched out, my router is still under the control of a cargo ship, which claims to actually be an aircraft carrier. What is to be done about blufferboat?
Judging from the first 25 replies, the slashdot readership is suffering from an overdose of eggnog. Here's a link (which has links to results from every ISP), which shows latency under load often measured in seconds. http://www.dslreports.com/spee... The problem with this survey is that there are now plenty of folk that get sub-30ms latencies on their internet - which is what those using bufferbloat fixes get, and the question was if you or your isp was driving improved hardware to get those results. Problem seems to be 99% of the results are worse than that, still, 4+ years after the code to fix first arrived in Linux.
i notice that none of the pages that are linked to in this article have anyone's name on them. Is this the same crackpot who was complaining about this non-existant problem a few years ago?
...is what slows my connection speed down. Fuck, I could have a gigabit connection and would spend 80% of my time waiting for the next version of ad.doubleclick.net, etc. Really? Bufferbloat? I wish!
The latency measurements in the article are meaningless. Reducing seconds of latency to milliseconds! Where is point a and b? The driver layer adds ten seconds of latency? None of this makes sense.
You are a wien-er.
captcha = viewable
I don't have problems with bufferbloat, i have problems with speed throttling of my ISP once my data-cap of 3GB is used. Before reaching the cap I get 120 kbps download and once the 3GB is reached (due to ads and css on third party sites) my connection is throttled to 35 kbps. Fun times.
Comment removed based on user account deletion
Don't forget that dropped packets is the only reliable way for TCP to detect congestion, unless you want to use out-of-band signalling for congestion.
The simple answer why we don't hear an answer to "Have you or has your ISP fixed your bufferbloat yet?" is because the ISPs have been usin cut-through routing and forwarding that do not suffer from the buffer bloat. Modern internet does not suffer from this problem. Maybe your home lab with a Linux router does, but that is of no concern of the ISPs.
Now it is after I got my fiber connection it is all gone. My old *DSL connected at 50/10 mbit(errorfree) but I couldn't get anywhere near that(30mbit at most) and latency were way too high. Only place it caused me some problems was when I worked from home and the Citrix connection as I don't play online games.
You're right, of course. The trouble is, the latency increases aren't reasonable for common consumer networks under load.
Two speedtests I just did on my lightly-loaded hardwired home network (30Mbsp cable from Time Warner):
With QoS
Without QoS
Throughput is less (rather surprisingly less -- I may want to check some things) with my QoS rules that group connections into individually-throttled categories, but bufferbloat is sane-ish (a brief peak at 250ms was observed, but otherwise under 100ms).
Without QoS, bufferbloat starts at around 1000ms (x10 increase!) and goes up from there.
I'm currently using Shibby's version of Tomato-USB on an overkill dual-core Asus router to accomplish this, though I have used other consumer-ish hardware with reasonable success (including the venerable WRT54G/L/GS) using similar software.
The trick, as I see it, is primarily to ensure that the cable modem (and whatever is directly upstream of it at the head-end) never see enough throughput for their buffers to begin filling by keeping all nearby bottlenecks under my own control.
The other benefit of QoS is that on heavily bandwidth-constrained networks, some tasks can be given higher priorities than other tasks, which is easy when we control the neck of the bottle.
I dated a girl for a bit who had the cheapest Internet she could get: 2Mbps down. Her kids hated it, and web browsing with tablets and phones and laptops was terrible for all of them if anyone was streaming a video (badly) or downloading (slowly). Loud banter over who was "hogging the Internet" and ruining gaming was common, and not unreasonable. It got worse when people would visit. It was really bad.
Best case: They were taking turns using the Internet. In 2014.
After observing this and suggesting she get faster Internet ("no, it's not important to me," she said) I gave her a router with Tomato, did some obvious QoS priorities that were tweaked for that particular situation, and voila: The games worked fine. Web browsing was always quite responsive. Youtube worked (worked meh, but worked), and downloads and BT didn't trash any of the above. Anyone could do whatever they wanted, and the inevitable slowdowns were graceful while responsiveness remained good. The gamer of the house didn't get upset anymore seemingly-randomly.
But that's just one success story. I've been doing tricks like this for over a decade on a myriad of non-enterprise networks, using cheap hardware and thoughtful software.
(Now it's time for someone to pop up and tell me that I've done it all wrong, and that my results are impossible. This always happens on /. when I write about using Tomato and QoS to solve real, practical problems. I'm ready.)
Kid-proof tablet..
Is your moustache real?
You've done it all wrong! Those results are impossible!
Silence is a state of mime.
I have 32GB of ram in my system, 64mb barely registers on the radar. Nice try, thank you for playing.
So rise up, all ye lost ones, as one, we'll claw the clouds.
okay. so the internet provider is selling 100/100 Mbps to twenty people but the backbone uplink is only 1 Gbps.
so the solution to more snappy-ness is to limit YOUR CONNECTION (on your end) to fit into the REAL uplink capacity?
that is "move the buffer to something you control" which fits into the REAL uplink of 50 Mbps?
of course if the provider would upgrade the uplink to 2 Gbps there would be NO los of snappy-ness?
Comparison: Hosts gets 50mpg/400hp. Ublock gets 5mpg/30hp. What's better? Best hosts creator https://tech.slashdot.org/comments.pl?sid=10039961&cid=53555237/
APK
P.S.=> Hosts do far more for far less vs. UBlock (or any addons) & faster + more efficiently... apk
At least in Tranna
davecb@spamcop.net
I'll need to see your CCNA before I can accept your retort.
Kid-proof tablet..
Steely Dan?
A bullet may have your name on it, but artillery is addressed to " Whom It May concern"
I built my own router because I don't want any of these mass-produced, consumer piece of shit routers with more holes in them than swiss cheese.
Nice success story and the exact circumstances we were trying to make easier to solve with cake. (and the dream is more ISPs would just be doing it for you on their default supplied boxes)
I would like to benchmark more stuff like tomato's qos against cake, the equivalent (single!) command line for outbound would be:
tc qdisc add dev your_device root cake bandwidth 2mbit nat
which automatically applies per host fairness, qos, and queue length management.
inbound requires a slightly more complex setup but not much.
Pizza delivery for an I.C Weiner.....
oh crud.
Buffering is more useful for UDP, and is primarily intended to smooth out transient congestion on a network link or interface.
On a Carrier network, it's used to help deal with bursty traffic... TCP is a rather "slow" response mechanism which works fine in general, but can't handle congestion which comes and goes on millisecond or sub millisecond time scales.
For example, if an ISP has ten 100gig links running in a bundle, and a 1ms duration traffic burst fills one link up, the buffer will soak it up and allow for the load balancing to recalculate traffic flows across the other links. Without the buffer, those packets would drop, and all the TCP clients would react and throttle back, even though the overall bundle is still under 50% capacity, and even though the congestion has long since cleared before the packet times out.
The other place buffers are useful is when enforcing QoS on a network. They allow the router to evict higher priority traffic ahead of other traffic, so that when congestion hits you can still guarantee some traffic types.
Buffer bloat can't happen without congestion. Congestion is the real problem and talk of buffer bloat is a bit off-point. Sure, if you combat congestion with very large buffers (and hence significant queuing), you get increased latency due to the queuing. Reasonable increase in latency (say 20%) is not a huge hit on performance. Remember that you're trading that extra latency for lower probability of dropped packets.
You're correct that bufferbloat "only happens" when there's traffic. But I don't think you appreciate the current nature of internet traffic.
With web pages averaging 2 megabytes these days, you're "doing large file transfers" all the time. And if your iPhone kicks off an upload of its pictures, or your child starts watching videos, or your spouse starts their own web browsing/mail session, you're at the mercy of your router's queue management algorithm.
I don't think a "20% increase" in latency is reasonable, given that the Smart Queue Management (fq_codel, and soon cake) that's available in the Linux kernel (not to mention LEDE/OpenWrt/DD-WRT for your home routers) provide a "no-settings" way to limit lag/latency to an increase of only a few msec (or a couple dozen msec on a crummy DSL link).
maintain a hosts file is a PITA
With the speed of light through fiber, no one should ever see a ping above 300ms to anywhere in the world.
Even to places where there's no fiber connection? In a lot of places, the only route to the Internet with a throughput greater than the 0.15 Mbps of IDSL is through a satellite in geostationary Earth orbit, 36,000 km up. An ICMP ECHO request from a subscriber to a satellite ISP, such as Exede, needs to go up to the satellite and down to the destination network, and its response needs to come out of the network and then go up to the satellite and back down to the subscriber. That's 0.12 light seconds for each of four legs, already nearly half a second, plus whatever latency is in the destination network.
I've been using CeroWrt (https://www.bufferbloat.net/projects/cerowrt/wiki/ - the initial testbed for all of the bufferbloat work) for at least four years. For the majority of that time I had 1.5Mbps DSL service, but now I'm connected via a 12Mbps ADSL2+ link.
Prior to the installation of CeroWrt, it was painful for me to attempt to work remotely using an SSH tunnel if someone was watching a show via Netflix, but after setting up CeroWrt everyone was happy (me for not having to yell at my daughter and my daughter for being able to watch Netflix without me yelling).
With the 12Mbps link, it doesn't seem to be the ingress traffic that causes issues, but the egress traffic (at times, I upload large data sets). Without shaping the outbound traffic, I can see round-trip times in excess of 2 seconds which is just a bit excessive. ;-)
I recently installed LEDE (https://lede-project.org/) (an OpenWrt (https://openwrt.org/) fork) on a spare router (the same model as the CeroWrt router - WNDR3800) and it is obvious that the software continues to improve.
It appears that LEDE may be approaching its first stable release (https://forum.lede-project.org/t/criteria-for-first-lede-stable-release/552). If you have a spare router that is supported by LEDE, please consider installing a current build and report any issues found.
If you would like to learn more, here are a few random links to get you started:
I feel that the work that Dave (and everyone else that is involved) is so important that I send a few coins his way every month via Patreon. Here's his most recent update: "Where your donations go" (https://www.patreon.com/posts/where-your-go-7564906).
Dave, a belated Merry Christmas to you and I'm looking forward to a New Year where all of the efforts to tame bufferbloat and make WiFi fast benefit everyone.
Can you elaborate on some of the QoS settings you use? I'd like to mess around because speedtest.net gives me an F on bufferbloat, but I have no clue which knobs to tweak or what values are reasonable. HTB vs HFSC, service priorities, etc.
Remember that you're trading that extra latency for lower probability of dropped packets.
Not once you've gotten into the "bloated" range of buffer sizes. Increased latency from large buffers also increases the latency to signal to the sender that the route is congested. The sender will spend more time sending packets that will ultimately just get dropped. If the latency was lower, the sender would have known sooner to reduce its rate. Latency and loss go hand-in-hand once you get into unnaturally large buffers. I'm not sure the exact recommend buffer size, but I think it's around 10ms of the bandwidth. Many people are seeing 1,000ms+, which is 2 orders magnitudes above optimal.
UBlock can't do these as well as (or @ all) hosts do 4 speed, security, & reliability:
1.) Protect vs. bad sites (past ads)
2.) Protect vs. fastflux botnet C&C's
3.) Protect vs. dyndns botnet C&C's
4.) Protect vs. DGA botnet C&C's
5.) Protect vs. downed DNS (reliability)
6.) Protect vs. DNS poisoned dns
7.) Protect vs. trackers
8.) Protect vs. spam payloads
9.) Protect vs. phish payloads
10.) Protect vs. caps
11.) Get past dns blocks
12.) Keep off dns request logs
13.) Speed up 2 ways (adblocks/hardcodes)
14.) Work on anything webbound multiplatform.
15.) Ez data edit
16.) Block ads more efficiently in cpu/ram/I-O use
17.) UBlock now uses hosts (no DNS benefits vs. dns issues) - poor imitation = "sincerest form of flattery"
Hosts = native vs. illogically "Bolting on 'MoAr'" & not ClarityRay blockable like addons.
APK
P.S.=> Hosts (1st resolver) do MORE w/ less in fast kernelmode & before slow usermode addons
Hosts ~3mb vs. UBlock = 64MB -> http://cdn.ghacks.net/wp-conte...
APK Hosts File Engine 9.0++ SR-4 32/64-bit https://www.google.com/search?...
Ads rob speed, security (malvertising) & privacy (tracking).
Hosts add speed (hardcodes/adblocks), security (bad sites/poisoned dns), reliability (dns down), & anonymity (dns requestlogs/trackers) natively.
Works vs. caps & PUSH ads.
Avg. page = big as Doom http://www.theregister.co.uk/2... & ads = 40% of it.
Hosts != ClarityRay blockable (vs. souled-out to admen inferior wasteful redundant slow usermode addons)
Less power/cpu/ram + IO use vs. DNS/routers/addons/antivirus (slows you) + less security issues/complexity.
Compliments firewalls (blocking less used IP addys vs. hosts blocking more used domains) & DNS (lightens dns load).
Gets data via 10 security sites.
APK
P.S. - Safe https://www.virustotal.com/en/... (Verified by Malwarebytes' S. Burn "seen the code & it's safe" http://forum.hosts-file.net/vi... )
See subject & this link https://tech.slashdot.org/comments.pl?sid=10039961&cid=53557503/
APK
P.S.=> Hosts do FAR more (vs. far more threats) for FAR less using what you already natively have... apk
I have a spare Asus RT-N16.
Where do I get started with Cake?
Kid-proof tablet..
Your "seen the code & it's safe" link says "You are not allowed to access this resource" even with an account for hpHosts support forums.
bufferbloat is definitely still a thing.
I've been using this script for years to drop packets early to improve latency. it uses HFSC (built into linux since forever) and works great:
https://gist.github.com/eqhmco...
from that:
Congestion avoidance algorithms (such as those found in TCP) do a great job of allowing network endpoints to negotiate transfer rates that maximize a link's bandwidth usage without unduly penalizing any particular stream. This allows bulk transfer streams to use the maximum available bandwidth without affecting the latency of non-bulk (e.g. interactive) streams.
In other words, TCP lets you have your cake and eat it too -- both fast downloads and low latency all at the same time.
However, this only works if TCP's afore-mentioned congestion avoidance algorithms actually kick in. The most reliable method of signaling congestion is to drop packets. (There are other ways, such as ECN, but unfortunately they're still not in wide use.)
Dropping packets to make the network work better is kinda counter-intuitive. But, that's how TCP works. And if you take advantage of that, you can make TCP work great.
See subject: My program's tooltray icon rightclick popup menu lets you disable (or enable) hosts temporarily (or edit in/out entries in hosts).
APK
P.S.=> Nitpick away as an unidentifiable ac all you like - I've thought of your "objections" long ago & built them into the program... apk
See subject: That's all I have to say to "the trolling unidentifiable likes of you" as truth you're an unskilled "ne'er-do-well" that can't produce a better program for the same purpose yourself in GUI form, lol!
APK
P.S.=> I wonder HOW MANY TIMES I've handed you your ass beneath your "registered 'luser'" account on /. that you're trolling me by unidentifiable anonymous posts? apk
See subject & this link (Linux 64-bit executables are coming in it VERY soon & it does all major others already) https://community.embarcadero.com/article/news/16418-product-roadmap-august-2016/ & it already does MacOS X + Android & "IoT" also.
* I could port it to Lazarus/FreePascal though - but I want the codebase to stay "EXACTLY" the same done in the same compiler (less risk that way) & ports to *NIX for this program are easy enough - some WinSock2 vs. *NIX sockets diffs (easy), drive letters vs. mounted devices (easy), & finding *NIX API analogs to some Win32/64 API calls (should be easy enough).
APK
P.S.=> I'm not in the habit of helping an inferior competitor though & yes, Linux is inferior in many levels (mainly availability of applications vs. Windows)... apk
See subject: Mr. Burn of Malwarebytes' hpHosts probably has proxies you're using to hide your true IP filtered off OR you are too dumb to figure out how to use a forums!
APK
P.S.=> It's already obvious you're too unskilled & stupid to create a better hosts managing program than mine in GUI form - all you know how to do is stalk/troll by unindentifiable anonymous posts (that I shoot down w/ facts easily)... apk
See subject: E.G. - hostsman - it doesn't do hardcoded favorites @ top of hosts for fastest local memory resolution & protection vs. DNS redirect poisoning + dns being down. It also doesn't have a native 64-bit model (mine does). It also has a dependency on OTHERS' work (SQLite) so if bugs turn up in those libs/dlls, he has a problem & has to wait out a fix - I don't. My codebase is self-contained in a single multithreaded GUI executable.
57++ antivirus programs haven't found a bugs - there isn't any!.
Malwarebytes' own verfied it's code as safe too.
Dozens use it here speaking highly of my program & many 1,000's use it worldwide.
BOTTOM-LINE: You don't have a leg to stand on (which is why you post unidentifiably STALKING MY POSTS imo).
Seems like you're SCARED of the fact I'm doing well here! I love it...
APK
P.S.=> Keep "nitpicking" as an unindentifiable anonymous - I'll keep shooting down your "so-called 'points'" easily as always, lol... apk
See subject: Says all - *NIX folks'll like Delphi for Linux64 (does other major platforms already) https://community.embarcadero.com/article/news/16418-product-roadmap-august-2016/
* I was "holding off" (could port it to FreePascal via Lazarus IDE - both almost exact 'clones' of Delphi) due to not admittedly helping 'the competition' & waiting for Delphi to become complete for ALL platforms (most all if NOT all amongst the "major" ones...) + wanting it to be from the SAME EXACT COMPILER (less risk of bugs etc.) - that's coming VERY soon & I just MAY build ports to MacOS X (already possible) + Linux (or, possibly, open source it).
QUESTION: How would I 'drop you a line' when you left no point of contact in your message (e.g. email)?
APK
P.S.=> I'd "OpenSORES" it but after seeing what happened to GOOGLE's Chrome w/ EFast? No thanks & NO WAY - that's NOT going to happen to me or my work... apk
See subject & proof thereof here (only very partial, I can literally list MANY 100's more) https://it.slashdot.org/comments.pl?sid=9995967&cid=53488785/
APK
P.S.=> This populating hosts = best possible overall solution that's multiplatform & multidevice (hosts are everywhere & do FAR more for FAR less vs. more threats than ANY single other "so-called 'solution'") https://tech.slashdot.org/comments.pl?sid=10039961&cid=53557503/ via the BEST tool for hosts creation featured in the 2nd link I just posted... apk
I have an LTE Verizon Jetpack as my primary Internet connection, and the firmware is proprietary and not user-modifiable, and of course they refuse to implement bufferbloat mitigations on their own. So, no, it's not free from bufferbloat.
See subject: That's pretty short & it uses a quicksort (best overall sort for large & small datasets).
APK
P.S.=> My character mode version does it under 17 seconds but folks don't use tools like that for decades now (most want GUI & gui has overheads that slow it but ACCURACY is perfect & more important vs. speed only)... apk
See subject & in the meantime? I'm "Turning Japanese" https://www.youtube.com/watch?v=IWWwM2wwMww/
* LOL!
(It's my life...)
APK
P.S.=> Merry X-mas & happy new year (my email's in my program - I get ~ 2 mails a month, makes sense - I documented the hell out of it & how to use it)... apk