Slashdot Mirror
Comcast's Congestion Catch-22
An anonymous reader sends us to Telephony Online for a story about Comcast's second attempt at traffic management (free registration may be required). After the heavy criticism they received from customers and the FCC about their first system, they've adopted a more even-handed "protocol agnostic" approach. Nevertheless, they're once again under scrutiny from the FCC, this time for the way their system interacts with VOIP traffic. By ignoring specific protocols, the occasional bandwidth limits on high-usage customers interferes with those customers' VOIP, yet Comcast's own Digital Voice is unaffected. Quoting:
"The shocking thing is just how big a Pandora's box the FCC has appeared to open — and it just keeps getting bigger. When the FCC first started addressing bandwidth usage and DPI issues, it quickly found itself up to its knees in network management minutia. Not long after that, it followed another logical path of the DPI question and asked service providers and Web companies about their use of DPI for behavioral targeting. Now it seemingly has opened up huge questions about what it means to be a voice carrier in the age of IP. It's not hard to imagine the next step: What about video? Telco IPTV services are delivered in roughly the same way as carrier VoIP services — via packets running on the same physical network but a prioritized logical signaling stream. Is that fair to over-the-top video service providers?"
If Comcast is having major network congestion then why did they automatically double everyone's download speeds? I got a letter a few days ago saying that I now get 12 down rather then 6. Seems like a BAD idea if they are having congestion issues...
~ Mooga
It would have been first, but someone was on the phone.
If someone is doing very high traffic, enough to get into Comcast's temporary "QOS Low" category, they are probably sending and full rate. If you are sending at full rate, the typical end-host NAT and buffering alone will cause bad quality for VoIP (search for VoIP and BitTorrent for a lot of such tales). There is nothing Comcast's network management really does to affect things in this case anyway.
Comcast's network management should only cause additional VoIP issues when the big transfer STOPS and the VoIP call is made within only a few minutes (before the user's link is reclassed back into the "QoS normal" category).
Test your net with Netalyzr
I don't know what's going on in my area, but Comcast sucks ass out here. The connection will just flat-out drop for 10-20 seconds at a time. Really fucks you up when you're trying to play a game online. They've had techs out here a few times with no results. Thank God I just found out that Qwest has their 7Mbps DSL service out here. I'm crossing my fingers and hoping that it's better than fucking Commiecast's godawful service.
Why do ISPs insist on being more than just a pipe? It's so dumb no one wants them to be anything else. Do they just not feel useful when they are a pipe?
If you have VOIP, don't set your kitchen on fire during high congestion periods. Please people, a little take a little personal responsibility.
Leave the gun, take the cannolis.
Good lord, call the feds...
Why don't they just UPGRADE THE PIPES.
My god every other first world country has huge bandwidth where these types of things aren't even a consideration. Yet comcast just whines because you can't run everything and be fair on tiny pipes.
The ISP supplying my workplace regularly blocks HTTP for up to hours at a time. Nothing else, just outgoing(!) port 80. First connections get dropped silently, then after a while it moves on to forged TCP reset packets when trying to connect to anything. Which is pretty worrying because they're the only ISP available here.
because they want to milk the current technology and still profit from it as long as they can. I know I work with these guys. No need for FTTH until there's competition.
They'll just let the FTTH come in and then merge with that company or acquire them outright.
the typical slogan of "they bought us" comes into play here. As cable becomes telco that's just what happens. Only you have to wait until the market forces make that happen. Here in NY Verizon basically takes about 10 thousand households away from time warner every month. Merger aquisitions! It's on the shopping list, as Larry Ellison likes to say :)
"Catch-22" implies a no-win situation. Comcast (and the other ISPs) have done this to themselves. They advertise unlimited Internet access (or make it seem like they're offering unlimited access) and then get upset when someone tries to use it.
The ISPs should start advertising their download speed, upload speed, and bandwidth caps openly. Offer additional speed and bandwidth for a reasonable price. And if your infrastructure is such that sometimes you'll need to throttle someone, make it clear upfront how and when such throttling will happen.
Right now, on Comcast's sale page, they only list the download speed of their connections. I couldn't find their upload speeds or the bandwidth caps (which I know to be 250GB). As far as I know, Comcast customers have no way to check to see if their being throttled or if they're near the bandwidth cap.
It's really no surprise then that customers are upset.
If they are treating their own VOIP differently than other traffic then it isn't "protocol agnostic" at all.
Comcast has no interest in providing oppertunity for competing with themselves either in media delivery or for phone service.
The same goes for the telephone company, whom really has no incentive to provide you an internet connection
that would let you drop your telephone line for voip. This is the real problem with our internet infrastructure, the technology
and products are out there to give everyone a superfast connection, but why would the telephone company or the cable company do this,
they would be competing against themselves. It is a real shame that countries like Sweden, Japan and South Korea have faster internet options
available to them at cheap prices.
...and i hope their board of directors catches on fire.
Hmmm. Based on their massive profits, I would suggest they put aside, say, 1/10th of those profits to upgrade their network. That would fix it.
I mean, I realize that would keep Comcast execs from getting the gold-plated Bentley this month, but sacrifices have to made.
Actually, I don't really care whether they provide traffic priorization for VoIP and video, on a non-protocol-agnostic basis, as long as it is provider-agnostic.
Of course, should they do this, some clever monkeys will tunnel their bittorrents over VoIP just so their abusive software gets the best treatments. Much like overzealous block-everything-but-port-80 practice has led to everyone and his dog using port 80 regardless of content. I'm with Briscoe that we need to redefine "fairness", but I get the feeling the comcast NOC people aren't really up to the tast. Any wandering scientists about to lend a hand?
For the few of us who have access to multiple Internet connections for whatever (legit) reason, why not try your VOIP call on both, and see if the choppiness goes away on the 2nd one?
Comcast is a cable company, right? So isn't this just because their VoIP can be put in a separate Docsis channel (and prioritised accordingly), while 'regular' VoIP is sent as normal data?
Join the anonymous, help develop the network: http://www.i2p2.de
Consider yourself lucky.
Plan A:
Set up your home router to allow incoming SSH and set up your home computer to allow SSH tunneling. Then use an SSH client on your machine to route everything through your home computer.
Problem solved.
Well, until your employer catches on, in which case I hope your resume is up to date.
Plan B:
Be pro-active and find a new employer.
Knowledge is how to play a game, intelligence is how to win, wisdom is knowing what game to play.
If their VoIP isn't going over the public Internet, then it's really just telephone service that happens to use TCP/IP. That's very different from voice over Internet.
Voice- or video- over a private network is legally more akin to running a telephone company, and COMCAST may find themselves regulated as one.
Knowledge is how to play a game, intelligence is how to win, wisdom is knowing what game to play.
I don't want to spend an hour writing a treatise on this, but I do think I need to make a few things clear.
Several issues are convolved here, and the "right" answer to each individual issue is not obvious (at least not once one factors in political and business viewpoints), so the convolution is essentially a mess. Like the original poster of the story, I have to assume that the FCC decided intentionally to delve into the mess. Anyway, here are the real issues:
1. There is (as far as I know) no new technology here. The PacketCable specs, which define how cable operators (most of them, anyway) implement VoIP were released in 1999. Comcast (like other US cable operators) has been deploying this technology since about 2002. As far as I know, the only part of the spec that Comcast don't really implement is the security portion. In any case, the specs are public and have been so for nearly a decade.
2. There is a fundamental technical difference between over-the-top VoIP (i.e., service provided by a third party such as Vonage) and telephony provided by the cable company.
3. The cable company can differentiate between its VoIP (or any other service needing preferential Quality of Service (QoS)) and ordinary so-called "best-effort" traffic, which is what is used to carry everything else, including over-the-top VoIP.
4. The reason for this is that it is the only entity that has access to the Cable Modem Termination System (CMTS), which controls the microsecond-by-microsecond details of traffic flow over the plant between a customer and the cable operator's facilities.
5. It is reasonable (from the cable operator's point of view) that since it owns the CMTS (and CMTSes are not cheap either to acquire or to manage), it's not voluntarily going to let some other company control any part of its operation (especially since if that gets screwed up, the customer experience is impacted).
6. Looked at from the point of view of guarantees applied to services, this looks like a violation of net neutrality, since over-the-top operators have to fight for bandwidth against things like P2P and web browsing, while the cable operator's phone calls don't have to do so (they have QoS guarantees).
7. But there is no law against violating new neutrality (as far as I know, in the US anyway). IANAL.
8. One can also argue that although it *looks* like a violation of net neutrality, it is in fact not such a violation, since from the viewpoint of what is happening inside DOCSIS (the protocols used to manage bandwidth on the plant between the residence and the cable company), over-the-top VoIP looks completely different from the cable company's offering. From that technical viewpoint, they can be considered two different services, and hence it would (presumably) be fine even under net neutrality principles to treat them differently.
Those are the basic ideas (although of course I've just summarized; it would take a lot more space to really describe all the details). But the basic point here is that there are lots of issues and viewpoints, some business-related, some political, and some technical. And much though one might like to demonize one party or the other, in this particular case the issues don't really seem to lend themselves to such a simple analysis.
Disclaimer: this was a quickly-written post of my initial impressions given the rather sparse (and not unambiguous) information available.
Bandwidth is a commodity. As such is interchangeable: the provider of a commodity is in competition with everyone else providing the same commodity. They have to differentiate themselves based on price, which they can only do by cutting costs and increasing efficiency. Though market competition is in our best interests as consumers, it isn't in theirs. The last thing a company wants is for market competition to work efficiently to drive down their margins. That's why they will do everything they can to avoid selling a commodity: product differentiation, branding, and so on - strategies that effectively create mini monopolies (you don't buy an MP3 player, you buy an iPod; you don't buy shoes, you buy Nike).
That's the main reason. Another, which applies especially to monopolies (hello telecoms!), is price discrimination. A company would like to charge each customer as much as that customer can afford to pay, but they don't want to lose business with a price that's too high. By developing different classes of service they can coax more money from those able to pay more. The classic example is first-class seating on flights. How much a customer is able to pay may also depend on how much the service is worth to them. It may not cost the telecom company any more to provide bandwidth for, say, VoIP users than for WoW players, but VoIP customers may be able to pay more because it saves them money elsewhere.
It is the role of good market regulation to ensure competition works effectively to drive prices down towards costs. That is broadly good for consumers and for the economy as a whole. Companies - especially incumbent companies - should be expected to do everything in their power to fight to break the market. And they do.
"Comcast Digital Voice uses Internet Protocol and not the Internet. Comcast Digital
Voice calls travel on our private, managed network -- not over the public Internet. That makes
it superior to other 'Best Effort' services delivering phone traffic over the public Internet."
Source (emphasis mine): http://www.comcast.com/MediaLibrary/1/1/About/PressRoom/Documents/ProductsAndServices/digital_voice.pdf
My comments here are my own; I do not speak for my employer.
That is just way too much effort for pleasure. It's much simpler to twist a belt around your neck until you pass out. The rush is amazing! Every AC should try it a few dozen times a day. :-)
Comcast's digital voice service uses a different frequency on the broadband line, so it doesn't actually use bandwidth, and will work even if your internet connection is down.
Every other first world country has immensely higher population density.
Wrong, unless you're saying that Finland, Sweden, and Norway are not in the first world. http://en.wikipedia.org/wiki/List_of_countries_by_population_density
I live in Finland which has about 5 million persons at a population density of 15.6 per sq.km, while the US has about 300 million at 31 per sq.km, or double Finland's population density. Actually, about half of Finland's population is near the south coast (especially around Helsinki and Turku), while I'm in a rural area 300km north of Helsinki, so our regional population density is a bit lower. The largest town within 200km has about 80,000 people.
I have fiber to the house with 100/10 service available. The service is eur55 per month, including IP TV. If it's possible in the countryside in Finland, then it should be possible in most of US, where local populations and population densities are higher.
In fact, there are substantial areas of the U.S. with quite high population densities and local populations greater than all of Finland. Example: New Jersey, with 8 million persons at 438 per sq.km, and many millions more in adjacent areas. http://en.wikipedia.org/wiki/New_jersey
Your argument based on population density is a load of bollocks. You're just screwed by your ISPs.
Those who can make you believe absurdities can make you commit atrocities. - Voltaire
Well of course.
If they offer a netflix alternative expect that to be a better performer due to shaping as well.
Most people will just think the alternatives suck and choose comcast's service instead, never the wiser.
---- Booth was a patriot ----
Comment removed based on user account deletion
We're still not doing congestion control very well. DOCSIS 3.0, the new cable modem/hub standard, has many congestion management features, but they're a collection of features, not an integrated strategy.
Realistically, there are two QoS options a congestion strategy for general IP-based networks can deliver:
There are fancier reservation schemes, where you can reserve bandwidth, but they only work when all the players in the path cooperate, which tends not to happen. But there's no reason not to get the simple mechanisms above right.
Really. I mean that.
Most VOIP uses UDP, which, *by specification* does not guarantee delivery timeliness, order, or that the packets will even arrive at their destination. It's strictly a fire and forget protocol, and this should have been understood from the outset. While I understand the advantages it brings on well-managed networks, and the value it has for those who can tolerate dropped speech and calls, it should not be thought of telephony, as it is nowhere near as reliable as conventional POTS networks.
Even for VOIP designs which use TCP, or other delivery assurance mechanisms, IP itself does not guarantee a maximum latency. While the network provider may be able to manage latency, and even guarantee it for a given segment of network, there is currently no way to guarantee a maximum latency when travelling outside the provider's network. Furthermore, even if there existed a protocol mechanism which provided definitive latency management, the fact remains that most ISPs do not make latency, or even *bandwidth* guarantees. Building a realtime voice application on such an infrastructure remains risky, at best.
The common customer, who grew up with phone lines, does not understand how IP networks work, nor why a common internet connection is not going to provide them with a 100% reliable connection. Instead, they're going to expect the network to fully support realtime voice, because, "I've got up to 6 Mbs, and this VOIP phone only requires 128kbs connection..."
I don't mean to rip on VOIP, but the technology was designed for networks which can, and do, guarantee max latency and min bandwidth. This excludes the majority of residential broadband and DSL customers. Yet the VOIP companies conveniently forget this in their advertisements. It really isn't the ISP's problem, because they are delivering the service advertised.
The society for a thought-free internet welcomes you.
If comcast treats Vonage like their VOIP then you're vonage calls will never work... i think vonage needs to go across the internet.
I believe Comcast VOIP sends it's voice as marked packets that get diverted to some kind of soft-switch...
ISPs need to be held responsible for the bandwidth they sell! If they can't do it then they shouldn't advertise it.
ISPs should NOT be allowed to legally prioritize any traffic. This does NOT exclude the ability for ME to prioritize my OWN traffic making it my responsibility and freedom.
Now you network minded people will say: How do they know what traffic of yours is important to you when you and your neighbors peak above the base guaranteed rates they advertise?
The solution is more simple than many of the schemes they are trying or wishing to implement in the future: user flagged prioritization!
You flag your important traffic yourself they guarantee up to the advertised amount bandwidth will not be dropped.
Problem solved. If you flag too much they have the right to drop anything over the limit just like they can (have done and may be still doing) with not prioritized data we have today.
Democracy Now! - uncensored, anti-establishment news
From the article:
One solution would be to use the DPI boxes as intended, said Allot's Cullen, to inspect packets. In that way, Comcast or other providers could both manage congestion as well as provide some level of priority access to real-time applications like VoIP or real-time gaming. Those apps typically don't consume much bandwidth anyway, while benefitting greatly from enhanced prioritization.
If they do that, then what would stop those eeeeeeevil P2P folks from massaging their traffic so the packets look similar to VOIP?
If you're a zombie and you know it, bite your friend!
Comcast should be required to put in all of their advertising the fact that they prioritize their own VOIP traffic over VOIP from other companies.
FYI, this is exactly what Comcast is planning on doing. The actual date of when they stop analog service will vary based on the market, but already in the Connecticut area, they're no longer signing up new customers for "standard" cable (i.e., 70 analog channels, no box).
Of course, once everyone has a box, that's one less argument against a la carte pricing...
"Anyone who [rips a CD] is probably engaging in copyright infringement." - David O. Carson
No, the cable companies have been reducing the number of analog channels. RCN has completed "project crush", to get rid of them all. You thus need a cable box. Comcast still has some analog channels and the FCC has given them a hard time about removing them.
But this has NOTHING to do with the issue at hand. That's downstream video capacity, unrelated to PacketCable, a separately-multiplexed two-way service that uses some of the cable's scarce upstream capacity. "Congestion" is not usually caused by scarcity; that's an obvious, but wrong, assumption (sort of the flat earth theory of operations research).
When I last moved, I avoided Comcast due to their congestion/throttling. Got satellite TV and DSL instead.
For the past couple of days or so I was getting extreme latency on the network and I was like WTF is going on?
So I took at the firewall's RRD graphs which tracks the traffic latency and it shows average 400ms. Spiked to 2002ms. I was like..WTF is this crap?
I was gonna call them up and complain but figured some kids running bit-torrent and they will deal with em soon enough.
Now this news got out I guess I still need to call and complain.
But... Comcast's traffic shaping policies do not apply unless you've used over 75% of the upstream or downstream for 15 minutes straight, and even then only when the whole cable node is congested.
Basically, you just shouldn't expect to run some full-rate bittorrent and voip at the same time.
Ideally Comcast will upgrade their networks to meet demand, but the bandwidth-based policy they have sounds fair to me.
As many people have said, Comcast's phone service is not subject to traffic shaping because it does not come in with the other IP packets, it's allocated it's own slice to come in on.
I have comcast, and I've noticed as of a few weeks ago that any HTTP download greater than 100 mb will simply die halfway through.
newsgroup downloads will slow to 20kbs if the pieces are greater than 100 mb as well
I use perhaps 20gb a month.
Perhaps they wouldn't experience congestion if they UPGRADED THEIR INFRASTRUCTURE.
VLC FOR MAC IS DYING! IF YOU DEVELOP, PLEASE SAVE IT!!
Not sure exactly what kind of "VoIP" service Comcast offers here. Is this VoIP phone like Vonage but branded and sold by Comcast? Or is this digital phone. The company I work for sells both. The digital phone hooks up to the telco wiring outside the customer's home and other than that its just like a PSTN phone. They also sell VoIP. You have to have internet service to get it, and you hook up the phone to the modem inside your house. But the bandwidth is separate from the regular HSD service. So if Comcast's service is unaffected by traffic shaping policies, this may be the kind of VoIP service they're offering. And that makes sense.
If Comcast's traffic shaping is affecting third party VoIP, it's because third-party VoIP is running on the high-speed internet service. This isn't Comcast's problem except from a business standpoint. They aren't selling VoIP data connections, they're selling HSD service meant to be used for Internet browsing first and foremost. If they're service isn't up to snuff for VoIP (or VPN, online gaming, ect) they only have to decide if they are going to lose enough business from it to justify fixing the problem.
There is a gap between what can be done with Internet access from a provider and what they actually support. VoIP is one of those items and always has been, which is why I think people are crazy for making a VoIP line their only phone service. It's a area where you have two distinct services that must work together when the companies that provide the parts are not proactively working together and in some cases are competitors. You have a situation where no one is ultimately responsible for getting it working and you're left holding the bag.
The FCC doesn't even recognize VoIP as a real phone service. You know there are legal requirements for uptime on phone services? Like if it goes down the provider has to restore dial tone within 2 days? That simply doesn't exist with VoIP (even dedicated VoIP from your cableco). But no one is putting any handle on VoIP companies in their marketing, so they continue to act like it's exactly the same as a regular phone and people don't find out the truth till it goes down and they get quoted a week+ repair time frame.
The only downside: Everyone with a basic analog cable subscription would need a converter box.
And everyone with a TiVo Series 2 DT would instead have an expensive paperweight. There's no dual-tuner converter box for those.
Visual IRC: Fast. Powerful. Free.