Slashdot Mirror
Net Neutrality Opponent Calls Google a "Bandwidth Hog"
Adrian Lopez writes "According to PC World, an analyst with ties to the telecom industry — in a baseless attack on the concept of Net Neutrality — has accused Google Inc. of being a bandwidth hog. Quoting: '"Internet connections could be more affordable for everyone, if Google paid its fair share of the Internet's cost," wrote Cleland in the report. "It is ironic that Google, the largest user of Internet capacity pays the least relatively to fund the Internet's cost; it is even more ironic that the company poised to profit more than any other from more broadband deployment, expects the American taxpayer to pick up its skyrocketing bandwidth tab."' Google responded on their public policy blog, citing 'significant methodological and factual errors that undermine his report's conclusions.' Ars Technica highlighted some of Cleland's faulty reasoning as well."
If my server logs are any indication, then this is probably true. They spent 6 months hitting my server every 2 seconds at one point.
-1 Uncomfortable Truth
Phone companies are one of the single greatest causes of people talking. More people talking means more oxygen consumption. And the externalities of all that poisonous CO2 exhalation.
Phone companies are literally living off our dimes. And the Amazon and Sting and Al Gore don't even get a cut.
Google may be the largest provider of services and the aggregate use the most bandwidth, but the users are the users- what a thought.
If they think google is getting their bandwidth too cheap why aren't they just charging more?
...but Google apparently doesn't want me to.
The post loads perfectly. The post finishes loading. The post displays perfectly in its entirety. I start reading it. Then, after 15 seconds, it disappears and is replaced with a message saying "Your request took too long to complete. This is typically just a temporary error due to high network traffic or heavy usage of Blogger."
Thanks, Google. I love an application that claims there's an error when nothing's wrong, and displays the message in such a way that I can't even read the article that was displaying perfectly until you replaced it with your error message. Says a lot for the quality of Blogger.
After all, rich people and companies don't pay their "proportional" share of the cost of government, even though they benefit from it.
Hey, I just conducted a study and found out that my interconnect connection would be more affordable if Scott Cleland payed for my bandwidth costs.
There oughta be a law!
And just to be clear, is Scott Cleland proposing that well-run companies should be transferring their profits to all poorly-run companies, or just the poorly-run telecoms?
Is customers paying for individual bandwith, not the ISPs. And I will pay a isp to get a link and not use then? stupid, really stupid.
Religion: The greatest weapon of mass destruction of all time
Your customers who use google are already paying their fair share. Any bandwidth used by google for it's indexing is purchased from its ISP. The telcos just want to double dip.
Google is a content provider after all, maybe they should start charging AT&T. People pay to connect to the internet for the content, not to say they can connect to the AT&T network.
I'm not sure technical arguments are really necessary to demonstrate this as bunk. Google's services add a lot of value to a consumer's bandwidth. I would wager that their contributions exceed their consumption.
I use google, I use it because I want to or rather because the other search engines aren't that good. Here's the thing : I pay my freaking internet bills! Just for the concept of being able to use any web site I'd like. So the ISPs are already getting my money for google hits. Not only that, but google also pays for its bandwidth to an ISP already. This sounds like lame excuses 2.0 with a demagogic twist. How about you fuck off?
Copyright infringement is "piracy" in the same way DRM is "consumer rape"
"It is ironic that Google, the largest user of Internet capacity pays the least relatively to fund the Internet's cost
So how much does Google pay for it's usage of the Internet?
There's a local company offering a 1.5TB external drive when you order a 2mbit or faster internet connection. Since few people are likely to fill the drive up with holiday photos, the use for this combo is obvious.
ISPs and digital storage manufacturers benefit from online piracy. I'd wager the profits are greater than the loss the content producers face, and are of net benefit to the global economy.
But, my perspective on the issue is skewed. I've been a pirate since I was 7. :p
All rites reversed 2010
I was under the impression that Google purchased business/carrier Internet facilities (OC3/OC12/OC48/OC192 and Gig-E interconnects) just like any other major business.
Unlike shared residential services such as cable/DSL/FIOS, these are dedicated facilities. They are paying for all their bandwidth, whether they use it or not.
How can they be "hogging" what they are paying for?
Can You Say Linux? I Knew That You Could.
It is ironic that Google, the largest user of Internet capacity pays the least relatively to fund the Internet's cost
Economy of scale is not ironic. It is a appropriate, and makes sense to anyone who understands basic economics.
The consumers use bandwidth, and it is the consumers who should shoulder a significant cost of the bandwidth. Google, et al, need to pay for the redundant lines that connect their facility. It is true that due to different usage patterns, some consumer will pay out of proportion. It is also true that some taxpayers will pay for something they do not use. But such is life.
Let's say that I am in the city. I drive like 20 or 20 miles a day, and the roads I do use are well traveled and largely cheap surface roads. Then why am I paying taxes and high gas taxes to subsidize the suburbanites excessive travel and wear and tear on the roads? Well, for one thing I do not want them in the city. Second, i need them in the city to serve me. I am likely paying out of proportion of my direct use, but not me total use.
It is the same thing with taxes. Suppose I am in the top 25% of the income. I likely am part of the group that pays a huge percentage of the nations taxes, maybe even in excess of the proportion of money that I earn. This is caused by the fact that the bottom third of the wage earners pay almost no taxes. A family earning 30K, after deductions, maybe a token couple thousand. That is, of course, because we all get a deduction basic living expenses, just like business only pays on profit, actual humans pay taxes only on their excess income, and the more money you make, the more actual excess income you have. It is an observable that 50% of the population have almost no excess income, while, when on reaches the 10 20% of the wage earners, excess income becomes the majority.
On one hand this is bad, as it means I pay higher taxes. OTOH, this allows us to keep wages low, as it is possible to pay barely enough to keep a family together. If everyone had to pay, say, 10%, then many family might double their tax bill, which might force them to ask for raises, which they would need to have to survive. This might mean that a couple who had been earning $9 an hour each, might now need to ask for $10, which might be more than a business could afford without increasing costs.And since business do not increase cost proportionately, such an increase could end up costing more overall. Or at least this is the conservative arguments.
So, fairness is not really crap, but fairness is dangerous, as people will inevitable skew the facts to make themselves the victims.
"She's a scientist and a lesbian. She's not going to let it slide." Orphan Black
for, if they had PAID their share of the bargain and INVESTED the HUGE profits they made from OVERSELLING bandwith for all those years, there would be NO issue about bandwidth anywhere. actually, there arent any issues about bandwidth at all. there is a SUPPOSED problem about 'internet breaking down due to bandwidth' in united states only for around 3 years now, and nothing happened.
considering all the pointers at hand, i have decided that the supposed 'an analyst with ties to the telecom industry' is either a non person that is invented to propagate a shitty corporate agenda, or a corporate shill to attempt justifying controlling internet, YET AGAIN.
you americans are WAY too much tolerant of this 'lobbying' thing. way too much.
Read radical news here
The people who go to Google are the hogs. If your pricing model doesn't take into consideration your consumer's usage patterns, then FAIL.
I want to delete my account but Slashdot doesn't allow it.
They are overcharging the customers. End of story. They are gluttonous swine. Adelphia and Comcast are perfect examples.
I don't think Google is a "push" provider. Google does not use any bandwidth. It is the individuals consuming Google's services that are using the bandwidth and they are paying for it.
And if they don't want google to crawl google unlike a few others actually obeys the robots.txt file.
i thought once I was found, but it was only a dream.
This is the same company who had a monopoly on the US phone network, and only allowed AT&T phones to connect to it (for 'network stability' reasons), and only allowed AT&T answering machines.. make a buck on the line, and then make a buck selling the stuff that connects to it; Sounds like their still trying to play the same game :)
There's an awesome response on the googleblog which makes a good read:
http://googlepublicpolicy.blogspot.com/2008/12/response-to-phone-companies-google.html
the dipshit who propagated and placed this article in news outlets represent corporate asswipes who want to control the internet through charging whomever they want, whatever they want, so they can make internet to another cable tv network.
Read radical news here
network neutrality assures the equality of opportunity in business in internet world. the FAIR and FREE market that conservatives so blabber about.
if there isnt network neutrality, entrepreneurship will become impossible without amassing huge capital before attempting anything. for, noone will be able to set up a web service with 3-4 figure bucks and then proceed to become millionaires.
the big money who enjoyed ruling the world is annoyed with this prospect.
if the number of entrepreneurs rising to billionaires remain low, they can be controlled, and assimilated into the present 'club'. if it takes huge capital to establish big business, it further assures that noone that is contrary to your world view or interests will easily accrue capital and become a contender.
internet breaks this down. since 90s, innumerable people became powerful and influential through the newfound wealth through internet. and thats not only google, yahoo and similar, there are lots of small fish that do hundreds of millions $ of business, and annoy these big capital, because they dont share their views or accept the control they try to exert.
so basically this is an attempt by those people to assure that it stays the same -> someone is making big buck and amassing capital, but doesnt seem to 'fit in' with your club ? why, just call your chum and have them charge exorbitant rates for bandwidth, and sink their business. you dont even need a call actually, club does it itself, if we know anything about business history of the last 50 years.
therefore conservatives HAVE to be FOR network neutrality, if they ever want a FREE and FAIR MARKET in which people can actually have a chance of making money.
Read radical news here
I don't see a way to use robots.txt to limit the number of crawler hits per interval other than just denying it. So you can block it, but that's undesirable if you want people to find it. It's also undesirable to have a robot hit your site every two seconds if ShieldW0lf is saying the truth, but robots.txt only address it in a simplistic allow / disallow.
the keyword. whenever conservative corporate shill tries to fool people into believing them, they use this keyword 'taxpayer'.
whoooooooooooo. taxpayer. they are gonna raise my taxes so i should oppose ! for i dont want to pay bucks from my pocket !!!
yet, it is a total display of STUPIDITY, since there are NO relation in between taxes and the fees isps charge their customers. its THEIR own greed, but they want to fool people to otherwise.
speaking of which, i would like to ask this expert, WHY was isp industry OVERSELLING the bandwidth they did NOT have for the last 15 years ?
i dont know about america, but in many places of the world, selling something you dont have is considered a FRAUD.
Read radical news here
Google is making money, we want some of it, so if you help us get some of that money, we might give some of it to you consumers.. maybe.. if we're really nice generous companies.
a 'hit' doesnt require the bandwidth you are complaining about. its just a request. and first request is robots.txt. if you deny all in robots.txt, nothing happens.
its WAY STUPID to be complaining about a 4 k text file request creating any kind of load on a server.
Read radical news here
It isn't every day that the glossies "get it", or even pay attention.
Next: the Wall Street Journal (;-))
--dave
davecb@spamcop.net
Crawl-delay directive
Several major crawlers support a Crawl-delay parameter, set to the number of seconds to wait between successive requests to the same server: [1] [2]
User-agent: *
Crawl-delay: 10
I don't see a way to use robots.txt to limit the number of crawler hits per interval other than just denying it. So you can block it, but that's undesirable if you want people to find it. It's also undesirable to have a robot hit your site every two seconds if ShieldW0lf is saying the truth, but robots.txt only address it in a simplistic allow / disallow.
I'm not sure if any of the other providers implement this, but Google does. SiteMaps
Lets you specify how often to update certain content, what URLs to block. It's a more advanced robots.txt.
There's no place like
Crawl-delay directive
Several major crawlers support a Crawl-delay parameter, set to the number of seconds to wait between successive requests to the same server: [1] [2]
User-agent: *
Crawl-delay: 10
Further, not only do the Google crawlers obey the robots.txt described above (or other standards for robot exclusion), they also use HTTP's if-modified-since to make a conditional request. The file is only returned to the crawler if it has been changed. That saves a lot of time and bandwidth.
PC World will also lose out if double-dipping is allowed.
Beta is broken and the link to classic doesn't work. Stop wasting our time or there won't be anybody left here.
I don't see a way to use robots.txt to limit the number of crawler hits per interval other than just denying it. So you can block it, but that's undesirable if you want people to find it. It's also undesirable to have a robot hit your site every two seconds if ShieldW0lf is saying the truth, but robots.txt only address it in a simplistic allow / disallow.
Aside from saying the obvious "Well, keep robots.txt on from one crawling untill you are ready to get next one", as robots is really quite easy to generate dynamically, any webmaster facing the problem should be able to find out that google has google.com/webmasters/tools where you can log in, tell Google that some site is yours (verified by adding certain file there for a moment) and set them to crawl slower. :)
Aside from that... I would pay a hefty sum to get a domain crawled every 2 seconds. Bandwith would cost but the possibilities would be endless... Having all your newsposts appear immediatelly after written would mean that whenever something important appears in the news and such you can write something quickly and get all the people googling it to visit your site for a few hours...
It's my understanding that the robots.txt file is the first grabbed by a webspider, right? If you don't want anything crawling your site, just disallow /, no?
Bla bla its not fair, bla bla bla.
This isn't worth even printing, let alone having a discussion over..
---- Booth was a patriot ----
Doesn't Google PAY for the bandwidth it uses? Don't Google's users PAY for the bandwidth they use? If not, why not, and if so how is this not just a blatant attempt to double-dip?
So ISPs are losing money because of Google? Fine. They should do what Sprint did and block all access to Google. Let their customers use the "Internet" of the ISPs email and the ISPs news. Let's see how long that lasts.
ISPs need to wake up and realize that people don't want their email, don't want their home pages, don't want their internet "content", and almost universally don't want anything the ISP provides except a pipe to the outside world.
I was unaware of this directive, and it would be a good setting for me to use.
The problem with quotes on the internet, is that nobody bothers to check their veracity. -- Abraham Lincoln
This is about the stupidest argument the ISP's have come up with yet. No one would use the internet if there was nothing on it. I bet an argument they have not considered is that they should be paying Google for their users having access to their services. Seems like the ISP's are getting a good deal as is.
The Telcos are lying to us (a lie of omission): They carefully avoid estimating the reduction in total bandwidth consumed due to the optimization that search engines provide. Search engines serve as a repository of index information used to optimize our access to internet services and products. The net effect is reduced resource utilization.
Earth to telcos: Google is an example of a service that increases the value of the internet, which drives our willingness to pay for it. I have been an internet user since modem dialup days. My use of the service has increased during the last 18 years because it provides value. Google improves that value. It's a big win for the telcos and service providers, and they are trying to prevent us from recognizing that fact.
Free bandwidth indeed! Google pays for every bit of their bandwidth just like everyone else, probably with a bulk discount just like every other customer of a service with a predictable and large utilization.
All those rickrolls GoogTube serves each day would come back to haunt 'em...
np: Benni Hemm Hemm - Jag Tyclate Hon Sa LÃnnlÃi (Ein à Leyni)
"I'm not anti-anything, I'm anti-everything, it fits better." - Sole
There are a lot of things you can do. If Googlebot is using too much bandwidth, you could easily (man tc) add an outbound QoS limit to your webservers.
http://www.google.com/search?q=googlebot+IP+range
If you're unable to do this, there is the GoogleBot webmaster tools that let you manage your hit rate.
http://www.google.com/support/webmasters
They think Google is being unfair, block it outright. Why haven't they done this? Because they know that 1) They're in the wrong and 2) They would lose just about all of their customers You know there's more to it when the simple and obvious solution is not employed, and then forgotten about. Google is what the consumers want, the consumers have paid for their internet connection, as have Google, end of story? Hah I wish.
Can I leave this box empty?
That man in the middle would still be selling dial-up if it weren't for the Google offerings that consumers want, specifically Youtube. There are others too such as Hulu and Veoh and even the major TV networks' sites that stream episodes on demand, plus all the Shoutcast streaming radio stations.
What this is really about is whether the ISPs still have common carrier status, and how that conflicts with their vertical service integration for services like TV and phone. These ISPs are charging for what is either free or for less money elsewhere.
The solution is very simple. The FCC grants the ability for these anti-net-neutrality ISPs to charge whatever they like for whatever content they choose to carry over their networks, in exchange for the return of every government subsidy and grant given over the last five decades, with interest, in addition to the rescission of their common carrier status. The government can then take that money and give it to companies that will act like common carriers and build net-neutral data infrastructure.
You so totally misunderstood my post.
You're proposong all-or-nothing, I asked for something in between. Was it really that hard to understand?
Apple makes shiny, flimsy hardware that's designed to appeal to faggots who work in interior decorating and which breaks as soon as you breathe on it or hit it with a hammer.
its WAY STUPID to be complaining about a 4 k text file request creating any kind of load on a server.
I think it's WAY STUPID to misunderstand what I said and call me stupid based on your misunderstanding of what I said.
I'm not saying anyone complaining about Google hitting robots.txt. It wasn't just about that, it was about Google grabbing a real page somewhere on the site, every two seconds. Some pages on sites really are intensive to generate, it's good to crawl and have it available, but not every two seconds. Once a day should have been fine.
I was also saying that the suggested complete blocking was undesirable, an all-or-nothing approach, such as your deny all suggestion is not an intelligent answer to the problem.
Switched to who? Where I live, I have a choice of exactly two broadband providers, both of which are lacking in the customer service department, and both of who have business reasons for not supporting net neutrality.
Perhaps the government should start anti-trust investigations to see if there could be anything done to introduce more competition into the market.
Capitalism starts to fall down where there are too few capitalists (i.e., consolidation), not when there "too many".
Do you have proof that google crawled your page every 2 seconds? I've heard nothing of this.
I don't, I was going from what ShieldW0lf said happened with his(?) site.
Son, I know it feels like this Internet thing is awfully new, but it isn't.
As someone who was here before Google, let me tell you: There's a VERY good reason they're as big as they are. Without Google, the Internet is very small and useless.
I had the misfortune of trying Live Search a couple days ago, and it was very much like the old Internet: Enter in a search term, and good luck having the most useful results on the first page. You'll spend a month searching and finally find what Google found and placed on the first page.
Nobody who was there back then wants to live without Google.
It's been a long time.
Actually, it makes sense. It's cheaper to provide one big pipe to a company than lots of tiny ones to thousands of end-users, even though the total of all the tiny pipes' bandwidths matches that of the big pipe. But if this is the case, then the cost is primarily due to all these end-users wanting to connect to the Internet, not Google's bandwidth use. So the obvious solution to reduce costs is to eliminate end-users. Simple!
if a company actually HAS the resources to meet up the demands of the product/service they sold, let users use up their bandwidth all they want. when they are over their quota, you can sell them bigger packages.
after all, isnt this the whole point of commerce ? selling more products so you can make more profit ?
it ends up in the isp. they have committed more resources than they could afford, SOLD them from high prices up until today, knowing that noone would use them, and hence setting the bar of quota higher and higher, and now they are in the red. why ? well, they OVERSOLD their resources. didnt act responsibly. just like the bastards who had brought the credit crisis upon the world.
Read radical news here
Then why don't the ISPs publicize this and offer consumer home connections that are not oversubscribed and charge a higher price for it, while continuing to offer hit or miss oversubscribed connections at the current rates? Those who are happy with sometimes slow traffic can stick with it, and the rest of us can move up to the non-oversubscribed lines. And our additional payments should give them money to invest in more infrastructure.
I actually probably couldn't afford this idea myself right now due to working my way out of debt and getting ready for our first child, but I can certainly see a day when I would be ready to move up a more expensive, unshared connection if one was actually an option.
Beware of bugs in the above code; I have only proved it correct, not tried it.
The net neutrality debate isn't just about Google, it's about every content provider on the internet. Including Apple, Amazon, Microsoft, and Slashdot.
Nothing wrong with google getting a big fat share of the pie.
It has more need of calories to burn.
People use google ALL THE TIME, so it's like the queen bee in a hive. Give the most important things the most resources.
ISPs are like movie cinemas that never have to pay royalites for the content they show. So now they are saying the biggest stuidios should pay up because they are filling up their seats and customers are complaining about the queues...
So anti-net neutrality doesn't look good. It gets worse when you have one huge studio that arguably drives the industry. Sure it's clogs your intertubes, but your business depends on it...
After logging in slashdot still does not take you back to the page you were on. It's been that way for 20 years.
So the ISP, can't get away with charging the users more, they obviously can't charge google any more, they will just go somewhere else .
Seems to me like everyone is already paying for their data.
They just want their cake and they want to eat it as well.
I guess they have seen the music/video industry buy legislation they want some too
Google's users do.
Google happens to be the most popular search site, so the traffic to its servers is high. If someone else builds a service that users perceive as being as good, traffic will go there instead.
If it wasn't for all these users, the telecoms would have a really smooth running Internet.
Have gnu, will travel.
In the US, ISPs do not have, and never have had, "common carrier status".
I haven't RTFA but...it's very simple.
1) Google has their own backbone. They bought dark fiber, put their equipment on it, and run packets over it. They don't owe jack shit for this. They bought fiber so they wouldn't have to pay some inflated monthly fee to another service provider.
2) Google also pays for transit when their packets exit Google's network.
#2 covers it. Google pays fees, that's all they owe. Some other random ISP isn't owed a damned thing, if they are having problems they can either do #1 (light up some dark fiber to keep more traffic "on-net"), or #2 suck it up and pay for a larger pipe.
I wish Google would hit on me. :(
How many freakin times do we have to play the "you sell pipes now leave me alone" game? Just because companyX found a way to make $1000 off your $50 a month pipe dosent mean they owe you $950. Get over it.
And... will John Q Public ever figure out that the logic behind using up all the internet for free is a lie of biblical proportions?
They did use up all that money for investing in infrastructure to instrument the infrastructure for the convenience of the NSA.
From their point of view, it was a better investment -- since they've cooperated with the Government, they're in a better position to benefit from their legislation.
Now, that's NOTHING even close to a Free Market -- but here we are.
Technology -- No Place For Wimps! Grateful Dead and Jerry Garcia Chatroom -- http://www.wemissjerry.org
Good! This is a good discussion to have, even if started by a paid hack. The gamerz and script kiddies here need to realize that the general public cannot go on paying for their WoW patch downloads and pirated video downloads and YouTubes and whatnot. Bandwidth is a scarce resource and has to be paid for, just like electricity -- come to think of it, it *is* electricity at the end of the day. I fail to see why telecoms have to shoulder more of the load, or "the government" (i.e. taxpayers) have to pay for people's Bittorent habit. If there isn't a feasible pay-to-play model, then let's look at how Google might pay more of its share for the *traffic*, not just the path to the backbone. After all, those websites that it says are all user-paid and on somebody else's dimes are hosting their AdSense, duh, so they are getting paid without paying for the cost of doing business. I see an enormous amount of aggressive insolence on Slashdot by geeks who don't seem to grasp that we are not required endlessly to pay for their energy consumption. The Internet is a form of energy, not merely a media. The entire Net Neutrality fandango should be renamed Net Congestion. It's not about free speech, it's about *consumption*. http://secondthoughts.typepad.com/second_thoughts/2008/12/net-congestion.html
Of course they are doing this because Google is the biggest and now they're adding more HD content to Youtube so users can use a higher % of their PAID bandwidth but still. Google is one of the cleanest sites out there. If everyone got their content from a site that had flash adverts everywhere and an ungodly amount of pictures then we would have a serious problem from wasted bandwidth.
I'm sure Google pays to connect to the Backbone and I pay to connect to the backbone. Why should Google pay to allow me to pay to connect to the Google section of the Backbone (wow that was confusing).
Common Sense
The Telcos and ISPs know damn well that 99% of users will cluelessly blame Google.
That's all fine and well, but in my experience, most laptops will break as soon as you hit them with a hammer, not just apple laptops.
Just saying...
http://www.zombieapocalypse.tv/
A sitemap, which Google will follow, will give the crawler an indication of how often your content changes. If you don't want Google hitting you often, set a high change frequency.
Why do vehicular analogies keep popping up in ICT/Net related discussions??
The net neutrality arguement is incredibly simple;
On the one hand, net neutrality, bandwidth is sold as a service and no-one cares how that bandwidth is consumed; but, assumedly, the CONSUMERS (thats you, the people at home, with a net connection) of that bandwidth will end up paying for the share they use (if the ISPs are clever).
The counter arguement, from the telcos is similar but different, observe; bandwidth is sold as a service, and the telco has determined that by throttling the data rate FROM a certain location, they can sell that bandwidth twice; once to the CONSUMER (who pays for bandwidth used) and once to the PROVIDER (who pays their protection so the mob dont come around and smash their internet stores windows)...
In both arguements the gross ammount of bandwidth will be driven (assuming well run businesses) by the need to satisfy the CONSUMER DEMAND and invested in accordingly. In the latter arguement the people doing the investing also get to roll naked in giant piles of cash with $10k hookers :)
Frankly the former arguement sounds logical, the basis of good service business competition and the latter sounds like organised crime. But maybe Im just to simple to understand the logic of the latter arguement.
err!
jak.
That's almost as bad as saying something bad about Apple... let the flamewars begin.
Actually it's cool to attack Apple now. When the first iPhone was released, it was given a parade here. When a mysterious line formed in front of an Apple store near the 3G's release, suddenly it wasn't cool anymore and retro-active history kicked in. Oh, funny thing, those people were just in line for the next shipment of phones, not because of some rumor.
"I like to lick butts!" by MobileTatsu-NJG (#32700246) (Score:5, Informative)
No, but the telcos have and some of them turned into ISPs too.
DRM: Terminator crops for your mind!
A misuse of the term "ironic" and no one has mentioned Alanis Morissette yet? Where is this world heading to?
The filesystem is the package manager
This from companies that recieved *billions* in govt. subsidy to ensure a sound fiber-optic infrastructure that was never implemented.
The fact that Google forces YouTube visitors to download the same videos again and again, even if they have the video cached from a previous visit, would tend to undermine their arguments for NN.
Might want to fix that soon before it's dragged out to make them look foolish.
Here's a tip: a quick look with Wireshark shows that the embedded flv player is requesting the flv object from the server using a query string generated with random values each time it is initiated (even on history navigation). Our browsers then treat this as a request for a new object to be cached separately from the content already collected for that location. On a fast connection you can turn a 25MB video into 200MBs just by clicking back and forward a few times.
Please Google, remove the random values and let us use our caches again - not all of us have unlimited broadband, you know.
He's a bitch of comcast, microsoft et al. He's pathetic.
You can lead a man with reason but you can't make him think.
Do you have proof that google crawled your page every 2 seconds? I've heard nothing of this.
They've slowed down a bit, but yes... Here's the top few entries of my "Google Recent Visit Report"
2008-12-07 23:04:05.887876-04 /svensk/viewartpiece.php?artpieceid=73&EnvLID=1 66.249.70.156
2008-12-07 23:04:00.031226-04 /pyccko/viewjournalentry.php?entryid=9&EnvLID=1 66.249.70.156
2008-12-07 23:03:53.482872-04 /arabic/viewartpiece.php?artpieceid=79&EnvLID=1 66.249.70.156
2008-12-07 23:03:46.776078-04 /slovene/viewartpiece.php?artpieceid=80&EnvLID=1 66.249.70.156
2008-12-07 23:03:42.534327-04 /svensk/viewjournalentry.php?entryid=22&EnvLID=1 66.249.70.156
2008-12-07 23:03:33.373614-04 /latvian/viewartpiece.php?artpieceid=44&EnvLID=1 66.249.70.156
2008-12-07 23:03:26.681602-04 /pyccko/viewartpiece.php?artpieceid=124&EnvLID=1 66.249.70.156
2008-12-07 23:03:20.461428-04 /arabic/viewartpiece.php?artpieceid=63&EnvLID=1 66.249.70.156
2008-12-07 23:03:13.759767-04 /polski/viewartpiece.php?artpieceid=59&EnvLID=1 66.249.70.156
2008-12-07 23:03:07.066574-04 /polski/viewartpiece.php?artpieceid=67&EnvLID=1 66.249.70.156
2008-12-07 23:03:00.840937-04 /svensk/viewartpiece.php?artpieceid=42&EnvLID=1 66.249.70.156
2008-12-07 23:02:57.742184-04 /pyccko/viewartpiece.php?artpieceid=76&EnvLID=1 66.249.70.156
2008-12-07 23:02:47.4434-04 /hebrew/viewartpiece.php?artpieceid=81&EnvLID=1 66.249.70.156
2008-12-07 23:02:40.741955-04 /svensk/viewartpiece.php?artpieceid=59&EnvLID=1 66.249.70.156
2008-12-07 23:02:34.549513-04 /korean/viewartpiece.php?artpieceid=71&EnvLID=1 66.249.70.156
If you choose you can believe I've made those up, but I haven't. This is running out of my home office, and is still in beta, not being promoted at all.
-1 Uncomfortable Truth
Even accepting your argument (which is definitely debatable), how does this justify the *ISPs* charging Google a fee?
So false it hurts.
Why have 0 ISPs been charged with child pornography and copyright infringement? Same reason regular Telcos are not charged when a mobster uses the phone to call in a hit.
Dear all of Slashdot:
ISPs do not have common carrier status. Reformulate your argument in light of this fact.
Thanks,
Anonymous.
It is ironic that Google, the largest user of Internet capacity pays the least relatively to fund the Internet's cost; it is even more ironic that the company poised to profit more than any other from more broadband deployment, expects the American taxpayer to pick up its skyrocketing bandwidth tab
It is ironic that Comcast & AT&T which lay their lines with tax-payer money received as subsidies go ahead and charge the same tax payers a series of Fake Federal Charges on EVERY single bill.
It is ironic that comcast expects the american tax payer to pick up comcast's operating expenses while the profits go straight to fund the CEO's latest Carribbean purchases.
It is furthermore ironic that comcast continues to receive and demand federal subsidies towards providing rural connections, while refusing to provide such connections to any village and town AND suing said village that tries to provide its own connection.
"Doing what i can, with what i have." ~ Burt Gummer
Electricity supplies could be more affordable for everyone, if TV Broadcasters paid their fair share of the Generator's cost," wrote Clitland in the report. "It is ironic that Fox, who'se viewers are the largest user of generating capacity pays the least relatively to fund the Power company's cost; it is even more ironic that the company poised to profit more than any other from more HDTV deployment, expects the American taxpayer to pick up its skyrocketing electricity tab."'
The success of arguments like those claiming some application is "hogging" the Internet because hundreds of millions of people use it rely on people not knowing how the Internet works. I pay my ISP for access. They, in turn, pay others for their connections. The access for everyone is ALREADY paid for.....and the consequent traffic is what we are all paying for. Especially here in New Zealand where we have data quotas or caps (plan dependent).....and the more we use, the more we pay. No one can claim I'm not paying the freight....and Google needs to be tapped for cash, too. I'm sure they pay heaps of money for their many connections worldwide.
Only boring people are ever bored.
I believe a similar argument was used against the BBC when it first brought out Iplayer, the big difference however was it's use of a peer to peer arangement.
None of the urls you just posted are the same. Are you complaining that google indexes your site ? To do that they have to visit the various separate pages. Maybe if you didn't run everything through a couple of PHP scripts, they wouldn't put so much load on your server.
Yahoo has been guilty of large amount of spidering on my sites, but google is only once every week or so. But then I don't use php so much. If you don't like google doing what you see, then script a robots.txt file that changes according to the day/date whatever. Crontab might be your friend.
Zip! Nada! In fact when they send packets over my network connection they pay a *negative* amount! If you want to find the real bandwidth hogs, it is the Telcos.
All those people who browse http://localhost/ at ridiculous speeds without paying a cent are pretty greedy too.
... Back in the real world there is central node called "The Internet" that has to route all traffic. If you want to connect to someones network they may ask for money, if they want to connect to your network, you can ask them for money. How much money is up to you and them.
Our engineers are stealing all your ideas and putting them into pagerank. Mwahahahaha!
If the Tele Act of 96 was to get rural lines to 56k, then "FAIL." I'm a rural internet user (since 1984!). My line is 26.4 kbps. Isn't it 2008? Or have I been asleep?
Yes. Most of us have maybe two ISPs to choose from; the very lucky may have ten. But every Internet user has access to millions of web pages.
Hmmmm, which competitive market will produce better content?
Non-neutral ISPs will destroy their customers' motivation to use the Internet in the first place.
Must be nice. I have ZERO broadband ISPs. So my choices are...?
Telcoms, you need to get into your head that you are an utility. Run it like one or get the hell out of the business. You can't have "protection" from competition and at the same time complain about free market enterprises. And stop fighting against local governments across the country who want to provide the same utility at a much lower cost, lower headaches, and lower whining.
I'm not saying anyone complaining about Google hitting robots.txt. It wasn't just about that, it was about Google grabbing a real page somewhere on the site, every two seconds. Some pages on sites really are intensive to generate, it's good to crawl and have it available, but not every two seconds. Once a day should have been fine.
Fix your cache. Google doesn't submit forms, so everything they crawl should be cachable.
I used to be the senior tech guy for a high bandwidth company, so I have a little insight into this.
Someone else wrote a little bit about the differences in bandwidth cost associated with larger lines, which I can confirm.
As a little customer (residential or small business) you pay a lot more for bandwidth.
As a "little" customer just using 45Mb/s (T3) we paid a particular rate. Sorry, it's been a long time, and I don't remember how much. When we went up to 100Mb/s lines in datacenters, our price dropped to somewhere in the ballpark of $100/Mb/s. When we went up to GigE circuits, our port charges went up, but our bandwidth prices dropped. If I remember correctly, they were down to $15/Mb/s. That was with several GigE circuits, all using lots of bandwidth.
I don't have specifics on Google's network, but once you reach a particular size, you get to start playing in the realm of a provider, which means you can make your own peering arrangements, leasing dark fiber, etc, etc, etc.
Both sides always pay for the bandwidth. Effectively, everything is paid for twice.
I, as a Google user, am paying for bandwidth to be able to access Google.
I, as a webhost, am paying when Google spiders my hosted sites.
Google, as a webhost, is paying for people to access their site.
Google, as a "client", is paying when their software spiders.
Now, is it fair to pick Google out of the thousands of spiders that are crawling daily, and say that they owe based on that? Not really. Their spiders do crawl. They aren't terribly aggressively compared to many. Some download everything over and over. In the past, I've had to block some crawlers. Some are stupid, and will saturate a 100Mb/s line. I've never had one saturate a GigE line, but in time I'm sure it'll happen.
The company I worked for that had multiple GigE lines suffered more from people doing "wget -r example.com". It wasn't just the occasional user, it was many users, frequently. There were hundreds of gigs of data that they could recursively spider on some sites. Effected webmasters and webhosts should put in countermeasures to protect against this. Our primary one identified that a user was pulling more than normal. My "more than normal" was 10x what should be pulled by a customer on a good high speed connection. It would throttle just their traffic. Should they continue, it would start throwing "403" pages to them. Should that not help, their traffic would be blackholed.
Oddly enough, with millions of daily users, those countermeasures never drew complaints. The same millions of daily users would complain about anything. We'd receive emails that someone on a dialup in the arctic circle (just a random place far away from good connectivity) would complain if the page didn't render within 10 seconds.
The original document was funded by a couple ISP's. They want more money. They always do. They simply can't charge the outside connection, but they will reflect it back to their own customers.
If they charge Google, that opens up a whole can of unmanageable worms. So Google gets a bill from every ISP in the world for transit costs. Now they have to bill every company online with services that pull. One of my sites now pulls RSS feeds from hundreds of other sites, and aggregate them online on our site. Should my company be responsible for any more than my bandwidth bills? No. It's up to the other provider to make a decision based on their business on what to do. Do they stop allowing me to pull the RSS feed? Probably not. It in turn brings traffic to their site, and they make more advertising money. Some just shut down their feed over time. Those usually seem to be for other reasons. I've never been asked to stop pulling any RSS feed, and we've been doing this for years.
Serious? Seriousness is well above my pay grade.
.
No, satellite doesn't work on my steep north-facing lot. And a dish atop a 60 foot pole is not practical for several reasons I won't bother to re-iterate.
Wireless? Not even a cell phone worked where I live until just THIS YEAR. I'm hoping eventually some actual wireless internet connectivity will arrive (current estimates indicate 2012).
"Google offerings that consumers want, specifically Youtube."
Seeing as my friend started YouTube and it was popular before Google bought them I wouldn't give too much credit to Google.
“Common sense is not so common.” — Voltaire
Does anybody have any idea how much traffic is generated by every Microsoft PC having to download patches and service packs? It seems to me that if MS were able to deliver a secure and stable OS (and apps) the global network traffic would significantly drop. Not only in first instance by the patch transfers disappearing, but also in second instance by reducing the number of hijacked systems that are sending me SPAM! Just my two cents...
It's a giant game of chicken between the ISPs. They all know that they can't individually block Google or their users would raise Hell and try to move to a different ISP, and ISPs that didn't block Google would win BIG.
What they are really trying to do is hide part of the expense of being an ISP from the client users (homes and small businesses) who can more easily change their ISP to content providers who would be forced to pay to reach the people.
If this happens you'll end up with nearly free Internet service in competitive locations, but the barrier for entry for content providers will become impossibly high.
Well, aside from User-agent, Disallow I don't think any of the other commands are part of the official standard. Major search places like Yahoo/Google seems to actually care about the effects of searching so they follow those extensions AFAIK.
Then again, there is no law that says a bot has to follow robots.txt, but perhaps that myspace will change that.
Google is one of the only internet companies big enough to call these criminals out on this kind of crap. "We're hogging bandwidth? Fine. We'll null route all your customers to save your precious bandwidth and see how long they remain your customers."
robots.txt can't do it, but google's webmaster tools allow you to configure the crawl rate. Just set it to low if you want slower crawling. :)
Throw the bums out!