Unique Visitors = 1/10th of Unique IPs?

Max Fomitchev submitted a little blog entry where he proposes that the ratio of unique IPs to actual unique users is 10:1. This flies in the face of the numbers you usually see attached to these sorts of things. I'm not sure about the logic he uses to come up with these numbers either.

Comments

10 was arbitrary

[op12]

The 10 was a hypothetical...the only point was that you can't trust the number of recurring visitors that a site reports because they users come back with a different IP (obvious) and get counted twice. Couldn't one use cookies and IPs in combination to get a better gauge? The IP may change but the cookie would not. Sure some may delete it, but it'll still improve accuracy at least a little bit.

Re:10 was arbitrary

[Meagermanx]

Exactly. How many people log into their webmail or their MySpace or whatever from their home computers, their laptops, their work computers, the library, their school's connection, etcetera? Almost everybody.

Re:10 was arbitrary

[MikeFM]

My experience is that a lot of users use cookie killing software that removes cookies every time the browser is closed or just reject cookies altogether. Also many users seem to use multiple browsers and computers even within small time periods. Counting unique visitors is really quite difficult. Still, if all site's play by the same rules on counting the number still has some meaning. Unique IP address within a given timeframe is probably a decent metric still.

Re:10 was arbitrary

[Penguin Programmer]

I don't know about anyone else, but my IP address at any one location (home, for example) very rarely changes. Occaisionally, perhaps, if my internet goes out for a long period of time and DHCP gives me a new one, but not very often.

However, many of us visit the same site (/. for example) from many different places, even in one day. Today, I've already visited /. from two different machines with different IP's, and I'll visit it from one or two more by the end of the day.

So, I'm not sure which is the bigger cause of repeat visitors with different IPs each time: changing dynamic IP addresses or visits from different machines entirely.

Re:10 was arbitrary

[From A Far Away Land]

statcounter.com uses a cookie to determine if someone has visited before, and so people who don't retain cookies throws off my blog's stats. I see visiotrs counted as two, when clearly they must be the same person. Dialup with DHCP addresses also hampers accurate counts about who is returning, if they've deleted the cookie. And computer labs or vast networks like Community Net in SK use proxy servers, as does AOL I think.

Re:10 was arbitrary

[Anonymous Coward]

Couldn't one use cookies and IPs in combination to get a better gauge?

No. What happens when I visit the website from work, or from a library? What happens when I reinstall Windows? What happens if I disable cookies? What happens when my girlfriend uses my computer to visit your site?

Web statistics can only be trusted to measure server state. To measure things like number of visitors, browser marketshare, etc, you need to obtain your data by other means, e.g. surveys. If you try and use what your server sees to obtain this information, then not only will your statistics be wildly skewed, but they'll be skewed by an unknowable amount, making them absolutely useless.

People who proudly trumpet web statistics are ignorant. Some of them are wilfully ignorant - i.e. they'll try and say that if the numbers can be biased in both directions (i.e. you can undercount and overcount), then they cancel each other out. I refuse to believe anybody could actually be that stupid, so I tend to assume they are just sticking their heads in the sand because they like to believe the numbers, no matter how mistaken they are.

Every time I post this, I always get a bunch of people modding me down and complaining because I offended them by questioning their sacred web statistics. "How dare you say I didn't get three dozen visitors to my blog last week!" I really don't understand this unhealthy attachment to meaningless numbers.

Re:10 was arbitrary

[Anonymous Coward]

And this doesn't factor in all the people connecting to the same website from behind a single IP (NAT). At my job for example, I know of at least 4 or 5 people who read slashdot from the same IP. So, in essence, what this guy just said is that unique IP's != unique visitors. Wow... brilliant. If anyone didn't come to the same conclusion after thinking for 5 seconds they're a moron.

Re:10 was arbitrary

[bbsguru]

Oh Puh-lease! EVERYONE knows the REAL number is 11.32019 per IP address. This is just silliness! There is no magic number that works everywhere; fuggidaboudit. If it matters that much to KNOW the real number of unique visitors, ask each one for a scan of their right thumbprint, and then create a database. I thought so.

Re:10 was arbitrary

[op12]

My experience is that a lot of users use cookie killing software that removes cookies every time the browser is closed or just reject cookies altogether.

My bet would be in the grand internetscape, this number of users is actually quite small and that most do not reject or remove cookies (at least not often).

However, the point was a better gauge, not a perfect one. Requiring login would resolve most issues of users from different locations or even multiple users on the same computer, but few people are going to like a website they can't browse anonymously.

I think it's just interesting data for site owners and nothing more. I wouldn't overanalyze some site touting 3000 unique visitors a day or something.

Re:10 was arbitrary

[Anonymous Crowhead]

Gah, no kidding. Everyone who has had to explain this to a PHB, please raise your hand. "But, they're the server logs! How can they be wrong? This PDF has been downloaded 1,000,000 times. Look - there are 1,000,000 lines in the log that have this pdf file in it. Broken up into chunks by the webserver? WTF does that mean? Nope, 1,000,000 downloads of this PDF. This is were we need to put the money. This is the most popular thing we have. People want this."

Ugh.

Re:10 was arbitrary

[MikeFM]

I've been surprised on the cookie issue. I thought nobody would block cookies but then I've had to resolve a number of issues with users who do block cookies. Evidently a few anti-spyware type programs commonly used block cookies by default. A real pain in the ass to figure out to tell the truth. This whole anti-cookie thing drives me nuts since they really are harmless for the most part.

Proxy servers add some issues too. I'm pro-proxy as it does reduce load on servers, speed up the user's experience, etc. It does make tracking harder though and causes some hiccups with dynamic pages sometimes.

Re:10 was arbitrary

[Amouth]

I have found that when using cookies (for logon and stuff like tracking users) i only use them in SSL - If you use secure cookies most of the anti spyware stuff will let them through.

and honestly if you site has SSL support why not redirect your users to that at all times (i do if they are loged in - even for message boards)

for most people it doesnt' really slow anything down except major downloads which you force to use http anyways

Re:10 was arbitrary

[UnrefinedLayman]

This whole anti-cookie thing drives me nuts since they really are harmless for the most part.

Cookies are tracking mechanisms. If I walk into Safeway I don't want to have a sticky note with a barcode stuck to my arm that will stay there until 2038 unless I manually remove it that gets scanned every time I walk in and every time I pick something up off the shelf.

There's no reason I should leave a brightly colored trail of everything I ever do on the Internet. The only reason I've ever heard is that it provides information and storage is cheap. Information is power, power is beneficial. What kind of deal is that, where I lose my privacy and the provider receives benefit? No thanks, I'll continue to opt-out of the tag-and-release program.

Re:10 was arbitrary

[Bogtha]

and honestly if you site has SSL support why not redirect your users to that at all times (i do if they are loged in - even for message boards)

It totally eliminates public caching (e.g. ISP caches). A waste of bandwidth, a waste of CPU, and slower speeds (it can be a lot faster for users to get stuff from an intermediate cache than from the origin server).

Also, this isn't an option for anybody using name-based virtual hosting, which is incredibly common. There have been specifications published for getting around this, but browser support isn't there yet.

Re:10 was arbitrary

[FLEB]

There's no reason I should leave a brightly colored trail of everything I ever do on the Internet.

Actually, there is. It allows statefulness and memory on the stateless, call-and-response Web. Without cookies or something like them (401 logins are basically crippled cookies, with only a limited subset of manually entered values, and link-rewriting is basically hackneyed cookie emulation), the Web would have far fewer applications than it does. If you block cross-site cookies (and I think this is a common default, isn't it?), it's not like any one site can read down your visited sites list and make a profile of you.

Re:10 was arbitrary

[IpalindromeI]

Safeway can track you too, even if you use cash to pay for everything, by using video cameras with facial recognition. It may be a way off yet before it's ubiquitous, but I bet not that far. Casinos use it already.

Soon the signs will read: Shirts and shoes required. Face masks prohibited.

Re:10 was arbitrary

[jacksonj04]

Umm, every time you use your credit card to buy things you are tracked. Safeway knows a lot about you.

Re:10 was arbitrary

[ADRA]

How many user are sitting behind my company's single routable IP address? Hit: Its over 1000.

This sways the number in the opposite direction. The number the story's based on is completely baked. You could attempt to statisticly estimate the number of unique users/ip on your site with some effort, but you can't get a real concensus between one sight and the next. The reason is demographics. If you take a mobile enabled sight, you're almost always guaranteed to get at least 2 IP's per user(one mobile usability, one desktop ease), but if you take a corp-LAN, you're almost guaranteed to have 1-1 user/IP.

As for sites worrying about anonymity-type scatter IP, I think that most anonymity-type solutions are quite easy to detect if a site maintainer really tries to. Remember, the referrer record is your friend here. If the user decides to piss around and blank it out, just loop them back to the main site page or something or tell them to login. So, if user X is behind a proxy that drops referrer headers, tell them to login or 'goto hell'. If someone's using an open relay, drop them from your site all together. A hit doesn't count as a user if you don't let them in =)

Re:10 was arbitrary

[Phillup]

Cookies are tracking mechanisms.

They can be.

But, by making cookies unreliable you have forced people (like me) to come up with alternative methods (to cookies that is) that are much more reliable.

And, as a replacement for cookies... these alternative methods can also be used for tracking.

So, I guess... in a way... congratulations are in order.

You are no longer being tracked by a technology that is easy to detect and block.

Feel better?

Re:10 was arbitrary

[mattyrobinson69]

name based virtual hosts, ssl on every page (if http, redirect), fone-me.com - let me know which browser your having problems with.

Re:10 was arbitrary

[I_Love_Pocky!]

I'm not a grammar/spelling nazi. Still, I had to wonder about how it was that you spelled site correctly at several points in your post, and mispelled it as sight in several other places. Did you have few beers with lunch? Be honest. I won't tell your boss.

Re:10 was arbitrary

[Amouth]

"this isn't an option for anybody using name-based virtual hosting, which is incredibly common"

this isn't true.. i have never had any issue with this - you just need to have it set up right which is quite easy.. what are you using that has this problem?

Re:10 was arbitrary

[Abcd1234]

From the Apache docs:

Name-Based Virtual Hosting is a very popular method of identifying different virtual hosts. It allows you to use the same IP address and the same port number for many different sites. When people move on to SSL, it seems natural to assume that the same method can be used to have lots of different SSL virtual hosts on the same server.

It comes as rather a shock to learn that it is impossible.

The reason is that the SSL protocol is a separate layer which encapsulates the HTTP protocol. So the SSL session is a separate transaction, that takes place before the HTTP session has begun. The server receives an SSL request on IP address X and port Y (usually 443). Since the SSL request does not contain any Host: field, the server has no way to decide which SSL virtual host to use. Usually, it will just use the first one it finds, which matches the port and IP address specified.

Re:10 was arbitrary

[pixelpunk]

The only way to know for sure is to lock everyone out of your site and require them to log in with a username and pass. Considering some couples may share an account it's the only definitive way to know exactly how many users access your site in a given time frame.

Re:10 was arbitrary

[Technician]

The count in the article is biased toward having one user counted several times because of using several IP's from a pool on DSL service. It made no allowance for the number of unique visitors not counted because they re-used an IP address from the pool.

Re:10 was arbitrary

[MikeFM]

Then don't bitch when stuff doesn't work for you. If you refuse to let stores send you coupons then don't complain when you never have coupons to use at the check-out.

The only people who can access the information in cookies are people who put it there anyway. The vast majority of them are simply an anoymous key that keeps track of who you are. It's the same as me giving you a number when you walk into my business and then every thing you ask to buy being put in a matched basket until you are ready to checkout. Nothing is disclosed about you because of you using the number I give you that wouldn't be revealed anyway and it makes things a lot simpler than if I have to label a basket with "Male of about 6ft and 180lbs with red hair and freckles wearing a blue tshirt and dorky striped shorts.". Anything else that cookies track can be tracked just as easily using other methods.

Re:10 was arbitrary

[DocTim]

I believe Max Fomitchev is plain wrong. To count unique users most websites won't use an IP counter, but cookies. And cookies are more likely to undercount, if several people share one computer (and haven't set up separate accounts on that machine) - although that's probably balanced out by people logging on both at home and at work.

Re:10 was arbitrary

[caseydk]

If you take a mobile enabled sight, you're almost always guaranteed to get at least 2 IP's per user(one mobile usability, one desktop ease), but if you take a corp-LAN, you're almost guaranteed to have 1-1 user/IP.

And actually... this is even worse. If you access it via phone access, most carriers only have a handful of gateways to the external world. For Cingular, I believe there are two. So no matter how many people access the page using a Cingular phone, you'd see one of two gateway IP's.

Re:10 was arbitrary

[Amouth]

I realize this is a limitation of Apache.. (they claim it is one of SSL and i do agree to why they are saying that) but (and i am not being rude here) Apache isn't the only web server out there. and many others have found working solutions to this. Apache should (and i know they are working on it) find a way to make it not an issue with their software.

The web server you run is based on your need.. if you have a need to run 100 virtual hosts off one server with ssl on all of them and you only have one address for it.. Apache might not be what your looking for..

Re:10 was arbitrary

[Old Man Kensey]

I checked out http://www.fone-me.com/ and it did redirect to https://www.fone-me.com/, but that doesn't really prove anything. What's another SSL-enabled virtual host on the same IP? That will be the real test.

Re:10 was arbitrary

[mattyrobinson69]

xesia.com

Re:10 was arbitrary

[Abcd1234]

Uhh... that proves nothing. The exact same content is served when visiting either domain, so there's no proof that the server is actually serving material from different virtual hosts. In fact, I would contend that this is proof it *doesn't* work, as, just as the Apache docs suggest, the server is probably just picking the first vhost entry.

Second, even if I'm mistaken about the above, this does demonstrate another problem with name-based virtual hosting and SSL: the certificate is signed for a specific domain name. In this case, when I visited https://www.xesia.com/ I got an error from my browser, because the certificate is for www.fone-me.com.

Re:10 was arbitrary

[mattyrobinson69]

hmm, what browser are you using? xesia.com is completely different to fone-me.com in all browsers ive tested on.

the ssl cert is a problem, because ive not setup xesia.com for ssl. there are no links to https://www.xesia.com/ but i shall fix my vhosts so that it just doesn't work, rather than give an invalid cert warning.

Re:10 was arbitrary

[Abcd1234]

I'm using Firefox 1.5.0.3. And I denied the cookie xesia.com tried to set, in case that makes a difference.

Re:10 was arbitrary

[mattyrobinson69]

no, that shouldn't make a difference.

xesia.com works for me in firefox 1.5.0.1 and lynx 2.8.4 on gentoo, after disabling cookies in both browsers.

Re:10 was arbitrary

[NameCritic]

He didn't seem to think it was arbitrary. He stated it as fact. But if I see a number of unique visitors to my website AND I check the cities where they come from AND the browsers and operating systems they are using, I get a much better picture. For instance I doubt my unique visitor at 3pm in the afternoon is the same as the unique visitor that came to my site at 10am. Why? Unless at 10am he was at home in Buffalo, NY, and then drove to work in Thailand by 3pm, he probably isn't the same guy. Also he failed to mention that not all dsl customers get a different IP everytime we log in. Either way, he is guessing. I agree you can't count every unique visitor as unique and that some are returning visitors, but I doubt the 10 to 1 ratio is even close.

Re:10 was arbitrary

[Abcd1234]

Well, just in case I was on crack, I double checked using both Firefox 1.5.0.1 and Lynx 2.8.5rel.1. I visited these two links:

https://www.fone-me.com/
https://www.xesia.com/

Both forward me to '/welcome' and then display the exact same HTML. I can provide screenshots or the downloaded HTML as proof, if you like. :)

Re:10 was arbitrary

[mattyrobinson69]

Ah, i just tried those links. There must be an error in the virtual host definition for httpS://xesia.com, as http://xesia.com/ works fine.

Sorry about the confusion.

I should get that fixed in the morning.

Re:10 was arbitrary

[Abcd1234]

LOL, that was the whole point! :) You won't be able to make it work. At least, certainly not with Apache. See that Apache doc link I posted elsewhere. Apache, with HTTP*S*, will always pick the first virtual host, due to limitations imposed by SSL.

Re:10 was arbitrary

[mattyrobinson69]

Hmm, i'm using apache2, and without any use of voodoo, or other black magic: (you'l get fone-me.com's cert, but thats because i haven't bothered to get a self signed one for xesia.com)


HTTPS://xesia.com
HTTP://xesia.com
HTTPS://www.fone-me.com
HTTP://www.fone-me.com

http://www.fone-me.com/ will redirect you to https://www.fone-me.com/

Re:10 was arbitrary

[Abcd1234]

Well I'll be damned... and you're not using funky URL-rewriting tricks or anything? Something must have changed in the Apache implementation since I'd last looked this this stuff. I stand throughly corrected. :)

Re:10 was arbitrary

[mattyrobinson69]

nothing fancy, here's my /etc/apache2/sites-enabled/sites:

http://pastebin.ca/54877

using Apache/2.0.54

nothing much here

[Anonymous Coward]

IPs = Visitors*(DSL_FRACTION*VISITS_PER_TIME_PERIOD + CABLE_FRACTION)

If DSL_FRACTION = CABLE_FRACTION = 0.5 then we get

IPs = 5.5*Visitors or Visitors = IPs/5.5

thats the formula she uses to get the number of ips.no idea if the math is right,anyone who has an idea,please enlighten me

Re:nothing much here

[GigsVT]

I'm on DHCP on Cable. I don't think I'm the only one. I guess maybe the article wanted to show by demonstration how most statistics are made up on the spot.

Re:nothing much here

[Anonymous Coward]

14% of all people know that

Re:nothing much here

[zippthorne]

Well it appears to make the assumption that the visits are sparse enough for the DSL Ips to change every time (and also that cable IPs are static. Mine is not, though it doesn't change for months at a time)

So, what IS the typical holding interval for a DSL ip?

as for properly estimating, If there are good enough statistics to have separate numbers for both {known, relatively static IPs over a month} and {known dynamic IPs} you could find the ratio of returning static IPs and normalize the dynamic ones to match that ratio.

In fact, I'd be surprised if this wasn't already being done for many sites.

Re:nothing much here

[hotgigs]

Last line of article: "So do not trust stats, they ARE inflated." Like I have always heard in regards to statistics: "Statistics are like loose women. You can do whatever you want with them" Although I don't suppose most of us on Slashdot would know much about women, let alone loose ones...

Re:nothing much here

[level_headed_midwest]

From what I hear, most cable users are on DHCP. In fact, my cable provider doesn't even allow the option of a static IP whereas the DSL provider will (for an extra fee, of course.)

so...

[gEvil (beta)]

So, he's saying my website has 1/10th of a visitor?

Re:so...

[labalicious]

Yes, in fact... Only my foot has visited your site and it was thoroughly bored with it.

Re:so...

[Ruvim]

That's actually, ZERO Visitors, as you are not a visitor, but a host, so to say.

Re:so...

[Anonymous Coward]

If I were even more exact, I would say that he has not more than -9/10 visitors. Gee, how I'm sorry...

Re:so...

[frieko]

1. Come up with arbitrary retarded 'theory' 2. Post your stupid blog article on slashdot 3. ??? 4. Profit! At least Piquepaille's links were sometimes interesting...

Re:so...

[isaacklinger]

He's saying only one person out of ten people is unique.

Re:so...

[MrNougat]

Yes, the only visitors to your site are midgets.

Re:so...

[HermanAB]

Hmm, maybe you should not check your own website so often... ;)

Re:so...

[Opie812]

He's saying only one person out of ten people is unique.

Screw that, man. I'm unique just like everybody else.

Re:so...

[Ruvim]

Got to watch out for those negative visitors!

Re:so...

[50m31sl4sh.]

I tried accessing your website (http://evil.google.com/) but it seems to be unavailable, probably slashdotted?

Re:so...

[gkhan1]

Hey, I'm overjoyed for my 3/5ths of a user.

Neighbor's Wireless

[Anonymous Coward]

I help keep this in balance by using my neighbor's wireless, that IP has a load of unique users.

Cookies?

[Anonymous Coward]

Isn't this what cookies are for? Sure lots of people wash them. But I'm betting the majority of people do not.

Maybe not...

[Anonymous Coward]

I've hosted several servers from home for years at a time without my dynamic IP address ever changing, and I've known many others in the same situation. I think this 10x rule might be a bit extreme...

Re:Maybe not...

[Anonymous Coward]

I've had a primary MX on a dynamic IP on a cablemodem. About the only time I had to change the DNS was when the ISP shuffles their network (about every 2 years).

Already considered.

[locu]

I think anyone in the tech sector knows this already. It's pretty common knowledge that people surf from multiple IP addresses. How about a solution instead? How about adding MAC addresses to stats, or modem_id's (whatever the equivilent to them is a to a MAC) and have webbrowsers send that data too then stats applications can have something more unique to identify a unique customer by than an IP address.

Re:Already considered.

[Nos.]

Because its none of your business (you being a web"master"). I understand that my Ip is broadcast, but I'm not going to use a web browser that sends personal information about me or my computer, every time I hit a site. I can use an anonymous proxy now, and refuse/delete cookies, and know that I am not being tracked. Besides, MAC addresses can be altered as well.

Re:Already considered.

[Homology]

How about adding MAC addresses

Your MAC address survives at most until the next router.

Re:Already considered.

[Anonymous Coward]

How about adding MAC addresses to stats, or modem_id's (whatever the equivilent to them is a to a MAC) and have webbrowsers send that data too then stats applications can have something more unique to identify a unique customer by than an IP address.

Why would I want my browser to send you identifying information? This makes no sense to me.

Re:Already considered.

[locu]

The article is just stating the obvious. Just looking to present 1 of what will soon be many solutions to the problem probably. Nobody likes cookies, nobody likes sending personal data. There's always some critic to *everything* out there. And every advancement in technology always ticks somebody off.

Re:Already considered.

[Anonymous Coward]

What about firewalls. I'm sure the 40 or people surfing monster.com at my workplace are all seemingly using the same IP to monster.com. So perhaps it goes the other way - one IP equals 10 users.

Re:Already considered.

[ceoyoyo]

Perhaps people should be a little less concerned with tracking other people? Why exactly should uber accurate unique visitor statistics matter to websites? How about a little less effort tracking your visitors and a little more generating something interesting for them to see when they do visit?

Re:Already considered.

[Adam Hazzlebank]

Because a) It would be a gross invasion of privacy b) It doesn't solve the problem of a user surfing from multiple computers and c) It's technically infeasible.

Re:Already considered.

[dietrollemdefender]

...can have something more unique to identify a unique customer by than an IP address

So says the AC.

Re:Already considered.

[level_headed_midwest]

That's called Internet Protocol Version 6, where some of the address is your MAC address.

Re:Already considered.

[Ruds]

Why exactly should uber accurate unique visitor statistics matter to websites?

Because it impacts ad revenue? If IPs >> visitors, advertisers want to know this and lower the amount their willing to pay. If visitors >> IPs, websites want to raise their advertising rates.

Re:Already considered.

[poot_rootbeer]

How about a little less effort tracking your visitors and a little more generating something interesting for them to see when they do visit?

How are we supposed to know what our visitors find interesting? Hmm, maybe we could take a look at what current content they favor and use those statistics to guide the creation of new content...

Re:Already considered.

[ceoyoyo]

In that case we should definitely sacrifice anything to have more accurate visitor statistics. After all, advertising makes the world go around.

Somehow advertising on TV, radio, bill boards, foreheads, t-shirts, shoes, placards, shingles, phone books and jumbotrons manages to survive without putting house arrest style anklets on the entire population.

Re:Already considered.

[Geoffreyerffoeg]

That's called Internet Protocol Version 6, where some of the address is your MAC address.

No it's not. First, if you're on a NAT, your private IP doesn't help (and if you think ISPs are going to give away blocks of public IPv6 addresses to their customers just 'cause they have plenty, you misunderstand the ISP industry). Second, if I remember correctly, your IPv6 address only contains your MAC address for the private autoconfiguration address. If you're on the public internet - or any network with a DHCP server or with manual addressing - you don't have your MAC address embedded. Requiring each computer to include its MAC address in its IP address would just be wasting 6 bytes of your IP address.

And the most important thing is that TCP/IP works perfectly fine over token-ring, LocalTalk, carrier pigeon, or whatever other non-Ethernet link layer you have. And MAC addresses only apply to Ethernet networks.

Re:Already considered.

[Meumeu]

AFAIK, there is no NAT in IPv6 (and Wanadoo is testing IPv6 now and you can have a /48 block). IPv6 addresses contain your MAC address for link-local, site-local and global addresses (although if a certain bit is not set, the host part of the address is not your MAC address). And putting your MAC address in your IP address makes sense as you have 8 bytes for the host, you can put your MAC address and you don't need ARP.

Logging in from home vs. work PC

[hotgigs]

So if I access a site at work and then access it at home, that's two IP's (two different computers and two different ISPs) but one user. Seems a bit high to say 10 but I could see 2 - 3...

Assume = ASS + U + ME

[Orestesx]

"Presuming that the latter have static IPs the former draw different IP from some pool each time they connect." I have dsl and according to dyndns.org, my dynamic IP host, my ip was last updated on April 13. Just because someone is using PPPoE doesn't mean that each time they visit your site they will have a new ip.

Re:Assume = ASS + U + ME

[Kwiik]

Just because someone is using DSL doesn't mean they are using PPPoE

The DSL provider in my area uses an ATM network that the modem syncs in to, to connect to it's assigned PVC, and from there all IP addresses are requested through DHCP. Static IP addresses are also an option, but still using DHCP, as they assign the PVC to a DHCP pool that has one available IP address, being the IP address that you will always get.

On the other hand, the cable provider in my area uses dynamic IP addresses as well.. I have never ONCE seen *any* cable provider offer only static IP addresses. Our local cable company changes 60$ more than a consumer account for the base level business account, which is the first account type to offer a static IP address.

This dumb article makes an assumption that all cable providers provide static IP addresses.. WTF? and either way, any site that reports a unique visitor count had better be doing it over a given time period, i.e. 10,000 unique visitors in a given month

Am I my clothes or my reward card?

[spyrochaete]

I guess it depends on the definition of "visitor". Maybe the site tracks unique IP addresses, or maybe it ignores them where the client has a permanent cookie installed.

However, I can attest that my ADSL connection is pretty DHCP heavy. Sometimes my IP won't change for weeks, but I've had 5 or 6 IPs in 24 hours on several occasions.

The post is a bit misleading

[mapkinase]

Actually, the article is about exhaggeration of the number of visitors. Number 10 is just a vague unsubstantiated number given in the blog entry without supporting data to illustrate the magnitude of that exhaggeration.

Re:The post is a bit misleading

[poot_rootbeer]

Actually, the article is about exhaggeration of the number of visitors.

I thought it was well known to everybody in the web business already that IP addresses and unique visitors do not have a strong direct correlation. If the point of the article wasn't to support in particular the 10-to-1 ratio, then I'm not sure what the point was.

Use cookies

[Bruce Perens]

IP addresses do not correspond to users because of dynamic IP and proxies.

Cookies are a much better indicator of what browser you are communicating with.

Also, most spiders don't bother with cookies, so that's another way to tell something isn't a real user.

Unfortunately, some users disable cookies. And then all you can do is fall back on their IP address.

It would be nice to see cookie-tracking support in Open Source stats engines like awstats.

Bruce

Re:Use cookies

[Kalak]

http://www.analog.cx/docs/logfmt.html

Look for %u in defining a custom log format for analog, which can be used with the user report capability to give you session ID information (easily paired with Apache's mod_usertrack http://httpd.apache.org/docs/2.0/mod/mod_usertrack .html).

http://www.serverwatch.com/tutorials/article.php/3 504311 is a good read on the module as well.

You have to get your log formats set right, but I believe this is what you're looking for (I don't use awstats, but it's most likely possible in a similar fashion).

Re:Use cookies

[2008]

Yeah, but my IP address is a NAT with 3 computers on it (desktop, big laptop, small laptop). I dual-boot, so that's 6 operating systems. I mostly use Firefox, but I sometimes use IE or Konqueror, so that's 12 different browser installs, none of which share cookies, all of which are just one unique user. And I delete my cookies or reinstall an OS now and again.

Oh, and I have another Firefox+Konqueror at work, which I sometimes browse through via X-forwarding or nx (or physically being there, obviously)... so IP addresses and cookies both fail.

Good thing too, I don't think it's in my interest to be tracked.

Re:Use cookies

[gstoddart]

Unfortunately, some users disable cookies. And then all you can do is fall back on their IP address.

Of course we block cookies. Because most of the cookies you get offered are 3rd party to the site you're visiting and just crap so gator and all of that other junk could keep track of you. I only accept cookies coming from the site I'm visiting, and then only if I say YES. It took a very long time to teach a lot of people they needed to be more cautious with cookies, because there were a lot of privacy issues with them. If I could block my IP address, I would do that too. =)

It would be nice to see cookie-tracking support in Open Source stats engines like awstats.

But we don't want to be tracked. That's why we disabled the cookies in the first place. I got so damned tired of things like doubleclick trying to set cookies back in the day -- if you need to give me a cookie to make your site work, *maybe* I'll accept it. Giving me third-party cookies? No flippin' way!

Course, I'm pretty anti-advertiser and the like. So I'm probably not a good example. :-P

But...

[xpird]

Can he find a formula for the number of /. articles posted vs. the actual unique articles?

Re:But...

[qwijibo]

Sure, it's: total/unique

The problem is that x/0 is undefined.

Stopped reading after the first sentence.

[mightybaldking]

Both a grammatical and a math/logic error in the first sentence!

Maybe

[Kethinov]

I think the moral of the story here is that you can glean no true information as to how many visitors your site really has by unique IPs. This convenient unique visitors = 1/10th of unique IPs idea is no more accurate than simply assuming each new IP is a new visitor. There will be people who visit your site from 10 different locations and thus 10 different IPs, and there will be whole families on one IP visiting your site. Or perhaps one of those 10 different locations one person uses is used by others. Since we have no way of knowing these things, any sort of formula is as inaccurate as the next.

Nonsense

[MrNougat]

I see what the guy is saying - dynamically assigned IPs at clients mean that one person can view a site from multiple source IPs over a period of time. Both DSL and cable use dynamic IPs - but they are not often disconnected/reconnected, and when they are, DHCP is likely to pull the same IP address back anyway.

Besides that, think of all the people at work on internal LANs, each presenting the same public IP source address to the same web server. This effect more than makes up for the dynamic IP nonsense the blogger spouted off.

Maybe he's just mad about the size of his epenis, and is trying to compensate with illogic.

WTF?

[giorgiofr]

This argument is flawed. Logging to Slashdot now from my house and two hours from now from my friend's house should count for two visits, and so it rightfully does. The article writer seemed to have a problem with this? ZOMG 2 different IPs...
And if my IP has changed but I'm still here... that's because I haven't surfed for many hours at least otherwise the lease will be renewed and the address will stay the same. So it should still count for two visits. Duh.

Re:WTF?

[Anonymous Coward]

two visits yes but ONE PERSON you moron. thats what the writer is trying to say.
most site stats say unique visitors but in fact its unique ip addresses.

Re:WTF?

[gEvil (beta)]

So you are saying that by going to your friend's house, you are now a different person (aka, a different unique visitor)? It may count as two separate visits (which is not what is being argued here), but you are still the same unique visitor. Counting IPs alone does not provide an accurate representation of how many unique visitors view a site. Your example only helps to demonstrate this.

Re:WTF?

[mightybaldking]

It all depends on what we are counting. These numbers are primarily used for advertising. I can show your ad once to ten different people or 10 times to one person. Which do you prefer? Are we counting unique visits or unique visitors? You are correct if the former, but wrong if the latter.

Re:WTF?

[the phantom]

Two things:

1) He is interested in counting the number of unique visitors to a site. Not unique computers. If you visit a site from a home computer, a work computer, and a computer at some internet cafe somewhere, it will show up as three different IPs, and be counted as three different unique visitors. However, you are only one person, so, if we are concerned with counting unique visitors, it has just overcounted you by 2.

2) Also, even if you are using just one computer, there is a high likelihood that your IP will change over the couse of a month (if not more often). Assuming you are on DSL (cable, as well, but he ignores this fact), you are probably getting an IP address via DHCP, which means that the server issueing that address could issue a new one every time you connect. Over the course of a month, you may get several addresses, each one counting as a unique visitor. Again, you are being overcounted.

Re:WTF?

[Peyna]

Assuming you are on DSL (cable, as well, but he ignores this fact), you are probably getting an IP address via DHCP, which means that the server issueing that address could issue a new one every time you connect. Over the course of a month, you may get several addresses, each one counting as a unique visitor. Again, you are being overcounted.

My cable modem only gets disconnected if the power goes out, and if that does happen, it reconnects long before my DHCP lease expires. I've had the same IP for over a year.

Re:WTF?

[gEvil (beta)]

You somehow have been modded up. It seems that at least you and a moderator need to learn the definition of unique.

Re:WTF?

[fbjon]

The unique visitor statistic you're talking about is completely useless anyway. Another way to phrase it is: "how many pairs of eyes have seen this site, in total, ever". So if I see the site, I'm forever counted as having seen it once, and no more.

From the point of view of ad impressions, this would imply that ads have exactly one chance of having an effect on a visitor, no matter if there's several hours, days, or weeks in between views. But that's not how ads work, you want to show it to the same pair of eyeballs repeatedly, and also to a lot of different pairs as well.

Re:WTF?

[jafiwam]

1) He is interested in counting the number of unique visitors to a site. Not unique computers. If you visit a site from a home computer, a work computer, and a computer at some internet cafe somewhere, it will show up as three different IPs, and be counted as three different unique visitors. However, you are only one person, so, if we are concerned with counting unique visitors, it has just overcounted you by 2.

Well. He needs to realize it's NOT POSSBILE. You can make an educated guess, and seriously what's the difference between 800 visitors a month and 1200 visitors a month really? Leases, NAT, Proxies, reverse proxies, anonymizers, open WiFi, botnets, freenet, blah blah will all get in the way. Digging deep into senarios of end users to figure out what the typical one does and then extrapolating is the only way to do this. Unfortinately that is way outside the typical PHB and end user mom/pop website operator's ability. Open a log file once and look through all the crazy crap in there. You can guess, and that's about all.

2) Also, even if you are using just one computer, there is a high likelihood that your IP will change over the couse of a month (if not more often).

Not a safe assumption, depending on the ISP the setup is different. I have had enough ISPs to know it's VASTLY different. Charter cable has DHCP, but the MAC address on the modem dictates the IP you get and you can have the same IP for YEARS, pick it up, put the modem on the neighbor's line and it works there (if they had cable internet too) but you get YOUR IP not theirs. DSL has other setups and systems...

Assuming you are on DSL (cable, as well, but he ignores this fact), you are probably getting an IP address via DHCP, which means that the server issueing that address could issue a new one every time you connect. Over the course of a month, you may get several addresses, each one counting as a unique visitor. Again, you are being overcounted.

Yes, however it goes both ways. One AOL user can look like 50 users. Just where exactly it balances out, who knows. I assume it is different from site to site and when your user base changes, so does how you would guess the number of unique visitors.

Seriously, read through your logs a few months and you will see how complicated it gets.

In any case, it's pretty clear the article author lacks real knowledge on the subject.

Less

[teeseejay]

For sites with recurring traffic there are at least 10 times less unique customers than there are unique IPs.

The word you're looking for is "fewer," buddy.

Re:Less

[mightybaldking]

And what does 10 times less/fewer mean?

Re:Less

[Anonymous Coward]

The same thing bugs me as well. There are just too many mathematical luddites out there who had a bad experience with fractions in the fifth grade and now can't wrap their brains around phrases like "one tenth as many as". So, instead, they say "ten times fewer than", or n = m - 10m = -9m.

It's like saying "ten times as many as" versus "ten times more than". The first case is as intended, n = 10m. But the second case (though it's usually not used this way) should be interpreted as n = 10m + m = 11m. (Of course, in this case, when the multiplier is large, the inaccuracy is small.)

The other one I hate is "a million and a half", especially as applied to people. Things must be pretty uncomfortable for that extra half a person, unless the speaker really meant "one and a half million".

Comcast doesn't change

[Solkre]

I have Comcast Cable and to be honest, they never assign me a different IP address. I've had the same one for 6 months, the entire time in this new appartment. My old connection was the same, DHCP getting the same IP address over and over again. Does this guy account for my wife and I visiting the same website, being two different users, yet having a shared IP address?

fyi

[hurfy]

Fixed 4:1 ratio here tho i can't speak for anyone else.
1 office and 3 home computers.

10 seems a little excessive, timeframe probably matters to actual ratio: unique per day? month? year?

The author left out surfing from work

[Trails]

I do most of my news browsing at work, where several hundred people show up as one IP (home computer is exclusively for WoW).

Besides, the assumption that stated unique visitors = actual unique ip's is innacurate. Lots of companies track users with some kind of UID cookie, for more accurate stats. True, this isn't perfect either, and will reset when users wipe their cookies or it expires, but is probably closer to the real number than ip's.

Re:The author left out surfing from work

[Rick.C]

Or from a library, or from various campus PCs, or from a notebook with WiFi.

Okay... and?

[the_skywise]

Ooh, wow... IP's aren't good indicators of uniqueness... I'm sure the Slashdot editors will tell you how valid that is when they're troll hunting.

But I don't think dynamic DSL IPs are that big of a problem. What about DSL users that are connected 24/7? My DSL provider rarely kicks me off and I can hold the same IP address for weeks.

What about laptop users at wireless cafes or users who post/read from work? Surely the same IP that reads a tiny website from home is likely also the "same IP" that reads it from a nearby workplace.

Re:Okay... and?

[Charan]

For a while now, I've been connected to the internet through a DSL connection. My computer has been running ddclient, a daemon that scans my DSL router every 2 hours to get my external IP address. When it finds a different public IP address, it adjusts my DynDNS link. Importantly (for us), it also notes each change in a log file.

I can say with confidence that over a 9 month period, my computer has been assigned 49 different IP addresses. Average: 5.4 IPs / month.

This is a low estimate, since my computer could have missed some changes. It doesn't run continuously, and it updates only every 2 hours. Still, this figure shouldn't be too far off.

Plan 9

[Anonymous Coward]

If only the same was true with our little OS over here.

Erm.

[Rob T Firefly]

A) He's talking about IPs reported in site visitation stats, not available IPs "out there."

2) He skips a few major technical details about the IP system itself.

d) He's mulling over a random loopy theory in a personal blog post, which isn't quite news. If it were, I'd be William Randolph Hearst by now.

More than IP

[Billosaur]

If you are only using IP to generate your visitation metrics, then you're fooling yourself, for the reasons outlined in the blog. You can't guarantee an IP is unique to a user, any more than you can guarantee that a user is unique to an IP (think Internet cafe or library; different users, same machine with potentially the same IP)

You have to use a combination of log data to try and scope out exactly who's visiting: IP, browser type (can't count robots in your stats), membership id (if the site uses/requires it), and possibly cookie data (you assign a unique id when a user visits the site and they carry that data to every page). Here's a good breakdown of metrics processing and its pitfalls.

Max has just learned about DHCP

[upside]

... but didn't read the detail. A DHCP client gets the same IP address it had previously, so unless the pool is in short supply of free addresses it will get the same address as before.

And why does he suggest that DSL clients have static addresses while cable users have dynamic ones?

Also, most home users (I'm allowed a presumption too) have routers instead of bridge/modems or PCI card modems, and they are kept on all the time. While the router is powered on it will keep renewing the existing IP address.

I have a dynamic IP address but it's stable enough to run servers on it.

What about proxies?

[Anonymous Coward]

AOL represents a large chunk of traffic and all users appear to come from one of a dozen proxy servers. Did he factor this in?

No fucking duh

[SmallFurryCreature]

IP says very little. Dial-in users (yes they still exist) get a new one every time they dial-in.

I am a little puzzled by his assumption that DSL users get a new ip while cable users have a static one. I had a DSL account with a static ip and a cable with a changable one.

Also if you got a good ISP that doesn't drop your always on connection you won't be changing your IP all that often. Hell even my crap cable provider rarely changed my ip.

So no, for a "large" site you can't really determine unique visitors by ip. That only works for really small sites wich maybe get one visitor per ISP worldwide.

Even if IP was unique that still doesn't account for proxies. I have personally had to explain a couple of times to customers that no user X wasn't reloading the site constantly, that is the proxy for a university.

So my conclusion? Wake up and smell the coffee. Everyone who knows anything about measuring site metrics know that you can't make any claims based on ip's.

His 10:1 claim is just random guessing.

If you absolutly must know then you need to do what newspapers do when they study how many people read their newspaper (and therefore see the ads) vs how many newspapers are sold. Take a wild stab at it and choose the figure you think your are going to get away with.

Re:No fucking duh

[drinkypoo]

Also if you got a good ISP that doesn't drop your always on connection you won't be changing your IP all that often. Hell even my crap cable provider rarely changed my ip.

Even if it does drop connection, if you renew your lease within the TTL, you'll get the same IP anyway, unless the server is configured otherwise.

On Comcast, my IP only changed if I brought up machines out of order. So long as my server was the only thing hooked up to the cable modem (to do firewalling and NAT) then the IP never changed. If it went down or I was trying to do some troubleshooting and hooked up a different PC, then it got the firewall's IP, and the firewall got a new IP or something like that. Very strange.

wait a sec

[hurfy]

I forgot something.
What about the other way?

Do they see the 10 people on the office NAT as one IP ?!?

That would skew it in the other direction and average things out wouldn't it? Now 10 is definately excessive.

I thought it was the opposite.

[Black Perl]

First of all, a DHCP server is typically going to give you the same IP address each time your computer requests it, unless there are more users than IP addresses, in which case there will be some shuffling. But that tends to be when there are more users than available IPs.

There are entire domains hidden behind a NAT device of some sort. This would be many users per IP address. TFA didn't mention this at all.

So I think TFA is indeed arbitrary, and also wrong.

Re:I thought it was the opposite.

[spyrochaete]

Home internet users are usually grouped into big subnets of IPs that are constantly being shuffled around. IP addresses expire and reconnect on a schedule, houses have power failures, modems do something funny, or whatever. IPs in a huge pool are being retired and reassigned constantly. The bigger the pool, the lower the chance of obtaining the same one before someone else's modem nabs it.

If they offer it, ISPs charge extra for static IPs. Nobody would pay extra if it wasn't an issue.

Re:I thought it was the opposite.

[the melon]

Yeah, here at Sun there are nearly 40 thousand people that connect through 40-50 different proxy servers. That is a thousand to 1 in the opposite direction the article claims.

And yes, he seems to have no idea how DHCP really works. Even if your lease is expired you will get the same IP address unless the pool has been exausted and your address re-used. I see that as an extremly unlikely thing to happen because it would mean, as you say, that your pool is smaller than your installed base. If you pool is smaller then you will start having issues because x number of customers will always be without a connection because they can't get an address.

Had he mentioned Dialup users then I would be more inclined to agree because you are very likely to get a different address every time you connect.

Re:I thought it was the opposite.

[Anonymous Coward]

Had he mentioned Dialup users then I would be more inclined to agree because you are very likely to get a different address every time you connect.

Even then, what about the other people who end up with the address you were using last time? The "perfect" unique visitors count would have to take this into account too.

Re:I thought it was the opposite.

[arodland]

One place where it is true that users change addresses constantly is dialup; PPP generally doesn't care who you are, or have any concept of a "lease", so generally speaking you'll never get the same address two connections in a row. The number of dialup users is on the way down, but it's still significant (for reference, see Slashdot). But I agree that the effect should be canceled out (or more than canceled) by the effect of NAT routing at offices, schools, libraries, and other big institutions.

Re:I thought it was the opposite.

[XorNand]

True, but some broadband providers intentionally hand the client a different IP when their lease expires in order to:

a. Prevent subscribers from running servers without paying for a static IP. While dynamic DNS services can be a workaround much of the time, it doesn't work very well with SMTP or other cases where DNS caching can cause issues.

(or, if you ask the provider)

b. To decrease the likihood of crackers breaking in your computer.

Re:I thought it was the opposite.

[Anonymous Crowhead]

It think it's ISP-dependent. My address hasn't changed in a few years, but a co-worker's changes every month or two. I know, because I have to reconfigure something whenever it changes. We both have cable, but different companies.

Re:I thought it was the opposite.

[bogado]

--

        * MOD THIS UP INSIGHTFUL!!!! by Anonymous Coward Wednesday February 30, @09:34AM

So now we have phishing signatures? It seems like this signature make it looks lika other user have requested a mod up on him, I would mod it down if I had the points, just for the use of low tatics, even though I do think the comment content is okay.

Re:I thought it was the opposite.

[ranson]

Hmm, well TFA was speaking to DSL users, who generally connect via PPPoE. PPPoE, while it does dish out IPs a la DHCP, does not have a lease file that associates user mac addresses with the IP they were handed on previous connections; users generally will get a different one every time they connect just like dial-up (aka PPP). I would say router-based PPPoE connections drop and reconnect at least once per week on average.

Re:I thought it was the opposite.

[the melon]

Actually the oposite should be true. The bigger the pool the better your chances are for getting the same IP. When you release your DHCP lease from the server or it expires before your client renews it the server still has that address assigned to your MAC address. The only difference is the lease is inactive. If there are addresses in a pool that have never been assigned and a new client connects it will use those addresses first. If there are none then it will start checking the inactive leases. It will take one, not sure if it does them in order form oldest expiration or randomly, and ping the address to make sure it is not still in use. If it is not then it assignes it to the new client. If it is in use then it moves on to the next one and marks that address as unuseable. ISC DHCP will revisit unuseable address once it has exausted the inactive pool. Solaris DHCP will not revisit them until they have been marked good again by an admin.

So the longer you are inactive the greater your chances are that you will get a different address and the bigger the pool is the lower your chances are.

The above is of course null and void if the ISP deliberatly expires leases and forces users to new addresses. It has been my experience though that very few do this. Even my dealings with PPoE based DSL has indicated that as long as you stay connected via a keepalive then you will retain the same address for a while. If you restart your router with PPoE you are likely to change though.

I wonder what percentage of people with DSL and Cable do not use a firewall/NAT box?

Re:I thought it was the opposite.

[the melon]

The "perfect" uniuque visitor count would have to have something like an RFID or SmartCard based unique identifier that you carried with you and was required to use a web browser. Even cookies combined with IP uniqueness would not do it.

Implant an RFID in your palm and have the mouse be the reader.

Re:I thought it was the opposite.

[spyrochaete]

If you don't want to change settings every time your coworker's IP changes check out No-IP's dynamic DNS service. I've been hosting many live services from my home network for years including POP3\SMTP email and it's been impeccably stable, and totally free!

Re:I thought it was the opposite.

[TheRaven64]

Any kind of broadband[1] is always-on, and so needs an IP all of the time. The easiest (and therefore most common) way of implementing this is to give a user the same IP all of the time. Since users will be refreshing their DHCP leases at different times, it is difficult to shuffle them between users (and will break things if they have active connections at the time). If your modem/router/whatever is turned off for a period, however, then it is quite likely that your IP will have been allocated to someone else and you will get the next spare one.

When a DSL or Cable company says that they don't give you a static IP, it means that they don't guarantee a static IP. They are at liberty to change it at any time. For example, they might decide to redistribute their subnets if their customer concentration changes, to make their routing easier. This will not affect residential customers much, since they will just experience a brief disconnection. For commercial customers, who have a lot of DNS entries pointing at servers on their assigned IP, however, it would be disastrous. My cable IP has stayed the same for about a year. Before that it stayed the same for about two months.

One thing that the article author failed to take into account was HTTP proxies. Everyone who uses the web via a public terminal on my campus appears to come from the same IP, even though they could be any of several thousand people. Most of my ISP's customers go through a proxy (i.e. anyone who uses Windows and used their set-up software, or anyone who configured one manually). They used to use a transparent proxy for everyone, but they seem to have stopped that now. Other ISPs still do. When I was bothering to log visitors to my site (yes, I know it's down at the moment - server's hard drive died) I found that a significant proportion of my hits came from known proxy addresses - IPs that may have represented large numbers of individual users.

[1]Currently offered commercially, that I am aware of.

Re:I thought it was the opposite.

[insane_machine]

My IP address hasn't changed in at least 6 months, though it could be due to the fact that I have a router. I also keep my computer on most of the time.

Re:I thought it was the opposite.

[TheRaven64]

This sig has been in use for a while. I've seen it before, but it's obviously not a real comment; the link text isn't coloured and there bullet is not the right character. When I saw it first (a year ago?), it seemed an obvious joke, and I laughed a little. Perhaps you need to lighten up a bit?

Re:I thought it was the opposite.

[SydShamino]

If I used broadband the way my ISP (AT&T) envisioned it, it would connect to my single PC. Whenever my PC was on, I would be connected. Whenever it was off, I would disconnect.

But I don't use it like that. I have a NAT router attached to the DSL modem. The router maintains my connection 24/7. The keep-alive is set as long as is allowed - 6.999999 days or something like that. And, when it renews, I'm down for less than a second.

Thus I've maintained the same IP for as long as I have checked. Of course they could always choose to swap me around - I'm not paying for a static IP to prevent this. But the fact is that they don't usually bother; the DHCP software is designed to return the same IP to the same MAC address if it hasn't been given out to someone else since it was last released.

Thus, my one IP at home represents unique visits by everyone in my house. And given that we clear cookies regularly, they probably cannot tell any of us apart.

Re:I thought it was the opposite.

[W. Justice Black]

It doesn't quite work this way for PPPoE (which is why he singled out DSL). PPPoE is more like really-high-speed dialup.

I'm not sure about TFA's logic overall, but he had a point when it comes to most contemporary DSL.

Re:I thought it was the opposite.

[bogado]

Well, in the comment the character was right and the signature was correctly idented. The date and the fact that it is not a link is a clear signal of it's not being a real reply.

Well I didn't see the point of this kind of joke, and quite clearly it could fool someone, and having a good feedback in a fake response could aid him to get some extra karma. I don't think this is fair play or funny, but I don't care much to karma either, but many people here do care, and I was stunned to see this kind of "phishing" that would grant an off-topic comment. :-)

Re:I thought it was the opposite.

[ninjaz]

Yeah, here at Sun there are nearly 40 thousand people that connect through 40-50 different proxy servers. That is a thousand to 1 in the opposite direction the article claims.

That's a good point. However, the majority of the sites being visited through the Sun proxy are probably also being visited by the same users from home as well. So, while Sun may not be adding much to the skew, it's quite a leap to say it skews it in the other direction 1000:1

For example, I have visited 3 or 4 of the same websites from the following locations last week:

1) the computer lab at the university
2) my dsl at home
3) two different cybercafés

While the cybercafés and my university are probably both using proxies with a single IP, that still means my hits show up from 4 IP's while I am only one visitor.

Re:I thought it was the opposite.

[Matilda the Hun]

Yes, well, if I had mod points I'd give you +1 Overreacting. I guess we can't all get what we want.

Re:I thought it was the opposite.

[operagost]

PPPoE (used with some DSL) is the same way, of course.

Re:I thought it was the opposite.

[ModernGeek]

I believe in the Kentucky K-12 school system everybody now goes through the same proxy address. I know that everyone is on the same domain, so I am guessing it is just a giant VLAN connected with T1s.

Re:I thought it was the opposite.

[KingMotley]

The above poster obviously doesn't fully understand DHCP either. The IP pool for many scenarios are much smaller than their installed base. Dial-up users and DSL users included. You only have to have as many IP Addresses available in the pool as you can have online at the same time. And the DHCP server can reuse any unallocated address, it does not have to necessarily reuse them in longest since has been allocated strategy. Besides the technical issues, I agree with the above poster in general. The article is full of crap, as even for those people with dial-up, a single user may (Will most likely) use more than 1 IP, it's also quite possible the opposite is true. A bank of modems (reusing the same IP), where multiple people dial-in during different parts of the day and get the IP reused will only be counted as a "single" unique user. Trying to track unique visitors by IP is only a ballpark figure, and a VERY large ball park at that.

Re:I thought it was the opposite.

[sdnoob]

there are cable internet subscribers here who have had the same 'dynamic' ip address since the service launched five years ago. they assign ip's by your nic or router's mac, and they never change. they only reassign (currently unused ip's) when they reach the end of the address pool and have to start over from the 'top' of the list.

with dsl from the telco here, though, you could end up with a different ip every day (sometimes even more often than that), and it's very rarely ever one you've had before).

i agree, to a point, with tfa, but he fails to mention multiple visitors to the same site who are behind nat and share the same ip. those visitors, though, will more than likely be outnumbered by those whose ip's change regularily on a typical site.

the number of actual unique visitors will be lower than the number claimed, when using only ip address to determine a unique visitor. it's obvious skewing of the data, but what organization doesn't do that if it will benefit themselves?

Re:I thought it was the opposite.

[imsabbel]

Here in germany, ALL major dsl flatrate providers disconnect after 24h and give out a new ip on reconnect.
They have millions of suscribers, and IMHO, while i was logging, i _never_ got the same ip again.

That means, for slashdot, i alone would have been about 2000 or 3000 unique visiors, which should make up quite a lot for those people behind a NAT

Re:I thought it was the opposite.

[ePhil_One]

If they offer it, ISPs charge extra for static IPs. Nobody would pay extra if it wasn't an issue.

They shuffle address so they can charge extra. My ISP refuses DHCP renewals an instead forces my system to update to a new IP address. There's no sane reason to do this except to disrupt attempts to run "servers" without paying for the static IP

Re:I thought it was the opposite.

[Anonymous Coward]

The few ISPs that force people onto new IPs don't do it "to disrupt attempts to run servers". Something like a web server doesn't usually need connections more than a couple of seconds, and combined with DynDNS, they won't have a problem.

On the other hand, people downloading or playing online games will have their download aborted, and lose connection to the game. Those are the regular customers.

Plain and simple, they do it to annoy their customers, to show "you are nothing, we can do everything we like".

Re:I thought it was the opposite.

[aonaran]

And yes, he seems to have no idea how DHCP really works. Even if your lease is expired you will get the same IP address unless the pool has been exausted and your address re-used. I see that as an extremly unlikely thing to happen because it would mean, as you say, that your pool is smaller than your installed base. If you pool is smaller then you will start having issues because x number of customers will always be without a connection because they can't get an address.

But you seem to forget that that is why Cable has "static" addresses and DSL doesn't (most of the time) Cable uses DHCP DSL usually uses PPPoE with a rotating pool of addresses. Most of the time on PPPoE unless you choose your ISP wisely, or they charge you extra for the Static Address feature you WILL get a new address every time. ...and yes it is because they have a larger user base than the size of the pool.

Re:I thought it was the opposite.

[the melon]

Oh I do understand DHCP. What you are talking about is very implimentation specific. Sure, in alot of scearios the pool is smaller than the user base but from my experience that is not usually the case on Cable modem networks.

Re:I thought it was the opposite.

[bogado]

So, yes I agree it is an over reaction. :-D I just thought it was interesting enought to call atention to it.

Don't NATs skew the results too

[joeflies]

If you have a NAT and only one IP facing the net, but a bunch of machines using network address translation, can the web site determine how many unique users there are attached to that IP?

There's a kernel of an idea there...

[Anonymous Coward]

...which is that most ISPs deliver dynamic IP addresses, so users very well may be drawing multiple IPs over the period of time that they visit a website. But instead of running with idea and doing something useful with it, like conduct even a limited study (corrolate IPs & cookies, etc), he simply pulls some numbers out of his ass.

It's not so hard to come up with ideas like this - the real work is in verifying the idea, or coming up with a helpful method for figuring how those who care about such numbers can figure out what the real numbers are like.

It's lazy journalism when (1) the guy with the idea doesn't do any kind of validation of his concept before publishing, and (2) a big news site like Slashdot simply picks it up and runs shoddy work like this, tossing out a very potentially inflated 10x number...

Who cares

[Jesus IS the Devil]

Nobody knows exactly how many true unique visitors there are to a given website. And given the various ways to determine what is "unique", this muddles the pie further.

However, the important thing is, advertising rates aren't affected since they have been market-corrected for this. If an advertiser can make money, he will buy. If he can't, he won't. Whatever the true number is, it's already been factored in.

Re:Who cares

[Anonymous Coward]

Off-topic, but I love your screen name. And I completely agree. Jesus, Mohammed, any human who calls himself divine is really acting as the Devil instead and turning people away from facing God, to facing the human.

Is there ANYTHING on which Muslims and Christians can agree? Well, yes. Each believe in God.

It is only when they create distinctions as to which human they honor that the divisiveness starts.

Jesus IS the Devil. So is Mohammed.

NAT

[Anonymous Coward]

What about NAT and proxies?

NAT? 10 is too high!

[Mr.Ziggy]

There is no mention of NAT in his analysis. NAT is even included in some DSL modems these days from SBC. Lots of companies will have 100+ computers behind a single IP address.

Why is this guy's post news? Can I sit around and write bad formulas to get my blog linked by slashdot too?

AOL uses HTTP caching

[QuietLagoon]

All of the millions of AOL users visit websites via a couple of hundred cache servers. You won't see a lot of different IP addresses for the AOL visitors to your site.

I wonder if the other major ISPs do the same.

Re:AOL uses HTTP caching

[drinkypoo]

I wonder if the other major ISPs do the same.

No, most of the major ISPs just have an agreement with someone like Level3.net that handles dialin for them, and they only do caching for customers who pay for "high speed dialup" which is to say browsing through caching proxies that degrade image quality in order to reduce bandwidth consumption due to page loads.

Comment removed

[account_deleted]

Comment removed based on user account deletion

static/dynamic

[Anonymous Coward]

yeah, i agree with the other commenter who mentioned comcast. as a cable customer, although i technically have a dynamic ip address, i don't really. it changes maybe twice a year. although i would say that perhaps 2 to 1 or 3 to 1 would be a decent guess, maybe.

and he is totally not considering situations where it works the other way around. that is to say, he is not considering situations where different visitors come from the SAME ip address (as opposed to the other way around). if a huge lan exists, such as a university, which is behind a router, and 100 people from that lan visit the site, they will all appear to have the same ip. i think, right? taking that into consideration may throw the numbers in the opposite direction.

Just count visits.. why count ip's?

[xiando]

If you use http://www.mrunix.net/webalizer/ then it counts number of visits, the same IP is still the same visit if the user of that IP (re)loads a page within 30 minutes. After that it's a new visit... And it should be safe to assume one visit is one user. Saying that 1:10 is the ratio for IP/users is simply saying that every user will visit the site ten times - which seems like a worthless number without limiting it to a time-period and also, the number seems to be taken out of no-where.

Re:Just count visits.. why count ip's?

[dgatwood]

Except that with that scheme, AOL is about one user, more or less.

Re:Just count visits.. why count ip's?

[ceoyoyo]

Sounds accurate to me. Many hands, one brain. Or near enough.

NAT / DHCP

[f(x) is x]

Conversely, NAT, Proxy servers, CDNs (like Coral, CoDeeN), etc. decrease the number of IP addresses that access a page while having a large number of users see the content...

Stupid, stupid, stupid...

[C10H14N2]

Single IP addresses could be multiple people. Check.
Multiple IP addresses could be individual people. Check.
Cookies cannot be trusted to be persistant, since people routinely clear their caches. Check.

However,

Not all DSL customers are on dynamic ip.
Not all cable customers are on static ip.
The reverse of the above is also not true, so why even get into that?

So, what can we learn about IP address->Unique visitors from the above collection of information? ABSOLUTELY NOTHING.

However, you could come up with a reasonable approximation if you went to the effort of constructing a sample of known individuals and recorded their behavior and against the selection of IP addresses they use throughout a day/week/month/year. Hell, take a site like /. and run the stats on registered users. THAT would be [vaguely] useful. This, on the other hand, was just pure, pull-it-out-of-your-butt speculation.

Um... duh?

[UP_Minstrel]

Can we get a "From the 'Well, Duh' department" subheading for things like this?

Or, to completely skew his numbers...

[Dynedain]

We have 54 employees going through one firewall, and having one external IP address. On our company website, only that one IP address shows... So for that IP, it is not 1/10th of a unique visitor, it is 54 unique visitors. His numbers are baseless and skewed.

Re:Or, to completely skew his numbers...

[Anonymous Coward]

Funny, because so are yours.

This is pointless

[Who235]

You can use statistics to prove anything.
40% of people know that. . .

Re:This is pointless

[Churla]

Yes...

But 37.5% of all stats presented by people are made up on the fly.

Only about 2.31% of people know that by adding numbers after the decimal point the average person considers the number "more credible".

Re:This is pointless

[mightybaldking]

Do you have a source for this info?

Re:This is pointless

[Churla]

Yes,

But it's a dark dark place (the sun never shines there).

how about the opposite problem?

[danpsmith]

How about the people who connect from behind a router and have the same public IP, wouldn't that have the opposite effect. Sure these people _look_ like the same user, but could easily be a lot more than that.

If I send my sister a link on our home network, she could go to the site and it looks like the same visitor, etc. Everyone forgets about this too. Surely unique IPs != unique visitors, but it is somewhat close.

Caches, Proxies, NAT, etc. - weak article

[xmas2003]

While there's some validity behind what the article says (althought the 10x is ridiculously high IMHO), he fails to mention the other side of the equation which is multiple visitors from the same IP address. Your home network is almost certainly NAT'ed ... and Corporations are using proxies more and more for outbound connections. I.e. I know of one Fortune-50 company with a couple of hundred thousand employees that has a total of SIX (!) web proxies. You also see this with cache servers, especially stuff like AOL.

I agree you don't REALLY know for sure what the ratio of unique IP's to actual human visitors are (even assuming you filter out the BOTS) ... but my guess is that it is actually pretty close to 1:1 ... and in fact, unique IP's may actually undercount the number of unique visitors.

So let me get this straight

[GauteL]

This random blogger (*) proposes something fairly wild without any proof whatsoever. Slashdot reports it simply because it is a wild guess.

Hmmm... I think I'll guess that there are only 10 unique internet users in the world excluding Comic Book Guy [tm], maybe that will get me reported on Slashdot giving me 10 hits of sweet, sweet advertisement money.

(*) Well, I've never heard of him.

Not true

[BadassJesus]

There is about 500 people in my city routed throught one IP at our popular ISP in my city that makes 1 : 500 unique IP to unique user ratio, now consider many providers use this kind of routing.

At Wikipedia for example I saw my IP address under our local subject edits many times, never touched myself the article.

Re:NAT? 10 is too high!

[RingDev]

No joke, we have 800 people going out over one IP from here. Kinda a pisser when I hit the 'slow down cowboy! you just posted' message. As much as the stats are inflated by dynamic IPs and multiple logon points, they are deflated by NAT and Proxies.

-Rick

Crazy article

[TheLastUser]

You can't trust web stats, that much I agree with. The rest is a bunch of hand waving.

DSL customers do not get a new IP every time they turn on their computer. Maybe some do, but my IP changes maybe once every few months, max.

He fails to mention the effect of NAT'ing and mega proxies, both of which are in heavy use and have the OPPOSITE effect. All of AOL emerges through a small number of IP addresses, clearly more eyeballs than IPs.

I agree that IP != eyeball, but that's it, there could be more eyeballs than ips or less, who knows, and it probably varies from site to site, based on demographic. There is no way to know for sure. Cookies will only tell you the number of computers.

Re:Crazy article

[waferhead]

"DSL customers do not get a new IP every time they turn on their computer. "

People turn off their computers???

More datapoints needed

[satsuke]

I'd say he'd have to look at a specific population.

Among college students and younger, it may very well be 10:1, or worse.

For those of us accessing from work and home. That will be 2:1 assuming the same site.

For those of us behind corporate firewalls or other traffic aggregate points. It could very well be 1:1000, or higher.

Without some other data point, unique IP address statistics are next to worthless, except in "We had xxx,xxx average daily last month and xxx,xxx + xx,xxx average daily this month.

Re:More datapoints needed

[Anonymous Coward]

Why 10:1 for college students? My ResNet IP address hasn't changed since I've lived in this dorm room. Most students here don't use the labs except for stuff like MATLAB or programming.

unique IPs are so 1999

[sednet]

there are lots of challenges to accurately sizing an audience. i think that acceptance of audience size based solely upon unique IP addresses waned around 1999 when methods employing javascript and cookies came into vogue. there's always the option of counting registered users too.

all of these techniques help quantify unique users and monitor the trends in their online behavior. as far as noise in unique IP counting, i think that the biggest issue with relying soley upon unique IPs is that a simple count of unique IP addresses will include all robot noise. the major web searche engine spiders will not influence this count much, but the gratuitous IPs logged by script-kiddie bots can eclipse the human traffic on smaller sites.

Sounds familiar

[smooth wombat]

This talk about tracking people and determining the amount of visitors to a site is somewhat dated. Here is an article from October of 2005 in which, astonishingly, it is revealed that people are deleting their browser cookies so when they go back to a site they are counted as a unique visitor even though they may have visited the site yesterday.

What the author is pointing out is merely the obvious: when a site says they have X visitors they're making a guess. In fact, this link from April 30th both explains and shows why web site statistics are not accurate.

This need to say how many visitors a site has is nothing more than marketers trying to justify their costs. The trend to shove commercials down our throats using every conceivable idea including the possibility of preventing you from switching channels when a commercial comes on serves only the marketers since they're the ones who are reaping the most from inflated statistics.

Two factual flaws

[Anonymous Coward]

Firstly, cable users while on DHCP keep that address for weeks, thus one unique IP is one unique visitor. Secondly, I'd wager that a huge portion of internet surfing occurs at the workplace, which is masked behind a SINGLE IP: an enterprise of 5,000 or 10,000 employees is represented as 1 unique IP.

  Back in the day of modems and less inet prevailance, I generally thought the numbers balanced out. Today, I think there are many MORE visitors that unique IPs.

The Problem

[blueg3]

I almost knew not to read the article on the basis of the statement "the logic he used". The article did not disappoint. The logic he used is irrelevant. The whole argument is pointless because he tried to argue it logically. There are plenty of ways to inflate or deflate this number, however, as above comments have pointed out. One should not try to come at the answer with logic. Just measure it. (Yes, measuring it is not necessarily easy, but difficult-to-obtain right answer is always better than an easily-argued wrong one.)

NAT?

[NewmanBlur]

I'm no expert, but I have about 20 people in my office, connecting through the net through a NAT'ed firewall. If we all looked at this guy's blog at work, he would technically have 20 unique visitors coming from a single IP address, so that would actually deflate his number of visitors based on IP address. It seems like Max's law is a little one-sided.

Well then why the fuck did you post it Taco?

[hellfire]

I'm not sure about the logic he uses to come up with these numbers either.

News for Nerds! News! This isn't news. This is a random and arbitrary article that passes itself off as fact when it's just speculation. That's not news! Why the hell did you post it Taco if it isn't news???? Please.

Webmasters the world over know that the numbers we get aren't entirely accurate because of DHCP servers, NATs, and all the things we for IP addressing. That's not news and no webmaster really needs to be reminded of it!

What would be news is a statistical study that tries to provide facts to back up this 1/10 number, or a new, more accurate way to count people who visit your website.

And 1/10th of zero is...

[greenegg77]

Still zero.

Better approach

[Iphtashu Fitz]

When I worked for a search engine company we relied on a combination of IP address and HTTP cookie to identify unique users. True, many people disable cookies, delete them, etc. but by making use of multiple tracking methods you get a much more accurate idea of usage.

So, what he's saying is...

[zen611]

"Don't trust Stats. Except mine..."

today's math lesson

[Anonymous Coward]

1/10th = 1/10 of make sense

bad maths here

[Ankh]

There are so many factors here that focusing on one probably isn't sensible.

Some examples:

1. Even DSL users can often keep an IP address for several weeks, depending on the ISP.
2. On the other hand, any sensibly configured home network (OK, that's almost none of them perhaps) has a hardware firewall, or has multiple users with "connection sharing", so multiple users per IP.
3. Most offices use a firewall with NAT, so all of $BIG_COMPANY appears to be one IP address.
4. Some ISPs run an HTTP proxy -- AOL is one example -- so that static pages will only be fetched once per Expiry period (or once per day) even if everyone on AOL looks at them.
5. In any case, numbers of users is not the same thing as number of IP addresses; sites are reporting based on cookies or on login codes.
6. small numbers like 10 sometimes take on different values. Er, OK, no they don't but I'm bored.

I don't really know why it matters in any case. For advertising, clickthrough rate is more important than number of users, and they are not very closely related. Sadly, the poorer your site's navigation the higher the clickthrough rate (and the fewer pages on your site people will see each visit, as the ads take them away sooner).

This is news?

[Anonymous Coward]

This makes no sense.
a) As stated in the post, we don't know what this guy bases his information on.
b) 10:1 sounds rather inflated. I know plenty of people that have 1 computer per IP (NAT or not). So if the average is 10:1, then there has to be a lot of people with more than 20 computers behind an IP.
c) It's not really news, it's just some guy ranting about something.
c) We post it as NEWS anyway.

No cookies

[Gattman01]

I've been know to browse sites on my two machines at work ( with different realworld IP addresses ), then visit it later when I get home.
So I would appear as three unique visitors. Cookies wouldn't solve this problem.

Using just cookies wouldn't work if a person used multiple browsers on their machines, like me.

A Hybrid would work best.

I like using lynx so I can block cookies and not have to deal with adds. :-)

Visitors not trusted in the Web Analytics Industry

[boscowall]

I'm not sure about his article and his formula, but it is already a debate in the web analytics industry. http://www.omniture.com/blog/ Even using cookies it's nearly impossible to get correct unique visitor counts and that is why the industry is moving more towards unique visits, because a visit is a visit, it doesn't matter who the visitor was... The only way to really measure how far off visitor data could be is comparing unique customers (cusomter id) to the number of unique visitors they create (the customer id coming from a login). That way they could see the affects of multiple customers on multiple machines and browsers and also see the affects of multiple customers on a single machine and browser.

What about proxies?

[wbhauck]

Proxies could, especially ISP proxies (AOL, anyone) can hide potentially 10,000's of unique users.

Also, as far as i've seen DSL IPs don't change that often.

You know you're dyslexic when..

[OmgTEHMATRICKS]

The article title looks like "Unique Visitors = 1/10th of Unique PIs?"

On the contrary...

[abiessu]

For the sites I do traffic analysis for, I've noticed that certain end-users will have their IP address change during a session (i.e., between one page request and another, often within minutes or even seconds of each other). AOL users seem to be in this situation, along with one other major ISP (I forget which). When the IP changes that often, I started trying to figure out other ways to count unique visitors. I still haven't come up with anything particularly good.

So, if the main users of the sites in TFA are from AOL (or some such), the 10 to 1 effect is one of the likely possibilities.

However, TFA does appear to use speculation rather than actual numbers...

Re:On the contrary...

[ranson]

AOL user's actual IPs do not change mid-session, but the web proxy server that is making requests on behalf of the user changes all the time; my server logs even show certain objects on a single page for the same user are requested by different cache-*.aol.com servers.

Lots of info about that is here.. including the proxy IP list, etc... http://webmaster.info.aol.com/proxyinfo.html they say specfically "When a member requests multiple documents for multiple URLs, each request may come from a different proxy server. Since one proxy server can have multiple members going to one site, webmasters should not make assumptions about the relationship between members and proxy servers when designing their web site."

not /. worthy

[tobiathan]

This guy is so far off base, that this is really just sensational, and hardly worthy of being /.ed

CMP

[Anonymous Coward]

This article is from CMP media.

This is a technology article from CMP media.

...

...

What a waste of time.

[loftwyr]

So other than advertising a blog entry that, at best, is deeply flawed due to the complete misunderstnading of NAT, DHCP lease times and the fact that no reputable site uses IP addresses as a basis for their stats, what exactly was this article about?

The key is not the Ratio, it's the Revelation

[EvilMagnus]

It's that this is a Marketing Person who has realised that IP != Unique User.

That places him amongst a tiny minority of marketing people, even if his reasoning and ideas on methodology are just as batshit insane as the rest of his kin.

Re:The key is not the Ratio, it's the Revelation

[imidan]

Actually, if you go back and look at the rest of his blog, you'll find that he claims to be a developer. In fact, when he attended University of Tulsa, he was apparently surprised to find that some few students there were actually smarter than he was! So he's clearly a very smart developer!

No, but really, if you browse the rest of his blog, he just comes off sounding like a dumbass. Well, more of a dumbass than he sounds like just from this nonsense about unique visitors to his web site.

Re:The key is not the Ratio, it's the Revelation

[robertjw]

comes off sounding like a dumbass.

Appearantly not too much of a dumbass. Managed to get his blog slashdotted. He's probably buying a new car with his adsense check about now.

I respectfully disagree

[hummassa]

Estatistically speaking, and in the long run (meaning tending toward infinity) the "profiles" of people that browse site A and site B would fall in more or less the same -- meaning it's not the overcounting or the undercounting that cancel each other -- is the the over/undercounting for site A is more or less the same over/undercounting for site B.

Therefore, if site A has 10 million IPs logged yesterday and site B has 5 million IPs logged yesterday, it's fair to say that site A has double the traffic of site B.

That is, unless you can prove that the user "profiles" (PPP, PPPoE, people that use lots of different computers versus proxies, NATted people etc) are wildly different between site A and site B -- which can occur e.g. in pr0n sites (which would be blocked by most corporate proxies but not home computers etc), the number of distinct visiting IPs is a good measure of site traffic.

Re:I respectfully disagree

[Anonymous Coward]

Therefore, if site A has 10 million IPs logged yesterday and site B has 5 million IPs logged yesterday, it's fair to say that site A has double the traffic of site B.

Not fair at all. For example, if site A has Cache-Control: public set, but site B has Cache-Control: private set, site A isn't going to log a lot of IPs that do in fact visit their site, but site B will log every IP that visits their site.

Or, if they both have Cache-Control: public set, the IP:user ratio will change when something as simple as AOL tuning their caching parameters happens.

Not only do user profiles differ, things that have nothing whatsoever to do with the users can drastically skew the numbers.

That is, unless you can prove that the user "profiles"... are wildly different between site A and site B

That's a huge, implausible assumption, and the burden of proof is upon you to prove it, not upon me to disprove it.

Dynamic?

[ElephanTS]

I've got cable with supposed dynamic addressing but in truth it has remained the same for over a year. There was a time it changed every few days but I presume they upped the allocations or whatever. I think it's interesting what this guy is saying but (as many people are saying) the data is very confounded and many different reasons could apply.

Why Was This Dignified With A Link?

[CheeseburgerBlue]

I recently put up a blog post entitled, "The Secrets of the Universe, As Released From My Mystic Corn-Hole" and I got practically NO TRAFFIC. What's the deal here?

This tripe gets posted, but my butt-wisdom goes ignored?

It should be obvious to everyone that Slashdot is under the control of the Scientologists.

Govn't Analysis of Website Traffic

[Fireye]

I work for a contractor that maintains/develops/hosts a govn't site. I'm working right now to transition our Webstats Analysis tool off of an old copy of Webtrends (crappy product), to Urchin (decent product). Measuring Unique visitors is IMMENSELY difficult on a government site because you aren't allowed to leave cookies that exist beyond the user session!

    I _have_ to go by IP+User Agent. Cookies would make everything 10000x easier, but because of privacy laws, it's not usable.

The suits don't care

[Anonymous Coward]

All the suits want is a number, any number. When working for an online news company, I tried to explain to a group suits that the traffic numbers we had (unique IPs) wasn't really an accurate figure -- that its only real value was in comparison to itself over time, to show traffic growth.

To illustrate, I showed that a large portion (30% +) of the IP addresses visiting our (local news) site were originating in either Virginia or Washington State -- and that that was because AOL and MSN have their operations.

No one wanted to hear it. Someone suggested instead that maybe it was because there are large military installations in both places, and soldiers from our area were checking for local news. Plausible, but probably not to that extent.

Eventually, it was agreed that we'd keep telling advertisers what we had been telling them, which was how many 'visitors' the site had, based on IP addresses.

This kept everyone happy; our advertisers thought they knew what they were getting for the money, and our salespeople continued making sales with no ambiguity.

Nobody cares about accuracy -- they just don't want ambiguity. I've seen this applies to lots and lots of fields, not just web traffic stats.

My own stats say very different things.

[CFD339]

I did a quick analysis of a 250,000 line entry server log. I counted unique ip addresses, unique useragent cgi values, and then the number of unique combinations.

A useragent value looks like this: Mozilla/4.0 (compatible; MSIE 6.0; Windows 98; .NET CLR 1.1.4322)

Although even this is hardly reliable since useragent can be faked, and useragent isn't unique enough to be a client fingerprint -- its still helpful in this context.

One can make the assumption that a given user's "useragent" value isn't going to to change much on a day to day basis, though it will not stay the same over time as vesions get updated. GENERALLY speaking, the same IP address but different USERAGENT values would indicate different people from behind the same NAT firewall, or different users assigned the same DHCP address.

Here's what I got for results -- it looked like counting only unique IP's gave you only about 85% of the unique hits.

Total Hits Looked At: 249861
Unique IPs: 10309
Unique UAs: 1578
Unique Combos: 12232

Re:My own stats say very different things.

[robertjw]

Here's what I got for results -- it looked like counting only unique IP's gave you only about 85% of the unique hits.

So, shouldn't this actually (more or less) average out. You looked at unique combos of IP and useragent. Isn't it probable that some of those combos are actually the same user who's IP has changed, but has the same useragent as well as unique users behind a common firewall?

NAT?

[peacefinder]

While he has a point (I visit particular sites from at least two different networks on a regular basis) there are a couple things he doesn't take into account.

Firstly, every DSL I have ever worked with is a peristent connection. (Why would anyone bother with an on-demand DSL?) It may not be a fixed IP, but it is a pretty darn sticky IP. It only changes when one endpoint of the DSL circuit is reset, and that generally doesn't happen much more often than monthly. If you're tracking unique IPs over the life of your site, then sure... hits from IPs assigned to DSL users will be very inflated by this effect. But if you're tracking unique IPs in the last month or so, then the inflation will be pretty minimal.

(Incidentally, the same effect will occur as users change ISPs; your unique users will move to different IP pools each time they switch. Is that worth taking into account?)

Secondly, he doesn't take into account NAT. That's obviously not a big deal when counting home users, but it is a big deal when counting anyone on a corporate network. Tere could be one or one thousand unique users behind every NATted address... if you're counting just unique IPs, there could be a serious undercount of unique users in this space.

Wouldn't it be easier to say "Unique IP hits bear only a loose relationship to unique user hits" and leave it at that? The measurement may be inflated, but it is simiarly inflated for most websites. And when comparing one site to another, that seems good enough.

Poorly thought out

[shawnmchorse]

He's correct about visitors that visit from many different IP addresses, particularly AOL users that weren't really mentioned. The same AOL user can have several different IP addresses on just one visit to a web site, due to the way that their proxies and such work. I distinctly remember phpBB running into this issue because it wanted to associate each user's login cookie with their IP address for security, but with AOL users there's just no rhyme nor reason to their IP addresses.

The flipside that's not considered at all though is the number of places that have any number of unique visitors all using the same IP address. Everyone in my office will have the same external IP address for any web site we visit. I know many other offices are exactly the same way.

What does this mean overall? I think they balance out for the most part. Some people are overreported, some are underreported, and it probably all balances out in the end.

On the contrary

[unity100]

Real unique visitor numbers are always much higher than IPs, or anything that the stats programs comment as 'Unique Visitors'.

Stats program rely upon ips, browsers, other identifiable factors to asses what traffic is 'viewed'.

This is a thing of yesterday. Today there is much 'privacy' and 'security' hype going on, networks are showing the users to the remote server as they are a single user, people in their homes use privacy blocking stuff to prevent many identifying info about the client pc being submitted to the remote server that holds the site visited.

Thus, it is natural that many actual visitor traffic is now being interpreted as 'not viewed' traffic. But how much of it, what is the ratio ? That totally depends on the visitor spectrum of the site, hardware and measures they use in their own desktops and so on.

Unique Slashdot Articles...

[Anonymous Coward]

Unique Slashdot Articles = 1/10th of Total Slashdot Articles?

My dynamic DSL ip hasn't changed in 2.5 years

[sd.fhasldff]

"So, what IS the typical holding interval for a DSL ip?"

Anyone who has a router setup as the primary access point will always have the same IP (barring an ISP reset of some kind - or turning off the router for some reason). I've had the same IP for over 2.5 years now and my pathetic Cisco 678 router gets it via DHCP (I know, I checked in CBOS). The router needs to be rebooted every so often (which prompty causes it to reset the NAT table to some pseudo-arbitrary point in the past, disregarding any changes even through "written") and I still get the same address.

DHCP virtually always assign a MAC the same IP address as it was last seen with, providing it has not been taken already. Since IPs are assigned by FILO, that would mean the stack has been completely used. Unless the ISP has an extraordinarily tight supply of addresses (in which case you'd probably do better with a different ISP), that's not likely to happen if you use your computer regularly.

Put a stop to inaccurate statistics!

[suv4x4]

I urge you people! Mandatory registration on each site! Three names, e-mail, address, age, sex, and social security id!

I've ALWAYS had the same IP (going on 15 years)!

[sd.fhasldff]

My IP has been 127.0.0.1 for a really long time now. Ever since I got my first internet connection, actually. That must be why it's such a "nice" number and not those horribly complicated ones other people always seem to have.

IP + Cookie

[Anonymous Coward]

I work for one of the big four accouting firms and commonly attest traffic reports from websites that are pubilically listed. There is a requirement for this where internet traffic is published in financial statements.

The calculations used are pretty accurate and certainly not what you would get reported straight from Webtrends or similar. We take a unique user as a combination of unique IP and Cookie. However, we also identify and exclude cookie storms - this is where a browser refuses a cookie and so you get a new cookie for every hit (yes I do mean hit and not page, or visit).

I am sure that non-publically listed companies aren't quite as rigerous.

Eric

Re:IP + Cookie

[Anonymous Coward]

Should have mentioned that we also do lots of other filtering such as robot traffic etc.

A unique vistor become unique again after 24 hours or when 00:00 occurs though this will always be less than 0.01 of traffic for largish sites.

A little knowledge is a dangerous thing

[itdood]

ummmm, what about Proxy and NAT? If anything, the opposite of what this article concludes is true. What a waste of my 4 minutes. I want my money back!

Re:A little knowledge is a dangerous thing

[Anonymous Coward]

This may seem boring but is actually the cause of the dot com crash in 2000. People where investing in newly formed companies (with no financial history to see) based on their growth in traffic. The traffic numbers were bogus, the price crashed we all suffered greatly.

Think on!

Cookies optional

[RareButSeriousSideEf]

Do you have any kind of login-dependent features, from which to infer a user:IP ratio for the larger, no-cookie group? Is there any data on the relative IP diversity of users who login to a site versus those who don't?

To get a bigger set of sample data, hmmm... I suppose even if you have to allow for a % of cookie-blocking users, one could still code the more complex dynamic stuff against a server-side cache / session (to avoid forks), and then a noncritical "Uniqueness" cookie would probably still yield enough data for a meaningful estimate on the IP-count question.

(and with that, I'm off to go determine just how un-unique my visitors are...)

Lies

[dascandy]

There's lies, damn lies and statistics.

Binary?

[Spez]

1/10? In decimal, its 0.5 is it? Thats not that bad...

THE POINT IS

[Anonymous Coward]

My point is that many non-publicly owned sites report unique IPs as a measure of unique visitors, and this number is totally incorrect because of mixture of dynamic and static IPs.

I further insist that there is no general fomula except to conduct a study on how frequently folks visit the site and what's the ratio of unique IPs/cookies to unique visitors.

A study is needed indeed! But who is willing to participate and admit as a result that they have only a fraction of unique visitors?

Different user-agent does not mean different user

[lamber45]

While in some cases a different user-agent could arise for the reasons you cite, it might also be that someone views the same site with a different browser because the first one didn't work very well; or that someone has multiple computers (say a laptop and a desktop) behind the same personal firewall.

On the other hand, if there's a whole lab behind one IP address, it's quite possible that each system has the exact same configuration...

Re:I've ALWAYS had the same IP (going on 15 years)

[operagost]

I have bad news for you, buddy... I h4x0r3d j00 last night!!!!111

Yes, both of those are accurate.

[CFD339]

And it gets worse! If users also use an RSS reader, it has its own useragent signature. If you assign cookies, plus use useragent, plus use the ip, you start to get close I suppose, but there really is no good way to make valid measurements.

Nonsense

[Anonymous Coward]

Even if your lease is expired you will get the same IP address unless the pool has been exausted and your address re-used. I see that as an extremly unlikely thing to happen because it would mean, as you say, that your pool is smaller than your installed base. If you pool is smaller then you will start having issues because x number of customers will always be without a connection because they can't get an address.

The pool is normally smaller than the number of users. Otherwise, there would be no point in having a pool - each user could have his/her own IP address.

It works because, if you have enough users, they are never all using the internet at the same time.

UK Academia

[grahamsz]

I'm not sure if it's still true, but a few years ago it was the case that virtually all international web traffic for the entire UK academic network was funnelled through 15 or 16 cache servers.

Re:UK Academia

[ArtStone]

However, anyone with experience and tracking web users knows that all of the requests coming through most proxy servers have either an HTTP_X_FORWARDED_FOR or HTTP_CLIENT_IP http header that identifies the individual IP behind the proxy.

The default Apache web logs do not log the source IP behind the proxy, but the headers are available to PHP for analysis... getenv("HTTP_X_FORWARDED_FOR")...

The client IP may be a non-routable IP address (192.168.x.x, 172.16.x.x/12 127.0.x.x, 10.x.x.x), but in combination with the IP of the proxy, it should still be unique.... Of course, proxies can be nested making it more fun...

A proxy which does not divulge the actual IP of the address of its user is called an anonymous proxy - typically those are used by folks trying to hide their identity.

People who think they are anonymous just because they are behind an ordinary proxy misunderstand how proxies work.

One final caveat - the http headers are only as trustworthy as the proxy itself. Just because a proxy offers up a proxy header does not mean that the information being returned is valid. A proxy can be configured to deliberately mislead about the actual IPs.

A important factor in the IP/Unique visistor guestimate is AOL. AOL has a significant number of users (therefore can't be ignored).... AOL's typical user sits behind their proxy farm in Virginia which creates at least three problems:

1) many of their page requests are satisfied from the AOL proxy farm without ever sending a request to your web server (unless you force the AOL cache to not cache your pages) so you may have visitors that you never see if you mostly offer up static content.

2) If you forced requests, the requests from the same user will random appear over a range of proxy IP addresses - as AOL makes no effort to force a user through the same proxy every time. If the AOL user allows cookies, they can be tracked over the IPs... if they are blocking cookies, then the best you do is track them based on time proximity or adding session tracking to the page requests URLs....

3) Since the AOL users are all funnelled through Virginia (including AOL users in Europe unless they have changed that), any kind of geotargeting of users (think Google Adwords) will assume the AOL user is in Virginia. An AOL IP (even if they have bypassed the proxy) gives -no- indication of their geographic location.

Note to Slashdot technical people. Something (probably the random ads) is injecting a center tag that centers the entire content of the entire page randomly. Pretty uncool. :)

I don't think so.

[CFD339]

The period of time involved here is too short for a large number of users to have different IP's each time. I'd have to do more complete analysis to know for sure, for example if the IP's are in similar address groups or not.

Its never going to be close to perfect, but it clearly refutes the 10:1 statement in the article.

Page Tagging

[barnaclebarnes]

Most real web analytics services use page tagging to measure unique browsers. IP address analysis is just not valid. Real analytics packages screan out spiders and other non human traffic as well. Of course even page tagging with cookies is not totally fool proof - 2 people could use the the same computer with the same login and one person may use a different computer at home and at work but it is better than any other method for measurement.

Note: If you want to see how page tagging works the easiest way is to check out Google Analytics.

1:10 is not 1/10

[dasnipa]

the 1:10 notation would mean 1/11th not 1/10

How hard is it?

[dafragsta]

... to just register a cookie variable that's unique to that browser that doesn't have to expire for a long time and that key is checked in a database. Sure, it's not secure, but if you want to inflate the perception of unique visitors, there are always ways around that. Actually, just checking for the cookie alone lets you know if someone's been to the site before. There are lots of ways to track users coming and going even without logins, by using cookies.

Re:How hard is it?

[cnerd2025]

register a cookie variable that's unique to that browser

Two problems:

1. Not all users have cookies enabled
2. There is no such thing as an artificial "random" number generator

Out of those two, the first is the only one of consequence. Assuming all users enabled cookies, tracking would be a matter of date stamping and IP addressing. It would be the form of XXX_XXX_XXX_XXX-time_since_epoc. Having users actually enable cookies is another story... Many people have been tricked into thinking that cookies can begin running your computer when your eyes look away. The real truth is that unless cookies hold sensitive information unencrypted, they pose no real risk. If we really care all that much, why not get server data about the orginating MAC address. I'm sure you can find it. And all MACs are unique.

Bzzzt - Wrong

[craznar]

Neither have static IP addresses in most cases, Cable may be more likely to hold an IP lease longer, but it is basically dependant on the stability of the connection and the lease times the ISP gives out.

Alot of ISPs in Australia also offer sticky IP (== long lease time), makes traking naughty customers easier. I'm sure the US providers do this as well.

If a DSL customer has 10 times as many IP addresses as a Cable customer in a month, that's a reflection of the ADSL quality - rather than anything else.

Bullshit!

[angel'o'sphere]

I think i use this headlien from now on when I find bullshit!

The author has a point: the number of recorded IPs has no real connection to the number of users.

NAT router: lots of unique users with same IP.
Interesting site like /. : I connect to there with a different IP each day, and with several different IPs on several days if I use a different computer, e.g. at work.

However, the articel is bullshit, because the author reversed the explanaition:

While it is obvious that DSL visits would be inflated they are drawn from a limited pool. Given enough time the entire pool will be drawn by DSL customers thus giving the impression that there are as many unique visitors as there are IPs in the pool


So, the number of visitors equals the number of IPs? If there are indeed so many visits, that is not that bad.
However: if EVERY customer of that DSL provider visits the web site in question, you still ONLY see as many visitors as you have different IPs, but you have far more visitors than IP numbers. The opposite of the authors conclusion is true.

IMHO: the complete idea to measure visitors via IPs is moot anyway. If you want to measure that, use a userlogin/password and/or a cookie. User login/password combiantions you easy get by offering customization (e.g. liek amazon offering you stuff yoou might be interested in, concluded from your last boughts)

angel'o'sphere

Re:Bullshit!

[Anonymous Coward]

Nope! You can draw the entire pool even with few customers given time. Laws of statistics, you know.

Re:Bullshit!

[angel'o'sphere]

Sure you can. Never doubted that. But both ways are happening.

Afterall the statement of the guy is very inacurate what exactly he wanted to measure anyyway.

But lets asume a worst case scenario: 100 DSL users, 1 IP number -> with 50 visits on the measured site, no one knows if it was 50 times the same user or 5 users 10 times each or 50 users 1 time.

With 1 DSL user and 100 randomly shifting IPs after 50 visits you got a sampling of x 50 IP numbers (because 1 or 2 might have been used several times).

So. how can you distinguish the two experiments?
How can you "figure" in anyway in any of the two how many IPs in total and how many customers in total are connecting?

I find this approach very strange ;D

angel'o'sphere

Que?

[Ayanami Rei]

No modern webbrowser allows cookies to "leave a trail" of where you've been anymore.
Cookies are only sent in requests to the domains that created them in the first place.

I guess one valid point would be that ad-networks can sort of track you by determining what sites who use their banners you visit based on referrals headers. The issue there is:
1) There's no user-identifiable information that links your cookie to the pages you visited feature that adnetwork's banners aside from your IP address at the time
2) Adnetworks don't care _who_ you are. They use the cookie to determine what sites should be considered "related" so they can better determine how to distribute banners. (That is, banners that do well on one site might do well on sites visited by people who visited the first site)
3) You can always block referers to images/iframes hosted on external domains (adnetwork stuff). IIRC this is the default now on Firefox and Safari... so... its irrelevant now.

disproof

[TheSHAD0W]

I'm not certain what site Mr. Fomitchev was benchmarking, but on one website I administer, I get ~25,000 unique visitors per day, and nearly 20,000 downloads of the 3.5 megabyte software package on the site. Now, I don't believe very many people will be downloading multiple copies of the software, nor do I think most of the bots that hit the page are going to waste their downstream to grab copies, either.

Dumbass

[Anonymous Coward]

Not only is his maths flawed, but it is "fewer" not "less".

sense sation

[edis]

not one to ten, neither one direction thing.

then, why?

Comment removed

[account_deleted]

Comment removed based on user account deletion

A Helpful Mnemonic for the Dyspellic

[DrVomact]

I visit a webSITE.
I aim with my gunSIGHT.

In truth, you are all blameless. English is a terrible language, and people should try to avoid using it as much as possible. Of course, most already succeed without even trying.

Re:A Helpful Mnemonic for the Dyspellic

[Sj0]

Actually, most languages I know of tend to steal bits and pieces from english as time progresses. Japanese is a particular offender, because of the speed with which they were shunted into the 19th century. Others still do it, though.

Inconsistency on Slashdot

[Anonymous Coward]

"How are we supposed to know what our visitors find interesting?"

Ummm, to answer that question maybe you could try tracking your own damned web site, instead of tracking users?

Let's say you have four web pages. Page three gets the most hits. Golly, maybe people like the content on Page 3 better? Ya think?

You are lying to yourself if you think you need to track OTHER PEOPLE instead of tracking your content.

I don't get it. Why does the Slashdot crowd scream bloody murder when the government or industry uses RFID or some type of surveillance technology ... and yet turn around and argue that everything is groovy when some random webmaster does it?

Surveillance is surveillance. With limited court-supervised exceptions, it is always immoral and dishonest and should be illegal.

People don't turn off cookies and the like because they feel mildly pestered by a website's tracking. WE FUCKING HATE IT. Tracking is quite simply wrong behavior by websites, and indefensible.

so

[xmodem_and_rommon]

so why can't the virtual host just be sent along with the HTP data in the encrypted SSL stream?

Re:so

[Old Man Kensey]

so why can't the virtual host just be sent along with the HTP data in the encrypted SSL stream?

As I understand it, SSL certificates are tied to the name and IP of a secure server (this is to prevent man-in-the-middle attacks). So if you give every one of your secure virtual hosts its own routable IP, yes, you can virtual-host over SSL -- but you're not name-based-virtual-hosting any more. Otherwise your users are going to get warnings about the certificate not matching the server.

I don't know if it's possible to get umpteen certificates for servers with different names but the same IP. Does SSL certificate verification involve any kind of checks on reverse-resolved DNS data?

Mod Parent Down - Totally wrong

[stry_cat]

When I lived in DC, I had Verizon DSL. Connected via PPPoE and only had a total of two IP addresses in 3 years. Comcast where I live now, uses DHCP and again I've had the same IP address for 2 years and only 3 IP address in 5 years. Dialup is the only time I've ever gotten a different IP every time I connected.

Also where I work, has 4000 people + 1000 public internet servers all behind just a few IP addresses. The parent and the article are just totally wrong.

Re:I've ALWAYS had the same IP (going on 15 years)

[marcosdumay]

Lucky you! Mine is 127.231.17.68... Much harder to remember. Wait, the net mask is 255.0.0.0... Are we on the same ISP? That must be a huge ISP, to use an entire A space...