Unique Visitors = 1/10th of Unique IPs?

Max Fomitchev submitted a little blog entry where he proposes that the ratio of unique IPs to actual unique users is 10:1. This flies in the face of the numbers you usually see attached to these sorts of things. I'm not sure about the logic he uses to come up with these numbers either.

Comments

10 was arbitrary

[op12]

The 10 was a hypothetical...the only point was that you can't trust the number of recurring visitors that a site reports because they users come back with a different IP (obvious) and get counted twice. Couldn't one use cookies and IPs in combination to get a better gauge? The IP may change but the cookie would not. Sure some may delete it, but it'll still improve accuracy at least a little bit.

Re:10 was arbitrary

[MikeFM]

My experience is that a lot of users use cookie killing software that removes cookies every time the browser is closed or just reject cookies altogether. Also many users seem to use multiple browsers and computers even within small time periods. Counting unique visitors is really quite difficult. Still, if all site's play by the same rules on counting the number still has some meaning. Unique IP address within a given timeframe is probably a decent metric still.

Re:10 was arbitrary

[bbsguru]

Oh Puh-lease! EVERYONE knows the REAL number is 11.32019 per IP address. This is just silliness! There is no magic number that works everywhere; fuggidaboudit. If it matters that much to KNOW the real number of unique visitors, ask each one for a scan of their right thumbprint, and then create a database. I thought so.

Cookies?

[Anonymous Coward]

Isn't this what cookies are for? Sure lots of people wash them. But I'm betting the majority of people do not.

Re:nothing much here

[GigsVT]

I'm on DHCP on Cable. I don't think I'm the only one. I guess maybe the article wanted to show by demonstration how most statistics are made up on the spot.

Maybe not...

[Anonymous Coward]

I've hosted several servers from home for years at a time without my dynamic IP address ever changing, and I've known many others in the same situation. I think this 10x rule might be a bit extreme...

WTF?

[giorgiofr]

This argument is flawed. Logging to Slashdot now from my house and two hours from now from my friend's house should count for two visits, and so it rightfully does. The article writer seemed to have a problem with this? ZOMG 2 different IPs...
And if my IP has changed but I'm still here... that's because I haven't surfed for many hours at least otherwise the lease will be renewed and the address will stay the same. So it should still count for two visits. Duh.

Re:Already considered.

[Nos.]

Because its none of your business (you being a web"master"). I understand that my Ip is broadcast, but I'm not going to use a web browser that sends personal information about me or my computer, every time I hit a site. I can use an anonymous proxy now, and refuse/delete cookies, and know that I am not being tracked. Besides, MAC addresses can be altered as well.

The author left out surfing from work

[Trails]

I do most of my news browsing at work, where several hundred people show up as one IP (home computer is exclusively for WoW).

Besides, the assumption that stated unique visitors = actual unique ip's is innacurate. Lots of companies track users with some kind of UID cookie, for more accurate stats. True, this isn't perfect either, and will reset when users wipe their cookies or it expires, but is probably closer to the real number than ip's.

Re:nothing much here

[zippthorne]

Well it appears to make the assumption that the visits are sparse enough for the DSL Ips to change every time (and also that cable IPs are static. Mine is not, though it doesn't change for months at a time)

So, what IS the typical holding interval for a DSL ip?

as for properly estimating, If there are good enough statistics to have separate numbers for both {known, relatively static IPs over a month} and {known dynamic IPs} you could find the ratio of returning static IPs and normalize the dynamic ones to match that ratio.

In fact, I'd be surprised if this wasn't already being done for many sites.

I thought it was the opposite.

[Black Perl]

First of all, a DHCP server is typically going to give you the same IP address each time your computer requests it, unless there are more users than IP addresses, in which case there will be some shuffling. But that tends to be when there are more users than available IPs.

There are entire domains hidden behind a NAT device of some sort. This would be many users per IP address. TFA didn't mention this at all.

So I think TFA is indeed arbitrary, and also wrong.

Re:I thought it was the opposite.

[the melon]

Yeah, here at Sun there are nearly 40 thousand people that connect through 40-50 different proxy servers. That is a thousand to 1 in the opposite direction the article claims.

And yes, he seems to have no idea how DHCP really works. Even if your lease is expired you will get the same IP address unless the pool has been exausted and your address re-used. I see that as an extremly unlikely thing to happen because it would mean, as you say, that your pool is smaller than your installed base. If you pool is smaller then you will start having issues because x number of customers will always be without a connection because they can't get an address.

Had he mentioned Dialup users then I would be more inclined to agree because you are very likely to get a different address every time you connect.

Re:I thought it was the opposite.

[ranson]

Hmm, well TFA was speaking to DSL users, who generally connect via PPPoE. PPPoE, while it does dish out IPs a la DHCP, does not have a lease file that associates user mac addresses with the IP they were handed on previous connections; users generally will get a different one every time they connect just like dial-up (aka PPP). I would say router-based PPPoE connections drop and reconnect at least once per week on average.

Re:I thought it was the opposite.

[the melon]

Actually the oposite should be true. The bigger the pool the better your chances are for getting the same IP. When you release your DHCP lease from the server or it expires before your client renews it the server still has that address assigned to your MAC address. The only difference is the lease is inactive. If there are addresses in a pool that have never been assigned and a new client connects it will use those addresses first. If there are none then it will start checking the inactive leases. It will take one, not sure if it does them in order form oldest expiration or randomly, and ping the address to make sure it is not still in use. If it is not then it assignes it to the new client. If it is in use then it moves on to the next one and marks that address as unuseable. ISC DHCP will revisit unuseable address once it has exausted the inactive pool. Solaris DHCP will not revisit them until they have been marked good again by an admin.

So the longer you are inactive the greater your chances are that you will get a different address and the bigger the pool is the lower your chances are.

The above is of course null and void if the ISP deliberatly expires leases and forces users to new addresses. It has been my experience though that very few do this. Even my dealings with PPoE based DSL has indicated that as long as you stay connected via a keepalive then you will retain the same address for a while. If you restart your router with PPoE you are likely to change though.

I wonder what percentage of people with DSL and Cable do not use a firewall/NAT box?

AOL uses HTTP caching

[QuietLagoon]

All of the millions of AOL users visit websites via a couple of hundred cache servers. You won't see a lot of different IP addresses for the AOL visitors to your site.

I wonder if the other major ISPs do the same.

Or, to completely skew his numbers...

[Dynedain]

We have 54 employees going through one firewall, and having one external IP address. On our company website, only that one IP address shows... So for that IP, it is not 1/10th of a unique visitor, it is 54 unique visitors. His numbers are baseless and skewed.

Visitors not trusted in the Web Analytics Industry

[boscowall]

I'm not sure about his article and his formula, but it is already a debate in the web analytics industry. http://www.omniture.com/blog/ Even using cookies it's nearly impossible to get correct unique visitor counts and that is why the industry is moving more towards unique visits, because a visit is a visit, it doesn't matter who the visitor was... The only way to really measure how far off visitor data could be is comparing unique customers (cusomter id) to the number of unique visitors they create (the customer id coming from a login). That way they could see the affects of multiple customers on multiple machines and browsers and also see the affects of multiple customers on a single machine and browser.

What about proxies?

[wbhauck]

Proxies could, especially ISP proxies (AOL, anyone) can hide potentially 10,000's of unique users.

Also, as far as i've seen DSL IPs don't change that often.

My own stats say very different things.

[CFD339]

I did a quick analysis of a 250,000 line entry server log. I counted unique ip addresses, unique useragent cgi values, and then the number of unique combinations.

A useragent value looks like this: Mozilla/4.0 (compatible; MSIE 6.0; Windows 98; .NET CLR 1.1.4322)

Although even this is hardly reliable since useragent can be faked, and useragent isn't unique enough to be a client fingerprint -- its still helpful in this context.

One can make the assumption that a given user's "useragent" value isn't going to to change much on a day to day basis, though it will not stay the same over time as vesions get updated. GENERALLY speaking, the same IP address but different USERAGENT values would indicate different people from behind the same NAT firewall, or different users assigned the same DHCP address.

Here's what I got for results -- it looked like counting only unique IP's gave you only about 85% of the unique hits.

Total Hits Looked At: 249861
Unique IPs: 10309
Unique UAs: 1578
Unique Combos: 12232

Re:The key is not the Ratio, it's the Revelation

[imidan]

Actually, if you go back and look at the rest of his blog, you'll find that he claims to be a developer. In fact, when he attended University of Tulsa, he was apparently surprised to find that some few students there were actually smarter than he was! So he's clearly a very smart developer!

No, but really, if you browse the rest of his blog, he just comes off sounding like a dumbass. Well, more of a dumbass than he sounds like just from this nonsense about unique visitors to his web site.

Re:Use cookies

[gstoddart]

Unfortunately, some users disable cookies. And then all you can do is fall back on their IP address.

Of course we block cookies. Because most of the cookies you get offered are 3rd party to the site you're visiting and just crap so gator and all of that other junk could keep track of you. I only accept cookies coming from the site I'm visiting, and then only if I say YES. It took a very long time to teach a lot of people they needed to be more cautious with cookies, because there were a lot of privacy issues with them. If I could block my IP address, I would do that too. =)

It would be nice to see cookie-tracking support in Open Source stats engines like awstats.

But we don't want to be tracked. That's why we disabled the cookies in the first place. I got so damned tired of things like doubleclick trying to set cookies back in the day -- if you need to give me a cookie to make your site work, *maybe* I'll accept it. Giving me third-party cookies? No flippin' way!

Course, I'm pretty anti-advertiser and the like. So I'm probably not a good example. :-P