Slashdot Mirror
New Method of Tracking UIP Hits?
smurray writes "iMediaConnection has an interesting article on a new approach to web analysis. The author claims that he is describing 'new, cutting edge methodologies for identifying people, methodologies that -- at this point -- no web analytics product supports.' What's more interesting, the new technology doesn't seem to be privacy intrusive." Many companies seem unhappy with the accepted norms of tracking UIP results. Another approach to solving this problem was also previously covered on Slashdot.
Barcoded humans =)
new, cutting edge methodologies for identifying people....the new technology doesn't seem to be privacy intrusive
The Wookie defense in action!
time is a perception of a being's consciousness
time is your 6th sense, the wierd ones are 7+
Sending your PCs unique CPUID along with every HTTP request would be ideal for this. You could also group up websites and use this to track people across websites. It would be great for marketing and for law enforcement.
Oh, you all disabled your nice Intel CPUID? Why ever would you want to do that?
International Union of Private Wagons
Quimper, France - Pluguffan (Airport Code)
Ultimate Irrigation Potential
Uncovered Interest Parity
Undegraded Intake Protein
United International Pictures
Universidad Interamericana de Panamá
Unusual Interstitial Pneumonitis
Upgrade Improvement Program
Urinating In Public
User Interface Program
USIGS Interoperability Profile
Usual Interstitial Pneumonia of Liebow
Utilities Infrastructure Plan
We know some IP addresses cannot be shared by one person. These are the ones that would require a person to move faster than possible. If we have one IP address in New York, then one in Tokyo 60 minutes later, we know it can't be the same person because you can't get from New York to Tokyo in one hour.
If my company had computers in New York and Tokyo, I could ssh between them in much less than 60 minutes. . .
iMediaConnection starts a huge field test of tracking unique slashdot readers with their cutting edge technologies.
Surely not urinating in public, although that would be important to track, too.
--
Atlantis, a runaway hit, ball matching game for mac: http://www.funpause.com/
So you can use probabilistic means to identify unique visitors. That's not a paradigm shift, except for those whose paradigms are already very small.
Somehow I don't think this research is worthy of an NDA.
I'm not sure what the Flash is, but to me, scanning all the cookies your computer has had IS privacy intrusive.
What's so new about this? How is this news? Very little substance to the article, plus I've been using IPs, Cookies and Logins to track people for a long time.
No single test is perfectly reliable, so we have to apply multiple tests.
No kidding. This guy probably needs a wake up call.
We know some IP addresses cannot be shared by one person. These are the ones that would require a person to move faster than possible. If we have one IP address in New York, then one in Tokyo 60 minutes later, we know it can't be the same person because you can't get from New York to Tokyo in one hour.
Ok, so this is what normally is called a really stupid argumentation. I don't say that it can't be accounted for, but stating such a thing is nothing more than plain stupidity. Has this guy ever heard about that Internet thing ?
Flash can report to the system all the cookies a machine has held.
Uhmm, not a great argument to make people use it.
No one wants to know.
I don't think they don't want to know. They just don't want to see a sudden drop of ~50% of their user count from a day to the other. And it really doesn't matter if it's the truth or not. A drop is a drop.
I am putting myself to the fullest possible use, which is all I can think that any conscious entity can ever hope to do.
What's more interesting, the new technology doesn't seem to be privacy intrusive
The only mention of the word "privacy" on the linked web page is the term "Privacy Policy" at the bottom of the page.
John.
Unique IP
It would be great if submitters would add a sentence or two explaining the key acronym(s) in their article instead of assuming that everybody already knows about their pet interest. RTFA to find out? Why should I waste my time reading an article that might not be of any interest to me?
From the article:
" We know some IP addresses cannot be shared by one person. These are the ones that would require a person to move faster than possible. If we have one IP address in New York, then one in Tokyo 60 minutes later, we know it can't be the same person because you can't get from New York to Tokyo in one hour."
Everheard of ssh and similar tools to make that travel?
And they put this on slashdot. Ignorance, just pure ignorance...
Evolution of Language Through The Ages: 6000 BC : ungh, grrf, booga 2000 AD : grep, awk, sed
They make some silly assumptions that I don't think work with users using proxy agents, but in the end it still boils down to the existence of cookies. Which would be ok, if the problem they are trying to solve wasn't that users are deleting and not storing cookies at all. They do mention using Flash to store cookies, which I suspect will have to be the next area users will have to start cleaning up. But just because cookies don't overlap in time and the IP address is the same doesn't mean it's the same person. A bunch of users that use the same browser and share an IP address that always delete their cookies with this system will look like one user. Vastly under counting. Which I don't think web sites are interested in. Vast over counting is profitable. Under counting, not so much.
In the end there is no way they can even mostly recognize repeat web site visitors if the VISITOR DOESN'T WANT THEM TO.
The big problem is stated at the top of the article:
"We need to identify unique users on the web. It's fundamental. We need to know how many people visit, what they read, for how long, how often they return, and at what frequency. These are the 'atoms' of our metrics. Without this knowledge we really can't do much."
If knowing who unique users are is that important they need to create a reason for the user to correctly identify themselves. Some form of incentive that makes it worth giving up an identification for.
The article's "Sky is Falling" tone rests on a single factoid. "30 to 55% of users delete cookies" therefore current analytics products are out by "at least 30 percent, maybe more".
That is of course complete nonsense. Let's say we accept the author's assertion that different studies have given cookie deletion rates across that range. I can accept that a significant number of users might delete cookies at some point, but what percentage of normal, non-geek, non-tinfoil-hat-wearing users are deleting cookies between page requests to a single site in a single session? If it is 30%, then I will eat my hat.
Most cookie deletion amoung the general populace will be being done automatically by anti-spyware software and is not done in realtime.
The author clearly knows that even the most primitive of tools also use other metrics to group page requests into sessions, so even if 30% of users were deleting cookies, it would not result in a 30% inaccuracy.
Of course "researchers propose more complex heuristic that looks to be slightly more accurate than current pracice" does not make as good a story as "paradigm shift" blah blah "blows out of the water" blah blah "We've been off by at least 30 percent, maybe more." blah blah.
I develop web analytic software for a living.
There's only so much you can do to track users.
IP address, user agent, some javascript stuff for cookieless tracking.. the only real "unique" identifiers for any one visitor. It stops there.
Of course, using exploits in flash doesn't count, but supposedly this new method is "not intrusive."
I call BS because it simply can't happen.
If a user doesn't wanna be tracked, they won't be tracked. This story is just press, free advertisement, and hype for this particular company.
We have secretly replaced these Slashdot mods' sense of humor with a rusty nail. Let's see if they notice!!
Analyse this, bitch!
*slashdots his server*
Why do I have this feeling like this "cutting edge technology" involves the entrails of an animal and some form of divination?
"Trademarks are the heraldry of the new feudalism."
I wonder what they will think when they start getting impossible bit patterns, like 7734 and 6027734 and 5773857734?
I wonder if they'll notice?
Hexadecimal would probably put the joke way past them.
After some thought, I'd probably agree that step 4 is valid for the vast majority of web users.
The only way this might break is if a large number of people are sitting behind a proxy/cache. But if it is the case they have fallbacks.
The problem with cookie deletion is not that it happens, but that we've been relying on a single method for identifying people.
I'm so happy that we have other ways of tracking people. I mean, whenever I clear out my cookies, I'm thinking to myself, "But now how will the Man track my online activities?" Now I can clear out cookies and once again feel safe with the knowledge that somewhere, somebody knows everything I do online.
Not to mention a security flaw.
When you visit my site, you agree to download and run a Flash/ActiveX control that downloads all your cookies to slashdot.org, and then sends them to me, so that I can now present false credentials to slashdot.org to make it think that I have auto-login privledges.
Awesome design flaw there, but I highly doubt anyone is THAT stupid to put THAT big of a security flaw into a system.
I am unamerican, and proud of it!
"I'm not sure what the Flash is"
In this case I think the "Flash" being referred to is Macromedia's Flash plugin. He's not very clear though is he?
AM
When I read "paradigm shift" in the very first paragraph, my bullshit sensor sound such a loud alarm that it's hard to continue reading...
WTNT31 KNHC 230536 ...CENTER OF JOSE MAKES LANDFALL ON THE COAST OF MEXICO...WAS
TCPAT1
BULLETIN
TROPICAL STORM JOSE INTERMEDIATE ADVISORY NUMBER 4A
NWS TPC/NATIONAL HURRICANE CENTER MIAMI FL
1 AM CDT TUE AUG 23 2005
GETTING BETTER ORGANIZED AT LANDFALL...
A TROPICAL STORM WARNING REMAINS IN EFFECT FOR THE GULF COAST OF
MEXICO FROM VERACRUZ NORTHWARD TO CABO ROJO. THIS WARNING WILL
LIKELY BE DISCONTINUED LATER TODAY.
FOR STORM INFORMATION SPECIFIC TO YOUR AREA...INCLUDING POSSIBLE
INLAND WATCHES AND WARNINGS...PLEASE MONITOR PRODUCTS ISSUED
BY YOUR LOCAL WEATHER OFFICE.
DATA FROM THE MEXICAN RADAR AT ALVARADO INDICATE THAT THE CENTER OF
JOSE HAS MADE LANDFALL ON THE EASTERN COAST OF MEXICO.
AT 1 AM CDT...0600Z...THE CENTER OF TROPICAL STORM JOSE WAS LOCATED
NEAR LATITUDE 19.8 NORTH... LONGITUDE 96.8 WEST OR ABOUT 60
MILES... 95 KM... NORTHWEST OF VERACRUZ MEXICO AND ABOUT 90 MILES...
145 KM...SOUTH-SOUTHEAST OF TUXPAN MEXICO.
JOSE IS MOVING TOWARD THE WEST NEAR 9 MPH... 14 KM/HR... AND THIS
GENERAL MOTION IS EXPECTED TO CONTINUE DURING THE NEXT 24 HOURS.
ON THIS TRACK... THE CENTER OF JOSE SHOULD MOVE FARTHER INLAND INTO
THE MOUNTAINS OF EASTERN MEXICO TODAY.
MAXIMUM SUSTAINED WINDS ARE NEAR 50 MPH... 85 KM/HR...WITH HIGHER
GUSTS. JOSE SHOULD WEAKEN AS THE CENTER MOVES FARTHER INLAND. THE
ALVARADO RADAR INDICATED THAT JOSE WAS BECOMING BETTER ORGANIZED IN
THE LAST FEW HOURS BEFORE LANDFALL...AND THE MAXIMUM SUSTAINED
WINDS AT LANDFALL MAY HAVE BEEN HIGHER THAN 50 MPH.
TROPICAL STORM FORCE WINDS EXTEND OUTWARD UP TO 45 MILES... 75 KM
FROM THE CENTER.
ESTIMATED MINIMUM CENTRAL PRESSURE IS 1001 MB...29.56 INCHES.
RAINFALL ACCUMULATIONS OF 3 TO 5 INCHES...WITH ISOLATED HIGHER
AMOUNTS OF UP TO 10 INCHES OVER THE HIGHER ELEVATIONS...CAN BE
EXPECTED IN ASSOCIATION WITH JOSE. THESE RAINS COULD CAUSE
LIFE-THREATENING FLASH FLOODS AND MUD SLIDES.
REPEATING THE 1 AM CDT POSITION...19.8 N... 96.8 W. MOVEMENT
TOWARD...WEST NEAR 9 MPH. MAXIMUM SUSTAINED
WINDS... 50 MPH. MINIMUM CENTRAL PRESSURE...1001 MB.
THE NEXT ADVISORY WILL BE ISSUED BY THE NATIONAL
HURRICANE CENTER AT 4 AM CDT.
FORECASTER BEVEN
SCNR...
and are already tired explaining customers, why the unique visitors differ from ther built-in log-file analysis.
See CheckEffect for details.
I don't WANT to be tracked you insensitive cloud!
The article uses a lot of time to establish that this is a paradigm shift, when it's actually not. I do believe their idea is good, but basically it's just applying a lot of "possible" user identifiers and merge them together to form a unified result.
Some of the identifiers they haven't used are linkage on the site. If one page links to another, it might be the same user, if the pages are called in sequence.
On top of links "time" might be applied. Some links are expected to be clicked fast, others after some reading on the page.
Some may argue that linkage is what you want to determine in the following analysis, and can't therefore be used to determine the use in advance, but this is not true. The determination of the user uniqueness looks to see if its possible for the user to get from one page to an other, while the analysis want to determine if they did it.
-:) Oh no - not again.
www.rednebula.com
I mean, seriously folks-- there is a reason why these things are mocked.
UTF-8: There and Back Again
ROI is mentioned, along with the 'atoms' of their metrics: page hit count, popular URL count, URL dwell time, and returning visitors. When these metrics are used to produce reports, how valuable are these reports in ascertaining how ROI is affected by said metrics? For example, getting a neat funnel report of the path people take through a site and where the traffic drops off offers insight into popular paths and locations where people bail out, but apart from listening for errors, there is no further insight into why a person bailed.
What seems to be missing is gathering insightful information into what transpires while someone is on a particular page. I'd like to know the general trends in behavior, not just the server requests. I've found it more useful to be able to see the interactions with the content than reporting where people enter, traverse, and exit a site.
Macromedia Flash has a local shared object (LSO), which is similar to a cookie, but less known.
I presume the proposed tactic is to set a cookie with an id and add the same id to a LSO. That lets you see what happens to your cookies over time, as long as the LSO doesn't get deleted.
Since there is no LSO management tool by default, LSO's have better lifetimes than cookies.
There is a firefox extension howerer, that lets you view and delete those LSO's.
I expect this functionality will eventualy become more widespread, giving LSO's the same kind of reliability as cookies.
"If the same cookie is present on multiple visits, it's the same person. We next sort our visits by cookie ID"
Only after that they seem to continue the analys ("We know some IP addresses cannot be shared by one person. These are the ones that would require a person to move faster than possible", etc)
Thus turning off or regulary removing cookies will render their bleeding cutting edge technology useless? And how are cookies a 'breakthrought'?. Their only alternative to this seems to be;
You can also throw Flash Shared Objects (FSO) into the mix. FSOs can't replace cookies, but if someone does support FSO you can use FSOs to record cookie IDs.
I don't know what the fuzz is about
This is just basic logic, which any decent programmer should be able to come up with, even the M$ certified ones.
I think we can keep recursing like this until someone returns 1
For those who can't be bothered to read through all the buzzwords, here's the actual method used:
Each of these steps is applied in order:
1. If the same cookie is present on multiple visits, its the same person.
2. We next sort our visits by cookie ID and look at the cookie life spans. Different cookies that overlap in time are different users. In other words, one person cant have two cookies at the same time.
3. This leaves us with sets of cookie IDs that could belong to the same person because they occur at different times, so we now look at IP addresses.
4. We know some IP addresses cannot be shared by one person. These are the ones that would require a person to move faster than possible. If we have one IP address in New York, then one in Tokyo 60 minutes later, we know it cant be the same person because you cant get from New York to Tokyo in one hour.
5. This leaves us with those IP addresses that cant be eliminated on the basis of geography. We now switch emphasis. Instead of looking for proof of difference, we now look for combinations which indicate its the same person. These are IP addresses we know to be owned by the same ISP or company.
6. We can refine this test by going back over the IP address/Cookie combination. We can look at all the IP addresses that a cookie had. Do we see one of those addresses used on a new cookie? Do both cookies have the same User Agent? If we get the same pool of IP addresses showing up on multiple cookies over time, with the same User Agent, this probably indicates the same person.
7. You can also throw Flash Shared Objects (FSO) into the mix. FSOs cant replace cookies, but if someone does support FSO you can use FSOs to record cookie IDs. This way Flash can report to the system all the cookies a machine has held. In addition to identifying users, you can use this information to understand the cookie behavior of your flash users and extrapolate to the rest of your visitor population.
Please correct me if I got my facts wrong.
About 20% of my time on my last job was spent doing web analysis. It drove me insane.
The problem is with the word "accurate". To management, "accurate statistics" means knowing exactly how many conscious human beings looked at the site during a given period. However, the computer cannot measure this. What it can measure, accurately, is the number of HTML requests during a given period.
You can use the latter number to estimate the former number. But because this estimate is effected by a multitude of factors like spiders, proxies, bugs, etc., management will say "these stats are clearly not accurate!". You can try to filter out the various "undesirable" requests, but the results you'll get will vary chaotically with the filters you use. The closer you get to "accurate" stats from the point of view of management, the further you'll be from "accurate" stats from a technical point of view.
Makers of web analysis software and services address these problems by the simple of technique of "lying". In fact, a whole industry has built up based on the shared delusion that we can accurately measure distinct users.
Which is where this article comes in. The author has discovered the shocking, shocking fact that the standard means of measuring distinct users are total bollocks. He's discovered that another technique produces dramatically different results. He's shocked, shocked, appalled in fact, that the makers of web analysis software are not interested in this new, highly computationally-intensive technique that spits out lower numbers.
My advice? Instead of doing costly probability analysis on your log files, just multiple your existing user counts by 0.7. The results will be just as meaningful and you can go home earlier.
fish and pipes
``The author claims that he is describing 'new, cutting edge methodologies for identifying people, methodologies that -- at this point -- no web analytics product supports.''
And when you read down to how these "new, cutting edge methodologies" actually work, it comes down to: plant cookies, if that doesn't tell you what you need to know, look at the IP address. Then take into account that different cookies and different IP addresses can still be the same user, if they occur at different times.
It's clever, but it didn't take a genious to think that up. Why is nobody doing it? Well, because it's too much work and still doesn't give you any guarantees that your conclusions are correct.
It's nice how TFA is presenting this as the best thing after sliced bread and Longhorn, though.
Please correct me if I got my facts wrong.
"We need to know how many people visit, what they read, for how long,"
Even before tabbed browsing I would often have well over 10 browser windows open, if they measure what I have open for the time the window is open then they will get VERY scewed results.
Also I have two monitors so windows in the foreground do not always translate to windows I am focusing on. (also high resolution monitors could produce the same effect)
I wish them luck becuase they need it.
ERR 411[Max number of witty sigs reached]
Somebody please explain to me: why would you go to all this trouble to get a close estimate of how many unique visitors your site draws?
I'm personally always more interested in how many pages get requested, and which ones. The first gives me an impression of how popular the site is*, the second tells me which pages people particularly like, so I can add more like that.
The only reason I see for really wanting to track people is if your site is actually an app that has state. In those cases, you have to use a more bullet-proof system than the one presented in TFA.
* Some people object that it counts many people who visit once, then never again; but I consider it a success that they got there in the first place - they were probably referred by someone, followed a link that someone made, or the page ranks high in a search engine.
Please correct me if I got my facts wrong.
The article points to Magdalena Urbanska and Thomas Urbanski's original research paper which "reveal" its valididy through a "mathematical proof". (8 pages of formulas, so it must be true) Of course, anyone with a post 1990's knowledge of the thing called "internet" would know that mentioning the existence of a research paper has been replaced by the thing called a "hyper link".
Googling, I found little more than this link to ARF, an unknown organization boldly calling itself "The Research Authority" (with capitals, mind you).
No paper (at least not online) and no references to the institute or organization that Magdalena Urbanska and Thomas Urbanski might be affiliated with. Their daily lifes seem to be spent as "researchers" - whatever that may mean.
It looks like a hoax. A sad hoax. Because why would anyone want to hoax a story this sad?
Wow! THAT's a paradigm shift if I've ever seen one!
Macromedia have a page that allows you to modify what sites can do on your computer in regards to Flash:n /flashplayer/help/settings_manager02.html#118539
http://www.macromedia.com/support/documentation/e
One single method that would reliably allow a site to track its users would be that each user needs to log in, and then needs the "session cookie" on each page they visit, and if they delete it, hard luck, log in again. This method is just a step away from another one: Make the pages password-protected and give the password to nobody. Users tracked: 0. Pages visit: 0. Tracking reliablity: 100%.
Anagram("United States of America") == "Dine out, taste a Mac, fries"
"I highly doubt anyone is THAT stupid to put THAT big of a security flaw into a system."
Read the article, and the guy is proposing to build exactly that kind of a security flaw into the system.
Flash can use, basically, some local shared storage on your hard drive. This isn't really designed as cookie storage, and doesn't have even the meager safeguards that cookies have. (E.g., being tied only to a domain.) It's really a space that _any_ flash applet can read and write, and currently noone (with half a clue) puts any important data there.
This guy's idea? Basically, "I know, let's store cookies there, precisely _because_ any other flash applet, e.g., our own again from a different page, can read that back again."
Caveat: so can everyone else. I could make a simple flash game that grabs everything stored there, just as you described, and sends it back to me. Including, yes, your session id (so, yes, I can take over your session in any site you were logged in, including any e-commerce sites or your bank) and anything else they stored there.
Since it's used to track your movements through sites, depending how clueless that's programmed, I may (or may not) also be able gather all sorts of other information about you.
So in a nutshell his miracle solution is to build _exactly_ that kind of a vulnerability (not to mention privacy leak) into the system.
So, well, that's the problem with assuming that "noone could be THAT stupid". Invariably when I say that, someone kindly offers himself as living proof that I'm wrong. Soneone CAN be that stupid.
A polar bear is a cartesian bear after a coordinate transform.
right here from our "friends" at Omniture
so a visiting a page that you can adjust your privacy settings will actually compromise your privacy,
now you can see why the privacy GUI is on Macromedia's site and not built into the player, but thats not suprising
seems Flash is slowly becoming spyware, shame
This is not really "New Technology" or a "Paradigm Shift", or anything extraordinary. This is just another marketeer trying to start a "buzz".
I know plenty of software out there that perform multiple tests in order to establish uniqueness of visitors. Perhaps the current big-biz log-analyzing apps do not do it, but that doesn't mean nothing else does. There was a time when Real Programmers didn't trust cookies as the exclusive identifier. I even remember some popular Log Analyzer Perl script that used to check for the following:
- First, Cookies
- Then IP Address (Whether it is known to be dynamic or static)
- Then compares the IP Addresses by IP pool (ISP)
- Then checks the time between requests, so that requests of different IPs from the same IP pool, with the same User Agent come in within a pre-determined time, they are considered the same visitor.
- Also checks the time between requests from the same IP address, so that if a certain pre-determined time has passed between requests, and the IP address is known to be dynamic, and the User Agent changed, then it is probably someone else.
I do not recall the exact details of the analysis, but it was something along the lines of the above. And there were many scripts like that one.
Comparing IP addresses geographically and in a time-sensitive manner (coupled with other potentially identifying criteria, such as Cookies, User Agent, Screen resolution, etc.) has been known and done for years! In particular, these forms of unique visitor analysis was very popular during the days when you couldn't count on Cookies being supported by all browsers, or on savvy users accepting them -- you know, when dinosaurs roam the data center; way before everybody decided to rely on Cookies as the end-all-be-all of session identification.
If they all forgot that using Cookies exclusively was never a a very reliable solution, then that's their problem.
-dZ.
Carol vs. Ghost
Wow, I knew that the Flash settings UI was badly designed, confusing and annoying to use. I didn't know that it set up tracking with partner sites as well.
Besides, what steps does Flash take to ensure that any old web site can't just reset these permissions, or except itself to the 'no local storage' policy you set?
Don't bother visiting Macromedia's site at all:
find / -name libflashplayer.so -o -name libflashplayer.xpt -exec rm {} \;
If you really can't live without it, try this instead:
chmod --recursive 500 ~/.macromedia
That'll screw em up, random user agents.
Liberty freedom are no1, not dicks in suits.
10+5/2
Seriously, whats important REALLY is not the current statitic total for NOW, or TODAY,
its.... yes... TRENDS!!!
that PAGE X is increasing by 6% weekly.
or that page y is dropping in interest.
Its just like TV ratings, everyone knows its all CRAP and nonsense, except the DELTAS, the changes
if TV show X is going up 30% week, you know its HOT.
Think of it like qantum physics, you dont really know the location of the electron, just its DIRECTION. TIME CANNOT STAND STILL.
DIRECTION of MOTION is what you want which is WHAT PREDICTS the future.
What good is YESTERDAYS news, we need to know TOMMOROWS NEWS.
Liberty freedom are no1, not dicks in suits.
While web usage stats may indeed be inaccurate, it is so across the board. This means, everything that relies on it has the same amount of inaccuracy... Which in turn makes it, accurate in the market place.
For instance, considering everything else to be equal, an ad buyer wanting to pay $1 for one thousand unique eyeballs won't care whether it's spent at site A or site B, as long as they are using equivalent methods to measure traffic.
Another example. Say Google puts out a press release saying they have X% of web traffic. MSN comes out with Y%. Sure they both may be off, but still the ratio of the two will still be about the same.
It's like playing chess without the Bishop, except it doesn't just apply to one player, but to every player. There's no advantage for anybody and the net result is probably the same.
eTrade SUCKS
YOW! Are you ZIPPY the PINHEAD?
http://flashblock.mozdev.org/
Get it because it'll make you cool like everyone else (Go Go Gadget Peer Pressure!), keep it because you don't miss the ads and just one click brings up any content you do want, as well as whitelist features.
I8-D
If I request page A, then request page B and then go back to page A and grab it with a conditional request (and the server returns 302 not modified) wouldn't this obviously indicate I had been to page A before fairly recently? (assuming you have set cache headers such as to only allow private non-shared ISP proxies to store them)
What about people following a link with a referer from page A to page C when they haven't (according to your logs) been on page A? Doesn't this likely indicate page A has been cached/saved or is otherwise still open in another tab or window?
I suspect there is some decent post-processing of HTTP behaviour that could be done on old logs that hasn't been considered or implemented yet.
If you really want some useful analysis of your website then use some javascript to measure how long it takes users to actively fill out forms, or how quickly they navigate from page to page (one time use information)...you know to maybe actually improve your navigation and make it better for the user?
Why do people overestimate the importance of knowing exactly how many people are using your site and identifying them? That's a pretty useless practice. When and if the user wants to assert themselves they can register/login (if such a thing is applicable to your site).
"We know some IP addresses cannot be shared by one person. These are the ones that would require a person to move faster than possible. If we have one IP address in New York, then one in Tokyo 60 minutes later, we know it can't be the same person because you can't get from New York to Tokyo in one hour." - Surely this is flawed if the user is utilizing an Onion router (e.g. http://tor.eff.org/)?
The author said that if a cookie with an IP address in New York and one in Tokyo an hour later means it is a different person. I know that I have logged in to remote sites with a VPN connection and continue to browse the web. Tracking geography of an IP is not an ideal way to track individual users.
http://tor.eff.org/
First, a bit of background about the concept of a paradigm shift.
What Dainow is describing is no more a paradigm shift than the first person who put a spare tire in their car. Redundant usage of preexisting systems to increase reliability does not equal a paradigm shift.
That said, he seems to be on the right track to achieving his goals. (Or rather, the researchers seem to be on the right track.) They seem to be naively optimistic. User metrics is obviously in its infancy if this is the cutting edge. Foiling schemes like this would require a minimum of effort:
What we need to do is create a browser plugin that actively manages cookies by sharing them. By actively, I mean not just accepting/denying based on a set of rules, but coupling that method with active uploading/downloading of cookies from an alternate source. (Deletion is not the solution!) Maintain a pool of known user-metrics cookies that users can update from rather than from the intended originator. (I'm guessing that P2P distribution would turn out to be ideal for this.) How would the user-metrics companies deal with millions of computers surfing with identical cookies?... I'm betting they honestly might abandon cookie use.
I can't see that writing a plugin that does this would be very hard. Any takers? (I would if I weren't so damn busy lately.)
argumentum ad fallacium: Fallacy of defining a fallacy which allows one to dismiss the argument in question.
You seem to assume that they want to improve their site. In which case, yes, anonymous trends and anonymous user movement grouped by session id suffices. (E.g., to see if users give up and leave your site half-way through the marketting bullshit pages, before even reaching the product pages.)
But that's not the problem.
Whenever you see someone going on about how the _need_ to track and identify each user, and they _need_ accurate numbers and even personal details... that's your clue that it's purely an ad money problem. They couldn't care less about the site design or trends as such, they just need some bullshit in numbers to shaft the ad providers with, or viceversa for the ad providers to shaft you with.
The internet started as a pretty clean and ad-friendly place, but it quickly went downhill. The initial ad rates were basically for sites with 1 ad on the main page, that a lot of people actually looked at. But from there it went downhill with site operators trying to shaft the ad providers. (E.g., by trying to collect those rates per ad... for pages that had wall-to-wall ads on each page, scripts that clicked on an ad 1000 times a second, etc.) The ad providers in turn went on to not only try to defend themselves from this kind of fraud, but also to try to commit the exact kind of fraud on the companies paying to advertise.
Welcome to the War Of The Bullshit Metrics. Because that's basically what it is.
All this ranting and raving about unique trackable ids and such, is just the search of the perfect metric so an ad provider can (A) say to the site operator carrying the ads "nah, we're not paying you that much, because while you had users clicking, it wasn't all UNIQUE users", while at the same time (B) telling some clueless company advertising with you "our marketting campain was a huge success because X thousand unique users saw it, and Y thousand clicked on it, and <insert other bogus metrics used often just as a Chewbacca defense>, so you owe us a big wad of money."
The problem with these bullshit metrics is that not only they mis-represent, but often lead to counter-productive campaigns aimed at inflating the metric even if they cause _less_ sales. E.g., once you define the "click" as a metric of success, as opposed to a "sale", you get bullshit fake-UI ads, punch-the-monkey ads, and outright redirects that simulate a click. That's all stuff that isn't aimed at actually raising awareness/interest in a product, but at gaming a sick irrelevant metric.
And the problem is that bogus metrics be damned, the companies paying for it aren't seeing results out of it. They just see some metrics by which the ad campaign should have been a huge success, except the expected blip in sale tends to be missing completely.
The ad provider's solution? Trying to make that bullshit sound more credible. All this talk about uniquely identifying users and accurately counting everything, is mostly trying to make something sound like a science, when it's just bullshit. They want to present something like they have some solid scientific proof that for your X thousand dollars, you're accurately and guaranteed getting Y thousand whatever, within Z% accuracy.
And it's partially "Chewbacca defense". They deflect the attention from whether their service actually works, to how accurately they measured some bullshit irrelevant metric. And then dazzle your with some equally irrelevant pseudo-science bullshit as to why and how their measurements are so accurate. But again, completely avoiding the question of what it will do for _your_ _product_.
And the same kind of a war of the bullshit metrics sometimes also goes on inside a company. When marketting departments need to justify their budgets, what do you think they reach for? I'll tell you what. The exact same kind of bullshit metrics that show how accurately and scientiffically they got you X thousand hits. and Y thousand unique tracked users, and whatever.
So the next time you see such a "the sky is falling if we
A polar bear is a cartesian bear after a coordinate transform.
So how do you determine if your methodology is accurate? The fact that preliminary tests give you different answers than traditional methods doesn't really tell us anything. It just informs us that two different methods present two different results.
Read the EFF's Fair Use FAQ
A website can only work out how long you are viewing their page for by tracking the time between requests. If you just have a single page open for 18 hours it will be indistinguishable from having a single page open for 0.3 seconds.
I was debunking the poor logic, inappropriate assumptions and overall lack of fundamental understanding held by these researchers. After debunking the first four points, I changed gears. I'm tired of all these marketing bullshit artists trying to track my every page view and metric on what I do at their site. I'm especially tired of having to manage cookies and delete them on a regular basis. Sure each site only sends 1-10 cookies of a few bytes each, but that starts to add up when you don't stick to your main sites.
I think we need an open source project that will collaboratively "surf" web sites. The collaboration will mean that site cookies will be tossed in to a publicly accessible pool and shared amongst the workers. The pool might contain thousands of cookies for each site. Workers will get a site and a cookie from the main server and start surfing. The same cookie will be given out to many workers at the same time. My worker might hit NYT now and a machine in New Zealand might hit that same site with the same cookie just a few minutes later. The actual time delay is irrelevant, it's the sharing of the cookies and "co-ordinated attack" that's the key here.
The workers will of course present random, but valid client IDs to the sites. Some logic also needs to detect that cookie has been used too often and should be discarded. On command a client should access a site with no cookie and instead retrieve a new one. The rate and level of cookie sharing will vary so as to generate noise in any of the standard web metrics algorithms in use.
I don't think the workers will even need to accept the entire page or images from the page. Log-files from web servers only write information about requests, not about completions. What that suggests is that bandwidth usage will be minimal.
Maybe if we can generate enough noise these morons will stop trying to come up with more useless ways to invade our privacy and track our every on-line move. Once the advertisers start seeing their pay-outs go through the roof they may ask questions about what's going on.
I'm no privacy nut, but I wouldn't stand for being tracked as I walk around all day; I've no great desire to accept it as I "walk" around the 'net either.
Article X: The powers not delegated... by the Constitution...are reserved...to the people
Otherwise your production app would have used some kind of persistency and your job would have been a lot nicer. Thats what I would have explained to them anyway. Stupid as an excuse can only really go so far and even managers in such technical fields should be able to get that.
Quack, quack.
This guy misses one more way to identify users: make your web site using SSL and use SSL Session ID for identifications. Most web servers (for example tomcat and apache) can do this.
Many times what is as important isn't the existance of a piece of information, but the lack of a piece of information. If a particular object is referenced and that object is flagged to be cacheable at a browser (cache-control: private), and the reference wasn't an if-modified since request, then you could consider it a new visit. If however, a user references the page the object is imbedded in, but the object itself isn't referenced, then it is cached, and could be considered a return visit. This would work about as effectively as anything this report was talking about, and is something that as an individual method won't require a huge amount of processing to do the math on.
Golden Question:
Is this Method Patented?
I'd bet that it is, or will be soon, and this is just an ad to get someone to license their 'Method', or worse, get someone to implement it without knowing it's patented, so the lawyers can do their magic.
Abolish Copyright. Restore Freedom.
If you want to measure the success of your web site, look at the net income it generates. If you want to identify problem areas, use the available data intelligently, with full understanding of its limitations, and perform a well reasoned statistical analysis of that data.
The only thing gained by uniquely identifying users outside of financial transactions is the opportunity to violate their privacy.
I defy the "new" methodology to uniquely identify me on jrandomwebsite.com -- I block cookies until I know they're essential to support my own selfish desires, I block HTTP_REFERER, and I use onion routing. I choose not to participate in any corporate greed in which I do not have an interest. Count this, sucker.
Warning: This signature may offend some viewers.
I wonder what they will think when they start getting impossible bit patterns, like 7734 and 6027734 and 5773857734?
If a site requires Trusted HTTP Extensions, you will get a 403 error instead of a page. If your ISP requires Trusted Network Connect, you will get a DHCP failure instead of an IP address. Alsee predicts that dystopia will arrive by 2015 unless we drum up a significant backlash among residential Internet users.
The majority of users behind "web accelerator" proxies do not know how to delete cookies. Or if you're really anal you could impose "free reg. req." on all users behind such proxies.