Slashdot Mirror
Ask Slashdot: Where Can I Find "Nuts and Bolts" Info On Cookies & Tracking Mechanisms?
New submitter tanstaaf1 writes: I was thinking about the whole tracking and privacy train-wreck and I'm wondering why specific information on how it is done, and how it can be micromanaged or undone by a decent programmer (at least), isn't vastly more accessible? By searching, I can only find information on how to erase cookies using the browser. Browser level (black box) solutions aren't anywhere near good enough; if it were, the exploits would be few and far between instead everywhere everyday. Read below for the rest of tanstaaf1's question.
On Amazon, I haven't found a likely good book on the topic. There are books on protocols but I'm really only interested in how I can detect and track and block, and erase, and re-write and spoof all the tracking attempts on a case by case basis. Maybe a book on how to write my own tracker — or my own tracking blocker from scratch?
In theory it wouldn't seem to be that hard to uttlerly micromanage your own computer. Here's how I think it could be done:
(1) Have an explicit on/off switch, ideally OS based and trivial to control with a mouse-twitch, which turns internet access on and off as certainly as a mechanical light switch controls lights. Along with this, maybe the whole screen can change color, red-light green-light, to keep the user always aware of incoming or outgoing traffic. I should instant be able to get detailed information on any unexpected write or read request. Think unix "ps" or better. (Actually, a file system which allowed the owner to attached detailed memos and other information would be a nice touch...once litter builds up it quickly gets easy to hide real malware everywhere; that is a common technique used by embezzlers everywhere — create chaos and then hide your exploits within it).
(2) When the browser is started, make it start in a fresh virtual space / sand-box. Then copy into that space any "cookies" or other information I explicitly care to put into that space. I would, for example, put in site specific cookies to allow sites i whitelist to identify me. A good database of all the files in my virtual space, how they got there and what they are used for, would be really nice to see.
(3) As you browse you can block or not block ads and trackers; the add-ons already exist.
(4) When you decide to exit the browsing session, at least, the computer should save important cookies from sites you frequent for later restoration.
(5) The entire virtual space is then shredded and deleted.
This could all be done at a finer grain, I'm sure, but I wanted to lay out an overall strategy — and ask:
(1) What am I missing?
(2) Has this already been done and automated, say, under Linux? (I wouldn't expect Microsoft, Apple, or Google to facilitate this sort of security under their OS systems; foxes guarding the hen house and all that. However, even under Windows and OSX I can install virtualbox...)
(3) Why is it so hard to find the specifics of, step-by-step, how (not why or if) we are being conned and raped and what, specifically, can be done to stop it? Why are we screwing around with all these endless add-ons instead of striking at the root of the problem? Or have I not really identified the root?
I would appreciate any specific feedback on my scheme or, even better, a link or three.
In theory it wouldn't seem to be that hard to uttlerly micromanage your own computer. Here's how I think it could be done:
(1) Have an explicit on/off switch, ideally OS based and trivial to control with a mouse-twitch, which turns internet access on and off as certainly as a mechanical light switch controls lights. Along with this, maybe the whole screen can change color, red-light green-light, to keep the user always aware of incoming or outgoing traffic. I should instant be able to get detailed information on any unexpected write or read request. Think unix "ps" or better. (Actually, a file system which allowed the owner to attached detailed memos and other information would be a nice touch...once litter builds up it quickly gets easy to hide real malware everywhere; that is a common technique used by embezzlers everywhere — create chaos and then hide your exploits within it).
(2) When the browser is started, make it start in a fresh virtual space / sand-box. Then copy into that space any "cookies" or other information I explicitly care to put into that space. I would, for example, put in site specific cookies to allow sites i whitelist to identify me. A good database of all the files in my virtual space, how they got there and what they are used for, would be really nice to see.
(3) As you browse you can block or not block ads and trackers; the add-ons already exist.
(4) When you decide to exit the browsing session, at least, the computer should save important cookies from sites you frequent for later restoration.
(5) The entire virtual space is then shredded and deleted.
This could all be done at a finer grain, I'm sure, but I wanted to lay out an overall strategy — and ask:
(1) What am I missing?
(2) Has this already been done and automated, say, under Linux? (I wouldn't expect Microsoft, Apple, or Google to facilitate this sort of security under their OS systems; foxes guarding the hen house and all that. However, even under Windows and OSX I can install virtualbox...)
(3) Why is it so hard to find the specifics of, step-by-step, how (not why or if) we are being conned and raped and what, specifically, can be done to stop it? Why are we screwing around with all these endless add-ons instead of striking at the root of the problem? Or have I not really identified the root?
I would appreciate any specific feedback on my scheme or, even better, a link or three.
Now that Verizon has hooked up with AOL to share cookie data and personal information, it sure would be nice if the Verizon stealth cookies could be deletable.
Now that Verizon has hooked up with AOL to share cookie data and personal information, it sure would be nice if the Verizon stealth cookies could be deletable.
Just a quick question, can the browser insert its own Verizon stealth cookie into the request URL?
And if that can be done, can it be used to poison the data, or even crash the Verizon tracking system?
You should be able to find some pretty straightforward documentation on HTTP cookies, flash data storage, HTTP Local Storage, and browser fingerprinting (see https://panopticlick.eff.org/ ). The tracking services aren't doing anything fancy -- they're just sharing that identifier behind the scenes. When you visit website1 they assign an id to your browser (via a cookie, or whatever). When you visit website2, it loads a script from website1 that puts your id somewhere into the DOM that website2's scripts can read and website2 assigns that id to your browser as well. Website3 does the same, and so forth. Then, websites 1 through N share the browsing habits of your id amongst themselves and gain some insight into what your browser is doing.
Do you really need reason for beer? Wingman Brewers
Evercookie is how it is being done.
UBlock Origin is how to block a ton of stuff.
Both are open source so you can have a look at it.
But in case of Verizon, you're talking about tracking on a whole other level.
And since your MAC or IMEI device number are needed to allow you access on their network, there are no options to spoof that.
The best you can do is block what they throw at you as a result of that data.
Adding a VPN would make life for them pretty hard because no server side deep packet inspection can be applied.
You can be tracked and identified by a large number of ways. Its not just cookies, its anything you click on, its hidden variables, its the URL, applets, javascript, and even your IP address. Have you heard of a Firefox plugin called Ghostery? Look at all the things it blocks. That will give you more clues about how you are being tracked. Cookies are not in themselves bad. They were designed for developers to cache information so that they could remember what the user was doing when they clicked. Advertisers decided to use them for different purposes. Then agains, the web sites are partly to blame. They want to know what you were doing, what pages you liked, where you spend time. It lets them know what interests people. But the sites have found that by signing up for programs that track users across multiple sites, they can get a deeper understanding of their customer. So, they deploy tracking code/cookies/pictures so that the companies who track across multiple sites can get info to share with them. Its really complicated.
A good Samaritan did the heaving lifting... https://gist.github.com/atcuno...
An organization can store information in your browser that can uniquely identify you. Usually this is a session code. It can share this information with whoever they want.
If you are concern with privacy never type any uniquely identifying information into your browser. Since you don't know what can potentially be uniquely identifying then you should never use cookies. Given that makes the internet practically unusable then use a whitelist. Given that a whitelist is a pain then forget about privacy. Everyone knows your porn habbits anyway.
If you are that preoccupied by evil cookies, use Linux Tails like Edward Snowden recommends.
But really, you overestimate your value for those who allegedly "rape and con" you with cookies. They care about trends and patterns, not about you as a person, so browsing the web in a virtual space that you "shred" afterwards is more of a hobby than a necessity. Modern browsers are well-equipped to provide a decent level of privacy, there's no need to go thinfoil hat over this.
lucm, indeed.
The reason is because what you're describing is an enormous amount of hassle and people just don't want to deal with that much complication, unless they're wearing a tinfoil hat. And if they are they probably have already set up a system that resets itself every time it's rebooted. Just learn to stop worrying and love the cookies.
You can easily add your own X-UIDH header. It is likely that Verizon's proxy wouldn't add another if one were already present. It's also possible that the request would be sent on with two (or more) X-UIDH headers. Most programming is sloppy programming, so they probably didn't account for this correctly. It's extremely likely that random strings in the X-UIDH header would confuse the system.
As I mentioned, most programming is sloppy programming. People keep making the same mistakes. One common mistake is, what if that string that's supposed to be about 16 characters is instead 500,000 characters (500KB)? Or 2MB? Things might break. What if it contains null characters (ascii value 0)? A lot of things break when strings have embedded nulls. Strings that are used to query a database to get a user's information often break when single quotes and semicolons are present.
That said, it's also likely they use popular off-the-shelf, premade software for the proxy, and it's protected against the most obvious attacks. Their database query routines are probably written by their own programmers, and those programmers probably aren't security experts.
Obviously, trying to harm their systems could very well be unlawful, even criminal. "I just sent web requests" might be about as convincing to a judge as "I just waved my arms around (while holding a knife)".
I hope the bad guys don't mess with them too much.
Re "erase cookies using the browser" .SOL shared cross-browser tracking, flash and other deeper tracking options. .
Thats really all that can be done to average users by most ad brands legally as the settings and use allow that short or long term access by default. Beyond that and it gets to be equipment interference.
Lots of apps on different OS will find the super cookies, Local SharedObject
"Has this already been done and automated" Different browsers have add ons that can do that based on some level of settings.
The other option was the ISP level deal with brands to alter the users internet experience. Very hard to escape that one as it flows with the basic network.
The final option is the security services or police passing code to detect a user when a visit a site has to resolve the original ip or to classical track a browser of interest for a while.
Classically the option was for ISP backed cookies that only an ad brand could read (2008).
Later users are starting to understand more about Unique Identifier Header (UIDH) and terms like perma-cookie.
A provider using JavaScript to inject packets to show an ad. ie the provider starts altering or initiating data packets for branding, ads.
Other network systems used personalized marketing ie search terms, websites visits, time spent ie all data that a provider can log.
Re "Has this already been done and automated, say, under Linux?"
Search the Firefox add ons some listed are options like: Better Privacy, Self destructing cookies, Cookie time.
Re " Why is it so hard to find the specifics of" Its now been done at the server, isp, web 2.0 provider, social media site level.
Ads have followed the security services thinking, why be in the users machine, just become the network used for all connections.
re "Or have I not really identified the root?"
An average user is now buying generations of hardware and OS software, OS updates from an ad brand... using their ad brand search engine on their OS..
The internet in some countries will be provided by in totally by a social media company or via ad brand hardware. Collect it all.
Every packet in and out is then up for logging, over any browser. ie the classic ISP becomes the advertizing brand not just selling logs to third parties
The only easy solution might be a new virtual machine with a modern browser and OS on fast dedicated hardware every browsing session. ie a laptop or desktop computer just for the new VM to surf the internat.
Domestic spying is now "Benign Information Gathering"
As mentioned by others, cookies are only one mechanism.
For instance, if you install any wierd fonts on your system - that along with your IP makes your are quite easy to identify. Browsers allow javascipt to query the fonts installed on a system (that feature is really a privacy intrusion bug, IMHO)
You mean that their monetization rights can be trumped by your desire to retain privacy? Oh, wait....
---- Teach Peace. It's Cheaper Than War.
You COULD try attacking them (rather than not using their service) and then tell the judge that your attack was to protect your privacy. You could try that.
You could also try attacking them using a knife to their sysadmin and try telling the judge the same thing.
I wouldn't recommend either. I just use a different company.
Ghostery is an excellent add-on for Firefox.
Firefox hacks mentions a 20 cookie per client connection limit. That appears to be true today.
It's not a limit, that was a recommendation from the original Netscape Cookie Specification. It's up to the individual browser implementors what limits they place on cookie size, cookies/domain, total cookie counts and even max cookie age. I have seen some web sites in the Top 1000 Alexa site list, for example, spu out over 60 Set-Cookie headers in response to a simple GET / HTTP/1.1 request - and that's just for the source page, not even any of the linked resources.
If you're still browsing mainly HTTP pages (not HTTPS) from a desktop-based computer I highly recommend getting hold of Privoxy and configuring it to eat the Set-Cookie headers for you. I'm sure there are other tools out there that achieve a similar result, but cookies are only a very small part of the identity tracking problem.
Also need to change your MAC address each reload...
Layer 2 addresses don't pass routers, which operate at level 3, so your client media access control address is only visible if you don't have a router.
The outside global address of your modem is of course visible to your ISP. You could change that, but most cable ISPs either require that not change at all (without calling them ) or if it changes it takes an hour or two to reconnect. Since you even with a direct ethernet connection your MAC address is only visible to your ISP, the only reason to change it would be to hide your identity from your ISP. That's going to be a problem for billing. Your ISP knows who you are, there's basically no getting around that. So no point in messing with your ethernet address.
If you have PPoE service (such as some DSL), you could change your MAC, but since you're sending them a user name and password they still know who you are. So again, no point in that.
Many companies are tracking by MAC address. E.g., coffee shop Wi-Fi systems, retail analytics systems.
It is not surprising you didn't find any info on "nuts and bolts" cookies.
Because while nuts are popular ingredients for cookies, bolts are terrible.
So I suggest that you replace bolts with chocolate chips, these are much better and you are much more likely to find information about them.
No referers. No script. Just plain doc data. Problem solved.
If you have client-side logic on by default, somene will use it to track you. It's that simple.
Another approach would be fresh private tabs for every session and perhaps spoofing of plattform data.
I use Gostery and don't care to much about super-cookies. I use multiple browsers for multiple personas and tasks, which mitigates the problem a little more.
I don't use facebook and stuff like that, but I'm pretty deep in Googles camp, with my Android devices and my various Google Accounts. ... It's a trade-off.
I might try to cut lose entirely sometime in the future.
No script, no referers. No google. Problem solved.
We suffer more in our imagination than in reality. - Seneca
You are making a mountain of assumptions with absolutely nothing but cheeky speculation
"When life gives you lemons, don't make lemonade. Make life take the lemons back!" -- Cave Johnson
I have no idea why you were modded down. You're absolutely right.
The OP apparently learned just enough to form those ideas and questions, but decided not to continue reading and find out that (more or less) all his proposals are already done.
1. On/off switch for "internet access". There's bunches of ways to do this. Many laptops come with a hardware switch to turn off wifi... that can do the trick. Just about any firewall software could do it, and most have a "panic" mode (including the very naive /etc/rc.d/init.d/iptables). You can up/down the network interface quite easily. All those fill his need here, but I suspect he just has no idea what he even wants - he probably doesn't want an internet access on/off switch, but one JUST for the browser, in which case, use a (local) proxy.
2. Start browser in fresh (virtual) space, but pre-populated with saved cookies. I'm ignoring the virtual/sandbox stuff, as it's unnecessary (but could be done via docker, a vm, bsd jails, chroot, etc). The browser can clear any and all data at the end of a session already, and can optionally not clear the cookies. There's also a cookies setting for "keep local data only until you quit your browser" allowing cookies to be created, but then those created during the session go away when you exit the browser. There are exceptions, third party blocking, and cookie managers.
2 - b. A good database of all the files in "my virtual space"... use your file manager. If you want to know what was newly created, use existing filesystem tools. You can even check the ~/.mozilla or ~/.config/google-chrome into git and diff it afterwards, or use etckeeper to maintain it, or a IDS like tripwire. Whatever level of detail you want.
4. When you decide to exit the browsing session, at least, the computer should save important cookies from sites you frequent for later restoration.
Already done. See #2. If you want a partial save (only those you consider important, but not other ones you don't want), then you'll need to become more intimately involved with your cookie management. Start with the cookie manager and figure out what you want. Then script something to maintain your cookie DB as you see fit -that isn't really as hard as it may sound. The cookie DB is often a flat text file, or an SQLite DB. Google Chrome's is SQLite (on linux, ~/.config/google-chrome/Default/Cookies)... you can use "sqlite3" and sql to manage it directly, or script something using your favorite language.
5. Shred the virtual space on exit.... if you really want this, then a short shell script can do it. Create loopback encrypted filesystem; mount; copy skel of browser directory into it; start browser using that profile; when it exits, copy out the cookies, then unmount and delete the file. I doubt that's really what is wanted though - have you thought about all the side effects? ...
3. Why is it so hard to find the specifics of, step-by-step, how (not why or if) we are being conned and raped...
WHOA! Hold up. This is not rape. You can watch every bit of data go back and forth, and you can control every bit of what you send or accept. Worst case (you don't trust the browser), use a local socks proxy and do your filtering there.
Except media queries are performed locally within the browser - not on the server. Even if the media query is specified in a link element in the head, the CSS file is still downloaded even if it doesn't fulfill the query requirement.
https://scottjehl.github.io/CS...
Popisms.com - Connecting pop culture
Not really. For example, I'm said their system will EITHER add an additional header or it won't. One or the other is true.
I suppose the "assumption" is that their code isn't fundamentally different from ALL of the tens of millions of lines of code I've reviewed over a decades-ling career. I "assume" that the programmers tried to get their job done. I've looked at the security posture of a LOT of software. In essentially all cases, no -application- code ever sufficiently anticipated all possible types of malformed input. Programmers (other than security analysts) try to write code that works. That's their job. It's what programmers do. They write a routine that takes certain input and produces certain output.
What programmers don't do is they don't write code that assumes everything is wrong, that inputs are completely fucked. They (rightfully?) say "garbage in, garbage out". That was fine in the pre-internet world, and it's fine if security isn't a concern. For internet-facing applications, if security is important, that has to change to "garbage in is the normal, expected thing. We plan for any and all types of garbage. Garbage in, nothing out because the software fails fast and fails hard." (In other words, raise an exception immediately, and do so by default, in any case other than getting precisely the input you want, all the input you want, and nothing else.) People just don't write software that way. They write software based on the input being what it's supposed to be.
They also write software in reasonable ways. For example, if there is a way that's quick, easy, and obvious, most programmers will do it that way. What that means is that programmers use the same patterns over and over, exposing the same types of vulnerabilities over and over. Very often, I can correctly list off security vulnerabilities without ever seeing the software, just based on the name. For example, any time I see something called download.php, I know they probably coded it the most clear and obvious way. The most clear and obvious way has three security issues. When I actually look at the source code for something called download.php, it almost always actually does have at least two of the three vulnerabilities that I would predict. Because the programmer is doing his job, writing a PHP script that causes a file to be downloaded. He's not doing my job - figuring out how such a script could be exploited.
That's true, for wifi, access points can see your MAC address. I was thinking of home / office internet, where your MAC is only visible to your ISP (who already knows who you are).
https://www.qubes-os.org/
What I have been waiting for quite a long time is the extension, or the iCab of sorts, that'll slightly upgrade most of the borwser fingerprint (screensize changed by some pixels, 1% of my fontlist hidden) every 1/4h or so. :(
I truly believe this is not so difficult to prepare, and once it'll be done most of this fingerprinting issue will be over.
The only trouble for the moment is, I'm not a programmer myself
Herve S.
all is in the title. Too bad I don't have points now...
Herve S.
It got picky. For one thing, you may need to restart the resolver daemon. Also, if you have IPv6 running, you may need to set an IPv6 address in /etc/hosts as well - even if you never use it.
As others mentioned in this thread, people are using it - it "works", it just got harder.