Google Makes It Harder For Marketers To Collect User Data

cagraham writes "In a seemingly minor update, Google announced that all Gmail images will now be cached on their own servers, before being displayed to users. This means that users won't have to click to download images in every email now — they'll just automatically be shown. For marketers, however, the change has serious implications. Because each user won't download the images from a third-party server, marketers won't be able to see open-rates, log IP addresses, or gather information on user location and browser type. Google says the changes are intended to enhance user privacy and security."

Comments

And google will retain that info exclusively.

[Spamalope]

While I applaud the move, it is about competitive advantage for Google.

Re:And google will retain that info exclusively.

[dotancohen]

While I applaud the move, it is about competitive advantage for Google.

Google already knew which emails you have or haven't read. So does every other email client, web-based or IMAP / POP3.

Re:And google will retain that info exclusively.

[Anonymous Coward]

Yep. And the security angle is overrated for two reasons:

1. NSA
2.Most mailing software generates unique images to track opens, so you're still being tracked. It's actually decreases privacy for Google to auto-download the images.

Re:And google will retain that info exclusively.

[jaseuk]

Yes and the point the summary misses, is that the images are used to verify that you have received and viewed the e-mail. This is far more important than browser types / locations etc.

It also prevents some evil things, such as first time you hit the page you get a drive by, the second time (with cookie set) you get the actual image and all seems fine.

Jason.

Re:And google will retain that info exclusively.

[pradeepsekar]

The article does not state of all images would be cached automatically even if you have not read your mail. It only says that images would be served through a Google proxy server, which caches the images.

So if Google proxies and caches the images when you open the mail, there is no protection added from marketers, except for the fact that Google can scan the images for exploits.

And if Google proxies and caches the images as soon as the service receives the mail, marketers can verify if the address is a valid gmail address or not by just sending mails and waiting for Google to cache the image. Expect more spam if this is the case.

There will be true protection from email tracking only if Google caches the images in all emails it receives, even if the email address is invalid - and that would increase the load on Google servers quite a bit.

Re:And google will retain that info exclusively.

[EvanED]

As Google *always* cache the image, the sender does not knows anymore when or even if the image was viewed and, so, doesn't knows anymore if the email was opened.

If they have specific knowledge about Gmail. Unfortunately, mailers that don't would make the more dangerous assumption (that you read the mail) under that behavior.

But anyway, even that's not true because under Gmail's new setup, the first download will still come when the user opens the mail and loads the images. At least, that's the best information I can find on this. I also saw a comment somewhere a couple of days ago by someone who claimed to have tested that behavior, and checked that the load of the image came when the mail was opened.

Re:And google will retain that info exclusively.

[HiThere]

But if they're doing reasonable de-duping, then only the first person to click on the image will register. Everyone else will hit the cache. To avoid this every email would need a separate link to the picture.

Re:And google will retain that info exclusively.

[KiloByte]

And if Google proxies and caches the images as soon as the service receives the mail, marketers can verify if the address is a valid gmail address or not by just sending mails and waiting for Google to cache the image. Expect more spam if this is the case.

Verifying that foobar@gmail.com is a valid address doesn't give spammers any real information: the namespace is so full even most pwgen outputs point to existing names, as long as you don't have embedded numbers (on gmail, addresses seem to have numbers at the end).

Thus, that check can be quite simplified to "does a Markov chain say this string of letters is pronounceable?". Not a big benefit to a spammer. On the other hand, they don't get told anything about the recipient anymore.

While for a small mail provider this change might leak some info, for Gmail it seems to be nearly entirely positive.

I for one don't use Gmail for privacy reasons, and don't fetch remote images, but good luck training aunt Lucy about that.

Re:And google will retain that info exclusively.

[blippo]

Isn't this what everyone does today? I thought the whole point of tracker images was personal urls like 'img158294.png'.

It won't help the users privacy a bit, or actually make it worse since users can't ignore image attachments anymore - google automatically hit the tracker url for them...

Re:And google will retain that info exclusively.

[hairyfeet]

Do those even work anymore on anything other than XP? Because I fix PCs 6 days a week and I haven't seen one in years. The way most folks get infected nowadays is 1.- "Hey its your (insert friend's name) on (insert IM) and I found this great new thing that made my PC faster, just (click this link,push this button)". 2.- "You want to see teh lesbians? To watch this hot video just run 'IzNotViruzIzCodec.exe'". 3.- (insert friend name) just sent you an e-card for (insert holiday), just go here and receive your e-card!" 4.- "Oh noes, you have teh viruz OMG! Run 'IzNotViruzIzCleaner.exe' to get rid of it". That last one works well on old folks BTW

As for TFA yet again another change that fucks the user or takes a valuable tool away from the user while giving Google more power....are we even surprised anymore? the only nice thing about Google in the last year is only the hardcore Googleaid drinkers buy the "Do no evil" "don't be evil" horseshit, the rest of the world can see its as much bullshit as "think different" and "where do you want to go today", Google has become just as nasty as the other two and in some ways worse.

Re:And google will retain that info exclusively.

[icebike]

While I applaud the move, it is about competitive advantage for Google.

If you applaud this you haven't thought it out very far.

Almost ever SPAM has small uniquely named images embedded. Often single pixel images.
These are encoded to your email address. If you fetch this image, your email address is VERIFIED. You just did the spammer a favor.

If you were reading the email with a mail client, you would NEVER fetch these, because 1) spam is spam, and 2) most
email clients don't download images by default and most email recipients are just fine with that.

With Google pre-fetching all of these, every GMAIL address id Verified for the Spammers.

Its not a well thought out scheme at all. No sensible person would read Gmail with a web browser from now on.
The wise choice is to use a traditional Email Client, (something like Thunderbird, Kmail, k-9 mail, Evolution, etc), and set them not to load images at all.

Re:And google will retain that info exclusively.

[icebike]

De-duping of images that have unique names keyed to your email address? Really?

All the spammer has to do is watch his web server logs to know that the address was a REAL email address, because google will be hitting every one of those images.

Since spammers arrange to never receive bounced mail, the uniquely coded embedded image url has been the favorite tactic for email verification for over 10 years. Google is going to give these guys a gift.

Read your gmail from a email client (pop or imap) and hope Google is smart enough not to hit those links unless you are using a web browser.

Re:And google will retain that info exclusively.

[icebike]

Marketeers already know the address exists the moment they get a 200 on the RCPT TO: header. Spammers, using botnets, generally don't care about the maildelivery itself, for these the autodownload of images is extra information.

Spammers do everything in their power not to get bounce messages. They do everything they can to not personally contact your (google's) mail server.

The fact that uniquely encoded image URLs are embedded in virtually ALL spam and UCE should be proof enough for you that you haven't thought your argument through. Go look at your email raw view someday.

Re:And google will retain that info exclusively.

[Anonymous Coward]

This.

I work for an email marketing company. Since our customers are very keen on not being mixed in with spam, we (and I think I speak for most of our competitors in this respect) take care to ensure only legit (confirmed double opt-in) email accounts are listed, to keep our servers' reputation perfect. Understand that it is in the best interest of legit senders to make customers WANT to recieve their emails. Open images and the statistics they create are primarily used to fine-tune the emails sent.

These open pixel images have practically no value to spammers (hence very few spammers actually use them); sending out spam over botnets, they don't care if an email address exists. They might care if a batch of several thousand email addresses no longer exists, but tracking and logging individual recipients... that's damn expensive if you're sending to millions of email addresses.

This cache won't hurt spammers.
It hurts companies you have subscribed to receive email messages (I sure hope you trust the average Hotmail user's taste, since emails will change to suit their needs).
And I dare bet that pretty soon, Google will start selling this information, and then everybody will be hurt.

Re:And google will retain that info exclusively.

[StripedCow]

The solution is simple:

if(connection.ip_address in google_ip_addresses)
    write(connection, "Sorry Google, only the user may open this image!");

Re:And google will retain that info exclusively.

[StripedCow]

Uhmmm... what if google reads the images *regardless* of whether the recipient is known?

Re:And google will retain that info exclusively.

[icebike]

Funny little AC, still assumes spammers connect directly to google. How quaint.

Re:And google will retain that info exclusively.

[icebike]

Typically SMTP servers won't accept delivery if the recipient is unknown, although I suppose when you write your own SMTP server you can do pretty much as you want. I can't see any up-side for google to process mail for NO ONE.

If they don't accept the body of the message they wouldn't get the URLs.

Re:And google will retain that info exclusively.

[mysidia]

In which case, a broken link or no image will be shown to the user, and it might result in your message being marked spam.

Re:And google will retain that info exclusively.

[icebike]

Your example of devious spammers using single-pixel images is a really poor one. Google probably has all those images stored somewhere already.

No, its not a poor example. I get them all the time.
Here's one for you: (it took me all of 15 seconds to find one in my spam bin)

https://q4fg3t1i.emltrk.com/q4fg3t1i?p&d=3D3215081772,F2A9FJZZ,TC05EM,CMNC04,U007

Grep the raw text of your UCE and SPAM mails. LOTS AND LOTS of companies use emltrk.com In fact, that's all that company does.
But lots of companies don't use a service like that, they just have a bazillion random images on one of their own web servers.

Further, google can't very well decide by itself that Image A127867 sent to Joe Sixpack is the same as image B835234 sent to Martha Uptight.
After all, one genital selfie might look a whole lot like another.

Re:And google will retain that info exclusively.

[tlhIngan]

Verifying that foobar@gmail.com is a valid address doesn't give spammers any real information: the namespace is so full even most pwgen outputs point to existing names, as long as you don't have embedded numbers (on gmail, addresses seem to have numbers at the end).

Actually, it does. Because it tells the spammers that the recipient opened the email

Google doesn't fetch the image until you open the email. And the moment you do, Google just confirmed that the email was read. And that information is very valuable.

There are two ways Google can fix it:

1) Set "Don't load images" back as default again, as it is now and in every email client.
2) Simply load every image, so valuable information like that isn't revealed - the marketer just pays for bandwidth and gets zero information - they don't even know if the image is read. No storage requirements as Google can re-write the email to self-contain all the images.

Of course, Google is probably going into email marketing - given how Gmail has sprouted that "Promotions" tab (yes, you can turn it off, but don't you think it immediately foreshadows something? It's not Spam, but "Promotions" - what, spam that someone paid to bypass Google's filters?). And they don't need competition - best way to squash it is to starve out the existing marketers.

And of course, since Google's in the information business, selling that information is very valuable - Google knows what you like, so they can sell targeted ads into your inbox.

Re:And google will retain that info exclusively.

[chromas]

They'd have to hit the server with the unique name before they could find out if the data is in cache.

Re:And google will retain that info exclusively.

[mrt_2394871]

If you want to know if I've read an email:
request a return receipt
If I want to give you that information, I will.

Goodness, there's an existing, non-scummy way of working all this out which preserves user expectations of privacy and provides you with the information you actually want, not a poor proxy of it.

Re:And google will retain that info exclusively.

[gsslay]

How would you feel about your customers sending tracking images to you with orders/complaints/queries? Just to "fine-tune" whether they deal with you again? I imagine it could be statistically enlightening to see how quickly you open emails, how often, and how long the response takes. Not so keen?

I appreciate your efforts to ensure that your emails lists are on target and not spammy, many companies are not so diligent. (Particularly with confirmed opt-ins.) But you have no automatic right to collate any further information about your customers unless they intentionally provide it. Tracking images are sneaky and most certainly not used by your customers intentionally. There is a reasonable expectation of privacy when reading your own email on your own computer.

You're right about two things though. The days are long gone when spammers cared about whether an address was valid or not. They are not incurring any costs spamming to invalid addresses. All they care about is how many suckers they hook with a response. And yes, the cached image hits are yet more information being sucked up by google, that will inevitably be sold in some way in the future.

Cutting into their business

[Anonymous Coward]

Of course they're cracking down hard - stealing user data is Google's job...they don't like the competition.

They do see open rates

[Anonymous Coward]

The cache system honors no-cache headers. As long as your images are served no-cache, you do see exactly when the email was opened, since the GMail servers refetch it every time. If each user gets a unique URL, you know exactly who opened the email.

Re:They do see open rates

[Anonymous Coward]

Multiple tests by multiple individuals have shown that they do NOT honor any of the various no-cache headers.

Tracking unique users is still easy (using a unique URL) - but tracking how many times they opened the email, or where they opened it from (IP address) or on what platform is now lost.

Re:They do see open rates

[StripedCow]

Tracking unique users is still easy (using a unique URL)

Not if google simply opens all e-mail behind the scenes, regardless of whether the user exists or not.

Possible?

[Tim12s]

Well, pulling all the images certainly solves the problem of having to display emails with images. The only reason we (I) don't click the display-images button is because the images allow us to be tracked, the images may have some sort of exploit (rare). Originally this used to be due to limited download speeds.

I suspect caching the images allow pre-processing of the images and therefore making the whole system more secure by default. Images could therefore be displayed in full by default with images, preferably with some large images being intelligently excluded by default.

Google could release a mass marketing email API/gateway and monetise that allowing marketeers access to data regardless of whether you open the images/email or not. This is slightly more valuable information.

Re:Possible?

[EvanED]

I suspect caching the images allow pre-processing of the images and therefore making the whole system more secure by default.

I saw mention that Google will be transcoding them, so yeah, you should be more protected by exploits.

That said, I still turned off the showing of images by default because of the first issue you mention -- otherwise Google will still go out and download the tracking bugs.

Re:Possible?

[symbolset]

Image formats have been used to compromise browsers in the past, so automatically loading images in your webmail or email client is a bad idea. Fortunately this is just a change from the default behavior so you can turn it off in the options.

In fact, Microsoft just patched a .tiff image format exploit last Tuesday.

The fix that breaks things

[Kvasio]

This fixes: opening ratio, opening time, user's IP.

This breaks: spammers will now have confirmation is the @gmail email is valid or not.

Re:The fix that breaks things

[Cassini2]

If I were google, I would download images in all incoming messages regardless if they are intended for real email boxes or not. This would let them know which websites are being used for spam. The spam detector could use this information by pattern matching every image (regardless of relabling or website copying), and mark spam accordingly.

What this is really

[Rosco P. Coltrane]

is a monopoly tightening its grip on the market it monopolizes.

Harder for **Other** Marketers

[perpenso]

Yeah. The move is to make things harder for **other** marketers. For the marketer named Google it confers advantages.

change or same mistake I made about announcement?

[patrixmyth]

Is this a new change, because after I saw the google announcement, I saw a report that they would share all that data about loading of images with marketers. End result: safer images, but just as much information for marketers, as along as they make nice with Google as 'official' email marketers. Would love to be wrong. Here's my source, Ars Technica article.
http://arstechnica.com/security/2013/12/dear-gmailer-i-know-what-you-read-last-summer-and-last-night-and-today/

Awesome for spam/tracking

[saikou]

Actually, this is rather awesome for spam/tracking of "real" addresses.
Before silly users could refuse to load external tracking pixels with unique IDs, assigned to each email.
And now? It's auto-downloaded for everyone. Yay!

While absence of IP address, Referral (if tracking image was loaded via https) and Browser info is sad, "everyone now auto-loads images" waaaay outweighs it :P You won't hide from confirming that email address that easily ;)

Re:Awesome for spam/tracking

[Stonent1]

Now all the spammers will get their servers overloaded. If they send out millions of e-mails and they all immediately get "opened" by google trying to pull in the picture data.

Re:And when the next JPEG or PNG exploit comes alo

[Anonymous Coward]

No, because Google will scan the images for viruses and common inconsistencies, then convert them to raw pixel data, using there decoding libraries that don't have these exploits, and then re-encode them into consistent and buffer-overflow-free images, that will work on any old and/or bug-riddled operating system or browser used by the recipient.
I hope google will also re-sacale images when people embed 3000 DPI company logo's in HTML-emails.

if Google is smart, they download 1 copy, ignoring

[raymorris]

If Google is smart, they'll download approximately 1 copy of each image, ignoring the tracking ID in the URL.

"Most successful tech company in the world" suggests that they may in fact be smart.

Re:Down to a single info

[Anonymous Coward]

I suspect Google will load the image even if the gmail address is invalid, or else it would be an easy way to build a list of all valid gmail addresses. So your example does not indicate that it ended up in someones in-box (or spam box!), let along that someone actually opened the email.

Re:if Google is smart, they download 1 copy, ignor

[Anonymous Coward]

You make the tracking ID part of the image name. Set up a cgi to always return the same image regardless of what it is called. Use a fake hashed etag thingy so they are always different.
Google has to download the image to see if it is the same, marketing mission accomplished.

e.g. http://examplemarketing.com/images/gjdfkadfdhkhkfdhkdsfhkhfdsqiuqr.gif

Oh. Please send royalties to A.C. @ Slashdot.

Summary is wrong wrong wrong

[Dynedain]

This summary is garbage and complete misrepresents the implications of Gmail's change. (I already researched this last week and developed a solution to avoid cacheing with in-progress email images that might get replaced with final versions)

Every singe email marketing system already uses a unique image URL to identify a given recipient. This is frequently called a "tracking pixel" because it's usually a 1px transparent gif stuck in the corner of an email where it won't be distracting. In fact, this method has been used for web tracking as well for many years. It's how Google Analytics originally worked.

Since these unique images will still get loaded when an email is opened in Gmail, marketers will still be able to track your opens. What they won't see, however, is how many times you re-opened the email. And since the image gets cached and requested through Gmail's proxy, marketers won't get information about your machine like browser, IP address, etc. But if you click-through on a link, or you visited their site before (highly likely if you're on their mailing list) then they have most of that info anyways.

This caching by Gmail is primarily to speed up Gmail since it means images can be loaded and shared on Google's Content Delivery Network which is almost certainly faster than servers owned by the email campaign provider for image hosting.

Ad broker + NSA

[Anonymous Coward]

From the OP: "Google says the changes are intended to enhance user privacy and security."

I find this lie from google/doubleclick insanely funny yet darkly cynical.

To enhance user privacy and security, don't use services from this huge ad broker which has a small army of lobbyists working Washington to prevent laws that would harness our privacy, and which works with the NSA to rape our liberty and privacy. If you use gmail, you should have no expectations of privacy or security whatsoever. That would be insane. It is everything their prime directive is not - i.e. make money of your privacy.

Re:change or same mistake I made about announcemen

[Bogtha]

No, you are completely misunderstanding that article.

Before mail clients stopped loading images by default, it was possible to embed a "web bug" image in an email. Essentially a transparent non-image that is referenced with a unique ID for each user. When the email was viewed, the mail client would request this web bug, and their server could record a) that this particular user opened the email, b) when they opened it, and c) whatever information they could glean from a normal HTTP request - where in the world you are, what software you are using to read the email, what language you have your mail client configured to use, etc.

If at any point you click "Load images", you will be sending this information to whomever sent the email. It's just that by default this would not occur in the majority of mail clients.

Gmail are switching to proxying the images and loading them by default. This means that email senders will get a) and b) by default. You can remedy this by switching your Gmail settings back to the old default of not loading images by default.

However because they are proxying the requests for the images, the people sending emails no longer get access to c) - things like your IP address, location, software, etc.

You seem to have invented some kind of nefarious arrangement between email marketers and Google, but that appears nowhere in the article you link to. It does not describe Google sharing data at all. All the article describes is the fact that by default, email marketers can now get a) and b) by using web bugs - this is something you don't need an agreement with Google to use, it's a natural consequence of the technology in question. It's your browser that shares the data, and it does so by performing a normal HTTP request - this is information you send to each and every website you visit. There's no http://google.com/download-private-data-muhahaha.zip link that email marketers now have access to.

This change improves privacy and has no loss of privacy if you change your settings to not load images by default. If you leave the settings at their defaults, you gain privacy in some ways and lose it in others.

Worse, Google now blocks steganography too

[Anonymous Coward]

I'm surprised that everyone is focused only on how this affects advertisers. That might be just a decoy excuse for the modifications.

A far more fundamental change is that Google will now be transcoding all images, which inherently blocks the sender's ability to transmit steganographically hidden information with plausible deniability. I bet the NSA has been requesting Google to do that for ages, as it must have been an extreme headache to have to scan all images just to find the few with a hidden payload. No such payloads now.

Spooks aside, the effect of this on photography will probably be far more dramatic for the general population, since photographers often transmit precisely controlled images. Google's new transcoding means that Gmail is no longer suitable for sending bit-perfect images of known properties or quality, so we're going to have to put our images in archives from now on, which will be a pain to view.

It seems that Gmail is becoming strictly a conduit for advertising. Google is at least consistent in their evil.

Re: Worse, Google now blocks steganography too

[Anonymous Coward]

They're not caching attached images, they're caching linked images.

Re:Hah

[TheRaven64]

Not necessarily. A lot of email virus scanners will pre-fetch images and follow links in emails, for example. They'll do it even if they're just forwarding the mail to another server, and sometimes before the mail even gets to the delivery agent.

Re:Hah

[icebike]

Google caches content only for it's own Gmail client, so you have no choice of clients with regards to this caching.

That makes no sense.
If they only cache for their Gmail client, that would mean I DO have a choice, by simply using another client.

As for the disable feature, that is the FIRST thing I did. This feature does nothing to protect the user. Its all about giving google an advantage. It MAY be illegal. Its not at all clear that Google has the right to cache image files that were intended to be sent directly from my Brokerage account to me via an embedded URL in an email.

Schizophrenic google

[crossmr]

So do they want privacy or not?
On one hand they're claiming to serve up images by proxy to protect users privacy, on the other hand, they're using Google+ and youtube to force users to display their real name.

We had the issue where Google started forcibly customizing google services for you based on you signing up for Google+. When I signed up a couple years ago, it broke my news archive search, because it would only search news sites in Korea, and in Korean despite having everything in English and my account being created in Canada (I happen to be in Korea). While several months later that was actually fixed, they also went ahead and first removed the insanely useful timeline from the archive, and then just recently killed off the archive entirely, because who could ever want to read news more than 30 days old.

Butchering services, heavy handed user manipulation, my patience with google is quickly wearing thin.

Re:Hah

[Ash Vince]

Yep and in fact despite what I said earlier, this could be worse. If google pre-fetch every image for instance, then this could have some horrid consequences. Such as confirming e-mail addresses.

Jason

You all seem to assume you are the first people to realise this, ten to one says some Google engineer also realised this and so is just going to get the software to do a hit on the sending or linked server for every image, even if the email address it was sent to does not exist. Then, they can use the content of that image as an additional way to help identify unsolicited email.