British ISP Bombards Users With Deleted Emails

judgecorp writes "For three days, customers at British ISP Sky have been receiving a flood of old and deleted messages. The problem started when the company switched its email provider from Google to Yahoo. As it began to move accounts from one provider to another, it became obvious that the new provider could not tell which emails in the old system had been sent or deleted. Some users had up to 8000 old messages. The incident has been going for three days, as users are migrated. Sky is apparently unable to fix the problem — its best advice been to suggest users delete the old messages."

Comments

I'd be pretty pissed

[Anonymous Coward]

I'd be pretty pissed if my ISP shipped a lot of my personal data including deleted data to Google.

Re:I'd be pretty pissed

[Anonymous Coward]

You might want to read the summary again. Google already had the data. They shipped it to Yahoo, and Yahoo bungled it up.

Re:I'd be pretty pissed

[TheGratefulNet]

if they had only let the good yahoo engineers work from home, none of this fark-up would have happened...

(snark)

Re:I'd be pretty pissed

[radiumsoup]

actually, I'd wager it's Google who bungled it - they store items their own way - because they can - and when asked to move the old mailbox, my bet is that Google didn't use the IMAP protocol as written and transferred all the deleted mail in a folder marked something like "Google-like deleted folder but not really deleted just archived". When the Yahoo servers received it, they ignored it as non-standard and shunted the "deleted" (but not actually deleted) mail to the inbox.

If Google had actually deleted the mail instead of archiving it, then there wouldn't be a problem.

Re:I'd be pretty pissed

[Anonymous Coward]

OK, manager of a data centre and an ISP here. This happens very easily if you move from one IMAP/POP3 provider to another (even if just changing software on your mail server, sometimes even with a major version update). There are two issues normally:

For IMAP users, the way the IMAP server stores it's flags for Seen, Deleted etc. may not be recognised by the new software.

For POP3 users, whether or not an email has been downloaded is tracked by the client, based upon the UID for the message. If this UID is changed (different servers use different systems) the client will decide the message are new - where users decide to leave messages on the mail server (rather than deleting them after retrieval), this is a common problem.

Neither of these cases is necessarily the fault of Sky, sometimes it's just not possible to reliably import this information between mail servers, and in the case of POP3 users, it's just down to the fact that POP3 is not designed for leaving read messages on the server for multiple clients to pick up.

Re:I'd be pretty pissed

[Anonymous Coward]

GP fail but why would they ship deleted emails in the first place. Seems Google doesn't really delete anything.

Re:I'd be pretty pissed

[Anonymous Coward]

My experience with various imap configurations for gmail (mutt, emacs) agrees. To delete something, you have to move it to the [Gmail]/Trash folder, rather than going through the standard imap 'delete' procedure.

That said, it's pretty obvious if you look for a few minutes that [Gmail]/Trash should refer to a trash-like location, so it might be a good idea to not import it, or import it into the trash-like location. Unless the deletion process has been so borked (gmail 'deletes' by copying messages to the trash folder, if there's an ISP layer on top it might forget the following step of 'actuallly really imap-delete it from the Inbox and All folders') that the import process can't reconstruct the folders to figure out what messages are deleted. But that would probably be the fault of the Gmail/ISP integration, not Yahoo.

Re:I'd be pretty pissed

[Nerdfest]

People aren't getting paid to spread FUD about Yahoo yet though. Give it a bit.

Re:I'd be pretty pissed

[LordKronos]

Actually, I'm pretty certain I know exactly what happened, because I just handled a major migration to google and dealt with an issue like this. It's due to the way google uses labels instead of folders, and how they (mostly-transparently) expose them as folder via imap (though this is one of the few non-transparent side effects).

In google, when you "delete" a message via imap, it doesn't get deleted. Instead, google just removes the label. That message still exist with all of the other labels, and it also exists in "All Mail" (which is exposed via IMAP through the "[Gmail]/All Mail" folder). So, if you have new mail come it, it is by default in your INBOX and your "[Gmail]/All Mail" folders. When you then delete it from the INBOX via imap, it's still in All Mail.

The way to deal with this is to move the message into "[Gmail]/Trash" instead of deleting it. That will truely delete it. However, since that wasn't done all along, those "deleted" messages are now orphaned in "[Gmail]/All Mail". There is a potential way to resolve even this problem, but it depends on how the account has been used. If users have logged into Gmail directly and taken advantage of the "Archive" feature to remove a message from the inbox (without truely deleting it) then all bets are off. There is no way to differentiate intentionally saved messages from deleted-via-imap messages. However, if it has only been accessed via imap (and users haven't intentionally been trying to take advantage of the All Mail folder), then you can do the following via a script:

Go through every message in "[Gmail]/All Mail".
For each message, try to find that same message in another folder.
If you don't find it in another folder, then that message only exists in "[Gmail]/All Mail". You can then move it to "[Gmail]/Trash" to get rid of it.

Searching for messages 1 at a time is a bit slow, so you can optimize this by first building a list of all messages in other folders. If you just retrieve a few headers from every message, it's actually fairly fast. The "Message-ID" field is usually sufficient for this, but there may be messages here and there that don't have that header, so you'll have to have other headers to fall back on.

Re:I'd be pretty pissed

[interkin3tic]

I personally would be more pissed about having to go back to yahoo instead of gmail, but that's only because when I was using yahoo mail years ago, it sucked. Maybe they've improved since gmail set higher standards. I don't really care to find out.

Re:I'd be pretty pissed

[Anonymous Coward]

oh bullshit. It *is* the fault of Sky because they chose to change what they were doing without working around the issues their customers were going to see.

Re:I'd be pretty pissed

[LordKronos]

Google does delete stuff (at least from our view...who can say what they really do behind the scenes). This is just a case of Google only being a pseudo-imap system. It's compatible with imap clients, but it's got some oddities in how they expose their internal system to imap. See my post above for more info.

Re:I'd be pretty pissed

[mutube]

Surely they could just opt to sync every folder except the "[Gmail]/All Mail" folder. Doing that, and syncing [Gmail]/Trash to the Yahoo! deleted mail equivalent would sort it all out.

I think you're right though. Sounds like the people handling the migration just aren't very familiar with the Google IMAP interface.

Re:I'd be pretty pissed

[LordKronos]

Actually, I doubt it was Yahoo's fault either. This can't be the first time Yahoo has done an import from Google, can it (assuming they even handle the importintg for you...google doesn't)? If they had done this before, they'd have known about this problem with IMAP and the All Mail folder. So I'm guessing it Sky itself that made the mistake (or whatever consultant they hired to to do it) because of their unfamiliarity with Google's system.

Re:I'd be pretty pissed

[KingRobot]

Yes, this... I can't believe there aren't more people with this answer.

Re:I'd be pretty pissed

[nozzo]

Well they must have tested this thoroughly beforehand - and they would have known this would happen. It would have been courteous to fess-up and warn folks beforehand. Just good customer service.

Re:I'd be pretty pissed

[Anonymous Coward]

Let me guess you also think the internet lives in your cable modem.

Re:I'd be pretty pissed

[Dishevel]

If the users had actually deleted the mail instead of archiving it, then there wouldn't be a problem.

You realize that those email were not deleted.
Regardless of the summary the emails were archived not deleted.

Re:I'd be pretty pissed

[confuscan]

????? At the barest minimum, if you're going to make life for your customers painful (e.g.. an upgrade imposing downtime, degraded service, h/w swap outs, etc) THEN you at least give them a heads up about what's coming and if possible, some guidance on how to minimize the pain. That said, if you can avoid impacting your customers, you perform some due diligence, hell, even do some testing before hand.

Re:I'd be pretty pissed

[LordKronos]

Yes, for migrating off of google, that's also an option (which I thought of right after clicking submit), assuming the recipient system supports folders. I don't know about yahoo. I was messing with hotmail a few months back before our migration, and I think my reccollection was that they support IMAP to the inbox, but no arbitrary folders. In a case like that, you'd have to collapse all the mail down to 1 folder (the INBOX), so pulling All Mail would be the easiest way.

In my case, we had a similar sort of issue while moving onto google, so ignoring "All Mail" wasn't an option, so I resolved as indicated in my other post.

Re:I'd be pretty pissed

[drinkypoo]

Actually, I doubt it was Yahoo's fault either.

What are you, new?

Re:I'd be pretty pissed

[Anonymous Coward]

How is a technical explanation of what may be behind the problem bullshit - I made no comment on Sky's handling of the change over.

Re:I'd be pretty pissed

[mutube]

Ah right, I get you. I didn't realise the folder support might not be there - it would probably cause more trouble than it's worth pulling individual folders separately and trying to remove duplications (multiple-tagged). Since they've apparently already copied over the All Mail folder for all their users you're right that the simplest fix would be pull the Trash and remove anything that is in there from the user's accounts.

Seems like the sort of thing you'd test before migrating all your live user accounts.

Re:I'd be pretty pissed

[Big Hairy Ian]

So basically what you're saying is that 99% of corporate are done using a system designed initially as a hobby project by an intern at Sun?

Re:I'd be pretty pissed

[Anonymous Coward]

The Slashdot crowd will never grasp this. They are more interested in dreaming up big conspiracies and trying to find ways to shift blame to big evil corporations instead of their own stupidity.

This is absolutely correct. If users *choose* to use POP3 instead of IMAP, and choose to leave mail on the server, then they are also *choosing* to have duplicate mail historical problems eventually. They then blame their ISP for their own stupidity.

POP3 was never designed to keep track of message read/unread states on the server. It was designed to simply download the mail, and keep track of state via UID of the messages on the client. UIDs on stored messages are not permanent, nor does RFC require them to be. They can, and do change as systems get upgraded, etc. If the 'read' state was kept on the server, how do they think their other POP3 clients would know to download them?

So instead of these Slashdot wannabes admitting they were too clueless to use IMAP, or to perhaps use a webmail client to move their old mail out of the inbox to another folder, they blame the ISP.

Re:I'd be pretty pissed

[Anonymous Coward]

God damn you're an idiot. Yes you did.

"Neither of these cases is necessarily the fault of Sky,"

Yes, it's entirely their fault. They can't work worth a fuck.

Re:I'd be pretty pissed

[Patch86]

Neither of these cases is necessarily the fault of Sky, sometimes it's just not possible to reliably import this information between mail servers, and in the case of POP3 users, it's just down to the fact that POP3 is not designed for leaving read messages on the server for multiple clients to pick up.

What a ridiculous thing to say. Sky is one of the UK's biggest ISPs, and they're moving their email provider from probably the biggest webmail provider in the world (Google) to probably the second largest (Yahoo). Are you really telling me that it was beyond the abilities of any of these three parties to a) do a test migration to check for serious errors, and b) make sure their migration methodology/toolset was compatible with the two technologies at hand before making the move?

To let such a huge and obvious public-facing error go live to so many people smacks of absolutely shocking incompetence.

Re:I'd be pretty pissed

[Sesostris III]

The problem is that the 'All Mail' folder can legitimately be used as an archive folder, i.e. for those emails we want to keep, but not in the 'In Box'. I for one use it like that. Any idea that items in the 'All Mail' folder and only in the 'All Mail' folder may get moved to Trash would worry me intensely.

I think the only solution would be to let the end user decide (which seems to be the solution they've hit upon). The problem of "deleted" emails reappearing should not be too worrying, as it lets the end user know that they weren't really deleted in the first place!

Re:I'd be pretty pissed

[Anonymous Coward]

It's not an "error". Those users who *chose* to use POP3 instead of IMAP are getting exactly what they should expect from using POP3 instead of IMAP. POP3 was never designed to leave mail on the server and keep track of read/unread states. Any users affected can only blame themselves.

Re:I'd be pretty pissed

[Anonymous Coward]

Having been through a couple mail migrations between servers we've always warned people that if they are using POP3 to download mail they may receive duplicated mail if they are leaving copies on the server. There isn't really any way to work around it properly from the provider side.

Generally IMAP isn't a problem (assuming the message has been purged and not just marked to be deleted). Of course how Google handles IMAP is a bit odd with labels rather than folders may complicate things in this regard.

We'd instruct people how they could change their pop client settings to not leave messages on the server, and how to change back afterwards, but there is always the people who complain that they are getting duplicates because they refuse to read any service notices.

Re:I'd be pretty pissed

[nospam007]

This raises several questions:

Yahoo still exists?

Who on earth would move them from Google to Yahoo instead of the other way around?

Did anybody expect something different?

Who got paid for this?

Re:I'd be pretty pissed

[Anonymous Coward]

Indeed, this is just insane, especially considering the kind of XSS attacks yahoo have been failing to fix in their webmail system for the past few months, why would anyone subject their customers to the danger of new and interesting hijacks in an already thoroughly exploited system when they're using gmail and it just works, what kind of kick backs were involved?

Re:I'd be pretty pissed

[Anonymous Coward]

Well, they bought Zimbra, which was a frankly astonishingly impressive, if a little slow at the time, AJAX webmail interface, but i can't see that much of it has translated into their current offering.
Also bear in mind that account security is a bit of a sore point for yahoo mail http://www.theregister.co.uk/2013/03/08/yahoo_webmail_hijack_flare_on/

Re:I'd be pretty pissed

[Calydor]

So if their tests showed it couldn't be done without the (relatively minor) inconvenience of unflagging mails, should they just have shrugged and never changed a thing?

Re:I'd be pretty pissed

[lgw]

No one cares about POP3 or IMAP. Seriously. Those are back-end choices that needed to stay back-end. Sky should have worked this out so that mail shown in the UI before was the same as mail shown in the UI after. Everything behind the scenes was on the ISP to keep behind the scenes, whatever technical cleverness that would have required.

I've done email archiving products for a living. This sort of BS isn't trivial - you have to think it through, and do extensive testing - but it sure isn't rocket science.

Re:I'd be pretty pissed

[Patch86]

If the service "offers POP3 support", which presumably they do, then it is the service supplier's problem. Not the user's fault for using one of the two services on offer which happens to be the less sophisticated one.

And in any case, I've not read anything in this thread or TFA to imply that POP3 users are affected while IMAP ones aren't. The GGP points out a way in which the problem would occur for users of both technologies. If I use an IMAP client to "delete" an email on one day, there is zero good reason for it to reappear in my inbox a year later. That is a system error pure and simple.

Re:I'd be pretty pissed

[Sir or Madman]

Yes, because the average user knows the difference, am I rite?

There is a car analogy here somewhere, and it involves blaming the client for not having a Masters in Automotive engineering.

 

Re:I'd be pretty pissed

[Anonymous Coward]

No but they should have contacted their customers BEFORE they startet the process so I can decide if I want to go with this bullshit or do something about it.

Re:I'd be pretty pissed

[kasperd]

Google does delete stuff (at least from our view...who can say what they really do behind the scenes).

I could say, because in the past I have been working on the team responsible for making backup copies of Gmail on tape. But I think the underlying storage system has been replaced since I worked on it. But a few of the key points haven't changed.

Once a user delete a message it does become inaccessible to that user. But it doesn't mean that all copies will be gone instantly. Having all copies be gone instantly would completely defy the purpose of having backup copies in the first place. The point of backups is that you can get back stuff, which you have accidentally deleted.

You could design a system where you could guarantee that the last copy was deleted exactly x days after the user hit the delete button. But designing for such an exact guarantee would require more resources. Instead you could have a system where the guarantee was that the last copy would be deleted between x and 3x days after the user hit the delete button. This flexibility allows for a simpler and cheaper design. I don't know what sort of range Gmail operates with these days, neither if the number is published.

For a data transfer such as the one we are talking about here none of those deleted mails should be part of the transfer. If they were, that would sound like a mistake on Google's part, but I don't think that is what happened. I think your explanation sounds more likely.

If Sky had said they wanted any email deleted in the last x days to be restored from backup to be part of the transfer, then that would have sounded like a completely unreasonable request to a lot of people. I think Google would have argued, that it would not be in the users' interest to do so.

Re:I'd be pretty pissed

[Filgy]

....and after they bought Zimbra they sold it to VMWare. VMWare now owns Zimbra fully.. I think yahoo has nothing to do with it anymore. (We use Zimbra at my work for email, and it's pretty nice).

Re: I'd be pretty pissed

[Anonymous Coward]

Well, usually before doing massive batch data move, you should test with a reduced set in order to assure everything would work as expected.
Doing it blindly is just a no-no, imho

Re:I'd be pretty pissed

[Anonymous Coward]

So if their tests showed it couldn't be done without the (relatively minor) inconvenience of unflagging mails, should they just have shrugged and never changed a thing?

LOL. Relatively minor? Sorry, but not so. From the last couple of years, I've got thousands of messages I've saved, and probably 10-50 times as many that I deleted. If these all got mixed together, with no real way to know which was which without going through them 1 by 1, that would fucking bitch. Sky's recommended "solution" is an absolute joke:

"Delete any emails you may no longer want/require e.g. any that are older than a certain date."

So their solution is to just go through everything and determine 1 by 1 which to delete? Gee, thanks captain obvious. Or perhaps they mean just pick a cutoff date and delete everything before that. Equally useless.

Re:I'd be pretty pissed

[P-niiice]

It would be Yahoo's responsibility, at the behest of SKY, to have checked this out beforehand, as Yahoo is getting paid for this and should care about delivering a good migration. This makes the "new" Yahoo look very, very bad. My boss would have my ass for this.

Re:I'd be pretty pissed

[sjames]

A faster way would be to delete all in All Mail, then recurse through the other folders creating a duplicate tag in All Mail. Messages that only existed in All Mail before will be gone. All others are preserved.

Re:I'd be pretty pissed

[osu-neko]

So basically what you're saying is that 99% of corporate are done using a system designed initially as a hobby project by an intern at Sun?

99% of what we use and do today is the result of accidental discoveries. "Designed as a hobby project" is a more deliberate, intentional genesis. The key word here is "initially". The state of something today is not the same as its initial discovery or invention. "Designed by an intern" shows a more informed and qualified genesis than "resulted from an accident", but that doesn't necessarily make it better today.

Re:I'd be pretty pissed

[darknet-defender]

You might want to read the summary again. Google already had the data. They shipped it to Yahoo, and Yahoo bungled it up.

Actually, this is Google's fault I was migrating away from Google using POP3 to download my mail messages off their server. IMAP had been too slow with thunderbird and evolution. When it was done, I noticed that pop3 sends me my sent mail.

Re:I'd be pretty pissed

[darknet-defender]

And when the FBI/CIA/DHS and countless other Intelligence agencies read your mail in realtime cause your a dumbass using IMAP - its just you to blame

Re:I'd be pretty pissed

[BitZtream]

I've done the sync between gmail and Dovecot and Cyrus IMAP.

The 'tags as folders' thing isn't an issue for imap clients as the server tells the client (because its tagged) that the message is deleted.

I can safely say Yahoo is at fault. Having moved many millions of messages myself, there is no reason on Google's side that this 'HAD' to happen.

The Deleted flag is NOT one of the flags that Google doesn't handle properly.

If you simply run IMAPSync on gmail, it will work almost flawlessly, none of the issues Sky has had come up when dealing with Gmail to Dovecot or Cyrus.

They didn't properly handle the migration or they migrated to a shitty provider, either way, both Sky and Yahoo should have their asses handed to them for giving absolutely horrible service to their customers.

Re:I'd be pretty pissed

[darknet-defender]

I think Google makes it difficult and non-standard like this to make things difficult. I moved away from GMail because I simply don't trust Google anymore. It was a bitch to reorganize my mail and downloading the entire thing over imap just wasn't working without freezing my clients (and I only had 2gb of mail). POP3 worked perfectly, but I got everything that was in every folder. I was smart enough to empty my trash first! :)

Re:I'd be pretty pissed

[darknet-defender]

Google scans your e-mail and sells that info to advertisers. Even when you pay for it. Scary shit. Yahoo for business doesn't do this.

Re:I'd be pretty pissed

[LordKronos]

A faster way would be to delete all in All Mail, then recurse through the other folders creating a duplicate tag in All Mail. Messages that only existed in All Mail before will be gone. All others are preserved.

You would think so, wouldn't you. And that would probably work if this were a fully imap compliant system, but that's exactly the point (and why they got into this mess to begin with)...it isn't.

For gmail, the message in "All Mail" is the sole existence of the actual message in the system. The copies of the message in other "folders" are really just references to the copy in "All Mail". So deleting it from All Mail is not really a choice. I can't remember from my testing back then what would happen if you DID delete it from All Mail. Either google would reject the delete, or the message would disappear from every folder it was in and not even leave a copy in the trash (you had to move it to trash if you wanted that). I know google rejects deletes of the entire All Mail folder, but I don't recall the result for trying to delete individual messages.

Re:I'd be pretty pissed

[LordKronos]

You are right...it MOSTLY works flawlessly, but there are subtle issues you need to pay attention to. The All Mail thing usually would not be such a concern I would think. If you were migrating to a server that is fully imap compliant, then yes, those deleted messages would show up in an additional folder called '[Gmail]/All Mail' on the destination server, and you could just promptly delete that folder and you'd be back to what you would have expected.

However, the fact that these messages are showing up in people's inbox leads me to beleive they didn't do a full folder migration, and instead only copied the All Mail folder to the new inbox. Why would you do that? Well, as I mentioned in another post, my recollection of hotmail is that it supports INBOX access to imap, but doesn't support other folders. I don't know if Yahoo does a similar thing or not, but to me, that seems like the only excuse I could think of for copying All Mail to the inbox. Though even that isn't a good excuse...I don't see how they could justify letting their users lose their entire folder structure in the migration process.

Re:I'd be pretty pissed

[vakuona]

Google does not sell info to advertisers. Google sells adverts based on your info. If you are going to bash Google, then at least get your damn facts straight!

Re:I'd be pretty pissed

[rsborg]

Surely they could just opt to sync every folder except the "[Gmail]/All Mail" folder. Doing that, and syncing [Gmail]/Trash to the Yahoo! deleted mail equivalent would sort it all out.

I think you're right though. Sounds like the people handling the migration just aren't very familiar with the Google IMAP interface.

Which is ridiculous since one learns quite a bit about Gmail IMAP by having used it at all (just ask anyone having configured iOS+Gmail prior to Apple's adding the "Gmail" option for account setup) or reading the public documentation.

This is simply a case of the project team a) not reading very-available documentation on a very key part of the project b) not testing and uncovering this issue and c) doing a big-bang migration of all accounts. a+b+c = predicable clusterfuck.

Re:I'd be pretty pissed

[sjames]

That is a fairly ugly implementation.

Re:I'd be pretty pissed

[Ghaoth]

Good customer service requires that you think (or pretend to think) like a user. When dealing with thousands of customers, one must think at the lowest common denominator and that is someone who has NO idea what POP3 or IMAP is. Their email client has a facility to leave messages on the server, so some use that facility. The chaos that this can cause in migration would never be understood by the majority of users. I would suggest that those managing the migration were so wrapped up in the technical aspects of the migration that they forgot about the consequences of what they thought they were doing. Flawed methodology will ruin the best of intentions. The Sky is the limit with introverted, narrow minded incompetence. The results speak for themselves.

Re:I'd be pretty pissed

[Ghaoth]

"It would be Yahoo's responsibility"......and you know this how? I would suspect that in an operation of this size there would be various responsibilities and Sky would have a large say in the specifications and implmentation...maybe I'm wrong though.

Re:I'd be pretty pissed

[nospam007]

"Google scans your e-mail and sells that info to advertisers. Even when you pay for it. Scary shit. Yahoo for business doesn't do this."

Even though you're wrong, if they scan business mails of let's say paper-cup manufacturers, do you think showing them ads for paper-cups from another firm will generate many sales?

Re:I'd be pretty pissed

[jewens]

For IMAP users, the way the IMAP server stores it's flags for Seen, Deleted etc. may not be recognised by the new software.

Ok, I'll give you the Seen and Sent flags, but why is Deleted stored as a flag rather than by say, deleting it?

Re:I'd be pretty pissed

[Anonymous Coward]

No, you are the idiot who cannot speak English.

"Neither of these cases is *necessarily* the fault of Sky." - go away and work out what that means.

Re:I'd be pretty pissed

[Anonymous Coward]

For the general population of fuckwits who decide they did want that email after all.

Re:I'd be pretty pissed

[darknet-defender]

"Google scans your e-mail and sells that info to advertisers. Even when you pay for it. Scary shit. Yahoo for business doesn't do this."

Even though you're wrong, if they scan business mails of let's say paper-cup manufacturers, do you think showing them ads for paper-cups from another firm will generate many sales?

You obviously have no clue of how it works

Re:I'd be pretty pissed

[darknet-defender]

Google does not sell info to advertisers. Google sells adverts based on your info. If you are going to bash Google, then at least get your damn facts straight!

Prove they don't

Slight oversight?

[7x7]

Apparently they've never heard of IMAPsync ;-)

Re:Slight oversight?

[Anonymous Coward]

I don't think Yahoo even supports IMAP. Last I saw, the IMAP setup consisted of setting up a POP3 account and checking "leave messages on server".

Re:Slight oversight?

[7x7]

But Google *does* support IMAP. Getting files off Google with the deleted flag set should make it pretty easy to put only valid emails in to Yahoo. Their method likely uses something other than SMTP to get the mail in to Yahoo or there would be madness with the spam filters and rate limiters.

Re:Slight oversight?

[jonbryce]

They were pretty late to the game, but they do support IMAP.

http://help.yahoo.com/l/us/yahoo/mail/mobile/mail/imap-01.html

Re:Slight oversight?

[Anonymous Coward]

...or testing

"Deleted"

[Anonymous Coward]

I guess they used the Facebook definition of deleting contents...

Re:"Deleted"

[Kingkaid]

I would have hoped that "deleted" meant "deleted". How naïve of me.

Indeed. How else do you expect people to recover their email messages they accidentally moved to the trash, confirmed the move, and then deleted? ;)

Re:"Deleted"

[kasperd]

An email exists in more than one place. Deleting it from one system doesn't imply it is being deleted from all other systems. From the article it is not entirely clear which systems are in play, where it was deleted, and where it was not. From the description I get to think of two likely scenarios.

• Users downloaded the messages from the server to their own computer using the POP protocol. On the Google side it was configured such that a delete requested over the POP protocol was interpreted as archiving the message. The user then deleted the message from their own computer without thinking about the copy still in the archived mails "folder" on Gmail. Somebody messed up as the messages were transferred, and they all became available over POP again, and thus were downloaded again.
• The problem was on the sender side with messages in the Sent folder being misinterpreted as Outbox from where they were sent and then placed in a Sent folder. As old messages were being sent again receivers who had already deleted them are complaining as they show up again. They really had been deleted on the receiving end, but still existed on the sending end.

Re:"Deleted"

[John Hasler]

> On the Google side it was configured such that a delete
> requested over the POP protocol was interpreted as
> archiving the message.

Then they were not correctly implementing the protocol.

Re:"Deleted"

[kasperd]

Then they were not correctly implementing the protocol.

As long as the message is no longer visible to POP clients, they are indeed following the protocol. The POP protocol does not require the server to find every copy of the message in existence across all servers on the Internet and delete them. If downloading the mail to your computer caused the copy accessible through the webinterface to get deleted, people would be screaming about data loss. Gmail offers four possible configurations on what happens to the copy in the webinterface when the copy accessible through POP is deleted. It looks like the default is to do nothing, which treats them as independent copies.

I'm sure Sky could have gotten a different default setting for their own customers, if they had asked for it. But why would they have asked for a default setting, which would cause data loss for those users, who did not fully understand the system, they were working with.

POP is an outdated protocol, which was not designed for a world where users want to access their email on multiple devices. Rather than telling users they had to upgrade to IMAP or implementing POP in a way that was useless to most of the users, Google decided to implement something that was working well for the users.

Re:"Deleted"

[lgw]

As far as anyone can tell with Google, "deleted" does mean "deleted". However, most gmail users "delete" email by archiving it, not actually deleting it. In gmail, you only see mail if it is tagged in some way, or you search for it, so simply removing the "inbox" tag means you stop seeing the mail in your inbox (but it's still there in searches).

Having all of this not-really-deleted mail show up again and surprise users was a completely predictable side effect of moving mail from Google's idea of mail organization of everyone's else in the worlds. Clearly no one at Sky tested this migration, or even thought about it (well, presumably many engineers did and were ignored by management, as is usually the case).

I have a habit of never deleting emails

[eksith]

...Unless they're really useless or are too sensitive (I never send sensitive information via email, but despite my best efforts, I do get them sent to me). But I guess even that's not consolation that the information was private by any stretch of the imagination.

I keep trying to explain to people, email with Yahoo, Hotmail, Gmail etc... is like having a private conversation in a coffee shop or something. Yeah, you can get "some" privacy, but really, anyone can listen in or record if they really wanted to since you don't control the venue. Anything without PGP/GPG encryption is like that.

I can only imagine what this might be like for those folks. If this happened to me, and if I do delete messages, I'd be not only livid, but hosed as well. How can you sift through that much info in a single morning... or a week's worth of mornings?

Re:I have a habit of never deleting emails

[Anonymous Coward]

Just think of how bad it is for the people who use local email clients. The 'mail delivered' flag was lost, and messages were never deleted from the server. So they get a second copy of everything from before the transition.

Re:I have a habit of never deleting emails

[eksith]

Gotta completely ruin your week, if not a few weeks. This can not only ruin productivity, it can also potentially end a relationship with a client(s) or even cost new opportunities. Why would anyone trust someone who couldn't even get their emails in order (although if it's someone with understanding, they could potentially make it through), but that's an awful lot of duplicates.

An absolute disaster.

paid yahoo customer here

[Anonymous Coward]

I hate how Yahoo doesn't synchronize folders either on migration. There's so much to hate about yahoo but ultimately their low prices against google is what keeps them in business in the end of the day.
I don't often migrate email, but it's cheaper for me to do it than switch infrastructure...unfortunately.

Email by MurdochEmpire (TM)

[StoneyMahoney]

Telling the difference between emails that have been deleted and those that haven't, along with those that have been sent and those that haven't, costs extra. Doing it any other way means Progressivism wins!

Re:Email by MurdochEmpire (TM)

[operagost]

That's the weakest leftist troll I've seen for some time.

Re:Email by MurdochEmpire (TM)

[StoneyMahoney]

Grow a sense of humour already.

Re:Email by MurdochEmpire (TM)

[operagost]

But it wasn't funny.

Re:Email by MurdochEmpire (TM)

[StoneyMahoney]

If your dumb ass didn't notice, it was a two-part joke pointing directly at a very specific recent incident in American politics...

Re:Email by MurdochEmpire (TM)

[H0p313ss]

If your dumb ass didn't notice, it was a two-part joke pointing directly at a very specific recent incident in American politics...

Don't give up your day job.

Outrage! by Murdoch

[StoneyMahoney]

Simultaneously, Murdoch himself releases a statement calling for email service quality reform without realising he's contradicting his own company line.

Really?

[RedHackTea]

Wouldn't you try a test run on one user account before letting it fly for all users? Then this probably would have been caught. The bar has been lowered. Any takers?

Re:Really?

[kasperd]

Wouldn't you try a test run on one user account before letting it fly for all users?

You are absolutely right, that's how it should have been done. And you can take it in more steps. You don't have to go directly from one to all. Decide how many steps are appropriate and then increase the number of users exponentially between each step.

This sort of approach is standard practice at Google. I don't think Google made the decision to rush this through. I imagine Google was warning about the possibility of something going wrong, but Sky and Yahoo somehow reached an agreement, that Google was simply trying to hold onto the users for as long as possible. So they decided to ignore Google's "warning" and go ahead with a quick transfer of all the users. This is just my imagination of how it might have happened, I don't know if this is exactly how it happened.

A transition period where users are distributed across the two systems requires a special layer of frontends, which know which users are currently on which system. This adds complexity, and complexity increases the risk of bugs. For mails which are arriving via SMTP it is fairly easy to have frontends, which simply delivers copies to both Google and Yahoo. But for anything else it is hard to keep the systems in sync. Doing the entire migration in one go may have been choosen for simplicity. It is possible Sky wanted this approach because it was cheaper.

Re:Really?

[tqk]

Wouldn't you try a test run on one user account before letting it fly for all users?

I'm pretty much constantly amazed at what crap manages to be considered "competent IT staff" these days. This's at one of the biggest names out there too. There's a lot of blame to pass around here, from HR (for suggesting them), to management (for hiring them), to the drone who did this without bothering to first get their ducks in a row (research the problem maybe?). Then there's the moving from gmail to Yahoo bit. Wouldn't anyone be better than Yahoo, just based on their press over the last few years? Who made that ridiculous decision?

Re:Really?

[moeinvt]

Not migrating a few accounts at a time does seem to be an over-achievement in dumb-assery. Don't be too harsh on the IT staff however. I'm sure some manager/bean counter types made the decision to migrate based on cost only. Then, the project gets dumped on an already over-burdened IT staff under a tight schedule. Even "competent" IT people can screw up when they're given unrealistic requirements.

I'm not necessarily disputing your assertion, but at my former employer, the penny-pinchers were constantly making "cost saving" decisions with no understanding of the technical implications.

Avoid Yahoo mail servers

[Anonymous Coward]

Our ISP moved its email to Yahoo and my wife has had nothing but troubles since. Constant errors about being unable to connect - I think she went almost a week once without being able to access her emails over IMAP.

Re:Avoid Yahoo mail servers

[moeinvt]

"Our ISP moved its email to Yahoo "

I don't quite understand this. Why doesn't the ISP have its own mail servers? What does this look like to you as the end user? Are the server settings on your e-mail client referencing mail.yourisp.com or some yahoo.com server? When the change occurred, what happened to usernames and e-mail addresses from your ISP that conflicted with Yahoo! usernames?

then move it back jerks

[Anonymous Coward]

Dirty secret is, yahoos email passwords db has already been cracked wide open by a previous xss flaw, it is merely coincidence that i have had floods of btyahoo customers call me with "why do all my friends tell me i sent them 200 spam mails (headers confirm its from yahoo)", yet none of them are infected with anything, coincidence i tell ya, then wait till you hear POP3 problems with btyahoo garbage or their shtty software, go abroad on a business trip and there is a whole world of crap awaiting for you

the outsourcing of UK ISP mail is a total clusterfsck wherever you look, sky and virgin is just as bad with their fuckd up advert frames destroying googles UI, Murdoch and Branson have no dignity if they have to scrape for pennies at the bottom of a 468px banner

shite, all of it

Re:then move it back jerks

[Anonymous Coward]

Dirty secret is, yahoos email passwords db has already been cracked wide open by a previous xss flaw, it is merely coincidence that i have had floods of btyahoo customers call me with "why do all my friends tell me i sent them 200 spam mails (headers confirm its from yahoo)", yet none of them are infected with anything, coincidence i tell ya, then wait till you hear POP3 problems with btyahoo garbage or their shtty software, go abroad on a business trip and there is a whole world of crap awaiting for you

Wasn't a lot of spam from Yahoo caused by users who were logged in to Y in one tab in Firefox and browsing suspicious web sites (wares/porn) in another? The malicious dudes opened a hidden Iframe that accessed the Y account and spewed their spam. I saw something similar with Gmail when my account tried to spam an old email account I didn't use anymore.

What else can you expect from a MS subsidiary?

[Anonymous Coward]

MS never could handle email according to standards.

Don't use ISP email.

[hattig]

Never, ever, use ISP provided email.

Firstly, you might change ISP, so you lose your old email.
Secondly, they pull tricks like this.
Thirdly, they won't provide as good a solution as a dedicated email provider.

What I am wondering is if you can set up a new personal GMail account, and get it to sync your old emails from the ISP's own gmail service.

Re:Don't use ISP email.

[StoneyMahoney]

Back in' day, Steam account names were the same as the email address you signed up with. I used an email account I had from about 15 years ago when the only email address I had was an ISP-proscribed one with my dial-up package. Neither the email account nor ISP still exist and while it's possible to change the email address the account is connected to, but I can't change my Steam username. Anachronism!

Re:Don't use ISP email.

[SeaFox]

Couldn't you just set up both accounts in your mail client and just drag and drop the messages between the different sets of folders to upload them to the new account? I'm sure I've done something similar once before.

execs are assholes

[Anonymous Coward]

"...from Google to Yahoo"

Found the problem.

Actually, the problem is probably some dick up in the head office that had a "good idea."

Why switch from Google to Yahoo?

[Anonymous Coward]

I'm surprised Yahoo is still in business.

"Deleted"

[enabran]

I would have hoped that "deleted" meant "deleted". How naïve of me.

Why would an ISP have used Google in 1st place?

[aristotle-dude]

Shouldn't one of the basic services provided by an ISP be "email" meaning that they provide their own independent set of email servers?

Re:Why would an ISP have used Google in 1st place?

[petermgreen]

IMO the bundling of email service and addresses with internet access service is a historical artifact that adds lockin and reduces competition in the internet access market.

Re:Why would an ISP have used Google in 1st place?

[SeaFox]

Not providing email greatly lowers support costs because you don't have to help noobs through how to configure mail clients, getting blamed when their Outlook Express eats their old email, getting complaints about spam, etc. Plus you don't have to field hundreds of support calls when you bungle up moving the services on the back end, like here.

Re:Why would an ISP have used Google in 1st place?

[aristotle-dude]

IMO the bundling of email service and addresses with internet access service is a historical artifact that adds lockin and reduces competition in the internet access market.

So who do you work for? Google? Yahoo? Some other third party email provider? A dishonest ISP that lies to the customers and uses a third party for email? Here is a cluestick for you to hit yourself with. Lock-in is an artifact of a proprietary email schema. Google is an example of a proprietary extension of IMAP email. They don't "delete" you emails, they are just archived. Most email providers that offer IMAP protocol access will move the deleted email to the deleted folder and allow you to easily empty that folder whereas Google does not follow that standard. This story is an example of how Google's implementation is non-standard and causes a level of "lock-in".

If you are going to offer customers email but in the background use gmail then don't bother and just provide instructions on how to setup a Gmail account. I personally use both my ISP and a couple of cloud based providers but I am not a noob. You should not be lying to noobs.

Par for the course at Yahoo

[bbartlog]

Why anyone would switch their email provider *to* Yahoo I have no idea. At least I have an excuse, since I've had my email there for ten years and don't want to go to the trouble of changing it. But between the unexplained service outages (unable to retrieve emails, sometimes for hours at a time) and the security holes (get your contact list downloaded if you click on a bad link) I have no idea why anyone would choose it over other offerings, if they were starting fresh.

All the speculation and desceiptions are bunk

[rickb928]

Had they actually *tested* this, in advance, would we be discussing the various flaws in each mail systems' implementations, or their real/imagined problems?

No. We would not even know it happened.

Don't blame Google's use of IMAP flags and folders, blame Yahoo!'s apparent lack of planning. Or Sky. Or whoever. Plenty of blame at the receiving end.

If you're moving mail from some system into yours, the responsibility is yours to make it right.

And I've done this. Wait till ya hose up the passwords, my friend. Fun times.

IT happens

[Anonymous Coward]

Recently, I saw a large organization convert email systems. They are split into handful of business units that had their own IT staff and email systems. They made the announcement and told us how it would go. All your email, calendar, and contacts would be converted and everyone's mailbox size would be 25G. As the migration got closer, one business unit's IT kept making frustrating announcements while the other units went as smooth as expected. (I don't know the source of that units internal issues, but this is what was seen from the outside)

Announcement: Our mailboxes will be 2G in size instead of the 25G that everyone else gets.

Announcement: You will have to move your contacts by hand, we will take care of your calendar and email.

Announcement: You can't move personal email groups because the addresses are not saved in any known email standard.

Announcement: We tried to migrate a calendar but could not get it to work. We won't be able to migrate calendars.

Announcement: The automated email migration is taking longer than we expected, bu the vendor is working with us.

Announcement: The automated migration is taking too long, so we wont be able to do it. You will have to archive your own email.

Announcement: If you need someone to migrate your stuff, we can have a contractor do it for $200.00 for each mailbox.

Announcement: We recommend everyone purchase a license of Acrobat Pro and PDF your emails. Or email old messages to your self after the switch.

Announcement: Our existing system archives email after 1 year, you have to open each archived message once before you PDF it or it will PDF an empty email.

Announcement: Here is a trick to open up to 10 archived emails at once.

Announcement: We must remove every copy of the old email client (due to licensing) so all your old personal archives in that system will not longer be accessible.

And all the central mail groups needed to be recreated by the admin team with no conversion from the old lists. And the new groups could not contain outside email accounts. There were a hand full of mistakes that showed that someone had to do this by hand (scripts or automation would have create different mistakes).

I kind of feel bad for that IT team. Because if that was the flow of announcements going out, you can only imagine what they were dealing with inside the team.

   

Re:IT happens

[inglorion_on_the_net]

I guess the advantage of having multiple IT departments in the same organization is that when you give each of them the same tasks, you can see which ones are competent and which aren't.

I'm with the parent in feeling bad for the IT team that had all this trouble. On the other hand, one has to wonder why all the other IT departments managed to pull off the migration smoothly, but this one didn't.

Re:IT happens

[Anonymous Coward]

The only saving grace is if this group was accounting or some other group with legal requirements that made the switch much harder.

Re:IT happens

[girlintraining]

Mistakes with this level of both fail and visibility are invariably due to politics rather than technology. I'm guessing some mid-level manager decided their department was special and severely under-estimated the amount of resources required, and tried to in-house the migration for a political reason that now is obviously stupid but at the time seemed really smart, or at least said manager wasn't told it was doomed to failure.

Evolution

[Anonymous Coward]

Just think of how bad it is for the people who use local email clients. The 'mail delivered' flag was lost, and messages were never deleted from the server. So they get a second copy of everything from before the transition.

I use Evolution against a POP account. about once a year it seems to "lose its mind" and redownloads the entire mailbox. I suddenly have 9000 NEW but duplicate messages going back for a period of years. It's a pain!

The saving grace is that the messages are all NEW. Sort new messages first, scroll back a day, select and delete all. It still pisses me off though.

Right now I'm wondering if it is Evolution's fault or if the mail provider is moving servers or some such.

Re:Evolution

[Anonymous Coward]

My bet would be the latter. If they move it to a different server and are not careful to preserve the UID numbers on each message, then your POP client has no way to know they are the same messages.

Re:Evolution

[Anonymous Coward]

I use Evolution against a POP account.

You have my sympathy.

" its best advice been to suggest"

[Anonymous Coward]

Huh?

Dude! You're Getting The Cloud!

[Anonymous Coward]

Shouldn't one of the basic services provided by an ISP be "email" meaning that they provide their own independent set of email servers?

Most providers are switching to cloudy services such as Gmail, Yahoo!, Outlook(Hotmail). It's cheaper and they don;t have to maintain it. This is especially important when you aren't charging your subscribers an extra line item for email service.

Why do ISPs even offer email as a service

[bhspencer]

Who uses the email service provided by their ISP anymore? How may times have you received the email "I changed my ISP so this is my new email address"? If you are using the email service provided by your ISP you are doing it wrong.

Re:Why do ISPs even offer email as a service

[Anonymous Coward]

The ISP I work for has offered email services for 19 years, for many of our users it was their first and some cases only email account they have ever had. We don't enable email accounts for new users by default but we do still get people asking for an email address at times.

Reasons vary from wanting to have a separate account to receive the billing and service notices on, to liking the domain name to any other reason they might create.

"Deleted"

[Anonymous Coward]

If the deleted messages were actually deleted, this wouldn't've happened.

Reason 8000

[SCHecklerX]

to run your own mail server.

Re:Reason 8000

[fazey]

The problem I run into with that, is that people who email me are using these services. But I am right there with you on that.

Re:Reason 8000

[chihowa]

to run your own mail server.

Absolutely. With incredibly cheap backup MX services and the ease of using your ISP's SMTP server as a relay, there's no reason not to run your own, even if you're on a residential link.

IMAPsync

[Anonymous Coward]

Do you mean that hideous French perl script? Or something else of the same name?

I ended up rewriting huge portions of the French one when I used it five or six years ago, so I can testify that it is nighmarishly bad code and poorly supported.

I would have been better off writing a new script from scratch, given how hard that piece of crap was to work with back then. If we're talking about the same thing, please tell me it's been completely rewritten since I last saw it.

ISP hosted email is strange to me

[Anonymous Coward]

Who wants ISP hosted email? You have to switch when you change ISPs.

Furthermore, who wants to go from google to yahoo's email system? Stab me with a fork in the eye!

Unable to fix problem?

[gravis777]

Sky is apparently unable to fix the problem — its best advice been to suggest users delete the old messages

I can tell you a fix to the problem in two seconds - Go back to Google. You are going to risk pissing off your customers with this, then they will discover that they have worse spam filtering, to save some money? Sounds like Sky needs to rethink a few things - that is a major issue!

Also, Google with Imap and all my mail clients that plug into it have no issues with this. Why is Yahoo having an issue with this? Sounds like someone at Yahoo is just lazy and doesn't want to be bothered to do a few hours worth of programming - meaning bad customer service they are offering to the ISP which turns around to screwing the end customer. I mean, really, that is inexcusable!

Gmail does not use folders

[rcharbon]

Folks, any poster who talks about "folders" in Gmail is not someone to listen to. Gmail doesn't use folders. Messages are tagged with labels and sorted and viewed using those labels. I was modding posts down, but there are just too many that show no understanding of Gmail, yet pretend to have worthwhile points (shock!).

Re:Gmail does not use folders

[wisnoskij]

Which might of flown if sublabels did not exist. But they do, so "label" is just Googles name for folder.

Re:Gmail does not use folders

[EmperorArthur]

Folders with symlinks maybe.

The ease of use is incredible though. I love being able to have an E-Mail with multiple labels, and see/edit all of those labels easily.

It's 2013, and most E-Mail clients still treat an E-Mail with multiple labels as multiple copies. I can't easily search, edit, or even delete my E-Mail with anything but the gmail web interface.

"Just hit delete"

[BenSchuarmer]

an oldie but a goodie

Google is not good at deleting stuff

[Anonymous Coward]

A similar thing happend to me too. I switched from google to another provider. I deleted my emails (also the trash) on my google account but after that old emails came in over IMAP which i did delete several weeks ago. Normally they shouldn't have existed on the google server anymore.
I deleted my whole account but i can still acces my google calender over the private ical link...

Who uses ISP Email?

[wisnoskij]

Email is so incredibly useful, but far less useful if you have to change email addresses with every ISP.

I have entered my email in far too many important sites, let alone given it to people/organizations, for me to recover from having a change of address.

The real story

[Anonymous Coward]

..is that, despite USERS EXPLICITLY DELETING THEM, Google still retained all the messages. You can't have a problem with re-delivering an e-mail you don't have.

Google. Keeping every piece of information they can possible learn about you forever, whether you like it or not.

Amazing Catcha Moment - I got "Expunge."

Re:The real story

[vakuona]

Google makes it clear they do not delete emails. And they also make it very clear how you can permanently delete emails. Click on "delete" in the webmail interface.

It would have been very easy for Yahoo to provide users with a "clean" inbox. All they had to do was to map the gmail inbox to the Yahoo inbox, create a Gmail "all mail" folder, and recreate all other folders/labels to match what Gmail was providing. The only downside would have been where users had tagged emails with multiple labels, and they would have created duplicates where Google only has a single email, but that's about it.

The solution was simple. Yahoo borked it.

Obligatory World of Goo reference

[OneAhead]

It's just these inquisitive little goo balls dropping the Undelete button into the red liquid so that World of Goo Corporation will be destroyed.
Everyone is crazy but me!

-the dotslashing Sign Painter.

Test run?

[Air-conditioned cowh]

Surely if they did a test run on a few mailboxes they would have discovered this.