Yahoo's Amazing Disappearing Mail Servers

Golygydd Max writes "A Techworld story reveals that the reason Yahoo email has delivery problems is that the company's mail servers mysteriously close once in a while." From the article: "According to trimMail's Email Battles site, which recently monitored 16 of the company's advertised email hosts 240 times over a half hour period, only 133 of its probes were answered. Many of the servers were closed and unavailable. Overall availability ranged from 25 percent to 75 percent over the admittedly short test period. The average availability was 55 percent, with the worst of the servers available only 7 percent of the time."

Comments

Is this caveat emptor day?

[WebHostingGuy]

I was under the impression that Yahoo! mail was free. Isn't the rule you get what you pay for? Seriously though, why would Yahoo put a ton of money into something which is not a revenue generator. Free email is so yesterday.

And if you don't like what I am saying you can reach me at:

server-never-works@yahoo.com

and really let me know how you feel about this. Well, you can at least reach 7 to 55% of the time.

Re:Is this caveat emptor day?

[Alex P Keaton in da]

Actually, yahoo has a significant business email program...

http://smallbusiness.yahoo.com/business_services /
They also have a lot of people who use Yahoo as a web host (paid) and get email @theirdoman.com....
So Yahoo mail is not always free.....

Re:Is this caveat emptor day?

[grub]

Still it's way better than hotmail.

Print out the mail you wish to send then smear feces on it. Tie it to the leg of a blind pigeon. Release the pigeon then taking pot-shots at it. The results are still better than hotmail.

Re:Is this caveat emptor day?

[putzin]

I actually dropped Yahoo as my webhost provider for this very reason. Too many bounced emails at random times. The outages were usually short, probably less than a couple of minutes, but would happen more than weekly. Ironically, it seemed that ETrade always was on the bounce side for some reason. Anyway, I asked them that if I was paying $20 a month for what other companies where charging $5 a month, then why was the service flaky. They asked me to tell them exact times and dates, and would investigate and get back to me. And just like that, I was a member of Avahost.

BTW, in an interesting side note. Yahoo registered my domain name for me, but curiously, lost the details on the name and info they did the registration with. It took me 2 days on the phone between network solutions and Yahoo to get a hold of my domain name. And they wanted to know why I wanted to leave?

As long as one of them is up...

[jeffmeden]

All you need is one server to have a functioning email system. Can anyone say 'MX priority list'? Of course maybe when each user is taking in hundreds of pieces of spam a day things might get *a little slow*...

Re:As long as one of them is up...

[caluml]

Plus your average mail server will hold your mail, and keep trying for 5 days before giving up. So, as long as 1 of Yahoo's servers is up for a decent period in a 5 day period, mail shouldn't be lost. Course, users might be annoyed with the delays. :)

Re:As long as one of them is up...

[arivanov]

Yahoo is a heavy user of greylisting. I would expect any of their servers to break connections, refuse connections and even deploy firewall rules including tarpitting to anything their greylisting algorithm finds annoying. In fact I am pretty sure about the first two, dunno about the last item. I am planning on doing it on the servers I run, I would be surprised if they do not have it already. After all they have a huge department that does nothing else but mail for themselves and their resale customers.

Move along people, simply the dot.bomb times are back. Yet another metric company making big noises about the fact that someone BIG looks bad on their metric. Reason is most likely that the metric is badly designed and does not take current large scale mail handling practices into account. We have all been there a few years ago when everybody and his dog was pushing metrics around just before the bubble collapsed. Move along, nothing new here.

Re:As long as one of them is up...

[coaxeus]

Indeed, as soon as I read the headline of the article I knew some dork that didn't understand greylisting was behind it. I've implemented greylisting with MDaemon (along with it's other 9 or so anti-spam layers) with great sucess, and if you use decent monitoring tools, everything works just fine.

Re:As long as one of them is up...

[Ilgaz]

I also hope that they know the size of the mail system they are playing with. That mail system must have its very own schemes, countermeasures.

To see size of Yahoo mail:
http://www.senderbase.org/

ps: Some on that list are spammer friendly ISPs (non managed etc), that is the purpose of that system. They own spamcop.net too.

Re:As long as one of them is up...

[gid13]

Yeah, greylisting works great, if the sending mail servers behave too. My employer (a small ISP) uses it, and every now and then a remote server has some weird retry patterns that fuck everything up. Try explaining THAT to a customer.

Shouldn't be responsible for delivery problems

[pen]

Servers being down some of the time shouldn't cause large-scale delivery problems. Remember, when e-mail protocols were being designed, a lot of these servers were down for a good portion of each day. E-mail protocols were designed to deliver e-mail in whatever window existed. If the receiving server is down, the sending server will try again for a good while before giving up.

Also, as someone else has already mentioned, there are the MX priority lists...

Maybe they were having a bad day...

[Osrin]

This is hardly a news story. Tomorrow: 100% of people on my front porch stub their toe at 10:19am on Friday morning. Porch declared a national disaster zone, FEMA are organizing evacuation flights.

Going back in time?

[xRelisH]

My guess is that these servers along with Babylon 4 went back in time to aid the Vorlons and Minbari in the first Shadow war.

Re:Going back in time?

[DangerSteel]

... and some of you people wonder why you can't pick up chicks...

Dumb spam protection?

[rudegeek]

The average availability was 55 percent

Maybe they fight spam in stupid way by letting fake SMTP servers eat thier e-mails? Normal SMTP server will delay deliver while spam-bot will gave up. They not follow RFC from what I know. ;-)

Re:Dumb spam protection?

[rhizome]

It's also a common practice to use a delayed SMTP response to thwart spammers. Maybe this person's ping script doesn't account for any delay and thus returns an error.

Failover and clustering

[Pranjal]

Duh! That is why they have multiple redundant servers. When one server goes down the email is routed to another server. Personally I have never encountered a situtation where an email sent to my yahoo account did not reach me. Yahoo Groups is a different story. Emails used to disappear frequently when they merged with eGroups. Things have stabilized now, but sometimes emails sent to a group do not reach all the participants and it is not a receiver issue but a routing issue on yahoo groups servers. Overall the uptime should be close to 100%. Nobody cares what is happening behind the scenes, whether one server has 100% or 7%.

This is meaningless

[mcrbids]

Email is DESIGNED to handle failures of this kind. Assuming Yahoo is running some form of clustering, it's quite reasonable to think that systems will start/stop as load fluctuates. Availability of individual servers is largely irrelevant - it's the availability of the system at large that matters.

no wonder

[the_Bionic_lemming]

And here I thought the reason for my spam reduction was because of their filters...

Re:no wonder

[arivanov]

You are amazingly pretty much on target.

95%+ of the SPAM reduction on Yahoo is due to the use of greylisting. Essentially the mail server simulates that it is unavailable to anyone it does not know as a well behaved relay. A well designed MTA will come back and deliver the mail later and the server will accept it. A SPAM zombie will skip to the next target.

A probe will be judged a zombie until proven opposite. A probe that does not try to deliver mail or do anything usefull will cause the SPAM ranking of the originating IP to go up until firewall shielding rules are deployed. From there on you cannot even reach the servers in question until the entry expires. In addition to that well behaved MTAs go to MXes in a predictable order. Anything hitting MXes in a different order is immediately considered a SPAMBOT and will cause the greylisting code to either set a "refuse" with a high timeout on it or (if the code is there) to raise firewall shields outright to tarpit any connections from the BOT. This also essentially disallows you to test any specific host for MX connectivity without testing the entire MX pool in correct order. If you do, you guarantee yourself a blacklist entry which will be generated automatically on the fly.

By the look of it this pretty much summarises what has happened here. Quite funny actually. It is indicative of the current crop of "security companies" and professionals. They claim understanding of security without knowing how things are done.

Down or defense?

[Richard_at_work]

240 times over a half hour period is a high rate of connections per server (8 per minute per server), especially for email servers, so is it possible that Yahoo!'s servers were simply defending themselves against a perceived threat? Connection throttling was the first thing that came to mind on reading the blurb.

Methodology?

[jtorkbob]

Trying out the test, it *appears* that the tool connects to port 25 and checks for a banner. There is no information about timeouts. I presume that 'Closed' means 'Timeout' or 'Connection Refused', but they don't tell us.

It seems that Yahoo! has blocked their server or something; the tool works fine on my domains but I can't get a result for yahoo.com.

Yahoo is actually doing things /better/ here

[twoflower]

Yahoo is actually doing the right thing here, from a technical point of view. The worst thing you can do is have an MX that accepts connections but is not responsive enough to actually handle accepting a message at that point -- it's far better to stop accepting SMTP connections when you detect you're at your maximum capacity.

This is because SMTP clients who fail to get a connection will immediately try the next MX. If they get a connection, but can't send the message, they may back off and try again later, delaying the message further.

Never had any problems

[MadMorf]

I've used Yahoo for %100 of my personal email for the last 6 years.

And, as an email admin, I also use it to test systems, both mine and others, and it always works...

If the servers are up and down all the time, I've never noticed it...

I'd file this under FUD...

And in other discoveries

[dmeranda]

And in other brilliant studies:

* Ethernet packets found to collide sometimes, resulting in worldwide communications silence.

* Some traffic lights found to periodically turn red almost 50% of the time; transportation system grinds to a halt.

* Study finds that if you call someone every 15 seconds and ask "can you hear me now?", unexplicably none of your calls will be answered, in addition to getting strange looks.

* Fast food restaurant closes one of its eight queues at the shift change; six people starve to death as a result.

Um...

[Guppy06]

"240 times over a half hour period, only 133 of its probes were answered."

Well, if a single host tried to ping me once every 7.5 seconds for a half-hour, I'd want my hardware to ignore a few of them, too.