Slashdot Mirror
Why Google's Gmail Phishing Warnings Give False Positives (vortex.com)
Vortex.com is one of the oldest domains on the internet -- one of the first 40 ever registered, writes Slashdot reader Lauren Weinstein. So why does Google sometimes block the email he sends?
Here's why. First, my message had the audacity to mention "Google Account" or "Google Accounts" in the subject and/or body of the message. And secondly, one of my mailing lists is "google-issues" -- so some (digest format) recipients received the email from "google-issues-request@vortex.com"... Apparently what we're dealing with here is a simplistic (and frankly, rather haphazard in this respect at least) string-matching algorithm that could have come right out of the early 1970s...! [A]t least in this case, it appears that Google is basically using the venerable old UNIX/Linux "grep" command or some equivalent, and in a rather slipshod way, too.
In addition, the article concludes, "I've never found a way to get Google to 'whitelist' well-behaved senders against these kinds of errors, so some users see these false phishing warnings repeatedly.
In addition, the article concludes, "I've never found a way to get Google to 'whitelist' well-behaved senders against these kinds of errors, so some users see these false phishing warnings repeatedly.
With the huge volumes of data that Google handles, it's probably hard to do any better.
AI style approaches can fail in quite unpredictable ways, and I think Google likely much prefers that too much is blocked than failing to find something obviously fishy but that gets through the algorithm for some obscure reason.
Sometimes simple approaches are the way to go. You're going to have false positives and false negatives no matter what, the question is how much and in what circumstances. And this particularly scenario is unlikely to be all that common.
Yup. +1
Whitelisting good behavior leads to making it very desirable to impersonate people who have good behavior.
Tweak your mailer so that it sends mail from gi-request instead of google-issues-request, and don't mention "Google Account". Granted, this sucks, but the Internet routes around brokenness, and that's what you need to do in a situation like this. Is that a sad thing? Yes, of course. If we had a mail architecture that was pull- rather than push-based, maybe we could have nice things, but until that magic day, the whole thing is bubble gum and bailing wire, and it's honestly not Google's fault that that's so.
As another example of brokenness, I often get mail that is marked spam because it went through a mailing list expander and the headers didn't get rewritten, so that it fails DKIM validation. Yes, we can all rail about how evil and awful DKIM is, but the bottom line is that if you don't want that to happen, you rewrite the headers. Again, a system that's pull-based rather than push-based would make this a lot better.
So one of the "first forty registered domains" can't behave poorly? Get SPF and DKIM going and these problems go away. Or just bitch about it to slashdot, whichever!
C'mon, Lauren, with the 10's of millions of spams that google catches every day, some things are going to get caught by the filter that shouldn't be. Even if the filter is 99.99 effective that means there will be 1000 false positives in there...and yours is one of them. Shit happens. Adjust and move on.
Apparently what we're dealing with here is a simplistic (and frankly, rather haphazard in this respect at least) string-matching algorithm that could have come right out of the early 1970s...! [A]t least in this case, it appears that Google is basically using the venerable old UNIX/Linux "grep" command or some equivalent, and in a rather slipshod way, too. is drawing a trend and a conclusion from one data point.
No one cares how old long the domain has existed. It could go from pristine track record to spewer of spam in a single successful hack ... or a simple change of ownership.
What spam wounded, anti-spam killed. Sending email has become a dark art. It is impossible to find out if an email will actually reach a recipient or if an MTA will silently drop it or if the email will be sorted away into the spam folder. If the email is suppressed in any way and you find out about it, there's often no way of determining what caused it to fall out of favor. I used to make fun of people who would call to announce that they sent you an email. If an email is important, I do that now.
This is also my experience.
I have my own domain, I run my own web-server, been doing so for years.
About 50% of the time my email goes to spam - even when I'm sending to the same people, who've replied to me int he past and presumably also removed my email from spam.
If I have to email someone at gmail, I use protonmail.
Having said that I try not to email gmail accounts, because of the lack of privacy.
You are a product, not a customer, and a product goes in whatever box the Goog wants it to.
GMail won't normally mark your email as spam/phishing if you've implemented basic mail server identification such as SPF (Sender Policy Framework) and DKIM (DomainKeys Identified Mail). This is well known, and I guarantee that if the author bothered to search for why their mail ends up flagged by GMail he would hit at least one of these two terms in the first few results.
A recursive sig
Can impart wisdom and truth
Call proc signature()
You mean like that vortex.com front page?
Beware of the Leopard.
The AI is so advanced it looks exactly like a human running grep.
With the huge volumes of data that Google handles, it's probably hard to do any better.
GMail may be "hard to do any better", but dealing with spam is complex and labyrinthine.
My client began having conversations with a vendor last week, and as a result GMail put *all* subsequent E-mails into my spam folder, including ones from my (whitelisted) client addressed to the vendor CC'ing me. I only found out by accident.
One might *expect* a quick, easily identified control that says "whitelist this person" or "whitelist this company", but there isn't. You have to go to "Settings->Settings->Filters and blocked addresses", none of which terms are "spam", so the casual user can't just scan headings for the term.
You can't, apparently, just refer to the spam and say "whitelist that person", you need to create a new filter. You can't, apparently, say "@example.com" as a wildcard for the business, you have to identify an actual sender by complete address.
And of course, you have to discover that you need to do this, because GMail doesn't give any warning. (Surprising, since every time I use GMail from a different location it sends me a warning E-mail. Every. Single. Time.)
I'm not even sure why everything went to spam in the first place - I had sent E-mails to both the vendor and the client, so they should have been in my "recently used" list.
GMail has a pretty cryptic interface, compared to some of the other mail readers I've used.
I always figured it was because some suits at Google were shorting Paypal stock or something.
One look at Vortex.com's front page, and I would quickly classify it as spam too...
A million false positives are much more preferable than one false negative.
Time is what keeps everything from happening all at once.
I’m certainly not going to change the names of my mailing lists or treat the term “Google Accounts” as somehow verboten!
Why not? That seems like an easy way to deal with the problem.
Setting the source and tone aside is crucial to any good analysis of a subject, so setting the Vortex source aside, I've noticed two things that seem to be relevant here. The first is that while Google is usually pretty good at blocking spam without false positives, it seems to be getting worse at that task, rather than better. Furthermore, it is notably bad detecting when an email is or is not a scam.
The second, more important, point comes at the end of the original note, and it's that Google as a whole has virtually no functional feedback mechanism for error correction. It is very hard to get any attention at all from staff, and even if you can manage it, my recent interactions simply yielded brain-dead responses and endless run around. This was with a botched Google Wallet payment where the firm sent confirmation saying "This money is now yours." but never delivered it. After many hours of investigation, there is still no real answer or means for progress.
Everybody gets things wrong sometimes, but Google seems has strayed a long way from its original "Don't be evil." motto. It now seems to do whatever benefits its bottom line, and costs the least without any regard for accuracy or allowing people to help it fix its mistakes.
Even for today's Slashdot, that was a remarkably low-insight comment to get an insightful moderation. There are constructive approaches to consider, but at this point, why bother?
Freedom = (Meaningful - Coerced) Choice != (Speech | Beer^2), and sad sock puppets' bad mods avail them naught.