The place to ask this question isn't here, it's on the "spam-l" mailing list, which arguably has the highest concentration of the world's most experienced anti-spam researchers and developers. Simple techniques for tackling this have been repeatedly covered there over a period of many years, and their behavior is well-understood and predictable, making them viable choices for production systems. So I would suggest that you subscribe to that list (via listserv@peach.ease.lsoft.com) and repeat your question there, along with some indication of your MTA environment.
Meanwhile, here is some general guidance. First, do not waste your money on commercial products -- they're expensive, poorly-maintained, and in many cases (e.g. Barracuda) actually make the spam problem worse via backscatter. (There are now several thousand Barracudas on a communally-maintained blacklist, making it obvious to everyone working in this field that Barracuda is completely incompetent.) Second, do invest your money and time in open-source solutions: it is easy for anyone who possesses baseline competence in mail to craft their own, superior spam handling system using postfix or sendmail or another open-source MTA, DNSBLs, RHSBLs, judicious configuration, and other tools such as rbldnsd, mimedefang, SpamAssassin, ClamAV, and so on. Third, a little googling will reveal near-cookbook procedures for combining these pieces of software together into a useful system; which cookbook procedure is appropriate for you depends on your environment -- which brings me to the fourth point, which is that you need to perform log analysis in order to understand your particular mix of spam/not-spam. Everyone's is different, which is why one-size-fits-all solutions usually fail. Only after you have some clue about the size and shape of your problem will you be able to determine which approach(es) are likely to minimize both false negatives (FN) and false positives (FP).
As an aside, one set of highly effective anti-spam tactics involves enforcing RFC requirements that have been in place for many years: for example, all mail servers must have rDNS; that rDNS must resolve to a host which in turn resolves back to the IP; the domain of the host must exist; the host must HELO as a valid FQDN or bracketed-quad IP; the envelope-sender's domain must exist; the host must not HELO as you; the host must wait for the SMTP greeting before HELO'ing; the host must handle a multi-line SMTP greeting; the MX records for the host must point to valid IP space; and so on. Enforcement of these requirements yields differing rates of spam control (which is again why log analysis is crucial) but has the very valuable property that it can be done at low computational and bandwidth cost. Substantial experience with these suggests that enabling them and augmenting them with a few DNSBLs (especially the Spamhaus Zen zone) is enough to deal with the overwhelming majority of the spam problem at most sites, reducing what's left to a much smaller issue to be dealt with.
In re: "Unfortunately, monitoring lists and networks is about the only current alternative."
There are many alternatives to this, starting with: "Recognize that operating systems which are readily compromised by malware are broken and not acceptable for use."
If you choose to use an OS which is so intrinsically weak that it cannot survive exposure to the (unfirewalled) Internet without anti-virus, anti-spyware, anti-adware, etc., then you have chosen poorly, and no subsequent choice you make will compensate for that.
A followup point would be "Understand that it is not possible to 'clean' a malware-contaminated system. The only acceptable course of action is to wipe to bare metal, reinstall, and restore from backups." While it might have been partially true in
a limited sense that some malware could be removed by anti-whatever products, that's certainly not the case now: it's much more likely that malware will evade detection and removal. Of course, it serves the purposes of both anti-whatever companies and lazy system administrators to continue propagating this fiction, because if they actually had to scrub and rebuild systems as often as they're infested, they might have to face some hard choices that they'd rather not.
And an excellent set of auxiliary points may be found in Marcus Ranum's The Six Dumbest Ideas in Computer Security, where he enumerates the most egregious (and sadly, most common) mistakes made by nearly everyone, including supposed "experts" with strings of meaningless, worthless certifications after their names.
So there are plenty of alternatives -- but choosing them and implementing them requires vision and insight, two qualities badly lacking in many in the profession.
Marcus Ranum's "The Six Dumbest Ideas in Computer Security" rant/essay neatly identified the top culprit a few years back. The mistakes he outlined continue to be made on a daily basis by nearly everyone working in the field -- and most of those people compound those errors by layering on more mistakes. (Example: "Well, yes, the firewall is default-permit outbound, but that's okay because we have an IDS.") This approach inevitably fails, yet those practicing it profess surprise every time it does -- especially if they happen to be standing in front of a press conference announcing the latest data loss incident.
We will not make any headway on this, as a profession, until we stop making rudimentary mistakes such as the ones Ranum has identified, along with a few others that are worthy additions to that list. No initiatives, no certifications, no appliances, nothing will change that -- because none of those change the attitudes of the people who are building systems and networks. Until those people manage to step back from irrelevant
details like "which iframe exploit is current today?" and look at larger questions like "why are iframe exploits even possible?" or "why are browser exploits even possible?", then they will continue to waste effort "solving" the wrong problems.
Sadly, after observing this situation close up for many, many years, I've concluded that some, possible many, people will never get that far. They simply Do Not Get It, and despite essays like Ranum's or books like this one or anything else, they're not going to get it. And they will continue to fail, and so the systems/networks they've built will continue to fail. I'd say that will make for a bleak future, but -- look around! -- we're living in a bleak present.
here is absolutely no rationale reason for a COMIC STRIP SITE, for crying out loud, to try to use Flash...other than some inexperienced, junior person with little grasp of the InterTubes and less knowledge decided it was Way Cool and that it must be forcibly inflicted on the masses. This is clearly one of the dumbest things Scott Adams has ever done.
I would be happy to tell him that on his own site, but even after fabricating my location and my birthdate (and why does a COMIC STRIP SITE need to know that?), it still won't let me register so that I can rip him in an up-close and personal kind of way.
This is truly miserable. "A year and a half"? That's a year and half that someone could have been masturbating continuously, with far superior results.
This should be printed out in 72-point type and stapled to the forehead of any mail system administrator who hasn't already made their operation do exactly this. There are no excuses: numerous techniques for accomplishing this, even in multiple-server, multiple-tier environments have been well known for a decade.
Those who fail are likely to find themselves on numerous blacklists -- correctly listed as spammers.
GoDaddy is the single largest registrar of spammers, phishers, and the like. On the surface, that might sound odd, given that GoDaddy has published policies that say they'll take action, but the reality is that those are propaganda, no better. GoDaddy's enforcement of its own policies against abusers has been laughable: it's pretty obvious to everyone that they only do so with reluctance and in the face of bad PR. (See Usenet's news.admin.net-abuse.email for many discussions on this.)
This really isn't surprising, though: spammers and phishers buy domains by the hundreds, if not thousands, which makes them excellent customers. And if you're GoDaddy, you need that income (among other reasons) to fund your offensively sexist commercials.
How does this tie in? It's all about profits. Profits for GoDaddy are maximized by selling as many domains as possible and then holding them for ransom.
Given how weak and slow ICANN has been, this has been a viable strategy for a number of years; it remains to be seen if something meaningful will actually happen in this case, or whether GoDaddy will just continue cementing its reputation as one of the scummiest registrars out there.
Clarification: I meant that remark to apply only to various anti-spam vendors,
not to doctors or anyone else.
I'd like to add that there's nothing wrong with symptomatic treatment as
an adjunct to root-cause treatment; it's generally a good idea unless it gets
in the way (say by masking the problem, making it tougher to pinpoint). However, I think there is something wrong with only
doing symptomatic treatment when root-cause treatment techniques are well-known
and readily available. And that's what I think a number of those (anti-spam) vendors
are doing: I think they're letting others (with far less resources) do the heavy
lifting.
As an example of what I mean, check the docket of anti-spam cases over the past few years -- and see who the plaintiffs are. And aren't.
Two-part answer: first, there are many, MANY of those boxes out there
that have NDR enabled. They're all spamming. It's unlikely that anyone has
a complete list, but the two blacklists I cited in the original article
have quite a few of them. Latest one seen locally: barracuda.corning-cc.edu.
Second, Barracudas have been observed sending backscatter even when that switch
turned off. Not often, to be sure, but it's been noted -- and reported to Barracuda.
I've yet to see any response and/or fix from them.
Nonsense. It is perfectly RFC-compliant to issue an SMTP refusal
during the conversation with the SMTP client. This leaves responsibility
for the returning an NDR with that client, thus avoiding the generation
of backscatter/outscatter. Moreover, this has been a very well-known
best practice for the better part of a decade -- even casual and novice mail server
operators are well aware of it. Surely anyone claiming even minimal
expertise in the anti-spam area should not only be well aware of it, but
have long since done the necessary coding/configuration work to eliminate
the problem entirely.
I suggest using the search engine of your choice to search for "backscatter spam" and reading what you find. Among other things, you'll notice that since backscatter is spam, that sites emitting it are eligible for several blacklists in addition to the two I already cited. You'll also notice that there has been particular emphasis placed on eliminating it by numerous participants in the anti-spam community -- a community, I might add, that Barracuda is conspicuously absent from.
Note that Barracuda's products are notorious for generating spam.
Barracuda's engineers were informed of the problem years ago, provided with
a fix -- and stubbornly refused to address the situation. It's no wonder
that there are now thousands of Barracuda installations on various
blacklists. (Two examples:
Backscatterers
and
Backscatterer.org)
Barracuda doesn't seem to care as long as they make money.
A secondary point is that Barracuda's products are NOT open-source.
Oh, they're built almost entirely on open-source (an open-source operating
system, an open-source mail server, an open-source anti-spam scanner,
an open-source anti-virus scanner, etc.) but they're not open-source.
Essentially what they've done is take all of that open-source code,
slap a web front-end on it for the point-and-drool crowd, and then
sell it. They're not in this to help out the Internet or stop spam
or anything else admirable: they're in this to make money, and they're
perfectly willing (see first point) to make the spam problem worse if
it increases their profits.
They're not alone in that -- there are others out there who are in
business to profit from our collective misery. An excellent way of
spotting such companies is to ask the question: "What would happen
if the problem they claim to address was actually solved?"
If the answer to that question is "they would go out of business",
then their motivation for always treating the symptoms and never treating
the underlying cause will become clear.
I concur, ScrewMaster. Those same linkfarms are very often the ultimate
target of massive spam runs, which is why various attempts to identify
recently-registered domains and deny all mail from them until they're
N days old (N > 5, with various experiments choosing other values) have been made.
In my own research, I've frequently noticed that spam source or spam target
domains often have been de-registered by the time I run a WHOIS lookup on them.
Just about as often, I've noticed that their A records point to known linkfarms.
So I see this as entirely good thing for everyone but the scammers, and I applaud
Google for taking this step.
Now if we could just get ICANN to forbid this idiotic practice entirely,
we could make even more progress.
That (use of data harvested from ssh attacks) is entirely possible. Some of those attacks had to successful against some hosts.
So maybe one possible line of investigation would be to see if any hosts which
defended against ssh dictionary attacks (say, by throttling back or denying connections
from hosts that made too many ssh tries) were compromised. (I suppose it'll also be
necessary to assess the strength of their root passwords; sufficiently weak ones
might not require a concerted ssh attack to be compromised.)
Sure, this could be the wrong line of reasoning -- but given that we've all seen the ssh attacks you refer to, it's probably worth investigating.
To figure out what the compromise vector is, it's probably going
to be necessary to figure out what the compromised servers have in
common -- and how that differs from uncompromised servers. (Keeping
in mind that currently-uncompromised servers may have the same vulnerability,
and that attackers or their software just may not have gotten to them yet.)
I'd suggest enumerating factors such as OS, OS version, remote access methods
(ssh, ftp, etc.), Apache versions, Apache modules, add-ons like CPanel,
network/ASN, and so on -- anything could be a culprit at this point.
And that includes things that have nothing to do with Linux or Apache:
for example, it's possible that the attackers acquired root passwords
by infecting Windows systems used by administrators -- then just waited
for them to initiate ssh sessions to their servers. It'd probably be
best to leave all possibilities open and consider them equally likely
until evidence starts accumulating in favor of/against them. (In re-reading
that last statement, I suppose it sounds a bit trite. I'm just trying
to discourage premature conclusions that anything is at fault until
somebody can produce evidence to support saying so.)
...or so I predict. The maximum fines are but a tiny fraction
of his monthly income. The jail terms aren't a threat given overcrowded
prisons, the focus on the farcial War on Drugs (TM), the classification
of this as a "white-collar" crime, and the technical illiteracy of both juries
and judges when it comes to spam. Not to mention that Ralsky is easily
smart enough to have planned for this and no doubt has plenty of
high-priced legal talent at his disposal -- plus, I wouldn't doubt, a carefully
maintained stash of information on other spammers that he can use to
plea-bargain his way out of much of this.
All that remains is a book deal and eventual appearances on cable
news networks as "a spam expert". Oh, and he might have to "retire"
from spamming in the same way that Spamford "retired" -- by moving
on to junk faxing, spyware and typosquatting.
Every piece of significant Internet technology designed,
developed and deployed over the past 25-30 years has been
open-source. Offhand, I could list everything related to Usenet
and NNTP, Apache, perl, gopher, python, PGP, BIND, Firefox,
archie, AFS, NFS, X, LDAP, MIME, majordomo and mailman,
ruby, RCS, CVS, subversion, BSD Unix, Linux, sendmail, postfix,
courier, exim, P2P and associated tools, IRC, a bunch of ASF
projects, etc., etc., etc. These are the building blocks of what
most people perceive as the contemporary Internet -- and I'd
say that creating that has involved some serious innovation.
The biggest obstacle to innovation isn't open-source: it's
software patents and the associated legal thicket that's being
constructed to strangle innovation and thereby preserve the
profits of the incumbents. I note with interest the the overwhelming
majority of those engaging in this anti-innovation practice are
vendors of closed-source software -- who are thereby admitting
that they can't compete on merit, and so have to resort to
unethical legal maneuvers to quash their competition. Oh, and
the occasional open-source-is-bad propaganda piece.
Obvious point first: knowing where someone is doesn't tell you
what they're doing. They could be watching TV in their basement,
or they could be watching TV while getting a blood transfusion.
And so on. (And the training bags? Easy enough to have someone else
transport them around while the owner is elsewhere.)
And using such a technique could open up vulnerabilities, as in "Hmmm....
Johann is not in his assigned room in the team dorm at the Pan-Am Games, so this
would be a good time to plant the syringes there." I'm sure some creative
thought will reveal other possibilities.
More generally though -- and I speak as someone who's competed
at the national level and served on my sport's national board of directors --
everyone (including the IOC) knows that there's no way to stop anyone
from doping if they're sufficiently careful and sufficiently clever.
The tests just can't keep up with newly-developed methods, and the
boundaries between legitimate medications (e.g., anti-sting kits for
those who risk anaphylactic shock if stung by an insect)
and performance-enhancing drugs are often blurred.
The best clues are often available to coaches and other team staff,
who have detailed performance data on all athletes and should be
able to spot anomalies. However, they don't have much motivation
to share these observations -- with anyone. Which is why one of
the things that needs to happen is that the governing bodies for
each sport need to emphasize doping detection by coaches as much
(or possibly more) as they do results production...and that means "put it in their contracts".
And those of us who watch sports need to do something as well:
we need to lose our winning-is-everything, second-place-means-losing mentality. (That includes the media,
by the way.) That attitude fuels a number of unpleasant trends in
sports, not just doping. We need to keep in mind that the reason
athletes go to events like the Olympics is not to win -- but to participate.
When we show the same respect and admiration for the effort of the
last-place finisher in the 10K, or the basketball team that loses by 50,
or the skier who falls, as we do for the gold medal winners, then we'll
have done our part to remove part of the motivation/temptation that
drives doping.
And now we see that their official corporate policy is to issue threats against comedy troupes and bloggers.
Happily, there is a quick and easy fix for this, known as a "blacklist",
so momentarily I'll be adding bestbuy.com, bestbuyinkfinder.com,
and myrewardzone.com to it.
1. Please enumerate all of your mail system operational experience,
including your anti-spam experience and your participation in the
Internet's anti-spam mailing lists/newsgroups/etc.
2. If the algorithm isn't published for unrestricted peer review, then
it will be classified along with supposed secret cryptographic algorithms -- that is, snake-oil promoted by liars. Put up or shut up.
3. Brightmail is hardly the "de facto standard"; they have demonstrated
only middling competence, and their best work is properly classified as "amateurish". I would expect any junior mail system administrator to be
able to build an anti-spam system that outperforms theirs.
4. You have failed to demonstrate even a rudimentary grasp of
the spam problem on a practical level. While your supposed "solution"
might work in a restricted environment (and the same could be said
for many other proposed "solutions") it has no chance in the real world,
because you haven't accounted for a number of real-world issues --
including a huge number of compromised systems.
This approach is quite similar to that taken by the DCC.
Quoting from its home page: "The DCC is based on an idea of Paul Vixie and on fuzzy body matching to reject spam on a corporate firewall operated by Vernon Schryver starting in 1997. The DCC was designed and written at Rhyolite Software starting in 2000. It has been used in production since the winter of 2000/2001."
As is often the case, those who are new to the spam problem frequently
believe they are inventing something new, when it's most likely that they're
not -- the remaining question being whether it's workable or long-since
abandoned as (mostly) useless. Reputation systems like this are presently
somewhat useful, but it's worth noting that should they become widely
used, spammers might then find it worth the effort to exercise the
control they have over the 100M+ hijacked systems out there and
thereby poison the reputation system. While this could be done by
generating appropriate traffic, and that'd be moderately disruptive,
exerting control over a sufficient number of systems participating
in reputation assessment would be worse.
This therefore joins a long parade of specious claims (e.g., Spam as a technical problem is solved by SPF") made to announce the mythical
"solution" to spam, which of course does not exist. Does it have possible
value in mitigation? It would appear so, based on the track record of similar work (see above). Is it The Answer? Not even remotely close.
1. As those of us who have been fighting spammers for decades
have seen -- many times -- attempting to estimate global spam
trends based on single measurement points (no matter how large)
doesn't work. Moreover, attempting to estimate those same trends
based on brief time periods doesn't work either, since there are often
seasonal (or longer) fluctuations. So while Google's measurements
may be locally and temporarily valid, they have no meaning and no
value vis-a-vis the entire Internet.
2. Spammers, like any enterprise seeking long-term success,
have diversified. They're into adware, spyware, phishing, domain
typosquatting, domain kiting, drive-by downloads, and every other
possible line of "business" available to them. If any one spam operation
appears to be momentarily declining in output, it's almost certainly
because their focus has shifted to another form of abuse.
3. Address harvesting by spammers is now extremely
effective, thanks in large part to an estimated 100 million compromised
systems (and that number may be low). Any email address that's
actually used should be presumed to be in enemy hands shortly;
attempts at obfuscation are futile. It's a best practice to make
this assumption and plain defenses accordingly.
4. Google itself has been increasingly implicated in spam and
spam support activities -- the number of spammer dropboxes there
is increasing, as is the number of spammer web sites hosted by them.
And Google's abuse desk is non-responsive to even the most
well-documented complaints (that is, though where a victim has
generously done Google's homework for them and donated the
results). This is troubling.
It makes my short list, along with ZImbra, Scalix, and Openchange
(so far).
The nice things about JES are (a) it's rock-solid (b) it works well
with many mail clients, even horribly broken ones like Outlook
(c) while it doesn't have every possible calendar feature in the
world, it has all of the ones that people actually use (d) it scales
amazingly well -- it's really no problem to get it to support millions
of users (e) because it's been around for a while (including a prior
incarnation as a Netscape product) there's a pretty solid support
community for it (in addition to Sun) (f) it's flexible enough to support
integration with other products.
The bad things about JES are (a) the install is complicated, even
if you're very accustomed to complex installs (b) the documentation,
like much of Sun's documentation, is poorly written, verbose, uses
opaque terminology, and lacks cohesion (c) the log files are inscrutable
(d) it's somewhat bloated (somebody needs to trim all the legacy
code out of it) (e) it's overkill for anyone who just needs a mail server
(i.e., no calendaring).
But...given that you get mail, calendaring, LDAP, all rolled up in one
package -- it's at least worth looking at. I'm aware of any number
of places that have migrated from Exchange to JES, so at least
their requirements were met.
The software to set up a decent mail/calendar is
open-source, free, scalable, well-supported by the community,
and will run on commodity hardware. The same can be
said for routing and firewall infrastructure (OpenBSD,
OpenBGP, pf, etc.) as well as for inbound/outbound
email defenses (postfix or sendmail, Clamav, a judicious
selection of DNSBLs and RHSBLs, and possibly SpamAssassin).
All of these are mature products that have been deployed
(and extensively documented) -- it's really not that difficult
to architect a solution that uses them and make it a reality.
(I know, I've done it. Multiple times.) Better yet, using
these products alleviates the need to rely on vendor
support -- expensive, usually worthless, and often less
clueful than customers.
The bottom line is that I would expect any mid-level
mail systems admin (5 years experience, and no,
Exchange does not count) to be able to spec out,
architect, and build infrastructure like this -- with
some possible help on the networking aspects from
a firewall/router jockey. It's just not that hard. Sheesh,
back when I worked in academia (at a large midwestern
university in an athletic conference which can't count)
these sorts of problems were routinely dispensed with
so that we could concentrate on the hard stuff.
Oh, and one note about outsourcing: "class breach".
Given that Microsoft has already (and repeatedly)
proven that it cannot operate a mail server in anything
remotely approaching a professional manner, and that it
has only a dim grasp of security (at best), anyone outsourcing
email to them is just begging for subsequent litigation from
faculty/staff/students.
Qmail -- whatever its security merits, and it does have
some -- is not suitable for use on the public Internet
because it fails to comply with not only de jure standards
(RFCs) but de facto standards (best practices). The author
has refused to correct these defects -- which is certainly
his prerogative as an author, but has as a byproduct serious
operational impact on not only users of the package, but
other mail server operators who communicate with those run
by users of the package.
It's my professional recommendation (based on nearly
three decades of experience running mail systems) that
qmail be avoided in favor of superior packages such as
sendmail, postfix, and exim. (Although the latter,
unfortunately, appears to be making increased use of
an abusive, spam-supporting feature known as "callbacks".)
These packages are actively maintained and their authors
have made, and continue to make, efforts to make them
standards-compliant and well-behaved (despite the increasing
stress placed on them by all forms of mail-related abuse).
As an aside, readers interested in the history of qmail
should query a Usenet archive for "a tribute to the programming
style of Eric Allman".
Sometimes I find myself idly wondering how many miserable
failures of opt-out proposals will be necessary before people
get a clue that opt-in offers the only possible way to success.
Then I snap out of it and remind myself that of course some
people have a clue, and that's precisely why they continue to
put these proposals out (or to enthusiastically back them): doing
so serves their purposes nicely. It allows them to proudly say
that "they've taken the lead in protecting privacy" while of course
they're doing everything they possibly can to do the opposite.
(They do this, of course, because they're well aware that few people
would opt-in to have telemarketers bother them, or to have spammers
clog their mailboxes, or to have their personal data collected.)
This situation is unlikely to change in the forseeable future.
Just as it's given us ineffective anti-telemarketing measures,
just as it's given us ineffective anti-spam measures, the outcome
of this process will inevitably give us ineffective anti-privacy-invasion measures.
Which is why it's probably best to just ignore this nonsense and instead
use technological means to either deny data to invaders or feed them bogus data.
I was saying that competent "spammers and other miscreants" have far more sophisticated, self-organizing C&C methods already deployed. Storm's authors appear to have eschewed those, at least for the moment -- and arguably, they may have made a tactical decision to do so for reasons that aren't clear to any of us yet.
Please note as well that if the descriptions of traffic disruption caused by Comcast that we've seen so far are accurate, they're insufficient to stop communication channels using UDP, spread-spectrum techniques, burst transmissions, and so on. Moreover, it's not clear (at least not to me) whether Comcast has deployed this at the borders of their network -- or internally as well. If it's only the former, then clearly P2P communications within Comcast's own network (currently populated with millions of 0wned systems) aren't affected at all.
Slashdot Mirror
Meanwhile, here is some general guidance. First, do not waste your money on commercial products -- they're expensive, poorly-maintained, and in many cases (e.g. Barracuda) actually make the spam problem worse via backscatter. (There are now several thousand Barracudas on a communally-maintained blacklist, making it obvious to everyone working in this field that Barracuda is completely incompetent.) Second, do invest your money and time in open-source solutions: it is easy for anyone who possesses baseline competence in mail to craft their own, superior spam handling system using postfix or sendmail or another open-source MTA, DNSBLs, RHSBLs, judicious configuration, and other tools such as rbldnsd, mimedefang, SpamAssassin, ClamAV, and so on. Third, a little googling will reveal near-cookbook procedures for combining these pieces of software together into a useful system; which cookbook procedure is appropriate for you depends on your environment -- which brings me to the fourth point, which is that you need to perform log analysis in order to understand your particular mix of spam/not-spam. Everyone's is different, which is why one-size-fits-all solutions usually fail. Only after you have some clue about the size and shape of your problem will you be able to determine which approach(es) are likely to minimize both false negatives (FN) and false positives (FP).
As an aside, one set of highly effective anti-spam tactics involves enforcing RFC requirements that have been in place for many years: for example, all mail servers must have rDNS; that rDNS must resolve to a host which in turn resolves back to the IP; the domain of the host must exist; the host must HELO as a valid FQDN or bracketed-quad IP; the envelope-sender's domain must exist; the host must not HELO as you; the host must wait for the SMTP greeting before HELO'ing; the host must handle a multi-line SMTP greeting; the MX records for the host must point to valid IP space; and so on. Enforcement of these requirements yields differing rates of spam control (which is again why log analysis is crucial) but has the very valuable property that it can be done at low computational and bandwidth cost. Substantial experience with these suggests that enabling them and augmenting them with a few DNSBLs (especially the Spamhaus Zen zone) is enough to deal with the overwhelming majority of the spam problem at most sites, reducing what's left to a much smaller issue to be dealt with.
There are many alternatives to this, starting with: "Recognize that operating systems which are readily compromised by malware are broken and not acceptable for use." If you choose to use an OS which is so intrinsically weak that it cannot survive exposure to the (unfirewalled) Internet without anti-virus, anti-spyware, anti-adware, etc., then you have chosen poorly, and no subsequent choice you make will compensate for that.
A followup point would be "Understand that it is not possible to 'clean' a malware-contaminated system. The only acceptable course of action is to wipe to bare metal, reinstall, and restore from backups." While it might have been partially true in a limited sense that some malware could be removed by anti-whatever products, that's certainly not the case now: it's much more likely that malware will evade detection and removal. Of course, it serves the purposes of both anti-whatever companies and lazy system administrators to continue propagating this fiction, because if they actually had to scrub and rebuild systems as often as they're infested, they might have to face some hard choices that they'd rather not.
And an excellent set of auxiliary points may be found in Marcus Ranum's The Six Dumbest Ideas in Computer Security, where he enumerates the most egregious (and sadly, most common) mistakes made by nearly everyone, including supposed "experts" with strings of meaningless, worthless certifications after their names.
So there are plenty of alternatives -- but choosing them and implementing them requires vision and insight, two qualities badly lacking in many in the profession.
We will not make any headway on this, as a profession, until we stop making rudimentary mistakes such as the ones Ranum has identified, along with a few others that are worthy additions to that list. No initiatives, no certifications, no appliances, nothing will change that -- because none of those change the attitudes of the people who are building systems and networks. Until those people manage to step back from irrelevant details like "which iframe exploit is current today?" and look at larger questions like "why are iframe exploits even possible?" or "why are browser exploits even possible?", then they will continue to waste effort "solving" the wrong problems.
Sadly, after observing this situation close up for many, many years, I've concluded that some, possible many, people will never get that far. They simply Do Not Get It, and despite essays like Ranum's or books like this one or anything else, they're not going to get it. And they will continue to fail, and so the systems/networks they've built will continue to fail. I'd say that will make for a bleak future, but -- look around! -- we're living in a bleak present.
I would be happy to tell him that on his own site, but even after fabricating my location and my birthdate (and why does a COMIC STRIP SITE need to know that?), it still won't let me register so that I can rip him in an up-close and personal kind of way.
This is truly miserable. "A year and a half"? That's a year and half that someone could have been masturbating continuously, with far superior results.
Those who fail are likely to find themselves on numerous blacklists -- correctly listed as spammers.
This really isn't surprising, though: spammers and phishers buy domains by the hundreds, if not thousands, which makes them excellent customers. And if you're GoDaddy, you need that income (among other reasons) to fund your offensively sexist commercials.
How does this tie in? It's all about profits. Profits for GoDaddy are maximized by selling as many domains as possible and then holding them for ransom. Given how weak and slow ICANN has been, this has been a viable strategy for a number of years; it remains to be seen if something meaningful will actually happen in this case, or whether GoDaddy will just continue cementing its reputation as one of the scummiest registrars out there.
Clarification: I meant that remark to apply only to various anti-spam vendors, not to doctors or anyone else.
I'd like to add that there's nothing wrong with symptomatic treatment as an adjunct to root-cause treatment; it's generally a good idea unless it gets in the way (say by masking the problem, making it tougher to pinpoint). However, I think there is something wrong with only doing symptomatic treatment when root-cause treatment techniques are well-known and readily available. And that's what I think a number of those (anti-spam) vendors are doing: I think they're letting others (with far less resources) do the heavy lifting.
As an example of what I mean, check the docket of anti-spam cases over the past few years -- and see who the plaintiffs are. And aren't.
Two-part answer: first, there are many, MANY of those boxes out there that have NDR enabled. They're all spamming. It's unlikely that anyone has a complete list, but the two blacklists I cited in the original article have quite a few of them. Latest one seen locally: barracuda.corning-cc.edu. Second, Barracudas have been observed sending backscatter even when that switch turned off. Not often, to be sure, but it's been noted -- and reported to Barracuda. I've yet to see any response and/or fix from them.
Nonsense. It is perfectly RFC-compliant to issue an SMTP refusal during the conversation with the SMTP client. This leaves responsibility for the returning an NDR with that client, thus avoiding the generation of backscatter/outscatter. Moreover, this has been a very well-known best practice for the better part of a decade -- even casual and novice mail server operators are well aware of it. Surely anyone claiming even minimal expertise in the anti-spam area should not only be well aware of it, but have long since done the necessary coding/configuration work to eliminate the problem entirely.
I suggest using the search engine of your choice to search for "backscatter spam" and reading what you find. Among other things, you'll notice that since backscatter is spam, that sites emitting it are eligible for several blacklists in addition to the two I already cited. You'll also notice that there has been particular emphasis placed on eliminating it by numerous participants in the anti-spam community -- a community, I might add, that Barracuda is conspicuously absent from.
Note that Barracuda's products are notorious for generating spam. Barracuda's engineers were informed of the problem years ago, provided with a fix -- and stubbornly refused to address the situation. It's no wonder that there are now thousands of Barracuda installations on various blacklists. (Two examples: Backscatterers and Backscatterer.org) Barracuda doesn't seem to care as long as they make money.
A secondary point is that Barracuda's products are NOT open-source. Oh, they're built almost entirely on open-source (an open-source operating system, an open-source mail server, an open-source anti-spam scanner, an open-source anti-virus scanner, etc.) but they're not open-source. Essentially what they've done is take all of that open-source code, slap a web front-end on it for the point-and-drool crowd, and then sell it. They're not in this to help out the Internet or stop spam or anything else admirable: they're in this to make money, and they're perfectly willing (see first point) to make the spam problem worse if it increases their profits.
They're not alone in that -- there are others out there who are in business to profit from our collective misery. An excellent way of spotting such companies is to ask the question: "What would happen if the problem they claim to address was actually solved?" If the answer to that question is "they would go out of business", then their motivation for always treating the symptoms and never treating the underlying cause will become clear.
I concur, ScrewMaster. Those same linkfarms are very often the ultimate target of massive spam runs, which is why various attempts to identify recently-registered domains and deny all mail from them until they're N days old (N > 5, with various experiments choosing other values) have been made.
In my own research, I've frequently noticed that spam source or spam target domains often have been de-registered by the time I run a WHOIS lookup on them. Just about as often, I've noticed that their A records point to known linkfarms. So I see this as entirely good thing for everyone but the scammers, and I applaud Google for taking this step.
Now if we could just get ICANN to forbid this idiotic practice entirely, we could make even more progress.
That (use of data harvested from ssh attacks) is entirely possible. Some of those attacks had to successful against some hosts.
So maybe one possible line of investigation would be to see if any hosts which defended against ssh dictionary attacks (say, by throttling back or denying connections from hosts that made too many ssh tries) were compromised. (I suppose it'll also be necessary to assess the strength of their root passwords; sufficiently weak ones might not require a concerted ssh attack to be compromised.)
Sure, this could be the wrong line of reasoning -- but given that we've all seen the ssh attacks you refer to, it's probably worth investigating.
To figure out what the compromise vector is, it's probably going to be necessary to figure out what the compromised servers have in common -- and how that differs from uncompromised servers. (Keeping in mind that currently-uncompromised servers may have the same vulnerability, and that attackers or their software just may not have gotten to them yet.)
I'd suggest enumerating factors such as OS, OS version, remote access methods (ssh, ftp, etc.), Apache versions, Apache modules, add-ons like CPanel, network/ASN, and so on -- anything could be a culprit at this point.
And that includes things that have nothing to do with Linux or Apache: for example, it's possible that the attackers acquired root passwords by infecting Windows systems used by administrators -- then just waited for them to initiate ssh sessions to their servers. It'd probably be best to leave all possibilities open and consider them equally likely until evidence starts accumulating in favor of/against them. (In re-reading that last statement, I suppose it sounds a bit trite. I'm just trying to discourage premature conclusions that anything is at fault until somebody can produce evidence to support saying so.)
...or so I predict. The maximum fines are but a tiny fraction of his monthly income. The jail terms aren't a threat given overcrowded prisons, the focus on the farcial War on Drugs (TM), the classification of this as a "white-collar" crime, and the technical illiteracy of both juries and judges when it comes to spam. Not to mention that Ralsky is easily smart enough to have planned for this and no doubt has plenty of high-priced legal talent at his disposal -- plus, I wouldn't doubt, a carefully maintained stash of information on other spammers that he can use to plea-bargain his way out of much of this.
All that remains is a book deal and eventual appearances on cable news networks as "a spam expert". Oh, and he might have to "retire" from spamming in the same way that Spamford "retired" -- by moving on to junk faxing, spyware and typosquatting.
Every piece of significant Internet technology designed, developed and deployed over the past 25-30 years has been open-source. Offhand, I could list everything related to Usenet and NNTP, Apache, perl, gopher, python, PGP, BIND, Firefox, archie, AFS, NFS, X, LDAP, MIME, majordomo and mailman, ruby, RCS, CVS, subversion, BSD Unix, Linux, sendmail, postfix, courier, exim, P2P and associated tools, IRC, a bunch of ASF projects, etc., etc., etc. These are the building blocks of what most people perceive as the contemporary Internet -- and I'd say that creating that has involved some serious innovation.
The biggest obstacle to innovation isn't open-source: it's software patents and the associated legal thicket that's being constructed to strangle innovation and thereby preserve the profits of the incumbents. I note with interest the the overwhelming majority of those engaging in this anti-innovation practice are vendors of closed-source software -- who are thereby admitting that they can't compete on merit, and so have to resort to unethical legal maneuvers to quash their competition. Oh, and the occasional open-source-is-bad propaganda piece.
Obvious point first: knowing where someone is doesn't tell you what they're doing. They could be watching TV in their basement, or they could be watching TV while getting a blood transfusion. And so on. (And the training bags? Easy enough to have someone else transport them around while the owner is elsewhere.)
And using such a technique could open up vulnerabilities, as in "Hmmm.... Johann is not in his assigned room in the team dorm at the Pan-Am Games, so this would be a good time to plant the syringes there." I'm sure some creative thought will reveal other possibilities.
More generally though -- and I speak as someone who's competed at the national level and served on my sport's national board of directors -- everyone (including the IOC) knows that there's no way to stop anyone from doping if they're sufficiently careful and sufficiently clever. The tests just can't keep up with newly-developed methods, and the boundaries between legitimate medications (e.g., anti-sting kits for those who risk anaphylactic shock if stung by an insect) and performance-enhancing drugs are often blurred.
The best clues are often available to coaches and other team staff, who have detailed performance data on all athletes and should be able to spot anomalies. However, they don't have much motivation to share these observations -- with anyone. Which is why one of the things that needs to happen is that the governing bodies for each sport need to emphasize doping detection by coaches as much (or possibly more) as they do results production...and that means "put it in their contracts".
And those of us who watch sports need to do something as well: we need to lose our winning-is-everything, second-place-means-losing mentality. (That includes the media, by the way.) That attitude fuels a number of unpleasant trends in sports, not just doping. We need to keep in mind that the reason athletes go to events like the Olympics is not to win -- but to participate. When we show the same respect and admiration for the effort of the last-place finisher in the 10K, or the basketball team that loses by 50, or the skier who falls, as we do for the gold medal winners, then we'll have done our part to remove part of the motivation/temptation that drives doping.
We've known for years that Best Buy are spammers; see, for example, http://mainsleazespam.com/companies/bestbuy.html and http://groups.google.com/groups?q=best+buy+spam&ie=UTF-8&oe=utf-8&rls=org.mozilla:en-US:official&client=firefox-a&um=1&sa=N&tab=wg for some background on that.
We also know that their personnel steal private customer data, see for example http://consumerist.com/consumer/insiders/leaks-how-geek-squad-investigated-its-own-porn-thieves-328654.php and http://consumerist.com/consumer/the-rollercoaster-ride-of-pride%2C-shame%2C-and-morality/the-10-page-geek-squad-confession-+-stealing-customers-nudie-pics-was-an-easter-egg-hunt-257108.php and http://consumerist.com/consumer/investigations/video-consumerist-catches-geek-squad-stealing-porn-from-customers-computer-271963.php for background on that.
And now we see that their official corporate policy is to issue threats against comedy troupes and bloggers.
Happily, there is a quick and easy fix for this, known as a "blacklist", so momentarily I'll be adding bestbuy.com, bestbuyinkfinder.com, and myrewardzone.com to it.
1. Please enumerate all of your mail system operational experience, including your anti-spam experience and your participation in the Internet's anti-spam mailing lists/newsgroups/etc.
2. If the algorithm isn't published for unrestricted peer review, then it will be classified along with supposed secret cryptographic algorithms -- that is, snake-oil promoted by liars. Put up or shut up.
3. Brightmail is hardly the "de facto standard"; they have demonstrated only middling competence, and their best work is properly classified as "amateurish". I would expect any junior mail system administrator to be able to build an anti-spam system that outperforms theirs.
4. You have failed to demonstrate even a rudimentary grasp of the spam problem on a practical level. While your supposed "solution" might work in a restricted environment (and the same could be said for many other proposed "solutions") it has no chance in the real world, because you haven't accounted for a number of real-world issues -- including a huge number of compromised systems.
This approach is quite similar to that taken by the DCC. Quoting from its home page: "The DCC is based on an idea of Paul Vixie and on fuzzy body matching to reject spam on a corporate firewall operated by Vernon Schryver starting in 1997. The DCC was designed and written at Rhyolite Software starting in 2000. It has been used in production since the winter of 2000/2001."
As is often the case, those who are new to the spam problem frequently believe they are inventing something new, when it's most likely that they're not -- the remaining question being whether it's workable or long-since abandoned as (mostly) useless. Reputation systems like this are presently somewhat useful, but it's worth noting that should they become widely used, spammers might then find it worth the effort to exercise the control they have over the 100M+ hijacked systems out there and thereby poison the reputation system. While this could be done by generating appropriate traffic, and that'd be moderately disruptive, exerting control over a sufficient number of systems participating in reputation assessment would be worse.
This therefore joins a long parade of specious claims (e.g., Spam as a technical problem is solved by SPF") made to announce the mythical "solution" to spam, which of course does not exist. Does it have possible value in mitigation? It would appear so, based on the track record of similar work (see above). Is it The Answer? Not even remotely close.
1. As those of us who have been fighting spammers for decades have seen -- many times -- attempting to estimate global spam trends based on single measurement points (no matter how large) doesn't work. Moreover, attempting to estimate those same trends based on brief time periods doesn't work either, since there are often seasonal (or longer) fluctuations. So while Google's measurements may be locally and temporarily valid, they have no meaning and no value vis-a-vis the entire Internet.
2. Spammers, like any enterprise seeking long-term success, have diversified. They're into adware, spyware, phishing, domain typosquatting, domain kiting, drive-by downloads, and every other possible line of "business" available to them. If any one spam operation appears to be momentarily declining in output, it's almost certainly because their focus has shifted to another form of abuse.
3. Address harvesting by spammers is now extremely effective, thanks in large part to an estimated 100 million compromised systems (and that number may be low). Any email address that's actually used should be presumed to be in enemy hands shortly; attempts at obfuscation are futile. It's a best practice to make this assumption and plain defenses accordingly.
4. Google itself has been increasingly implicated in spam and spam support activities -- the number of spammer dropboxes there is increasing, as is the number of spammer web sites hosted by them. And Google's abuse desk is non-responsive to even the most well-documented complaints (that is, though where a victim has generously done Google's homework for them and donated the results). This is troubling.
It makes my short list, along with ZImbra, Scalix, and Openchange (so far).
The nice things about JES are (a) it's rock-solid (b) it works well with many mail clients, even horribly broken ones like Outlook (c) while it doesn't have every possible calendar feature in the world, it has all of the ones that people actually use (d) it scales amazingly well -- it's really no problem to get it to support millions of users (e) because it's been around for a while (including a prior incarnation as a Netscape product) there's a pretty solid support community for it (in addition to Sun) (f) it's flexible enough to support integration with other products.
The bad things about JES are (a) the install is complicated, even if you're very accustomed to complex installs (b) the documentation, like much of Sun's documentation, is poorly written, verbose, uses opaque terminology, and lacks cohesion (c) the log files are inscrutable (d) it's somewhat bloated (somebody needs to trim all the legacy code out of it) (e) it's overkill for anyone who just needs a mail server (i.e., no calendaring).
But...given that you get mail, calendaring, LDAP, all rolled up in one package -- it's at least worth looking at. I'm aware of any number of places that have migrated from Exchange to JES, so at least their requirements were met.
Why aren't you?
The software to set up a decent mail/calendar is open-source, free, scalable, well-supported by the community, and will run on commodity hardware. The same can be said for routing and firewall infrastructure (OpenBSD, OpenBGP, pf, etc.) as well as for inbound/outbound email defenses (postfix or sendmail, Clamav, a judicious selection of DNSBLs and RHSBLs, and possibly SpamAssassin).
All of these are mature products that have been deployed (and extensively documented) -- it's really not that difficult to architect a solution that uses them and make it a reality. (I know, I've done it. Multiple times.) Better yet, using these products alleviates the need to rely on vendor support -- expensive, usually worthless, and often less clueful than customers.
The bottom line is that I would expect any mid-level mail systems admin (5 years experience, and no, Exchange does not count) to be able to spec out, architect, and build infrastructure like this -- with some possible help on the networking aspects from a firewall/router jockey. It's just not that hard. Sheesh, back when I worked in academia (at a large midwestern university in an athletic conference which can't count) these sorts of problems were routinely dispensed with so that we could concentrate on the hard stuff.
Oh, and one note about outsourcing: "class breach". Given that Microsoft has already (and repeatedly) proven that it cannot operate a mail server in anything remotely approaching a professional manner, and that it has only a dim grasp of security (at best), anyone outsourcing email to them is just begging for subsequent litigation from faculty/staff/students.
Qmail -- whatever its security merits, and it does have some -- is not suitable for use on the public Internet because it fails to comply with not only de jure standards (RFCs) but de facto standards (best practices). The author has refused to correct these defects -- which is certainly his prerogative as an author, but has as a byproduct serious operational impact on not only users of the package, but other mail server operators who communicate with those run by users of the package.
It's my professional recommendation (based on nearly three decades of experience running mail systems) that qmail be avoided in favor of superior packages such as sendmail, postfix, and exim. (Although the latter, unfortunately, appears to be making increased use of an abusive, spam-supporting feature known as "callbacks".) These packages are actively maintained and their authors have made, and continue to make, efforts to make them standards-compliant and well-behaved (despite the increasing stress placed on them by all forms of mail-related abuse).
As an aside, readers interested in the history of qmail should query a Usenet archive for "a tribute to the programming style of Eric Allman".
Sometimes I find myself idly wondering how many miserable failures of opt-out proposals will be necessary before people get a clue that opt-in offers the only possible way to success.
Then I snap out of it and remind myself that of course some people have a clue, and that's precisely why they continue to put these proposals out (or to enthusiastically back them): doing so serves their purposes nicely. It allows them to proudly say that "they've taken the lead in protecting privacy" while of course they're doing everything they possibly can to do the opposite. (They do this, of course, because they're well aware that few people would opt-in to have telemarketers bother them, or to have spammers clog their mailboxes, or to have their personal data collected.)
This situation is unlikely to change in the forseeable future. Just as it's given us ineffective anti-telemarketing measures, just as it's given us ineffective anti-spam measures, the outcome of this process will inevitably give us ineffective anti-privacy-invasion measures.
Which is why it's probably best to just ignore this nonsense and instead use technological means to either deny data to invaders or feed them bogus data.
Please note as well that if the descriptions of traffic disruption caused by
Comcast that we've seen so far are accurate, they're insufficient to stop
communication channels using UDP, spread-spectrum techniques, burst
transmissions, and so on. Moreover, it's not clear (at least not to me) whether
Comcast has deployed this at the borders of their network -- or internally
as well. If it's only the former, then clearly P2P communications within
Comcast's own network (currently populated with millions of 0wned systems)
aren't affected at all.