I've tried both some custom filters using 'Bayesian' categorization and also used SpamAssassin. Both have proved *highly* effective?
I'm currently using SpamAssassin, and although it seems to properly identify 90% of spam, it still misses a bunch, and it even occasionally marks valid emails as spam, which is a BIG no-no in my books. It means I still have to swarm through every message looking for names I recognize, just in case I nuke an important email. (I've played around with a bunch of "levels" to set it at, but to get that number high enough to let 100% of the valid emails through, it seems to let more than 50% of the spam get through).
I haven't tried any of the Bayesian stuff (yet), but I imagine it'll have a similar hit-ratio.
The problem is that if your spam-filter blocks even ONE non-spam email, it's unacceptable.
As for the public DB of spam messages, I can't see it doing much if any good - all it will do is force spammers to completely personalize/randomize each mail they send out (move a bunch of words around, swap paragraphs, add nonsense tags everywhere), so no sort of quasi-CRC check or even fuzzy-algorithm'ed spam detector could recognize it.
I'm afraid the grandparent is right - whitelists are the only way to block as much spam as possible, while guaranteeing all valid emails get through.
(ps. I like the concept of having a daily, automatically generated.GIF file with some password in it that anyone wanting to get on your whitelist reads and types in - no need to have a "handshake" of sorts before they can send you email).
I have to agree to being a little offended by his Mozilla comment, but in a way, he's right. Most slashdotter's know that Mozilla is fast becoming a better browser than IE, but can it ever take over (back?) its market share? Not until it's the default browser on Joe HomeUser's fancy new computer from the store. Or until Mozilla's install is a literal one-click effort from a web page, and people start posting links to it on every page they create.
I switched from Netscape to IE quite a few years ago (not because it was already installed, but because Netscape started to suck). Now I've switched (back) to Mozilla, because I'm one of those power-users who loves to customize and use all the new whiz-bang features. But the average user doesn't even KNOW there's a "preferences" area - all they care about is that their favorite sites look good and work properly. Unfortunately, I find myself occasionally having to revert back to IE to view a site because some DHTML-this or ActiveX-that doesn't work properly (sure, we can blame the web-site developer, but the average guy will happily blame his browser first).
Internet censorship is no better than a Nazi bookburning. Doesn't make a difference if they're blocking printed text or unicode.
True, but you can't be angry at the guy who invented fire, just because someone's using fire in bad ways.
Yes, China's communist practices of censorship are not a Good Thing, but just because companies produce filtering technology and sell it to them doesn't make THEM bad. The filtering tech can be useful if used properly.
It's the age old dispute that applies here - "guns don't kill, people do".
Capt. Braddock: Okay no more bullshit
[to Dave, talking fast]
Capt. Braddock: was there or wasn't there a woman?
Dave: Are you serious?
Capt. Braddock: Yes I'm goddamn serious.
Dave: Fuzzy Wuzzy was a woman?
- Hear No Evil, See No Evil
With broadband being so popular, all you need is a family member/friend who also has it, and write a little script to zip up and ftp/scp the backup file to their computer. They do the same to you - should a fire break out, it's doubtful both of you will be affected. If it is a concern, just add more people in the "ring".
No need for big backup tape drives or burners, and no hassle once you have it up and running. (Of course, the usual "test your backups" mantra still applies - no sense backing things up if you're not doing it right).
You can also use a dynamic DNS service and client apps so you don't have to constantly updating IP addresses when the ISPs change them.
Legislation will never even put a dent in the amount of spam you or I receive, because the Internet is global. Legislation is bound to a single state/country. Even if something as mighty as the U.N. decreed it unlawful to send spam, it wouldn't affect nations not part of the U.N. Legislation could only work if every single country in the WORLD buys into it, AND actively enforces it.
In fact, most North American ISP's (and I'm sure thousands in other countries) are doing a great job of finding and killing spam accounts as they flair up. But most of the regular spams being sent today are from open relays hosted in other (often third-world) countries, or from foreign ISPs who encourage the business (the more bandwidth used from them, the more money they make - they don't care HOW it's used). Unless we're willing to close the borders (and destroy one of the greatest aspects of the Net), this will always hold true.
I'm afraid the author of this article is correct - email, as we know it, is dying a quick death. The whitelist concept is the only spam-proof technical, and legal, solution there is.
I don't remember the last time a PC crash stopped me in the middle of using my computer--I suspect it was playing UT, which is hardly Windows's fault.
If a single app brought down your whole machine, then yes, it IS Windows's fault. No OS should ever crash because of an errant process. I have yet to experience a kernel-fault in Linux, even though I've had plenty of applications and server processes die, yet my Windows XP box still crashes occasionally due to a single program (sure, XP is miles ahead of older versions for stablity, but it still has a few more miles to go).
It wouldn't take much to flood a DSL line with enough traffic to render it useless, no matter what router is on the home user's end. Only the ISP's routers could block a DoS attack like that.
I was pointed to a an interesting site that sells (possibly illegal in your area) boosters, bridges, antennae and other miscellanea for 802.11B.
This isn't an endorsement - I have no need in my small place, and haven't tried any of this myself, but it sure would be fun playing around with some of this stuff.
When someone has a patent on a technology, it doesn't prevent others from stealing it and using it themselves. It just means the original owner of the patent must now take them to court to win damages back, assuming they're aware of the theft in the first place.
It's akin to software companies - they can sell their games without any copy-protection whatsoever, and just hope everyone follows copywrite laws, but this obviously doesn't happen very often.
We've tried banning IP's for abusers of our site - works great, until one of them is an AOL'er, as AOL only uses a small number of massive proxy machines for all of their users (ie. millions of AOL users appears as 10-20 different IP's).
People certainly make legit backups of game CDs... for $50 or more a pop, I don't want to risk not being able to play a game because I was careless and allowed it to get scratched. Especially when each modern game requires the CD to be in the drive to play, I probably switch CDs 3 or 4 times a day to get my gaming fix. Often they're left shiny side up, begging for something to be dragged across its surface and permanently ruin it. When I'm using a burnt CD, I no longer have to fear of losing my investment.
Sure, it's easy to Print Scrn an ebook and then OCR it to text, but when you're talking about super high-quality multimedia content, feeding it back though a video/sound card is bound to introduce noise (or spelling mistakes in the case of OCR), which is unacceptable to most audio/video-philes. Of course, a simple DivX rip of a DVD is unacceptable to the same crowd, due to its lossy compression, so perhaps copy-protection is a moot point until home bandwidth is reasonable enough to copy the full, raw VOB files from a DVD.
Besides, anyone can crack these schemes through brute-force like the author did, but the real fun (and bragging rights) is in actually cracking the encryption, no?
Another proof by example that Google is a Good Thing, they're working on keyboard shortcuts for quickly getting to your search results without needing to pry your sticky fingers from the keyboard. Best of all, it uses the familiar k and i for all us vi[m] users.
Ok, so does this mean that an email containing ANY advertisement within it must be prefixed with ADV:... what about the many legitimate mailing lists (for jokes, stock quotes, music/dvd/game schedules, etc), that use these banner-ads and text-ads within the message to fund their production (like 90% of the Web uses). The main content of the email is NOT unsolicited, yet it does contain unsolicited pieces.
Every SysAdmin in the world is going to automatically send any ADV: emails to the bit bucket, making the delivery of these types of lists a virtual nightmare (and subsequently causing them to go to a subscription model, meaning we lose even more of the wonderful freebies the Net is known for.)
To be fair, Lessig does point out that there needs to be human intervention, but I'm where is the line drawn?
"Theft of cable TV costs the industry an estimated $6 billion a year, according to the National Cable Television Association."
I hate these kind of fabricated numbers - the question is, would the 11 million people who are supposedly stealing cable and sat services (more detail here) have really bought 6 billion bucks worth of programming and pay-per-view if they didn't have their illegal access. I think their number would be far lower.
That's like the recording industry claiming massive theft when someone downloads a popular single they heard on the radio - would that person have actually gone out and purchased the CD for that song if the file-sharing apps weren't around? I doubt it, at least most of the time. I know I download hundreds of tunes that I never would have considered buying in the first place (but may now purchase because I get to hear what the CD sounds like - but that's a different argument...)
If I had access to free pay-per-view, I'd watch almost every movie out there, as I'm a huge movie buff. But I don't have free access, and I've never purchased a single pay-per-view program - how can the cable company claim any losses?
Most of the hackers out there who find the holes in Windows and other commercial O/Ses don't have access to any source code, yet they still find the holes.
Although the renaming of system utils is funny - I bet you could break 90% of the script-kiddie tools out there just by installing Windows in a non-default directory (like C:\Linux - really confuse them - although you'll probably break half of the windows utils too...)
As with everything in life, it's not about which product is better - it's about how they're marketed.
For months, all I saw on TV were Spider-Man ads, major hype in newspapers, magazines & comic strips, and all I heard on the radio were the two big hits from the Spider-Man soundtrack (as the DJ's would constantly remind us): Sum 41's "What We're All About" and Chad Kroeger & Josey Scott's "Hero".
Yet I didn't see a tenth of the advertising for AtoC (and it seemed like a lot less than they did for the Phantom Menace). I believe Lucas intentionally cut down on the merchandising and marketing partnerships with big companies like Pepsi (who owns Pizza Hut, Taco Bell, KFC) this time round, for whatever reason.
The younger generation spends half their TV time watching MTV, and a constant barrage of videos & ads for Spider-Man vs. the occasional ad for AtoC means the Spider-Man marketing folks knew what they were doing.
I've seen both, and although they're flawed in various minor ways, I'm rate them awfully close.
With the major advances in robot technology, A.I., computer vision, etc. etc., I'm very surprised they'd even consider using people again. The cost associated with maintaining a human crew's life support, food, and environment is huge (not to mention how much larger the craft must be to hold all this, and how much more fuel it takes to get out of Earth's atmosphere, AND bring them all back, AND the usual huge risk of loss of life...). I think it would be better spent building a better robot.
Obviously, the robots can't do everything themselves, but humans on earth can reasonably control them (it would take anywhere from 3 to 22 minutes for a one-way communication from Earth to Mars, depending on their respective orbits around the sun).
Unless we're ready to start terraforming, I don't think it's cost-effective to send humans.
Although it's absolutely amazing Microsoft is still getting away with this, it's certainly business as usual.
As detailed in Jerry Kaplan's excellent book Startup: A Silicon Valley Adventure , about the rise and fall of the GO Corporation, one of the first anti-trust cases to be brought up against Microsoft involved a very similar license (circa late-80's, early-90's).
Basically, every retailer who wanted to sell Microsoft products (and who didn't - even then it was very popular software) had to sign a contract with Microsoft stating that for every competitor's product they sell, they had to pay a 100% royalty back to Microsoft! (you read that right - here's a quick example: if the retailer buys both a MS product and a GO product for $50 a piece, and typically doubles the price to $100 to make a profit, they'd have to pay Microsoft $50 if they sold the GO product, so the retailer is basically forced to sell the GO product for double their usual markup ($150) - 50 to GO, 50 to Microsoft, and 50 to themselves). And as icing on the cake, the retailer wasn't allowed to mention the terms of the contract to anyone.
The only way GO eventually found out was from a rare retailer who had seen the contract, but decided not to sign it (and therefore not to sell any MS products in his store).
Bizarre? I'd say. Illegal? Oh yeah. I think that's a text-book definition of anti-competitive behavior. And it's basically the exact same thing they're doing to the schools - the school still has to pay Microsoft for using a competing product.
Sadly, the DOJ didn't pursue it to closure because they couldn't get enough witnesses (they were too scared to lose Microsoft's business).
(OT: it's a great book, read it if you get a chance - it should have been required-reading for all dot-coms).
In the evolutionary ladder, SpamBots are just crawling out of the ocean. It's only a matter of time before they will look and feel exactly like a human surfer - HTTP is too simple a protocol to allow otherwise.
It'd be a simple matter to write a SpamBot that gets pages with a random delay in between hits, that spoofs the Referer, User-Agent, and other headers, that actually downloads images to/dev/null, etc.
So how does one fight it? Unfortunately, you just have to do what most big weblog sites are doing - obfuscation, either with random text strings, or by dynamically generating an image with the email address in it.
Or, you can offer an online HTML form that will send the email out to the address you have on file (but don't display it in plain text anywhere). Not a great solution, as the email will come from your server, and most people like to use their own mail agent, but at least there's no way for the SpamBot to gain access to the email list.
I think trying to develop automated "traps" for SpamBots is a battle we just can't win, at least until every ISP in the world doesn't allow their users to send out spam (read: never).
I think it'd be more like 1/100th of the cost for a developer to do that change instead of paying lawyers to draft up C&D letters, at least initially. But it would cost more in the long-term, because people will no longer deep link to the site, and they'd lose out on all sorts of ad revenue from those extra clicks they WOULD have had.
Why can't they understand that a website only gets popular from word-of-mouth (ie. links on other sites), and popularity means more ads and subscriptions.
Slashdot Mirror
I haven't tried any of the Bayesian stuff (yet), but I imagine it'll have a similar hit-ratio.
The problem is that if your spam-filter blocks even ONE non-spam email, it's unacceptable.
As for the public DB of spam messages, I can't see it doing much if any good - all it will do is force spammers to completely personalize/randomize each mail they send out (move a bunch of words around, swap paragraphs, add nonsense tags everywhere), so no sort of quasi-CRC check or even fuzzy-algorithm'ed spam detector could recognize it.
I'm afraid the grandparent is right - whitelists are the only way to block as much spam as possible, while guaranteeing all valid emails get through.
(ps. I like the concept of having a daily, automatically generated .GIF file with some password in it that anyone wanting to get on your whitelist reads and types in - no need to have a "handshake" of sorts before they can send you email).
I switched from Netscape to IE quite a few years ago (not because it was already installed, but because Netscape started to suck). Now I've switched (back) to Mozilla, because I'm one of those power-users who loves to customize and use all the new whiz-bang features. But the average user doesn't even KNOW there's a "preferences" area - all they care about is that their favorite sites look good and work properly. Unfortunately, I find myself occasionally having to revert back to IE to view a site because some DHTML-this or ActiveX-that doesn't work properly (sure, we can blame the web-site developer, but the average guy will happily blame his browser first).
Yes, China's communist practices of censorship are not a Good Thing, but just because companies produce filtering technology and sell it to them doesn't make THEM bad. The filtering tech can be useful if used properly.
It's the age old dispute that applies here - "guns don't kill, people do".
No need for big backup tape drives or burners, and no hassle once you have it up and running. (Of course, the usual "test your backups" mantra still applies - no sense backing things up if you're not doing it right).
You can also use a dynamic DNS service and client apps so you don't have to constantly updating IP addresses when the ISPs change them.
In fact, most North American ISP's (and I'm sure thousands in other countries) are doing a great job of finding and killing spam accounts as they flair up. But most of the regular spams being sent today are from open relays hosted in other (often third-world) countries, or from foreign ISPs who encourage the business (the more bandwidth used from them, the more money they make - they don't care HOW it's used). Unless we're willing to close the borders (and destroy one of the greatest aspects of the Net), this will always hold true.
I'm afraid the author of this article is correct - email, as we know it, is dying a quick death. The whitelist concept is the only spam-proof technical, and legal, solution there is.
I don't remember the last time a PC crash stopped me in the middle of using my computer--I suspect it was playing UT, which is hardly Windows's fault.
If a single app brought down your whole machine, then yes, it IS Windows's fault. No OS should ever crash because of an errant process. I have yet to experience a kernel-fault in Linux, even though I've had plenty of applications and server processes die, yet my Windows XP box still crashes occasionally due to a single program (sure, XP is miles ahead of older versions for stablity, but it still has a few more miles to go).
It wouldn't take much to flood a DSL line with enough traffic to render it useless, no matter what router is on the home user's end. Only the ISP's routers could block a DoS attack like that.
This isn't an endorsement - I have no need in my small place, and haven't tried any of this myself, but it sure would be fun playing around with some of this stuff.
It's akin to software companies - they can sell their games without any copy-protection whatsoever, and just hope everyone follows copywrite laws, but this obviously doesn't happen very often.
What... you mean you don't feed your family with Squirrel Fishing?
Wait... maybe this is a good thing... :)
People certainly make legit backups of game CDs... for $50 or more a pop, I don't want to risk not being able to play a game because I was careless and allowed it to get scratched. Especially when each modern game requires the CD to be in the drive to play, I probably switch CDs 3 or 4 times a day to get my gaming fix. Often they're left shiny side up, begging for something to be dragged across its surface and permanently ruin it. When I'm using a burnt CD, I no longer have to fear of losing my investment.
Besides, anyone can crack these schemes through brute-force like the author did, but the real fun (and bragging rights) is in actually cracking the encryption, no?
Another proof by example that Google is a Good Thing, they're working on keyboard shortcuts for quickly getting to your search results without needing to pry your sticky fingers from the keyboard. Best of all, it uses the familiar k and i for all us vi[m] users.
Ok, so does this mean that an email containing ANY advertisement within it must be prefixed with ADV:... what about the many legitimate mailing lists (for jokes, stock quotes, music/dvd/game schedules, etc), that use these banner-ads and text-ads within the message to fund their production (like 90% of the Web uses). The main content of the email is NOT unsolicited, yet it does contain unsolicited pieces.
Every SysAdmin in the world is going to automatically send any ADV: emails to the bit bucket, making the delivery of these types of lists a virtual nightmare (and subsequently causing them to go to a subscription model, meaning we lose even more of the wonderful freebies the Net is known for.)
To be fair, Lessig does point out that there needs to be human intervention, but I'm where is the line drawn?
... but what about the rest of the world?
Does Google have any plans to become a public company, so all of us devoted fans can start throwing our hard-earned money into your IPO?
I hate these kind of fabricated numbers - the question is, would the 11 million people who are supposedly stealing cable and sat services (more detail here) have really bought 6 billion bucks worth of programming and pay-per-view if they didn't have their illegal access. I think their number would be far lower.
That's like the recording industry claiming massive theft when someone downloads a popular single they heard on the radio - would that person have actually gone out and purchased the CD for that song if the file-sharing apps weren't around? I doubt it, at least most of the time. I know I download hundreds of tunes that I never would have considered buying in the first place (but may now purchase because I get to hear what the CD sounds like - but that's a different argument...)
If I had access to free pay-per-view, I'd watch almost every movie out there, as I'm a huge movie buff. But I don't have free access, and I've never purchased a single pay-per-view program - how can the cable company claim any losses?
Although the renaming of system utils is funny - I bet you could break 90% of the script-kiddie tools out there just by installing Windows in a non-default directory (like C:\Linux - really confuse them - although you'll probably break half of the windows utils too...)
For months, all I saw on TV were Spider-Man ads, major hype in newspapers, magazines & comic strips, and all I heard on the radio were the two big hits from the Spider-Man soundtrack (as the DJ's would constantly remind us): Sum 41's "What We're All About" and Chad Kroeger & Josey Scott's "Hero".
Yet I didn't see a tenth of the advertising for AtoC (and it seemed like a lot less than they did for the Phantom Menace). I believe Lucas intentionally cut down on the merchandising and marketing partnerships with big companies like Pepsi (who owns Pizza Hut, Taco Bell, KFC) this time round, for whatever reason.
The younger generation spends half their TV time watching MTV, and a constant barrage of videos & ads for Spider-Man vs. the occasional ad for AtoC means the Spider-Man marketing folks knew what they were doing.
I've seen both, and although they're flawed in various minor ways, I'm rate them awfully close.
Obviously, the robots can't do everything themselves, but humans on earth can reasonably control them (it would take anywhere from 3 to 22 minutes for a one-way communication from Earth to Mars, depending on their respective orbits around the sun).
Unless we're ready to start terraforming, I don't think it's cost-effective to send humans.
As detailed in Jerry Kaplan's excellent book Startup: A Silicon Valley Adventure , about the rise and fall of the GO Corporation, one of the first anti-trust cases to be brought up against Microsoft involved a very similar license (circa late-80's, early-90's).
Basically, every retailer who wanted to sell Microsoft products (and who didn't - even then it was very popular software) had to sign a contract with Microsoft stating that for every competitor's product they sell, they had to pay a 100% royalty back to Microsoft! (you read that right - here's a quick example: if the retailer buys both a MS product and a GO product for $50 a piece, and typically doubles the price to $100 to make a profit, they'd have to pay Microsoft $50 if they sold the GO product, so the retailer is basically forced to sell the GO product for double their usual markup ($150) - 50 to GO, 50 to Microsoft, and 50 to themselves). And as icing on the cake, the retailer wasn't allowed to mention the terms of the contract to anyone.
The only way GO eventually found out was from a rare retailer who had seen the contract, but decided not to sign it (and therefore not to sell any MS products in his store).
Bizarre? I'd say. Illegal? Oh yeah. I think that's a text-book definition of anti-competitive behavior. And it's basically the exact same thing they're doing to the schools - the school still has to pay Microsoft for using a competing product.
Sadly, the DOJ didn't pursue it to closure because they couldn't get enough witnesses (they were too scared to lose Microsoft's business).
(OT: it's a great book, read it if you get a chance - it should have been required-reading for all dot-coms).
It'd be a simple matter to write a SpamBot that gets pages with a random delay in between hits, that spoofs the Referer, User-Agent, and other headers, that actually downloads images to /dev/null, etc.
So how does one fight it? Unfortunately, you just have to do what most big weblog sites are doing - obfuscation, either with random text strings, or by dynamically generating an image with the email address in it.
Or, you can offer an online HTML form that will send the email out to the address you have on file (but don't display it in plain text anywhere). Not a great solution, as the email will come from your server, and most people like to use their own mail agent, but at least there's no way for the SpamBot to gain access to the email list.
I think trying to develop automated "traps" for SpamBots is a battle we just can't win, at least until every ISP in the world doesn't allow their users to send out spam (read: never).
Why can't they understand that a website only gets popular from word-of-mouth (ie. links on other sites), and popularity means more ads and subscriptions.