It was developed in 1994 and released in 1995. It is a meta search tool that pre-dates the patent and pre-dates Google. It is web and browser based. It has a toolbar that allows you to enter a search term. The pages are modified to highlight the search terms in the browser. While it was never patented, it is prior and appears to be the same thing as the patent claims.
It was developed at a time when every search engine such as Lycos, HotBot, AltaVista, etc. had a portion of the web indexed. WebSeeker searched them all simultaneously, and combined the results into a single list while removing duplicates. The "Filter Find" feature of WebSeeker would downloaded every page, re-index, and display the pages with your keyword highlighted in an embedded IE browser instance.
WebSeeker still exists, still sells, but does not have the market that it once did. Ironically, the dominance of Google meant that searching using multiple search engines wasn't needed anymore. It still works great to keep on top of a specific narrow search term because it remembers what you've seen and monitors for new pages. Still useful for research or competitive analysis in a narrow field.
I was one of the original developers on the project.
Yes! Although this view will not be popular on this site, I completely agree with the parent.
Our limited human knowledge (based on experimentation) of quantum mechanics seems to point to a world designed to allow for a creator that is outside of our space time to be omniscient and allow for free will. While this runs counter to our intuition, so do the results of the experiments.
Evolution seems to be accepted here as an explanation of all life. There is rarely a distinction drawn between micro evolution and macro evolution. They seem to be lumped together and evidence for one is accepted as evidence for the other. I encourage you to verify this for yourself.
I fully expect that the further we explore the boundaries of science, the more God will be revealed. Whether you see that or not will depend on your world view.
The author has accurately identified that we must look at spam in many different ways. The text of the spam message, while important, is not as important as the "tricks" used to elude filters as an accurate identifier of spam.
The spammers, by necessity must disguise the text to elude simple Outlook filters. By doing this, they have introduced more accurate, yet harder to detect, indicators that the message is spam.
By understanding the tricks used by spammer, and using "real intelligence" to detect the tricks, it becomes possible to accurately detect spam without relying on training and other burdensome processes.
Almost all of the methods for detecting spam are included.
Whitelist (by e-mail)
Blacklist (by e-mail)
Blacklist - RBL (by IP)
Dictionary - detects tricks used to throw off Bayesian analyzers
Bayesian Analyzer - Trainable
Challenge/Response for false positives
EMail Stamps - Give spammers the option to pay
Bouncer - Trick spammers into taking you off their list
Good Words - Make sure you get messages you are interested in
Anti-Virus Detection and removal
Script removal
Dangerous attachment removal
Detection and removal of Web Bugs
HTML "loudness" detection
SPF / MS Caller ID
Reply possible detection
There is more, and new techniques are added as they are shown to be helpful. Weighting of each of the techniques is simple. Administrators can keep control centralized, or give each user control over their own account and its settings.
There is an SDK available upon request for plug-in analysis.
Nothing is hidden. Every message gets a report. A web interface lets users see the quarantined messages and a detailed report on why it was not allowed through.
It learns the valid users. Or, the program links into LDAP or Active Directory, or RADIUS (for ISPs), or allows the import of users and automatic generation of passwords.
It may be the most complete anti-spam system developed to date. It does not rely on one technique or method, but rather combines them all, and then allows for new techniques.
We use Spam Sleuth Enterprise from Blue Squirrel which only partially relies on IP blacklists. It allows you to assign points for a blacklisted IP, but more importantly, it also lets you assign points for a bayesian analysis (good or bad), a invalid SMTP server, a invalid MX record, profanity, bad words, good words, blacklisted e-mail addresses, regular expression power filters, etc.
A malfunctioning IP blacklist will give a message more points, but only a fraction necessary to send the message to dev/null
Thought of in another way is that the decision of whether the message is spam or not is distributed among lots of "decision makers" The weight of those decision makers is determined by the number of points they are allowed to assign to a given message.
We also use Spam Sleuth Enterprise to protect our server from SoBig.F. We just look for the text "X-MailScanner: Found to be clean" and set it to enough points to delete the message. It takes the load off of our internal servers.
In my humble opinion, the best one was missed. Spam Sleuth from Blue Squirrel makes Bayesian just a feature of a much larger anti-spam solution. If you want just Bayesian, you can shut off all the other capabilities. That seems to be to be sort of like taking the spark plug wires off of 5 of your 8 cylinders.
Spam Sleuth also does all the other things like Whitelists, Blacklists, RBL, Challenge-Response (Turing), etc. It combines the results to determine "spaminess" and takes action.
Another advantage of Spam Sleuth is that it begins working without Bayesian, until it can build up a set of messages it can use for training. It also lets you correct any mistakes before training so you don't get a bad statistical data set.
It is naive (no pun intended) to think that Bayesian will be able to perform better than a multi-view solution.
SkipWire was able to upload and download programs and data from any Palm OS wireless device.
It did conversions between standard formats like.RTF,.DOC,.PDF,.JPG,.GIF,.PNG and the formats for the Palm, like PalmDoc, Fireviewer, TinyImage, etc.
You could do everything from your Palm browser. You could convert both directions and e-mail. It accepted e-mail attachments, and then converted them and made them available for download.
If you didn't have the right program to view the data, then it provided the links so you could download the free trial, and you could even purchase it.
If you didn't have enough memory, you could just upload something for storage, delete it, and then get it back later.
It was awesome!!!
It still works, but isn't very up-to-date. Probably a victim of the economy.
Palm, Handspring, or Sony ought to pick up the company and update it.
The Challenge-Response system works great provided that the system also uses Bayesian or other methods to accurately detect spam. That way the filters can be "aggressive" while still giving Grandma a way to tell the family that Grandpa is on Viaga.
The Spam Sleuth program from Blue Squirrel has added Challenge-Response. They call it the Turing Test. The same program also has other methods built-in like Bayesian, EMail Stamps, Simulated NDR (Bounce), Whitelists (Friends), RBLs, executable attachment detection and removal, regular expressions, etc.
It appears their Enterprise version works with any e-mail server, but the POP3 version is Windows only:(
Spam is not going to stop. It will continue despite laws and regulations which do not apply world-wide and are difficult or impossible to enforce.
There will not be a "new" SMTP because the existing one is too well established.
There have been many wonderful suggestions posted as previous stories and also as responses to previous stories. Many, perhaps most, of the great solutions require a critcal mass of people to adopt a technical solution at the server. None of those will happen.
The best solution will be individuals or companies adopting products like Spam Sleuth or Spam Sleuth Enterprise which have a variety of detection methods including Bayesian (statistical analysis), EMail Stamps (shift cost to sender), Bouncing (trick the spammers), as well as the usual Whitelists, IP Blacklists, e-mail address Blacklists, etc.
Just like computer viruses, those people who use the technical solutions will be immune, and those that don't will continue to suffer. The tools exist. Slogging through spam each day is a choice.
Is there prior art that would invalidate this patent?
I know there aren't a lot of Bill Gate's fans hanging out here, but what about The Road Ahead (copyright 1996) as prior art. I'm pretty sure the concept of paying per e-mail was mentioned in the book.
Maybe the EMail Stamp concept could be added to Spam Sleuth so it would work with all e-mail accounts, all clients, and all servers.
There has been much discussion of white lists, spammer list, RBLs, simulated 550 bounces, bayesian analysis, etc.
On Linux, these solutions either exist, or are being built, and on Windows there is Spam Sleuth for individuals with ISP accounts, or an Enterprise version for companies, ISPs and schools.
The proposed solutions are not mutually exclusive. Most proponents of Bayesian analysis recommend also using a white list. Add RBLs, Profanity Filter, Bad Word Filter, Valid Sender tests, etc. and it really works great. Keep the spam for a "short time" to train the Bayesian Analyzer, and just-in-case an important message slips through.
These are not solutions where everyone has to comply, or it doesn't work. These tools will stop the spam immediately for those who use them. They instantly have an effect, although probably minor, on the spammers.
We could e-mail everyone a notification about it. Wait... that could take a long time and would be spam.
If we have a way to approach 100% adoption and install software on everybody's server, lets install Spam Sleuth Enterprise or another similar tool and solve the problem completely.
We can't even get administrators to patch their systems. What are the odds on getting this thing widely used?
Slashdot Mirror
I read the patent and WebSeeker from Blue Squirrel appears to me to be prior art.
It was developed in 1994 and released in 1995. It is a meta search tool that pre-dates the patent and pre-dates Google. It is web and browser based. It has a toolbar that allows you to enter a search term. The pages are modified to highlight the search terms in the browser. While it was never patented, it is prior and appears to be the same thing as the patent claims.
It was developed at a time when every search engine such as Lycos, HotBot, AltaVista, etc. had a portion of the web indexed. WebSeeker searched them all simultaneously, and combined the results into a single list while removing duplicates. The "Filter Find" feature of WebSeeker would downloaded every page, re-index, and display the pages with your keyword highlighted in an embedded IE browser instance.
WebSeeker still exists, still sells, but does not have the market that it once did. Ironically, the dominance of Google meant that searching using multiple search engines wasn't needed anymore. It still works great to keep on top of a specific narrow search term because it remembers what you've seen and monitors for new pages. Still useful for research or competitive analysis in a narrow field.
I was one of the original developers on the project.
Our limited human knowledge (based on experimentation) of quantum mechanics seems to point to a world designed to allow for a creator that is outside of our space time to be omniscient and allow for free will. While this runs counter to our intuition, so do the results of the experiments.
Evolution seems to be accepted here as an explanation of all life. There is rarely a distinction drawn between micro evolution and macro evolution. They seem to be lumped together and evidence for one is accepted as evidence for the other. I encourage you to verify this for yourself.
I fully expect that the further we explore the boundaries of science, the more God will be revealed. Whether you see that or not will depend on your world view.
The spammers, by necessity must disguise the text to elude simple Outlook filters. By doing this, they have introduced more accurate, yet harder to detect, indicators that the message is spam.
By understanding the tricks used by spammer, and using "real intelligence" to detect the tricks, it becomes possible to accurately detect spam without relying on training and other burdensome processes.
Blue Squirrel's anti-spam products detect spam using the text as well as detecting the tricks used to disguise the text.
Almost all of the methods for detecting spam are included.
- Whitelist (by e-mail)
- Blacklist (by e-mail)
- Blacklist - RBL (by IP)
- Dictionary - detects tricks used to throw off Bayesian analyzers
- Bayesian Analyzer - Trainable
- Challenge/Response for false positives
- EMail Stamps - Give spammers the option to pay
- Bouncer - Trick spammers into taking you off their list
- Good Words - Make sure you get messages you are interested in
- Anti-Virus Detection and removal
- Script removal
- Dangerous attachment removal
- Detection and removal of Web Bugs
- HTML "loudness" detection
- SPF / MS Caller ID
- Reply possible detection
There is more, and new techniques are added as they are shown to be helpful. Weighting of each of the techniques is simple. Administrators can keep control centralized, or give each user control over their own account and its settings.There is an SDK available upon request for plug-in analysis.
Nothing is hidden. Every message gets a report. A web interface lets users see the quarantined messages and a detailed report on why it was not allowed through.
It learns the valid users. Or, the program links into LDAP or Active Directory, or RADIUS (for ISPs), or allows the import of users and automatic generation of passwords.
It may be the most complete anti-spam system developed to date. It does not rely on one technique or method, but rather combines them all, and then allows for new techniques.
A malfunctioning IP blacklist will give a message more points, but only a fraction necessary to send the message to dev/null
Thought of in another way is that the decision of whether the message is spam or not is distributed among lots of "decision makers" The weight of those decision makers is determined by the number of points they are allowed to assign to a given message.
We also use Spam Sleuth Enterprise to protect our server from SoBig.F. We just look for the text "X-MailScanner: Found to be clean" and set it to enough points to delete the message. It takes the load off of our internal servers.
Hope this helps somebody.
Spam Sleuth also does all the other things like Whitelists, Blacklists, RBL, Challenge-Response (Turing), etc. It combines the results to determine "spaminess" and takes action.
Another advantage of Spam Sleuth is that it begins working without Bayesian, until it can build up a set of messages it can use for training. It also lets you correct any mistakes before training so you don't get a bad statistical data set.
It is naive (no pun intended) to think that Bayesian will be able to perform better than a multi-view solution.
SkipWire was able to upload and download programs and data from any Palm OS wireless device.
.RTF, .DOC, .PDF, .JPG, .GIF, .PNG and the formats for the Palm, like PalmDoc, Fireviewer, TinyImage, etc.
It did conversions between standard formats like
You could do everything from your Palm browser. You could convert both directions and e-mail. It accepted e-mail attachments, and then converted them and made them available for download.
If you didn't have the right program to view the data, then it provided the links so you could download the free trial, and you could even purchase it.
If you didn't have enough memory, you could just upload something for storage, delete it, and then get it back later.
It was awesome!!!
It still works, but isn't very up-to-date. Probably a victim of the economy.
Palm, Handspring, or Sony ought to pick up the company and update it.
Kyocera 7135 is nice, but the form factor is not pocketable.
GPS would be nice, but not necessary.
I've found it easy to use and it automatically configured and read in my contacts from Eudora. I hate Outlook, but I think it's also supported.
Any e-mail I get, I can block with some type of rule. I even wrote a regular expression to detect comments inside words (a new trick of theirs).
The Spam Sleuth program from Blue Squirrel has added Challenge-Response. They call it the Turing Test. The same program also has other methods built-in like Bayesian, EMail Stamps, Simulated NDR (Bounce), Whitelists (Friends), RBLs, executable attachment detection and removal, regular expressions, etc.
It appears their Enterprise version works with any e-mail server, but the POP3 version is Windows only :(
The laws are necessary because it forces the spammers to modify their messages in such a way that they (the spammers) won't get caught.
Then, programs like Spam Sleuth can easily detect the deception and remove the messages.
Pass the laws, and use technology. The solutions exist already. Receiving spam is a nuisance (bandwidth). Seeing spam is a choice.
Let's pass the laws. If, or rather, when they don't stop the spam, let's turn to technological solutions.
I hope I'm wrong. If the laws are sufficient, and enforcement is effective, e-mail will be better for it.
There will not be a "new" SMTP because the existing one is too well established.
There have been many wonderful suggestions posted as previous stories and also as responses to previous stories. Many, perhaps most, of the great solutions require a critcal mass of people to adopt a technical solution at the server. None of those will happen.
The best solution will be individuals or companies adopting products like Spam Sleuth or Spam Sleuth Enterprise which have a variety of detection methods including Bayesian (statistical analysis), EMail Stamps (shift cost to sender), Bouncing (trick the spammers), as well as the usual Whitelists, IP Blacklists, e-mail address Blacklists, etc.
Just like computer viruses, those people who use the technical solutions will be immune, and those that don't will continue to suffer. The tools exist. Slogging through spam each day is a choice.
It has educational links by Grade/Topic.
I know there aren't a lot of Bill Gate's fans hanging out here, but what about The Road Ahead (copyright 1996) as prior art. I'm pretty sure the concept of paying per e-mail was mentioned in the book.
Maybe the EMail Stamp concept could be added to Spam Sleuth so it would work with all e-mail accounts, all clients, and all servers.
On Linux, these solutions either exist, or are being built, and on Windows there is Spam Sleuth for individuals with ISP accounts, or an Enterprise version for companies, ISPs and schools.
The proposed solutions are not mutually exclusive. Most proponents of Bayesian analysis recommend also using a white list. Add RBLs, Profanity Filter, Bad Word Filter, Valid Sender tests, etc. and it really works great. Keep the spam for a "short time" to train the Bayesian Analyzer, and just-in-case an important message slips through.
These are not solutions where everyone has to comply, or it doesn't work. These tools will stop the spam immediately for those who use them. They instantly have an effect, although probably minor, on the spammers.
If we have a way to approach 100% adoption and install software on everybody's server, lets install Spam Sleuth Enterprise or another similar tool and solve the problem completely.
We can't even get administrators to patch their systems. What are the odds on getting this thing widely used?