Spam and virus filtering in an efficient manner for anyone is a major issue, and it has already been mentioned that there are multiple ways of accomplishing this. In designing your process, think in terms of dropping or rejecting email from the process loop as soon as possible.
At the SMTP server
Drop email from known blacklisted servers via your email server access file
Allow email from known whitelisted servers or addresses
Use RBL lists
Filter out mis-behaving SMTP servers, ones that don't follow standard protocols
Disable ESMTP commands that give the spammer access to your local users lists (VRFY, etc..)
Only relay email from authenticated servers and users
Impose a size limit to messages (50k) if possible.
At the SMTP Filter Proxy Server or LDA
Allow emails from recipient-based whitelists
Drop emails from recipient-based blacklists
Process Tagged messages (from TMDA)
Run your faster classification programs: clamav (for viruses), bogofilter (bayesian)
Run your slower classification programs: procmail, spamassassin
Just remember to shortcut the process along the way. If email can be dropped or tagged for any reason, do so immediately and quit processing it.
Re:GPL has anti-DRM built in?
on
Linus on DRM
·
· Score: 1
I don't think you get the concept here. Drop the FUD regarding Linux, the GNU GPL, and license contamination. It's unrelated to this topic. Just because a binary is run on Linux doesn't mean you have to expose your precious source code.
An on topic observation: A DRM-enabled kernel would allow commercial distributors of embedded devices or special-purpose systems to use Linux as a possible platform, even if the software they intend to run may only be licensed legally under DRM provisions. They would have to think in terms of using a user-space daemon with one-way hashes or public key encryption for license and CRC validation.
Regardless, it's largely a moot point, which is the gist of this argument. It doesn't matter, really, if Linux is DRM-enabled if the end-user refuses to play by the rules.
Language and implementation are irrelevant. That is my point. Security breechs happen. Bad code happens. Cracks happen. If he wanted to make a general plea to programmers to use more secure tools, he didn't have to focus on Unix/Linux and C to do it. He could have chosen any language, any interpreter, and made exactly the same point. Instead, he chose to point his finger at a specific group of developers to see what kind of response he could raise. He succeeded in getting over 130+ posts on/.
I don't feel it necessary to expound on this further. There are plenty of comments here and published papers on writing clean, secure, and efficient code, regardless of which language is used.
...about the recent paper detailing the cracking of a JVM?
Of course he is, because that would make the article larger in focus than what he wants, which is to push buttons. C may be a riskier language than some, but even the big-dog interpreted languages have their own problems.
Although S tar is a nice utility, as mentioned by previous posters. You can get pax directly from the OpenBSD people. Debian also packages pax, if you run Linux.
Try out LVM from Sistina. It's part of 2.4.x, why not use it?;-) In fact, you can "apt-get install lvm10" on Debian Woody. If you want to compile it yourself, it's pretty easy. Sistina did a good job wrapping up the software and kernel patches with a well documented installation procedure. I couldn't live w/o my LVM.
The first problem I see with this is that there is no reason to use a passwordless key. The only reason you would want to do so is to actually sign the data file. Signatures should NOT be done automatically, since they mean nothing. If you really want to sign the data file w/an automated signature, create a key just for signing, but use your 2048 bit key for encryption. Remember, you don't need a password to encrypt data.
Now, I haven't looked at the code yet, but I'm assuming that the python script is simply a wrapper around an HTTP server, a MD5 or SHA1 hashing algorithm (for the filename), tar, bzip2, and some meta database to keep an index of files backed up.
That being said, it's not a bad concept. If you have a trusted friend that will allow you to back up files on his machine, this wrapper should operate nicely. Otherwise, you can always do it by hand.
Not so novel idea: use a dedicated snort box that uses its alerts to add rules to a firewall, switch, and/or router. Check out the snort FAQ and contrib directory for ideas.
Better yet. Replace pieces of the app with C/C++ modules that can be accessed from within python. Use C/C++ only as a faster-implementation of a module.
I can't imagine a peristaltic pump would be difficult to build. Take a look at this animation
to see what I mean. It's simply an electric motor with three or four arms to which rollers are attached. The rollers pinch the tubing and squeeze it along it's track until it lifts off the tube on the other side. In the mean time, another roller contacts the tube on the far side and continues the sequence. Simple, clean, and cheap. An added advantage is that you can't burn out these pumps when no fluid is present...
Pumps are only expensive because the include nice housings, some logic circuits to control the motors, and include service warranties. It shouldn't be too hard to construct your own parastolic pump with some very simple components: a 12 volt battery, a small electric motor ($3.50 at radio shack, though you *may* want a little more power), some ABC plastic for the disk that holds the rollers, the rollers themselves, tubing, screws, and a piece of wood to carve the 1/3rd circle track. If they're already wiring up relays for the controller board, purchase electronic solenoid valves for fluid control. The system is less complicated, fewer moving parts, and more robust.
...the variants of programming languages available to us. Just when you think you've got your "favorite" language pegged, along comes another to tempt and tease you. Just check out the dmoz.org site out for a list of programming
languages. It's nuts!
Re:where have I heard that before
on
Deadly Perversions
·
· Score: 5, Insightful
hmmm... oh, yeah...
Stephenson, Neal,
"Snow Crash", Bantam Books, Inc. 2000, ISBN 0553380958.
Is your employer's concern over licensing or over distributing source code? Linus Torvolds has made it quite clear that the Linux kernel has provisions for commercial modules, given that the module follows the API requirements, such as providing a COPYRIGHT notice in the initialization structure of the module itself. Proprietary modules are not an issue with it comes to the kernel.
I believe that any changes you make to the kernel itself in order to integrate your module need to be GPL'ed. For example, let's say you have to make a patch to the Block IO API in order to enable some functionality of your driver. That patch must be GPL'ed. Your driver is still safe.
If your company is concerned with distributing source code, regardless of license, then you have another problem. In that case, you will be in the business of compiling kernels if you don't want to deal with a physical module. Compiling kernels and distributing them, which you have all the rights in the world to do, adds overhead to your company's responsibilities that they probably don't want.
So, when it comes to the Linux kernel, you should talk to Linux kernel developers and Linus Torvalds or his legal team in particular. Remember that the Linux kernel does have provisions for legal use of commercial and proprietary licensed drivers. Looking only at the GPL is not enough in this case.
If you would like to see a nice comparison of VCD, DVD, and DivX recordings, take a look at the grid table at http://www.vcdhelp.com/comparison.
This site has lots of good info, a definite bookmark!
You (the reader) may not agree in the general distrust I have for binary applications from third parties, but I would rarely run one of these applications outside of a chroot environment. There is an implicit trust that the software provider is only working under your best interests, but can you truly trust someone you don't know personally or someone who won't guarantee that their program won't destroy your OS installation?
My general attitude about such software is to give it a shot, but only in a chroot jail. This particular software, the Folding@home binary, has been unsuccessful in running under such an environment. It will start running, but it won't retrieve data sets from the servers.
Whose loss is this? Not mine. If I can't get something running in a satisfactory environment after spending a reasonable amount of my valuable time on it, I don't run it.
Has anyone had success in running the Linux binary in a chroot?
There's an interesting distinction that I haven't browsed upon yet in the comments (though, I'm not looking very hard). GPL allows you to sell the software, but why would anyone want to do that? Anyone could just as easily download it from the net for free. That's one of the founding principles of GPL software and the Free Software movement.
Let's apply this knowledge. For example, Red Hat doesn't necessarily sell software. It distributes the software and charges for the distribution. The software is otherwise freely available for download in source code or binary format. Customers are buying service and convenience, not software. This may be walking a thin line legally, but thin lines have been known to be found in favor of defendants more often than procecutors.
DOH, correction. It seems to me that clamscan (Clam Antivirus command line scanner) is the one doing the multithreading, not amavis. Very cool tool. Check it out if you have a chance, people.
Not sure what winbindd is, but I'm assuming it's a DNS server modeled after the ISC Bind (DNS Server)?
Clam Antivirus is a virus scanner, a command-line tool used to scan files for virus signatures. It will report whether it finds a virus or not. AmaVis is used as a filtering daemon for email. It unpacks MIME messages into multiple files, decompressing them if necessary, and runs the virus scanner over each file. If it finds a virus with its tools, it reports the results to the following (configurable, of course): the admin, the sender (I shut this off because of spoofing), and the receiver (you can shut off alerts sent to recipients that are off-site). The entire email is saved in a quarantine directory; it is not deleted.
The virus definitions file is updated by the members of the Open Antivirus project. Subscribe to their email list to get bleeding-edge, just found definitions. Otherwise, just let the clam antivirus updater fetch the definitions when the project updates them (1-2 day delay after a new virus is identified -- or at least it seems that way). Talk to the OpenAV guys for legit frequency info.
The only reason why we don't go with a commercial product is because most of these products charge by the number of recipients or users on the system, often requiring client licenses for each user as well. McAffee wanted WAY too much money for what we wanted to do, especially considering that we already have Nortan Antivirus installed on the Windows and Mac machines (University site license). Why pay for something we already have?
To date, I haven't seen a virus come through amavis+clamav yet, but that is my own personal experience and that of our users.
Spamassassin is a different beast entirely. I use procmail scripts to intercept messages bound for email lists (served from ecartis) and filter them for spam. I also filter out VIRUS warnings sent by AmaVis. These filtered items get saved to a "spam" and a "virus" folder, and I wrote a cron job to report how many emails it finds in these folders twice per day. It's valuable to send these to individual recipients on the system, but not to a list.
Procmail is an important piece of the spam filtering process. Postfix can do content filtering, so I think it's certainly possible for me to have spamassassin tag EVERYTHING coming into the system, just like AmaVis does now. I just haven't pieced together how to do it yet. It would eliminate the need for users to run procmail recipes and drop the number of processes run on the server.
If you want to disinfect files, go commercially funded/grown software... that is, at least until the Open Antivirus people or another group come up with virus definition files that include instructions for disinfecting files.
Postfix: mail transport agent (MTA); packaged by most Linux distros; runs on many other platforms; easy to cinfigure; flexible; modular; secure; highly scalable; written in C by the venerable Wietse Venema; IBM Public License
AmaVis: Antivirus filtering daemon; packaged by most linux distros; multi-threaded (recognized multiple CPU's); sends out email alerts; very configurable; supports many antivirus scanners; works well with postfix; written in Perl; GPL
Clam Antivirus (clamav): virus scanner; written in C; fast; virus definition update tool included; uses virus definitions from the Open Antivirus project; (does not disinfect, just identifies); GPL
SpamAssassin:
Perl-based Spam filter; use with Procmail; client-server architecture (one daemon); Perl Artistic License
Our application of the above software seems to work quite well. We server about a thousand users (about 100 "heavy users"), and the average server load rarely gets above 0.21 with a Dual AMD 1500+ MP that provides SMTP, IMAP, and POP all w/SSL enabled.
This sounds interesting, but IANA Geologist. It also sounds risky, trusting in the good faith of "Mother Nature" not to do anything drastic, like push up a new mountain range and bring all of the "not yet consumed" nuclear waste. Do we really want to meddle with an obviously powerful natural force as plate tectonics? Again, IANAG, so I have no answers, only questions regarding your suggestion.
IMHO, Yucca Mt. is the US's best viable solution to the Nuclear waste disposal problem. People who are not familiar with the daily operation of Nuclear power don't really understand the massive amounts of training and beurocracy that surrounds it. Plant operators undergo more testing and requalification training than any airplane pilot. It's a stressful job, but the knowledge that they possess is extremely important when emergencies or problems arise.
Let's say a simple thing, like brush goes out in an electric motor used to power one of the many water pumps in the system goes bad. Procedures need to be followed to the letter and documented extensively. There is nothing that goes on in the plant that isn't recorded.
I'm getting way off topic. The point I'm trying to make is that ANY procedure involving nuclear power and the waste it produces will be studied, tested, documented, and basically beat like a dead horse. Why? 1) Safety is important. 2) The "Public Eye" is watching 3) If the Nuclear industry doesn't get it right the first time, they won't ever have a second chance. This includes transportation of waste to Yucca, and the handling and protection of it once it's there.
Perhaps. I've placed too much trust in the NRC (Nuclear Regulatory Commission), the operators, the engineers, and security forces. IMHO, they deserve it as much as they deserve our support.
Nevada is whining because they have the best site available and don't want the waste. So what. The need is greater than Nevada's discomfort; it's time they start seeing that.
Slashdot Mirror
At the SMTP server
At the SMTP Filter Proxy Server or LDA
Just remember to shortcut the process along the way. If email can be dropped or tagged for any reason, do so immediately and quit processing it.
I don't think you get the concept here. Drop the FUD regarding Linux, the GNU GPL, and license contamination. It's unrelated to this topic. Just because a binary is run on Linux doesn't mean you have to expose your precious source code.
An on topic observation: A DRM-enabled kernel would allow commercial distributors of embedded devices or special-purpose systems to use Linux as a possible platform, even if the software they intend to run may only be licensed legally under DRM provisions. They would have to think in terms of using a user-space daemon with one-way hashes or public key encryption for license and CRC validation.
Regardless, it's largely a moot point, which is the gist of this argument. It doesn't matter, really, if Linux is DRM-enabled if the end-user refuses to play by the rules.
I don't feel it necessary to expound on this further. There are plenty of comments here and published papers on writing clean, secure, and efficient code, regardless of which language is used.
...grammar take two.
Of course he is, because that would make the article larger in focus than what he wants, which is to push buttons. C may be a riskier language than some, but even the big-dog interpreted languages have their own problems.
Although S tar is a nice utility, as mentioned by previous posters. You can get pax directly from the OpenBSD people. Debian also packages pax, if you run Linux.
Try out LVM from Sistina. It's part of 2.4.x, why not use it? ;-) In fact, you can "apt-get install lvm10" on Debian Woody. If you want to compile it yourself, it's pretty easy. Sistina did a good job wrapping up the software and kernel patches with a well documented installation procedure. I couldn't live w/o my LVM.
Now, I haven't looked at the code yet, but I'm assuming that the python script is simply a wrapper around an HTTP server, a MD5 or SHA1 hashing algorithm (for the filename), tar, bzip2, and some meta database to keep an index of files backed up.
That being said, it's not a bad concept. If you have a trusted friend that will allow you to back up files on his machine, this wrapper should operate nicely. Otherwise, you can always do it by hand.
Not so novel idea: use a dedicated snort box that uses its alerts to add rules to a firewall, switch, and/or router. Check out the snort FAQ and contrib directory for ideas.
Better yet. Replace pieces of the app with C/C++ modules that can be accessed from within python. Use C/C++ only as a faster-implementation of a module.
Pumps are only expensive because the include nice housings, some logic circuits to control the motors, and include service warranties. It shouldn't be too hard to construct your own parastolic pump with some very simple components: a 12 volt battery, a small electric motor ($3.50 at radio shack, though you *may* want a little more power), some ABC plastic for the disk that holds the rollers, the rollers themselves, tubing, screws, and a piece of wood to carve the 1/3rd circle track. If they're already wiring up relays for the controller board, purchase electronic solenoid valves for fluid control. The system is less complicated, fewer moving parts, and more robust.
...the variants of programming languages available to us. Just when you think you've got your "favorite" language pegged, along comes another to tempt and tease you. Just check out the dmoz.org site out for a list of programming languages. It's nuts!
FYI, Red Hat != Linux. Red Hat == Linux Distribution Please use the correct reference.
I hope that helps. If you are ever in need of info, always hit Google.
IANAL, and IANAE, but here goes...
Is your employer's concern over licensing or over distributing source code? Linus Torvolds has made it quite clear that the Linux kernel has provisions for commercial modules, given that the module follows the API requirements, such as providing a COPYRIGHT notice in the initialization structure of the module itself. Proprietary modules are not an issue with it comes to the kernel.
I believe that any changes you make to the kernel itself in order to integrate your module need to be GPL'ed. For example, let's say you have to make a patch to the Block IO API in order to enable some functionality of your driver. That patch must be GPL'ed. Your driver is still safe.
If your company is concerned with distributing source code, regardless of license, then you have another problem. In that case, you will be in the business of compiling kernels if you don't want to deal with a physical module. Compiling kernels and distributing them, which you have all the rights in the world to do, adds overhead to your company's responsibilities that they probably don't want.
So, when it comes to the Linux kernel, you should talk to Linux kernel developers and Linus Torvalds or his legal team in particular. Remember that the Linux kernel does have provisions for legal use of commercial and proprietary licensed drivers. Looking only at the GPL is not enough in this case.
If you would like to see a nice comparison of VCD, DVD, and DivX recordings, take a look at the grid table at http://www.vcdhelp.com/comparison. This site has lots of good info, a definite bookmark!
My general attitude about such software is to give it a shot, but only in a chroot jail. This particular software, the Folding@home binary, has been unsuccessful in running under such an environment. It will start running, but it won't retrieve data sets from the servers.
Whose loss is this? Not mine. If I can't get something running in a satisfactory environment after spending a reasonable amount of my valuable time on it, I don't run it.
Has anyone had success in running the Linux binary in a chroot?
Let's apply this knowledge. For example, Red Hat doesn't necessarily sell software. It distributes the software and charges for the distribution. The software is otherwise freely available for download in source code or binary format. Customers are buying service and convenience, not software. This may be walking a thin line legally, but thin lines have been known to be found in favor of defendants more often than procecutors.
DOH, correction. It seems to me that clamscan (Clam Antivirus command line scanner) is the one doing the multithreading, not amavis. Very cool tool. Check it out if you have a chance, people.
Clam Antivirus is a virus scanner, a command-line tool used to scan files for virus signatures. It will report whether it finds a virus or not. AmaVis is used as a filtering daemon for email. It unpacks MIME messages into multiple files, decompressing them if necessary, and runs the virus scanner over each file. If it finds a virus with its tools, it reports the results to the following (configurable, of course): the admin, the sender (I shut this off because of spoofing), and the receiver (you can shut off alerts sent to recipients that are off-site). The entire email is saved in a quarantine directory; it is not deleted.
The virus definitions file is updated by the members of the Open Antivirus project. Subscribe to their email list to get bleeding-edge, just found definitions. Otherwise, just let the clam antivirus updater fetch the definitions when the project updates them (1-2 day delay after a new virus is identified -- or at least it seems that way). Talk to the OpenAV guys for legit frequency info.
The only reason why we don't go with a commercial product is because most of these products charge by the number of recipients or users on the system, often requiring client licenses for each user as well. McAffee wanted WAY too much money for what we wanted to do, especially considering that we already have Nortan Antivirus installed on the Windows and Mac machines (University site license). Why pay for something we already have?
To date, I haven't seen a virus come through amavis+clamav yet, but that is my own personal experience and that of our users.
Spamassassin is a different beast entirely. I use procmail scripts to intercept messages bound for email lists (served from ecartis) and filter them for spam. I also filter out VIRUS warnings sent by AmaVis. These filtered items get saved to a "spam" and a "virus" folder, and I wrote a cron job to report how many emails it finds in these folders twice per day. It's valuable to send these to individual recipients on the system, but not to a list.
Procmail is an important piece of the spam filtering process. Postfix can do content filtering, so I think it's certainly possible for me to have spamassassin tag EVERYTHING coming into the system, just like AmaVis does now. I just haven't pieced together how to do it yet. It would eliminate the need for users to run procmail recipes and drop the number of processes run on the server.
If you want to disinfect files, go commercially funded/grown software... that is, at least until the Open Antivirus people or another group come up with virus definition files that include instructions for disinfecting files.
AmaVis: Antivirus filtering daemon; packaged by most linux distros; multi-threaded (recognized multiple CPU's); sends out email alerts; very configurable; supports many antivirus scanners; works well with postfix; written in Perl; GPL
Clam Antivirus (clamav): virus scanner; written in C; fast; virus definition update tool included; uses virus definitions from the Open Antivirus project; (does not disinfect, just identifies); GPL
SpamAssassin: Perl-based Spam filter; use with Procmail; client-server architecture (one daemon); Perl Artistic License
Our application of the above software seems to work quite well. We server about a thousand users (about 100 "heavy users"), and the average server load rarely gets above 0.21 with a Dual AMD 1500+ MP that provides SMTP, IMAP, and POP all w/SSL enabled.
This sounds interesting, but IANA Geologist. It also sounds risky, trusting in the good faith of "Mother Nature" not to do anything drastic, like push up a new mountain range and bring all of the "not yet consumed" nuclear waste. Do we really want to meddle with an obviously powerful natural force as plate tectonics? Again, IANAG, so I have no answers, only questions regarding your suggestion.
IMHO, Yucca Mt. is the US's best viable solution to the Nuclear waste disposal problem. People who are not familiar with the daily operation of Nuclear power don't really understand the massive amounts of training and beurocracy that surrounds it. Plant operators undergo more testing and requalification training than any airplane pilot. It's a stressful job, but the knowledge that they possess is extremely important when emergencies or problems arise.
Let's say a simple thing, like brush goes out in an electric motor used to power one of the many water pumps in the system goes bad. Procedures need to be followed to the letter and documented extensively. There is nothing that goes on in the plant that isn't recorded.
I'm getting way off topic. The point I'm trying to make is that ANY procedure involving nuclear power and the waste it produces will be studied, tested, documented, and basically beat like a dead horse. Why? 1) Safety is important. 2) The "Public Eye" is watching 3) If the Nuclear industry doesn't get it right the first time, they won't ever have a second chance. This includes transportation of waste to Yucca, and the handling and protection of it once it's there.
Perhaps. I've placed too much trust in the NRC (Nuclear Regulatory Commission), the operators, the engineers, and security forces. IMHO, they deserve it as much as they deserve our support.
Nevada is whining because they have the best site available and don't want the waste. So what. The need is greater than Nevada's discomfort; it's time they start seeing that.
http://www.openssh.com/press.html