Spammers interested in using SPF will also implement DKIM. SPF and DKIM aren't about preventing spam, they are about authenticating the sender and using that to build a reputation.
Also, stop using a catchall account. Spammers will dictionary attack your domain and you will continue to get more and more spam for the rest of your life. Instead of *@yourdomain.com, set it up so your catchall accounts look like username-*@yourdomain.com.
Actually you should probably have spam assassin use SPF and DKIM instead. It's actually designed to do stuff like "no mail from someone@aol.com unless the spf record says their sending IP is allowed to". Plus, the ISP's configure it so you don't have to. It's an industry standard.
Spamassassin has all that stuff built into it already. You may need to install some SPF and DKIM perl modules.
In my work environment, we don't support linux on the desktop. We allow it, but we don't spend time making images, testing it with our hardware and software, etc. Our server infrastructure is 90% unix based.
I can ghost a Windows image onto one of our windows machines and have a person up and running on it in under 30 minutes. If there is a problem, we have a knowledge base full of solutions. Our infrastructure automatically deploys patches and updates. Employees that are supposed to be working spend very little time working on fixing their PCs.
The people that use linux on their desktop take 2-3 hours to install it. They have to think about maintaining their PC's and applying patches and figuring out what software works. For the 1-2% of the organization that runs !Windows on their desktop, there is no reason to put any time into supporting it.
We could just as easily standardize on Linux for our desktops and set up the support infrastructure for it and it would be people who wanted to run windows that would be spending time maintaining their systems. But we don't.
Interestingly, there is one terminal emulator that every developer or sysadmin runs to interface with the linux boxes: xterm. The windows developers get an install of cygwin and run xterm. The people with linux on their desktops are also running xterms. If someone needs to run a specific X-based, linux gui they are probably going to use it over ssh X-forwarding to their desktop, regardless of the OS they are running.
All of your thoughts are valid, but probably not in this case.
Sure, you've worked with all sorts of huge 100GB+ db's. It's really easy to say what a table scan of a 100m table can take on mysql. Again, I'm sure this poster isn't looking for db's in the 100gb scale, especially if he's asking slashdot.
I'm betting a simple solution that will meet the needs of the poster using mysql can be done very quickly and cheaply. I'm also sure that you would end up spending around the same amount on oracle or db2 if the data ever needed to scale up to the 100's of gb range. That's probably not the case here, but I guess we'll never know if the poster doesn't give more details.
I'm guessing the administration overhead of oracle or db2 would make it not worth while. Due to the fact he's asking slashdot about his problem, the expensive db's are probably out of reach for him. This is especially true because he says "cost matters a lot."
I made the assumption that the reporting application already exists and runs against the current schema. If it doesn't, ETL is an option with MySQL just as much as any other database. Again, I doubt it will be a problem if cost matters -- they probably have very simple reporting needs.
60k queries/day is less than 1 per second. Even 6-7 times that is still only 6-7 a second. MySQL 5.1 has partitioning. MySQL was originally designed for data warehousing applications (that is what the isam table structure was all about) and there are many documented > 100GB databases in existence.
If mysql is working for you now, you should look at mysql scaling options. For example, if you are worried about reporting queries, replicate the database to a second machine for running the reports against. Mysql replication works great for this sort of application. Also, if your dialer application is only performaning read queries, you can spread those across replicas too.
Knowing the current 'size' of your database would help -- if it's a dual processor box with 1 or 2 gb of ram, there are still a few affordable forklift upgrades before you need to worry too much about one box or mysql's performance (assuming your indexes are set right).
Also, MySQL Cluster was designed by/for the telecomm industry -- the original commissioners were performaning analysis on call records or something of the such.
MySQL can definitely do whatever you want it to. Why switch?
A CPAN id is your username for uploading stuff onto CPAN.
I would say it depends on the level and expectations of the job. As a senior perl programmer, I probably wouldn't hire you because you're not involved in the culture of perl. You do know what CPAN is and were able to take a decent guess on what a CPAN id is, so I probably would hire you.
I think it was one of the Live Journal guys at OScon that said, "If your server starts to swap, you've lost the battle".
With all of our 64bit 4GB of ram minimum hosts floating around, there is no longer a point to having swap -- if you server really is swapping, it's under a huge load and the io is making the problem worse. Let the OS kill a few processes to get it back under control
Telco's in Canada are still mostly a monopoly. Most people only have one choice (Telus in the West, Bell in the East). It's slowly changing. If they tried looking, they'd find that they could probably get a Bell line in (there are CLEC/ILEC type agreements in place, I believe) or maybe even Sprint.
Canada is even more helpless as far as Telcos and Cable goes than in the US. While we have better coverage for highspeed internet, you typically can choose Cable from Shaw and DSL from Telus (in the west, in the east it'll probably be Rogers and Bell). There is no competition. There are only three real options for cell phone providers as well -- Telus, Bell and Rogers. There used to be more, but not any more -- they keep merging into fewer companies.
Comparing the T series to an Inspirion is like comparing apples (not Apples) to twinkies. Inspirion laptops are the Dell consumer grade machines. They are flimsy and heavy compared the Dell Latitudes.
Latitudes are nice machines, though not nearly as nice as the T series:)
Normal PBX systems can cost in the $10k amounts to do half of what a $5k Asterix system can. Plus, if you are truly a geek, you can setup your own home PBX off normal phone lines.
Wow, what does 10K get you in the world of phone systems (outside of asterisk)? The last install I did was $80K *before* handsets.. That was for about 40 people -- expandible to 80ish..
I've had the pleasure of seeing him speak twice. The first time was around '99 at UBC at some sort of thing ActiveState was putting on. At that point it was just plain cool to see the guy who publishes all the excellent books I'd been buying.
The next time was at the MySQL conference last April. After he did a keynote, I went up and talked to him. He had a few William Gibson references in his presentation. I asked him about that and we chatted for a few minutes. We exchanged business cards and that was about it. I was hoping to get him to sign his book of essays which I had picked up earlier at the conference, but I forgot in the middle of the conversation..
As lame as it may seem, it was pretty cool to get to talk to one of my heros. He's quite the optimitic dreamer and does it without an invasive corporate vision.
It's hard to believe that oreilly only has 15% of the tech book market -- 80% of my library is oreilly.
Is there anything OSS that offers WebDAV, Apache support, BSD/Linux support and Active Directory-LDAP authentication with support for Windows and Mac clients?
Doesn't Apache + mod_webdav + auth_ldap support all of this? Can't you just point any webdav client at apache and have web based file storage?
If you want normal people to access it, put up a web page with instructions on how to access it.
We're using a Wiki for this sort of thing. In particular, we are using TWiki. I'm sure other wiki's have the features we are using.
We have a template for adding new hosts and we use the %SEARCH{}% stuff to generate tables of devices. We even use the %CALENDAR{}% plugin to map out when events happen.
NFS sucks. Use something like CVS to keep your webroots in sync and have each server host it's own copy of the content.
You get to get out a massive single point of failure (the NAS) and you get a little closer to linear scalability (adding another webserver doesn't put more load on your NFS box).
That wasn't really an option. They were in a 2U chassis that was already loaded up with fans.
My personal experience with WD has been terrible. I swore off the drives many years ago, but it was hard to turn down 10K RPM drives. We took a risk getting WD drives and it didn't pan out. We've since (using 3ware's CLI) swapped out all of the WD drives for Seagates. Again, no down time, plus we were going to larger drives.
We had 3 failures of WD drives in the first 3 months of operation. It's been 4 and no seagate drives have failed.
WD's website says:
Enterprise storage systems built using WD's new 10,000 RPM Enterprise Serial ATA hard drive are highly reliable, fast, low-cost systems that meet the budget and storage needs of today's IT manager.
I figured "highly reliable" "enterprise serial ATA" drives could live in a typical "Enterprise" Chassis in a well air conditioned colo center.
In retrospec, we didn't notice any speed/performance loss when we went to the 7200RPM drives.
I agree. 3ware cards are great. They have excellent linux support and an excellent reputation. I use them under linux and windows.
Two areas you may run into problems with:
1) The cards require a lot of power and riser cards can be troublesome.
2) They aren't the fastest cards on the market. They do use a custom PATA=>SATA bridge, even on the 9500 cards. That being said, they are still blazingly fast and reliable.
I'd recommend getting the 9500 boards. The 8500's didn't support onboard cache and there is a significant speed improvement on the 9500 boards.
We have a 9500 in our db server. We were using the Western Digital raptor drives. It is a RAID10 of 6 drives. Over the course of 8 months, we lost 12 drives. That's right, 12 drive failures. Through this, we didn't experience a single second of downtime.
1) Install Cygwin X server 2) Start Cygwin X Server 3) Run a local window manager (the scripts are pretty much set up to do this out of the box) 4) Open an xterm 5) ssh to your remote server 6) type in xclock &
There is a big difference between SSO and having a central account database.
Single Sign On means your enter your username/password once and you can connect to any resources you are allowed to.
For example, you'd log into your Windows XP PC. Without entering your password again, you can browse other computers in the AD Domain. You start Outlook and you never have to enter your login info for your exchange server. From there, you point IE at your intranet server and it uses the kerberos ticket to log you in without a password.
In the unix world, it means getting a ticket with kinit. From there, ssh, ftp, mozilla and so forth should be smart enough to not require you to log in again.
RH's support isn't $349/system/year good. Out of the 3 tickets I've opened with them, they've closed 1. 2 I straight up gave up on and found my own work around -- google groups was more affective.
I pay far less for Cisco TAC on my switches. I've experience much more comprehensive support from Cisco.
That being said, most administrators I know are more confortable running Centos. It's free, the packages are high quality and they aren't depending on people in India for support. Most of the servers that run linux aren't worth $349.
You might also want to check out Ebay for some Cisco LRE stuff. There was a video floating around where John Chambers was doing video conferencing with LRE over speaker wire, phone wire, barbed wire and a wire fence -- the really impressive part was that it was 2 or 3 times longer than the max recommended distance.
That's only a problem if you're sending from the account that is getting spam. A lot of people have 'catchall' style domains -- messages sent to *@domain funnel to one pop account. This is brutal when dealing with spam -- a lot of spammers run dictionary attacks at domains and this just populates their lists with your addresses.
Probably the only thing you can do is drop the addresses that are getting the bounces. In fact, it's best to configure the mail server to deny those addresses at the SMTP level and not let it re-queue another bounce to the bounce. Well, in this situation anyway.
Slashdot Mirror
Spammers interested in using SPF will also implement DKIM. SPF and DKIM aren't about preventing spam, they are about authenticating the sender and using that to build a reputation.
Coincidently, today is the second day of the Authentication and Online Trust Alliance Summit: http://www.aotalliance.org/summit2007/.
Also, stop using a catchall account. Spammers will dictionary attack your domain and you will continue to get more and more spam for the rest of your life. Instead of *@yourdomain.com, set it up so your catchall accounts look like username-*@yourdomain.com.
Actually you should probably have spam assassin use SPF and DKIM instead. It's actually designed to do stuff like "no mail from someone@aol.com unless the spf record says their sending IP is allowed to". Plus, the ISP's configure it so you don't have to. It's an industry standard.
Spamassassin has all that stuff built into it already. You may need to install some SPF and DKIM perl modules.
In my work environment, we don't support linux on the desktop. We allow it, but we don't spend time making images, testing it with our hardware and software, etc. Our server infrastructure is 90% unix based.
I can ghost a Windows image onto one of our windows machines and have a person up and running on it in under 30 minutes. If there is a problem, we have a knowledge base full of solutions. Our infrastructure automatically deploys patches and updates. Employees that are supposed to be working spend very little time working on fixing their PCs.
The people that use linux on their desktop take 2-3 hours to install it. They have to think about maintaining their PC's and applying patches and figuring out what software works. For the 1-2% of the organization that runs !Windows on their desktop, there is no reason to put any time into supporting it.
We could just as easily standardize on Linux for our desktops and set up the support infrastructure for it and it would be people who wanted to run windows that would be spending time maintaining their systems. But we don't.
Interestingly, there is one terminal emulator that every developer or sysadmin runs to interface with the linux boxes: xterm. The windows developers get an install of cygwin and run xterm. The people with linux on their desktops are also running xterms. If someone needs to run a specific X-based, linux gui they are probably going to use it over ssh X-forwarding to their desktop, regardless of the OS they are running.
All of your thoughts are valid, but probably not in this case.
Sure, you've worked with all sorts of huge 100GB+ db's. It's really easy to say what a table scan of a 100m table can take on mysql. Again, I'm sure this poster isn't looking for db's in the 100gb scale, especially if he's asking slashdot.
I'm betting a simple solution that will meet the needs of the poster using mysql can be done very quickly and cheaply. I'm also sure that you would end up spending around the same amount on oracle or db2 if the data ever needed to scale up to the 100's of gb range. That's probably not the case here, but I guess we'll never know if the poster doesn't give more details.
I'm guessing the administration overhead of oracle or db2 would make it not worth while. Due to the fact he's asking slashdot about his problem, the expensive db's are probably out of reach for him. This is especially true because he says "cost matters a lot."
I made the assumption that the reporting application already exists and runs against the current schema. If it doesn't, ETL is an option with MySQL just as much as any other database. Again, I doubt it will be a problem if cost matters -- they probably have very simple reporting needs.
60k queries/day is less than 1 per second. Even 6-7 times that is still only 6-7 a second. MySQL 5.1 has partitioning. MySQL was originally designed for data warehousing applications (that is what the isam table structure was all about) and there are many documented > 100GB databases in existence.
Hey,
If mysql is working for you now, you should look at mysql scaling options. For example, if you are worried about reporting queries, replicate the database to a second machine for running the reports against. Mysql replication works great for this sort of application. Also, if your dialer application is only performaning read queries, you can spread those across replicas too.
Knowing the current 'size' of your database would help -- if it's a dual processor box with 1 or 2 gb of ram, there are still a few affordable forklift upgrades before you need to worry too much about one box or mysql's performance (assuming your indexes are set right).
Also, MySQL Cluster was designed by/for the telecomm industry -- the original commissioners were performaning analysis on call records or something of the such.
MySQL can definitely do whatever you want it to. Why switch?
A CPAN id is your username for uploading stuff onto CPAN.
I would say it depends on the level and expectations of the job. As a senior perl programmer, I probably wouldn't hire you because you're not involved in the culture of perl. You do know what CPAN is and were able to take a decent guess on what a CPAN id is, so I probably would hire you.
I think it was one of the Live Journal guys at OScon that said, "If your server starts to swap, you've lost the battle".
With all of our 64bit 4GB of ram minimum hosts floating around, there is no longer a point to having swap -- if you server really is swapping, it's under a huge load and the io is making the problem worse. Let the OS kill a few processes to get it back under control
Telco's in Canada are still mostly a monopoly. Most people only have one choice (Telus in the West, Bell in the East). It's slowly changing. If they tried looking, they'd find that they could probably get a Bell line in (there are CLEC/ILEC type agreements in place, I believe) or maybe even Sprint.
Canada is even more helpless as far as Telcos and Cable goes than in the US. While we have better coverage for highspeed internet, you typically can choose Cable from Shaw and DSL from Telus (in the west, in the east it'll probably be Rogers and Bell). There is no competition. There are only three real options for cell phone providers as well -- Telus, Bell and Rogers. There used to be more, but not any more -- they keep merging into fewer companies.
Comparing the T series to an Inspirion is like comparing apples (not Apples) to twinkies. Inspirion laptops are the Dell consumer grade machines. They are flimsy and heavy compared the Dell Latitudes.
:)
Latitudes are nice machines, though not nearly as nice as the T series
Wow, what does 10K get you in the world of phone systems (outside of asterisk)? The last install I did was $80K *before* handsets.. That was for about 40 people -- expandible to 80ish..
I've had the pleasure of seeing him speak twice. The first time was around '99 at UBC at some sort of thing ActiveState was putting on. At that point it was just plain cool to see the guy who publishes all the excellent books I'd been buying.
The next time was at the MySQL conference last April. After he did a keynote, I went up and talked to him. He had a few William Gibson references in his presentation. I asked him about that and we chatted for a few minutes. We exchanged business cards and that was about it. I was hoping to get him to sign his book of essays which I had picked up earlier at the conference, but I forgot in the middle of the conversation..
As lame as it may seem, it was pretty cool to get to talk to one of my heros. He's quite the optimitic dreamer and does it without an invasive corporate vision.
It's hard to believe that oreilly only has 15% of the tech book market -- 80% of my library is oreilly.
Is there anything OSS that offers WebDAV, Apache support, BSD/Linux support and Active Directory-LDAP authentication with support for Windows and Mac clients?
Doesn't Apache + mod_webdav + auth_ldap support all of this? Can't you just point any webdav client at apache and have web based file storage?
If you want normal people to access it, put up a web page with instructions on how to access it.
We're using a Wiki for this sort of thing. In particular, we are using TWiki. I'm sure other wiki's have the features we are using.
We have a template for adding new hosts and we use the %SEARCH{}% stuff to generate tables of devices. We even use the %CALENDAR{}% plugin to map out when events happen.
NFS sucks. Use something like CVS to keep your webroots in sync and have each server host it's own copy of the content.
You get to get out a massive single point of failure (the NAS) and you get a little closer to linear scalability (adding another webserver doesn't put more load on your NFS box).
My personal experience with WD has been terrible. I swore off the drives many years ago, but it was hard to turn down 10K RPM drives. We took a risk getting WD drives and it didn't pan out. We've since (using 3ware's CLI) swapped out all of the WD drives for Seagates. Again, no down time, plus we were going to larger drives.
We had 3 failures of WD drives in the first 3 months of operation. It's been 4 and no seagate drives have failed.
WD's website says:
I figured "highly reliable" "enterprise serial ATA" drives could live in a typical "Enterprise" Chassis in a well air conditioned colo center. In retrospec, we didn't notice any speed/performance loss when we went to the 7200RPM drives.
I agree. 3ware cards are great. They have excellent linux support and an excellent reputation. I use them under linux and windows.
:)
Two areas you may run into problems with:
1) The cards require a lot of power and riser cards can be troublesome.
2) They aren't the fastest cards on the market. They do use a custom PATA=>SATA bridge, even on the 9500 cards. That being said, they are still blazingly fast and reliable.
I'd recommend getting the 9500 boards. The 8500's didn't support onboard cache and there is a significant speed improvement on the 9500 boards.
We have a 9500 in our db server. We were using the Western Digital raptor drives. It is a RAID10 of 6 drives. Over the course of 8 months, we lost 12 drives. That's right, 12 drive failures. Through this, we didn't experience a single second of downtime.
I'm yet another satisfied 3ware user
I've heard that gold is softer, so the connections mold together better...
1) Install Cygwin X server
2) Start Cygwin X Server
3) Run a local window manager (the scripts are pretty much set up to do this out of the box)
4) Open an xterm
5) ssh to your remote server
6) type in xclock &
Like, say, a laptop AC Adapter?
There is a big difference between SSO and having a central account database.
Single Sign On means your enter your username/password once and you can connect to any resources you are allowed to.
For example, you'd log into your Windows XP PC. Without entering your password again, you can browse other computers in the AD Domain. You start Outlook and you never have to enter your login info for your exchange server. From there, you point IE at your intranet server and it uses the kerberos ticket to log you in without a password.
In the unix world, it means getting a ticket with kinit. From there, ssh, ftp, mozilla and so forth should be smart enough to not require you to log in again.
RH's support isn't $349/system/year good. Out of the 3 tickets I've opened with them, they've closed 1. 2 I straight up gave up on and found my own work around -- google groups was more affective.
I pay far less for Cisco TAC on my switches. I've experience much more comprehensive support from Cisco.
That being said, most administrators I know are more confortable running Centos. It's free, the packages are high quality and they aren't depending on people in India for support. Most of the servers that run linux aren't worth $349.
If I were RH, I'd be afraid.
out.
That's only a problem if you're sending from the account that is getting spam. A lot of people have 'catchall' style domains -- messages sent to *@domain funnel to one pop account. This is brutal when dealing with spam -- a lot of spammers run dictionary attacks at domains and this just populates their lists with your addresses.
Probably the only thing you can do is drop the addresses that are getting the bounces. In fact, it's best to configure the mail server to deny those addresses at the SMTP level and not let it re-queue another bounce to the bounce. Well, in this situation anyway.