You can't mod down one comment 30 times. Nor can one comment be modded down so many times you get banned. If you have a better explanation behind it, I'm all ears,
Perhaps you were just being a douchebag? Just checking...
The binary package management really says it all.. you shouldn't be running anything but compiled source on a performance cluster.
Wow - how many performance clusters do you run again?
Not that I run a "performance cluster" as such - but I do run a bunch of machines that are very busy, all on Debian.
You know what? We compile the couple of programs where CPU is the bottleneck from source. We also compile Cyrus IMAP from source because we apply a pile of patches, but if someone else was packaging up all those patches in upstream, I'd be happy for them to be compiled there. Disk IO is the issue with Cyrus, and a custom compile won't help with that.
Yeah, we build our own kernels as well - that's another point that's worth the effort to customise./bin/ls though? I don't think it matters to anyone on a high performance cluster. Just so long as the cluster apps are optimised then the rest is just noise - better to have a system that's less work for your administrators so they can concentrate on what's important.
Yeah, we have a wodge of perl that sits there scanning logs, getting stats out of the policy daemon (Postfix policy daemon is very handy) and generally making educated guesses as to the probability that a site really is nothing but a spam source. Once it's satisfied, they go into the early block list which doesn't even get a reverse DNS lookup, just accept and then drop. We're thinking of moving to accept and teargrub actually, assuming we can do it in a way that doesn't overload the machine with idle processes.
The biggest is connecting and then failing to complete an SMTP transaction. There's also enumeration and some heuristics based on reverse lookups and netblocks. I don't know how much of it's readily sharable, it's all internal stuff that we put together over the years at FastMail, and like most of our internal systems, it has deep and ugly hooks into everything. We use open source packages, and contribute quite a bit back to those projects, but I doubt anyone wants to see the perl duct tape that holds it all together, and we don't really want to share it all because unfortunately security-through-obscurity still helps in the anti-spam arms race!
Our mx servers have a list of over a million machines which are blocked from talking SMTP to us for three days thanks to past bad behaviour. In a single hour nearly 200,000 of them tried multiple SMTP connection attempts.
Personally I use rsync for backing up both my Windows and Linux clients with BackupPC - but yeah. That.
Linux isn't exactly perfect at non-fiddly exact rebuilds either... it's very easy on a home machine to get layers of legacy cruft that are hard to reproduce. At least all my production Linux systems are a scripted rebuild (10 mins approx) away from perfect recreateability - but with that comes the expense of making all changes through the change management system first, so it's slower than just logging in and fiddling crap.
Good Windows administrators have similar reproduction techniques for their servers (no, I'm not debating that there are a LOT of bad Windows admins out there. There are quite a few bad Linux admins too)
Sure. Just don't buy cheap arse shit systems that will expose that it's more brittle when things are unreliable. You can lose the data on any filesystem, reiserfs will just take the entire tree with it.
And reiserfsck --rebuild-tree will shit itself if you have reiserfs images stored in your filesystem.
That said, it's a tool that has its place, and it's a bloody good filesystem for storing large amounts of email on big _reliable_ drive units without your performance sinking through the floor.
(believe me, the other "poster child" filesystems have their own issues too)
I'm hanging out for btrfs to stabilise though. I already run my local maildirs on it, and am quite happy.
Definitely - and the problem is that there really are only two protocols, POP3 and IMAP4, in anything like wide use.
Hotmail/Microsoft had a chance to do something great with their Outlook protocol, but being Microsoft they were more concerned with keeping it proprietary than with making it useful to the world at large by freeing it.
There are enough things out there now (Gmail and Opera's client being the ones I know best) looking at a less folder-centric view of the world that it would be good to have a way to share that view between systems. At the moment everything just tries to shoehorn things that don't really fit onto IMAP and you get the same information loss that you get with POP3 fetching and webmail talking to it when you're not at home - duplication of effort and loss of flagging.
I haven't talked much publically about my eventual design goals for my own mail server (I've been wanting to write one for a long time!) but one of the main things is an "embedded mode", where there's a replica of your mailbox running locally on your own computer, talking a much richer protocol back to the mail server, and talking IMAP or POP3 or protocol de jure to your client program. Sure it takes more space this way, but it would be such a fast experience for users, with reliable offline mode and backups.
Also, a larger embedded mode where an organisation could install a local mail server but still have it fully replicated to a larger ISP or mail provider company who provided incoming MX, outbound queuing, spam and antivirus, etc - but without the user seeing the lag times to connect to a machine potentially on the other side of the world, and not making the mailstore useless when the network link to the world was down.
But I ramble... I haven't actually implemented much of this yet, and plenty is still in the "ideas bouncing around my head" stage.
Re:Warning: Gmail IMAP support is ASCII only!!!
on
Free IMAP On Gmail
·
· Score: 1
I think this move certainly does put a lot of pressure on us, good IMAP support has been one of our long term advantages which this clearly impacts.
Which really leaves support, features, reliability, security, privacy and openness as things we have to concentrate on.
(Rob is my boss)
Re:Warning: Gmail IMAP support is ASCII only!!!
on
Free IMAP On Gmail
·
· Score: 1
Not directly, but we've mainly been holding off doing anything with sieve because Ken has been rewriting some of the underlying code and we're waiting until that stabilises.
Re:Warning: Gmail IMAP support is ASCII only!!!
on
Free IMAP On Gmail
·
· Score: 2, Interesting
Hey, fantastic to see a response. Even though you guys are opening up right into a major segment of our customer base (I work for FastMail.FM, and good IMAP access is one of our selling points, so much so that I spend a lot of my time enhancing and bugfixing Cyrus), I'm glad that IMAP is an option for Gmail users, because IMAP is an all round better protocol than POP3 for serious use (and people are less likely to lose email if the only copy isn't on a flaky laptop hard drive somewhere).
That said, IMAP doesn't map well to gmail's style of doing things already, and it's also less of a fit to how people would _like_ to use email generally. Single mailbox IDLE, no "submit" command that both sends an email out and copies to your Sent mailbox, etc.
Do you know if Google has any plans to develop a newer protocol, and if so if you'd be willing to share it so a larger base of implementations could develop around it? Unfortunately I have both a young family and a non-existant travel budget so I can't easily get to the conferences, but I'm really interested in improving mail access protocols to keep non-centralised email relevant in these days of Facebook and similar services sucking users into them.
(yes, I do work for FastMail - was wondering if we'd get mentioned in this thread)
Oh - and we're responsible for most of the bugfixing that's happened in the past few releases of Cyrus thanks to being early adopters and thanks to me spending far too much time reading C code for my sanity.
I read up to 10% of paternity tests come out with a different father
Yeah, but that's the cases where there's enough suspicion that they run the test. I'm surprised that the figure is so low in that case.
Re:Wrong: A good RBL is worth its weight in gold.
on
Choosing a Good DNSBL
·
· Score: 1
I was going to see who you are (since I'm a sysadmin for an Australian email provider as well) but your website links to a photo which links to a redirection to an HTTPS SVN repository that tells me to get fucked in no uncertain terms. Hmm.
I'm more inclined to believe the people who have actually done a diff between the COPYING file in the linux source tree and the GPL as shipped by the FSF which shows that Linus hasn't modified anything other than a clarification at the top to remind people that he didn't add the optional "or later version from the FSF" wording when he applied the licence change to GPL way back when and doesn't intend it either.
But feel free to believe Groklaw instead. There's a lot of good stuff there, but the GPL3 fanboyism is a bit of a crock.
I guess you've never been guilty of flying while ethnic, or flying while being shifty looking, or (heaven forbid) flying while having a similar name to someone on the s00p3r s3kr1t watch list.
Slashdot Mirror
You can't mod down one comment 30 times. Nor can one comment be modded down so many times you get banned. If you have a better explanation behind it, I'm all ears,
Perhaps you were just being a douchebag? Just checking...
Your profile page - I only see one thing modded down and that was: this one. Yep, douchebaggery.
Care to post the other username (assuming there really is one) with all the downmodded comments so we can pass judgement on you?
The binary package management really says it all.. you shouldn't be running anything but compiled source on a performance cluster.
/bin/ls though? I don't think it matters to anyone on a high performance cluster. Just so long as the cluster apps are optimised then the rest is just noise - better to have a system that's less work for your administrators so they can concentrate on what's important.
Wow - how many performance clusters do you run again?
Not that I run a "performance cluster" as such - but I do run a bunch of machines that are very busy, all on Debian.
You know what? We compile the couple of programs where CPU is the bottleneck from source. We also compile Cyrus IMAP from source because we apply a pile of patches, but if someone else was packaging up all those patches in upstream, I'd be happy for them to be compiled there. Disk IO is the issue with Cyrus, and a custom compile won't help with that.
Yeah, we build our own kernels as well - that's another point that's worth the effort to customise.
Yeah, we have a wodge of perl that sits there scanning logs, getting stats out of the policy daemon (Postfix policy daemon is very handy) and generally making educated guesses as to the probability that a site really is nothing but a spam source. Once it's satisfied, they go into the early block list which doesn't even get a reverse DNS lookup, just accept and then drop. We're thinking of moving to accept and teargrub actually, assuming we can do it in a way that doesn't overload the machine with idle processes.
The biggest is connecting and then failing to complete an SMTP transaction. There's also enumeration and some heuristics based on reverse lookups and netblocks. I don't know how much of it's readily sharable, it's all internal stuff that we put together over the years at FastMail, and like most of our internal systems, it has deep and ugly hooks into everything. We use open source packages, and contribute quite a bit back to those projects, but I doubt anyone wants to see the perl duct tape that holds it all together, and we don't really want to share it all because unfortunately security-through-obscurity still helps in the anti-spam arms race!
Our mx servers have a list of over a million machines which are blocked from talking SMTP to us for three days thanks to past bad behaviour. In a single hour nearly 200,000 of them tried multiple SMTP connection attempts.
Yes, I'd believe those numbers.
By "Joe Sixpack" you mean cute innocent looking girl of course.
Personally I use rsync for backing up both my Windows and Linux clients with BackupPC - but yeah. That.
Linux isn't exactly perfect at non-fiddly exact rebuilds either... it's very easy on a home machine to get layers of legacy cruft that are hard to reproduce. At least all my production Linux systems are a scripted rebuild (10 mins approx) away from perfect recreateability - but with that comes the expense of making all changes through the change management system first, so it's slower than just logging in and fiddling crap.
Good Windows administrators have similar reproduction techniques for their servers (no, I'm not debating that there are a LOT of bad Windows admins out there. There are quite a few bad Linux admins too)
Sure. Just don't buy cheap arse shit systems that will expose that it's more brittle when things are unreliable. You can lose the data on any filesystem, reiserfs will just take the entire tree with it.
And reiserfsck --rebuild-tree will shit itself if you have reiserfs images stored in your filesystem.
That said, it's a tool that has its place, and it's a bloody good filesystem for storing large amounts of email on big _reliable_ drive units without your performance sinking through the floor.
(believe me, the other "poster child" filesystems have their own issues too)
I'm hanging out for btrfs to stabilise though. I already run my local maildirs on it, and am quite happy.
Definitely - and the problem is that there really are only two protocols, POP3 and IMAP4, in anything like wide use.
Hotmail/Microsoft had a chance to do something great with their Outlook protocol, but being Microsoft they were more concerned with keeping it proprietary than with making it useful to the world at large by freeing it.
There are enough things out there now (Gmail and Opera's client being the ones I know best) looking at a less folder-centric view of the world that it would be good to have a way to share that view between systems. At the moment everything just tries to shoehorn things that don't really fit onto IMAP and you get the same information loss that you get with POP3 fetching and webmail talking to it when you're not at home - duplication of effort and loss of flagging.
I haven't talked much publically about my eventual design goals for my own mail server (I've been wanting to write one for a long time!) but one of the main things is an "embedded mode", where there's a replica of your mailbox running locally on your own computer, talking a much richer protocol back to the mail server, and talking IMAP or POP3 or protocol de jure to your client program. Sure it takes more space this way, but it would be such a fast experience for users, with reliable offline mode and backups.
Also, a larger embedded mode where an organisation could install a local mail server but still have it fully replicated to a larger ISP or mail provider company who provided incoming MX, outbound queuing, spam and antivirus, etc - but without the user seeing the lag times to connect to a machine potentially on the other side of the world, and not making the mailstore useless when the network link to the world was down.
But I ramble... I haven't actually implemented much of this yet, and plenty is still in the "ideas bouncing around my head" stage.
http://www.emaildiscussions.com/showpost.php?p=433703
I think this move certainly does put a lot of pressure on us, good IMAP support has been one of our long term advantages which this clearly impacts.
Which really leaves support, features, reliability, security, privacy and openness as things we have to concentrate on.
(Rob is my boss)
Not directly, but we've mainly been holding off doing anything with sieve because Ken has been rewriting some of the underlying code and we're waiting until that stabilises.
Hey, fantastic to see a response. Even though you guys are opening up right into a major segment of our customer base (I work for FastMail.FM, and good IMAP access is one of our selling points, so much so that I spend a lot of my time enhancing and bugfixing Cyrus), I'm glad that IMAP is an option for Gmail users, because IMAP is an all round better protocol than POP3 for serious use (and people are less likely to lose email if the only copy isn't on a flaky laptop hard drive somewhere).
That said, IMAP doesn't map well to gmail's style of doing things already, and it's also less of a fit to how people would _like_ to use email generally. Single mailbox IDLE, no "submit" command that both sends an email out and copies to your Sent mailbox, etc.
Do you know if Google has any plans to develop a newer protocol, and if so if you'd be willing to share it so a larger base of implementations could develop around it? Unfortunately I have both a young family and a non-existant travel budget so I can't easily get to the conferences, but I'm really interested in improving mail access protocols to keep non-centralised email relevant in these days of Facebook and similar services sucking users into them.
We still have better support at FastMail though :)
(yes, I do work for FastMail - was wondering if we'd get mentioned in this thread)
Oh - and we're responsible for most of the bugfixing that's happened in the past few releases of Cyrus thanks to being early adopters and thanks to me spending far too much time reading C code for my sanity.
Except the injected "bad" urchin.js would of course not contain this, hence defeating the whole point...
Wow, you run a porn site too?
I suspect you could get 300 volunteers for that easily enough too...
Wow - how did you flush the caches between those? How did you make sure there were no other programs requiring disk access at the same time? How...
*sheesh*. Very much so. Nicely caught by the disclaimer of course.
I read up to 10% of paternity tests come out with a different father
Yeah, but that's the cases where there's enough suspicion that they run the test. I'm surprised that the figure is so low in that case.
I was going to see who you are (since I'm a sysadmin for an Australian email provider as well) but your website links to a photo which links to a redirection to an HTTPS SVN repository that tells me to get fucked in no uncertain terms. Hmm.
Ah, the blind leading the blind.
I'm more inclined to believe the people who have actually done a diff between the COPYING file in the linux source tree and the GPL as shipped by the FSF which shows that Linus hasn't modified anything other than a clarification at the top to remind people that he didn't add the optional "or later version from the FSF" wording when he applied the licence change to GPL way back when and doesn't intend it either.
But feel free to believe Groklaw instead. There's a lot of good stuff there, but the GPL3 fanboyism is a bit of a crock.
Ahh, that was SCO's "mistake" - asking the question too close to lunchtime. I'll remember to ask after lunch!
No, no.. they would say:
..."
"It depends... we can't commit to any real answer because there's not enough case law to be sure..."
"... but we're happy to keep digging on your dime
Seriously, every time we've asked a lawyer about anything IT related the answer has been a definitive "maybe".
That takes you into the guts of a distribution, but not much further than "make menuconfig" into the kernel itself. Woot.
Not that there's anything wrong with Linux From Scratch, but a deep diving kernel expedition it isn't.
I don't believe you talked about "slash and yaoi" in the same paragraph as you said "just straight sex". I mean, sheesh.
I guess you've never been guilty of flying while ethnic, or flying while being shifty looking, or (heaven forbid) flying while having a similar name to someone on the s00p3r s3kr1t watch list.