I think you're confusing the half-baked challenge solutions of today with what a properly designed solution could do. Yes, C/R is annoying when you have to sift through your mailbox to separate spam from Challenges. When they look like any other E-Mail with not even a standard formatting to identify them. When the procedure varies between clicking a link, replying or even quoting some gibberish text from the mail (oh and don't get it wrong or it won't work), etc.
C/R would be widely accepted if you think more of the way skype does it. A simple dialog box, one click, done. This is the kind of integration I'm thinking of and I'm pretty convinced that even people like Mr.Pogue would happily accept it if it reduces their spam input to zero.
I for one would highly prefer to skim over, at worst, a dozen challenges a day if that saves me from scanning my spam box with thousands of mails regularly to look for false positives...
1. Admins would not have to keep the identity lists at the MTA up to date. The users do that themselves (their MUA interfaces with the MTA).
2. Trying to implement all this at the MUA level would mean adoption problems (chicken/egg) and networks/servers still bogged down with garbage traffic.
3. The individual definition of spam doesn't matter. Every user grows their own whitelist which would live primarily in the MUA (maybe bundled with that private key file for easy export) and automatically sync to any last-hop MTA that the user chooses. I would imagine this synchronization to be weaved transparently into the equivalents of today's POP/IMAP protocols. You are free to whitelist that viagra-guy, that will have no influence on *my* whitelist.
4. Identity theft would be a very small problem because a whitelist entry can ofcourse be revoked at any time. Furthermore a hijacked identity only enables the spammer to reach recipients that have whitelisted this identity and likely only once. That's too much effort for too little reward, most spammers wouldn't bother.
5. Normal Mailing lists are a no-brainer. The list-serv has an identity, too. Whitelist that and you're good to go. Ofcourse there remains the problem of spammers signing up and spamming the list. This cannot be solved, only mitigated through moderation, due to the nature of the beast.
I think our opinions don't differ all that much. I'm proposing a technical solution across MUA/MTA that would work 100% with zero false positives but cannot realistically be deployed "any time soon(tm)". You're proposing a technical solution at the MUA only that works today but can never get better than around 90% accuracy and has occassional false positives.
You say the latter should be good enough. I say: No, we could do better, if only we could get rid of SMTP in a day.:-)
Sorry, I disagree. The term "spam" is very well defined, you can look it up in pretty much every dictionary. For 99% of us spam is defined as: Stupid advertisement that I have never requested.
Yes, there are corner cases but claiming that the problem is not well defined not only ignores years of research into that area, it also ignores common sense.
No. SMTP is a store and forward protocol, think of it kind of like TCP but with mail messages instead of packets. Sender identity is irrelevant as is challenge/response for reliable message delivery, which is what SMTP is all about. In fact, both those methods reduce the reliability of message delivery, and are therefore bad.
Well, then let me put it differently. SMTP is fit for it's initial purpose (reliable message delivery) but that doesn't match today's requirements. Today we want more than reliability, we want a way to decide *who* is allowed to communicate with us. This requirement cannot be layered on top of SMTP.
You can add sender identity and challenge response at a higher level, namely controlled by the mail software and encoded in the text of the messages.
Wrong. Plenty of half-baked C/R systems (TDMA and the ilk) do what you propose and try to solve it all at the MUA-level. All of them fall down on at least one inherent property of the SMTP protocol: Bounce messages. There is no way to distinguish between legit and (possibly maliciously) forged bounce-notices. Thus as of today it's an all or nothing decision: Do you want to know when your message didn't reach your recipient and cope with the extra spam or do you choose to not care and at the same time give up the very property (reliability) that you praise SMTP for?
This only reinforces the adoption problem. If there was a solution that worked perfectly then it would likely spread like fire (with plugins for all popular MUAs) and people would soon forget that spam ever existed. But there is no such solution, only half-baked hacks that may or may not work to some degree. Worse yet, these hacks do nothing to reduce the load on our networks and servers. They only mask the problem instead of solving it.
Again, a true solution needs to be implemented at both the MTA and the MUA level. There are some tricky problems to tackle surrounding legit group communications (mailing lists) but they are solvable with sender identity.
Finally, I can easily outline a spam-resistant system:
* The sender identity is computed by the MUA (based on a password)
and stored as a private key that is portable to any other MUA.
* Message signing is mandatory. No MTA accepts unsigned messages.
* The destination MTA keeps a whitelist of allowed senders for each recipient.
Unknown senders are either challenged with a captcha or the recipient is
asked for authorization. Ofcourse a recipient can also pro-actively
whitelist specific senders or domains.
* Wrt bounces: Each sent message contains a cryptographic cookie (added by the MUA)
that expires after a user-defined interval. Bounces must quote this cookie or will
be silently discarded by the receiving MUA.
Feel free to explain how you would model these properties on top of SMTP. Also feel free to point out flaws. I wrote this up in under 10 minutes but am very convinced that a bullet-proof spec could be drafted in a day or two. Only problem: It's not compatible to SMTP. There's not even a remotely sane migration-path from SMTP. Which I'll take as proof that SMTP is broken for today's requirements.
If the spam-problem could be solved by tacking a few layers on top of SMTP then why has nobody done it yet? I can tell you why: because it's not that easy.
The fundamental building blocks for any technical solution to the spam problem are sender identity and challenge/response. Neither can be implemented on top of SMTP. Neither can be implemented as an extension to SMTP without incompatible changes to the protocol semantics.
If you want to insist that SMTP is "not broken" then please present your solution for reliable, yet spam-resistant, bounce messages.
Yes there are technical solutions and they're not even hard to implement.
One trivial approach would be mandatory message signing (cryptographic identity) combined with challenge/response. Whenever someone wants to mail you for the first time your mail server would, depending on your preference, either ask them to solve a captcha or you to permit mail from that sender.
From that point on the sending identity would be on your whitelist and you could exchange mail freely.
This can be built today, on top of freely available technology such as PGP/SMIME, without changing e-mail semantics (Push-model) at all. Ofcourse this can be optimized further and there quite a few other options that would work equally well.
The social aspect playing a role here is the chicken/egg problem or "how to offer a transition-path". It's not happening because we cannot replace all existing SMTP infrastructure over night. If we could then the problem would already be solved.
I think what GP meant when he said E-mail is fundamentally broken is that SMTP is fundamentally broken.
There are trivial technical solutions for the spam problem if only we could get rid of SMTP. Ofcourse "we" can't but my hopes are that google may do it eventually. They could roll out a new system on a large enough scale to actually make it stick.
I think what grandparent was trying to point out: The industry will rather develop yet another drug against cold (because they can sell that to 10mio people a year) than a drug against a lethal but rare disease (because that can only be sold to, say, 5000 people a year).
I would agree that from an idealistic point of view a "free market" is the wrong incentive here. But I cannot think of a better alternative either. Our society is driven by greed so the current state of affairs may even be the lesser evil...
This is coming from someone who looked at private offices and decided that would kill our small team collaboration work [maybe offering better, but maybe not] and would cost us a ton of money.
If that's your opinion then I'm grateful that I don't have to work for you. You have either found an amazingly rare breed of programmers (those that function well in a noisy environment) or you simply have no idea how programmers actually work. I strongly suspect the latter.
Read up on some of the comments from the "trenches". We don't make up this stuff about "conversation mode" and "focus mode". We don't ask for offices with doors because we like status-symbols. We ask for them because we can work better that way by pretty much every metric.
How did you come to the conclusion that separate offices would kill your team's collaboration work? Do they literally yell across the room "Joe, can you review my last checkin?" or spontanously summon flashmob meetings?
Yes, working in one big room can work well for up to maybe 10 people. But I have witnessed time after time that it simply doesn't scale beyond that. People have a natural tendency to take the shortest path to solve their problems and when the shortest path means walking (or yelling) across the room then that will be used. No policy helps that. Furthermore there's always a "new guy" around asking a constant stream of questions, there's always some important gossip to exchange and there's always someone walking around behind your back.
As much as we like to deny it, we're still animals. You can not defy psychology. Someone talking or just walking behind your back *will* disturb your concentration. Most of the time you don't even notice because we all have developed filters against such distractions. But keeping those filters up constantly costs energy. Energy that can not be used for productive work anymore.
In each new economy "loft" that I have worked in so far there were some people who'd regularly come in very early, stay in when everybody else went for food, or stay very late. When asked about that they all had the same answer: "These are the best (read: only) times where I can actually get shit done."
So, for god's sake, if you want to get the most out of your employees then give them choice. Some people *like* to work in a big-room, maybe because they're really that rare breed or (my pet theory) because they think they can make up for their slacking with socializing. But most tech workers, and programmers in particular, will happily take the office with a door and will thank it with a highly improved performance.
You're mixing too much personal taste and preference into this.
I don't like the java language either but the concurrent gc exists (which mitigates GC spikes), IDEs and coding practices are free for you to decide and your (maybe justified) antipathy for "lack of const" or "jar hell" are very minor aspects in a decision for or against a programming environment.
Further, if you take a look at the real world you'll notice that dynamic languages like python and ruby are gaining ground against java, not C++. That's because developer productivity matters above everything else and many people begin to realize that these languages provide better ROI for common web tasks - despite being slower in terms of instructions per second.
Nobody writes webapps in C++ because the language is not suited to the task.
Well, sounds like your "j2ee architects" didn't know what they were doing. Not uncommon in that profession if I may add...
So next time just stick blame where it belongs: on the incompetent staff.
Even if java was too slow for a particular task in your environment then it's not java to blame but the people who chose to use it. Or can you imagine a brick&mortar architect to blame it on his ruler when one of his buildings collapses?
Well, at least you're not caving up as I would have expected, so I'll give you the benefit of the doubt.;-)
Yes, technical projects get out of hand very often. What made me suspicious was the scale you claimed while at the same time blaming it single-handedly on a programming language.
If this royal screwup really happened as you described then I assume it has been due to the usual mixture of sheer incompetence and unrealistic ambitions. From a technical POV there are *very* few areas where I could imagine the raw java (JVM) performance to become a bottleneck. In a well architected system the choice of programming language simply doesn't matter much in terms of scalability because it's static overhead. Highly optimized C or even assembler code may get you 1000 OPS/sec out of a machine instead of 800 OPS/sec with java. The ongoing effort (bugs, maintenance, expensive specialized staff) to sustain such micro-optimizations usually dwarves the benefit, though, and is only really done when you're dealing with something that can not be scaled out and is very expensive to scale up (i.e. big iron).
I think I'll stick with the mythical man month perspective unless you can point me to some of their new, diverse products?
I know they have the xbox and zune now but if these figures are right then we're talking 45000 people here - or the population of a small city. They must have one hell of a lot of projects going on to keep so many people busy, or the numbers are just inflated.
Oh, I am working at a company that designs such applications for a living and have been involved with fairly large scale deployments (~15 racks), too. Furthermore I can do basic math.
You on the other hand don't even seem to realize that you're talking about investements in the upper two digit range here. Nobody spends that kind of money without a prior evaluation. Nobody spends that kind of money when the evaluation suggests that operational costs will explode by a factor of 16.
I worked in a place that went from C++ to java and the machine count went from 300 to over 5000 to maintain the overall performance and handle the growth at the same time. Needless to say supporting 5000 machines is not as easy as 300 and the IT costs really cut into the overall profit.
Dude, as I said in my other post, you're full of it. That figure is not only highly unrealistic, it is also totally meaningless without context. If you really have been involved in such a high profile operation then you surely won't mind to describe some of those tasks that forced you to "suddenly and unexpectedly" scale out by factor 16 after switching to java?
Needless to say supporting 5000 machines is not as easy as 300 and the IT costs really cut into the overall profit.
Yes, owning and operating half of a midsized datacenter "by accident" may cut a tiny bit into the bottomline. Enough so that it makes your story sound like total bullshit.
This is utter nonsense. Worse yet, if you *really* had implemented real apps in all these languages then you wouldn't be talking like that. If you feel an urge to "show off" on slashdot then please do it on a subject where you at least have a remote clue and don't spread FUD, kthx.
Anyone who has implemented a large scale webapp (or other n-tier app fwiw) in any language knows that per-node performance becomes less relevant with every node you add. The static overhead of one language over another is only relevant very early and *very* late in the scalability curve and normally the productivity benefits outweigh that overhead easily. The significance of such overhead is further reduced by Moore's Law (CPUs only get faster) and by the fact that most apps are primarily I/O-bound.
It was just interesting to see exactly how stupid these creatures are: They would AAWWWRRRK! at the oncoming bus which must have been going 2MPH before the rather loud SQUELCH of the bird being smooshed under the tires.
As one of my relatives is^Wwas a seagull I can explain this behaviour. It's actually fairly simple: When seagulls rest they normally float on the water. They don't have to be scared of "big boxy things" approaching because normally that would be a ship and that would just gently push them out of the way - not even disturbing their sleep.
Therefore seagulls have never had a need to develop defensive strategies against human land vehicles - or anything else that's walking on wheels.
To make matters worse the brain of a seagull is really small and their mental abilities are somehwat limited, akin to a 0.5MHZ CPU.
Thus, you are probably right that their small mind falsely classifies the blacktop as "water" at first. "Big boxy thing approaching at 2MPH" is not so slow anymore when your single thread of execution is blocked with sorting out the unfamilar sensory input: "Why is this water so hard?"
Many people who want to deploy "100 desktops" (or rather "100 servers" in this context) will first want to hire competent staff to manage said hosts. For OSS unix-management stuff I'd point to puppet, cfengine, FAI (debian specific) and others. As usual there is not "one tool to rule them all" but a set of building blocks that competent staff will assemble into something suitable to the task.
Admittedly I have only glimpsed at TFA. But what the hell does "manage" mean in this context? Last time I checked our hosts ran a mixed set of services, most of which are best (and most comfortably) "managed" from the command line - by editing config files.
What is MS gonna do, create a GUI frontend for every piece of OSS unix software out there? Preferably a unified one?
Maybe I wasn't clear about that point but we obviously do not set our machines to auto-update from some distro-repository. We set them to auto-update from our local mirror which only receives packages after they have been tested and blessed on a dedicated host. I think in a "moderm enterprise" you would call that a controlled rollout and my question was about how we're supposed to implement that on solaris without package or dependency management. Feel free to enlighten me; how do you do it?
Oh yes, we learned about that part. If that's what you want then Sun sure is a good partner. For us those "services" (or "solutions") were generally a bit opaque. We like to know what's going on and how to fix it when it breaks. Thus we take sun for what it is in our eyes (a hardware company) and don't throw money after turnkey-solutions.
ou don't synchronize Solaris machines automatically. If you do that it means your application should be running in something else or that you don't know what you are doing. Solaris servers will run the basic services that keep your company alive and profitable.
Erm. Yes, maybe that "something else" is linux then? I have no idea what you mean by "basic services" and why you think those never need to be updated. And I seriously wonder how you ensure availability for these services when you can't mirror them over multiple hosts - which obviously involves keeping all those hosts in sync?
I know this problem very well from the dark days when I was still writing java. There doesn't seem to be a satisfactory solution, it's always a tradeoff.
While reading this thread I realized a funny thing: This particular annoyance totally vanished from my day-to-day headaches when I switched to python about a year ago.
It's a bit wierd because Python doesn't even use braces so one would expect it to be even harder to identify where a block begins and where it ends. But the opposite has been the case for me: The clean syntax and language design has led me to write, on average, shorter blocks with very little nesting.
I don't hate VeriSign because they have a license to print money. I hate them because their boundless greed seriously damages all our efforts to educate joe sixpack about what is a "secure website" and what is not.
No kidding, they're charging a thousand bucks for a cert that turns your address bar green in some browsers. What's next, $2000 for a purple address bar? $3000 for color of my choice? $10000 for a.gif of the CEO's hairy butt?
VeriSign wants joe-sixpack to believe that "green addressbar" (or purple, or hairy) means more trustworthy than "just the padlock". This is false education and completely misses the point.
Malicious parties can get a "green adressbar"-cert just like your your bank can. Yes, there are more hurdles and paperwork involved. Up to the point that a criminal organization with the ressources to pull of and profit from a bank phishing scam might indeed consider them "mildly annoying". No one in their right mind can tell me that this green-bar-shenigan will bite enough into a scammer's bottom line to put them out of the game.
No, the green-bar is not more than a scam of its own. Designed for the sole purpose of paying the next Yacht for some VeriSign execs.
So how do we solve the real problem?
Remove the implicit trust for VeriSign and all other "certificate authorities" from the browser. The browser should display a warning, along with the cert details, whenever I visit a SSL site for the first time.
It doesn't matter if the cert is signed by myself or by some external "authority" who at one time in the past called up a possibly spoofed phone number and collected money from a potentially stolen CC. Display the warning in both cases!
I'm connecting to a site that intends to establish trust with me for the first time. Alert me of that fact. Tell me to double-check my addressbar, heck, display the full URL directly in the popup window. Tell me that if this is a banking site then I should call them up and ask for a fingerprint to verify against the cert. Tell me that if this site pretends to be my banking site that I know I have previously used then I should be especially suspicious and rather type in the URL manually again.
Then, when I have followed that procedure (or chose to not care and just click OK), save the cert and don't nag me again in the future.
It all boils down to: VerSign certs are not more trustworthy than self-signed certs.
Once you've realized that, the solution becomes painfully obvious...
I briefly looked into it privately and like the idea very much. But it didn't feel mature enough (yet?) to base our business on it, both in terms of software and in terms of "will this still exist in year?".
Slashdot Mirror
I think you're confusing the half-baked challenge solutions of today with what a properly designed solution could do.
Yes, C/R is annoying when you have to sift through your mailbox to separate spam from Challenges. When they look like any other E-Mail with not even a standard formatting to identify them. When the procedure varies between clicking a link, replying or even quoting some gibberish text from the mail (oh and don't get it wrong or it won't work), etc.
C/R would be widely accepted if you think more of the way skype does it. A simple dialog box, one click, done.
This is the kind of integration I'm thinking of and I'm pretty convinced that even people like Mr.Pogue would happily accept it if it reduces their spam input to zero.
I for one would highly prefer to skim over, at worst, a dozen challenges a day if that saves me from scanning my spam box with thousands of mails regularly to look for false positives...
I'll try to keep this short:
:-)
1. Admins would not have to keep the identity lists at the MTA up to date. The users do that themselves (their MUA interfaces with the MTA).
2. Trying to implement all this at the MUA level would mean adoption problems (chicken/egg) and networks/servers still bogged down with garbage traffic.
3. The individual definition of spam doesn't matter. Every user grows their own whitelist which would live primarily in the MUA (maybe bundled with that private key file for easy export) and automatically sync to any last-hop MTA that the user chooses. I would imagine this synchronization to be weaved transparently into the equivalents of today's POP/IMAP protocols. You are free to whitelist that viagra-guy, that will have no influence on *my* whitelist.
4. Identity theft would be a very small problem because a whitelist entry can ofcourse be revoked at any time. Furthermore a hijacked identity only enables the spammer to reach recipients that have whitelisted this identity and likely only once. That's too much effort for too little reward, most spammers wouldn't bother.
5. Normal Mailing lists are a no-brainer. The list-serv has an identity, too. Whitelist that and you're good to go. Ofcourse there remains the problem of spammers signing up and spamming the list. This cannot be solved, only mitigated through moderation, due to the nature of the beast.
I think our opinions don't differ all that much. I'm proposing a technical solution across MUA/MTA that would work 100% with zero false positives but cannot realistically be deployed "any time soon(tm)". You're proposing a technical solution at the MUA only that works today but can never get better than around 90% accuracy and has occassional false positives.
You say the latter should be good enough. I say: No, we could do better, if only we could get rid of SMTP in a day.
For 99% of us spam is defined as: Stupid advertisement that I have never requested.
Yes, there are corner cases but claiming that the problem is not well defined not only ignores years of research into that area, it also ignores common sense.
Well, then let me put it differently. SMTP is fit for it's initial purpose (reliable message delivery) but that doesn't match today's requirements. Today we want more than reliability, we want a way to decide *who* is allowed to communicate with us. This requirement cannot be layered on top of SMTP.
Wrong. Plenty of half-baked C/R systems (TDMA and the ilk) do what you propose and try to solve it all at the MUA-level. All of them fall down on at least one inherent property of the SMTP protocol: Bounce messages. There is no way to distinguish between legit and (possibly maliciously) forged bounce-notices. Thus as of today it's an all or nothing decision: Do you want to know when your message didn't reach your recipient and cope with the extra spam or do you choose to not care and at the same time give up the very property (reliability) that you praise SMTP for?
This only reinforces the adoption problem. If there was a solution that worked perfectly then it would likely spread like fire (with plugins for all popular MUAs) and people would soon forget that spam ever existed. But there is no such solution, only half-baked hacks that may or may not work to some degree. Worse yet, these hacks do nothing to reduce the load on our networks and servers. They only mask the problem instead of solving it.
Again, a true solution needs to be implemented at both the MTA and the MUA level. There are some tricky problems to tackle surrounding legit group communications (mailing lists) but they are solvable with sender identity.
Finally, I can easily outline a spam-resistant system:
* The sender identity is computed by the MUA (based on a password)
and stored as a private key that is portable to any other MUA.
* Message signing is mandatory. No MTA accepts unsigned messages.
* The destination MTA keeps a whitelist of allowed senders for each recipient.
Unknown senders are either challenged with a captcha or the recipient is
asked for authorization. Ofcourse a recipient can also pro-actively
whitelist specific senders or domains.
* Wrt bounces: Each sent message contains a cryptographic cookie (added by the MUA)
that expires after a user-defined interval. Bounces must quote this cookie or will
be silently discarded by the receiving MUA.
Feel free to explain how you would model these properties on top of SMTP.
Also feel free to point out flaws. I wrote this up in under 10 minutes but am very convinced that a bullet-proof spec could be drafted in a day or two. Only problem: It's not compatible to SMTP. There's not even a remotely sane migration-path from SMTP. Which I'll take as proof that SMTP is broken for today's requirements.
So, was that well defined enough?
If the spam-problem could be solved by tacking a few layers on top of SMTP then why has nobody done it yet?
I can tell you why: because it's not that easy.
The fundamental building blocks for any technical solution to the spam problem are sender identity and challenge/response. Neither can be implemented on top of SMTP. Neither can be implemented as an extension to SMTP without incompatible changes to the protocol semantics.
If you want to insist that SMTP is "not broken" then please present your solution for reliable, yet spam-resistant, bounce messages.
Yes there are technical solutions and they're not even hard to implement.
One trivial approach would be mandatory message signing (cryptographic identity) combined with challenge/response.
Whenever someone wants to mail you for the first time your mail server would, depending on your preference, either ask them to solve a captcha or you to permit mail from that sender.
From that point on the sending identity would be on your whitelist and you could exchange mail freely.
This can be built today, on top of freely available technology such as PGP/SMIME, without changing e-mail semantics (Push-model) at all.
Ofcourse this can be optimized further and there quite a few other options that would work equally well.
The social aspect playing a role here is the chicken/egg problem or "how to offer a transition-path". It's not happening because we cannot replace all existing SMTP infrastructure over night. If we could then the problem would already be solved.
I think what GP meant when he said E-mail is fundamentally broken is that SMTP is fundamentally broken.
There are trivial technical solutions for the spam problem if only we could get rid of SMTP.
Ofcourse "we" can't but my hopes are that google may do it eventually. They could roll out a new system on a large enough scale to actually make it stick.
I think what grandparent was trying to point out: The industry will rather develop yet another drug against cold (because they can sell that to 10mio people a year) than a drug against a lethal but rare disease (because that can only be sold to, say, 5000 people a year).
I would agree that from an idealistic point of view a "free market" is the wrong incentive here. But I cannot think of a better alternative either. Our society is driven by greed so the current state of affairs may even be the lesser evil...
So, having a QA department makes better software? Someone at microsoft must have missed the memo...
If that's your opinion then I'm grateful that I don't have to work for you.
You have either found an amazingly rare breed of programmers (those that function well in a noisy environment) or you simply have no idea how programmers actually work. I strongly suspect the latter.
Read up on some of the comments from the "trenches". We don't make up this stuff about "conversation mode" and "focus mode". We don't ask for offices with doors because we like status-symbols. We ask for them because we can work better that way by pretty much every metric.
How did you come to the conclusion that separate offices would kill your team's collaboration work?
Do they literally yell across the room "Joe, can you review my last checkin?" or spontanously summon flashmob meetings?
Yes, working in one big room can work well for up to maybe 10 people. But I have witnessed time after time that it simply doesn't scale beyond that.
People have a natural tendency to take the shortest path to solve their problems and when the shortest path means walking (or yelling) across the room then that will be used. No policy helps that. Furthermore there's always a "new guy" around asking a constant stream of questions, there's always some important gossip to exchange and there's always someone walking around behind your back.
As much as we like to deny it, we're still animals. You can not defy psychology. Someone talking or just walking behind your back *will* disturb your concentration. Most of the time you don't even notice because we all have developed filters against such distractions. But keeping those filters up constantly costs energy. Energy that can not be used for productive work anymore.
In each new economy "loft" that I have worked in so far there were some people who'd regularly come in very early, stay in when everybody else went for food,
or stay very late. When asked about that they all had the same answer: "These are the best (read: only) times where I can actually get shit done."
So, for god's sake, if you want to get the most out of your employees then give them choice. Some people *like* to work in a big-room, maybe because they're really that rare breed or (my pet theory) because they think they can make up for their slacking with socializing. But most tech workers, and programmers in particular, will happily take the office with a door and will thank it with a highly improved performance.
You're mixing too much personal taste and preference into this.
I don't like the java language either but the concurrent gc exists (which mitigates GC spikes), IDEs and coding practices are free for you to decide and your
(maybe justified) antipathy for "lack of const" or "jar hell" are very minor aspects in a decision for or against a programming environment.
Further, if you take a look at the real world you'll notice that dynamic languages like python and ruby are gaining ground against java, not C++. That's because developer productivity matters above everything else and many people begin to realize that these languages provide better ROI for common web tasks - despite being slower in terms of instructions per second.
Nobody writes webapps in C++ because the language is not suited to the task.
Well, sounds like your "j2ee architects" didn't know what they were doing. Not uncommon in that profession if I may add...
So next time just stick blame where it belongs: on the incompetent staff.
Even if java was too slow for a particular task in your environment then it's not java to blame but the people who chose to use it.
Or can you imagine a brick&mortar architect to blame it on his ruler when one of his buildings collapses?
Well, at least you're not caving up as I would have expected, so I'll give you the benefit of the doubt. ;-)
Yes, technical projects get out of hand very often. What made me suspicious was the scale you claimed while at the same time blaming it single-handedly on a programming language.
If this royal screwup really happened as you described then I assume it has been due to the usual mixture of sheer incompetence and unrealistic ambitions.
From a technical POV there are *very* few areas where I could imagine the raw java (JVM) performance to become a bottleneck. In a well architected system the choice of programming language simply doesn't matter much in terms of scalability because it's static overhead. Highly optimized C or even assembler code may get you 1000 OPS/sec out of a machine instead of 800 OPS/sec with java. The ongoing effort (bugs, maintenance, expensive specialized staff) to sustain such micro-optimizations usually dwarves the benefit, though, and is only really done when you're dealing with something that can not be scaled out and is very expensive to scale up (i.e. big iron).
I think I'll stick with the mythical man month perspective unless you can point me to some of their new, diverse products?
I know they have the xbox and zune now but if these figures are right then we're talking 45000 people here - or the population of a small city.
They must have one hell of a lot of projects going on to keep so many people busy, or the numbers are just inflated.
Oh, I am working at a company that designs such applications for a living and have been involved with fairly large scale deployments (~15 racks), too.
Furthermore I can do basic math.
You on the other hand don't even seem to realize that you're talking about investements in the upper two digit range here.
Nobody spends that kind of money without a prior evaluation.
Nobody spends that kind of money when the evaluation suggests that operational costs will explode by a factor of 16.
Dude, as I said in my other post, you're full of it.
That figure is not only highly unrealistic, it is also totally meaningless without context.
If you really have been involved in such a high profile operation then you surely won't mind to describe some of those tasks that forced you to "suddenly and unexpectedly" scale out by factor 16 after switching to java?
Yes, owning and operating half of a midsized datacenter "by accident" may cut a tiny bit into the bottomline.
Enough so that it makes your story sound like total bullshit.
This is utter nonsense. Worse yet, if you *really* had implemented real apps in all these languages then you wouldn't be talking like that.
If you feel an urge to "show off" on slashdot then please do it on a subject where you at least have a remote clue and don't spread FUD, kthx.
Anyone who has implemented a large scale webapp (or other n-tier app fwiw) in any language knows that per-node performance becomes less relevant with every node you add. The static overhead of one language over another is only relevant very early and *very* late in the scalability curve and normally the productivity benefits outweigh that overhead easily. The significance of such overhead is further reduced by Moore's Law (CPUs only get faster) and by the fact that most apps are primarily I/O-bound.
As one of my relatives is^Wwas a seagull I can explain this behaviour.
It's actually fairly simple: When seagulls rest they normally float on the water.
They don't have to be scared of "big boxy things" approaching because normally that
would be a ship and that would just gently push them out of the way - not even disturbing
their sleep.
Therefore seagulls have never had a need to develop defensive strategies
against human land vehicles - or anything else that's walking on wheels.
To make matters worse the brain of a seagull is really small
and their mental abilities are somehwat limited, akin to a 0.5MHZ CPU.
Thus, you are probably right that their small mind falsely classifies the blacktop as "water" at first.
"Big boxy thing approaching at 2MPH" is not so slow anymore when your single thread of execution
is blocked with sorting out the unfamilar sensory input: "Why is this water so hard?"
Many people who want to deploy "100 desktops" (or rather "100 servers" in this context) will first want to hire competent staff to manage said hosts.
For OSS unix-management stuff I'd point to puppet, cfengine, FAI (debian specific) and others. As usual there is not "one tool to rule them all" but a set of building blocks that competent staff will assemble into something suitable to the task.
Admittedly I have only glimpsed at TFA. But what the hell does "manage" mean in this context?
Last time I checked our hosts ran a mixed set of services, most of which are best (and most comfortably) "managed" from the command line - by editing config files.
What is MS gonna do, create a GUI frontend for every piece of OSS unix software out there?
Preferably a unified one?
Excuse me while I go laugh my ass off.
Maybe I wasn't clear about that point but we obviously do not set our machines to auto-update from some distro-repository. We set them to auto-update from our local mirror which only receives packages after they have been tested and blessed on a dedicated host. I think in a "moderm enterprise" you would call that a controlled rollout and my question was about how we're supposed to implement that on solaris without package or dependency management. Feel free to enlighten me; how do you do it?
Oh yes, we learned about that part. If that's what you want then Sun sure is a good partner. For us those "services" (or "solutions") were generally a bit opaque. We like to know what's going on and how to fix it when it breaks. Thus we take sun for what it is in our eyes (a hardware company) and don't throw money after turnkey-solutions.
Erm. Yes, maybe that "something else" is linux then?
I have no idea what you mean by "basic services" and why you think those never need to be updated. And I seriously wonder how you ensure availability for these services when you can't mirror them over multiple hosts - which obviously involves keeping all those hosts in sync?
I know this problem very well from the dark days when I was still writing java.
There doesn't seem to be a satisfactory solution, it's always a tradeoff.
While reading this thread I realized a funny thing: This particular annoyance
totally vanished from my day-to-day headaches when I switched to python about
a year ago.
It's a bit wierd because Python doesn't even use braces so one would expect
it to be even harder to identify where a block begins and where it ends.
But the opposite has been the case for me: The clean syntax and language
design has led me to write, on average, shorter blocks with very little
nesting.
Ha, interesting note, I'll have to think about that. :-)
I'm on 1920x1200 and often browse in full width (like when I wrote that post).
You are right in that I should probably just let it flow but that doesn't work so well with a 217 chars wide input box.
Generally you should be able to get my intended formatting (without auto-wraps) by making your browser wider or reducing the font-size a bit.
Sorry for the inconvenience, I'll try to make my browser window
narrower next time.
Count me in, parent and grandparent are dead-on.
.gif of the CEO's hairy butt?
I don't hate VeriSign because they have a license to print money.
I hate them because their boundless greed seriously damages all
our efforts to educate joe sixpack about what is a "secure website" and
what is not.
No kidding, they're charging a thousand bucks for a cert that turns
your address bar green in some browsers. What's next, $2000 for a purple address bar?
$3000 for color of my choice? $10000 for a
VeriSign wants joe-sixpack to believe that "green addressbar" (or purple, or hairy)
means more trustworthy than "just the padlock". This is false education and completely
misses the point.
Malicious parties can get a "green adressbar"-cert just like your your bank can.
Yes, there are more hurdles and paperwork involved. Up to the point that a criminal
organization with the ressources to pull of and profit from a bank phishing
scam might indeed consider them "mildly annoying". No one in their right mind can
tell me that this green-bar-shenigan will bite enough into a scammer's bottom line
to put them out of the game.
No, the green-bar is not more than a scam of its own.
Designed for the sole purpose of paying the next Yacht for some VeriSign execs.
So how do we solve the real problem?
Remove the implicit trust for VeriSign and all other "certificate authorities" from the browser.
The browser should display a warning, along with the cert details, whenever I visit a SSL site for the first time.
It doesn't matter if the cert is signed by myself or by some external "authority"
who at one time in the past called up a possibly spoofed phone number and collected
money from a potentially stolen CC. Display the warning in both cases!
I'm connecting to a site that intends to establish trust with me for the first time.
Alert me of that fact. Tell me to double-check my addressbar, heck, display the full URL
directly in the popup window. Tell me that if this is a banking site then I should call
them up and ask for a fingerprint to verify against the cert. Tell me that if this site
pretends to be my banking site that I know I have previously used then I should be
especially suspicious and rather type in the URL manually again.
Then, when I have followed that procedure (or chose to not care and just click OK),
save the cert and don't nag me again in the future.
It all boils down to:
VerSign certs are not more trustworthy than self-signed certs.
Once you've realized that, the solution becomes painfully obvious...
I briefly looked into it privately and like the idea very much.
But it didn't feel mature enough (yet?) to base our business on it,
both in terms of software and in terms of "will this still exist in year?".