Slashdot Mirror
Whitehouse Emails Were Lost Due to "Upgrade"
I Don't Believe in Imaginary Property writes "We now know how the Whitehouse managed to lose about five million emails. It seems that they 'upgraded' their Lotus Notes system, which had an automatic retention and backup system, for Microsoft Exchange, which did not support the automatic system. So they changed it to a manual process, where aides would manually sort emails one by one into individual PST files, which they call a 'journaling' archive system. They're still building a replacement for the retention system. Right when they had one finished, the White House CIO complained that it made Microsoft Exchange too slow, so they hired yet another contractor to build another one, causing a senior IT official to quit in protest. So they still haven't completed the project after almost eight years, and rely on humans to sort millions of emails."
ridiculous!
"Strategic Incompetence"
It's not wasting time, I'm educating myself.
read the summary and understood the Whitehouse is blaming Microsoft? hahahahahahahahahahahahahahahaha
There is nothing that will happen for the rest of the week that can make me more light hearted than this. HAHAHAHAHAHAHAHAHA
Support NYCountryLawyer RIAA vs People
C'mon. After 8 years they still couldn't come up with something? As a Government employee I know how slow things can be, but 8 years?
It's almost dumb enough to be true but it's a transparent lie. They made backups before they switched systems and those backups should still exist. If nothing else, M$ has a copy of the pst files because they can and would. Windows is no way to run a government.
Friends don't help friends install M$ junk.
It's getting harder and harder to tell the difference between subterfuge and sheer incompetence.
+1 IDisagreeSoHeMustBeATrollOrAnAstroturferOrAShill
Honesty, why throw out what works?
Our government is a black hole of wasted money and half-assed IT projects that some contractor is stringing out to make the most money.
I think that claim alone is pushing the limits of credulity.
So now you're telling me it wasn't Alzheimer's that wiped out Reagan's memory?
What was that quote about never ascribing to malice?
It's a well put-together story (plausible enough) but I'm still skeptic though.
Maybe we've just seen too many lies :)
Tie two birds together: although they have four wings, they cannot fly. (The blind man)
How Much Exchange and Windows lowers total Cost of Ownership. Sure This isn't MSFT's fault that lies strictly with with IT department, but if MSFT worked better with others this wouldn't be so much of a problem.
Once your locked into MSFT's system you can leave easily. I have 15 year old email boxes that load up just fine in thunderbird, Apple Mail, pine, etc.
i thought once I was found, but it was only a dream.
What happened to the backups they apparently DID have prior to the "upgrade"? Shirley they have these, correct?
Napoleon Bonaparte is credited with saying "Never blame on malice that which can adequately be explained by incompetence."
Best Slashdot Co
... IBM is Democrat and Microsoft is Republican?
Prior Planning Prevents Piss Poor Performance. This shouldn't be taken as a Micr$oft bash as much as an example of poor planning. After having administered both Lotus Notes and Microsoft Exchange I can say that ditching Notes for Exchange isn't a bad move. But doing so without planning out the migration path is. Any large scale project should involve a considerable outside contracting firm that would have automated measures in place. You could even plug in a server appliance before your front end Exchange servers that would automatically archive off mail messages being sent to/from the White House staffers. Another example of US government being inept. Just look to how the US air traffic control centers still operate with equipment that is so outdated that some units are out of commission because they can't order vacuum tubes to service them...
The White House's failure to follow records retention laws was due to deficiencies in Microsoft software?
I predict this will lead to a civil, thoughtful Slashdot discussion which results in many useful recommendations for avoiding similar problems in the future.
I recommend fire.
Computation and Turing machines and all of that science stuff are just theories. The so called "experts" tell us that these things can sort email, but it's better to trust the people to get about the business of the government. I wonder if Bush looked into the eyes of Bill Gates and saw his soul? Maybe if we allow enhanced interrogation methods, we can recover the email? Ahh, too easy, I could go on for hours but I'll stop now.
1. Blame it on Microsoft
2. Post on Slashdot
3. ???
4. Profit!
simon
http://support.microsoft.com/?scid=kb%3Ben-us%3B258243&x=8&y=15
throw new NoSignatureException();
I myself have lost over 6000 emails in two separate incidents involving Outlook.
That being said...que the following comments just to save everyone time.
1. Bush is a Nazi...Cheney Too
2. Impeach them all.
3. Most corrupt in history. blah blah blah
4. Throw in a global warming shot...you pick which.
5. Haliburton!
6. Not a real democracy.
7. Incompetent (now that is a valid charge)
8. Long shot but..."Democrats are no different".
9. Someone will say "Fuck You"
10. Incredulity on the part of EU slashdotters.
I think that covers it. Might as well just disable comments.
I just finished setting up our email retention system, which took a grand total of 3 months from the initial inquiry until complete date. Now I work for a smaller, more agile organization than the Whitehouse, but that doesn't mean that it should take 32x as long for them to complete a similar task. To any technical person it is painfully obvious that this was the intended results of their actions, email retention isn't that hard.
Never ascribe to malice, that which can be explained by incompetence.
Especially in government. Those of you in the private sector may think you know how bad PHBs and other ridiculous management types can be, but they can't hold a candle to the layers upon layers of pointless stupid decision-making that goes on in government. Then add Microsoft software to the equation.
Seriously, the Bush administration sucks, but the missing e-mails have nothing to do with nefarious scheming. The deciders aren't nearly competent enough.
Well we meant to backup up all these terrible incriminating emails but wouldn't you know it, there was a technical glitch.
Basically this comes down to either:
The Government was Incompetent.
or...
The Government is lying and covering up.
Hmmm.... Mr Rock, meet Mr Hardplace.
the incompetence permeates all levels of this White House.
LoB
"Anyone who stands out in the middle of a road looks like roadkill to me." --Linus
Seriously, this is the least bullshit excuse the could come up with? If ANY corporation in the US tried this kind of thing, the wrath of SARBOX would rain down on them like you wouldn't believe.
Even given the staggering incompetence of the Bush administration in nearly all aspects, this just doesn't pass the laugh test.
1) To bad the Whitehouse isn't using an e-mail system like millions of other people. Wait they are. Like it or not MS Exchange is everywhere.
2) To bad the requirement for e-mail archiving and retention is unique to government. Wait, most publicly traded companies have legal and compliance requirements to do so.
3) To bad there is no market for software to archive and retain e-mail on one of the most common e-mail platforms. Wait, there is, and its huge.
4) To bad nobody has nobody has developed technology for this market. Wait, there are dozens of solutions.
To bad no one is getting fired, imprisoned or impeached over this one.
You know...they probably off shored it to China, I hear you need really small fingers to sort those .PST files manually!!!
"My immediate reaction is "WTF? What kind of moron doesn't make things 64-bit safe to begin with?" Linus
rely on humans to sort millions of emails
Damn_registrars has no butt-hole. Damn_registrars has no use for a butt-hole.
George W Bush, Dick Cheney, Bill Gates, and Steve Ballmer -- the real axis of evil!
Ha ha, its what they get for using that virus spreader called MicroSoft Exchange!
Are the people involved in this process, from conception to the current state, being held liable for criminal negligence?
They're being heavily fined and potentially imprisoned for a blatant disregard for government policy?
Is there anybody in a position to make in-depth enquiries regarding the processes involved in this fiasco, who has the wherewithal and political clout to actually do something about it?
I didn't think so. Now bend over and get ready for another "Oops, we did it again!" situation.
Finally had enough. Come see us over at https://soylentnews.org/
and rely on humans to sort millions of emails.
No problem. They had the job outsourced to India.
Rome taught me patience and assiduous application to detail. Virtues which temper the boldness of great, general views.
...could they not use Auto Archive to put them to a PST instead of doing it manually. You have the pain of the PST files but you don't have to rely on imbeciles to manually archive stuff. Eventually rotate the PSTs onto removable media and store them in a nice safe place that the government already owns.
So, backup onto floppy and post to underground storage next to the accelerators at Fermilab then.
the audacity of bullshit.
The First Dog could take a dump on the sidewalk and the White House Secretary of Dog Poop could forget to pick it up and the collective of slashbots would be calling for an impeachment.
Give it a rest, already. There are far worse crimes than hiring the low bidding, incompetent IT contractor. God only knows there is no shortage of incompetent IT people out there.
At best given the explaination provided, the emails are not lost, they are simply unsorted.
Also, what about backup tapes? You don't do a major upgrade without a backup. Even the most slackjawed IT yokel (like me) knows that.
Better hurry!
Ha ha ha!
...shenanigans. No one can be that incompetent. There are several products that will archive Exchange email. I've set one up here at my work not knowing anything about it and in a week it was working fine, it's not that hard.
utter bollocks. just unbe-fucking-lievable.
*every* backup system should result in a set a of data offsite or in a storage area never to be touched again.
even if you use incremental backup every nth backup should be a complete archival read only copy re the previous sentence.
the *very* worst case should be the last major backup is in a format that is not readable with the current system and some red faced admins need help to read read the data.
5 million emails? jesus wept.
add the conspiracy theory factor into the mix and you have something that, on the face of it, sounds unbelievable.
as one of our politicians in the UK said to another a short while ago "you cannot have it both ways, you were either ignorant or incompetent - and neither is acceptable".
be held accountable for this debacle?
"The doctor can't fix your severed hand because our IT department can't boot our server." Talk about a blue screen of death.
So, assuming for a moment their story is true and it *is* just negligence, incompetence, and stupidity; it is still FEDERALLY CRIMINAL negligence, incompetence, and stupidity, Right? Books will be thrown at those responsible, yes?
Those who fail to understand communication protocols, are doomed to repeat them over port 80.
"The Bureaucracy must expand to meet the needs of the expanding bureaucracy." I wonder how much those human email sorters get paid? I bet one of them sabotaged the Exchange server
I judt got a nre Kinesis keybiartf so please excusr ant egregiou typos.
If only someone could come up with a device that can do repetitive work without error and without getting bored. Some sort of electronic mechanism that could look at a certain field in an electronic document and then put the associated text into an electronic bucket labeled for an individual.
Why hasn't someone invented something like this?
One of my first projects after moving to Vancouver was a couple of test installations of pre-release Exchange server back in 1996. Since then I've worked constantly with every version of Exchange in all kinds of backup situations. Early versions of Exchange were a bitch to restore but it's gotten better.
However, there has *always* been a way to retain and archive emails automatically from Exchange and no shortage of migration utilities from notes to Exchange. The reasons stated in the article just don't wash. No one, not even the newest tech school grad could come up with a system like that currently in use.
However, it may in fact not be intentional malice from the start but more likely an existing state of incompetence that was taken advantage of to hide traces or misdeeds or at least to make finding any evidence difficult.
This still doesn't address the use of non-government email systems for official business by Rove and other Republican members. According to the laws of the United States this is all highly illegal. Don't you care at all about what your government is doing or do you think whatever you do won't make any difference?
If you don't want to repeat the past, stop living in it.
I lost a lot of email using Outlook and PST files for backups. After reinstalling Windows from scratch and trying to import the PST files into the new Outlook, 9 times out of 10 it couldn't import the email. That is why I switched to Thunderbird which has better success of backing up email files and importing them after a RRR (Reboot Reformat Reinstall).
Remember, Slashdot does not have a -1 disagree moderation, and no, troll, flamebait, and overrated are not substitutes.
They should just switch to hosted E-mail services with Google Apps :-)
Seriously, though: the White House should not be in control of White House IT services. An independent agency should be responsible for that. It's all part of checks and balances.
I work for a fortune 500 company. We migrated from Notes to Exchange about 5 years ago. I liked Notes more.
As an aside, Notes had support for POP3 so you could use any mail client you wanted.
Exchange has its problems. One time a few years ago our company "focal" (lead supporting 50K people) could not resolve a bug which crept into my profile without deleting me from the system and re-creating my account. Unfortunately when this happens, all of the group mail lists and recurring meeting notices with my name got dropped. There was no way to recover aside from manually correcting the lists. This caused headaches for several months afterward.
There is no good way to back up mail on an Exchange server. What I do is to create two rules which run whenever I send mail, and whenever mail comes in. They copy the message to an "archive" folder which is on another network share. Eventually the size will grow so that you'll have to archive your archive. The bottom line is that the burden of backup falls upon the end user.
Why does Microsoft feel the need to re-invent things that already have standard (and superior) solutions?
--
This space for rent
You can't argue that using Exchange in this case did not lower the total cost of ownership for the Whitehouse, by all indications, it would have been much more expensive for them if the emails had been preserved.
"Thanks for all the money you paid to us. We've used it to buy off ISO among other things" -Microsoft
Dumbocrap propaganda
We need to have a single gov wide IT department and we need to keep it mostly in house with little to no contractors in it as they are slowed down by red tape. The armed forces can keep there own system under this as well as long as it is not the Navy/Marine Corps Intranet tape set up they same must have the same contractor rules aka the gov has full control and can step in at any time and take over.
Never attribute to malice that which can be adequately explained by stupidity. Hanlon's razor
See, Bush isn't incompetent, his government IT staff is. Just like our friends in Oklahoma... Bad IT developers
David Gewirtz of the Outlook Power and Domino Power magazines has published a book on the subject titled Where Have All the Emails Gone?
It's written to be read by a non-technical crowd, so if you pick it up be prepared to skip some chapters which go over networking and e-mail application basics. It's still a very interesting read in that provides some fascinating history going back to the first e-mail system used in the White House and works forward to the current controversy. None of the administrations is blameless in their handling of information.
Al Gore lost the 2000 election due to an "upgrade" whereby the 538-member Electoral College was replaced by the 9-member Supreme Court.
Knowledge is how to play a game, intelligence is how to win, wisdom is knowing what game to play.
How does the whitehouse website deal with AJAX queries?
Did they hire an intern named Jax who can type real fast?
a: saved to tape and sent to a vault on a daily basis
b: recorded by the NSA, who also saves and backs up data
So, it's all a load of bullshit - they're thinking that the public is stupid enough to buy it, or, simply kick it down the road another month or two until the ADHD press finds something shiny to get distracted by like Miley Cyrus Boobs or another blast from Trainwreck Spears.
RS
Shoes for Industry. Shoes for the Dead.
This being Slashdot, everyone and their grandmother is tripping over each other trying to:
a) laugh at the government
b) laugh at Microsoft
c) insist that wiping out the emails was a conspiracy
It's funny how when this happens to thousands of companies around the world it's normal but when it happens to the government it's a conspiracy and/or major incompetence. With all due respect, it's perfectly normal. People who apply double standards and claim that this doesn't happen to large organizations on a daily basis should really sit this one out because you're not being honest.
My former big Corporation had a policy of deleting
all sent email. They did everything possible to
erase any email trails.
I know what that policy is. it's the "fuck you, we're covering our tracks and blaming microsoft" policy. I'm sorry, that doesn't fly around here. Someone broke the law. The white house CIO seems like the prime suspect. the presidential records act was violated, for all those who say, "what law?". Lost in an upgrade is what I expect from bill's plumbing and computer fixin's. It's even what I expect from enron. It is not what I expect from the white house. It shows an unparalleled level of incompetence and hubris.
When will he end up in jail?
They're using their grammar skills there.
Wow! ...just wow...
Lorem ipsum dolor sit amet, consectetuer adipiscing elit.
When you're thinking in economics, it's easy to do the math:
If the penalty for deleting mails that you are by law required to archive is less then the penalty for whatever those mails document, then it's the better choice to delete them.
It really is that simple.
And the only solution around that is one that's got its own problems, namely when you are required to have/show records in a case, and you can't or don't, you are assumed guilty and the penalty for deleting or not keeping those records is in addition to the penalty of the crime.
Assorted stuff I do sometimes: Lemuria.org
If the results of an election seem wrong, don't let lawyers get involved and muck up the process of sorting the thing out.
whitehouse.org has copies of many of the deleted emails sent to them by mistake.
Sorry folks, I seem to have gone off thread here </rant>
"Suppose you were an idiot...and suppose you were a member of Congress...but I repeat myself." Mark Twain
One of my favourite MS oddities.
Of course, this is what 90% of organizations seem to do anyway.
Endless arguments over trivial contradictions in books written by ignorant savages to explain thunder in the dark.
Its a good thing that all those "IT Gurus" in the Government have their Bullshit CompTIA, Microsoft, what the hell ever certifications!
Whenever there is an issue in the press that I happen to know quite a bit about, I find the reporting to be inaccurate, and the stories to misrepresent the issue, sometimes amazingly so. My skeptical nature leads me to suspect that most every story suffers from this, particularly if non-technical people with an axe to grind cover technical issues. Also when technical people cover issues they don't understand, like contracting, outsourcing, politics and people. And don't forget the technical "reporters" have an axe to grind, too.
In this case, the article is correct in some small amount of the particulars, and entirely wrong in most of the implications and conclusions. The issue is being incredibly misrepresented, partly from ignorance on the part of the reporters, but also because people (I think that includes the "reporters" in question) seem to need there to be a malicious reason for why things are a particular way, or why they happened like they did.
So far as I am aware no one involved at all is blaming MicroSoft for anything. As much as slashdotters would like that to be the case, it doesn't seem to be true.
I've read the court documents, and the court orders, and the judge involved misunderstood many of the technical issues. Of course, in this case, slashdotters want to ascribe vast wisdom to the jugde, rather than to heap scorn on him. Why? Because BUSH=BAD, and if the judge does something that seems to affirm that, then the judge _must_ be right, right?
Conversations with the actual people tasked with the actual work on the actual systems actually involved in making actual backup tapes from the actual servers containing the actual MS Exchange instances containing the actual mailboxes holding the actual emails make it seem very, very, very unlikely that emails have, indeed, gone missing.
It is also very, very, unlikely that any political apointee in the chain can affect any of the archival/retrieval process.
None of this keeps people with an axe to grind from mis-characterizing Ms. Payton's statements, or from completely misinterpreting that there must be some conspiracy or vast incompetence involved.
Remember all those times when someone has said "any half decent system admin would keep backups of pertinent data"? Ask yourself why it is that you believe it hasn't happened here. So, why is it? Is it because of what you read in the newspapers? See on the television news? Read online?
What if the tapes really are there? What if, in fact, there are tens of thousands of backup tapes? What if there are fulls, incrementals, and differentials, taken on a daily schedule? Covering every server in the network? What if the email users have no quota or limit so mail folders can grow indefinitely, so that they don't need to copy any of it locally to keep it? What if the people whose job it is to safeguard the data really have done so? How would you know?
What if what you read and see is misrepresenting the issue? Consider that it isn't news to say "We're not sure we agree with the report claiming emails are missing. We need to validate the report that claims missing email, we need to audit the NARA archive to find any anomalies, and then we need to fill in the gaps from our voluminous backups."
What if you said that, and it got so mangled in the reporting that came across as a denial of a problem?
Be careful when the news becomes wish fulfillment. Just because you want Bush and Cheney to be found guilty of having caused the deletion of volumes of email, it doesn't mean it actually happened.
If you've ever worked with government at any level you would recognize this situation as good planning. A senior IT person quit and I surmise that he mentioned to someone that the system was flawed, illegal, or both. It would be interesting to hear depositions of the past and present IT staff and aides involved in this process Even if we need to water board them. And disk and tape forensics is always an option, unless they were also "upgraded".
-- Wondering how long until the internet becomes fully corporatist, like television.
We've faced a similar issue and finally found a company (global relay) that does the archiving automatically. In government where you have to keep email indefinitely, finding a balance between mailbox size and performance can be a challenge. We were doing the pst thing and let me tell you, not the best idea in the world but honestly with exchange there aren't a whole lot of choices with this. Even worse a pst has a 2 gig limit and with some of my users that didn't take long to hit. Add on the fact that pst's are notoriously fragile it's not that far fetched that some emails could have been lost. but all of them, well they really must have screwed up or someone found a plausible excuse...
The industry is full of stories like this, for years. Exchange by default can't handle it - it's still a workgroup server at heart, and subject to many OS and filesystem limits. Does Hotmail.com even rely 100% on Exchange, or is it still UNIX at the core?
De-centralized email storage and PST files?? COME ON!
It is almost CERTAIN to expect that they knew this would cause emails to be lost and take the system from bad to worse. Even a junior IT person fresh off the boat would say this was CRAZY to attempt, with FEWER benefits and increased risk. In the corporate world, this would be met by massive civil lawsuits and possibly criminal charges. Any "contractor" the WH employed would know this for a fact.
So given that such warnings had to have been given and they went ahead anyways, you have to wonder if strategic "loss" of emails was perfect cover for an email purge. Given the shady nature of these characters, I'm sure this was a calculated "feature".
That's like calling Vista an "upgrade" from Windows 2000 or even from XP.
Good grief, I have never seen anything that Exchange / LookOut could do that couldn't already be done better in Notes (except perhaps at screwing things up.) Every job I have ever worked at used Lotus, until I came here. I routinely see backups fail and corruption of data on Exchange / LookOut. Heck, it can't even handle multi-gigabyte mail accounts like Notes, which are commonplace in the Fortune 500 companies I worked for. And I won't even touch the surface of all the database features that are built into Notes.
No serious SOX-compliant company uses Exchange / LookOut, so no serious administration should either.
People call the White House incompetent, but when it comes to things like getting rid of evidence and avoiding any actual consequences for their supposed incompetence, they are masters.
1 Invoke a terrifying internal and external enemy
2 Create a gulag
3 Develop a thug caste
4 Set up an internal surveillance system
5 Harass citizens' groups
6 Engage in arbitrary detention and release
7 Target key individuals
8 Control the press
9 Dissent equals treason
10 Suspend the rule of law
Dear Mr. Gates;
Remember how we kept your company from being split up into different pieces and imposing various fines and regulations that would limit your operations as a result of your abuse of your monopoly?
It's a shame this favor we did for you did not allow you to fix your exchange product so that we would not lose all these important emails. *wink* *wink*
Open Source Java DAO Generator
Well, it sound slike their IT people just didn't get how Exchange archiving works. Which, granted, is pretty ass backward from Notes.
In Notes, you create a archiving policy. Say, "all messages older than 6 months." You then toss these emails to a designated server/mailfile and the users can access them all peachy-key, although typically you set the ACLs on the mailfiles to not allow any deleting.
In exchange, it works quite differently (with PST files being saved to a network drive of some sort) and it sounds like their IT people just had no idea how to implement it.
Don't get me wrong, I vote for the people who seem the least corrupt and incompetent. I'm just under no illusions that things will actually change for the better.
Simple Solution: Symantec Enterprise Vault. It integrates with Exchange journaling to save every single e-mail (even BCC data) and attachment, indexes the content, and does all of this outside of Exchange, with retention policies. You can also archive your Exchange e-mails to keep your Exchange stores to a reasonable size.
The problem is that it is illegal to use government email systems for non-official business like running a political party. This makes it mandatory to have a second system available to those people who are active in party operations. Then you have to rely on people to use the right system for their messages. Even with good intentions, people will make mistakes, and there will be gray areas.
Mea navis aericumbens anguillis abundat
is an excuse that doesn't fly for 5-yr olds. Why should the administration, especially this administration, be allowed to get away with it?
Throw the book at 'em. History would have been happier if Al Capone had gone down for murder, conspiracy, extortion, and racketeering, but at least the income tax evasion charge took him down. That is, I would like to see these people impeached and sent to prison for the rest of their lives for human rights violations, the warrantless wiretapping, or any one of the other egregious crimes they've committed, but I'll settle for them going down for violating Federal law requiring the retention of public records.
Do what you can, with what you have, where you are.
No, I don't mean CNN, NBC, or even the ones that a bit more out there like Huffington or even amateurs like Kos.
I mean comedians. Yes, The Daily Show and The Colbert Report are probably doing a better job covering political events, because the mainstream press won't bother with the stupid stuff the White House does, nor does the less conventional media, because it's more funny and sad than informative.
Lewis Black once got his loudest laughs based on his aneurysm joke that stems from a sentence that's nonsensical without its context. Now that this administration has seemingly tossed rationality, common sense, and sanity in the general direction of South America, his character probably doubled his blood pressure meds for destroying one of his funniest jokes at the time.
At the same time, Bush and buddies give him plenty of material to gleefully mock and angrily stew over at the same time, so it's all good. So far, there are no sedition laws... yet.
"We are Microsoft. You shall be assimilated. Competition is futile."
OK, let's ignore Twitter's use of sockpuppets (which has gotten so blatent, it hardly needs to be pointed out) and focus on the more serious issue, that he like to throw out anti-Windows and pro-Linux cliches without regard to context.
Here the context is a bunch of missing emails. There's no evidence that they were lost due to a Microsoft screwup. Officially, federal IT screwed up. But given the Bush administration's previous attempts to avoid archiving incriminating emails (such as relying on outside mail servers, which is not just illegal, but really horrifying in terms of national security) it's not impossible that this "mistake" was deliberate. Somewhere Rosemary Woods is smiling.
So shut up Twitter. The whole world doesn't revolve around your Linux obsession.
I'm looking forward to losing a few other things out of the White House during the November 2008 upgrade
Nice oblique Arthur C Clark homage. Nearly spit my coffee onto my keyboard.
Some drink at the fountain of knowledge. Others just gargle.
Remember the Senate investigation into President Bill Clinton's investigations into his political opponent's backgrounds?
The investigation came down to a presidential staff member asking the Secret Service for a list of who had been in and out of the White House in the last 4 years (previous to Clinton taking office).
Well, the Senate grilled the crap out of a few Secret Service members and all they did was provide a list of people to the president (or his staff) about who had been in and out of HIS house recently.
I believe that after that fiasco, the Secret Service would rather have a system that DOESN'T WORK, rather than one whose data can be used against them (or the president).
I watched the grilling on CSPAN and one Secret Service agent got a new one reamed in front of the whole nation. I bet he is at a post in Alaska now.
- I live the greatest adventure anyone could possibly desire. - Tosk the Hunted
Any sufficiently advanced malice is indistinguishable from incompetence. Yes but the alliteration and parallelism works better this way. Malice and Magic start with the same letters. And Incompetence nicely parallels Technology.
thus it's both funnier and more thought provoking the original way. Your version is a more cynical editorial comment, the original is a more philosophical outlook that perhaps incompetence is more prevalent than malice but all too often we humans are given interpret the former as the latter
.
Some drink at the fountain of knowledge. Others just gargle.
The facts are, this administration failed to obey Federal law by not retaining emails. They also, against the law, spied on American citizens. Why aren't there full-blown investigations into what those emails were about, and who was responsible for making the decisions?
Those responsible need to be arrested and put in jail, period. No one is above the law in this country, including/especially the President and his cronies.
We need transparency to what the President is doing to protect us, The People, from corruption and misdeeds. He represents us to the world and Bush has presented us a malicious, bumbling, idiots. Frankly, I can't wait until he is gone and hope that someone finds some evidence that he or Cheney can cover up of their misdeeds. It is high time they went to jail for their crimes!
Trouble is, the investigators are also flying the 'L' on their foreheads, geniuine or not, and the would-be 'heroes' aren't fighting for this cause.
what our tax dollars are spent on.
I'm glad I live in a country that doesn't frivilously squander tax dollars.
Oh wait I do.
I am Bennett Haselton! I am Bennett Haselton!
There are a couple of SOX compliant archival solutions for Exchange that can get the job done just fine. If the SEC can mandate email retention on a 4 person investment banking firm, the White House sure as hell better be able to retain their important communications. I've never understood the Federal government mentality of building everything from scratch. My only exposure to government has been at the city level and they are more than willing to use off the shelf systems to get the job done. What is it about the Feds? Is it because of all the mandates and regulations that they place on themselves that they paint themselves into a corner where the only way to comply with all of those mandates is to write the damn software themselves... I mean, outsource it to contractors.
Or maybe it's your sockpuppets, who in a few minutes will make an appearance to heartily agree with you.
And there's a fair bit of that or there wouldn't be a national deficit that is so large it's starting to develop its own gravitational pull..
Insert
And no amount of lying will change that basic fact, nor the fact that every mail server recipient host also has a copy of the intentional fraudulent emails from the White House.
-- Tigger warning: This post may contain tiggers! --
The AC that seems to have taken up poor twitter's cause is probably feeling dumb right now.
Web2.0: I love when people Flickr my cuil and digg my boingboing until my google is reddit and I start to yahoo
WTF! The whitehouse has a CIO?
In the past the interns had to stay up all night with the paper shredders to get rid of the evidence. These days you just have to misplace a couple of backup tapes and fail to follow data retention procedures. The communication still takes place but the evidence trail is easier to get rid of.
right here:
http://www.emc.com/products/family/email-xtender-family.htm
Exchange isn't something you can take out of the box, install while you are having a few beers, and expect great results. A large Exchange deployment requires planning. Everything from the server and network architecture to the storage subsystems needs to be thought out. This includes backups and archiving.
There are lots of companies that make tons of products to do this. CA, Symantec, and EMC, just to name a few.
This upgrade looks like a convenient cover for something more sinister.
-ted
Why do I have a feeling that if I sent an email to the President and Vice President threatening to kill them that they wouldn't have any problem retaining it? Maybe I should do it just to prove that they do have an effective email retention policy. ;)
I, for one, think this is excellent progress. Previously "losing" information on this scale would have needed a major fire, or a move between administrative buildings or a flood or similar catastrophe.
We can now achieve all of this without the hassle of packing information into boxes and mis-lableing them, and without the human cost of employing the four horsemen of the apocalypse to manage IT.
We should all be thankful.
Nullius in verba
the non-government email systems were probably used by Rove & Co because the government ones were so fucked up.
It's getting harder and harder to tell the difference between subterfuge and sheer incompetence.
Because the bullshit is so deep?
This is why we have not closed the borders. I can see the room now with thousands of alien workers reading every white house e-mail for sorting. Nice.
the non-government email systems were probably used by Rove & Co because the government ones were so fucked up
No, the non-government one were used because when those people (Rove, Bush, their staffers, etc.,) were working on campaign or party-related stuff, it's ILLEGAL for them to use the government systems. So, if they HAD used those systems for that sort of messaging, everyone would then complain about THAT.
Don't disappoint your bird dog. Go to the range.
I guess they have more horse lawyers in charge of IT now...
There exists no way of exchanging information without making judgments. --Bene Gesserit Axiom
It's true that putting incompetence first produces better rhythm and a very tight resonance with the original.
But the other way around is more insightful into human behavior.
Sufficient incompetence can accidentally mimic malice.
But a large part of malicious scheming is not just to avoid detection until it's too late - but to avoid responsibility when the result of the scheme comes to light. Thus when one is responsible for keeping evidence against oneself, two design goals for a malicious scheme are destroying the evidence beyond recovery and doing so in a way that is a plausible accident.
If the apparent cause of the accident is lack of due diligence when such was required it still doesn't adequately deflect blame. But if the apparent cause is incompetence to perform the requirements, it DOES deflect blame. It's not the actor's fault that he wasn't capable of performing the function. It's not the boss' fault because the actor had adequate credentials. Malice disguised as incompetence is an example of protective mimicry.
This if malice is indistinguishable from incompetence it's sufficiently advanced.
Bantam Dominique roosters crow a four-note song. Once you've heard it as "Happy BIRTHday" you can't NOT hear it that way
... for defamation.
Their "upgrade" was a downgrade from Lotus Notes to an Exchange server. Many financial services companies, are on Exchange, and have to record a lot of emails -- apparently they are saying that all the companies required to follow the law, cannot due it because they are on Exchange.
I know -- this excuse isn't going to fly with Slashdot -- but on its face, it is bad PR for Exchange.
>>"ad space available -- low rates!!!"
Not one post about how this could have been avoided by the use of an Open Standard?
Hang your heads in collective shame Slashdotters! Shame!
Moving forward in time we'll see that the MS Exchange format will be revised, updated, made incompatible with the format now in use, and the White House will loose more emails.
Don't blame this on the President, this is Congress's fault. Congress not already requiring that our US government store data files in Open Standard format(s) should be grounds for Congressional impeachment due to gross negligence.
Does this explain why Rove used the non-government system for 90%+ of his government-related emails?
10-50% just for convenience reasons and poor training I can see. 90%+?
No. Freaking. Way.
If you don't want to repeat the past, stop living in it.
After 3rd world elections and a 3rd world currency the US government's IT infrastructure is going the same direction ;)
If you recall, Clinton Administration had a similar email retrieval problem with that administration was asked to provide emails to DOJ.
It just doesn't get any easier than that...
I think that this is pretty complex...
First I think they showed real incompetence by choosing to use Exchange for anything, but especially for something as important as government use. I wonder how many SQL injections the Whitehouse servers have seen.
Anyway, next they decided to hide criminal and immoral acts by feigning incompetence.
Finally, their actual incompetence in the working of this whole scenario shows great incompetence.
However, this administration has repeatedly proved the theory that the bigger the lies you tell, the more people will believe them. At this point, the Bush Kool-Aid drinking spurious neo-cons and mouth breathers that still believe this administration's nonsense will believe just about anything.
I'm betting another level of incompetence exists in that this whole thing came about because the dummies were actually discussing all their lies and crimes in e-mail and they didn't realize they couldn't just delete them.
Later,
Tachyon
âoeIf you tell a lie big enough and keep repeating it, people will eventually come to believe it. The lie can be maintained only for such time as the State can shield the people from the political, economic and/or military consequences of the lie. It thus becomes vitally important for the State to use all of its powers to repress dissent, for the truth is the mortal enemy of the lie, and thus by extension, the truth is the greatest enemy of the State.â
Joseph Goebbels
While it's easy to point to a "White House Conspiracy" about this kind of thing, I don't think it's very realistic at all.
Not only would putting together a conspiracy like this be extraordinarily difficult given the number of people involved who would potentially leak information about said conspiracy, it would be a terrific success of social engineering coupled with a vastly incompetent plan to begin with. Hardly a month has gone by in the Bush administration where there wasn't a scandal, big news or a big decision of some sort that hit the newswire.
It would be far easier to use an external e-mail address to simply avoid the archiving requirements than to push an archiving implementation project out 6+ years. It would be far less conspicuous to delete individual e-mails than to delete days and weeks of e-mail. It would be far easier to destroy an entire archived collection than to find and hunt down e-mail from specific days and weeks.
During election cycles its easy to get sucked into propoganda, and sometimes its even fun. But more often than not, when you get sucked into political propoganda, there is something else that you should really be paying attention to.
Sending a guy who works for white house IT to jail might make you feel better, but it sounds like sour grapes to me - sourced from an overall frustration with the administration. Most of us here have worked on projects that went south at some point or another, and in large part it wasn't because of a lack of technical knowledge, but instead from other factors. We're talking about missing e-mail. How many of us work for companies that can find e-mail messages that are 6 years old? Most enterprises that I work with only started mass data retention projects within the last 2 years, and I would guess that there are a few days here and there where things didn't quite work as expected.
Yeah, it seems convenient that these particular days went missing, but with loose parameters, any day of missing e-mails in the last 8 years could be tied to a white house scandal.
1. Nerdy content that average people can't fathom
2. Anti MS
3. Anti Bush
My wife works for USDA. On several occasions, she has gone a week or more without email due to botched Lotus Notes upgrades. Mostly, I think it was due to incompetent contractors, but, considering the times I've had a gun put to my head and forced to use Notes (over a slow WAN connection - the very definition of torture) I am sure some of that is intrinsic to the application. I'm not sure what they use now, she does 90% of her email on her blackberry, but I think they have gone to Exchange/Outlook.
Give a man a fish and you have fed him for today. Teach a man to fish, and he'll say "WHERE'S MY FISH, YOU IDIOT?"
Except the existing state was a fully functional backup system with appropriate safeguards and redundancy. The replacement was an utter clusterfuck that had at least one fix squashed just before implementation. Per the article, the guy in charge quit because they kept blocking proper solutions.
One can only read that there is deliberate stonewalling & willful incompetence because if they were actually so incompetent as to not be able to archive email with a sane solution after 7 years, then the Whitehouse network would have crashed & burned long ago. They did not, ergo, they cannot be as incompetent as would be required for this to be true incompetence.
I love how they are blaming Microsoft for loss of emails, didn't they think to back up the data manually instead of just building one. They need to make a book called: White House Email Backup for Dummies :-D
This is fascinating, as archiving emails has been a fundamental requirement for the financial services industry for years. We use EMC Legato. It has hooks into all sorts of systems. Exchange, or even messages sent through bloomberg terminals.
I'm certain there are at least a dozen companies offering similar solutions. This is a purchase order and maybe a few months of consulting time.
I thought Bush was going to run the government like a business. The first MBA President, blah blah blah.
How is it that his idea of business looks more like an Organized Crime Syndicate?
Just like the popular "downgrade" from Vista to XP, the white house should "downgrade" from Exchange to Lotus Notes.
1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24
I figure since Microsoft is willing to take the hit to the chin, they'll get some lucrative tax break for the next 20+ years. Or when there's a big Class Action lawsuit, they'll get Govt. bailout and make it all go away.
Was that fire in the building next to the White House several months ago burning the disc drives that contained the backups of all those emails?
Sigh.
Life takes interesting turns, but the most interest is when you're off the beaten path.
Imagine somebody from the new administration gets an order to replace notes with exchange, because this is what his boss has negotiated with MS. He gets consultants in to replace the email system, not to implement the back-ups. Email system is replaced, success is reported.
Ups, the back-ups are missing, people have realized. Sorry, no budget was allocated for that! Moreover, nobody in the new administration is interested in backing up the emails. OpEx are easier to get than the CapEx, so the volunteers are put on the task. At the same time least priority project gets no attention and gets the worst or inadequate people to manage it, and fails repeatedly.
Anyway, the administration made it through the both terms fine, lost emails being their least important problem. Does not this explains it all?
I guess nobody there ever heard of using qmail? Implementing a full-blown SMTP, POP3, and IMAP server with full and complete archiving of every message is trivial with it. Hell, I can build a cluster with these tools in a day or so to cover even a pretty heavy load.
And... I will not allow any proprietary email systems into my organization in any way, shape, or form.
At least they weren't lost in the floods of 1967.
668: Neighbour of the Beast
Dog eats homework, blames aliens.
Science advances one funeral at a time- Max Planck
If true, amazing.
If you have worked in corporate IT in the financial arena over the last few years and have been paying attention you probably have noticed the increasingly stringent regulations by the government and non-government regulatory bodies regarding the retention of e-mails.
Yet the Office of the President of the Unites States apparently has no retention strategy at all?
I think you're full of shit, sir.
The article's descriptions, while highly politicized to make it Bush's fault, accurately describes experiences at *many* large companies. Exchange has "something" for archiving, and they DO call it Journalling. But all it does is copy stuff to a mailbox. Period.
This process started as I recall in E2k-SP3 and became part of E2k3. What you do with it after it gets to the mailbox is your problem, not Exchange's. Either you manage that mailbox by hand or you create or buy a system that does it for you.
Tp put it another way, Postfix has an "always BCC" option that will effectively do the same thing - it will BCC every single message that goes through it to an SMTp address. But what you do with it when it gets delivered to that address is not Postfix' problem. You can even do LDAP with Postfix to determine your BCC maps that way. Functionally, the two are identical.
Have you ever gone through the federal procurement system? It's insane. So you are made to install E2k and use it's Journalling system. But wait the system won't allow you any more money to get a back-side database and extraction system. So what do you do, hot shot? You can bitch and moan all you want, it won't change things. You can say you'd stand your ground but doing that wont change the facts. So what do you do? You know what you do, you make some gruntwork and have someone do it by hand, hoping that it will demonstrate the atrocity of the situation. You cry out "Help, help, I'm being repressed. Come see the stupidity and intransigence of the system" but just as in the movie, nobody comes to your aid.
And the beat goes on.
Does it suck? Abso-friggin-lutely! Does it happen? Abso-friggni-lutely!
In a related story, the White House CIO's nose got longer and longer and longer.
... and you are paying for it with your taxes.
I'm just guessing that the telecom companies' various subpoena-less wiretaps might have saved some internet traffic. And don't tell me that our nation's intelligence services aren't monitoring internet traffic to/from government facilities (if they're not, then they're not doing their job). Some of this stuff has to be saved somewhere.
Ben in DC
"It's the mark of an educated mind to be moved by statistics" Oscar Wilde
To keep the smell away they'll have to be sure the bullshit stays deep...
--- I am known for the ones who want to find me on the net. Is that a privacy risk or a privilege? One might wonder..
Clinton bombed 8+ soveriegn nations. Bush is only up to 3.
Because I like to buck the slashdot trend of not actually RTFA before commenting, I read the article and also some of the discussion of it. Found this near the end and decided to repost it here (mainly because I'm too lazy to write up my own thoughts about the subject). I AM NOT the original author of this post, "The Real Bill Anderson" is. Here is the link to the Ars discussion where I found this: http://episteme.arstechnica.com/eve/forums/a/tpc/f/174096756/m/953006491931/p/4
.pst files (semi-tongue-in-cheek prod: if they were text files it's be far easier to search and examine!) being stored and named by government employees it is entirely *expected* that stuff goes missing and may or may not be found later.
... tell me it's never been seen outside of government? Go ahead. Accounts of new manager not doing old manager's projects come from all sectors.
It seems few commenters on this thread have been involved with the process of compliance archiving and restoring. The law does not require an automated system.
Why is that important? Because often, government or not, if it isn't required you don't get funding for it. Get it and you'll see mass complaints about the government going beyond it's requirements, pork passing, etc.. Yes, we the technical crowd agree there *should* be one. But how often does what we say *should* happen get passed on because should != required?
I've witnessed many an org pass on "shoulds" to get the "have-tos". No politics involved.
It's important also to note what is missing. The congressional report linked earlier goes into some detail, though not enough IMO. But what we do see is that it isn't all Bush or even Bush related emails. It's "components". Given the description of what can at best be described as an ad hoc method of
Again, no politics need be involved for this. Perhaps sad-but-basic office politics, and maybe high level politics. But the system and processes described are far from plausible, and in my experience in this industry over the last half-decade *common*.
Yes we can agree that the system sucks, and is ridiculous and non-scalable. And we can agree that the techies in the positions should have known that. But that does not mean BBB (Big Bad Bush) had anything to do with it. Indeed one thing from the congressional report mentioned earlier is that the office of the Vice President had lots of missing stuff. That has a familiar ring to it. Yes, read the GAO linked report all of it. VP Gore's office had stuff "missing".
No, that's not a defense-by-childhood-argument. There is a reason behind it. It's similar to tracking a server issue and it happening on the shift before you. Gore's office rightly concluded that that the FRA (Federal Records Act) did not apply to the OVP (Office of the Vice President), and neither did the PRA (Presidential Records Act). Therefore they were missing because they were not required to be there.
While we are under the impression that the records acts noted above require "all records", it is not true. For example, the the VP emails their [insert family relation here]] about dinner tonight or a movie tomorrow, those are not records covered by the acts. Nor should they be. We the people have no business reading those emails. Many may be shocked to learn the Sarbanes-Oxley (SOX) likewise does not require all email to be kept either. Again, this is sensible.
And it is nonsense. How does one decide what is a regulated email, and what is not? Who decides? This represents a fallacy in the notion that you can exclude certain emails from the system or rules. As a result most entities archive all of it. Which introduces other issues.
Regarding the current CIO not implementing something a prior CIO was working on
It is too easy and too simple to conclude that anything that relates or touches the "White House" is automatically controlled by the POTUS.
"they changed it to a manual process, where aides would manually sort emails one by one into individual PST files"
This is total nonsense, a generic backup application copies the files to tape. The only skill involved being the ability to swap the Monday tape for the Tuesday tape. The backup system copies all file types regardless of whether they are PST or not.
davecb5620@gmail.com
"A couple of years of not archiving emails due to configuration errors"
.. :)
.. :)
Like, where are all the tapes of the nightly, weekly and monthly SYSTEM BACKUPS. They do keep backups of the IT system of the Government of the worlds greatest democracy.
Like, I worked for a ten man architect outfit and even they managed to figure out that they needed backups. We didn't have to hire in a special contractor, we bought a HP Surestore tape unit
Disallow
was: Re:Interesting take on what REALLY happened
davecb5620@gmail.com
Do you understand how the Exchange MTA works? Do you understand the nature of mailbox storage on an Exchange server both pre and post E2K3? It sounds like you are confusing standard mailbox file folders with PST files which aren't even used for mail storage on a standard Exchange server.
For whatever reason, the current system is a clusterfuck and far too much time has passed without a resolution and with visible stonewalling on the part of the government at getting a solution to be complete absence of malice as the article points out.
Global rules can be implemented on an Exchange server to do the replication required to perform some sort of journaling even if an external system is required to implement more in-depth rules based upon content and/or recipients.
If you don't want to repeat the past, stop living in it.
Jeez, you're a twit. Even if I had mod points (and I haven't had them for a long time) it takes more than one moderator to knock off even a single point. Back when they still published Karma scores, it was about 25 karma points for a score point. So five different moderators would have to use all their points just for that one score point. To reach your current abysmal level, it would take 10 users.
Oh, wait, there's this evil conspiracy against you, right. It couldn't possibly be true that your posts are so irritatingly stupid that lots of different people mod you down, and continue to do so even when your starting score is -1!
Forget the conspiracies. Microsoft has no reason to shut you up. Clowns like you help Microsoft, because you convince people that the open source world is dominated by self-righteous idiots.
And enough with the sock puppets! If you had any creativity, you could maybe make it look like you're more than one person. But you don't, so you can't. You just make a bad reputation worse.