Slashdot Mirror
White House Email Follies
Presto Vivace forwards a link detailing a recent House Committee on Oversight and Government Reform hearing on the White House missing emails mess. David Gewirtz's report, carried in OutlookPower and DominoPower (in 6 parts, keep clicking), makes for scary reading. "If, in fact, the bulk of the White House email records are now stored in bundles of rotting PST files, all at or above their maximum safe load-level, that ain't good in a very big way... I object to using the inaccurate and inflated claim of excessive cost as a reason to avoid compliance with the Presidential Records Act."
512 MB RAM, 20 GB disk, 200 GB transfer, five datacenters. $19.95/month.
Hmmm, I suppose there is another theory which states that this has already happened...
They just need some excuse for "losing" dangerous email messages...
After all, it's not like there aren't answers to the question "how shall I archive my user's email for legal and regulatory purposes?" (Disclaimer- I work for a player in that market, but we're not on the first page of results for that search. So I don't feel too bad. Oh, wait - )
Given all the convenient archival problems, every executive branch email should be archived as a PDF and digitally signed and time stamped by a secure server with the private key in protected hardware. The archive needs to be outside of the executive branch.
is why are the dems allowing the White house off? They should be paying to have all the PST's restored. By now somebody has told them that the white house lied about the costs of the PST files. The need to go after them for perjery as well as getting the emails.
What really bothers me is that not this white house makes nixon and reagan look like boy scouts, but that the dems PROMISED to go after them, and really has done nothing.
I prefer the "u" in honour as it seems to be missing these days.
Is anyone out there still thinking that this White House operates at all near the level of minimum performance required from people in its job?
Anyone still think all this incompetence that always protects Bush and his team is some kind of accident?
--
make install -not war
I find the fact that the US Government runs on Lotus Notes more scary than the fact that they don't have any sort of backup strategy.
I bet if you go over to the IRS, those guys have a rock-solid backup going back many years.....
All their interesting stuff went through private mail servers at the RNC to evade responsibility for document retention under the Presidential Records Act. The RNC systematically destroys its emails and Bush has even invoked executive privilege in ordering the RNC to defy Congressional subpoenas to produce them.
"We lead the world in computerized data collection"
To hear that email from the White House has been 'deleted', 'misplaced' or simply 'missing' is truly a slap in the face to the American people.
This stuff pisses me off completely...that and the 'dangling chad debacle'.
~ Ron Fitzgerald
Here I am, some lowly line level system tech for a smallish town, and I'd be handed my marching orders were I even a quarter as incompetent as the white house staff seems to be. Which leads me to suspect:
1) Either they are that incompetent, and it's just a symptom of big government not knowing it's ass from it's face
OR
2) These people are purposefully appearing this inept.
Either option isn't pleasant, and both lead to a serious problem with our government where there will likely be no repercussions from this.
But then, we all knew that already, didn't we?
Mod me down with all of your hatred and your journey towards the dark side will be complete!
Considering how much we're spending in what are arguably other countries' wars, I'd find a claim of "excessive cost" for anything laughable.
Someone just needs to 'leak' that one of those archived emails contains a transcript of Osama bin Laden endorsing Barack Obama for president. Then stand back because we are all going to be sprayed with WH archived emails.
Technically, murder-suicide does not violate the golden rule.
Elect a president who doesn't believe in government and watch the government begin to fail at everything.
Hic iacet Arthurus, rex quondam rexque futurus.
So much for those that say watching CSPAN coverage of legislative hearings is as boring as watching paint dry.
The article, despite being spread across multiple pages, characterises the hearing fairly, so I won't bother reiterating except to say that the committee members were indeed uninformed, the witnesses were somewhere between clueless and dishonest, and the politics injected into the situation (notably from the Republicans) was so thick that I wondered whether anything could be agreed upon or any of the issues resolved. Hell, by the end of it, I doubt anyone really knew what the technical issues were, myself included.
The saving grace was watching (no one could hear what he was saying) the soft-spoken White House archivist and remembering the joke about how to tell the difference between an introverted and extroverted geek. Instead of shoes, it was microphones.
Your government in action, folks. The bad guys trying to cover up, the good guys trying to find out what's going on, and both groups taking its cues Microsoft weenies.
There, fixed that for you.
Big, corrupted PST files? No problem. Just get Stellar Phoenix PST Repair. "Stellar Phoenix can repair PST files in all scenarios including the common issues listed below ... Oversized PST files with 2Gb problem.
Recovers from encrypted files. Recovers deleted e-mails." U.S. Government price $249 with CD. Immediate download available. Recommended by PC Magazine.
This little problem can be overcome. Just get some image copies of those tapes out to the Internet Archive or Wikileaks, and all the technical problems will be quickly dealt with, the data will go on line, and it will all be indexed.
Are you new here?
Honestly, this is a classic, almost Hollywood-style presidential-aid-villain-style tactic.
First, you dry up all the funding to something so that you can later claim there was not enough money to do it right.
And in the process of doing it "not right" some important stuff gets lost so the people in charge can't be charged later (which they can't anyway, because presidents make a habit of indemnifying their successors and most of the senior staff around them, because if they wouldn't, their successor wouldn't indemnify them...).
Still wondering why people actually get out of their bed and vote?
Windows 2000 - from the guys who brought us edlin
Elsewhere in the world destruction of evidence is taken as guilt. Is that not the case in the USA?
In the USA, it matters a whole lot who you're talking about whether or not XYZ counts as guilt.
Format decides the maximum size of a PST.
www.voiceofthehive.com - Beekeeping and Honeybees for those who don't.
"It depends on what the meaning of 'is' is."
Excuse me, but please get off my Pennisetum Clandestinum, eh!
The cost of retrieval is normally included in evaluating any backup solution. If I need a particular SLA I need to evaluate the cost of a given solution in restoring, not just backing up. If I didn't evaluate that during my analysis I have to eat that cost when providing the service. And accessing PSTs via code is easy, so it's not like something couldn't just suck the mail out of the PST as needed. That said a PST isn't an archival format as far as I'm concerned. Oh, and "PSTs above their safe level" depends entirely on what the format of the PST is.
www.voiceofthehive.com - Beekeeping and Honeybees for those who don't.
" Mr. ISSA. Okay. So here we have a situation where the Clinton Administration is on a platform that has to be phased out. Simply, they lost the war of who is going to supply emails. A period of time goes on in which Yes, we are dealing, to Dr. Weinstein's concern, with getting good archives, but we are also dealing with the fact that I can't play my Betamax tapes any more, either, and I can't seem to find anybody who has a Betamax player any more."
Maybe Mr Issa should look here. And Republicans are the ones who lose wars these days.
Meanwhile, the General Services Administration just saved a million bucks of taxpayers money with Notes.
I cannot stand Microsoft Exchange in any of it's versions. It is nothing but an I.T. headache of the worst kind. Try backing up the mail store, I dare you. After spending several thousand more dollars you'll be close but no cigar.
In my former place of employment we used a lot of OSS for things like web, email, database, etc. Even Samba. We had a few MS-SQL environments but I stayed as far from those as I could. For email we used Qmail with a SquirrelMail front end, and for web it was Apache/Plone and databases were MySQL.
The nice thing about Qmail is it stores email in user home folders. They're flat files that are easily replicated and backed up.
When the new administration came in the Director of Admin was paranoid about the fact that I.T. could see her email folder. So they went out and spent a shitload of money on AD, Exchange, etc.
That was a year ago. They still don't have it all running.
Without getting into the technical issues ...
Where's the righteous geek who will release all these emails ? Lost, sure (hack cough right). Someone has a real copy. And, that copy is for blackmail purposes....
Somewhere, someone has to do the right thing and release this stuff.
Want to improve your Karma? Instead of "Post Anonymously", try the "Post Humously" option.
With the morons they have on staff up there - and that includes Bush - they can't be sure all sorts of incriminating stuff isn't in them. In fact, they probably assume there is.
So they stonewall.
Read TFA. They're making estimates of the cost of recovery of the PST files as wildly off the mark. They're claiming it would cost $50K just to recover ONE PST file! And half a million bucks to recover 5,000 PST files!
That's deliberately false testimony - i.e., perjury.
Face it, folks. This country is being run by criminals now - just like in Warren Ellis' comic, "Reload". Look up Sibel Edmonds on Google and see just how bad it is.
Richard Steven Hack - This sig is TOO GODDAMN SHORT TO DO ANYTHING USEFUL WITH! MORONS!
Slashdotters may think that it looks bad for the President's email systems to be horribly unreliable. Compared with what was probably in those emails, this is nothing. In fact, this whole missing email thing is brilliant, and from the Pres's perspective, a job well done. The missing content gives the Pres a get-out-of-jail-free card. Not that he really needs one, since our Constitution makes kings out of our Presidents here in America, with the ability to do whatever they damn well please.
Japanese scientist: Technically, sir, tomatoes are fags. Military scientist: He means fruits.
If you want the context that would make it surprising if the white house did anything other than hide every aspect of what they did, take a look at the book Worse Than Watergate. Or get it on audio book and listen to it. It certainly contains some bias, but that's an unfortunate and unnecessary detriment to a text that very thoroughly documents numerous counts of unreasonable and often illegal attempts to maintain a monumental shroud of secrecy over everything this administration does.
I found the bit about him blatantly violating Texas law by keeping all his gubernatorial records completely sealed very interesting. This was well documented and easy to find out about before he was elected in 2000, but the media barely touched it. It turns out it was a pretty good indicator of future performance.
Can anyone tell me how to set my sig on Slashdot?
IANAAmerican, but surely this is a non-partisan issue of heedless and convenient law-breaking, not a squabble solved by fiery press releases from Democrat HQ.
Please, USA, lock these criminals up already.
you had me at #!
Maybe citizens who have some free time on their hands and the experience/clearance necessary can make an organization to do these things for free for Govt agencies. That would be pretty cool because it could be a non-partisan group who does this.
But really, what are the odds that ANYONE in govt would want this? Too bad though.
My abilities are only limited by my imagination
And yes.. People will store gigabytes of email on an exchange server... Usually when they are emailing large videos, photoshop files, or do Desktop publishing work. Though I wonder what the Whitehouse doing to take up that much space.
Most email systems are poorly factored information because they duplicate a message for every last reader of a given message. It would save a lot of space and traffic if a given attachment or message was stored in one and only one place rather than replicated en-mass.
Of course, the security for centralizing items properly without being read by non-recipients complicates things, but shouldn't be a show-stopper. Also, the retainment date cutoff of the central server and individuals may be different, which makes some people want to be pack-rats if they can't trust the central system to keep stuff long enough.
A related problem is that people often CC copy everybody and their dog to cover their butts. Thus, we get bajillion messages that don't relate to us.
The whole idea of email needs a big rethink. It's become a jungle monster.
Table-ized A.I.
Not really sure what you are getting at here. And one really has to wonder what took a year for an AD/Exchange install. Or are we including waiting for 64 bit hardware to deploy W2k3/E2k7?
www.voiceofthehive.com - Beekeeping and Honeybees for those who don't.
I'm always curious to know why people bring up irrelevant facts at times like this. I don't get it.
Are you a moral relativist, arguing that no one can ever be expected to behave better than the previously-established worst? If so, your position leads to Zimbabwe or something like it overnight.
Are you a political apologist, like the dishonest bastards who used to bring up the behaviour of the United States every time anyone criticised the Soviet Union?
Or are you an idiot, who thinks that every question only has two sides, and if you're not for our answer (the Republicans, apparently) you must be for their answer (the Democrats, I guess, in this case)?
I'm personally betting on idiot.
Here's a clue: every interesting question has more than two sides, and other people's bad behaviour, no matter how recent and no matter how geographically nearby, is completely irrelevant applying even the most modestly objective standard to someone's current behaviour.
And in this case there is a clear, obvious, relatively uncomplicated objective standard to apply: the good of the nation and trust in government. By that standard, the Bush administration has failed on multiple fronts, and the Whitehouse e-mail mess is objectively one of them.
I've been doing email admin for a couple of decades now...I started off running a Fidonet system, and have managed email servers for Fortune 500 companies. I don't use Exchange. After Opus and Maximus, all of the mail servers I have worked on were Unix. And under Unix, even with the beast that is Sendmail, it is *easy* to make a complete duplicate of all email going through a server. And as the emails are stored as text files, you are really only limited by the size of your file system's maximum file size. Archive those files monthly, and you won't see any performance degradation. Use a file system that uses compression, and run RAID 01, and you could probably go for a year, depending on the system speed, and mail volume.
Such an archive email server could easily run in front of the exchange server, and be transparent to it.
Is the US government so clueless? Or did they do what they did on purpose?
ttyl
Farrell
CAN-CON 2019 - Ottawa's only book oriented Science Fiction Convention! October 18-20, Sheraton Hotel, Ottawa, Canada h
Congress has the POWER to ungag Sibel Edmunds. And yes, the majority (though not super majority) dems have the ability to do this. WHY HAVE THEY NOT?
I prefer the "u" in honour as it seems to be missing these days.
In the USA, it matters a whole lot who you ARE. Office of the President kind of trumps most legal arguments.
.. paranoid crackpot leftover from the days of Amiga.
Clinton did it != acceptable.
And on "teachers" wasn't Nixon's "14 missing minutes"* the template?
*I think I am off on the number of minutes, but you get the idea...
emt 377 emt 4
We hope your rules and wisdom choke you / Now we are one in everlasting peace
Thinking that a president might be violating the law is treasonous? That's perhaps the most unamerican thing I've ever heard uttered.
dragonhawk@iname.microsoft.com
I do not like Microsoft. Remove them from my email address.
Outlook has PST (Personal Store) and OST (Offline Store) files. PSTs are basically just local mail folder collections. OSTs are used to maintain local replicates of Exchange server mailboxes (so you can still use your email even if you're on the road). In Outlook 2003 "Cached Mode", Outlook also uses OSTs even when connected to the Exchange server, and synchronizes to the server in the background.
http://support.microsoft.com/kb/208480
PST and OST files -- I'll call them "Outlook stores" -- are both built around the same file format. There are two variants. The original format, which Microsoft sometimes called "ANSI", is limited to 2 Gi byte total size, and 64 Ki items per table. The table limit affects the number of items you can have in a folder, as well as the total number of folders you can have in a PST. (Outlook stores from Outlook 97 and earlier also had a table limit of 16 Ki items, but could be auto-upgraded in place to large tables in newer Outlook versions.)
http://support.microsoft.com/kb/197430
These store limits affected OST and PST alike, so even if you had a nice, capable Exchange server, you could still encounter problems with Outlook store limits.
http://support.microsoft.com/kb/288283
With Outlook 2003, Microsoft introduced a new Outlook store format. It's sometimes called the "Unicode" format. I'm aware of no documented limits on the file format. I'm sure there are some, but Microsoft doesn't document them. Microsoft didn't document the ANSI PST limits until long after they started causing data loss, either.
http://support.microsoft.com/kb/830336
In versions of Outlook prior to 2002, if you exceeded the store format limits, Outlook would give no immediate indication. The file would keep getting bigger, as the software didn't have checks for the limits. But it would corrupting things, too. In short, silently loosing data.
Eventually, the Outlook store would get so damaged it would stop working. Microsoft provided a utility to truncate the file to 2 GiB to make it work again, loosing more data in the process.
http://support.microsoft.com/kb/296088
In Outlook 2002, Microsoft added some code to check the limits of the store, and warn/stop if you reach them.
http://support.microsoft.com/kb/305108
In Outlook 2003, along with the Unicode format, Microsoft added a parameter at which it would consider a Unicode store "full", even though the format can keep going. The stock limit is 20 GiB; you can increase it with a registry tweak.
http://support.microsoft.com/kb/832925/
"ANSI PST" does not mean PST is a standard file format; that refers to the character sets/encodings the file uses.
Exchange Server uses an entirely different on-disk storage format, called EDB. There are technical limits, but they're insanely huge (16 TiB per store, 5 stores per database group). Exchange starts to run out of hardware resources (memory, mainly) long before you hit the file size limits. There are license-based size limits in some versions/editions of Exchange. 16 GiB in 2000 Standard, and 75 GiB in 2000 Standard SP2.
dragonhawk@iname.microsoft.com
I do not like Microsoft. Remove them from my email address.
Actually we implemented an Rsnapshot server to run every 15 minutes and we kept 8 hours of backups. The nice thing was Rsnapshot did incremental very well.
As to the AD/Exchange conversion, I was still there when all the gear and software rolled in. It all came in during April 2007. I was talking to the other systems guy and asked what his gut estimate was for lights-on. He figured January 2008, here we are in March and it still isn't up. That's because they laid me off and the other systems guy all of a sudden got stupid on them. It's too funny.
Want to improve your Karma? Instead of "Post Anonymously", try the "Post Humously" option.
> The only way to be sure about archiving email is to copy all messages immediately on the server-side. Incidentally, it's easy to do that with
> Exchange (and most other enterprise email systems).
Qmail: compile qmail with QEXTRA set and use the qmail-tap patch from init7.
Or postfix+always-bcc.
But you still need some kind of archiving solution anyway...
Windows 2000 - from the guys who brought us edlin
I suppose you could cite incompetent admins, but an incompetent admin will always lead to data loss, sooner or later.
Microsoft deserves blame for the lack of limit checking in the PST code in Outlook, and the data loss which follows, but that is a problem which only affects stand-alone Outlook installations, not well-run Exchange systems. it is not because it is a superior mail server. Exchange actually has some selling points when coupled with Outlook. I don't think I'd call it worth the costs in the big picture -- you can get much of the same 20% that people want using more lightweight, easier to manage, Free and free software. But Exchange does have some unique features. It's a resource hog, and touchy if mis-handled, but it's not entirely without merit.
dragonhawk@iname.microsoft.com
I do not like Microsoft. Remove them from my email address.
from the FBI to blackmail politicians, but her own files that all other candidates disclose?
Why in the world are they using client-side PST files as mail archives in the first place? That's no proper place for record preservation. Personal email storage should only contain messages that are current and relevant to the user who they belong to.
The right way to archive email messages is on the server side. At my company, our MS Exchange server uses a mail archiver which automatically stores all incoming and outgoing messages as they pass through. So even if I delete a message from my mailbox (I use Evolution + IMAP), it's still available in the archive.
We also have a web-based archive search tool. Granted, it's slow as molasses and sometimes doesn't come back from an archive search at all, but the point is the deleted email messages are still there. I think...
Microsoft has three major database engines.
.EDB extension and have transaction logs in separate .LOG files.
.MDB extension. Jet Had has been around forever, in Access and in MDAC (the embedded database technology that's part of Windows and/or Visual Studio, depending on who you asked and what year it was).
.MDF file extension. Microsoft bought it from Sybase, along with the rest of what became MS SQL Server.
One is ESE, Extensible Storage Engine, also called "Jet Blue". I think this was invented for Exchange. ESE is also used for Active Directory, Windows Desktop Search, and various other systemy things (like the DNS and WINS databases). ESE databases typically use an
The second engine is Jet Red. This is the one used by MS Access, and typically had an
The third engine is the one used by MS SQL. It uses an
There was supposedly an effort inside Microsoft to adopt the MS-SQL engine as the storage backend for Exchange, but they found it didn't work very well for that. (Same situation as with MySQL and it's various database engines: One size does not fit all.)
The Microsoft party line is that eventually, Jet Red will go away, to be replaced by the MS-SQL engine. So eventually MS Access will end up using the MS-SQL engine. This may be here in Access 2007; I haven't been keeping track.
"Jet" may be an acronym for "Joint Engine Technology", or that may be a backroynm, and the word "Jet" was just used because it sounds cool.
dragonhawk@iname.microsoft.com
I do not like Microsoft. Remove them from my email address.
I've been involved in things technical since 1974, and it shames me to be forced to acknowledge that people in my field - that strange digital world where "True or False" is a fundamental fact of life - are participating in obfuscating a basic fact:
This Administration is crooked, and there are enough equally corrupt Republicans in the House and Senate to shield them from the consequences that their criminality merits.
All technical details are moot.
Orwell: "In a Time of Universal Deceit, telling the Truth is a Revolutionary Act"
There is no right to feel safe thru security vaudeville at the expense of everyone's freedom, privacy and tax money.
Utterly trivial on any non-MS system from even before MS Exchange existed so long as you have two computers. Alias it to somewhere else as well - in the most simple case call it archive@backup if you like and copy that to other media at whatever interval you wish. Extra licencing costs and weird file formats are the only impediment to doing that on MS systems as well. On recent versions of MS Exchange backups are possible without a great deal of pain (backups good enough for bare metal recovery used to only be possible by stopping most Exchange services for the duration of the tape run).
Exchange 2000 and 2003 are pretty similar, and SIS works the same in both. In Exchange 2007, from what I've read, newly-created Information Stores only do SIS on attachments, not message bodies. So if I send a 20 KB message with a 100 MB attachment to five people, it uses 20 KB * 5 = 100 KB + 100 MB, or 100.1 MB. Still a worthwhile gain, in my book, although I think Microsoft's decision to just drop SIS support for bodies is typical of their arrogance.
:)
I don't know about about Louts Notes, but given that it's the "other big player" in the commercial email market, I was thinking it would do SIS, too. From what you say, I guess that's not cut-and-dry in Notes land, either.
dragonhawk@iname.microsoft.com
I do not like Microsoft. Remove them from my email address.
for all the sensitive stuff.
Engineering is the art of compromise.
The USA's vast reserves of goodwill, built up over the 20th century, has largely been eroded over GW's rule. Sure we might have laughed at Clinton and his blowjobs, but GW's illogical, immoral and illegal activity makes us very worried. At least GW's leaving, but his replacement better be a lot better and needs to send some very reassuring messages - not just to Americans but to the rest of the world too.
Engineering is the art of compromise.
dragonhawk@iname.microsoft.com
I do not like Microsoft. Remove them from my email address.