White House Email Follies

Presto Vivace forwards a link detailing a recent House Committee on Oversight and Government Reform hearing on the White House missing emails mess. David Gewirtz's report, carried in OutlookPower and DominoPower (in 6 parts, keep clicking), makes for scary reading. "If, in fact, the bulk of the White House email records are now stored in bundles of rotting PST files, all at or above their maximum safe load-level, that ain't good in a very big way... I object to using the inaccurate and inflated claim of excessive cost as a reason to avoid compliance with the Presidential Records Act."

Comments

What?

[palegray.net]

all at or above their maximum safe load-level What exactly is the safe load level for a PST file? If you're talking about stuff that's not reliably archived, the answer is "there isn't one." I recall reading a story a while back about a debacle wherein several thousand emails were "inadvertently" deleted... what the hell is so hard about implementing a sane backup policy? It's email, not terabytes of images or anything.

Re:What?

[cridanb]

the safe load level of a pst is around 2gb , after that it is subject to corruption
as to the amount of data , email systems are the largest systems on earth these days encompassing tens of terabytes of data in their live stores and tens of petabytes on tape

and yes they should have an archiving system not just doing tape back up tape

p.S if they used an enterprise email system like lotus domino this would not be a problem after all thats what the CIA uses

Re:What?

[vertinox]

What exactly is the safe load level for a PST file?

About 1.9 GB on an older PST file and anymore will crap out.

Outlook 2003 and greater will allow 20gb files, but they become horrendously slow after 5 to 10 gb.

And yes.. People will store gigabytes of email on an exchange server... Usually when they are emailing large videos, photoshop files, or do Desktop publishing work. Though I wonder what the Whitehouse doing to take up that much space.

Certainaly it wasn't powerpoints on intelligence reports.

Re:What?

[NeverVotedBush]

I think it's more likely that G.W. doesn't use e-mail because he can't find the "sendification" button.

Re:What?

[aeschenkarnos]

after that it is subject to corruption

This is the Bush Administration. It started subject to corruption.

Lost

[arigram]

They just need some excuse for "losing" dangerous email messages...

PST files for archiving

[call-me-kenneth]

Really, that's an accurate write-up - click past a couple of pages to get to the technical details. It'd be hilarious if it weren't so tragic.

After all, it's not like there aren't answers to the question "how shall I archive my user's email for legal and regulatory purposes?" (Disclaimer- I work for a player in that market, but we're not on the first page of results for that search. So I don't feel too bad. Oh, wait - )

digitally signed and time stamped archive

[wkk2]

Given all the convenient archival problems, every executive branch email should be archived as a PDF and digitally signed and time stamped by a secure server with the private key in protected hardware. The archive needs to be outside of the executive branch.

The real question

[WindBourne]

is why are the dems allowing the White house off? They should be paying to have all the PST's restored. By now somebody has told them that the white house lied about the costs of the PST files. The need to go after them for perjery as well as getting the emails.

What really bothers me is that not this white house makes nixon and reagan look like boy scouts, but that the dems PROMISED to go after them, and really has done nothing.

Who cares?

[MillionthMonkey]

All their interesting stuff went through private mail servers at the RNC to evade responsibility for document retention under the Presidential Records Act. The RNC systematically destroys its emails and Bush has even invoked executive privilege in ordering the RNC to defy Congressional subpoenas to produce them.

Re:Who cares?

[cfulmer]

There is a completely acceptable reason for going through the RNC, and it's the same reason that the Clinton administration used similar services at the DNC: there are two competing acts. You've mentioned the Presidential Records Act, which is intended to protect the official records of the White House. There's also the Hatch Act, which (among other things) prevents government computers from being used for political activities. Emails regarding political activities went through the RNC servers (or, in the case of the Clinton Administration, the DNC servers); emails regarding activities as President, i.e. the Presidential Records, are supposed to go through the White House email system, where they are backed up and archived. So, you cannot infer an intent to violate the Presidential Records Act merely from the fact that outside services were used.

Re:Who cares?

[MillionthMonkey]

There's also the Hatch Act, which (among other things) prevents government computers from being used for political activities. Emails regarding political activities went through the RNC servers (or, in the case of the Clinton Administration, the DNC servers); emails regarding activities as President, i.e. the Presidential Records, are supposed to go through the White House email system, where they are backed up and archived. So, you cannot infer an intent to violate the Presidential Records Act merely from the fact that outside services were used.

There, I helped you out with a few BOLD tags. Your mistake is assuming that an email either falls under the scope of partisian political activity or represents communication at official levels regarding government business. They sent emails that were both.

When you're having an email conversation (for example) about which U.S. Attorneys should be fired by the president for prosecuting Republican offenses or for not going after Democrats in election years, and what the cover stories for the firings should be, you're mixing political partisan activity and official government business. Since these emails were illegal for government officials to be sending, they obviously didn't use the White House email infrastructure to send them. Even these guys weren't that stupid. They were dumb enough, though, to indicate in WH emails when they were going to continue certain conversations, regarding planned activities to be carried out in an official capacity, in nongovernmental channels (RNC, gwb43.com, Yahoo Mail) to avoid them from ever becoming public.

But the purpose of the Hatch Act (passed in 1939) isn't just to protect Outlook servers from private or partisan use- it forbids the use of any federal agencies or resources to assist in partisan activities. That would include both WH email servers and the U.S. Department of Justice.

Re:wow

[laird]

The problem, I believe, is that the Presidential Records Act has no enforcement provisions or penalties for non-compliance. Thus, if the White House prefers to ignore it, there's no risk in doing so. So if the value of non-compliance is higher than the value of compliance, which is the case right now, the PRA loses.

This is not simply a case of incompetent IT staff setting up a system badly. The White House had an email system that by all accounts worked very well, archiving everything properly, and it was shut down and the staff let go, and the new system was set up by someone over-ruling their own IT staff in order to make sure that it couldn't work properly. That means that someone made the decision to spend a lot of time and money to eliminate a system that worked properly, to replace it with a system that didn't, over-ruling the recommendations of their own IT staff, which can only have been done intentionally.

What would be ideal would be for the PRA to be given real teeth so that the cost of violating it becomes clearly higher than the cost of not hiding whatever it is you want hidden. Given the extremely high value of keeping embarrassing or illegal behavior secret, the penalty needs to be extremely high as well, as it is for destroying evidence. That is to say, courts should presume that the records that were destroyed were incriminating. Judges take destroying evidence of a crime quite seriously.

Big, corrupted PST files? No problem.

[Animats]

Big, corrupted PST files? No problem. Just get Stellar Phoenix PST Repair. "Stellar Phoenix can repair PST files in all scenarios including the common issues listed below ... Oversized PST files with 2Gb problem. Recovers from encrypted files. Recovers deleted e-mails." U.S. Government price $249 with CD. Immediate download available. Recommended by PC Magazine.

This little problem can be overcome. Just get some image copies of those tapes out to the Internet Archive or Wikileaks, and all the technical problems will be quickly dealt with, the data will go on line, and it will all be indexed.

Re:Delete the White House

[dpilot]

> operates at all near the level of minimum performance required

We should be so lucky to see such a high standard.

> Anyone still think all this incompetence that always protects Bush and his team is some kind of accident?

I would rather. The alternative explanation is EVIL and probably treasonous.

Re:wow

[postbigbang]

While there's no seeming penalty, civil or criminal, there is a bigger penalty: ongoing confidence in government. The hubris and arrogance has become intolerable. This is just one symptom of a government gone berserk. Vetoing a bill on torture was another. Sliming the US House of Representatives because it won't pass a bill allowing the telcos to violate the very tenets of liberty in the constitution is another. The list is long. The list is sad. These are evil days, my friends.

Re:Delete the White House

[Master of Transhuman]

Treasonous? Ever heard of Sibel Edmonds? She's got your treason in detail. Do a Google or go here. And here's the Wikipedia entry.

Outlook storage

[DragonHawk]

What exactly is the safe load level for a PST file? There's been lots of replies to this, but I figured I'd organize a coherent and correct one.

Outlook has PST (Personal Store) and OST (Offline Store) files. PSTs are basically just local mail folder collections. OSTs are used to maintain local replicates of Exchange server mailboxes (so you can still use your email even if you're on the road). In Outlook 2003 "Cached Mode", Outlook also uses OSTs even when connected to the Exchange server, and synchronizes to the server in the background.

http://support.microsoft.com/kb/208480

PST and OST files -- I'll call them "Outlook stores" -- are both built around the same file format. There are two variants. The original format, which Microsoft sometimes called "ANSI", is limited to 2 Gi byte total size, and 64 Ki items per table. The table limit affects the number of items you can have in a folder, as well as the total number of folders you can have in a PST. (Outlook stores from Outlook 97 and earlier also had a table limit of 16 Ki items, but could be auto-upgraded in place to large tables in newer Outlook versions.)

http://support.microsoft.com/kb/197430

These store limits affected OST and PST alike, so even if you had a nice, capable Exchange server, you could still encounter problems with Outlook store limits.

http://support.microsoft.com/kb/288283

With Outlook 2003, Microsoft introduced a new Outlook store format. It's sometimes called the "Unicode" format. I'm aware of no documented limits on the file format. I'm sure there are some, but Microsoft doesn't document them. Microsoft didn't document the ANSI PST limits until long after they started causing data loss, either.

http://support.microsoft.com/kb/830336

In versions of Outlook prior to 2002, if you exceeded the store format limits, Outlook would give no immediate indication. The file would keep getting bigger, as the software didn't have checks for the limits. But it would corrupting things, too. In short, silently loosing data.

Eventually, the Outlook store would get so damaged it would stop working. Microsoft provided a utility to truncate the file to 2 GiB to make it work again, loosing more data in the process.

http://support.microsoft.com/kb/296088

In Outlook 2002, Microsoft added some code to check the limits of the store, and warn/stop if you reach them.

http://support.microsoft.com/kb/305108

In Outlook 2003, along with the Unicode format, Microsoft added a parameter at which it would consider a Unicode store "full", even though the format can keep going. The stock limit is 20 GiB; you can increase it with a registry tweak.

http://support.microsoft.com/kb/832925/

"ANSI PST" does not mean PST is a standard file format; that refers to the character sets/encodings the file uses.

Exchange Server uses an entirely different on-disk storage format, called EDB. There are technical limits, but they're insanely huge (16 TiB per store, 5 stores per database group). Exchange starts to run out of hardware resources (memory, mainly) long before you hit the file size limits. There are license-based size limits in some versions/editions of Exchange. 16 GiB in 2000 Standard, and 75 GiB in 2000 Standard SP2.