White House Email Follies

Presto Vivace forwards a link detailing a recent House Committee on Oversight and Government Reform hearing on the White House missing emails mess. David Gewirtz's report, carried in OutlookPower and DominoPower (in 6 parts, keep clicking), makes for scary reading. "If, in fact, the bulk of the White House email records are now stored in bundles of rotting PST files, all at or above their maximum safe load-level, that ain't good in a very big way... I object to using the inaccurate and inflated claim of excessive cost as a reason to avoid compliance with the Presidential Records Act."

Comments

Re:Delete the White House

[Master of Transhuman]

Treasonous? Ever heard of Sibel Edmonds? She's got your treason in detail. Do a Google or go here. And here's the Wikipedia entry.

Outlook storage

[DragonHawk]

What exactly is the safe load level for a PST file? There's been lots of replies to this, but I figured I'd organize a coherent and correct one.

Outlook has PST (Personal Store) and OST (Offline Store) files. PSTs are basically just local mail folder collections. OSTs are used to maintain local replicates of Exchange server mailboxes (so you can still use your email even if you're on the road). In Outlook 2003 "Cached Mode", Outlook also uses OSTs even when connected to the Exchange server, and synchronizes to the server in the background.

http://support.microsoft.com/kb/208480

PST and OST files -- I'll call them "Outlook stores" -- are both built around the same file format. There are two variants. The original format, which Microsoft sometimes called "ANSI", is limited to 2 Gi byte total size, and 64 Ki items per table. The table limit affects the number of items you can have in a folder, as well as the total number of folders you can have in a PST. (Outlook stores from Outlook 97 and earlier also had a table limit of 16 Ki items, but could be auto-upgraded in place to large tables in newer Outlook versions.)

http://support.microsoft.com/kb/197430

These store limits affected OST and PST alike, so even if you had a nice, capable Exchange server, you could still encounter problems with Outlook store limits.

http://support.microsoft.com/kb/288283

With Outlook 2003, Microsoft introduced a new Outlook store format. It's sometimes called the "Unicode" format. I'm aware of no documented limits on the file format. I'm sure there are some, but Microsoft doesn't document them. Microsoft didn't document the ANSI PST limits until long after they started causing data loss, either.

http://support.microsoft.com/kb/830336

In versions of Outlook prior to 2002, if you exceeded the store format limits, Outlook would give no immediate indication. The file would keep getting bigger, as the software didn't have checks for the limits. But it would corrupting things, too. In short, silently loosing data.

Eventually, the Outlook store would get so damaged it would stop working. Microsoft provided a utility to truncate the file to 2 GiB to make it work again, loosing more data in the process.

http://support.microsoft.com/kb/296088

In Outlook 2002, Microsoft added some code to check the limits of the store, and warn/stop if you reach them.

http://support.microsoft.com/kb/305108

In Outlook 2003, along with the Unicode format, Microsoft added a parameter at which it would consider a Unicode store "full", even though the format can keep going. The stock limit is 20 GiB; you can increase it with a registry tweak.

http://support.microsoft.com/kb/832925/

"ANSI PST" does not mean PST is a standard file format; that refers to the character sets/encodings the file uses.

Exchange Server uses an entirely different on-disk storage format, called EDB. There are technical limits, but they're insanely huge (16 TiB per store, 5 stores per database group). Exchange starts to run out of hardware resources (memory, mainly) long before you hit the file size limits. There are license-based size limits in some versions/editions of Exchange. 16 GiB in 2000 Standard, and 75 GiB in 2000 Standard SP2.