Bush Administration's E-Mail Deluge May Overload Archive System

Lucas123 writes "The Clinton administration generated 32 million e-mails. Bush's administration has generated 50 times as much data — 140TB, 20TB of which is email — which soon will have to be archived through a new government-built records management system. The new system may not be up to the task because the technology behind it may not be able to handle the sheer volume of data along with the fact that the Bush administration has been slow in providing the National Archives and Records Administration (NARA) with needed information about the records, according to a Computerworld story. Questions have also been raised about millions of missing e-mails from between March 2003 and October 2006. 'It wasn't until this summer that an intensive effort began to share information,' said Ken Thibodeau, director of NARA's Electronic Records Archives."

Comments

What's up?

[Anonymous Coward]

It hasn't helped that the Bush administration has been slow in providing NARA with needed information about the types and volume of data that will need to be archived. It wasn't until this summer that an intensive effort began to share information, Thibodeau says.

I can understand the reasoning that for national security, some information needs to be kept secret. The thing is, the more I hear of this administration's obfuscation of their communications and dealings, I can't help but wonder what in the World they are hiding.

Shadowy Government

[GMonkeyLouie]

Whenever I receive news that information that we're supposed to have access to from the Bush administration has gone missing, it makes me queasy. There's so much secrecy surrounding random little things that it's started to make me paranoid. Maybe it's just me wanting to blame the last eight years on a scapegoat, but I feel like someone at the top is trying to hide something really big and succeeding.

Re:Shadowy Government

[StopKoolaidPoliticsT]

Perhaps you're too young to remember, but Clinton's administration had a problem with missing emails during investigations too (Lewinsky, why hundreds of FBI records on their political enemies ended up in the White House, illegal campaign donations from China, etc).

I'd say it's par for the course and if you think just one side is doing shady stuff, it might be because you're a bit partisan.

Re:Shadowy Government

[GMonkeyLouie]

Well, Clinton never tried to insist that his VP wasn't part of the executive branch, never tried to put Harriet Miers on the supreme court... Actually I think the shadiest person in the administration is Cheney. He's certainly one of the only members of the 2001 Bush team left, and he keeps so many secrets! Also, he shot a man in the face one time. I love adding that to the end of my Dick Cheney rants. Is that too partisan?

Re:Shadowy Government

[mhollis]

I have understood this outgoing administration to be more than secretive. they're positively paranoid and the only administration in memory that was similar was Nixon. All internal memos have been classified first. Declassification only happens when there is a strong and abiding reason why the memo should be declassified. Contrast that with Clinton, where all internal memos are not classified, unless there was a strong and abiding reason why the memo(s) should be classified.

When Bush announced that his administration would immediately prepare for a transition (before the 4th of November, which was election day in the US), I assumed that the first course of action was that this Bush administration would do what the last Bush administration did: [Rip] the hard drives out of their computers and tried to erase "sensitive" computer files in the White House and West Wing.

To say that the Clinton Administration started with a "clean slate" was an understatement. Later, Clinton lawyers ignored the dangers of historical archive deletion when faced with Republican destruction of historical records. Presumably, they wanted a "pass" from future Republican administrations.

Republican administrations tend to be very secretive. Democratic administrations tend to not. I shall expect the Obama administration shall have to purchase all new computers -- or at least hard drives -- in order to simply start up in their first week. This is a horrid waste of taxpayers' money all in the name of whitewashing one's past deeds (for good or ill).

Due to record-keeping, we now know that Nixon did know about the Watergate break-in. And we do know that he was very interested in its coverup. Nobody can be prosecuted at this time for that (those who were found guilty have all ready served their time). I would be very interested to know if Reagan's CIA planted the stacks of AK-47s used as evidence by his administration that the attack on Grenada was justified. And we still do not know everything about the Iran-Contra affair. These historical records are worth keeping because, well after the Statute of Limitations, America gets another look at how an administration dealt with the world.

It is a shame that any Administration is that interested in "rewriting history" in order to unfairly burnish a legacy, which in the case of "W" is hardly salvageable.

Re:Number of emails generated.

[GMonkeyLouie]

Well, most of that 400GB from Clinton's administration was dirty pictures of interns. In all seriousness, though, I don't think the problem will be finding a way to store all that data. The real kicker will be finding information you need in it. Seems to me like the best way to hide relevant and/or damaging e-mails would be to have them stored right alongside truckloads of chain letters.

How much is spam?

[houghi]

How much of that is spam? I can imagine they are not allowed to delete spam. Spam has increased, so this would mean that all of it is still there.

The rest can mean a lot of different things. I am forced to work (otherwise no food) with 150MB excel files that I would love to put in a database and would take up at least 10 times less space. And I am not even talking about speed increase and ease of use, because somebody else has the file open, so I can not change the content.

Or perhaps Clinton did not keep everything. Or ...

Re:Number of emails generated.

[hedwards]

It isn't storage and it isn't finding it, the problem is preserving it long enough to look through and index it. I'm sure that Google and companies that do similar work have the technology to do it. I'm also quite sure that for the right price the Federal government could obtain software to do most of the heavy lifting.

The problem is that the Bush administration deliberately migrated only partially to a new system leaving it in a state of constant risk for bit rot and corruption. It's hard to say how much of it has already been lost due to incompetence.

And remember this is tax payer dollars and a Republican President, I'm sure he's OK with us writing a check for millions upon millions of dollars to correct his inept decision.

Throwing storage won't solve Exchange's issues

[CFD339]

There's an inherent architectural difference between storing mail in a database built on Microsoft's JET technology, and one which stores its data in something that is (although distinctly odd) very much like an xml data store. The Domino architecture makes segmenting the archive into manageable parts by date, by person, or by any combination thereof much simpler.

Essentially, the Domino architecture results in exactly what you describe -- throw more storage space at it and you can keep storing more data. The Microsoft architecture does not.