Slashdot Mirror


Bush Administration's E-Mail Deluge May Overload Archive System

Lucas123 writes "The Clinton administration generated 32 million e-mails. Bush's administration has generated 50 times as much data — 140TB, 20TB of which is email — which soon will have to be archived through a new government-built records management system. The new system may not be up to the task because the technology behind it may not be able to handle the sheer volume of data along with the fact that the Bush administration has been slow in providing the National Archives and Records Administration (NARA) with needed information about the records, according to a Computerworld story. Questions have also been raised about millions of missing e-mails from between March 2003 and October 2006. 'It wasn't until this summer that an intensive effort began to share information,' said Ken Thibodeau, director of NARA's Electronic Records Archives."

17 of 169 comments (clear)

  1. Number of emails generated. by Ice+Wewe · · Score: 5, Insightful

    "The Clinton administration generated 32 million e-mails. Bush's administration has generated 50 times as much data -- 140TB, 20TB of which is email -- which soon will have to be archived through a new government-built records management system.

    Well, to be fair, email wasn't quite as popular during Clinton's administration as it is now. Then again, the 400GB of e-mails that the Clinton administration must have generated (if it is 50 times less than 20TB) must have been rather hard to store when he left office.

  2. First, not enough emails... by Anonymous Coward · · Score: 4, Insightful

    ...Now too many many emails.

    Whining is Washington's most favorite thing to do.

    1. Re:First, not enough emails... by Anonymous Coward · · Score: 1, Insightful

      Whining is Washington's most favorite thing to do.

      What Bush is about to do is like giving a teacher a stack of books from the library and saying "here's my research paper".
      What Bush has already done is cut out chapters from those books, so that his teacher won't be able to get the full picture.

      Thus you both have too much and not enough.

  3. Text only, no html by Teun · · Score: 5, Insightful
    Start by mandating text only mail.

    No more fancy signatures and html crap will cause a 60-80% drop in volume if not more.
    Mandate the Usenet way with replies after the original, (it will) teach people to cut irrelevant repeats.
    Stop the addition of stupid and ineffective disclaimers.

    Teach the use of (ftp) servers for sharing large documents, no more Microsoft sized attachments, send a link.

    --
    "The likes of Facebook and WhatsApp are free to those whose privacy is of zero value."
    1. Re:Text only, no html by ai3 · · Score: 4, Insightful

      I would rather buy another hard disk than waste precious time editing the mail I'm replying too, in most cases it simply isn't necessary. For Usenet it's a different story as many people read it, so it's worth the effort.

    2. Re:Text only, no html by malkavian · · Score: 5, Insightful

      Longer email threads seem to end up forwarded and brought to the attention of many people you never expected at the outset.
      Judicious editing of the emails to include only the relevant sections for the replies, giving the context of the emerging thread of conversation means that someone being brought up to speed with that segment of the conversation doesn't need to trawl through masses of irrelevant junk to get at the meat of the issue.
      I tend to do it as an efficiency gain, rather than taking storage space into account. All comes back to that quote you hear people come out with after sitting through a bad movie "Well, that's an hour of my life I'll never get back". It may only be a few minutes at a time, but they mount up over time. Plus, crafting things to cut to the heart of the matter puts things into sharp perspective, and means people are far less likely to digress, saving even more wasted time.

    3. Re:Text only, no html by kevin_conaway · · Score: 4, Insightful
      Did you stop paying attention to email in 1995?

      No more fancy signatures and html crap will cause a 60-80% drop in volume if not more.

      I know you hate it when your mom or the boss' secretary at work sends out a cutesy formatted email but some people can actually use HTML email effectively in lieu of sending a document or a link

      Mandate the Usenet way with replies after the original, (it will) teach people to cut irrelevant repeats.

      Irrelevant repeats for you may be important context for someone else.

      Stop the addition of stupid and ineffective disclaimers.

      Often times, those disclaimers are required by law. Most people don't add them for fun or to make themselves feel important.

      Teach the use of (ftp) servers for sharing large documents, no more Microsoft sized attachments, send a link.

      FTP? Are you serious? Sending documents by carrier pigeon is more secure and reliable than FTP

    4. Re:Text only, no html by Reality+Master+101 · · Score: 4, Insightful

      Sheesh. I call this phenomenon "technological puritanism". All tech must be ugly! 80 columns should be enough for anyone! Fixed-width fonts were good enough for my granddaddy, they were good enough for me, and they should be good enough for everyone! Words are worth a thousand pictures! Get off my damn lawn!

      Nothing personal, but if people like you were in charge of the world, we'd all be living in gray, cast concrete cubes. Think of the efficiency! No more wasted paint. You can just make a bigger house by stacking the blocks and adding a ladder.

      Most of us *like* color, pictures, paragraphs, and most of all, convenience. Use FTP when I can just add an attachment that goes directly to the source? Give me a frickin' break. No one gives you respect points when you prove how miserably you can live.

      Let's put this in perspective... that 120 terabytes costs 12,000 dollars in hard drives. Retail at Fry's. The entire output of the Bush Administration costs less than what they probably spend on coffee in a month.

      P.S. And, yes, this is from someone who used a teletype in high school, and was ecstatic when we got a 300 baud modem (whoa! It's almost 3 times faster than the ol' 110!) and a Televideo terminal. Those days were not better.

      --
      Sometimes it's best to just let stupid people be stupid.
    5. Re:Text only, no html by Joce640k · · Score: 2, Insightful

      140 1TB Hard disks (plus another for RAID) probably costs less than a couple of government office chairs so what's the problem?

      [Most likely the fact that it's in secret, proprietary formats and spread across hundreds of PCs instead of being archived by the mail gateway]

      --
      No sig today...
    6. Re:Text only, no html by the_other_chewey · · Score: 2, Insightful

      Stop the addition of stupid and ineffective disclaimers.

      Often times, those disclaimers are required by law. Most people don't add them for fun or to make themselves feel important.

      I don't know about the situation in the USA, but in most parts of the world, this is exactly the reason.
      "Because everybody else does it" is another.
      In multiple european countries, those disclaimers are entirely worthless, and even in some cases came back
      to bite those using them in court by proving that the sender was aware that some piece of information might
      end up in the wrong place.

      Disclaimers don't replace common sense or encryption.

    7. Re:Text only, no html by jesterzog · · Score: 2, Insightful

      If it anything like our corporate mail server I would bet you the number one space filler is people making minor changes on documents then reattaching them and forwarding them back to the same 50 people who just got the previous version of the document, repeated over 100 iterations as the email soon becomes a 2GB mess.

      In our organisation (government but non-US) we just give people a document management system and we educate people about why they should use it. If that doesn't work we point out it's policy and make them use it, because if they don't then it means that important records might go missing and we could end up in trouble if anyone officially requests the records we hold on any particular topic.

      Nobody sends attached documents, they send links to documents in the DMS. After using it for a short time everyone seems to appreciate the benefits of only having a single master copy of documents.

  4. Re:What the hell does the summary say? by jimicus · · Score: 4, Insightful

    Besides, only 140TB (or 20 TB)? That's child's play for any competent DB admin, never mind only about $2k worth of hardware to hold it.

    Assuming that none of it's been put into the archival system yet, that means they're dumping 140TB on it in one go.

    You index 140TB on $2k worth of hardware and come back to me when you're done. Hopefully I won't have died by then.

  5. Re:Shadowy Government by rtfa-troll · · Score: 5, Insightful

    No; you are partisan when you think an accusation against one side can be answered by an accusation against the other side. They are both bad (they are US politicians; corruption is so endemic that it's legal and called lobbying), but Clinton's presidency ended about eight years ago and isn't something worth discussing now.

    The questions are; how to make sure Bush follows the law for what he still does? How to make sure Obama doesn't start off like Bush?

    --
    =~ s,(.*),<sarcasm>$1</sarcasm>,g if any_point_you_wish();
  6. Dear staff by keraneuology · · Score: 4, Insightful

    It has come to my attention that as I prepare to leave office my previous instructions to make all email and other documentation available to the shredder was incorrect. The correct policy is to make everything available to the archiver. If you have any concerns please feel free to pick up a copy of the standard presidential pardon boilerplate from my secretary's desk. Thank you, W

    --
    If the g'vt kept the data on you that google does you'd better believe you'd be calling it "doing evil"
  7. Re:What the hell does the summary say? by Kent+Recal · · Score: 2, Insightful

    Maybe not $2k worth of hardware but $200k will do. Which is still peanuts in government terms. They probably spend that amount on paperclips and toilet paper in the pentagon alone.
    Honestly, storing and indexing 140TB of e-mail is a trivial task when you can apply a six digit budget to it.

    If their "archival system" blinks at the sight of 140TB of mostly text then it doesn't even deserve the name.

  8. Not the same thing. by Frosty+Piss · · Score: 4, Insightful

    Perhaps you're too young to remember, but Clinton's administration had a problem with missing emails during investigations too (Lewinsky, why hundreds of FBI records on their political enemies ended up in the White House, illegal campaign donations from China, etc).

    Yes, but there is a magnitude of difference in importance between lost emails about blow jobs and a little dirty money, and emails about the loss of privacy and civil liberties of US citizens, torture of POWs, and the various other nastiness that GWB et al are suspected of. Much different.

    --
    If you want news from today, you have to come back tomorrow.
  9. The problem is Law, and convention, not volume by omb · · Score: 2, Insightful

    As with almost all problems where electronic/internet technologies bump into real life issues eg privacy, non-repudiability and simple confidence it is because the Law has not kept up with technology, and that in the USA is the responsibility of the Congress. Writing was thousands of years old, and the printing-press more than 300 years old when the Constitution was adopted in September 17, 1787. The drafters understood the technology.

    Today we are blessed with ignorant self serving legislators who do not, and are far too happy to follow hard-case makes bad law hurd thought, eg children, porn, paedophilia, drugs and terrorism. The courts have long held that you can read post-cards, but that if your letter-in-an-envelope is opened then a felony is committed or the information is normally in-admissible.

    For this to work people have to start encrypting and signing their e-mails and the Congress and the SCOTUS must enforce identical rules for electronic and hand-written communication.

    Specifically you can not go out and discover the entire contents of someone's library and papers in a law suite, and expect to go on a search-engine enabled fishing expedition.