Slashdot Mirror
Bush's Electronic Archives Threaten To Swamp National Archives
ColdWetDog writes "The New York Times reports that the soon-to-be-disbanded Bush / Cheney White House threatens to overload the National Archives with close to 100 Terabytes of data. This includes the Barney Cam and even 'formats not previously dealt with.'
By way of comparison, the Clinton White House dumped less than a single terabyte into the archives. Of course, Mr. Cheney, always the Good Citizen, tried to help out when he 'asserted this month in a court case that he had absolute discretion to decide which of his records are official and which are personal, and thus do not have to be transferred to the archives.'
Glad to see that somebody over there is trying to clean up the cruft for posterity."
100TB? eh, i can do that in a week
'formats not previously dealt with.' If they can't open on a typical machine, they are probably just corrupt. Wait a second...
While the National Archives obviously must catalog and make available all the data in some form or another (honestly I do not know their rules & regs for that sort of thing, and it seems a good bit is missing), the mere act of storing 100 terabytes hardly seems all that daunting. NewEgg has 1TB Samsung Spinpoint harddrives available for $100 with free shipping. You can't tell me the folks over at the National Archives couldn't afford 100 of those plus some additional hardware to oversee the transfer of all applicable data to the drives for storage, at least until a better solution could be found. Hell, have the Bush/Cheney crew do it for them and stick the drives in a closet somewhere until they can sort through all that mess. It doesn't take a $144 million computer system to handle 100 TB of data.
Mr. Cheney, always the Good Citizen, tried to help out when he 'asserted this month in a court case that he had absolute discretion to decide which of his records are official and which are personal, and thus do not have to be transferred to the archives.'
Thereby making what he was doing immune to FOIA requests. Nice.
It's just unreal how unabashedly criminal Cheney is. Nobody ever calls him on it. Anyone in a position to do anything about him (other than Dennis Kucinich anyways) strangely...doesn't.
Of course, he's also the same guy who shot a hunting buddy in the face. And had the victim apologize.
Far more dangerous than W. Will not be sorry to see him go. Good riddance. Go retire on your inflated Halliburton stock and please leave my country alone.
Weaselmancer
rediculous.
Perhaps within all that data is a DVDrom or two of real juicy material. More importantly, there may (should) be enough info in there to
convict the former Bush administration's true criminals. Destroying that would be "destroying evidence": Supposed to to a 'serious crime' in the US. If most of it is Windows word/email, that thankfully compresses over 1000:1 to 'human readable' text. Programs such as 'antiword' would be my first line of attack if I was in-charge of this mess. Thankfully I'm not.
BillSF
"I'm told researchers like to come and dig through my files, to see if anything interesting turns up," Mr. Cheney said. "I want to wish them luck, but the files are pretty thin. I learned early on that if you don't want your memos to get you in trouble some day, just don't write any."
This really says it all, doesn't it? I mean, wasn't this essentially Nixon's view on things? That if the president (or his puppet master, vice-president Cheney) deems it not for the public's purview, it's none of your damn business? I mean, what part of PUBLIC office does this numbskull not understand? (Excuse me, the mastermind understands, just doesn't care.)
Sickening. What's even worse is that no one's gonna make this administration accountable for anything they've done. In fact, I'm sure no one's gonna really take a hard look at what exactly this administration has done until a looong time later; everyone's too preoccupied with moving on.
Data capacities and volumes appear to be following the approximate path of Mooreâ(TM)s law â" doubling every 12â"18 months. If Clinton submitted 1 TB eight years ago, I would expect Bush to submit (very) roughly 1/8 * (2 + 4 + 8 + 16 + 32 + 64 + 128 + 256) = 64 TB.
A president ten years before Clinton would probably only have submitted ~1 GB of digital data. This is to be expected.
Stop complaining. Google handles this volume of information every minute.
Is any of it actually interesting?
If someone isn't too bothered about criminal acts when in power, explain to me why they should worry about data deletion being illegal?
Risk, it's how they make decisions to attack countries.
conflict of interest with the RIAA and the MPAA :]
They have to store it without losing any of it. That means redundant storage distributed geographically. The cost of doing this is pretty significant.
What's more, the interest is not just historical, they should be able to access this information immediately. For example, when the new administration is looking to negotiate a deal with North Korea, they need to know exactly what the old administration was doing and why. They need to know what overtures the US has made and why. Additionally, they need to know what overtures the North has made and what they could mean. It will save the new administration lots of time to read the old administration's analyses instead of having to generate their own. Theoretically, the transition team should be assuring that this kind of institutional knowledge is passed, but in reality something always gets missed.
With this amount of data, you are looking at something a lot more complicated than a mysql database and a web based front-end. To be quite honest, I would be surprised if there is any off-the-shelf software capable of this task.
weirdest thing I ever saw: scientology advertising on slashdot.
The Cheney is about if records he does as President of the Senate are covered by the law. The position of the Vice President is that only work ordered by the President are covered those done as being a leader of the Legislative Branch are not. Also the group bring the suit are tryng to get access purely political conversations which the national archives have already ruled are not to be archived.
If all you haters really want to look at something slimy how about the case was made that the NSA was "not an agency of the government" so keeping communications by the national archives was not required.
Delete the spam and you'll probably save 80TBs, just be careful about deleting things from Nigeria...
Any one else thinking they are trying to hide something of interest in all that pile of ... data?
If taxation is legalized theft, then Capitalism is a prolonged rape followed by a slow death.
Problem is, some of this information is classified, and most of it isn't. Some of it legitimately contains private information that should not be made public yet (things like job applications). It contains personal information that should probably be permanently removed from what the public sees, such as employee social security numbers.
What isn't classified or private needs to be available to the public, what is classified or private needs to be available to people with the proper credentials. Some of it will be automatically declassified in 5 years, some of it in 7 years.
In short, somebody has to look at it before it is added to an index. Probably security will dictate that the classified information is stored on different machines, probably different networks, than the publicly available stuff. You might be able to write an algorithm that automates this process, but Google certainly isn't it.
weirdest thing I ever saw: scientology advertising on slashdot.
Who knew a Speak and Spell could amass that much data? Maybe it's the high scores from Minesweeper and Solitaire that did it? Perhaps Bush's tutorials from Mavis Beacon's typing?
I'd really have thought with Bush's user level and Cheney's underhanded nature you'd have been able to archive their entire administration on a few floppies.
The contingency plan, quietly approved by the National Archives on Nov. 7, emphasizes the difficulties posed by large numbers of White House records created with proprietary commercial software.
weirdest thing I ever saw: scientology advertising on slashdot.
As a last addition, 1 TB of data was added to the archives. It contained all the news footage, replays, and parodies of the shoe-throwing incident.
Than go driving with Ted Kennedy.
And where in the Constitution does it say that the Executive cedes power to Congress by a mere passage of a law (FOIA)? I thought the actual Executive was in charge of that branch.
For all of you constitutional purists, I'd ask where in Article I Congress has the right to limit executive authority, whether it's FOIA, FISA, the War Powers Act, "congressional oversight," etc. I am looking but can't find in the Constitution where Congress can do this without ratifying a new Amendment.
Oh, and we are still waiting for that big Clinton records dump into his library that we've been promised. I guess we'll have to wait until Hillary is elected president in 2016 - wait, re-elected in 2020 - before they dump everything.
Slashdot "libertarians": Small government for me, big government for those I disagree with. -1, I disagree with you
100 TB! Wow! That's about 60 hard disks or so! And I don't even want to imagine how much money you'd have to pay for those drives - it'd be a *four digit sum*!
100TB seems like a lot, but can you imagine how much data the Obama administration might leave? Considering that they seem more 'connected' then most other politicians.
/. but can you imagine the baggage left by some of these administrations?
there is always talk of obsolete formats here on
they say it is often more relevant then the comment above, all we know is its called the Sig!
I believed that US had a law forbidding the use of the governmental emails and the like for personnel things and vice-verse (one with a specific law, the other due to the use of governmental resources for personnel use).
But i may be wrong...
i /i
Does anyone else find it ironic this is the same administration that couldn't keep track of a few years worth of official emails? I seem to remember a lack of storage space being one of the excuses, too.
If you can't delete the records, just hide multiple pieces in soo much garbage data that you would be dead and gone by the time anyone was able to put it all together.
Wanna fight ? Bend over, stick your head up your ass, and fight for air.
That should be enough for anybody.
The cost of that cleanup, of course, will be borne by taxpayers, not industry.
If you think this is bad today, this is only the tip of the ice burg. The national archives better ramp up for a drastic increasing curve of data to store as each new president is elected.
Not that i have the answer, but i can see it happening. Just look at the exponential increases in personal information for the average citizen.
---- Booth was a patriot ----
Video of the Risk game is already up.
IOW, these guys have been cleaning anything relevant and creating lots of BS crap to put in there. By the time it is figured out, Cheney will be dead and W will be senile. Hopefully, these traitors are able to at least be caught on tax fraud.
I prefer the "u" in honour as it seems to be missing these days.
Maybe we can find out how someone can have the goodwill of nearly every nation on earth (even Libya offered to help!), a tame congress, wartime expediency in letting anything you like get through, signing statements to change anything you don't like, a booming economy, a military grudgingly ready even to abandon the Geneva convention if ordered and access to experts on every subject yet STILL muck it up so badly.
Is there an 18 minute gap in the Barney Cam tapes?
He may have been loud, but he wasn't very firm. When Bush effectively nullified the ban on the military using torture with a signing statement, McCain said and did nothing. When congress tried to extend the ban to prevent the CIA from using torture as well, McCain voted against it.
He may have been against torture at some level, but not as much as he was in favor of getting the nomination. When the two goals came into conflict, he caved.
--MarkusQ
(1) Conspiracy to get the United States involved in a war, in order to personally profit himself and his business associates (Halliburton).
(2) [As a result of (1)]: Treason
I don't pay attention to any hate sites. But his actions have been pretty blatant.
Obama's probably already stored a few TB just with the fundraising email campaigns.
It's rare that you're presented with a knob whose only two positions are Make History and Flee Your Glorious Destiny.
Barney cam... FTW!
I don't know if it was worth the 10 minutes of my life I just spent watching it, but it was cute.
If you can read this... 01110101 01110010 00100000 01100001 00100000 01100111 01100101 01100101 01101011
Is it just me or does 100 Terrabytes seem rather paltry, especially if a good chunk of that is video? I have a 2 TB NAS I use for video clips and it's not enough...
My guess is that the "unknown formats not previously dealt with" refers to shitloads of the photos being in ridiculously large RAW or TIFF format, or digital vids in likewise raw formats that can only be accessed with software like Premiere, AfterEffects, etc.
Maybe Bush & Co. didn't know they could adjust the settings on their cameras? Heh, every camera was good for maybe 5 shots?
No, he loudly and publicly proclaimed his absolute unconditional refusal to endorse torture. Then, when his bill to prohibit it was quietly circumvented, he said and did nothing. Given the opportunity to vote for a revised bill that would have had teeth (by specifically prohibiting the CIA from torturing people, thus closing the loophole) he voted against it.
And then, on the campaign trail, he continued to play up his POW history and his objection to torture.
That isn't wisely refraining from shouting at your colleges, that's showing your true colors and folding like a hypocrite when it counts, and hoping the saps you pander to are too dumb to notice.
--MarkusQ
I really do believe most people who post here are very smart and intelligent people involved in some of the most important fields we currently have: technology of all kinds.
So why are we paying attention to this? Does this matter? Does it matter that some content is going to stored away (as IF the public will be able to access it easily and as IF the information will even matter)?
To me, this is a sign of Slashdot turning into Fox News and any other mainstream media. Slashdot is supposed to be the other side to me, the proponent of open source (led by SF) software and open standards of all types. Beyond that, science and technology breakthroughs, which are immense in the science section.
I know why this is a Slashdot story. It has the word terabyte in it. Therefore it goes in hardware? This should go in politics (the debauch of Slashdot IMO right next to user sent mail). Politics have no influence on technology and laws are only put in place to suppress it and admit we have no solution currently to a problem. Drugs: Let's jail all these people to our now mostly privatised jail system. Why? Because they had merely them. How? Because the US government pushed Afghanistan to produce more poppy seeds and become the world's largest producer for heroin. Check the facts, and do not read mainstream news.
The cost for 100 TB of data storage from Newegg is currently $9322. So the cost for 100 backup systems, distributed geographically, would probably be less than a million dollars, after government bulk purchase rates are figured in. When we are giving 7,000 times that amount to the banks, etc., and hoping that they will be able to survive and pay the loans back, I don't think the cost is too much to worry about, when the figure calculated is an obscene amount of redundancy.
I concur with your $1,000,000 estimate. It might be 10 times more to use all of the newest and very most reliable hardware and with armed Blackwater guards, the sky's the limit. The cost might also be 10 times less with used parts from eBay and bargain real estate, which is basically falling from the sky thanks to Cheney/Bush, but my guess is that $1 million is right around the center point where value and cost are at the optimum ratio -- more than adequate performance, but not so much more than adequate that money is wasted. The only thing you missed is that the cost of the failed asset redistribution plan is not a mere 7000 times the estimated cost to store the White House's files, it's 700,000 times more. 7000 times one million is only seven billion. Henry Paulson's taxpayer giveaway to failed "lenders" costs seven hundred billion USD, so you need another factor of 100. 7000 times is already a dramatically large multiplier but the reality is 100 times worse again, and considering risk analysis, it's even worse than that.
Redundancy in information storage is considered desirable because each copy reduces the total risk of losing all copies, or increases the probability of retaining at least one copy of each bit if you're a glass-half-full type. But in bailouts, redundancy normally is the result of criminal recidivism and considered a bad thing itself, in addition to the crime leading to bailout, because each repeat suggests higher risk of subsequent needs for bailout, recidivism.
Obviously, a foreclosed homeowner or bankrupt recent student with outstanding education loans (for education that might or might not have been outstanding) or car owner with no means of paying back our loans has made one or more financial mistake and would benefit from a personal bailout. Equally obvious are two crucial differences between us and AIG, Citi and Bear Stearns. Correctly estimating the risk of loans is not our primary duty, and we have each made just one such mistake, or very few. We're better at their jobs than they are. At a maximum, any one person is highly unlikely to ever be guilty of one or few of those mistakes because credit applications include existing debts and past repayment history. We should of course assume that some borrowers have multiple loans they can't repay, but the point remains, the number is very small compared to seven hundred billion.
The bottom line is that a bankrupt homeowner, or bankrupt recent college graduate, or bankrupt credit-enabled purchaser of such a poor investment as a depreciating motor vehicle, each demonstrate just one poor financial judgment, in a field which is not our specialty and not a measure of our professional ability. AIG, Citi and Bear Stearns, demonstrated incompetence in their own professions, in their roles as our lenders and as investors in more such loans, so unwise they necessitated their own, brand-new euphemism: "troubled" assets. They have made each and every one of the same mistakes as all of their customers, multiplied by the total number of their financially overextended customers. The fact that lenders' behaviors have led to a need for even one bailout demonstrates redundant incompetence by each company needing bailing out, compared to one or few financial mistakes by working borrowers in need of a bailout. A broke teacher or carpenter or other laborer is still good at his job, still good for something. The same cannot be said of a broke banker. Failed financial companies are by definition an unjustifiable cre
"I can't imagine how things could get any worse!" (some guy) "That could just be failure of imaginatioÂn on your p
Government employees aren't supposed to be using government computers for non-government purposes. I can't complain if Cheney has a few shopping lists and personal emails lying around, but truly personal files should be few and small. If he claims that any substantial portion of his files are personal, he's either lying or he has been misusing government property.
No need to draw attention by stopping something; flood the infonet with too much bad information and create a bigger haystack. Not to mention all the FUD that can be done much easier now. You could even put out almost true information with slight crazy distortions to make the truth look bad.
The IMPORTANT INFORMATION WAS LARGELY DESTROYED ("mistakenly lost") and whatever might be useful will take forever to dig out and make any sense of.
Criminal neglect never applies to politicians and don't think that they do not know this.
Democracy Now! - uncensored, anti-establishment news
Hey I'm tired. I know I have typos in there. Hell, why does English need this old junk like the letter C anyhow? K and S fill the void just fine.
Democracy Now! - uncensored, anti-establishment news
I don't understand the problem. 1tb hard drives are about 120 USD. 120*100=1200 USD. Ok, then you need backups. Either way, it is only a couple thousand dollars. What is the problem? Do the national archives need a federal bailout or something?
I am pretty damned sick of these "Troll" mods that I see being used against anybody who has a real opinion other than the "mainstream". This site is supposed to be for DISCUSSION, and there can be no true discussion without alternative views.
Yes, I have been victim, but I have seen many, many other victims as well.
Let's try to be ADULTS here, people, and accept the idea that others can hold opposing views without being "trolls"!!!
"the soon-to-be-disbanded Bush / Cheney White House threatens to overload the National Archives with close to 100 Terabytes of data"
Does that include the lost emails, the ones that were on the non-existent backup tapes ?
davecb5620@gmail.com
So is the complaint that the bush administration has designated too much material to national archives, thus overflowing the system, or is it that Cheney is not giving the NY Times enough of his materials? I mean, you really can't have it both ways. Either you complain that there's too much stuff being stored, or you go after your boogeyman Cheney. I would think that the reason there is so much being stored is because of the grief that the administration has received about deleting ANY records, for the innuendo that it contained a smoking gun that would unveil the "ultimate evil". Good for Cheney for not playing that game.
http://www.beanleafpress.com
This includes the Barney Cam and even 'formats not previously dealt with.' By way of comparison, the Clinton White House dumped less than a single terabyte into the archives.
2^(8/1.5)=40, so the fact that Bush's term ended 8 years after Clinton's would automatically account for 40% of the increase. Reckoning that digital records are more widely used now than even a decade ago easily takes care of the remainder.
Yeah, 100TB is a lot of data, but no more than expected. The National Archive better gear up for at least 10PB from President Obama, given those two factors alone.
Dewey, what part of this looks like authorities should be involved?
The swamp of data will have to be severely redacted to fit into the archives. I guess that the records of secret renditions and pre-war Iraq intelligence have to go to make way for the pet video.
I mean, you've got to set priorities.