JFK Library Launches Largest Presidential Online Archive

Lucas123 writes "The JFK Library launched what it is calling the largest presidential online archive, offering the public 117TB of data related to John F. Kennedy's presidency. The four-year project digitized a plethora of analog material including 200,000 pages of documents; 300 reels of audio tape containing more than 1,245 individual recordings of telephone calls, speeches and meetings; 300 museum artifacts; 72 reels of film; and 1,500 photos. 'As young people increasingly rely on the Internet as their primary source for information, it is our hope that the library's online archive will allow a new generation to learn about this important chapter in American history,' said Carolyn Kennedy, the wife of the late John F. Kennedy, Jr., who was on hand at the opening of the archive."

Comments

But

[TheL0ser]

But does it show who really shot him?

Re:But

[Anonymous Coward]

It was you and me.

Re:But

[Anonymous Coward]

It was our rhetoric. If we would just not critize the ruling elite, no one important would ever die.

Re:But

[geekoid]

The Comedian.

Re:But

[Locke2005]

Everbody knows the Phone Company shot JFK!

Re:But

[petermgreen]

he travelled back in time shot himself, I saw it on the BBC ;)

Where is the quick-link

[Just_Say_Duhhh]

to all of the Marilyn Monroe film and photos?

Re:Where is the quick-link

[Anonymous Coward]

The one film you're really looking for is in some rich guy's private Monroe collection (I think there was an article here on the issue).

Re:Where is the quick-link

[jaymzter]

I'd be more interested in documents regarding Kennedy's ineptitude regarding the Bay of Pigs, how he dragged us deeper into the Vietnam War, his complicity in the overthrow of Ngo Dinh Diem, and the personal monument the Russians built for him, the Berlin Wall. When it comes to Presidents being in over their heads, Kennedy made Bush and Obama look like true statesmen.

Re:Where is the quick-link

[PopeRatzo]

I'd be more interested in documents regarding Kennedy's ineptitude regarding the Bay of Pigs, how he dragged us deeper into the Vietnam War, his complicity in the overthrow of Ngo Dinh Diem, and the personal monument the Russians built for him, the Berlin Wall. When it comes to Presidents being in over their heads, Kennedy made Bush and Obama look like true statesmen.

George Bush is not fit to light John F. Kennedy's cigar. Not as a man and not as a president.

Re:Where is the quick-link

[operagost]

Thanks for all the evidence to back your assertion. You could have at least specified 41 or 43. Anyway, at least JFK lowered the outrageous income taxes.

Re:Where is the quick-link

[Anonymous Coward]

I'd be more interested in documents regarding Kennedy's ineptitude regarding the Bay of Pigs, how he dragged us deeper into the Vietnam War, his complicity in the overthrow of Ngo Dinh Diem, and the personal monument the Russians built for him, the Berlin Wall. When it comes to Presidents being in over their heads, Kennedy made Bush and Obama look like true statesmen.

Here's McNamara's account of that: The Fog Of War

Re:Where is the quick-link

[PopeRatzo]

You could have at least specified 41 or 43.

U pick 'em.

This just in

[GameboyRMH]

Julian Assange called in for questioning regarding a MASSIVE US government data leak!

Re:This just in

[Quiet_Desperation]

You fool! You may have initiated a conspiracy theory singularity! Run! just... run!

Finally got on

[thetoadwarrior]

All day during work it seemed to be down. I can get in now and it'll be interesting to sift through their material.

Rest assured that this collection has been ...

[Super Dave Osbourne]

cleansed of anything really useful or insightful. One thing can be assumed here is that many three lettered acronym departments in the governmint have scrubbed the collection top to bottom years ago.

Re:Rest assured that this collection has been ...

[geekoid]

You're an alarmist idiot.

Re:Rest assured that this collection has been ...

[oodaloop]

Uh, what? In the Clinton years, warehouses full of documents from this time were declassified. Top Secret communications from the Bay of Pigs invasion were made unclassified, just as an example. Scores of authors pored over the documents and wrote tons of books based on them. I read one recently: Oswald and the CIA, based almost entirely on declassified documents the CIA and other intel agencies held on Oswald. I would say there is a plethora of classified documents from the Kennedy administration that have been available unclassified for 15 or more years now.

Re:Rest assured that this collection has been ...

[Anonymous Coward]

I don't think you can scrub this information. It's mainly paper and celluloid-based and water would destroy it.

Re:Rest assured that this collection has been ...

[Count Fenring]

Well, that most likely depends on what you consider useful or insightful. If you're looking for conspiracy theories or state secrets, well, that's not actually what Presidential Archives contain or were meant to contain. It's not censorship, it's the collection policy; the Presidential Archives aren't comprehensive collections of state documents, but of documents associated with the president, which includes a subset of state documents in so far as the President's life coincides with his work.

I think some stuff is still classified, although that's dwindling as various items reach their years-of-disclosure. Virtually none of it would have any relevance to whatever your pet theory is, even if your pet theory happened to be correct, because, frankly, anything too secret for eventual release (i.e. the kind of illegal shenanigans that most conspiracy theories postulate) wouldn't have an official documentation trail to begin with.

Other than that, they're mostly accountable to NARA - the National Archives and Records Administration, who are mostly concerned that archival standards are being followed - they're more interested in preserving the collection than in doing any kind of censorship.

And, frankly, the kind of censorship you suggest isn't really feasible in the case of the JFK collection, anyway. By far the bulk of that specific collection is un-catalogued - I think they have something like 30,000 feet of untouched records and objects, and they're by and large a closed collection (except for copies of J.F.K.'s books, which they serve as a repository for, and a few other open collections, like the Ernest Hemingway collection). It would take immense manpower to even start sorting through that collection, much less do some sort of "SCRUB THE SECRETS" action, especially considering that the search would have to be carried out by people with the clearance to read the secrets, who had the know-how to search uncatalogued archival records.

Wow, imagine that

[Rosco P. Coltrane]

that's like 25,000 DVDs of Marilyn Monroe pr0n...

Oops

[Anonymous Coward]

small correction: Carolyn was his DAUGHTER not his wife.

Re:Oops

[madaket]

Caroline was his daughter

Re:Oops

[digitalsolo]

Indeed. JFK Jr. did have a wife named Carolyn, but she died in the plane crash that killed him. Caroline Kennedy, who attended, was JFK's daugher (and, obviously, JFK Jr. sister, not wife).

Re:Oops

[Anonymous Coward]

sister, uncle, go figure those kennedy's. but page scans 100mb each (page 2 of the fa) - that does sound suspicious.

Re:Oops

[Count Fenring]

Really? Because that's not remotely bad for archival tiffs. I believe the BPL's digitization program (one of the more under-funded such in many respects) routinely produces multi-GIG files; I think 16gb for some of their maps.

Scrub

[handy_vandal]

"One thing can be assumed here is that many three lettered acronym departments in the governmint have scrubbed the collection top to bottom years ago."

Perhaps those three-letter acronyms have, themselves, been scrubbed from the collection.

How does the saying go? "The greatest trick the Devil ever pulled was to convince people that he doesn't exist."

Re:Scrub

[Anonymous Coward]

How does the saying go? "The greatest trick the Devil ever pulled was to convince people that he doesn't exist."

Dick Cheney died?

His wife was on-hand?

[Sandor at the Zoo]

said Carolyn Kennedy, the wife of the late John F. Kennedy, Jr., who was on hand at the opening of the archive.

Didn't she die along with him, in the plane crash? The wiki says so.

Jeepers, what did /. do to this text field to make it so hard to paste a link?

Re:His wife was on-hand?

[Anonymous Coward]

Yeah.. thats JFK jr's SISTER not wife unless the zombie apocolypse has started..

He married his sister?

[Anonymous Coward]

Are they saying John John married his sister? I knew the Kennedys were odd but not that weird.

As opposed to

[Locke2005]

The G.W. Bush Presidentially library, which I believe contains no less than six coloring books now...

Re:As opposed to

[kenrblan]

I don't think the coloring books have been made available online.

Re:As opposed to

[Locke2005]

They won't be, at least until GW finishes coloring them.

And by the way, GW could have faded silently into obscurity, but he just _had_ to publish a book (Decision Points) attempting to justify the poor decisions his administration made... which makes him a fair target for derision all over again.

Re:As opposed to

[PopeRatzo]

I don't think there will ever come a time when it's not topical to point out that the first president of the 21st century was a nincompoop.

Re:As opposed to

[Jim_Maryland]

Yes I realize you are trying to be funny but you might be interested in reading an article on the GWB archive.

Electronic Info Dominates George W. Bush's Archive

On Jan. 20, 2014 — five years to the date after Bush left office — citizens will be able to request access to his administration's archives through the Freedom of Information Act.

What would be nice is to see all this information available through searching online, but as I understand it, request will need to go through the National Archives for processing :(.

Re:As opposed to

[operagost]

Shall we color them presidentially? Ironic mistakes are ironic.

Re:As opposed to

[treeves]

That might have been an intentionally mistake, for humorous effect.

And the smallest presidential library ....

[i_want_you_to_throw_]

would belong to this guy.

These are the same clowns...

[jameskojiro]

These are the same clowns that blocked the JFK Documentary film that the History Channel produced from airing.

Re:These are the same clowns...

[Anonymous Coward]

If by JFK Documentary you mean dramatized miniseries about the Kennedy family, then no, these are not the same people, since none of them worked for The History Channel.

Re:These are the same clowns...

[PopeRatzo]

These are the same clowns that blocked the JFK Documentary film that the History Channel produced from airing.

Or are they the same clowns that blocked the Ronald Reagan film that CBS produced from airing?

Appropriate resolution and compression?

[Anonymous Coward]

Didn't RTFA, but if the figures in the summary are correct, they suggest something's highly out of whack here. Either there's a LOT more than just what's mentioned in the summary, or those in charge of archival need to look into using appropriate resolution when scanning / encoding everything, and use of appropriate compression when saving it all.

If you assume that what's listed in the summary is everything included:
200,000 pages of documents
1,245 individual recordings of telephone calls, speeches and meetings (let's assume each is two hours, so 149,400 minutes of audio)
300 museum artifacts (let's assume 50 images per artifact, so 15,000 images)
72 reels of film (let's assume this is video, ten hours per film, for 43,200 minutes of film)
1,500 photos

Total: 409,100 "items", if you consider each document page, image, or minute of audio / video to be an "item" -- and that's using pretty darned generous assumptions.

117TB = 119,808GB = 122,683,392MB / 409,100 items = 299.88MB per item

Even making those generous assumptions, that's still an *average* of around 300MB per image, document page, or minute of audio/video. Seems a bit excessive, to me, unless there's a lot more to the story...

Re:Appropriate resolution and compression?

[Anonymous Coward]

The Kennedy Presidential Library includes more than 8.4 million pages of Kennedy's personal, congressional and presidential papers and over 40 million pages from more than 300 other individuals who were associated with the Kennedy administration or mid-20th century American history. In addition, the archives hold more than 400,000 still photographs, 9,000 hours of audio recordings, 7.5 million feet of motion picture film and 1,200 hours of video recordings. Digitization efforts are ongoing and additional material will be added to the archive as it is scanned and described.

The documents, photographs, and audio and video recordings are digitized at high resolution to preserve fidelity, thus resulting in large file sizes for each item. One standard document page consumes about 100MB of data, and the size of the archive, inclusive of all media, is expected to increase to approximately 152TB by 2016.

Yep its what woud expect. The summary is wrong. The storage is the cheap part, so they are saving every last bit of data they can (1 page = 100 MB).

Have you ever considered spending time learning instead of preaching?

Re:Appropriate resolution and compression?

[PopeRatzo]

Even making those generous assumptions, that's still an *average* of around 300MB per image,

Good point. That's a lot of storage space for documents that will mainly be heavy black lines meant to protect the Hunt brothers and the Koch brothers.

Re:Appropriate resolution and compression?

[Anonymous Coward]

So what you're saying is, there's something out of whack (as I suggested), and it's the article summary (which I also suggested as a possibility). And that it's off by a factor of three or more (depending on how wide of the mark my guesses were.)

And then you're having a dig about preaching, while in the process preaching yourself.

Is that about the size of it, Mr. Pot?

Re:Appropriate resolution and compression?

[Count Fenring]

Is there something out of whack? Because, if they're making high quality (say, 20% resolution from the archival tiffs) photos available, 100mb seems perfectly reasonable to me. If it includes overhead (successive scaling images, for example), it becomes downright conservative.

The previous guy's tone was out of line, but he's more or less right, in that 100mb images (and significantly larger videos, and potentially equivalent or larger audio) is probably more than reasonable for the display-quality assets for a federally funded digital library. Hell, the BPL, whose digitization program is funded off a fairly small, non-repeating grant, routinely keeps 2-4gig archival images around; they scan at around 16gigs for maps with fine detail.

Young People?

[Anonymous Coward]

Its making the data accessible to everyone not in Washington.

Re:Young People?

[Anonymous Coward]

Welcome to the lost land of Camelot. *snicker*

Carolyn Kennedy?

[nebaz]

Carolyn Kennedy died with JFK Jr. during the plane crash. Perhaps Caroline Kennedy, his sister was on hand at the opening of the archive?

Politics of the Undead

[handy_vandal]

Oh yes, Dick Cheney died, all right.

Then his Zombie corpse ran for Vice-President ... and won.

Re:Who was the best president?

[Just_Say_Duhhh]

Not that being president qualifies you in any way to light anyone else's cigar, but why argue over which former president is better than another? Is there any doubt as to which president is the most improtant? The answer is simple:

OUR CURRENT PRESIDENT! He is the only one that matters - he is the only president who can have an impact on the lives of all living Americans (and a large portion of the citizens of the earth). Debating about past presidents is a waste of time. Pay attention to the one sitting in the oval office, and spend a few extra minutes wondering who will take his place when he's done.

John F. Kennedy

[Anonymous Coward]

The first president elected because of television.

How best to make historic papers available online?

[Anonymous Coward]

I've been going through a number of document archives for my MA and I can honestly say that they're all a readability / searchability nightmare! Can you name ANY archives that present a President's papers both as tif documents AND as really well-tagged XML? Do you know of any research archives with a killer UI and search functions? The taxonomies of these things always seem so limited.

My top five requests include:

• Boolean search
• Full-text search
• Image search
• Cross-linking to other relevant archives
• Ability to access on my iPad!

What do you all think?

Re:How best to make historic papers available onli

[bberens]

Meh, give it a few months before some Google engineer gets bored and adds a couple tweaks to their indexer for the archives. Then you can use the various google features like site: to search the archives.

117TB? Really?

[wyr_taliesin]

Do they really mean 117TB?

Some quick sums on file sizes for online digitized material:
200K pages - say a generous 100K/page = 20GB
300 reels of tape + 1245 recordings, say 1000 hours of audio at about 1MB/minute = 60GB
72 reels of film, say 50 hours at a generous 2GB/hour (DVD quality) = 100GB
1500 images + [images of] 300 artifacts at a generous 1MB each = about 2GB
Total: 182GB
What are they doing with the other 116.8 TB? Digitizing the genome of every woman he slept with?

Re:117TB? Really?

[Anonymous Coward]

How are you coming up with 100K per page?!? A page will have been scanned to TIFF or JPEG2000 with minimal or no compression.
Probably more like 3-10MB per page, depending on grayscale or color scanning.

Your images and film estimates similarly seem quite low for preservation-quality digitization.

Re:117TB? Really?

[yodleboy]

have you been to the library of congress site? i've pulled up several images in their collections that are over 10MB

Re:117TB? Really?

[Count Fenring]

If they're doing serious archival tiffs to least significant detail, the files might be much, much bigger than that. The BPL's original scans come in at multiple GIGABYTES per scan; they actually have to drop the final files down significantly from their original size, in order to fit into their allotted space.

Re:117TB? Really?

[Anonymous Coward]

Bad article full of misinformation. Who is quoted as saying 117TB?

I believe this is more accurate: http://www.jfklibrary.org/About-Us/News-and-Press/Press-Releases/Caroline-Kennedy-and-David-Ferriero-Unveil-Ground-Breaking-Digital-Archive.aspx

From the partners involved I would imagine they are storing TIF's in higher quality and publishing lower quality to the web.

Your math is probably right on.

Useless information

[Anonymous Coward]

What people really want to know is who killed JFK.

Re:Useless information

[Count Fenring]

Well, then they probably want the killer's archives, not JFK's. I mean, it's not like he had paperwork on his own assassination.

Re:Who was the best president?

[PopeRatzo]

why argue over which former president is better than another?

I don't think you mean to imply there is nothing to be gained from analyzing history.

There's every reason to evaluate former presidents. Choose the ones that did the best job and then encourage the current occupant to learn something from them. Consistently, there is one modern president who is given higher marks than all the others (hint: he didn't star in any movies with a monkey). I don't think it's unreasonable to say that there's something for the current president to learn from that success.

Re:How best to make historic papers available onli

[Count Fenring]

You gonna pony up the scratch for the OCR and indexing work that would take? I mean, if you've got a spare couple million or twelve, it'd be pretty awesome - us liberry folks'll get right on it.

Seriously, though - that would be incredibly awesome, but libraries don't get the kind of funding or have the kind of manpower it would take to do those things with the level of accuracy that would be required. And these collections are HUGE... again, I think JFK has something like 30,000 feet of archival records that haven't even been cataloged yet.

I think it's actually through a UMass-funded project that they're getting this stuff up to begin with; the library's internal admin was running everything off of an Access database as recently as last year.

subject

[Legion303]

"the late John F. Kennedy, Jr., who was on hand at the opening of the archive."

Creepy.

Re:How best to make historic papers available onli

[Anonymous Coward]

From the looks of it this is an attempt to do just what you are asking for, without even spending federal money.