Slashdot Mirror
NHK Plans 50-Year Digital Archive
"Programs from the archive will be accessible via personal computers in NHK's Tokyo broadcasting center, which will be linked to the archive via fiber optic lines. An NHK official also said that the company plans to make the archive accessible to outside users some time in the future. So, anyone have a list of good old-school anime this archive will contain?"
This is truly an ambitious project -- I wonder how many terabytes it will take to digitize that many years worth of film and TV. On the other hand, it seems to be inevitable, so it's only sensible to start now -- storage techniques and prices are sure to improve over the course of the project. I just hope they make the films available under reasonable terms. Note, this article was first posted to tetsujin.org, which is worth checking out for you Japanophiles.
I appreciate this has some merit, I mean history is great and all.
But is this really worth the amount of money that's being spent on it? I'm sure there are better things to do with that money, even if it would be to buy a new chair for everyone in the office.
I suppose if i were going to be even more paranoid, this is complete BS, designed just to make themselves look better, and generate more income for themselves, completely apart from any merit that it may have. More media getting everyone to trust them, feed them their opinions.
I can't see how anyone could approve of this, much less endorse it as people here seem to be doing.
Well, if it's high enough bitrate, MPEG is far better than film. After all, that's all DVD-Video is.
I'd guess they'd archive a perfect-quality copy, and allow remote access to a toned down version.
---
Wow, it's encouraging to hear that. I don't have any first hand experience with it, I based my statements on things I've read, such interviews with film restorers like Robert A. Harris. People like him always talk gravely about what appallingly bad condition old movies are usually in. I suppose things must have gotten better, or perhaps they only take good care of the new stuff, and "old" films are left to rot. Dunno. Interesting info, though. Thanks for posting that.
Free Hans!
Ok, so she's born - I assume you don't want "pr0n" of NP until she's 16, so you want to see NP "pr0n" from ages 16-66 - 66 year old woman pr0n?
Oh, wait; I forget. Multiple cameras. You could get her in her prime, for ten years with five cameras. So forget all that, keh?
--Giving to trolls for the benefit of us all
Film is analog, it's "format" (har har) is created by actual light rays -> when you convert to digital you lose detail, always.
Now there would be a school of thought that says that as analog deteriorates over time a digital copy is better in the long term. But in no way is digital as good "if it's a high enough bitrate".
--Giving to trolls for the benefit of us all
Well, CD's have pleanty of bandwith to accuratly reproduce sound with a high dynamic range, low noise floor, and wider then necessary headroom.
DVD's don't have any of this. They are nice enough to make a TV screen look good, but compared to the original (which could be a D1 (digital) tape or 35mm film) they suck.
(ot- the main reason records have a reputation of good sounds is the fact that most recent CD's have virtually no dynamic range.. The audio signal is compressed(meaning dynamic range is reduced) and the volume is pushed to maximum to make the sound seem louder. Records typically don't have this done to them as much. Find some good Classical music CD's from the 80's, and listen for the huge difference in dynamic range we should be taking advantage of now...)
Blessed are the pessimists, for they have made backups.
Amen, brother!
As someone who every single week watches a strip of film that has been running for five years (replacing another copy that had been running for over a decade), I can tell you that the detail of the 35mm version of the Rocky Horror Picture Show exceeds *tremendously* the (very well produced) DVD. The scratches on the film are filtered out by your brain, while the fringe on the lamp in Columbia's bedroom is sharp in the film version, only hinted at on the DVD, and a blur of color when shown on a S-VHS connected televsion. While that's a small detail, things like expressions on the actor's faces when they are across the room come through on the film, while they don't on the DVD.
Hell, I'm really pissed at the fact that DVDs will be the 8-track of the video world. They are better than VHS, but with HDTV on the horizon, they are also very much in danger of dying off quickly. It's almost a shame they came out - will the public (who just started buying into DVDs) go to a new format to support the new digital and High Definition televisions?
I have a 5.1 DTS full digital setup, and the nicest thing about DVD is the sound. The video quite frankly stinks. Artifacts are substantially more annoying (IMHO) than a lesser quality signal. The only thing that really is bad when it comes to 35mm is the low framerate on a large screen causes "mental flicker" when panning or during a very active scene... the frames don't blend into a stream.
BTW - if you want to see a very nicely produced DVD, check out the 25th Anniversary Rocky Horror Picture Show. I can strip out images and zoom in way past the limit of NTSC -- but the 35mm film is still much better. My votes for worst DVDs go to Metropolis or Neon Genesis Evangelion, Volume 1 (animation in general is encoded poorly with MPEG2). I was waiting for the DVDs of Eva to come out before collecting them. I may very well get the series on VHS instead (the production company changed after volume 1, so I'll give it another chance).
Factoid - Of all the existing Star Trek series, which will look best in 2020? The original series... it was shot in 35mm, and the masters are still in archive, while the rest were shot to analog videotape. It's also a popular enough series to be remastered to whatever high definition format is available then.
--
Evan
"$30 for the One True Ring. $10 each additional ring!" -- JRR "Bob" Tolkien
Does this mean streaming video of Iron Chef will be available 24x7? I may never leave my computer again (not that i do very often anyway). .^
^.
The King of the World likes digital formats, and prefers them. But they're not as good as the original 35mil, if you can get it, which you never can. Really though, I wasn't stating opinion and just fact - you loose a huge amount of detail when going digital. If you were to screen a theatre-sized image the affects of digital vs analog would be clear - digital sucks when it comes to archives.
The King of the World needs to go over there now.
--Giving to trolls for the benefit of us all
You can't watch film from the original archive streamed for two very good reasons -- it is very very difficult to stream and it ruins the film. Furthermore, just because it is archived, why does it have to be for public consumption? They're not exactly public records. They are private property, for the studios to use or abuse as they please, and I for one think it's great that they want to archive them for the future, regardless of the motive behind it. Perhaps this is some extension of the bullshit "free as in speech" nonsense pitched by the Open Source fools?
What is with this whole "free as in speech" metaphor anyway? If I say something that does not inherently give you the right to use my speech in any way you see fit, especially not the right to reuse my speech as your own. This "free as in speech" nonsense has got to go. Pick a different metaphor that actually makes sense, instead of one designed to garner sympathy for your cause from people who haven't actually thought about what it means.
Sorry.
I get a little worked up sometimes when I hear the insane ravings of the "information wants to be free" crowd (arrrrggh! Another nasty one)
I'll go back to bed now.
MPEG4 -> DivX ;-) is the way !
--
1% APY, No fees, Online Bank https://captl1.co/2uIErYq Don't let your $$$ sit in a no-interest acct.
The Inatheque http://inatheque.ina.fr/ (in french) records since 1995 all french radio and TV broadcasts. Everything is stored in a bunker-like complex and they have a high-speed network that the journalists use to access the archives. I was working at Andersen Consulting when they participated in the specifications for the project, and what I saw was quite impressive, especially the hardware involved.
Good point. Although I would imagine things like that are already pretty well taken care of by the Library of Congress, the National Archives, the various presidential libraries, etc.
Free Hans!
Actually, the same group who encoded and packaged Fight Club also did the Rocky Horror DVD. Both have the same better than NTSC quality tracks. In addition, the packaging for both were very similar. AFAIK, these are the only two DVDs so far put out by Pox's new premium DVD studio.
Oh, and both featured Meat Loaf, although he didn't have women's breasts in Rocky.
--
Evan
"$30 for the One True Ring. $10 each additional ring!" -- JRR "Bob" Tolkien
I wonder what the rates will be to access the information in the library? Does this remind anyone else of Stephenson's Snow Crash, when everything the gargoyles (people who document their surrounding digitally) record is uploaded to the Library of Congress, and the gargoyles receive royalties contingent on the material's popularity? Between this and the newly announced implantable GPS system, the future is looking even more science fiction-y...
Free music from Jack Merlot.
After the impending nuclear halocaust, I hate to think the only survivors will be Cockroaches, Pokemon and the Power Puff Girls....damn.
;-)
I just hope for humanity's sake, they don't archive Star Trek Voyager!
Someone better slip they guys at the network a fifty to make sure Babylon 5 is preserved for all time. (Along with visual records of Natalie Portman, Mel C, Alison Hannigan and Sarah Polley)
Hardware prices will always drop, but you can't just put things off forever.
As well as converting to digital, they are also putting all the data on smaller cassetes. As http://www.strl.nhk.or.jp/open99/co-4/shosai-e.htm l
explains, the data/cassete volume ratio increases by 20 times(the small cassetes hold the same amount of data as D-3 cassetes but have a much smaller volume)
X-Has-Sig: yes
Sure they can do it, hell they can do anything they want, of course it may not be legal....
but since its not going outside their corporate boundries in the first incarnation, i doubt it would have any legal problems....
perhaps there will be legal problems if/when they open it up to outsiders, but i wouldn't be able to do anything but speculate if they were in america, and they're not so i have absolutely no idea of the legalities
Need a Catering Connection
I am sure NHK has a lot of influence, right? The people in charge must have some really "high-up" connections, right?
So why don't they have the Japanese government ask the aliens to send a giant (but thanks to advanced alien technology, inexpensive) radio reflector into space so that after a few decades, our signals will come right back to us! Then we can have them received by a hyper-accurate radio telescope/transmitter on Earth that will send them back out again! No one will ever know! (As opposed to some radical new storage technology suddenly appearing on Earth!)
The process could continue until we can invent a cheap archiving process capable of permanently storing all of our valuable data.
If planetary disaster were to strike, I'd hate to think that the only remaining A/V content would be a few burned out Napster servers with nothing but Britney Spears / Limp Bizkit tracks, and a couple of hard drives containing Cartman's mom in a German schisse video.
and it shows
GORE LIEBERMAN 2000 MILITIA
What's with all this anti-Japanese bullcrap?
OK, first of all, this sailor-suit and tentacle-porn stuff is not indicative of Japanese popular culture, or anything Japanese for that matter. They're spin-offs of Japanese pop culture popularized by...... AMERICANS! (much like Pokemon.)
I'm Japanese and have lived in Japan all my life, and I have yet to see any of this "hentai" porn. It's probably more popular in the US than here.
Also, there is no pornography whatsoever on mainstream Japanese TV. (Pornography laws are strict, even hardcore porno stuff is all censored)
NHK (Nippon Housou Kyoukai) is a non-commercial channel, so you get stuff like news, documentaries, informative stuff. No commercials. Better than the "appeal-to-the-lowest-common-denominator" crap you get elsewhere.
I hope it becomes avaliable.. Anybody remember Time Gang Tuesday? I would love to get copies of that show. I only have 2 aging videotape episodes.
The truth shall set you free!
Theoretically, digital data lasts forever. Practically, it doesn't.
... technology *CHANGES*.
Sure, you may make a few dozen copies, keep them in a controlled temp vault, etc, and ensure that at a minimum 2 copies are readable (and make more as necessary), but,
MPEG is accessible now. Will it be accessible 10 years in the future? Likely, yes. 20 years? Uh... about the only things we can read from 20 years back now is ASCII data and programs that use a well-defined fileformat (like the z-machine). Other proprietary fileformats have to keep getting refreshed to the latest and greatest, a monumental task in itself. 30 years from now, MPEG might just be a distant memory, with few, if any, computers even able to play back the streams (and also, find a reader for the media it is on).
So perhaps NHK would be copying the film onto archival film (film, vinyl (LP) and paper are one of the few mediums available that store data for a long time), and at the same time digitizing it. The digital copy is to be *searched* and *retrieved* quickly and easily, while the film is kept in enviromentally controlled vaults. Less wear and tear on the film, and the data is accessible in a more "today-ish" (i.e., in the next 10 years or so) format.
Analog media: format doesn't 'wear out', but the media does.
Digital media: media doesn't 'wear out' (easy to recopy data elsewhere), but the format does.
If you want to store stuff for archiving in digital form, make sure you include a complete computer, reader, schematics and mechanical diagrams, technical documents on how to read the media and how it is encoded, gas generator, enough gas to extract all data to 'modernize' it as well as chemical breakdowns... just to keep a few movies in MPEG.
This is NHK, non-commercial, government funded TV. It'll be 50 years of period dramas (Jidai Geki, from whence came Lucas' Jedi), boring news programs and some REALLY, REALLY good (although occasionally faked) documentaries.
If you don't want to repeat the past, stop living in it.
How much Japanese programming have you REALLY seen? This is NHK, remember. Publicly-funded, non-commercial television. Educational and occasionally odd but really funny anime, great documentaries, historical dramas, news.
We used to watch TV all the time in Japan. When we moved to Canada we watched North-American programming for the two weeks we were staying in a hotel then made the conscious decision not to receive any broadcast TV at home. 60+ channels of 95% crap.
If you don't want to repeat the past, stop living in it.
NHK I believe is the largest Japanese Television company (comparable to FOX? CBS?) and has an enormous impact on Japanese society (and with the increasing popularity of Japanese animation and film here in the US, need I say more?). We already have a good idea about just how important Japan is to us and the rest of the world. NHK has now embarked upon an ambitious project that will most likely be of great benefit to the world. How is this not important news? Do you mean to tell me that if it isn't happening in America, it isn't important? So if it isn't about Microsoft, Intel, or O.J. Simpson, it isn't news? Most of the media in the US is already so ratings driven that we hardly get any real news to speak of from major new sources such as TV and newspapers. In giving us news we would never have heard of from other sources, news that come from outside the small world we live in. Slashdot is doing us a service. Particularly when they are telling us of a possible future news source (NHK archive). I find it incredible that people complain about the relevance of this article.
"Only Real Men Have FABs." -W. J. Sanders III
I personally think it would be much more entertaining, amazing, and useful if someone were to succeed in archiving all of the web-page content, links, information, and summaries, that make up the browsable portion of the Internet.
I'd be even neater if it could be maintained. Jeez...just think: The world's largest, most diverse information source...akin to Asimov's Encyclopedia Galactica. (never done)
What's this Submit thingy do?
It's probably NHK putting their own material into the archive. There is a heck of a lot of it.
Disclaimer:
I'm working on a similar project for a major US Network, and have actually demo'd my project for the guys from NHK.
-- 73 de KG2V For the Children - RKBA! "You are what you do when it counts" - the Masso
I'm not entirely sure how this would work, but I can't help but think that television stations shouldn't have full rights to all of their content. In order to be able to offer public access to any of its content at any time, one would think the station would have to do quite a bit of renegotiation with the owners of said content. Normally, money exchanges hands each time something is broadcast, but this idea wouldn't work quite the same way with digital content. Does anyone have any more details or speculation on how this might be accomplished?
Imagine the possibilities, an archive storing every pr0n video made in the last 50 years! Not to mention Natalie Portman :)
Only the State obtains its revenue by coercion. - Murray Rothbard
As the body of this posting states, prices in hardware will drop. These kinds of drops are cause for changes of plans. The bureaucracy involved in this sort of matter is high and can only slow down or kill the project all together.
This kind of advancement also implies freedom of information (which I'm completely for), but there will always be men in suits that will have agendas to slow down the project.
Are you saying you think DVD-Video is better than 35mm film?
Free Hans!
Wait! Before I get moderated down, think: How many times have you wished you had taped something because a few years later, you finally realized it was insanely funny/cool/etc. OR think about the other possibilities: no need for TIVO/Replay because you can get all the previously broadcast videos on your computer from the archive site? Granted, this will take alot of bandwidth, but isn't this what the Internet2 is supposed to be experimenting with: real-time or high-bandwidth applications with next-generation fibre backbones and faster routing architectures? (Wow, that was a lot of buzzwords)
Anyways, I'm sure that if a service like this was set up, and advertized well, by any of the major networks today (maybe not CBS... j/k) they could create a large profit stream from all the people willing to put up with banner ads, or perhaps even short commercials at the beginning or end of downloaded archive video.
Anyways, I really don't see why any of the media giants hasn't created somthing like this yet.
BTW: If someone patents this in the US, consider this prior art.
After that...
Decay of digital media can of course be overcome by simply making a new copy, but you have to be careful. Failure of digital media is usually a catastrophic failure; i.e., you lose everything all at once. With film, if it fades, you still get an image, just a faded one. With digital media, it's all or nothing. One day, poof! The disk is unreadable. You have to make sure and create a new copy often enough so that there is no danger of that happening.
The thing is, film can be an archival-quality medium. With things like dye-transfer prints and silver-based separations, the image can last a long time with no degradation -- thousands of years in theory. Too bad major movie studios don't care about that. Many films have been lost permanently due to improper storage. Ironic that they lobby Congress for more protections for their "valuable property" while said property is rotting in a vault due to negligence. Does anybody wanna bet they're going to be just as sloppy with digital media? "Oops, we waited too long between making fresh copies, and the media it was stored on has become unreadable -- it's gone forever. Oh, well."
Free Hans!
For example, consider the Vietnam conflict. Much has been written about this episode in US history, and much of what's been written reflects the author's personal or institutional agenda. What if we had access not only to the original Walter Cronkite newscasts, but also to BBC, French, Chinese, and Vietnamese daily television broadcasts for the same period? Or even an open-source archive of reporting from individuals (something like that described in "Mother of Storms" by John Barnes) I think something like this is a good first step towards removing the (often-unwanted) mediation that is so prevalent in modern culture.
Someone else mentioned the Gargoyles in Stephenson's "Snow Crash" who constantly uploaded everything to the Library of Congress and were paid by access. I say this is an excellent proposition, but privatize it and make a million. As television and video become part of the historical record, it's high time we stopped seeing these records as ephemera and begin giving them the respect they deserve. Even if they are stilted and mediated, they reflect current viewpoints and attitudes, just as blackface minstrels did in just over a century ago.
The crimes of eBay are a disgrace to it's pig latin heritage!
Hardware prices will always drop, but there is a point where projects become economically viable (for whatever reason) - in their case, perhaps this is now.
-- Matthew - matthew.gream@pobox.com, http://matthewgream.net
Why just films, what about news stories from the war, JFK's asasination, moon landings.
MPEG quality doesn't really cut it (AKA DVD quality isn't broadcast quality).
Most people think DVDs look good. If you ever get to a TV studio, ask to see some original footage - it blows DVD away. Your just used to looking at noisy signal.
The technical problem is exactly that, and is actually more political than technical. What format(s) do you save the data in? It looks like the networks will continue to store the actual tapes for a long time, but they want to be able to do "Non Linear Editing" on line too. So, do you store broadcast quality video, or do you store a low res proxy? How about both, but then scrub the high res version after a time (Remember, the tape is still around)
A lot of the reson for thsi kind of stuff is this. Researchers want to know what they have. Let's say they are doing a story on "Foo". They will look up every tape they have on "Foo", and order all 50 tapes, to see what the footage is. Most of these tapes are A)One of a kind, and B)Have to be shipped. If they had a way to show what was on the tape to the researcher, so the tape didn't have to be handled, things would be a lot easier. The thing is, how good does the proxy have to be?
Base your answer on the fact that these archives can have on the order of 5 million hours tape in them, and grow by the order of 100-200 hours/week
-- 73 de KG2V For the Children - RKBA! "You are what you do when it counts" - the Masso
yeah, I don't like anime. I must hate the Japanese.
I also don't like Italian food. I must hate Italians too.
Your logic is infallible.
One of the many things I hate. thingsihate.org
Feed the trolls...
While not all over 35 yeas old, here is my list (in no particular order -- it also doubles as a film recommendation list, BTW):
Animal House, The Blues Brothers and Terminator pale in comparison.
--
I'm happy to see that there is at least one person left in the world with two working eyeball in his head who can see the difference between film and video.
By the way, if you plan on running your brand-new print of Rocky Horror as long as your previous one (over a decade, whew!) you might want to consider using FilmGuard. It's a non-evaporative cleaner that is applied using rollerpads. You can learn more about it by going to Film-Tech.com and clicking on the "cleaning" link in the bottom browser frame. Film-Tech also has some nice weblog-like forums where projectionists from all over the world share tips and advice.
Free Hans!
50 years of video is alot, wouldn't alot of the footage be lost among itself? I mean, so what if they archive it... Who's going to find it all and what kind of data access system would hold it all and be able to access it fast enough for 1000's of people to watch it at the same time, terabytes of raid on fiber? sounds extreamly expensive.
- Phawx ( www.blist.net / phawx@phreaker.net )
Now, if you include the hardware that makes it all instantly and automagically accessable, there's no way it's going to take up less space (unless it's compressed - see above).
In other words, if they're really intending on using this video library to store all their archives for easier accessibility and preserving the original quality to protect from deterioration of analog data over time, then I would expect that it is bigger than their old analog storage facility, especially if they built it big enough to hold future (still-to-be-produced) video.
A lot of film people are very uncomfortable with the quality of mpegs.
It would be a shame to see the quality of film sacrificed for the sake of easy access.
The other side of the coin, though...film degrades over time, while digital appears to be forever.
I say "forver", cuz people used to say CDs would last "hundreds" of years, but many think the metallic substrates will decay enough to be unrecoverable after only 20 years or so.
Well, I'm no video archiving guru, so if someone wants to comment on mpeg quality or long term data stability, I'd like to hear it.
Treatment, not tyranny. End the drug war and free our American POWs.
See my user info for links.
Don't forget Giant Siezure Robot, and Mr. Sparkle!
One of the great boons of this archiving will be the ability to look back on how people viewed the world--how they predicted the future and so on. Many of these predictions are available in print or online, but video is a quick dosage most of the time. As a resident of another nation, I'd be interested to see the perceptions of the world through the slant of television.
I wonder if this opens up a world of lawsuits based on false coverage.
The official site with more information is available at:
NHK Digital
or NHK Digital(English translation)
X-Has-Sig: yes
i doubt that they would have broadcast demon tentacle hentai on the air. unless....
anyway, this kinda stuff might be useful in an archive to future generations in case the world WERE to be merging with a dimension of tentacled monsters... yanno, as reference materials.
"Ask me about Loom"
Just think... decades' worth of Doctor Who. (Not to mention The Goodies.)
Anime! Anime! Anime! *droooooool*
But seriously, I have to echo several others: can they do this? Aren't there copyright issues at stake?
---