Slashdot Mirror
'Hobbit' Creates Big Data Challenge
CowboyRobot writes "In the past five years there has been an 8x increase in the amount of content being generated per every two-hour cinematic piece. Although 3D is not new, modern 3D technologies add from 100% to 200% more data per frame. In 2009, Avatar was one of the first movies to generate about a petabyte of information. The Hobbit: An Unexpected Journey was shot in a new digital format called High Frame Rate 3-D, which displays the movie at 48 frames per second, twice the standard 24-fps rate that's been in place for more than 80 years."
But with digital storage transcending some other limitations of conventional projection techniques, it's not just framerate that directors are now able to play with more easily; it's the length of movies themselves, which stats suggest just keep getting longer.
Increasing file sizes also means dampening piracy. Or did anyone think this was a quest to improve quality?
I read TFA, and nowhere does it say how big The Hobbit was.. only that Avatar was about a Petabyte. Why isn't this stated anywhere? It's very frustrating, and also makes the article less useful, since its entire premise is that "The Hobbit creates big data challenge" with no specificity.
Am I the only one who longs for the return of an intermission? If only for a little relief rather than ducking out for 3-4 minutes and missing that one important little line of dialogue on which the whole thing pins?
So there I was, scribbling down some notes off the PC screen by hand, when I reached for the keyboard and Ctrl-S'd.
It was so classy. I'm sure it would help with the theater owners concession sales as well.
There may be no "I" in team, but there's also no "F" in way.
This story is barely good marketing fluff. There is no informational value here beyond, more frame rate = more data. Post processing is hard.
Sometimes I wonder why journalism is dying, then I read crap like this. I want my 5 minutes back.
...so your 3rd eye can maintain perspective.
But with digital storage transcending some other limitations of conventional projection techniques, it's not just framerate that directors are now able to play with more easily; it's the length of movies themselves, which stats suggest just keep getting longer.
Digital storage transcends limitations? And the limitations were due to projection techniques? So, storage is better at doing something than projection is? Yeah, projection techniques were bad at storing my files. Storage transcends that limitation and can store my files.
Here's a better version:
Movies are also getting longer.
Time is money. I think you're more likely to get in seat catheters before you intermissions.
which stats suggest just keep getting longer"
And in the Hobbit's case, longer, and longer, and...... just waay too long. LOTR movies had 1000 pages of book to fill them with interesting content. Hobbit, not so much. In many of the scenes you can almost feel the director guy just out of camera view making that "stretch" motion with his hands.
Tolkien writes that Hobbits are between two and four feet (0.61–1.22 m) tall, the average height being three feet six inches (1.07 m).
I'm ok with advances in technology and the new challenges it creates. What I'm not OK with is a director deciding to make the source material "better" by changing the narrative. Jackson completely gutted Tolkien's Hobbit, rearranged the important events, and has replaced a light-hearted adventure story with the dark themes from LotR. Mr. Peter Jackson, why do you hate the work of JRR Tolkien?
The Admin and the Engineer
From 100% to 200% more data - so what? That's not a big step compared to the steps we had in the past like going from 1.44MB 3,5" floppies to 650MB/700MB CDs. Or from those CDs to 4.4GB DVDs.
Sounds like a job for handy, dandy DVD Shrink!
I don't respond to AC's.
Directors have never understood this, which is why most of the time they aren't responsible for the final edits. Length is one of the primary filtering factors for me to decide what movie to see. Anything substantially over 2 hours had better be a damned good movie.
there is a reason for them to be long(>2 hours).
I'm a HUGE Tolkien fan, and went to the LOTR Extended Version Trilogy Marathon recently before seeing The Hobbit.
I was surprised at how well the longer versions of the films held up, after not watching them for around five years.
However, The Hobbit film was a let down on several levels, most of which I won't go into here. My main complaint? You do not need three films to tell the story. PJ has thrown in everything but the kitchen sink into The Hobbit, and it drags. Even the uber-videogame-esque "escape from the Goblins" scene drags... Too much of a good thing can ruin a film.
I would also say the same thing about the last Batman film. Too long and drawn out. Scenes that should be edited or removed alltogether. Thats why they call it the Directors Cut!
It makes me wonder if there aren't people involved in the film such as producers or editors who tell guys like PJ or Nolan, "hey bro, you might want to trim things down, just a smidge... You know, just to kind of keep the flow of the film going"
We play the game with the bravery of being out of range
The main benefit for going digital was to cut production costs down, you don't have to drag dailies around or courier anything beyond a few hard drives when you go digital (heck you can torrent between locations), but now you have to store data electronically which shifts the cost to running servers or some other backup.
The effective resolution of film is infinite, with good optics on your digital scanner you could go down to any resolution you want for easy of digital editing. Store the film in a vault and forget about it for 90 years.
The Hobbit was also shot (and maybe shown?) at 4K resolution.
That's another bump in the data size.
nope caps proved in canada that it wont dampen piracy it always finds a way
At the London preview screening Peter Jackson said that because 48fps + 3d is 4x the frames it's taken longer to render and the last scene with the coins was only finished a couple days before the premiere. He did mention the complexity in moving coins though
You've probably made one of the top 10 most idiotic comments on Slashdot. And that's saying a lot--it's like winning the Special Olympics. Among retards, you stand out!
I have all the intermissions I want because these days I watch movies / TV shows / videos exclusively on my computer (Netflix, Hulu, YouTube, (borrowed) DVDs, etc.) I do this partially as a "Fuck You" to the MPAA, but mainly because the "home experience" is by far more preferable to me than the "theater experience".
If you think enterprise storage subsystem can have usable capacity of 1024GB (1PB). So if you go beyond, just buy 2 of those (DS8700 comes to mind, but there are others by EMC and Hitachi)
It's interesting to note that for typical footage, as frame rate goes to infinity, so should the compression ratio.
100 to 200 percent more data for "modern" 3D technologies. Stereoscopy adds exactly 100 percent more data, as it has since the late 1800s. There is no new 3D. There wasn't for Avatar, there wasn't for Captain EO, and there wasn't for any of the other marketing-oriented attempts to act like the red and blue glasses were the "old" 3D while the polarized lenses are the "new". Didn't you kids have ViewMaster? Sheesh.
Cloudiot: A person who does not see offsite storage as a way to lose control over access to his or her own data.
I wonder why this is news... I guess it's a good time to invest in ENTERPRISE level storage!!
The Hobbit was shot in 5k resolution, and in "true 3D" with two cameras on every mount. And shot at 48fps (5k x 48 fps x 2 cameras) is MASSIVE film stock. Add multiple shoots for setup, testing lights, and then the actual acting. Not to mention all the digital elements that have to be stocked at high resolution as well.
Yup that's a LOT of data. But then so was Star Wars. I suppose with those really long copyrights it means THEY have to keep all that digital treasure safe.. Because they only sell the stripped out Home copies. The Hobbit was really shot for theaters 10years from now when 4k becomes commonplace. I used Star Wars because while Licas was counting his Billions, the actual MOVIE and resources on film spent 20 years rotting in a damp basement. Because every scrap of these movies has to be copyrighted for 95+ years they don't want any bits leaking out... But saving EVERYTHING it's really hard... Even if they make a billion dollars from it.
I still don't see what the "news" is. I mean Banks are so big they fill LTO 5 tapes with transaction data several times a DAY (not accounts, just the individual records of card swipes and such in real time) . I guess banks don't have to keep that 95 years though...
How much storage would they need if they dropped all of the "short people walking" bits except for one? Once again, lossy compression schemes come to the rescue!
No matter "big" the movies data is, no matter how many fps they film it at, no matter how many 3d effects they add in it still sucked and so do the majority of the movies that rely on special effects. Digital special effects make film makers lazy. In the past decade how many big budget special effects crazy movies have been made that were classics of film vs how many classic movies in the past decade didnt have computer effects? Quentin tarantino alone has made more quality movies in the past 10 years than big budget special effects films.
So bottom line is make it fancy as you want but that alone doesnt make it a good movie.
How does this have anything to do with Big Data? Storing large amount of data isn't the important part, it is being able to analyze that data. You do not analyze a movie's data file. You just load and display the movie, which can easy be stored in one large continous file. A Big Data problem would be Netflix trying to determine what kinds of movies to recommend, not storing and then displaying a long movie to users.
-- All that is necessary for the triumph of evil is that good men do nothing. -- Edmund Burke
Tolkien's work is love-it-or-hate-it and unfortunately I fall squarely on the "hate-it" side. I guess it's good to know that we can enjoy hours of tedium at a higher-than-normal frame rate, though.
I mean they did the 1000 page Battlefield Earth in under 2 hours in film and that turned out great...
Look at even LOTR, special editions probably make it a 12 hour film for the same page count.
I remember watching Ghandi in the theaters in 1982, and it had an intermission because it was too large for a single reel, so there was 10 minutes while they changed the reel. Spartacus, and Ben Hur were also over 3 hours. They are just listing the longest upcoming movies expected, with no actual analysis of percent movies over 2 hours and 3 hours per year. It's multiple anecdotes pieced together, ignoring all data contradicting the hypothesis.
Call me when the average time of a Disnet movie is 150 minutes or higher. Disney fires directors who refuse to keep them at 90 minutes (90 minutes is the time for one extra show a day, 2-3 shows more than the Hobbit). Makes for better results in the box office.
Learn to love Alaska
Comment removed based on user account deletion
The effective resolution of film is infinite, with good optics on your digital scanner you could go down to any resolution you want for easy of digital editing.
Nope.
Comment removed based on user account deletion
Everybody wants to go to digital distribution, but I refuse to accept compromise for the experience. I want uncompressed sound and stunning visual clarity in my movies, not some overly compressed barely HD content with stereo sound split to 5.1 false channels.
Everybody wants to move to the cloud but I live in a G8 country where my bandwidth is throttled and still stuck at 20th century download speeds and upload speeds that are barely better then dial-up.
So yes, the next big challenge for big data is to deliver on the promise of offering high quality, high bandwidth cloud solutions that don't actually suck.
I haven't thought of anything clever to put here, but then again most of you haven't either.
And what do we use this giant-screen, high frame rate, 3D movie to film? Megan Fox french kissing Amanda Seyfried?
No. A bunch of fat midgets and other tiny freaks, and two old guys, one with a layer of bird poop down his temple.
(-1: Post disagrees with my already-settled worldview) is not a valid mod option.
You think that's big kiddo? Think about all the shit the brain sucking spy devices like google and facebook gather all the time. 24/7 365 Think about what science experiments produce, like nuclear research looking at subatomic particles. Think about storing all network traffic data globally, internet and phone calls.
Now. that's big.
You activate the app before the movie starts, it vibrates in your pocket st the start of "pee friendly" zones.
"There is more worth loving than we have strength to love." - Brian Jay Stanley
I noted with interest a few job titles in the credits that I hadn't noticed before. In particular, there were 4 "Data Wranglers" for The Hobbit.
I remeber Avatar with Ang, Prince Zuko and Katara. Good movie. M. Night Shyamalan is awsome!
lol, my parents also owned a video store from 1984-1992 and you are spot on. After a time it's like watching reruns of the same tired plot regurgitated with a new set of players. To this day I don't watch a lot of movies. After it's been a job it takes some of the fun out of it. I haven't seen The Hobbit yet, but it's one of the few I will go see in the theaters simply because I grew up reading the book multiple times, as well as having the 4 record audio dramatization. I still listen to it frequently. It's a great story and I hope Peter did it justice. Length doesn't necessarily mean quality story line though. I felt the LOTR battle scenes were about twice as long as necessary, and some of the embelishments changed the mood of the moment taking something away from the original authors intention IMHO. I'm still glad he made them, and enjoyed watching them. It's just a different story than you get reading the books.
I don't believe in karma, I just call it like I see it.
the longer movies that are coming out. I was getting really fed up of more and more films creeping under the 90 minute mark. For kids films I understand keeping them shorter as most kids won't stay interested for the duration of a 2 to 3 hour film. As an adult though, I appreciate the extra character development and depth that can be provided in a longer film (not that I would ever use Avatar as an example of a film where the extended time was well used).
Lutheran churches in the US seem to operate as much on coffee as they do on faith. However, there are great differences depending on the nationality of those who started the congregation. Go to a German Lutheran church and the coffee is a little on the weak side. Go to a Norwegian Lutheran church and your eyes will pop open after the first sip.
The Hobbit, scarcely 300 paper pages of children's story in print, leads to a big data problem. Now there's excessive bloat if I ever saw any.
Learn from the mistakes of others. There isn't enough time to make them all yourself.
my roomate's sister-in-law makes $77 hourly on the computer. She has been without a job for 5 months but last month her income was $17859 just working on the computer for a few hours. Go to this web site and read more http://www.cloud65.com/
Of course movies are longer! Movies began with double features and a cartoon and newsreel. They did away with double features and cartoons (sniff) when exhibitors wanted to churn the theater every 90 minutes like they were flipping burgers. When we spend a quarter of a hundred dollars, (sounds bigger that way) and then pay the same amount on drinks and snacks, we really want more than 90 minutes minus the 8 minutes of credits.
Longer films actually provide more product, better story-telling opportunities (not always used well), and a better value for our time and money. The Hobbit seemed too short for real Tolkien fans. Avatar's length was appropriate for the story. Maybe we all want better stories? If all one wants of a movie is to pass the time in an airliner, one really is not interested in the film. Perhaps one only wants a TV game show. IMHO.
I start rolling my eyes when I start fidgeting in the cheap ass chair cause my ass is going to sleep and I am tired of holding a bag of popcorn. Sure its a three hour movie, but you get there a little early cause of lines and you dont want to be that douche that walks in during the opening scene bumping everyone cause they want the middle seats for them and their 4 buddies, then you sit there for 20 min worth of previews.
So that 3 hour movie usually turns into 3:45, and you all know that most of that extra hour is just because they feel the need to drag along on less important stuff as if it were the main focus.
may be to convince the studio to keep the third installment, given that their best case audience is pretty much limited to those who saw and were happy with the first two installments. The dragon will presumably be slayed in the second; will there be enough to bring the non-died in the wool LOTR buffs back for the third?
The most interesting aspect about 48 FPS is that the Bluray spec doesn't cover 1080p48, so we'd need a new standard and new devices for home cinema use if 48 FPS will take off (as it should).