Cool Matrix Filming Techniques

webword writes "Here's how those cool scenes from the Matrix were filmed (go here). Not that I want actually buy one of these cool cameras, but I hunted around to find out how to get one and how much they cost. You can get one here. This brings up a quick question: How are people keeping up with the latest and greatest filming techniques?" What? An advance in cinematography that doesn't involve a farm of Linux machines?

Comments

Keeping with the trend

[Syn.Terra]

"How are people keeping up with the latest and greatest filming techniques?" Simple. They watch MTV (where the first video featuring the frozen-in-time effect was around 1995, though I forget who the artist was). Though the article was right: it's good that only a select few can do things like this, otherwise we'd see so much of this frozen effect (as it's been done already in Lost in Space and a multitude of music videos) that it would lose its spark. I personally would like to see more of an effort on writing than on camera angles, but that's just me.
------------

This was covered before.

[quasipunk guy]

This same thing was covered here last February. While I usually ignore reposts, it does get annoying. AFAICT, this was posted because it was on a "big" news site, and the poster didn't know it had been done before. /. doesn't owe us anything, but it never hurts to check.

Farms of Linux Machines?

[Anonymous Coward]

It is my understanding that they used a farm of FreeBSD machines for rendering special effects in The Matrix. You can read about it here:

http://www.freebsd.org/news/press-rel-1.html

Ben.

Unix Involvement

[bifrost]

Actually, if you'd been paying attention, they used FreeBSD instead of Linux for the Rendering on the movie. If they used FreeBSD instead just for the rendering, why would they use it for the camera operation?

Fad or fascination?

[Paolo]

These setups have been used in a lot of commercials recently too. The problem is, as always in the film industry, that the setup is much too expensive for small studios to pioneer some amazing things with them. Often it is the independent filmmakers who revolutionize techniques in cinematography, for they aren't afraid to experiment with techniques or styles.

I think we'll be moving more in this direction, in terms of filmmaking production, as well as satellite distribution to theaters. Viewers always wish to be awed, so I don't think this style will be a fad.

Re:matrix DVD

[waynem77]

There's a red pill in the background. Click it, and they go into "The Making Of...". Clever.

Cheaper things with time and cameras...

[fingal]

Slighly offtopic, but still quite fun:-

• take an slr camera
• expose a complete film with the lens cap on. The film is therefore at the end of its roll, yet no light has hit any frames yet.
• point the camera at some moving item, for example a person doing semaphore stylee things, with the camera on a tripod (it's probably a good idea to have the person against a neutral dark background)
• take off the lens cap!!!
• trigger the camera in long-exposure mode
• while you have the shutter open, wind the film back to the beginning at an uneven speed stopping occasionally.
• develop the complete film print, joining the adjacent prints into one single long picture.

What you will have is a single image with an uneven version of time across the image. When the film was moving fast then you get a pretty much blurred version of the image, but at the points where you stop the film movement you have frozen exposures of the object. If the object has a clearly developed movement from start to finish then you get a nice mix of space vs time.

Like I said pretty off-topic, but quite an amusing thing to do in these long winter evenings. Thanks to Zoe Millington for coming up with the idea.

Re:matrix DVD

[Roast Beef]

That's true. In addition to the "Making of" special that was shown on HBO, the DVD has a special showing how "bullet time" was done. It shows stage by stage, from the setup of the cameras, to the jumpy shot done in front of a green screen, to the final composited scene. "What is Bullet Time?" is chapter 19 or 22 (both work), and you might want to also check out chapter 33, which is a behind-the-scenes about the rooftop fight.

Nothing Spectacular

[Accipiter]

This is nothing new. As a matter of fact, there's actually a really cool Behind-the-scenes featurette on the Matrix DVD. They actually show each step of the process, from the camera circle / GreenScreen, to the Computer rendering of the walls, to finished product. Good stuff.

Whoever thought up this technique was brilliant. The design is simple, but obviously *very* effective. Basically, the design is simply a row of cameras (usually circular with varying height) that film on a central position. The cameras are all exactly synched with eachother, and film simultaneously. During the editing process, the film from each angle is played at the same time, and frames from each camera are used as input to the final master. So say Camera 1 is at position A, Camera 2 is at position B, and Camera 3 is at position C. All three cameras are filming one central point. During editing, Frame 1 is taken from Camera 1, and the next two frames in succession are taken from the next two cameras, all from the SAME TIME POINT. (Since all cameras are exactly synched, you get 3 different angles of the same shot.)

When the editing is finished, the shot appears to rotate around the central film point.

-- Give him Head? Be a Beacon?

WAY Better Articles on Virtual Cinematography Here

[Drog]

VFX Pro has two indepth interviews from last April with John Gaeta, the visual effects supervisor for The Matrix, regarding this new virtual cinematography technique, dubbed "bullet time" by the Wachowski brothers. One is here and the other is here.

You don't even need a camera...

[CJ Hooknose]

Heck, I've been thinking about this for a while. I am currently rendering a small, stupid 3.5 second animation that does the "freeze-n-pan" thing used in the Matrix, all in POVray. After about 3:10 EST, you can go here and see the result. The source code will all be available here so people can see how I did it. I'm sure all the raytracing folks out there know exactly how to do this, but if non-raytracing folks are curious....

The patent is linked from their web site.

[Ungrounded Lightning]

I noticed when reading it that it made four claims. Three related to a linear array of "cameras" (i.e. lenses) working on a single strip of film. The fourth substituted "video storage means" for the film, apparently implying substituting video cameras for the row of lens/box "cameras".

The various descriptions on the web site gave me the impression that a series of independent film or video cameras would have even more potential: Varying spacing to accellerate/decellerate the pan, varying positioning to pan in 3-D, switching to full motion at varying speeds at selected intervals, etc. And the authors appeared to understand this potential. So I wondered at the omission in the patent.

Then I checked some of the references, and discovered that such (at least with film) had already been patented before - far enough back that the patents are expired.

So it looks like doing this with an array of independent film cameras is prior art. Video cameras in any configuration except evenly-spaced along a straight or curved line (which is covered by claim 4) also appears to be open, and you might break claim 4 by treating it as a special case of the previous expired patent with the obvious substitution of video cameras for film cameras.

Essentially all the patent covers is a camera with many lenses and synchronized or sequenced shutters, projecting onto a common film strip, along with a multiple-video-camera model of it.

true 3D visualization...

[Uberminky]

One thing that I love about that (far overused) effect where they fly around the scene in slow motion is that you can capture consecutive frames and make stereograms of the scenes. I've made a couple of (granted, *very* low-quality) stereograms (cross your eyes to see them) from the movie, which I had fun doing... I think it'd be awesome to be able to watch the WHOLE MOVIE in true 3D tho... I wish that would catch on....

http://php.indiana.edu/~dgsharp/stereograms/

Cameras are only a tiny part of the story

[cying]

I know one of the people who did the R&D on bullet-time sequences in "The Matrix"; he recently gave a seminar at U.C. Berkeley along with Jon Gaeta where they discussed how the bullet-time sequences were done.

First, the difference between bullet-time sequences and the GAP commercial sequences is a big one:

Freeze-time shots (e.g., the GAP commercial) are easy to do. All shots are taken simultaneously of the scene, and you don't need to worry about the motion of the subjects in the scene.

Bullet-time shots actually have to move in very slow motion. At the seminar, they said that although they had many cameras firing sequentially over the camera path, they were unable to place cameras close enough together to capture sufficient frames during really slow segments of movement (if you watch The Matrix bullet-time sequences, you'll see that initially the motion starts out very slow, and gradually speeds up)

The way that Manex solved this was to use computer vision techniques to interpolate the necessary "in between" frames. This is especially difficult since the motion in some shots (i.e. Keanu Reeves' arm waving in a circle in the air) have motion that isn't linear (meaning that the compute can't simply compute the pixels along a straight line from one frame to another). Manex used a lot of combined interpolation techniques to achieve the results in the movie.

In addition, obtaining consistent camera lighting, film grain, and film speed parameters proved difficult. They used cameras that were all uniform in make and model, but had to image process the frames to achieve consistency.

Second, you may notice that all bullet-time sequences were captured on a green screen! One of the reasons they did this was because the angle of rotation is actually more than 180 degrees. (This is also a difference from the GAP commercial) So how did they insert the background?

Well the answer is, they re-created the backgrounds. Manex used image-based modelling and rendering techniques that were based on work done by Dr. Paul Debevec at U.C. Berkeley. You can read more about the FACADE photogrammetric modelling system and The Campanile Movie (which I helped work on) by following the link.

Manex's techniques greatly improved upon the work at U.C. Berkeley; they showed an OpenGL real-time demo of the sub-way and government building lobby shots from the movie at the seminar; very cool stuff.

Hope that sheds some light on how effects in The Matrix are really done.

-- Charles

Re:Matrix and rendering farms

[mr]

You have to understand, Mr. Plant is a dyed in the wool Linux advocate. So, from his perspective, if it's not using GNU/Linux its not cool. (And if it happens to be cool, it MUST be an accident.) So, keep this in mind when he writes on a topic.

The comment "What? An advance in cinematography that doesn't involve a farm of Linux machines?" was a jab at FreeBSD. Note the only OpenSource-Matrix reference commonly published is the BSD farm used for graphic rendering.

Given this quote from TimeTrack s own web page:
"We even have an optional mechanical shutter system which requires no electrical power."
It would seem the use of GNU/Linux technology for this camera system is a situation where the extra technology is not needed.

Trying to put your choice of technology where it is unneeded is the hallmark not of advice, but the same kind of tactic Microsoft uses....that of "All problems look like a nail(microsoft part number x-095687-002) and you hit it with a hammer(microsoft part number 94374)"

For him, the OpenSource movement is Linux, and not the OpenSource movement has Linux as a part of the movement.

(For everything bad said about Mr. Stallman, at least he has respect for others. When Mr. Stallman was looking for his access badge, Mr. Plant told him that he should tell the guard "he was Richard fucking Stallman, and didn t need a badge." Mr. Stallman pointed out that why should he make the guard s life difficult. The guard is only doing his job.)

Re:Matrix and rendering farms

[mvw]

For everything bad said about Mr. Stallman, at least he has respect for others.

Come on. Let's keep the Linux-BSD competition a friendly sportive one, it should not been carried out at the personal level.

Anyways thanks for that RMS story. By the way, he does not crumble to dust if brought in contact with BSD, as this link from an Australian Unix user group meeting proofs. :-)

Here is the caption:
Peter Wemm trying to convince Stallman to adopt the Berkeley Licensing conditions. You'll notice (but not recognize) that Stallman is holding a FreeBSD CD-ROM set in his right hand. It obviously doesn't taste as good as the Australian wine in his left hand.

How do they ensure consistent exposure?

[drix]

As anyone who has ever photographed a picture before can tell you, just a tiny difference in the exposure settings - aperture, shutter speed, flash level, film type - will produce a dramatically looking picture. I've seen the setup they used on the Matrix on "Making of the Matrix" on the DVD, and basically it looked like a whole bunch of Canon EOS-1N RSs strung together in an ascending circular fashion (for the Trinity scene). I'm curious how they ensured that all the colors and shadow/highlight detail matched perfectly with what they were filming so they could do a seamless transition from filmed footage to effect. Obviously, I guess they locked the exposure details, but what about film? Do they load the cameras with film stock and just shoot on that? Is computer correction involved?

Also, I never could figure out how they did they scene with the woman in red where they froze everyone in time until reading this. Can anyone elaborate more and how they varied the length of the exposure to produce that effect?

--

Sell Linux as a movie studio and die

[heroine]

Look at every small utility coming out nowadays, every company making pledges, and every diagnostic ever done to Linux and it's a server and only a server. Send an email to VA Linux with the word "video" in it and you'll be exiled from California forever. The guys whose job it is to promote Linux professionally are actually the most vehemently opposed to its use as a movie platform. They do not want it to be used for anything but a web server. It might be because the internet and the movie industry are at each other's throats. It might be because changing a marketing scheme which isn't broken is a bad idea. They sell web servers and they don't want Linux to get any other reputation.

Saw this on Splat!

[jfunk]

I saw a fair bit about this camera on Splat!, a show that comes on Teletoon here in Canada (I don't know if it comes on in other countries).

Basically, the guy who invented it is an engineering school dropout. They had an interview with him which was very interesting.

Re:CineFX magazine

[vtweb]

oops- Name of the magazine is Cinefex- their website is www.cinefex.com

Technical aspects of this technique

[Anonymous Coward]

Actually, the Manix camera is a smaller version of a camera that we developed for Big Fish Films (or Check here) in 1997, and which we are refining now.

That camera used a bunch (360!) of independent 35mm cameras, which could be arranged in a full 360 degree circle. This yields 12 seconds of film (at TV frame rates - 15 seconds at movie rates).

The cameras were controlled by a computer, which can assign a time delay to each camera independently. So, the system can be used to freeze a scene, do a "virtual dolly", do both, ramp the frame rate from any speed to any other speed, etc.

The reason for using this many cameras is that the quality of the final motion sequence is much better if you get the whole thing on film than it is if you have to do computer interpolation. Additionally, the cost of computer generating frames is VERY high.

So, the technical troubles with a multi-camera system are (in no particular order):

Synchronization: Even if you trigger a bunch of cameras at the same time, the shutters won't open at the same time. We call the delay "Lag Time", and it is dfferent for every camera we've tested. It doesn't matter if you have a consumer camera (Nikon N50, Canon EOS Rebel) or an expensive professional camera (Nikon F5, Canon EOS 3) - each individual camera, even the same model number - will have different timing.

Exposure: A previous poster mentioned the problems with subtle variations in exposure creating problems. Bingo! The trouble is that still cameras are meant to be consistent from frame to frame, not from camera to camera. Even a $2500 professional camera body will have variations of about +- 1/3 stop from camera to camera. When you sequencs these frames, the film looks like it was taken with a 1940's 8MM camera - bright/dark/flickery - terrible.

Lenses: Like the shutter, lenses can have a profound effect on the "look" of the frame. The exposure, color, and focus will be different from lens to lens. The denter of the frame will be fine, but the edges can be a problem (because of edge, the perspective warping of different lenses can be a bit, well, different)

Rotation / Focus / Setup: Remember, somebody has to point all the cameras at the right spot (or spots), focus them, possibly adjust zoom, etc. There's a whole lotta room for error here. Luckily, there's a machine, known as a rank, which is used to correct minor translations and rotations.

Spacing: Someone had mentioned that the cameras were too close together, thus requiring computer interpolation. Yup, that's a problem.

There are a few advantages to the multi-camera technique, as well:

Directional Flexibility: Each camera can be pointed at whatever you want. You can do pans, tilts, different zoom levels, freezes, virtual camera motion, etc. In fact, with 360 cameras, you could do them all in the same scene!

Timing Flexibility: This one says it all. Simple example: Go 30 frames/sec. on even numbered cameras, then freeze odds. The final film looks like a regular dolly around the subject for 6 seconds (subject is in motion), then we go around the subject for 6 seconds again while the action is frozen. Start doing speed variations and smooth timing curves, and you can get some interesting effects (want to see things happen in reverse time? go ahead. ...)

Well, I've wasted enough of your time and bandwidth. Just thought I'd tell people about the system that has done most of the freeze effects that have been done (about 75 shoots, versus 35 for all others combined).

Cheers.

Re:Hey, those are neat!

[Uberminky]

Be sure and email me if you make any good ones! I'd love to see'em! :) I just started making a couple of new ones for my site from

http://www.virtualcamera.com/samplework.html

which has some good fly-arounds of stuff from commercials they've made. I'd love to have 2 FireWire video cameras and strap them together, and film a short movie or something... =) Get some good polarized filters and two nice digital video projectors, and some polarized glasses..... NICE. =)

You can "exctract" the third dimension from just about anything if you're careful. Someone turns their head, a car drives by... Problem is getting a good clean couple of frames to choose from where no other objects move and stuff. And if you're filming them yourself, you've gotta be careful if you "circle" your target, because it can screw things up, making it hard to see (as I found out the hard way ;).

One movie I'd love to try this with is Enemy of the State, as they panned like crazy in that movie (not to mention a couple of the "fly-arounds"). I'd love to see anything you come up with tho. :) (It's a shame my video capture setup sucks so much... if anybody's got The Matrix on DVD and would mind making a few NICE resolution versions of the ones I made... DO IT! Geekkind will thank you! :)

A little more info...

[garagekubrick]

Just for those who don't want to go reading everywhere else...

The really impressive thing about those Bullet Time sequences is not really the camera setup (and yup, they were Canon EOS) - which is one of those ideas that several people seem to have come up with at once - but rather the frame interpolation work Manex did, as explicated in the article above. That was truly flawless stuff, as the individual character elements (live action plates) of Keanu, et. al had frames rendered by a puter to complete the range of motion in an automated fashion.

But even more impressive is that the background plates for those shots were also developed from photographic stills - taken from several different angles on the set or location, and a computer then developed 3 dimensional geometry so that there was a model of said set and textured properly without having to do it by hand. The potential application of this for filmmaking is enormous, and a massive step towards the truly virtual set.

I'm really, really hoping Manex clinch the Oscar this year, for their work truly was groundbreaking as opposed to the Phantom Menace team who refined existing techniques to, admittedly, unbelievable highs. However, none of their work really goes out there in terms of future potential apps. It's a given, though, that ILM will be getting the lil statue this year. Tis a pity.

And for anyone who wants to read about FX advancements, the Bibles are Cinefex, VFXpro, and ALWAYS, American Cinematographer. Read em and weep.