Cool Matrix Filming Techniques

webword writes "Here's how those cool scenes from the Matrix were filmed (go here). Not that I want actually buy one of these cool cameras, but I hunted around to find out how to get one and how much they cost. You can get one here. This brings up a quick question: How are people keeping up with the latest and greatest filming techniques?" What? An advance in cinematography that doesn't involve a farm of Linux machines?

Comments

The patent is linked from their web site.

[Ungrounded Lightning]

I noticed when reading it that it made four claims. Three related to a linear array of "cameras" (i.e. lenses) working on a single strip of film. The fourth substituted "video storage means" for the film, apparently implying substituting video cameras for the row of lens/box "cameras".

The various descriptions on the web site gave me the impression that a series of independent film or video cameras would have even more potential: Varying spacing to accellerate/decellerate the pan, varying positioning to pan in 3-D, switching to full motion at varying speeds at selected intervals, etc. And the authors appeared to understand this potential. So I wondered at the omission in the patent.

Then I checked some of the references, and discovered that such (at least with film) had already been patented before - far enough back that the patents are expired.

So it looks like doing this with an array of independent film cameras is prior art. Video cameras in any configuration except evenly-spaced along a straight or curved line (which is covered by claim 4) also appears to be open, and you might break claim 4 by treating it as a special case of the previous expired patent with the obvious substitution of video cameras for film cameras.

Essentially all the patent covers is a camera with many lenses and synchronized or sequenced shutters, projecting onto a common film strip, along with a multiple-video-camera model of it.

Cameras are only a tiny part of the story

[cying]

I know one of the people who did the R&D on bullet-time sequences in "The Matrix"; he recently gave a seminar at U.C. Berkeley along with Jon Gaeta where they discussed how the bullet-time sequences were done.

First, the difference between bullet-time sequences and the GAP commercial sequences is a big one:

Freeze-time shots (e.g., the GAP commercial) are easy to do. All shots are taken simultaneously of the scene, and you don't need to worry about the motion of the subjects in the scene.

Bullet-time shots actually have to move in very slow motion. At the seminar, they said that although they had many cameras firing sequentially over the camera path, they were unable to place cameras close enough together to capture sufficient frames during really slow segments of movement (if you watch The Matrix bullet-time sequences, you'll see that initially the motion starts out very slow, and gradually speeds up)

The way that Manex solved this was to use computer vision techniques to interpolate the necessary "in between" frames. This is especially difficult since the motion in some shots (i.e. Keanu Reeves' arm waving in a circle in the air) have motion that isn't linear (meaning that the compute can't simply compute the pixels along a straight line from one frame to another). Manex used a lot of combined interpolation techniques to achieve the results in the movie.

In addition, obtaining consistent camera lighting, film grain, and film speed parameters proved difficult. They used cameras that were all uniform in make and model, but had to image process the frames to achieve consistency.

Second, you may notice that all bullet-time sequences were captured on a green screen! One of the reasons they did this was because the angle of rotation is actually more than 180 degrees. (This is also a difference from the GAP commercial) So how did they insert the background?

Well the answer is, they re-created the backgrounds. Manex used image-based modelling and rendering techniques that were based on work done by Dr. Paul Debevec at U.C. Berkeley. You can read more about the FACADE photogrammetric modelling system and The Campanile Movie (which I helped work on) by following the link.

Manex's techniques greatly improved upon the work at U.C. Berkeley; they showed an OpenGL real-time demo of the sub-way and government building lobby shots from the movie at the seminar; very cool stuff.

Hope that sheds some light on how effects in The Matrix are really done.

-- Charles

Technical aspects of this technique

[Anonymous Coward]

Actually, the Manix camera is a smaller version of a camera that we developed for Big Fish Films (or Check here) in 1997, and which we are refining now.

That camera used a bunch (360!) of independent 35mm cameras, which could be arranged in a full 360 degree circle. This yields 12 seconds of film (at TV frame rates - 15 seconds at movie rates).

The cameras were controlled by a computer, which can assign a time delay to each camera independently. So, the system can be used to freeze a scene, do a "virtual dolly", do both, ramp the frame rate from any speed to any other speed, etc.

The reason for using this many cameras is that the quality of the final motion sequence is much better if you get the whole thing on film than it is if you have to do computer interpolation. Additionally, the cost of computer generating frames is VERY high.

So, the technical troubles with a multi-camera system are (in no particular order):

Synchronization: Even if you trigger a bunch of cameras at the same time, the shutters won't open at the same time. We call the delay "Lag Time", and it is dfferent for every camera we've tested. It doesn't matter if you have a consumer camera (Nikon N50, Canon EOS Rebel) or an expensive professional camera (Nikon F5, Canon EOS 3) - each individual camera, even the same model number - will have different timing.

Exposure: A previous poster mentioned the problems with subtle variations in exposure creating problems. Bingo! The trouble is that still cameras are meant to be consistent from frame to frame, not from camera to camera. Even a $2500 professional camera body will have variations of about +- 1/3 stop from camera to camera. When you sequencs these frames, the film looks like it was taken with a 1940's 8MM camera - bright/dark/flickery - terrible.

Lenses: Like the shutter, lenses can have a profound effect on the "look" of the frame. The exposure, color, and focus will be different from lens to lens. The denter of the frame will be fine, but the edges can be a problem (because of edge, the perspective warping of different lenses can be a bit, well, different)

Rotation / Focus / Setup: Remember, somebody has to point all the cameras at the right spot (or spots), focus them, possibly adjust zoom, etc. There's a whole lotta room for error here. Luckily, there's a machine, known as a rank, which is used to correct minor translations and rotations.

Spacing: Someone had mentioned that the cameras were too close together, thus requiring computer interpolation. Yup, that's a problem.

There are a few advantages to the multi-camera technique, as well:

Directional Flexibility: Each camera can be pointed at whatever you want. You can do pans, tilts, different zoom levels, freezes, virtual camera motion, etc. In fact, with 360 cameras, you could do them all in the same scene!

Timing Flexibility: This one says it all. Simple example: Go 30 frames/sec. on even numbered cameras, then freeze odds. The final film looks like a regular dolly around the subject for 6 seconds (subject is in motion), then we go around the subject for 6 seconds again while the action is frozen. Start doing speed variations and smooth timing curves, and you can get some interesting effects (want to see things happen in reverse time? go ahead. ...)

Well, I've wasted enough of your time and bandwidth. Just thought I'd tell people about the system that has done most of the freeze effects that have been done (about 75 shoots, versus 35 for all others combined).

Cheers.