CompuServe Thread

all, I was asked to upload a description of my "workaround" technique for camera tracking in 3dsr4. Here's the text of a file "cam-trak.txt" which I intend to upload for the good members here. Many TV and film CG effects call for the integration or compositing of CG elements into live action. If the camera isn't locked off completely this will involve matching the motion of a live camera with a 3d Studio camera. In the best case scenario you will get frame by frame camera position, tilt, roll, and FOV information. Even this would necessitate some translation of the tilt data to camera target x,y,z data. This could be done in IPAS or with r4's keyscript language. More often than not there will be no camera head data for various reasons: it's too expensive, takes too long for the crew to setup, wasn't calibrated correctly etc. In this case you can still salvage the situation to a degree by ensuring you have a "subject" of known dimensions in view throughout the shot. Ideally, the subject should be an inanimate object with some angular characteristics – cubic, triangular things are good. Also get some measurements of start/end camera distance to subject, camera height, angle to subject and other props or features in the shot, camera lens info etc. A helicopter or crane move is going to be almost impossible to match successfully – you may want to pass on these types of nightmare projects. The following text describes a technique I successfully used for a 3d Studio rendered special effects shot composited over live action involving a camera dolly for the film "Virtuosity". It was for a holographic head/VR kiosk which delivers a news report to the passers-by in a busy downtown plaza. The CG head lip-syncs about 10 seconds worth of optically motion-captured dialog and floats atop a kiosk – a prop of known dimensions. It was quite an achievement for me as an animator since nobody else in the studio's PC department had yet succeeded in camera-tracking other shots in "Virtuosity" even at video resolution. The clients had to resort to 2d tracking in post. This shot was to be at 2k resolution and occupy a fairly large portion of the frame (vertically) at the end of the move. Therefore the tracking would be quite a bit less forgiving. Once in 3ds, build your reference objects in real world dimensions (architectural units) and keyframe just the start and end positions for the camera over the shot's duration, adding ease to and ease from. Digitize the live plates (backgrounds) at the highest possible quality – these could be film or video plates. Keep these plates on the fastest, local hardrive of the system you intend to perform the camera tracking – this ensures the fastest possible load times. Also very helpful is to composite a reference number (or timecode) onto each plate that corresponds to the frame number in the keyframer. For example the background which goes with frame 120 in the keyframer would have the number 120 composited unobtrusively in a corner or wherever is convenient to avoid the action portion of the plate. I found the camera control/perspective matching IPAS/plug-in of 3dsr4 the best way of aligning my 3ds camera to the background. You get a reasonably detailed colour image which shows far better detail than the grayscale, proxy view available in the regular 3ds modules. It speeds things up considerably to resize your backgrounds in advance to avoid having the plug-in do this every time it loads one. Since my project was for film, the backgrounds were scanned at 2048×1536 resolution. To get the highest accuracy in the plug-in, I needed to run the 3ds main display at 1600×1200, which left an active background area in the plug-in of 1417 x 1131 (according to the status line). Therefore I made resized versions of the background plates at 1417 x 1063 – to maintain their 4:3 aspect ratio. It would've been nice to load the backgrounds at full resolution allowing camera adjustments pixel for pixel, but that wasn't possible with my display card or monitor. On my system (Pentium 100, 128mb RAM, Diamond Stealth 64 4mb VRAM, Seagate Barracuda 2.1 gb scsi fast and wide) backgrounds loaded in around 8 or 9 seconds and camera adjustments could be made very quickly and accurately using the cursor arrows or direct x,y,z data entry. The way to effectively bring camera keyframe data into the 3d editor for adjustment with the IPAS was to start the animation on frame 1 in the keyframer and reserve frame 0 for copying camera key info back to the 3d editor. Here's a quick summary of the repetitive steps I used to match the motion: 1) study the live action played in real-time or slow motion and decide where you need to add more keyframe data. Use timecode or some form of frame counter to reference this point. At that frame add keyframes for camera position, (also FOV and roll, if necessary), camera target position. 2) copy those keyframes from the current frame to frame 0. Make a note of what frame you are working on. 3) in the 3d editor make the camera view active and full screen. Enter the camera preview plug-in and load the correct background plate for that frame. Use wire frame shading mode. Make the adjustments necessary to align your geometry with the reference object(s) in the background. When satisfied change the background to none – otherwise the background will automatically be loaded when you return, like it or not – and exit, saying yes to update the camera. 4) in the keyframer, copy the camera and target keys from frame 0 back to the frame they came from. Careful notes and checklists should prevent mistakes here. Repeat until you feel happy. Render a test animation with wireframe geometry over the background (again, frame numbers really help). You can use the render/region command to save time, especially with film resolution. Carefully analyze the test and note problem areas. Repeat the above steps on those frames. Remember: fewer keyframes is generally better, to avoid introducing bumps that weren't originally there. Where there are natural bumps and jiggles in the live camera, which is quite typical and normal, you will have to add more keyframes to those areas to match the bumps. It took me about 5 sessions of 2-3 hours to get a visually perfect match that I was happy with. Actually, after the 3rd session the visual effects supervisor was happy enough, but I wanted to make it better for my own satisfaction so I spent the extra time. Check out the scene which should appear in the first 5 minutes and is possibly the first CG effect in the film. See for yourself if you think the motion tracking worked with this method. All the people involved seemed very happy with the shot. copyright 1995 Martin Foster