Matchmoving is a visual effects technique that allows the insertion of computer graphics into video footage. In this tutorial we'll start by taking a look at how we can track our footage using Boujou. We'll then jump into Maya and use that tracking information, along with a custom HDR image, to create and render a 3D object which we can insert into the scene. We'll then use After Effects to composite the 3D object into the original video. The techniques used in this tutorial can be applied to almost almost all of your 3D integration shots.
First we need to convert our video into sequence of images so that it can be read by Maya and Boujou. Open After Effects and go to “File > Import > File…”.
Navigate to your footage, select it and click "Open" to load the video into After Effects.
To create a composition with the same characteristics as our source footage, just drag the video to the new composition button.
You should now be able to see the video along with it's timeline.
We now need to change our composition's frame rate to match the frame rate in Maya. Go to “Composition > Composition Settings…”
As the frame rate for this footage is 30fps, I'm going to set the frame rate to 30 frames per second and click "Ok".
To render the video, go to “Composition > Add to Render Queue”.
Once the render queue has opened, click on the word "Lossless" in the output module section.
In the output module settings, select “JPEG Sequence” from the dropdown and then press “OK”.
Click here to change the name of your rendered image sequence.
Select the folder where you want your sequence to be rendered to (remember that we're going to be creating a lot of images!) Now change the name of your video to match this format : “Name of your video.[####].jpg”. The ".[####]" is extremely important as it ensures that our images are named sequentially. Also, make sure that you use a dot and not a under score in the filename, as Maya won't recognize the sequence if it is not in the above format.
We're now ready to render the sequence, so click "Render".
Open Boujou and click on “Import Sequence”.
Find the sequence that you rendered with After Effects, select the first file and click "Open".
Click on “OK”.
Click on the “Edit Camera” button.
We need to make sure that Boujou is working with the same frame rate as our sequence, so change the frame rate to 30 and click “OK”.
Click on the “Track Features” button.
I usually leave the settings as default, so just make sure “All frames” is active and click on “Start”.
Boujou will now start the initial tracking process.
Now that Boujou has finished, we have 2-dimensional tracking data for our shot. However, in order to integrate a 3D object into the footage we need to calculate the position and rotation of the camera. To begin this process, click on the “Camera Solve” button.
Make sure that “All frames” is active and click on “Start”.
Boujou will now start calculating the scene camera's position and rotation.
Once Boujou has finished tracking, we now need to align our scene to match the footage. Scroll through the video and select 2 or more points that lie in a straight line, in this case on the tabletop. These points will form our X-axis.
Click on the “Scene Geometry” button.
Click on “Add Coord Frm Hint”.
Select the X-axis from the "Type" dropdown and click on “Connect to Selected”.
With our X-axis in place, we now need to define at least one other axis so that Boujou can correctly orient our scene. Close the previous window and now select 2 or more points that lie in a straight line, but at 90 degrees to our X-axis. This will be our Z-axis and in this case travels away from camera across the tabletop.
Click on the “Scene Geometry” button.
Click on “Add Coord Frm Hint”, select the Z-axis from the "Type" dropdown and click on “Connect to Selected”.
If you can confidently select points in order to calculate the Y-axis, do so and repeat the process as before, making sure to select the Y-axis from the "Type" dropdown before clicking "Connect to Selected". In this scene, we're unable to select Y-axis points so we'll move straight on to the next step.
Click on the “Scene Geometry” button, then click “Update Coord. Frame” and close the window.
Activate the “Ground Plane” checkbox in the overlay tab on the bottom-right of the Boujou window.
We can now see the floor plane that has been calculated for our scene. It should stay in place throughout the video (try scrolling through the footage to make sure that this is the case.) If it does not however, repeat steps 24-31 above, choosing different points to orient your scene.
To make sure that our track is correct, we're going to create some test objects. Click on the “Add Test Objects” button.
Select the object of your preference and close the window, I usually choose “Ladybird”.
Play the video to see if the object stays in place throughout. As with the ground plane overlay, if your object does not sit correctly within your footage, your scene might not be oriented correctly. If this is the case you should repeat steps 24-31 above choosing different points for each axis.
We now need to export our tracking information to Maya. To do this, go to “Export > Export Camera Solve”.
Browse to your project folder and choose a filename, change the export type to “Maya 4+” and then make sure the move type corresponds to your footage. As this footage was shot using a handheld camera, I've chosen "Moving Camera, Static Scene". With this done, click “Save”.
After opening Maya, go to "File > Open" and open the file that we just exported from Boujou. Your viewport should look similar to this image.
In the menu at the top of the active viewport, go to “Panels > Perspective > Camera_1_1”.
We will now be looking through the matchmoved camera created for us by Boujou.
Go to “Create > Text” and click the option box.
Enter your text and use the available options to add bevels to the text object. Click "Create" when done.
As we aligned our scene within Boujou, our new text object should automatically sit on top of the table in our original footage.
Rotate, scale and position the text as desired, making sure that it remains sitting right on top of the grid.
Go to “Window > Rendering Editors > Hypershade”.
Select "Create Mental Ray Nodes" from the dropdown on the left, and click “mia_material_x” in the "Materials" group to create a new material.
With the new material still selected, press "Ctrl-A" to open the attribute editor and change the diffuse color of your material to your desired color, in this case a red/orange. We also want to change the following options : Reflection Color to grey, Reflectivity to 1.000, Glossiness to 0.350 and Glossy samples to 0.
Close the Hypershade. Right click our text object in the viewport and go to "Assign Existing Material > mia_material_x1" to apply our new material to the text.
Open the “Render Settings” window.
Select "mental ray" from the “Render Using” dropdown. Now change the following options in the Common tab : Choose a file name prefix (in this case I have added a folder structure before the name itself to ensure that the files are saved to the correct location,) change the image format to TIFF, change the “Frame/Animation ext” to name.#ext, set the "Start frame" and "End frame" values to match that of your animation, select “Camera_1_1” from the "Renderable Camera" dropdown, and make sure that the rendered image size matches that of your footage (in this case I have selected the HD 1080 preset.) With that done we can now close the render settings window.
We now need to make sure that Maya's frame rate matches that of our footage. Go to “Window > Settings/Preferences > Preferences”.
In the "Settings" category change the time to match that of your footage. Here I've selected “NTSC (30 fps)”. Click "Save" When done.
Re-open the Render settings window.
In the Quality tab we need to change the following options: set the "Max Sample Level" to 2, "Multi-pixel Filtering" to "Gauss", activate "Jitter", "Raytracing Reflections" and "Refractions" to 3, "Raytacing Max Trace Depth" to 6, set "Motion Blur" to "Full" and set the "Motion Blur By" value to 2.000.
Go to the “Indirect Lighting” tab and create an Image Based Lighting Environment. Then activate "Final Gathering" and change "Accuracy" to 200, "Point Density" to 0.100 and "Secondary Diffuse Bounces" to 2.
If the attributes editor isn't visible, click "Ctrl-A" to activate it. Click the folder icon next to "Image Name" and browse to your HDRI file.
Go to “Create > Lights > Area Light”.
Making sure that the light is selected, go to “Panels > Look Through Selected” in the menu at the top of the active viewport.
Position the light so that the shadows cast will match direction of the shadows in the original footage.
Go to the Attributes Editor with the light selected and activate “Use Ray Trace Shadows”. With this done, change the “Shadows Rays” to 15.
With the light still selected go to “Edit > Duplicate”.
Select the duplicated light and go to “Panels > Look Through Selected” in the menu at the top of the active viewport.
Position the new light at the same height but on the opposite side of our text. This helps to produce softer shadows in the final render.
Go to the attributes editor of the new light and turn off "Use Ray Trace Shadows".
In the active view menu go to “Panels > Perspective > Camera_1_1”.
Now we need to create a surface to act as a shadow-catcher. Go to “Create > Polygon Primitives > Plane”.
Move and scale the plane so that the shadows of the text will fall within it's boundaries.
Go to “Window > Rendering Editors > Hypershade”.
Select "Create Maya Nodes" from the dropdown on the left, and click the “Use Background” material in the "Surface" group. This material allows our plane to receive shadows and reflections, but ensures that we can still successfully composite the final render on top of our original footage.
With our new "Use Background" material selected, go to the attributes Editor and reduce "Reflectivity" to 0.
Apply this material to the plane, by right clicking the plane object and selecting "Assign Existing Material > UseBackground1".
We're now going to double-check that our objects sit exactly on top of the groundplane. In the menu at the top of the active viewport, go to “Panels > Orthographic > Side”.
Move the objects into position if required.
Go back to the “Camera_1_1” camera.
Create a test-render of the scene by clicking the "Render" icon in the main toolbar.
You can see in our render how the object integrates perfectly within the scene. If the shadows are cut off at this point, close the render view and scale up your shadow-catcher object. Make sure to do another test-render to ensure the problem has been fixed.
Select the “Rendering” menu from the main toolbar dropdown and go to “Render > Batch Render” to begin rendering the sequence.
When Maya finishes the rendering process jump over to After Effects and go to “File > Import > File…” .
Navigate to find your newly rendered sequence, select the first frame and, making sure that the “TIFF Sequence” option is active, click on “Open”.
A window asking for the alpha channel will pop up. Select “Straight - Unmatted” and click “OK” to import the footage along with it's alpha channel.
We now need to import the original video sequence. Go to “File > Import > File…”.
Browse for the sequence of images we initially rendered out from After Effects, select the first frame and, making sure that the “JPEG Sequence” option is active, click “Open”.
You should now see both sequences in the project bin.
Drag the .tif sequence we rendered from Maya to the new composition button.
Now drag our original .jpg sequence into the composition we just created, making sure to position it below the .tif sequence.
Our two layers are now composited together in the viewport.
If you zoom in, however, you'll notice that there's currently a glitch. This is because whilst the foreground sequence begins on frame 1, the background sequence begins on frame 0.
To correct this error, zoom into the time line using the small slider at the bottom of the screen, and drag the green bar representing our background sequence 1 frame to the left.
With this done, you can now see that the glitch has now disappeared.
All we need to do now is render the video. Go to "Composition > Add to Render Queue" and set your desired output options.