Over the course of this extensive 7 day series. Shray Khanna will guide you through the entire process of creating a next-gen, low poly vehicle using Maya and Photoshop. Throughout the series, Shray covers the complete next-gen pipeline in detail. In day 5 we'll prepare the model for the map baking process and render out Ambient Occlusion and Normal maps using xNormal.
Also available in this series:
- "Next-Gen Armored Car" Day 1
- "Next-Gen Armored Car" Day 2
- "Next-Gen Armored Car" Day 3
- "Next-Gen Armored Car" Day 4
- "Next-Gen Armored Car" Day 5
- "Next-Gen Armored Car" Day 6
- "Next-Gen Armored Car" Day 7
Now for generating the Normal maps. We already have the first two steps completed i.e. The High poly Model and the Low Poly UV-mapped model. Now for the third and main part which is normal smoothing on our low polygon model. For this we have to select edges and apply smoothing to them . So to get started we'll select the turret and Go to Normals>Set Normal Angle...
The "Polygon Soften/Harden Edge Options" window will open, it has only one option which is "Angle". For the initial stage set the "Angle" to 45. This "Angle" means that if two faces are at less than 45 degrees, the edge in between them will be softened. And where the "Angle" is more that 45 degree, the edge in between them will be hardened.
Now all the edges will either be softened or hardened depending on their angles. There are a few additional techniques we can use when generating Normal maps. One is, wherever we have edges stitched together on our UV's, those edges should be soft. And where the UV's are not sewn, the edges should be hard. A example of the stitched UV's is shown below.
Now to make the selected edges soft, keep the same selection and go to Normals>Soften Edge (This will soften the edges.)
Now following the same rule, select the Lower Edge that is not stitched, and make it hard via Normals>Harden Edge.
Now select the two edge loops shown below, as these are stitched on our UV's, make them soft.
Now select the edge loop around the small indent. As we can see the UV's are stitched, but if we make the UV's stiffen the smoothing will become very distorted and the normal map won't come good, because the angle for this edge is nearly 90 degrees (1). So where we have an angle of 80 degrees (or above) we use hard edges only and thus have to separate the UV's (2).
Now select the other edge loop and repeat the same procedure on this one too.
So here is the final output of the turret base, with all the smoothing applied to the low poly mesh.
Next select the Gun nozzle and set it's Normal angle to 45, and then apply proper smoothing groups to it.
Now select the small nozzle, and apply the same smoothing to it by pressing "g" (use last command.) You'll notice that the front edge gets softened, but the UV's are not stitched along this edge (1). So select the loop and go to Normals>Harden Edge and that will make this edge hard (2). Here is the final result (3).
Now select the nozzle base and apply the same 45 degree smoothing to it. Then select the middle edge loop and make it soft by going to Normals>Soften Edge (1). Next select the side edges and make them hard (2) Split the UV's as well.
Similarly select the corner edges of the rear portion and split the UV's as shown below.
On the rounded part, apply the same smoothness of 45 degrees. The outcome is perfect for this.
Select the next piece and make all the upper edges soft using the "Soften Edge" option. As they're all stitched in the UV's and have an angle of lower then 60 degrees (1). Then select the 2 lower edges and split the UV's from these edges (2).
Now select all these hardened edges one by one and split their UV's
Now select the two upper hatches and apply smoothing with 45 degrees only (1). Then select the handle edges and make them soft (2).
Now select the edge rings of the handle, and make them hard (1). Here is the final result (2).
Now select the turret light and apply a smoothness of 45 degrees to it (1). Then select the edge ring of both cylinders and make them soft (2).
Now select the lower ring of the base and soften them (1). And here's the final smoothed result (2).
Now select these hooks on the upper hatch and apply the same smoothing (1), then select the lower ring of the cylindrical base and make them soft (2).
Now select the rear mounted gun and apply a smoothness of 45 degrees (1), then select the edges that need to be softened and hardened accordingly, using the same work flow as before (2-3).
Similarly select these loops and make them soft as they're stitched (1). Do the same on the edges below (2). And here is our final result (3).
Now select the large antenna on the turret and apply the same smoothing (1). Then select the upper ring of edges on the small antenna, loop them and make them soft (2).
Do the exact same thing with the small antenna on the other side.
Now select the Front wheel arch and apply the same smoothing to it. We can notice that there's an edge in the middle of the wheel arch which is of no contribution to the shape, so just select that loop and delete the edge. Then join the open vertices using the Split Polygon Tool, also make them soften along side.
Now move on to the front lights and apply the same smoothing, Then select the edges that remained hard and make them soft, and vice versa.
Here's the follow up on the same mesh. Select the edge ring around the center cylinder (1) and make them soft. Then select all the loops around the bottom part and make them soft as well (2).
The next shape is very rounded and only needs soft edges.
Using the same techniques, try to achieve the result shown below on the front mounted gun.
Now select the front hatches, and first apply the same smoothing to them. Then move to the manual part of selecting the edges and making them soft and hard where required. Finally, select the edges that need to be separated in the UV's and cut them.
Here's the follow up on the same mesh, just separate the UV's where there is hard edges, and it will be done.
Now just by using the same techniques on the remaining parts, we can complete the smoothing of the car.
Here is another two parts which are done the same way.
Now we can move onto the main body of the car, so first apply the same smoothing to it (1). Then select the front edge and make it soft (2) (as these are stitched on the UV's) Finally, select the side edges (3) and split the UV's (4).
Now select the Rear wheel arch and apply the same smoothing, and that will work for it.
Now repeat the same procedure on the shovel.
Next select the small parts of the shovel bracket and apply a smoothness of 75 degrees, as it has mostly low polygon cylindrical shapes.
Now select the tail light and apply a smoothness of 75 degrees to it (1), then select the inner edge loop of the base and make it hard (2).
Now select all the small parts on the right side (except the shovel) and make a group (Using Ctrl+g). Duplicate the group and then invert the Scale to -1 on the X-axis and place it accordingly.
Now select the exhausts and apply the same smoothing to them. Then manually select the edges (whichever are required) and soften or harden them accordingly.
Next select the wheel and apply a smoothness of 45 degrees to it. Then select the edge loops that are not stitched in the UV's and make them hard.
Now select these edge loops (1) and make them soft (as their UV's are stitched.) Then select the inner edge loop shown and make it hard (2). Finally replace the other three wheels with this one, so you don't have to repeat the same procedure on each of them (3).
And this should be the final result of your Low poly car after all the smoothing has been applied.
Now go to "Polygons" tab and select Rendering. This will change your UI from Polygons to Rendering.
Now with the main body selected, go to Lighting/Shading>Transfer maps...
A new window has opened ("Transfer Maps") In the first column of the window you'll see "Target Meshes" and because you already had your low poly mesh selected, it has assigned it as your target mesh. The second column is for the "Source Meshes" and that will be your high poly model. And the third is "Output Maps", for our result we only require the Normal Map for now, so select the "Normal" option and it will give you the below options.
Now for the High poly source mesh we need the base mesh and all the nuts and bolts we have placed on it. So first hide all the high poly meshes, select the nuts and make a group of them. (so it will be easier to select later) Next select the group and the high poly main body mesh and say "Add Selected" under the "Source Mesh" column. This will add all the high polygon meshes required for Normal map baking.
Now browse to the folder where you want to save the Normal Map, name it "Main_Body" and change the file type to "Targa(TGA)".
Now set the "Map Space" to Tangent Space and under "Connect Output Maps" check "Assigned Shader"(1). This will apply the generated normal map directly to the applied shader so we can view it instantly. Then in the "Maya Common Output" menu, set the "Map Width" to 2048, (as this will be the texture size for all our maps). And do remember to check the option "Keep Aspect ratio" so our map Width and Height will remain uniform.
Now under the "Mental Ray Common Output" menu, change the "Map Width" to 2048 and the "Number of Samples" to 2 (1). Then in the "Advanced Option" menu, set the "Search Method" to Closest To Envelope (2). (I'll show you soon what an envelop is).
Change the "Max search depth (%)" to 30 (1). And leave the option for "Match Using" set to Geometry Normals (2) .
Now in the "Target Meshes" column, change the "Display" from Mesh to Both (1). You'll notice a pink colored "Envelope" will appear over your low polygon mesh. The main use of this envelope is to contain all the high and lowpoly meshes inside it, as much as possible. For this purpose, set the Search Envelope (%) to 0.8 and you'll notice the Envelope increases in size to cover all the meshes (2).
Now as everything is set and ready to go, just click on "Bake" (1) and it will start baking the Normal map for the Main body. You can track the progress at the bottom of the Maya viewport (2).
When the baking completes, hide all the other meshes and keep only the Main Body. Click on the "High Quality" view button at the top of the Maya viewport (1), and you'll be able to see your Normal Map applied, and I'll bet you can notice the difference (2)(3).
But if you look closely you'll notice few artifacts in the map, this happened because of a few things we didn't take care of. So now we have to check the UV's and smooth the edges, and bake the normal map again.
So now to solve those artifacts We have to split the UV's wherever there are Hard edges at 90 degree angles, and also soften the edges that are at lower angles.
Also take care of the wheel arch's UV's as well.
Now we're ready to go again, but before we do, We have to remove the applied normal map from default material, so to do this, go to Window>Rendering Editors>Hypershade (1) and Select the "Lambert1" material. Click on the "Input Connections" icon (2). This will show the connections to this material in "Work area" below. Now select the 2 connections just before the "Lambert1" material and delete them (3).
Now one thing I didn't do was close down the "Transfer Maps" window. I only minimized it so I don't have to select all the source meshes again. So select the Main Body mesh again and click on "Bake".
Now in "High Quality" mode you can check to see if the artifacts are gone, and the output is looking very neat.
Now again go to"Hypershade" and delete these 2 nodes connected to the "Lambert1" material. Now you have to repeat this step every time before generating a new Normal map. So there won't be any texture applied to the base material and applied to every mesh.
Now select all the assets on the turret, and the turret itself. Everything except the mounted gun, and combine them all (1). Then duplicate it, and in the UV Editor, click on the highlighted icon (2), This option helps you see the Overlapped and flipped UV's of the selected mesh. Now here's some info of which color denotes what :
1. Light Blue color : Proper UV's with no flipping and no overlapping.
2.Dark Blue color : Proper UV's with no flipping, but overlapping.
3.Light Red color : Flipped UV's
4.Dark Red Color : Flipped and overlapped UV's
5.Purple Color: Flipped and non flipped, overlapped UV's
Now whenever we need to generate Normal maps, there should be no overlapping. UV's at all. Flipped UV's will work, but NO OVERLAPING. So now using the color chart info, select the overlapped UV's only and move them out of the 0-1 range (3).
Now go to "Transfer Maps" with the low poly turret selected and click "Add Selected". Then repeat the same procedure of adding the High poly meshes as "Source Meshes".Browse for a new file in the same folder and name it "Turret". The rest of the options can be left as it is. And hit "Bake".
And here is the result, we can now see the details of the High poly mesh on our low poly.
Now once again remove the applied nodes on the "Lambert1" material, and then remove the overlapping UV's of the Wheel.
Now again go to "Transfer maps" while selecting the low poly wheel, and add the High poly as a "Source mesh" (1). Browse to the same folder and name it "Wheel"(2). Hit Bake and here's our final result (3).
Now select all the remaining small parts, combine them and then duplicate the mesh, now remove the overlapping UV's one by one (using the same procedure as before).
Again we repeat the same procedure of adding the High poly mesh and changing the file name to "Parts". Hit Bake and here are the results.
Now for our Ambient Occlusion maps we'll be using "XNormal" , as it's quite user friendly and generates fast results. XNormal is free and can be download at http://www.xnormal.com. Now to generate Occlusion maps with XNormal, we need to have both our Low poly and High poly meshes exported and saved as OBJ's from Maya. Select the Main Body and go to File>Export Selection... and save the file out with the name "Main_Body_Low". In the same way, Export the Turret, Small Parts and the Wheel Low poly model. (Export the ones which don't have Overlapped UV's).
Now using the same method, export the entire High poly body as one mesh, and the High Poly Wheel as another.
Now open xNormal, and in the main window click on "High Definition Meshes" (1).In the main window, right click and Select "Add Meshes" (2) and select the "Armored _Car_High.obj" (3).
Now as your high poly mesh is added, click on "Low Definition Meshes" (1) and in the main window, Right click and select "Add Meshes" (2) and add the "Main_Body_Low.obj" (3).
Now the low poly mesh is added under it. Now click on "Baking options" (2), and click on the Output File Browser (2) and save the file as a TGA with the name "Main_Body_Occ".
Now change the size of the map to 2048x2048, the edge padding to 4 and Bucket size to 16.
Now check off the "Normal Map" and "Ambient Occlusion" boxes, and change the "Antialiasing" to 4x (1). Click on the "Ambient Occlusion Options" box. This will open a new window, change the "Rays" to 256 (2) and now finally click on the big round "Generate Maps" button (3).
Now a new window will open that will show the render progress of our maps. To see the progress, we first need to change the option from "Do NOT Notify Tile Updates" to "Notify Tile Updates" (1). Now you can track the progress in main window and in the lower bar (2). Here is our final result (3).
Now again go to the "Low Definition Meshes" tab and Right click and select "Remove Mesh From List" (1) and add the "Turret_Low.obj". Then in the "Baking Options" tab, save the Output file as "Turret_Occ.tga", and hit the "Generate Maps" button, but this time we will get an error message stating the OBJ mesh has N-sided faces, so we have to remove them from the low poly mesh.
Now back in Maya, select the "Turret Low Poly" mesh and go to Mesh>Cleanup...(1), in the options window check the "Cleanup Matching Polygons" and the "Faces With More Than 4 Sides" options (2). And then "Export Selected" again (3). Now back in xNormal, just hit the "Generate Maps" button and it will generate the map now.
Now in xNormal, add the low polygon mesh of the "Small Parts". And change the name of the Output file to "Small_Parts_Occ.tga". And hit the "Generate Maps" button. This time again we get an error but a different one. It says that the position vertices count is not the same. Now this means that the number of vertices we have in our mesh is lesser or greater that the number of UV vertices we have in the UV editor.
Back in Maya, select the low polygon mesh of Small Parts and apply "Cleanup..." again. Check only the option for "Faces with Zero Map Area" and keep the rest off. This will select the polygons that have no map area in the UV editor. Delete these selected polygons and Export the low poly mesh again. Back in xNormal, hit the "Generate Maps" button once again and it will generate the Occlusion map.
In xNormal, add the "Wheel_High.obj" as the "High Definition Mesh" and the "Wheel_Low.obj" as the "Low Definition Mesh". Change the output file name to "Wheel_Occ.tga" and hit the "Generate Maps" button to generate the Occlusion map for the wheel.
Here is the resulting map for the Wheel.
Now open all the generated Occlusion maps in Photoshop, keep the wheel map as your base (1). Select the turret map and go to "Channels", it has RGB and an Alpha channels, the alpha channel has the selection of the turret UV's only (2). Now to select it press and hold "Ctrl" and click on the "Alpha1" channel, now press "Shift" and drag the selection onto the Wheel map (3).
Repeat the same procedure with the "Small Parts" map and the "Main Body" map. Here is our completed occlusion map, save this map as "Occlusion.TGA".
Now open all the generated Normal maps in Photoshop, and take the Wheel as your base layer.Go to your Occlusion map and select the "Small Parts" layer and by pressing and holding "Shift", drag it onto the "Small Parts" normal map, as this normal map doesn't have an alpha map of the selection. We use this occlusion map's layer as our selection, and then drag it onto the "Wheel" normal map.
Now that the "Small Parts" layer has been added, repeat the same process with the "Turret" and "Main Body" normal maps. Here is our completed Normal map, save it out as "Normal.TGA".
Now back in Maya, go to Window>Rendering Editors>Hypershade (1), and select the "Lambert1" material with the cursor and drag and drop it into the "Work area" (2) . Then select "File" from the side bar of Hypershade (3).
Now double click the "File Node" and then in the "Attribute Editor", click on the "Image Name" box (1) and browse to the "Occlusion.tga" (2-3).
Now using the middle mouse button, click and drag and drop the file to a new "Blinn1" material. An option box will open, select "Color" (1) and then select all the low poly meshes. In Hypershade, "Right click" and hold on the "Blinn1" material and select "Assign Material To Selection" (2). Now create a new "File Node" and browse to the "Normal.tga" image . If it's not visible in the node, "Right click" and hold on the node and select "Refresh Swatch" (3).
Now middle mouse click and drag and drop the "Normal Map File Node" onto the "Blinn1" material and select "Bump Map" in the list (1). Now select the newly created "Bump3d" node and in the "Attribute Editor", change the "Use As" option from "Bump" to "Tangent Space Normals" (2) . Finally go to the "Textures" tab in Hypershade and "right click" and hold, select Edit>Delete Unused Nodes (3).
And here is our Low poly model with the Normal and Occlusion maps applied.