Mesh artifacts, broken verts, overlapping faces and other forms of "Bad Geometry" are an unfortunate part of building 3D models, we've all experienced them at one time or another. These errors can appear for any number of reasons and they're not always easy to spot.
In today's tutorial, you'll get a detailed overview of Maya's Cleanup options and how they can be utilized to help get rid bad geometry that is sometimes hard to find and will cause issues when exporting meshes to real-time game engines.
Step 1
So we'll start with a simple polygon Sphere to test all the bad geometry possibilities which can be corrected using Cleanup.
Step 2
Now to start with the cleanup go to Mesh>Cleanup... and it will open a new window called "Cleanup Options", which has a variety of options which I'll explain as we move on.
Step 3
Now for our first option, we need to create a type of bad geometry on our sphere, as the default one doesn't have any kind of artifacts present. So for that simply select the "Split Polygon Tool" and add a few cuts on some faces to make them n-gons (faces surrounded by more than 4 edges.)
Step 4
Now open the Cleanup window again, this time using Hotbox. Press and hold Shift+Right-click and select "Cleanup". As you see these bad sections of geometry are easily visible on the sphere, but what if you had created something complex, like a "Roller Coaster" ride and some of these bad sections are left. How are you gonna find them? That's why "Cleanup" is there to help you find them.
Step 5
But first let's take a brief overview of the "Cleanup Effect" options. Under "Operation" if you select "Select Matching Polygons" and apply it, this will select only the affected area based on what you have highlighted.
If we choose "Cleanup matching polygons" it will automatically correct the affected areas without even showing that area. At first it might seem like a great idea to let Maya auto-correct all the artifacts, but Maya actually does a pretty poor job fixing your geometry. You should (almost) always set this option to “Select matching polygons”. So we can see what is wrong with our geometry and decide how we want to fix it.
Now there is a "Scope" option, in which if you select “Apply to selected objects” Maya will only perform the Cleanup on the objects that are currently selected in your scene. If you set it to “Apply to all polygonal objects”, Maya will perform a Cleanup on all the polygonal objects in your scene.
It's always better to work on each asset at a time as it will avoid any kind of confusion. It's also always good to keep the "keep construction history" option on as well, otherwise after using cleanup, Maya will automatically delete your asset's history.
Step 6
Now we'll move forward to the "Fix by Tessellation" tab, and select "Faces with more that 4 sides" (these are called "N-gons") which we've already created on the sphere. So after this, select the sphere and hit "Apply", and you'll notice it has already selected the affected faces.
Now I haven't given a brief of the "4-sided faces" option as we only use this one when we need our mesh to be in Triangles only. As of now most of the game engines accept quads, so we really don't need that option because we don't consider 4 sided faces bad geometry.
Step 7
Now there are two different ways to correct this issue. One we can simply use "Cleanup matching polygons" in the Cleanup options and hit Apply and it will connect the open vertices automatically. But that gives us some extra triangles that we don't want and may affect the shading on our asset.
Step 8
Or two, you could simply use the "Split Polygon tool" to connect the open edges where you want.
Step 9
Now for our next Fix "Concave faces", simply move the vertices and create few concave faces.
Step 10
Now under "Cleanup Options" select "Select Matching Polygons" and "Concave Faces" and hit "Apply" and it will select the affected faces only.
Step 11
Now we can simply correct it by moving the vertices to make the shape a convex face. or if our model doesn't allow us to do that, we can simply break the quad into 2 triangles using the "Split Polygon Tool".
Step 12
Now to show you guys what "Faces with hole" is, we have to create one. For that create a cylinder shape intersecting with the Sphere, and then Boolean it with the sphere using "Intersection".
Step 13
Now Delete the remaining faces of the Boolean intersection. This last remaining face with the hole in it, in which all the vertices are open (not connected), is called a Face with a Hole. These will give you terrible results when smoothed out and should always be taken care of.
Step 14
Now using the Cleanup Settings, select "Select Matching Polygons" under the Cleanup Effect options and set the "Fix by Tessellation" option to "Faces with Holes", you'll get the affected face highlighted.
And again if you set the Cleanup Effect to "Cleanup matching polygons" it will connect all the open vertices automatically, but you also do the same thing using the "Split Polygon Tool" if you want to control the smoothing group manually.
Step 15
Now for our next artifact "Non planar Faces" just select any one vertex, and move it inwards, and this has created 4 Non-Planar faces surrounding that vertex. Non-planar faces are polygons with at least four vertices that do not lie on the same plane. In other words the vertices of a Non-planar face no longer create a flat surface in 3d space. As we can see in the second image below.
Step 16
So applying Cleanup with "Non Planar faces" it will highlight only the Non Planar faces, and automatic Cleanup will connect the diagonal vertices so that the quads have now changed into triangles. A triangle is always a Planar surface in whichever form you place it or move the vertices. Manually you can also move the vertices and change the face from Non-Planar to planar, obviously only if your model allows it.
Step 17
Now we'll move on to "Lamina faces", which are actually 2 faces which share the same vertices and sit on top of each other with merged vertices. These kinds of faces are very hard to find as they blend in pretty well, you can only spot it if you find unusual smoothing on any part of your mesh, where smoothing groups wont work, and applying subdivision also will create some artifacts in this area. Now to have this shown as an example, duplicate an existing face over another and merge all 4 vertices.
Step 18
So applying cleanup will highlight that one extra face over the other one. And if auto cleanup is applied, it will directly delete this extra top face. Auto Cleanup works best for the Lamina faces issue.
Step 19
Now we'll have a look at the last case of bad geometry which consists of 3 types of them. First "Nonmanifold Geometry" is when 3 or more faces share the same edge on an object, for this simply select any edge and make an extrusion out of it. There are now 3 faces all extruding from one single edge.
Step 20
Now to clean this up if we use "Select Matching Polygons" it will select the two vertices through which these faces are connected. And if you do Auto Cleanup it will simply split these two vertices and hence split all three faces altogether.
Step 21
You can simply check this auto Cleanup by selecting the extruded face and moving it in space. An alternative and the best way to get rid of this kind of Nonmanifold geometry is to delete it directly, or simply separate that particular face.
Step 22
The second type of Nonmanifold Geometry is where two or more faces share the same vertex, yet they share no edge. Just like in the previous case, if we collapse that same edge then we have a face that is connected to numerous faces with just one vertex on the same object.
Now if we do an Auto Cleanup it will again split that vertex and all the connected faces will separate from that vertex, but yet again the best solution in this case is to Delete off that extra polygon face or simply separate it.
Step 23
Now the third and last type of Nonmanifold geometry, is when two or more adjacent faces have opposite normal directions. To test that, simply select a few faces and then go to Normals>Reverse, and you'll notice the hidden faces on the object. Generally these kinds of faces are pretty visible on the object, but on some bigger assets they're really hard to find and that's when Cleanup is there to help.
Step 24
In this particular case Auto Cleanup is also the best thing to do as it will automatically select the minority faces which have opposite normals and apply the Reverse normals to them.
Step 25
Now there are three more kinds of bad geometry according to Cleanup, but in reality they're not harmful or not acceptable i.e. "Edges with Zero Length", well technically it's not zero but it's close to "0" like "0.01 or 0.001". Selecting this option will highlight the edge that falls into this criteria, and Auto cleanup will simply collapse this edge.
Step 26
Same goes with the "Faces with Zero Geometry Area", it doesn't hurt having them if required.
Step 27
And The last one in Cleanup is "Geometry with Zero Map Area" that means the UV space of any particular face is near to zero, and that won't happen if you have done proper UV mapping. But just in case you need to find out if you've missed any faces on any asset, this can be a very helpful way to find out.
Conclusion
I hope this will give you a very clear picture of what kinds of bad geometry there can be in an asset and how we can rectify them, either manually or using the cleanup command of Maya.
