Calculating Tangents for your mesh

Recently I had the chance to improve the visuals of a 3D engine by fixing the previous algorithm of normal mapping. One of the problems was in the calculation of the tangents required per every vertex.

You can find more information about Normal mapping here.

For calculating the tangents of a mesh let's assume that we have a list of vertex indices, and a list of vertices. Something like this:

std::vector<Vec3i> indices;
std::vector<Vertex> vertices;

Also, our vertex object is going to look like this:

class Vertex final {
    Point3f position;
    Vec2f uv;
    Vec3f normal;
    Vec4f tangent;
};

Calculating tangents

With that in mind, the function to compute tangents will look like this:

void computeTangents(
    const std::vector<Vec3i>& indices,
    std::vector<Vertex>& vertices
) {
    size_t vtxCount = vertices.size();
    std::vector<Vec3f> tanA(vtxCount, Vec3f(0,0,0));
    std::vector<Vec3f> tanB(vtxCount, Vec3f(0,0,0));

    // (1)
    size_t indexCount = indices.size();
    for (size_t i = 0; i < triangleCount; ++i) {
        size_t i0 = meshIndices[i].x;
        size_t i1 = meshIndices[i].y;
        size_t i2 = meshIndices[i].z;

        Point3f pos0 = vertices[i0].position;
        Point3f pos1 = vertices[i1].position;
        Point3f pos2 = vertices[i2].position;

        Vec2f tex0 = vertices[i0].uv;
        Vec2f tex1 = vertices[i1].uv;
        Vec2f tex2 = vertices[i2].uv;

        Vec3f edge1 = pos1 - pos0;
        Vec3f edge2 = pos2 - pos0;

        Vec2f uv1 = tex1 - tex0;
        Vec2f uv2 = tex2 - tex0;

        float r = 1.0f / (uv1.x * uv2.y - uv1.y * uv2.x);

        Vec3f tangent(
            ((edge1.x * uv2.y) - (edge2.x * uv1.y)) * r,
            ((edge1.y * uv2.y) - (edge2.y * uv1.y)) * r,
            ((edge1.z * uv2.y) - (edge2.z * uv1.y)) * r
        );

        Vec3f bitangent(
            ((edge1.x * uv2.x) - (edge2.x * uv1.x)) * r,
            ((edge1.y * uv2.x) - (edge2.y * uv1.x)) * r,
            ((edge1.z * uv2.x) - (edge2.z * uv1.x)) * r
        );

        tanA[i0] += tangent;
        tanA[i1] += tangent;
        tanA[i2] += tangent;

        tanB[i0] += bitangent;
        tanB[i1] += bitangent;
        tanB[i2] += bitangent;
    }

    // (2)
    for (size_t i = 0; i < vtxCount; i++) {
        Vec3f n = vertices[i].normal;
        Vec3f t0 = tanA[i];
        Vec3f t1 = tanB[i];

        Vec3f t = t0 - (n * dot(n, t0));
        t = normalize(t);

        Vec3f c = cross(c, n, t0);
        float w = (dot(c, t1) < 0) ? -1.0f : 1.0f;
        vertices[i].tangent = Vec4f(t.x, t.y, t.z, w));
    }
}

Conclusion

This algorithm has two parts, the first part (1) will compute 2 vectors perpendicular to the normal for that vertex. In this part, the uv coordinates play an essential role for the direction of these 2 tangent vectors: tangent and bitangent.

The second part (2) will compute the final tangent vector and the essential w coordinate, which will tell us if the tangent needs to be inverted. (As an example, if we have a symmetrical mesh with on half of the uv mapping inverted, we will need to invert tangents on the vertices with inverted uv's).

Please, keep in mind that this is only one part of the normal mapping visual effect. If you want the full normal mapping effect, you will need to do extra work inside your shader after the execution of this algorithm:

• Feed the indices and vertices vectors to your shader.
• Apply the skinning transform to every vertex normal and tangent.
• Compute the binormal with: binormal = cross(normal, tangent);.
• Construct the tangent space matrix with: [normal, tangent, binormal].
• Unpack the texture normal with: nrm = (tex(uv) - (0.5, 0.5)) * 2.0.
• Multiply that unpacked texture normal with the tangent space matrix.

Looks like something scary but if it is done step by step, it can be achieved in a reasonable amount of time :D.

Update:

We discovered a bug where the tangent vector was always facing the oposite direction. We fixed it by switching signs while computing the tangent and bitangent vectors.