The story behind our Avatars

In order to create our avatars, several steps have been conducted:

Step 1: Photo Scanning

At first, we perform a photo scan of the client that is to be digitized. We capture all kinds of facial expressions of this client in order to make it possible to animate the digital imprint of the client later on.

Programs: Photogramrammetry Rig & Agisoft

photographs being taken from young guy
3D model created using the photogrammetry proces

Step 2: 3D Models

After taking several photographs, we use Photogrammetry to convert them into 3D models. During this process, we use approximately 15 million data triangles to get the best result.

Program used: Agisoft

Step 3: Cleaning the Base

In this step, the 3D model created is morphed into the target face. The base template consists of a clean topology, UVs,  and mouth cavity. It is called the base since it will be used for the creation of expressions and textures in the following steps.

Program used: Autodesk Maya

3d model morphed into target face
created skin texture for skin color and details such as pores and sweat

Step 4: Textures

Textures are applied to the surface of the 3D model to give it a more realistic appearance. This can be done by selecting the appropriate texture images and applying them to the model. Examples of textures are skin color, and details such as pores, wrinkles, and sweat.

Programs used: Substance Painter, Adobe Photoshop, and Marmoset.

Step 5: Expressions

During the fifth step, we add expressions to the 3D model to make it appear more lifelike. This can be done by selecting the appropriate facial expressions and applying them to the model. The expressions need to be added to all 60 poses, allowing for us to animate the face, expressions are created from the photosets. Furthermore, a skeleton is added, so we are able to move the face. The final expressions are connected to the base, which was created in step 3.

Program used: Autodesk Maya

Expressions and animations being added to base
assets such as eyes, teeth, and hair are added to model

Step 6: Assets

Any additional assets that may be needed to complete the 3D model or environment are gathered. This can include things like additional 3D models, textures, and other elements (e.g., hair, eyes, and teeth). We are working on Deep Learning algorithms to automatically generate hair and teeth.

Programs used: Python, Cuda, and Autodesk Maya

Step 7: Unreal 4

Finally, we use Unreal 4 to bring the 3D model and environment to life. This can be done by importing the 3D model and assets into the game engine and using its tools and features to create a realistic and engaging experience.

Program used: Unreal Engine 4

different skin complexities