Textures and Materials

Materials

Animaze items such as avatars, props, environments, are rendered using a complex shader (material), called The Animaze Shader. It is built modular, such that it can be customized to the needs of the rendered item. Every mesh has its own material that holds the configuration for the Animaze Shader and references to all textures bound to the mesh. Currently, a mesh Material can configure the Animaze Shader and it is the only way one can change the way an item is rendered.

Textures and materials for 3D avatars

The Animaze Shader lighting model is Blinn-Phong and has custom methods to compute specular, environment reflections, refractions etc., so all texture maps will have to be adjusted having this shading model in mind. The Animaze Shader does not follow the Physically-Based Shading equations currently.

For a better performance, you can group all the components that have the same material. In the picture above, grouped components share the same color.

Feature breakdown table:

Feature breakdown table for 3D avatars

Animaze uses a component system for the  Animaze Shader, each  component adds a feature to the item shading, e.g. Environment Mapping component adds the skybox reflection to the surface rendering. Some of the material components need extra info in the form of float values (e.g. fur length for the Fur component), geometric info (e.g. normals) or texture maps (RGB/RGBA/Grayscale textures).

Furthermore, texture maps encode per texel information in their RGB (and Alpha in some cases) channels that require careful tuning and are compliant to specific Animaze shading rules. Texture channel rules are described for each component in particular, although some of the texture maps are shared among them, the rules apply the same.

Some components are solely unique and they exclude each other out when one tries to add them together. For example, the fur and skin dot mapping components cannot coexist on a single mesh material. The reason behind this, is that the Animaze Shader model is built this way to reduce the performance expense of the models.

 

Material Components can be customized by the data fed to the Animaze Editor (e.g. the geometry, the textures) and can also be customized within the Editor's Material Editor (e.g. changing the diffuse texture, setting a tint color etc.). All components that can be changed only from the Animaze Editor are marked accordingly by (*).

Render Queues

Animaze materials can control the rendered surfaces render queues. The basic concept is that we need to have a distinction between opaque surfaces and the transparent ones, as the transparent ones need to modulate the color of the previous rendered surfaces (the opaque ones). Having render queues brings more control to how the meshes are displayed.

Queues are sorted as follows:

  • Environment – drawn first. Commonly used for skybox.
    • Opaque – opaque meshes are drawn front to back so we can leverage depth testing and have less overdraw.
    • Fur – fur meshes are very peculiar, they are instanced meshes, where the first layer is opaque and the rest are transparent or blended. Meshes rendered through this queue are sorted back to front.
    • Alpha Test – rendered back to front, pixels under a certain alpha value are discarded. This is a more efficient way of adding transparent surfaces.
  • Blended – transparent meshes, rendered back to front.
  • Overlay – reserved for UI

 

The current Animaze Shader supports the following Material Components: 

Diffuse

Applies diffuse color to the rendered surface based on a provided texture.

Requirements:

  • diffuse texture – RGBA texture. The diffuse texture holds diffuse color info of the rendered surface stored in RGB channels.
  • Alpha channel is used to specify transparency
    • 0 (black) areas for full transparency
    • towards 255 (white) surfaces get increasingly opaque.

Tangent space normal mapping

Applies normal mapping to the rendered surface based on the provided texture. For the  best results, we recommend baking the normal map in a software that is compatible with Autodesk (i.e. Mudbox, Maya, Substance).

Requirements:

  • normal map texture – The texture holds normal map info stored in Tangent (or texture) space.
  • geometry exported with Normals info

Learn how to use Animaze by FacerigLearn how to use Animaze by Facerig

We recommend baking the normal map after exporting the .fbx file from the software you are working with, so that the UV maps do not get overwritten.

The details present in normal maps are static: for now, they cannot be dynamically altered at run- time. If you want dynamic things like expression wrinkles that appear/disappear based on expression changes, don’t put them in the tangent space normal maps.


Environment Mapping

The AnimazeShader maps an environment cube texture (the Skybox) to the mesh surface. The Environment texture is provided by Animaze System, specifically the Skybox item cube texture.

Requirements:

  • Geometry exported with Normals info
  • (optional) affected by the Green channel in the Specular Map texture.
  • (*) Environment Multiplier – float value. A multiplier that can boost the intensity of the environment reflected on the surface.

Learn how to use Animaze by Facerig

Environment Mapping Tint

Env Map Tint component color tints the specular and environment reflections. Under the hood, the Animaze Shader uses a color map (RGB) to modulate against the current specular and environment reflections, with the intent to emulate how light behaves on colored metalic surfaces, such as gold.

Requirements:

  • Environment Map
  • Color texture - RGB 

Additionally, the result can also be influenced by tweaking the values of Environment Mapping and Lighting components.

ProTip: Desaturating a bit the color texture will have a more natural effect as a metal surface. 

Skinning

Animaze does all the skeletal animation skinning on the GPU. This component informs the Animaze system to add skeletal animation to the item in question.

Requirements:

  • The geometry needs to hold skinning information. Vertex weights are capped to 4 influences.

If there are more than 4, then the extra influences will be detected at import in Animaze Editor and will redistribute them automatically. This can lead to an unwanted edgy result of the geometry, so we recommend to limit the influences to 4 per vertex.

Learn how to use Animaze by Facerig

Lighting

Adds lighting to the rendered model. Removing this component results in a pre-lit or unlit model that displays the diffuse texture (if the Diffuse component is added) without any shading changes.

The AnimazeShader supports maximum 4 lights of any type: directional or omni.

Requirements:

  • Geometry exported with Normals info
  • Geometry exported with Tangent or Binormal info
  • Specular Map Texture – RGBA texture. The specular map uses two channels to customize the specular behavior on the surfaces.
  • Red channel holds the specular exponent info (also known as shininess or glossiness), so it controls how blurry or sharp the specular shine is rendered.
    • higher values make the shine small and sharp
    • lower values make the shine large and blurry.
  • Green channel holds the specular intensity info
    • 0 means no specular shine
    • 255 is max shine intensity.
  • · Blue channel holds the emissive intensity information
    • lower the values, less emissive and more influenced by lighting
    • higher values, more emissive.

Learn how to use Animaze by FacerigLearn how to use Animaze by Facerig

                               with light                                                             without light


Specular Mapping

The specular mapping is a component that can fine control the specular reflection on a rendered surface.

Requirements:

  • Specular Map texture – RGBA texture. The specular map uses two channels to customize the specular behavior on the surfaces:

Learn how to use Animaze by Facerig

  • Red channel holds the specular exponent info (also known as shininess or glossiness), so it controls how blurry or sharp the specular shine is rendered:
    • higher values make the shine small and sharp
    • lower values make the shine large and blurry
    • Green channel holds the specular intensity info:
      • 0 means no specular shine
      • 255 is max shine intensity

Learn how to use Animaze by Facerig

Emissive – as part of Specular component

Adds the emissive component to the surface shading. The Emissive just boosts up the diffuse color based on the specular map blue channel.

Requirements:

  • Specular Map Texture – RGBA texture.
  • Blue channel holds the emissive intensity information:
    • lower the values – less emissive and more influenced by lighting
    • higher values – more emissive.

 

Bekmann Specular

The Animaze shader provides two types of specular highlight models: the classic Blinn-Phong distribution and the Beckmann distribution. The Beckmann specular is used in combination with Subsurface Scattering and Translucency components, to better approximate how light refracts, scatters and reflects on the skin-like surfaces, but it is not limited to that.

Requirements:

  • Roughness - float value - a parameter between 0 and 1 measuring the surface roughness. Small values are shinier, larger values are rougher.

Learn how to use Animaze by FacerigLearn how to use Animaze by Facerig

                 Scatter + Translucency                                                    Beckmann specular


Translucency

The translucency component simulates the light going through the shaded object. It is used in the shading of skin or skin-like materials in combination with Subsurface Scattering and Bekman specular, to better approximate how light refracts, scatters and reflects on the surface.

Requirements:

  • A map obtained either through bake process or manually painted.
    • AO (ambient occlusion map) with inverted normals through baking (in order to obtain the thickness of the model) or manually painted
    • white areas > the light passes through easier
    • dark areas > the light passes through harder

Learn how to use Animaze by Facerig

Tint Color

Material component that tints the rendered surface with a certain color.

Requirements:

  • (*) Tint Color RGBA:
    • RGB tints the surface
    • Alpha makes the surface transparent.

Learn how to use Animaze by Facerig

                              with Tint                                                                  without Tint


Alpha Test

This component discards any pixels under a certain alpha value. Alpha test is done just after computing the diffuse color. It sets the transparency of objects.

In the example below, Fluffo’s eyelashes (left picture) have 0 transparency, you can see the result of the alpha test in the right picture.

Requirements:

  • (*) Alpha Test reference value – float value

Learn how to use Animaze by FacerigLearn how to use Animaze by Facerig


Fur

The Fur component renders the model in layers, creating the effect of a furry surface; it can be attained for short hair fur effects only.

The component makes use of the normals and vertex color geometry info to compute a general fur direction, where the Red and Green channels give the direction in the Tangent and Binormal space. Furthermore, a surface can have areas without any fur or with shorter fur. This aspect can be controlled using a fur texture mask. The quality of the fur rendering  also depends on the values of fur length, number of fur layers and  a global fur direction multiplier.

Requirements:

  • Geometry (meshes) exported with Normals info
  • Geometry exported with Vertex Color info:
    • Red and Green channels represent the direction in Tangent and Binormal plane.
    • Alpha Channel is the fur length factor
  • Diffuse texture – RGBA texture
  • Fur Mask:
    • Red channel controls the mix between normal specular and aniso specular
      • 0 is normal
      • 255 is specular
      • Green channel is an additional layer alpha control. Alpha is the length, all lower value pixels are discarded, everything above will render at a calculated opacity, the green channel adds an additional transparency control.
    • Blue channel is the fur aniso specular intensity

Learn how to use Animaze by Facerig

  • Alpha Channel is used as fur length, controlling a smoother blend between fur areas (values closer to 255) and furless areas (values closer to 0) of the rendered surface.

Learn how to use Animaze by Facerig

  • (*) Fur Direction – float value
  • (*) Fur Length – float value. Value expressed in model space scale, controls the maximum fur length.

  • (*) Mesh Layers – numeric value. The number of mesh layers determines the fuzziness of the fur rendering, a high number will make the surface seem very dense, while a low number will make the avatar look like it extrudes layers.

 

Subsurface scattering – SSS

Subsurface scattering is necessary for the rendering of materials such as skin and flesh.  If  subsurface scattering is not used, the material may look unnatural, like plastic or metal. This component works by simulating how light penetrates a translucent surface like a grape for instance, and is absorbed and scattered and exits the surface at a different location. When light rays are traveling and hit a surface there are many different things that occur all at once. Some of the light is going to be reflected off, giving specular light.

However, with certain materials that have a level of translucency some of the light rays are actually going to be absorbed into the surface. Once inside, the light rays will scatter all around and exit the surface at different locations, providing subsurface scattering.

The Animaze Shader is capable of emulating skin and flesh, for a more natural look of the model.

Requirements

  • scatter texture RGB:
    • Red channel – range of sub surface scattering
      • 0 value is for no subsurface scattering
      • 255 max subsurface scattering. Should fade towards 0 on mapping seams and  on solid details such as piercings/jewelry.
    • Green channel – direct light attenuation
      • 0 means no direct light (only ambient and radiosity)
      • 255 full direct light intensity.
    • Blue channel
      • 0 means amount of diffuse light that scatters in the subsurface
      • 255 max value means that the amount of light bounces right off.

A good default value would be 0.9 (229 229 229) for skin areas. Should fade towards 0 on mapping seams and on solid details such as piercings/jewelry. The UV maps must be unique within 0-1 range, and the materials mustn’t overlap.

 

Skin dot mapping

Dot mapping (or dot gradient-mapping) is used as a fast skin shading approximation solution, that is cheap resource-wise

Requirements:

  • dot gradient map texture which is used as a lookup table and describes how the  light  behaves on a curved surface (it tints the diffuse component).
  • a skin texture RGBA (GBA not used) >> where Red channel is used to control the V axis in texture space of the dot map texture.
  • dot map texture is provided with the Animaze Editor

The skin map represents a fake subsurface scattering map, which the whiter it is, the smoother the model’s skin will look. If this map is missing, the shader will consider the upper half of the dot map.

Clamping the texture will prevent wrapping artifacts when mapping a texture onto an object and you don't want the texture to tile. UV coordinates will be clamped to the range 0-1. When UVs are larger than 1 or smaller than 0, the last pixel at the border will be used.



Animated texture

The animated texture component takes the texture and runs a frame by frame animation based on a tiling method. 

  1. Animated Diffuse Maps

Requirements:

  • diffuse texture RGBA
  • (*) number of rows and columns in the diffuse texture
  • (*) framerate

Animaze supports more uses for animated textures created using Live2D Cubism technology for hybrid avatars. 

In addition to Animated Diffuse Maps (that were available before) you can now also have:

        2. Animated Normal Maps

        3. Animated Emissive (Specular) Maps

The Live2D (Cubism) animated textures details are alpha-blended on top of the existing static textures, and the result of this blend is what is used on the model.  Having a correct (static) texture as a base is still important!

The animated textures, if used for Normal Maps, are used for dynamically enhancing the details on the avatar's normals. It’s a great way to accentuate the avatar’s expressivity without increasing the polycount.

These animated textures for normal maps built with Live2D (Cubism) assets work in synergy with the tracked animation inputs of the avatar. It's important that the Live2D (Cubism) assets that are used for animated textures have their parameters named correctly for the behavior you want to obtain. For example, a general use-case is the BrowUP/Down animations for 3D, and for 2D ParamBrowLY, ParamBrowRY animations - surprise & frowning.

  • if ParamBrowLY si ParamBrowRY are 0, then the opacity will be set to 0 >> default position
  • if ParamBrowLY si ParamBrowRY are -1 or 1, then enables frown (-1) & eyebrow up (1)

Animated Alpha Noise

The Animated Alpha Noise component adds an alpha animation for some effects. Requirements:

  • needs a RGBA noise texture
  • a set of noise parameters set from Animaze Editor.

 


Mouth AO

The AnimazeShader has a simple way to deal with the ambient occlusion of the mouth in all avatars.

Based on some animation inputs, when closing the mouth, the light that is affecting the mouth interior is attenuated based on a monochrome occlusion texture map, and the effect is reduced when opening the mouth.

The mouth interior usually has a different material than the one outside of the mouth, therefore, at import, the Animaze system will identify the two materials and will break the mesh into two separate ones, each assigned with its own texture map.

The Fake Mouth AO component applies solely to avatars, because it needs mouth animation inputs. Adding it to props would only lower the performance (due to loading a texture and processing useless instructions) without any graphical benefit.

Requirements:

  • Mouth AO Texture – RGB texture.
    • the texture holds the minimum brightness value in the red channel, for when the avatar’s mouth is closed.
    • the green channel is reserved for mouth fully opened.
      • blue channel it’s used for the tongue. The tongue can dynamically travel from inside the mouth to outside the mouth so we need to mark what parts of the tongue need to be occluded less when the tongue sticks out.

Cel-shading

Cel-shading or “toon shading” is an intentional non-photorealistic rendering which is generally used to make 3d graphic content appear with flat colors. 

A common theme of cel-shading which gives it that effective cartoon animation look is the use of bold outlines.

Learn how to use Animaze by Facerig

Cel shading is supported by two separate material components, the main difference being the amount of fine-tuning and detail control needed for a particular look:

  • Dual-tone cel-shading that has a set of parameters to control Diffuse intensity, Specular highlights, and Edges.
  • Multi-tone cel-shading that uses textures to control Diffuse intensity, Specular Highlights, and Edges in far more fine detail.

 

Edge outline

The Animaze shader implements two outline methods that can be used simultaneously:

  1. Dot based edge detection - the edges of the same surface are darkened. This method uses the dot product between the surface normal and light direction and decides based on a threshold if a pixel is on the outline of the rendered mesh or not.
  2. Expanded shell / Outline shell - the darkened outline is rendered on the exterior of the surface

 

Currently, both outline methods render a darkened outline, there is no support for tinted outlines.

Cel shading (no textures) - Dual Tone Cel Shading

  • it can be added in Animaze Editor as mesh component

Learn how to use Animaze by Facerig

 

Diffuse Threshold - light intensity threshold that decides between applying 1st and 2nd tone diffuse intensities

Learn how to use Animaze by Facerig

Diffuse minimum value - 1st tone diffuse intensity value

Learn how to use Animaze by Facerig

Diffuse maximum value    - 2nd tone diffuse intensity value

Learn how to use Animaze by Facerig

Specular Threshold - light intensity threshold that decides between applying 1st and 2nd tone specular intensities

Specular minimum value - 1st tone specular intensity value

Learn how to use Animaze by Facerig

Specular maximum value - 2nd tone specular intensity value

Learn how to use Animaze by Facerig

Edge Threshold - threshold used by the dot-based edge detection method

Learn how to use Animaze by Facerig

Edge minimum value - the intensity of the surface that is not the edge (factorizes the diffuse and specular tones)

 Learn how to use Animaze by Facerig

Edge outline width -  used by the expanded shell/outline shell method. Controls the size of the outline. Currently, the edge is always black.

Learn how to use Animaze by Facerig

 


Cel shading w/ texture - Multi-tone cel-shading

Multi-tone cel-shading enables a finer detail of the rendered surface through use of various textures, that control the intensities of the diffuse, specular tone, and dot-based edge detection.

Diffuse Tones Texture

  • The grayscale texture that maps light intensity (u coordinate) to diffuse tone intensity (texel color).

Texture used

 Learn how to use Animaze by Facerig

Texture used

 Learn how to use Animaze by Facerig

Texture used

 Learn how to use Animaze by Facerig

  • Animaze shader samples a single row, so the size of the texture can be width x 1 pixel
  • Supports fine details by choosing to build the texture with steps or gradients

Texture used

 Learn how to use Animaze by Facerig

Texture used

 Learn how to use Animaze by Facerig

For maximum cel-shading effect, make sure to use point filtering on texture (by default Animaze imports textures with linear filtering).

Learn how to use Animaze by Facerig


Specular Tones Texture

The grayscale texture that maps light intensity (u coordinate) to specular highlight tone intensity (texel color).

  • Animaze shader samples a single row, so the size of the texture can be width x 1 pixel
  • Supports fine details by choosing to build the texture with steps or gradients

Texture used

 Learn how to use Animaze by Facerig

Texture used

 Learn how to use Animaze by Facerig

For maximum cel-shading effect, make sure to use point filtering on texture (by default Animaze imports textures with linear filtering).

Learn how to use Animaze by Facerig

 

Edge Tones Texture - used for the dot-based edge detection

The grayscale texture that maps the dot product between surface normal and light direction (U coordinate) to edge tone intensity (texel color).

Texture used

 

  • Animaze shader samples a single row, so the size of the texture can be width x 1 pixel
  • Supports fine details by choosing to build the texture with steps or gradients

Texture used

 

Texture used

 

For maximum cel-shading effect make sure to use point filtering on texture (by default Animaze imports textures with linear filtering). 

Edge outline width - used by the expanded shell/outline shell method. Controls the size of the outline. Currently, the edge is always black.

Shadows

Animaze rendered surface can cast and receive shadows, both processes being independent from each other.

Casting shadows makes the rendered surface be rendered through a shadow pass, building up a shadow map. Receiving shadows makes the Animaze Shader map the already rendered shadow map.

All three numerical values are used to get rid of the shadow mapping artifacts.

Slope bias factor - scale factor that is used to create a variable depth offset for each polygon when rendering the shadow map

Slope bias units - constant depth offset when rendering the shadow map;

Depth bias - screen space value that is used to offset the object depth before comparing it to the shadow map value.

Additional components can be added to a selected mesh within the Animaze Editor:

Specular Normal Mapping – an extra normal map to be used instead of the regular one when computing reflection and specular, usually used for eyes meshes.

Static Color – a color component which overrides any other color texture on that mesh; usually useful for particle systems.

Texture Alterator – basically a gradient color mask used for dilation of the pupil which can be fed by animation data. In order to attain this, you must have a unique mapping for the eye, placed in its center. The mask with full intensity (white) must have the size of the pupil, while the gradient from white to black must have the size of the iris.

©2020, Animaze, Inc. and the Animaze logo are registered and/or unregistered trademarks of Holotech Studios Inc.