Runtime MetaHuman Lip Sync
NEW: Enhanced Realistic Model for Premium Visual Quality

Real-time, offline, and cross-platform lip sync for MetaHuman and custom characters. Now featuring two quality models: Standard for broad compatibility and Realistic for premium MetaHuman experiences.

UE 5.0 - 5.6
Blueprints & C++
Windows, Android, Meta Quest
Multiple character systems

Two Quality Models to Suit Your Needs

Choose between our Standard model for broad compatibility and efficient performance, or the new Realistic model for enhanced visual fidelity specifically optimized for MetaHuman characters.

NEW REALISTIC MODEL

Enhanced Visual Quality

Realistic MetaHuman Lip Sync Example
Realistic Model - Premium Quality
  • MetaHuman-exclusive with advanced facial animation
  • Higher visual quality with more natural mouth movements
  • Ideal for cinematic experiences and close-up interactions

Best for: High-fidelity MetaHuman projects where visual quality is paramount

Standard Model

Standard MetaHuman Lip Sync Example
Standard Model - Broad Compatibility
  • Works with MetaHumans and all custom character types
  • Optimized for real-time performance on all platforms
  • Full compatibility with local TTS

Best for: Mobile, VR/AR applications and custom character systems

Custom Character Lip Sync Example 2
Standard Model with Custom Character
Custom Character Lip Sync Example 3
Standard Model with Different Viseme System

Real-Time Character Lip Sync Made Simple

Runtime MetaHuman Lip Sync provides a comprehensive system for dynamic lip sync animation, enabling characters to speak naturally in response to audio input from various sources. Despite its name, the plugin works with a wide range of characters beyond just MetaHumans.

Real-Time & Offline Processing

Generate lip sync animations in real-time from microphone input or pre-recorded audio

Universal Character Compatibility

Works with MetaHuman, Daz Genesis 8/9, Reallusion CC3/CC4, Mixamo, ReadyPlayerMe, and more

Multiple Animation Standards

Supports FACS-based systems, Apple ARKit, Preston Blair phoneme sets, and 3ds Max phoneme systems

Flexible Audio Input

Process audio from microphones, imported files, text-to-speech, or any PCM audio data

Latest Tutorial - High-Quality Lip Sync
High-Quality Lip Sync with AI TTS
Live Microphone Tutorial →

Latest Video Tutorials

Get up to speed quickly with our comprehensive video tutorials covering both the Standard and Realistic models, various audio sources, and integration techniques.

High-Quality Lip Sync with AI TTS

Learn to use the Realistic model with ElevenLabs & OpenAI for premium quality lip sync.

Realistic Model External TTS

High-Quality Live Microphone

Real-time lip sync from microphone input using the enhanced Realistic model.

Realistic Model Microphone

Standard Live Microphone

Efficient real-time lip sync using the Standard model for broad compatibility.

Standard Model Microphone

Local Text-to-Speech

Offline lip sync using the Standard model with local TTS for complete independence.

Standard Model Local TTS

Standard with External TTS

Combine Standard model efficiency with high-quality external TTS services.

Standard Model External TTS

Complete Setup Tutorial

Comprehensive guide covering plugin setup and configuration fundamentals.

Setup Guide Fundamentals

Key Features

Easy Blueprint Integration

Set up lip sync animation with just a few Blueprint nodes. Choose between Standard and Realistic generators based on your project needs.

Creating Runtime Viseme Generator
Standard Model Generator
Creating Realistic Lip Sync Generator
Realistic Model Generator

Animation Graph Setup

Integrate lip sync into your character's animation blueprint with dedicated blending nodes for each model.

Blend Runtime MetaHuman Lip Sync
Standard Model Blending
Blend Realistic MetaHuman Lip Sync
Realistic Model Blending

Customizable Parameters

Fine-tune the lip sync behavior with adjustable parameters for interpolation speed and reset timing for both models.

Interpolation Speed

Controls how quickly lip movements transition between visemes. Higher values result in more abrupt transitions.

Reset Time

The duration in seconds after which lip sync is reset when audio stops, preventing lingering mouth positions.

Technical Details

Model Comparison

Realistic Model

✓ MetaHuman-exclusive

✓ Enhanced visual quality

✓ More natural mouth movements

✓ External TTS compatible

Standard Model

✓ Universal character support

✓ Optimized performance

✓ Mobile/VR friendly

✓ Local TTS compatible

Viseme System

The plugin uses a standard set of visemes (visual phonemes) that are mapped to your character's morph targets:

Sil (Silence)
PP (p, b, m)
FF (f, v)
TH (th)
DD (t, d)
KK (k, g)
CH (ch, j)
SS (s, z)
NN (n)
RR (r)
AA (aa)
E (e)
IH (ih)
OH (oh)
OU (ou)

How It Works

1

Audio Input

Audio data is received as float PCM format with specified channels and sample rate

2

Viseme Generation

The plugin processes the audio to generate visemes using the selected model (Standard or Realistic)

3

Animation Driving

Visemes drive the lip sync animation using the character's pose asset or advanced facial controls

4

Real-Time Application

The animation is applied to the character in real-time with smooth transitions

Platform Support

Windows
Android
Meta Quest
More platforms coming soon

Android/Quest: Disable x86_64 architecture in project settings for compatibility

Requirements

Unreal Engine

UE 5.0 - 5.6

For MetaHuman Characters

MetaHuman plugin enabled in your project (UE 5.5 and earlier)

For Audio Capture

Runtime Audio Importer plugin

Powerful Integrations

Runtime MetaHuman Lip Sync works seamlessly with other plugins to create complete audio, speech, and animation solutions for your Unreal Engine projects.

Runtime Audio Importer

Import, stream, and capture audio at runtime to drive lip sync animations. Process audio from files, memory, or microphone input.

Learn more

Runtime Speech Recognizer

Add speech recognition to create interactive characters that respond to voice commands while animating with lip sync.

Learn more

Runtime Text To Speech

Generate realistic speech from text offline with 900+ voices and animate character lips in response to the synthesized audio.

Learn more

Runtime AI Chatbot Integrator

Create AI-powered talking characters that respond to user input with natural language and realistic lip sync. Perfect for the Realistic model.

Learn more

Complete Interactive Character Solution

Combine Runtime MetaHuman Lip Sync with our other plugins to create fully interactive characters that can listen, understand, speak, and animate naturally. From voice commands to AI-driven conversations with premium visual quality using the Realistic model, our plugin ecosystem provides everything you need for next-generation character interaction.

Custom Character Support

Despite its name, Runtime MetaHuman Lip Sync works with a wide range of characters beyond just MetaHumans. The Standard model provides flexible viseme mapping for various character systems.

Flexible Viseme Mapping

The Standard model supports mapping between different viseme standards, allowing you to use characters with various facial animation systems:

Apple ARKit

Map ARKit blendshapes to visemes

FACS-Based Systems

Map Action Units to visemes

Preston Blair System

Classic animation mouth shapes

3ds Max Phoneme System

Standard 3ds Max phonemes

Custom Character Example 2
Custom Character Example 3

Documentation & Support

Get started quickly with our detailed documentation covering both Standard and Realistic models. From basic setup to advanced character configuration, we're here to help you succeed.

Comprehensive Documentation

Step-by-step guides for both lip sync models, MetaHumans, and custom characters

Video Tutorials

Latest tutorials covering Realistic model and various integration methods

Discord Community

Get real-time help from developers and users

Custom Development

Contact [email protected] for tailored solutions

Demo Project Walkthrough

Add Realistic Lip Sync to Your Characters

Bring your MetaHuman and custom characters to life with real-time lip sync animation. Choose between our Standard model for broad compatibility or the new Realistic model for premium MetaHuman experiences.

NEW Realistic Model

Enhanced visual quality with more natural mouth movements specifically for MetaHuman characters

Standard Model

Universal compatibility with MetaHumans and custom characters, optimized for all platforms