MetaHuman Engine

Give AI a real-time interactive digital body

Browser-native 3D digital human engine with voice and dialogue capabilities.
Zero-config · Offline-ready · Open-source MIT

Quick Start · Features · Performance · Architecture · Documentation · Changelog · 中文

---

✨ Demo

🚀 Try it live →

Experience a fully interactive 3D digital human right in your browser. No installation or API keys required!

---

📸 Preview

3D Avatar with emotion-driven expressions and real-time dialogue

---

🚀 Quick Start

Prerequisites

• Node.js ≥ 22
• npm ≥ 10

Installation

# Clone and install
git clone https://github.com/LessUp/meta-human.git
cd meta-human
npm install

# Start development server
npm run dev

Open http://localhost:5173 — your 3D avatar is ready!

💡 No API key required. The engine automatically falls back to local mock mode for out-of-the-box demos.

🐍 Optional Backend: A FastAPI backend example is available in examples/backend-python/ for custom integrations.

---

🎯 Features

🎭 3D Avatar Engine

Feature Description GLB/GLTF Support Load custom models or use built-in procedural avatar Emotion-Driven Happy, surprised, sad, angry moods map to expressions Skeletal Animation Wave, greet, nod, dance triggered by dialogue

import { digitalHumanEngine } from './core/avatar'; digitalHumanEngine.perform({ emotion: 'happy', expression: 'smile', animation: 'wave', }); Note: The project uses Vite path aliases. See Path Aliases for configuration.

🗣️ Voice Interaction

Feature	Description
TTS	Browser-native SpeechSynthesis voice output
ASR	Browser-native SpeechRecognition voice input
Smart Muting	Auto-pause TTS when user speaks
Voice Detection	Visual feedback during recording

import { ttsService, asrService } from './core/audio';

await ttsService.speak('Hello! How can I help?');

asrService.start({
  onResult: (text) => dialogueService.send(text),
});

🧠 Intelligent Dialogue

Feature	Description
Multi-Modal Response	Returns { replyText, emotion, action }
Streaming	Real-time token-by-token via SSE
Graceful Degradation	Falls back to local mock when API unavailable
Session Management	Persistent conversation context

import { dialogueService } from './core/dialogue';

const response = await dialogueService.send({
  text: 'Tell me a joke',
  sessionId: 'user-123',
});
// → { replyText: '...', emotion: 'happy', action: 'laugh' }

---

⚡ Performance

Verifiable metrics from npm run build and npm run test:run:

Metric	Value
Landing bundle	~160 KB gzipped (lazy 3D excluded)
Full bundle	~430 KB gzipped (includes Three.js)
Unit tests	189 passing (Vitest)
Type check	tsc --noEmit strict mode

Runtime FPS and memory depend on the user's device and GPU; no synthetic benchmarks are shipped.

---

🏗️ Architecture

┌─────────────────────────────────────────────────────────────────┐
│                          UI Layer                                │
│   ChatDock · TopHUD · ControlPanel · SettingsDrawer             │
└─────────────────────────────────────────────────────────────────┘
                                │
┌─────────────────────────────────────────────────────────────────┐
│                       Core Engine Layer                          │
│   Avatar · Dialogue · Audio                                     │
└─────────────────────────────────────────────────────────────────┘
                                │
┌─────────────────────────────────────────────────────────────────┐
│                       State Layer                                │
│   chatSessionStore · systemStore · digitalHumanStore            │
└─────────────────────────────────────────────────────────────────┘
                                │
┌─────────────────────────────────────────────────────────────────┐
│                      External Services                           │
│   Three.js · Web Speech API · OpenAI API                        │
└─────────────────────────────────────────────────────────────────┘

State Management

Three focused domains minimize re-renders:

Store	Responsibility
chatSessionStore	Message history, session lifecycle
systemStore	Connection status, errors
digitalHumanStore	Avatar runtime state (expression, animation, audio)

📖 Architecture Docs →

---

📁 Project Structure

src/
├── core/                          # Engine modules
│   ├── avatar/                    # 3D rendering & animation
│   │   ├── DigitalHumanEngine.ts  # Unified driver
│   │   └── constants.ts           # Expressions, animations
│   ├── audio/                     # TTS & ASR services
│   ├── dialogue/                  # Chat transport & orchestration
│   │   ├── dialogueService.ts     # API client + HTTP/SSE transport
│   │   ├── dialogueOrchestrator.ts
│   │   ├── dialogueRequestMeta.ts
│   │   └── characterPresets.ts
│   └── createServices.ts          # Service container
├── components/                    # React components
│   ├── viewer/                    # 3D viewport (DigitalHumanViewer.tsx)
│   ├── ChatDock.tsx               # Chat interface
│   ├── TopHUD.tsx                 # Status bar
│   ├── ControlPanel.tsx           # Quick controls
│   ├── VoiceInteractionPanel.tsx
│   └── ui/                        # Shared primitives
├── store/                         # Zustand stores
│   ├── chatSessionStore.ts
│   ├── systemStore.ts
│   └── digitalHumanStore.ts
├── hooks/                         # Custom hooks
├── pages/                         # Route pages
└── lib/                           # Utilities

Path Aliases

This project uses Vite path aliases configured in vite.config.ts and tsconfig.json:

Alias	Maps to
@/*	src/*

---

🌐 Deployment

GitHub Pages (Frontend)

npm run build:pages

1. Set VITE_API_BASE_URL in GitHub Repository Variables
2. Push to master — CI auto-deploys
3. Live at: https://lessup.github.io/meta-human/

📖 Deployment Guide →

---

🛠️ Scripts

Development

npm run dev              # Start dev server (port 5173)
npm run preview          # Preview production build

Build

npm run build            # Production build
npm run build:pages      # GitHub Pages build
npm run build:analyze    # Build with bundle analyzer

Quality

npm run lint             # ESLint check
npm run lint:fix         # Auto-fix ESLint issues
npm run format           # Prettier formatting
npm run format:check     # Check formatting without writing
npm run typecheck        # TypeScript check

Testing

npm run test             # Vitest watch mode
npm run test:run         # Run tests once
npm run test:coverage    # Coverage report
npm run test:ui          # Vitest UI mode

---

🧰 Browser Support

Feature	Chrome	Edge	Firefox	Safari
Core Engine	90+ ✅	90+ ✅	90+ ✅	15+ ✅
TTS (Speech Synthesis)	90+ ✅	90+ ✅	90+ ✅	15+ ✅
ASR (Speech Recognition)	90+ ✅	90+ ✅	❌ Not supported	❌ Not supported

ASR Limitations: Speech recognition requires Chrome or Edge due to Web Speech API limitations. Firefox and Safari users can use text input instead.

---

📚 Documentation

• Quick Start — Get running in 5 minutes
• API Reference — Backend API documentation
• Architecture — System design
• Configuration — Environment variables and settings
• Contributing — Contribution guidelines
• Changelog — Version history

---

🛣️ Roadmap

See CHANGELOG.md for released features and GitHub Projects for upcoming work.

• Core 3D avatar rendering
• Voice interaction (TTS/ASR)
• Streaming dialogue
• Service discovery and endpoint failover
• Custom avatar upload
• Multi-language TTS

---

🤝 Contributing

We welcome contributions! Please see our Contributing Guide for details.

1. Fork the repository
2. Create feature branch: git checkout -b feat/amazing-feature
3. Commit changes: git commit -m 'feat: add amazing feature'
4. Push: git push origin feat/amazing-feature
5. Open a Pull Request

Follow Conventional Commits.

---

📄 License

MIT © LessUp

---

Built with ❤️ to make digital humans accessible to everyone

⭐ Star us on GitHub · 🐦 Follow on X