handcrafted-persona-engine

At a glance

What it is	What you need	How long to first pixel
Voice-driven Live2D character with LLM brain, real-time TTS, and streaming-ready output.	Windows x64, NVIDIA GPU with CUDA, ~16 GB free disk.	Download → double-click → pick a profile.

---

• Overview
• See it in action
• Getting started
• Install profiles
• Screenshots
• Features
• How it works
• Use cases
• Deeper docs
• Community
• Contributing
• Support

Overview

Persona Engine listens through your microphone, thinks with an LLM guided by a personality file, speaks back with real-time TTS (optionally voice-cloned), and drives a Live2D avatar in sync. You can watch the character inside the built-in transparent overlay, or pipe it into OBS over Spout for streaming.

The included Aria model is rigged for the engine's lip-sync and expression pipeline out of the box. You can bring your own model too — see the Live2D Integration Guide.

[!IMPORTANT] Persona Engine feels most natural with a fine-tuned LLM trained on the engine's communication format. Standard OpenAI-compatible models (Groq, OpenAI, Ollama, …) work too, but you'll want to put care into personality.txt. A template (personality_example.txt) ships in the repo, and the fine-tuned model is available in Discord.

See it in action

Getting started

[!IMPORTANT] Requires NVIDIA GPU with CUDA (Windows x64). ASR, TTS, and RVC all run on CUDA via ONNX Runtime — CPU/AMD/Intel are not supported.

1. Download PersonaEngine-<version>-win-x64.zip from Releases.
2. Extract somewhere with ≥ 16 GB free. Models land in a Resources/ folder next to the exe.
3. Double-click PersonaEngine.exe and pick an install profile when prompted. Models and the NVIDIA runtime are downloaded, hash-verified, and installed automatically.

Re-run the picker

PersonaEngine.exe --reinstall

Flag	Purpose
--profile=try|stream|build	Skip the picker and use the named profile
--repair	Re-download anything that fails hash verification
--verify	Re-hash installed assets and report mismatches (no downloads)
--offline	Refuse to touch the network — fail fast if assets are missing
--non-interactive	Treat any prompt as fatal (pair with --profile=…)
--skip-gpu-check	Bypass the GPU capability gate (not recommended)

The asset directory layout changed when the in-app installer landed. Existing Resources/Models/ and Resources/Live2D/Avatars/ trees from older builds are ignored — the installer re-downloads into the new locations on first launch. Free up ~16 GB before starting; delete the old folders once the bootstrapper finishes.

Install profiles

	Try it out	Stream with it	Build with it
Best for	First look, small downloads	Everyday streaming	Production, highest quality
Listening (Whisper)	Tiny	Small	Large-v3 Turbo
Voice (TTS)	Kokoro	Kokoro	Kokoro + Qwen3 expressive
Lip-sync	VBridger	VBridger	VBridger + Audio2Face
Approx. download	Smallest	Mid	Largest (≈ 16 GB)

[!TIP] Picked Build-with-it? You still have to flip the switches. The profile downloads the bigger models, but the UI defaults keep the light ones active until you toggle them:

• Voice panel → set mode to Expressive (Qwen3)
• Listening panel → pick the Accurate Whisper template
• Avatar panel → enable Audio2Face lip-sync

Full walkthrough in INSTALLATION.md.

Screenshots

Features

How it works

A single turn flows through these stages:

1. Listen — microphone audio, Silero VAD picks out speech.
2. Understand — fast Whisper watches for barge-in; accurate Whisper transcribes the final utterance.
3. Contextualize (optional) — Vision module reads text from a chosen window.
4. Think — transcription + history + context + personality.txt go to the LLM.
5. Respond — LLM streams text, optionally tagged with emotions like [EMOTION:😊].
6. Filter (optional) — keyword + ML profanity pass.
7. Speak — TTS (Kokoro or Qwen3) synthesizes the response; espeak-ng fills phoneme gaps.
8. Clone (optional) — RVC retargets the voice in real time.
9. Animate — phonemes drive lip-sync, emotion tags trigger Live2D expressions, idle animations run between turns.
10. Display — subtitles, avatar, and roulette render to the built-in overlay and/or Spout outputs for OBS; audio plays through the selected device.
11. Loop — back to listening.

Use cases

• VTubing & streaming — AI co-host, chat-reactive character, fully AI-driven persona.
• Virtual assistant — animated desktop companion that actually talks back.
• Interactive kiosks — guides for museums, trade shows, retail.
• Education — language practice partner, historical-figure Q&A, tutor.
• Games — more conversational NPCs and companions.
• Character chatbots — immersive chats with fictional characters.

Deeper docs

• INSTALLATION.md — profile picker, CLI flags, LLM + personality setup, overlay vs Spout, building from source, upgrading, bootstrapper troubleshooting.
• CONFIGURATION.md — every appsettings.json field, annotated.
• Live2D.md — rigging requirements and the VBridger parameter spec for custom avatars.

Community