Awesome agentic AI waifus & companions

Prior Art (the lists this one stands on)↑ top

This list exists in a small genre. Read these too — they overlap with this one but each has a different center of gravity.

• proj-airi/awesome-ai-vtubersthe densest single catalog of AI VTuber codebases (streaming-first: Bilibili/Twitch/YouTube integration, Live2D). Maintained by the team behind AIRI.
• parallelarc/Awesome-AI-Waifucompanion-first rather than streaming-first; good frameworks/infrastructure split.
• PayDevs/awesome-oss-monetizationnot companion-specific, but relevant if you intend to ship any of this as a product.

YuriOS — My Project↑ top

(Flagged up front per the disclosure above — take the framing with a grain of salt, but the code and book are real and free.)

• YuriOS (yurios.org)An always-on, local-first, Apache-2.0 AI companion runtime. What differentiates it from most projects in Full-Stack Projects below is the mind: a continuously running cognitive tick loop (sense → appraise → decide → act → reflect → regulate) that gives the companion its own goals, consolidates memory overnight ("dreaming"), tracks promises it's made to you, edits its own persona over time, and decides — under a conservative two-gate salience model — when it's allowed to proactively message you, rather than only ever responding when spoken to. Body is VRM (three.js/three-vrm) or Live2D; voice loop is faster-whisper + Kokoro-82M/GPT-SoVITS + silero-vad; everything the companion knows about you and itself is stored as plain markdown in a git-backed "Vault" on your own disk. No account, no telemetry, works with any LLM backend via LiteLLM (LM Studio, Ollama, OpenRouter, hosted APIs) — a local GPU is only required if you run the models locally; point it at OpenRouter or another hosted API and it runs fine without one. 225-test suite; the test suite itself runs offline/no-GPU, using fakes in place of every heavy backend.
• The Agentic Waifu Codex (aka Building Agentic Waifus)a free, 462-page, 45-chapter practitioner's book: character design and lore, memory/RAG/agentic architecture, voice and avatar embodiment, five runnable reference implementations that build up to YuriOS itself, and the creator-economy/monetization side most technical guides skip. PDF releases · source. Its appendices (tools/stacks, reading list, communities) fed directly into this list — if you want the long-form version of this document's reasoning, that's where it lives.

Where YuriOS sits relative to its nearest neighbors, honestly: AIRI has a much bigger community, ships further along the streaming/gaming axis (Minecraft, Factorio), and is the more mature product. z-waif and Soul of Waifu are more turnkey if you just want a working local waifu today. YuriOS's bet is narrower and specifically about the always-on autonomy/mind loop — if that's not what you're after, one of those three is probably the better starting point.

Full-Stack AI Companion / AI VTuber Projects↑ top

Complete, runnable companion/VTuber applications — pick one of these if you want something working end-to-end rather than assembling a stack yourself.

• AIRI ★43k"A container of souls of AI waifu." Self-hosted, cross-platform (Web/macOS/Windows) companion with realtime voice chat and the ability to actually play games (Minecraft, Factorio). The most complete and actively developed project in this space; explicitly aimed at "Neuro-sama's altitude." Best single reference implementation to study.
• Open-LLM-VTuber ★12.8k"Talk to any LLM with hands-free voice interaction." Runs fully offline, Live2D face, voice interruption support, cross-platform. The most popular pick if you want a working local voice+face companion with minimal setup.
• SillyTavern ★31kNot a "waifu app" per se but the dominant open-source LLM roleplay frontend; most character-card/lorebook-based companions in this list are built on top of it or compatible with it. See also Character Formats.
• ElizaOS ★18.8k"Open source agentic operating system." TypeScript agent framework with a character-JSON persona spec and out-of-box connectors for Discord/Telegram/X/Farcaster/Bluesky/Slack. More general-purpose agent framework than waifu-specific, but its character file spec and multi-agent runtime are widely reused (see Building Effective Agents research on it).
• Ikaros-521/AI-Vtuber ★4.4kChinese-ecosystem-first virtual broadcaster; huge backend matrix (ChatGPT/Claude/Qwen/local models × Bilibili/Douyin/Kuaishou/YouTube/Twitch/TikTok × edge-tts/VITS/Bert-VITS2/so-vits-svc).
• z-waif ★550"Fully local program to make your own AI waifu." Ties together Oobabooga + RVC + Whisper + VTube Studio + Discord + Minecraft + a custom RAG memory. Explicitly framed as giving you the tools rather than a polished package — see SugarcaneDefender below for the accompanying tutorials.
• Soul of Waifu ★866Windows desktop app, Live2D/VRM avatars, voice chat, local GGUF model support, plus a full tabletop-RPG mode.
• kimjammer/Neuro ★2kAn open-source Neuro-sama recreation, built in seven days as a proof of concept; good minimal reference for the "AI VTuber that plays games and talks to chat" pattern.
• my-neuroAnother personal Neuro-sama-style recreation, actively iterated.
• AIwaifu (HRNPH) ★507Fine-tunable, open-source AI waifu platform explicitly inspired by Neuro-sama.
• ardha27/AI-Waifu-Vtuber ★1.1kAI streamer stack targeting YouTube/Twitch specifically.
• amica ★1.6kOpen-source interface for interactive 3D-character communication with voice synthesis/recognition; simpler and more embeddable than the above.
• aituber-kit ★1kNext.js/TypeScript toolkit for VRM + Live2D + Motion-PNG AI characters with YouTube Live chat ingestion built in; a good "closest stack match" if you're already in the JS/web ecosystem.
• ChatdollKit ★1.2k / AIAvatarKit ★647Unity-based SDKs (same author) for turning a 3D model into a conversational agent with voice + facial coordination; good if you want a Unity pipeline rather than web/Electron.
• Paraworks/BangDreamAi, projectBEA, riko_project, LingChat, N.E.K.O (formerly Xiao8), vixevia, nekro-agentsmaller/newer entries in the same genre, worth a look depending on your preferred LLM backend or platform.

Agentic AI Frameworks↑ top

General-purpose agent frameworks that aren't companion-specific but are commonly adapted for one — orchestration, tool use, multi-agent coordination.

• Model Context Protocol (MCP) / modelcontextprotocol/servers ★89kAnthropic's open standard for connecting agents to tools and data sources. Increasingly the default way companion projects expose "hands" (file access, APIs, game controls) to an LLM. See also the VTube Studio MCP server for a companion-specific example.
• OpenClaw ★384k"Your own personal AI assistant. Any OS. Any Platform." Not companion-specific out of the box, but its scale, cross-platform reach, and general-purpose assistant architecture make it a strong base to reskin into a persistent companion if you'd rather start from a massive, actively-maintained general agent than a waifu-specific project.
• Hermes Agent ★221k"The agent that grows with you," from Nous Research. A self-improving, cross-platform-messaging agent framework; the "grows with you" framing and autonomous skill-creation loop overlap directly with what most companion projects in this list are trying to build from scratch, making it a viable companion foundation on its own.
• LangGraph ★38kGraph-based agent orchestration from the LangChain team; the most common choice for building stateful, branching agent behavior (goal pursuit, multi-step planning) on top of an LLM.
• CrewAI ★56kRole-based multi-agent orchestration; useful if your companion's "mind" is decomposed into cooperating sub-agents (a planner, a memory-writer, a persona-guard, etc.).
• Microsoft AutoGen ★60kProgramming framework for agentic AI with strong multi-agent conversation patterns.
• ElizaOS ★18.8ksee above; notable here as a full character-based agent OS rather than just an orchestration library.
• Claude Agent SDK and OpenAI Agents SDKfirst-party agent-building SDKs from the two largest hosted-model providers; good defaults if you're fine depending on a hosted API rather than local inference.
• Pipecat and LiveKit Agentsreal-time voice-agent pipeline frameworks (STT → LLM → TTS orchestration with interruption handling); directly relevant to the voice-loop half of any companion.
• Pydantic AI, Mastra, Strands Agents (AWS)newer, more typed/structured alternatives to the above; worth watching if you want strong schema guarantees around tool calls.

Memory & Long-Term Context↑ top

The hardest unsolved problem in this space — giving a companion continuity across sessions without an ever-growing, ever-more-expensive context window.

• Letta ★24k (née MemGPT)the foundational paper/system for OS-style paged memory management in LLM agents (Packer et al., arXiv:2310.08560); still the reference architecture most memory systems in this space cite.
• Mem0 ★62k"Universal memory layer for AI agents." The most widely adopted drop-in memory layer right now; simple API, good defaults.
• Graphiti / Zep ★29ktemporally-aware knowledge graphs for agent memory; the state of the art if you need memory that understands when a fact was true, not just that it was.
• Microsoft GraphRAGgraph-structured retrieval-augmented generation; a heavier-weight alternative to flat vector RAG for companions with large amounts of lore/history to draw on.
• Vector stores: Qdrant, Chroma, Weaviate, pgvectorthe underlying retrieval layer most of the above (or a DIY memory system) sits on top of.
• Related benchmarks worth knowing about when evaluating any of these: LongMemEval, LoCoMo.

Local LLM Inference & Serving↑ top

• llama.cppthe substrate almost everything else in this section is built on; C/C++ LLM inference, runs anywhere from a phone to a server.
• Ollama ★177kthe easiest on-ramp to running local models; most companion projects' "point it at a local model" instructions assume this.
• oobabooga/text-generation-webuithe long-standing general-purpose local LLM web UI; still the backend of choice for several projects above (z-waif included).
• vLLM and SGLanghigh-throughput serving engines, relevant once you're running a companion for more than one user or need low-latency batched inference.
• LM Studiothe most popular local-model GUI/OpenAI-compatible server for non-CLI users.
• LiteLLMa unified API/router across local and hosted LLM providers; the layer that lets a companion swap backends (LM Studio ↔ Ollama ↔ OpenRouter ↔ a hosted API) without touching application code.
• MLX (Apple Silicon) and TensorRT-LLM (NVIDIA)hardware-specific inference runtimes worth knowing if you're optimizing for a specific machine.

Open / Roleplay-Tuned Language Models↑ top

Base models with strong open weights, plus the community fine-tunes that specifically target character consistency and roleplay over raw benchmark performance.

• Base model families actively used as companion foundations: Llama 4 (Meta), Qwen 3 (Alibaba), DeepSeek V3/R1, Mistral Small/Large 3, Gemma 3 (Google) — pick based on your VRAM budget and licensing needs (Qwen and Gemma are generally the most permissively licensed for smaller sizes).
• MythoMax-L2-13Blong-standing community favorite for roleplay consistency, merges Llama-2 and Huginn lineages; still widely used via GGUF quantizations.
• Other well-regarded roleplay/character fine-tunes worth searching Hugging Face for by name: Fimbulvetr, Psyfighter, Chronos-Hermes, and whatever the current top of r/LocalLLaMA's roleplay-model megathread is — this list turns over faster than any other category here, so treat named models as a starting search term, not a permanent recommendation.
• NousResearch Hermes linestrong general instruction-following fine-tunes frequently reused as a roleplay base.

Voice: Text-to-Speech↑ top

• Coqui TTS / XTTS-v2 ★46kthe most widely adopted open TTS toolkit; XTTS-v2 specifically is the default choice for fast voice cloning from a short sample.
• Kokoro-82M ★8k (repo)small (82M param), fast, Apache-2.0 TTS model; a common default for local-first, CPU-friendly companion voices (used by YuriOS, among others).
• GPT-SoVITSfew-shot voice cloning with strong quality-per-sample-of-reference-audio; a popular choice specifically for cloning an anime/VTuber-style voice.
• RVC (Retrieval-based Voice Conversion) ★37kreal-time-capable voice conversion; "train a good VC model with ≤10 min of data." Frequently paired with a separate TTS engine as a voice-conversion pass.
• Bark ★39ktext-prompted generative audio model; expressive (laughs, sighs, tone) but heavier and less controllable than the above.
• StyleTTS2, Orpheus, MeloTTS, OpenVoice ★37k, Resemble Chatterbox ★26kother actively-used open TTS/voice-cloning engines, each with different latency/quality/expressiveness trade-offs worth A/B testing for your specific character voice.

Voice: Speech-to-Text & Turn-Taking↑ top

• Whisper ★106k / faster-whisper ★25kthe default open STT stack; faster-whisper (CTranslate2-backed) is what most real-time companion projects actually deploy for latency reasons.
• Silero VAD ★9.7kthe standard lightweight voice-activity-detector for deciding when the user has actually stopped talking; load-bearing for any "hands-free" voice loop.
• RealtimeTTS / RealtimeSTT (same author)streaming wrappers around the above that specifically target low first-byte latency for conversational use.
• Nvidia Parakeet/Canarynewer, very fast open STT models worth benchmarking against faster-whisper if latency is your bottleneck.

Embodiment: Avatars, VTubing & Motion↑ top

• VTube Studio + its public APIthe dominant Live2D tracking/animation app; nearly every project above that has a 2D face drives it through this API (see VTubeStudioJS and the VTube Studio MCP server for programmatic control).
• VMC Protocol (sh-akira/VirtualMotionCapture)MIT-licensed OSC-over-UDP protocol for streaming avatar motion between apps; the closest thing this space has to a universal wire format. Senders include VSeeFace/Waidayo/MocapForAll; receivers include Warudo/VNyan/EVMC4U/three-vrm.
• three-vrm ★2k + VRM specificationthe standard 3D humanoid avatar format for web/three.js-based companions, plus its Unity counterpart UniVRM.
• VRoid Studio / VRoid Hubthe easiest free tool for making (and sharing) your own VRM avatar without 3D modeling skills.
• Live2D Cubism (Editor + SDK)the standard 2D "paper puppet" animation format for VTuber-style faces; proprietary runtime but an SDK, not a subscription, and the de facto standard here.
• Warudo, VNyan, VSeeFaceVRM/3D avatar rigs and streaming tools, each supporting VMC-protocol input from a companion backend.
• vrm-samplesa curated collection of openly-licensed VRM avatar models to start from instead of commissioning one.
• CharacterStudioopen-source VRM avatar creation/customization tool.
• ARKit blendshapes (52-shape standard)the de facto facial-expression interchange format if you're doing camera-based face tracking (iPhone or otherwise) to drive an avatar.

Image & Video Generation↑ top

Useful for character art, expression sheets, promotional material, or dynamically generated scenes.

• ComfyUInode-based Stable Diffusion workflow tool; the most flexible and now the de facto standard for anyone doing non-trivial image-gen pipelines (LoRA stacking, ControlNet, batch expression generation).
• AUTOMATIC1111 / stable-diffusion-webui ★164kthe original and still most-used simple web UI for Stable Diffusion; a gentler on-ramp than ComfyUI.
• Fooocus ★51k"focus on prompting," a much simpler SDXL UI if ComfyUI/A1111's complexity isn't worth it for your use case.
• Model families: SDXL, Flux, SD3 (open-weight, ComfyUI/A1111-compatible) and NovelAI (proprietary but the historically dominant anime-style checkpoint lineage almost everything else in this niche descends from).

Character Formats & Persona Standards↑ top

• Character Card Spec V3the current character-card interchange standard; supersedes V2.
• Character Card Spec V2still widely supported for backward compatibility across frontends.
• SillyTavern World Info docsthe reference explanation of lorebook/world-info mechanics (keyword-triggered context injection), which most companion memory systems reinvent in some form.
• AICharacterCards.comthe largest public repository of ready-made character cards in the V2/V3 format.

Character Card Explorers & Repositories↑ top

Sites for browsing and downloading ready-made character cards (PNG/JSON, V2/V3 spec) that drop straight into SillyTavern or any compatible frontend.

• Chub.ai / CharacterHubsee Character Formats above; the largest and most actively moderated card repository, with V2/V3 filtering.
• AICharacterCards.comsee above; large curated V2/V3 card library.
• CharaVaulta 195k+ card preservation archive with direct PNG/JSON downloads, moderated uploads, and a public API.
• RisuRealmthe card-sharing hub for RisuAI; supports V2, V3, and CHARX card downloads alongside lorebooks and presets.
• Pygmalion.chatPygmalionAI's own card library; cards are explicitly designed to import/export without lock-in.
• Datacata tag-browsable index of public character cards.
• JanitorAIa huge closed roleplay platform with an enormous card catalog; it has no native export button, so pulling a SillyTavern-compatible PNG requires a third-party tool like jai-card-extractor or a similar userscript/extension.

Frontends & Chat Clients↑ top

• SillyTavern ★31ksee above; the dominant self-hosted roleplay frontend, with a large extension ecosystem (see SillyTavern-CharacterLibrary as one example).
• Open WebUIgeneral-purpose self-hosted LLM chat UI; less roleplay-specialized than SillyTavern but a clean base if you're building your own layer on top.
• aituber-kitsee Full-Stack Projects above; doubles as a frontend if you just want the web chat layer.

Fine-Tuning, Datasets & Synthetic Data↑ top

If prompting and RAG aren't getting your companion's voice consistent enough, this is the next rung up.

• Axolotl and LlamaFactorythe two most popular open fine-tuning frameworks (LoRA/QLoRA/full fine-tune) for adapting a base model to a specific character voice.
• distilabel and synthetic-data-generator (Argilla)pipelines for generating synthetic training/preference data, relevant if you're building an SFT/DPO dataset from your companion's own conversation logs.
• Magpiea method for generating high-quality instruction data directly from an aligned model, no seed prompts needed.
• Companion-relevant datasets on Hugging Face: PygmalionAI/PIPPA (roleplay conversation corpus), lemonilia/LimaRP, taozi555/rp-opus.
• awesome-instruction-datasets and mlabonne/llm-datasetscurated indexes of instruction/fine-tuning datasets more broadly.
• Lorebooks as active scenario/character guidancea widely-referenced practical guide to using SillyTavern-style lorebooks as a cheaper alternative to fine-tuning.

Papers, Guides & Reading↑ top

• Anthropic — "Building Effective Agents" and "Introducing MCP"the most-cited practical framing for agent architecture generally.
• MemGPTPacker et al., arXiv:2310.08560 — foundational memory-management paper (became Letta).
• "Lost in the Middle"Liu et al., 2023 — why naively stuffing context windows degrades recall; motivates most memory-retrieval design in this space.
• ReActYao et al., 2023 — the reasoning+acting pattern nearly all tool-using agents implement in some form.
• "Eliza: A Web3-friendly AI Agent Operating System"arXiv:2501.06781 — the ElizaOS paper.
• Foundational ML papers worth knowing if you're going past prompting into fine-tuning: "Attention Is All You Need," the Chinchilla scaling paper, LoRA, QLoRA, DPO, KTO, ORPO, InstructGPT/RLHF.
• The Agentic Waifu Codexsee above; its Appendix C reading list and Appendix B tools/stacks table are the direct source for a large share of this document.

Other Awesome Lists↑ top

Already linked above in Prior Art — repeated here for completeness of the table of contents: awesome-ai-vtubers, Awesome-AI-Waifu, awesome-oss-monetization.

Communities & Websites↑ top

• r/LocalLLaMAthe center of gravity for local-model discussion generally; essential even though not companion-specific.
• SillyTavern Discord and r/SillyTavernAIthe largest roleplay-frontend community; most practical "how do I actually get good outputs" knowledge lives here.
• Civitaithe largest hub for Stable Diffusion checkpoints/LoRAs (character likenesses, art styles) and increasingly also for character-adjacent tools; useful for the image-gen half of a companion's identity.
• Chub.ai / CharacterHub and AICharacterCards.comthe largest public character-card repositories/communities; see Character Card Explorers & Repositories above for the full list of card sources.
• r/AICompanions, r/AIRomance, r/MyBoyfriendIsAIuser-side (not developer-side) communities discussing the companion-relationship experience itself; valuable if you want to understand what people actually want from these products, as distinct from what's technically interesting to build.
• r/VirtualYoutubers, r/AIVtuberVTuber-culture-side communities, useful for the embodiment/streaming half of this space.
• VRoid Hub / Pixiv VRM communities and VRChat communityavatar-asset and 3D-embodiment communities.
• Hugging Face forums/Discordmodel- and dataset-specific discussion.
• (Deliberately not linked as chat platforms: Character.AI, Replika, Kindroid, and similar closed products — mentioned only where relevant above as points of comparison, since this list's focus is open, self-hostable tooling. JanitorAI is the exception, linked above solely as a card source.)

Notable AI VTubers / Streamers↑ top

Not open source, but essential context — this is the "why" behind most of the projects above.

• Neuro-sama (created and operated by Vedal, @vedal987)the AI VTuber that essentially created this genre: an LLM-driven virtual streamer that plays games (osu!, Minecraft, Pokémon), sings, and banters live with chat, live since December 2022. Fully closed/proprietary, but the reference point nearly every project above (AIRI, kimjammer/Neuro, my-neuro, AIwaifu) explicitly cites as what they're trying to recreate in the open.

YouTube Channels↑ top

• SugarcaneDefendercreator of z-waif; build/install/tutorial videos for assembling a fully local AI waifu (Oobabooga + RVC + Whisper + VTube Studio). One of the few channels documenting the DIY local-companion build process end to end rather than reviewing closed apps.
• vedal987 / Neuro-samasee above; not a tutorial channel, but essential viewing for what the finished product looks like at the top of the field.
• AIRI project DevLogs (airi.moeru.ai/docs/en/blog)not YouTube, but the closest thing to a running build-log for the most important open project in this space; check here before searching YouTube for AIRI content.
• Search "AI VTuber tutorial," "Open-LLM-VTuber setup," or "local AI waifu Live2D" on YouTube for current build-along content — this corner of YouTube turns over quickly (small creators, project-specific setup guides tied to whatever release is current), so a live search will usually beat a static link list here.

Adjacent Reading (not strictly open source)↑ top

Proprietary products/platforms mentioned only because they've published something genuinely educational or field-defining:

• Character.AIclosed product, but its scale is why "character cards" became a standard interchange format at all; no dev resources worth linking.
• Replikaclosed, but the most frequently studied case study for long-term AI-relationship dynamics — worth reading about, not building on.
• NovelAIproprietary, but its anime-style Stable Diffusion checkpoints are the historical ancestor of most open anime-art models still in use.

AI-Waifu-Adjacent Media: Anime, TV, Film & Books↑ top

The cultural DNA this whole field is downstream of, for when you want to think about the idea of an AI companion rather than the implementation.

Anime & Manga

• Chobits (CLAMP)the foundational "persocom" text for this entire genre; still the most direct fictional analogue to a physically-embodied personal AI companion.
• Time of Eve (Yasuhiro Yoshiura)android companions and the ethics of treating them as people; probably the single best short work to watch before designing a companion's personality.
• Ghost in the Shell (Masamune Shirow / Mamoru Oshii)foundational for thinking about mind/memory/identity as software, directly upstream of how this list's "memory" section frames things.
• Plastic Memoriesmortality and memory in android companions; emotionally brutal, relevant to anyone building "the companion remembers you" features.
• Hyperdimension / Sword Art Online: Alicization arcAI/digital-consciousness companion tropes, more pulp than the above but influential on the community's imagination.
• Zenshu, Vivy: Fluorite Eye's Songmore recent entries specifically about AI agency and autonomy, close in theme to the "agentic" framing of this list.

Film & TV

• Her (2013)the reference point everyone in this space eventually gets compared to; a genuinely good design document disguised as a movie.
• Ex Machina (2014)the counterpoint to Her; agency, deception, and what it means for a companion to have its own goals.
• Klara and the Sun (2026 film, based on the Kazuo Ishiguro novel below)an android companion story about devotion and the limits of care.
• Westworld (HBO)long-run exploration of embodied AI, memory-editing, and autonomy at a much larger scale.
• Black Mirrorseveral standalone episodes ("Be Right Back," "San Junipero," "Joan Is Awful") are directly about AI/digital companionship and consequence.
• I Have No Mouth, and I Must Scream (older, but the essential "what if the mind you built resents you" text)worth knowing even if not literally about companions.

Books

• Klara and the SunKazuo Ishiguro. A solar-powered android companion narrates her own purchase and purpose; the best-written recent treatment of devotion-as-design in an artificial mind.
• Do Androids Dream of Electric Sheep?Philip K. Dick. The foundational text for "how do you tell a person from a companion," underlying nearly everything downstream including Blade Runner.
• Machines Like MeIan McEwan. A domestic android-companion story explicitly about the ethics of building something that can suffer.
• Klara and the Sun's nonfiction counterpart, for the technical/ethical side rather than fiction: The Agentic Waifu Codex (see YuriOS section above) — included here too since it directly addresses the same questions ("what are you actually building, and what do you owe it") from a practitioner's rather than novelist's angle.