Open-source virtual teammates that take voice and video calls — and let you interrupt, redirect, or pause them mid-task without restarting.

Hop on a call with one. Send a follow-up text. Drop them a calendar invite. They remember who you are next time, what you talked about last week, and what they promised to do about it.

Most agents stop the moment you talk. They make you wait for a tool call to finish, then re-explain when you change your mind. Unity's teammates stay listening through everything — chat, voice, phone, video, screen-share — and treat your interjections, corrections, and questions as first-class inputs rather than interruptions to recover from. Whether the assistant is researching flights, drafting an email, or sitting on a live call with a vendor, you can ask "how's it going?", say "actually do X instead", or pause for ten minutes — without losing context.

It's built around long-lived state, not one-shot conversations. Contacts, projects, files, knowledge, and follow-ups persist as queryable structure — so a teammate remembers who Sarah is, what the Henderson project is about, and what they committed to on your behalf last Wednesday, regardless of which channel you raised it on.

Start here: console.unify.ai — try a teammate in 60 seconds • Overview • Quickstart • ARCHITECTURE.md

You          ▸  "Find me flights to Tokyo for next month."
Unity        ▸  (starts searching)
You          ▸  "Actually, also check trains to Osaka."
Unity        ▸  (adjusts the in-flight search — doesn't restart)
You          ▸  "Pause that, something urgent."
Unity        ▸  (freezes exactly where it is)
... five minutes later ...
You          ▸  "OK, resume. How's it going?"
Unity        ▸  (picks up where it left off, gives you a status update)

Unity        ▸  (on a live phone call with a vendor)
You          ▸  (in a side chat) "Don't agree to anything over $5k."
Unity        ▸  (the constraint reaches the call mid-conversation)

Unity        ▸  Three tasks running at once.
                  [0] research_flights   ██████████░░░  in progress
                  [1] draft_summary      ████████████░  in progress
                  [2] find_restaurants   ██░░░░░░░░░░  starting
                Each one independently inspectable, steerable, and pausable.

There are two paths, depending on whether you want to meet a teammate or run the whole stack yourself.

The lowest-friction path is the hosted product at console.unify.ai. Sign in with Google, get matched with a teammate, and start chatting in about a minute. No install, no Docker, no API keys to manage. Voice, video, telephony, and integrations are all turn-key.

Run the whole stack on your own machine. Runtime, persistence backend, LLM client, and Python SDK are all open-source — see Self-host below.

No signup required. The local installer auto-generates a synthetic API key for the bundled Orchestra and wires everything together. The only key you bring is one LLM provider key (OpenAI or Anthropic).

By default, Unity's open-core install is fully local: the runtime, the LLM client, and the persistence backend (Orchestra, via Docker) all run on your machine. The hosted product at console.unify.ai is optional — Unity does not depend on it for any local feature.

Prerequisites:

• Python 3.12+ (the installer will fetch it with uv if needed)
• Docker (runs the local Orchestra backend)
• PortAudio for audio support
  • macOS: brew install portaudio
  • Ubuntu/Debian: sudo apt-get install portaudio19-dev python3-dev
• One LLM provider key — OpenAI or Anthropic are the simplest paths

Install:

curl -fsSL https://raw.githubusercontent.com/unifyai/unity/main/scripts/install.sh | bash

The installer clones unity, unify, unillm, and orchestra as siblings under ~/.unity/, installs dependencies, creates a unity CLI shim in ~/.local/bin/, boots a local Orchestra in Docker, generates a local API key for the bundled Orchestra, and wires ORCHESTRA_URL and that auto-generated key into ~/.unity/unity/.env. No Unify account or external signup is required.

Add one model provider key to ~/.unity/unity/.env:

OPENAI_API_KEY=sk-...
# or
ANTHROPIC_API_KEY=...

Run the sandbox:

unity --project_name Sandbox --overwrite

At the configuration prompt:

If you're evaluating Unity as a runtime, start with option 2.

> msg Hey, can you help me organize my upcoming week?
> sms I need to reschedule my meeting with Sarah to Thursday
> email Project Update | Here are the Q3 numbers you asked for...

Other unity subcommands: unity setup, unity status, unity stop, unity restart, unity help.

curl -fsSL https://raw.githubusercontent.com/unifyai/unity/main/scripts/install.sh | bash -s -- --skip-setup

git clone https://github.com/unifyai/unity.git      ~/.unity/unity
git clone https://github.com/unifyai/unify.git      ~/.unity/unify
git clone https://github.com/unifyai/unillm.git     ~/.unity/unillm
git clone https://github.com/unifyai/orchestra.git  ~/.unity/orchestra

cd ~/.unity/unity
curl -LsSf https://astral.sh/uv/install.sh | sh
uv sync

cd ~/.unity/orchestra
poetry install
ORCHESTRA_INACTIVITY_TIMEOUT_SECONDS=0 scripts/local.sh start
# Copy the ORCHESTRA_URL and UNIFY_KEY it prints into ~/.unity/unity/.env

The installer copies .env.example to .env (intentionally minimal). For voice mode, live calls, hosted comms, LiveKit, Tavily, or visual caching, see .env.advanced.example and sandboxes/conversation_manager/README.md.

Unity follows the interaction-model / background-model split recently articulated by Thinking Machines — implemented at the harness level, against any LLM you already use.

A persistent interaction loop (the ConversationManager) stays present with the user across every medium. When work needs deeper reasoning than the conversation can produce instantly, it dispatches a background reasoner (the Actor), which writes Python plans over a back office of typed state managers. Crucially, every operation in the system returns a live, steerable handle — and those handles nest. A correction the user makes in chat propagates down through the dispatched action, into whatever manager call is currently running.

flowchart TB
    classDef interaction fill:#fce7f3,stroke:#be185d,stroke-width:2px,color:#1f2937
    classDef actor fill:#bbf7d0,stroke:#15803d,stroke-width:2px,color:#1f2937
    classDef neutral fill:#f9fafb,stroke:#9ca3af,stroke-width:1px,color:#374151
    classDef accent fill:#1f2937,stroke:#000,stroke-width:1px,color:#fef3c7

    User(["User"]):::neutral
    Mediums["💬 chat  ·  📞 voice / phone  ·  🎥 video / screen-share  ·  ✉️ email · SMS"]:::neutral
    Broker["⚡ Event Broker"]:::accent
    CM["<b>ConversationManager</b> · interaction loop (always present)<br/>per-handle steering tools: pause · resume · interject · stop · ask"]:::interaction
    Actor["<b>Actor</b> · background reasoner<br/>writes Python that composes primitives.*"]:::actor
    BackOffice["<b>The Back Office</b> · typed state managers, English-language APIs<br/>Contacts · Knowledge · Tasks · Transcripts · Files · Images · Web · Secrets · ⚙️ Functions · 📖 Guidance"]:::neutral

    User ==> Mediums ==> Broker ==> CM
    CM ==>|"act(...)"| Actor
    Actor ==>|"primitives.*"| BackOffice

    BackOffice -.->|"SteerableToolHandle"| Actor
    Actor -.->|"SteerableToolHandle + notifications"| CM
    CM -.->|"streamed responses"| User

Solid arrows are dispatch flow. Dotted arrows are the steering bus — every level returns the same SteerableToolHandle type, so steering signals propagate down through the call stack while results and notifications propagate up.

This is the demo no other framework can run. The user's mid-flight redirect doesn't abort the run, doesn't append a second prompt, and doesn't wait for the next tool boundary — it propagates through the live nested call stack as a typed signal.

sequenceDiagram
    autonumber
    actor User
    participant CM as ConversationManager
    participant Ax as Actor
    participant TM as TranscriptManager

    User->>CM: "find when Sarah last mentioned Berlin"
    CM->>Ax: act(prompt)
    activate Ax
    Ax-->>CM: handle_A (SteerableToolHandle)
    Note over CM: handle_A stored in<br/>in_flight_actions
    Ax->>TM: transcripts.ask(...)
    activate TM
    TM-->>Ax: handle_B (nested SteerableToolHandle)

    User->>CM: "actually include emails too"
    Note over CM: slow brain wakes,<br/>picks the steering tool<br/>for handle_A
    CM->>Ax: handle_A.interject("...also emails")
    Ax->>TM: handle_B.interject("...also emails")
    TM-->>Ax: refined results
    deactivate TM
    Ax-->>CM: notification (intermediate progress)
    CM-->>User: "scanning emails too..."
    Ax-->>CM: handle_A.result
    deactivate Ax
    CM-->>User: final answer

The clearest way to see what's distinctive about Unity is to draw the same diagram for adjacent projects, using the same visual language. Pink means persistent supervising loop (only Unity has one). Click to expand.

flowchart TB
    classDef agent fill:#bbf7d0,stroke:#15803d,stroke-width:2px,color:#1f2937
    classDef neutral fill:#f9fafb,stroke:#9ca3af,stroke-width:1px,color:#374151
    classDef dispatch fill:#fed7aa,stroke:#c2410c,stroke-width:2px,color:#1f2937

    User(["User"]):::neutral
    Channels["💬 Telegram · Discord · Slack · SMS · Nodes (devices)"]:::neutral
    Gateway["<b>Gateway daemon</b> · dispatcher<br/>per-session lane (1 active run); steer = abort + redeliver"]:::dispatch
    PiAgent["<b>Pi embedded agent</b> · single tool-calling loop<br/>no supervising loop runs in parallel"]:::agent
    Tools["<b>Tools</b> · core + plugin + MCP bridge<br/>core (web · exec · sessions_spawn) · 📞 voice-call plugin (discrete actions: initiate · speak · end) · mcporter → MCP servers"]:::neutral
    State["<b>State</b> · local-first artefacts<br/>JSONL sessions · workspace files (📖 SKILL.md · SOUL.md · AGENTS.md) · memory plugin (one slot at a time)"]:::neutral

    User ==> Channels ==> Gateway
    Gateway ==>|"start / abort run"| PiAgent
    PiAgent ==> Tools
    PiAgent <==> State

flowchart TB
    classDef agent fill:#bbf7d0,stroke:#15803d,stroke-width:2px,color:#1f2937
    classDef neutral fill:#f9fafb,stroke:#9ca3af,stroke-width:1px,color:#374151
    classDef trigger fill:#fed7aa,stroke:#c2410c,stroke-width:2px,color:#1f2937

    User(["User"]):::neutral
    Cron["⏰ cron + webhooks (automation triggers)"]:::trigger
    Surfaces["💬 CLI · TUI · Gateway (Telegram · Discord · Slack · SMS) · ACP (IDE)"]:::neutral
    AIAgent["<b>AIAgent</b> · single ~12k-LOC sync tool-calling loop<br/>steer() = inject text into next tool result; interrupt() = thread-scoped abort flag"]:::agent
    Tools["<b>Tools</b><br/>native tools · execute_code (ephemeral Python against fixed RPC stubs) · TTS / voice_mode / SMS (no live phone call) · delegate_tool · MCP servers"]:::neutral
    State["<b>State</b><br/>SQLite sessions + FTS5 · MEMORY.md / USER.md workspace files · 📖 SKILL.md library · memory provider plugin (mem0 · honcho · ...)"]:::neutral

    User ==> Surfaces
    Cron ==> Surfaces
    Surfaces ==> AIAgent
    AIAgent ==> Tools
    AIAgent <==> State

A small bit of history. This architecture has been running in Unity since 2025 — well ahead of the wider conversation about it. For the record:

• SteerableToolHandle (the universal steering protocol) — first commit September 23, 2025. That predates OpenClaw's first commit (Nov 24, 2025), Hermes Agent's interrupt() (Feb 3, 2026) and steer() (Apr 18, 2026).
• ConversationManager + dual-brain LiveKit voice — first commit November 12, 2025. That predates OpenClaw's voice-call plugin (Jan 11, 2026) by two months.
• The two-tier interaction-loop / background-reasoner pattern as a whole — operational since November 2025. The Thinking Machines paper that articulated the same architecture was published May 11, 2026, six months later.

We're not claiming foresight; the convergence is just interesting if you find architectural archaeology fun. Repo dates verifiable in git log.

Every public manager method returns one. The same ask, interject, pause, resume, stop surface, regardless of whether you're talking to the top-level orchestrator or a deeply nested knowledge query.

handle = await actor.act("Research flights to Tokyo and draft an itinerary")

# Twenty seconds later, while it's still working:
await handle.interject("Also check train options from Tokyo to Osaka")

# Or if something urgent comes up:
await handle.pause()
# ... deal with the urgent thing ...
await handle.resume()

When the Actor calls primitives.contacts.ask(...), the ContactManager starts its own tool loop and returns its own handle — nested inside the Actor's handle, which is nested inside the ConversationManager's. Steering at any level propagates.

Most agents emit one JSON tool call at a time and let the LLM stitch results together across turns. Unity's Actor writes a single Python program per turn over typed primitives.*:

contacts = await primitives.contacts.ask(
    "Who was involved in the Henderson project?"
)
for contact in contacts:
    history = await primitives.knowledge.ask(
        f"What was {contact} last working on?"
    )
    await primitives.contacts.update(
        f"Send {contact} a catch-up email referencing {history}"
    )

This runs in a sandboxed execution session. Variables, loops, real control flow. A contact lookup → knowledge retrieval → outbound communication becomes one coherent plan rather than three separate tool-selection turns — and the LLM can express intermediate computation directly instead of round-tripping through tool messages.

Live calls run as two coordinated brains:

• Slow brain — the ConversationManager. Sees the full picture: all conversations, in-flight actions, structured memory. Makes deliberate decisions. Runs in the main process.
• Fast brain — a real-time voice agent on LiveKit, running as a separate subprocess. Sub-second latency. Handles turn-taking and direct conversation autonomously.

They communicate over IPC. When the slow brain wants to guide the conversation, it sends one of:

• SPEAK — "say exactly this" (bypasses the fast brain's LLM)
• NOTIFY — "here's some context, decide what to do with it"
• BLOCK — nothing; the fast brain keeps going on its own

Screen-share frames and webcam frames stream to both brains simultaneously, so the fast brain can answer "can you see my screen?" without round-tripping, while the slow brain incorporates visual context into longer-running plans.

Unity maintains two persistent libraries that the Actor draws from on every session:

• FunctionManager — executable Python (with metadata and a venv) that the Actor composes into plans.
• GuidanceManager — procedural how-to prose: SOPs, software walkthroughs, multi-step strategies.

After a successful trajectory, a proactive reviewer loop (store_skills) can extract both — code worth keeping, and the procedural narrative for using it. The next session consults both before reaching for raw tools, by design.

Recurring and triggered work isn't configured with cron expressions or webhook YAML — it's described to the agent in natural language and stored as a Task with schedule and repeat (for cadences) or trigger (for event matches). When the time arrives or the trigger fires, a contained Actor run wakes up, reads the task's description, and figures out how to do it.

That same task can graduate over time. After enough successful description-driven runs, the storage-review loop can persist the trajectory as a stored function — at which point the recurring task runs in a hidden, headless lane against that function rather than re-planning from scratch each time. So "summarize my unread emails every Monday at 9" starts out as a paragraph the agent interprets, and gradually becomes an entrypoint it just calls.

Every fifty messages, the MemoryManager runs a background extraction pass over the new transcript window. It distills:

• Contact profiles — who people are, their roles, relationships
• Per-contact summaries — what you've been discussing, sentiment, themes
• Response policies — how each person prefers to be communicated with
• Domain knowledge — project details, preferences, long-term facts
• Tasks — things you committed to, deadlines, follow-ups

These end up in typed, queryable tables — not freeform transcript summaries.

┌─ In-Flight Actions ────────────────────────────────┐
│                                                     │
│  [0] research_flights  ██████████░░░  In progress   │
│      → ask, interject, stop, pause                  │
│                                                     │
│  [1] draft_summary     ████████████░  In progress   │
│      → ask, interject, stop, pause                  │
│                                                     │
│  [2] find_restaurants   ██░░░░░░░░░░  Starting      │
│      → ask, interject, stop, pause                  │
│                                                     │
└─────────────────────────────────────────────────────┘

Each action gets its own dynamically-generated steering tools attached to the slow brain's tool surface. You can inspect, interject into, pause, resume, or stop one action without affecting the others.

For the full architectural breakdown — async tool loop internals, event bus, primitive registry, hosted deployment SPI — see ARCHITECTURE.md. At a glance:

ConversationManager (interaction loop, event-driven scheduling)
    │
    │   Slow Brain ◄── IPC ──► Fast Brain (real-time voice + video, LiveKit)
    │
    ▼
CodeActActor (generates Python plans, calls primitives.* APIs)
    │
    ▼
State Managers (each runs its own async LLM tool loop)
    │
    ├── ContactManager        — people and relationships
    ├── KnowledgeManager      — domain facts, structured knowledge
    ├── TaskScheduler         — durable tasks, schedules, triggers, execution with live handles
    ├── TranscriptManager     — conversation history and search
    ├── GuidanceManager       — procedures, SOPs, how-to knowledge
    ├── FileManager           — file parsing and registry
    ├── ImageManager          — image storage, vision queries
    ├── FunctionManager       — user-defined functions, primitives registry
    ├── WebSearcher           — web research orchestration
    ├── SecretManager         — encrypted secret storage
    ├── BlacklistManager      — blocked contact details
    └── DataManager           — low-level data operations
    │
    ├── EventBus              — typed pub/sub backbone (Pydantic events)
    └── MemoryManager         — offline consolidation every 50 messages

1. A user message arrives on any medium. The slow brain renders a full state snapshot and makes a single-shot tool decision.
2. It starts an action via actor.act(...) → gets back a SteerableToolHandle, registered in in_flight_actions.
3. The Actor generates a Python plan calling typed primitives. Each primitive dispatches to a manager running its own LLM tool loop, returning its own steerable handle.
4. Meanwhile, the slow brain can start more work, steer existing work, or guide the fast brain during voice/video calls.
5. The MemoryManager observes message events and periodically distills conversations into structured knowledge.
6. The EventBus carries typed events with hierarchy labels aligned to tool-loop lineage, making everything observable.

Unity is one of four MIT-licensed repos that make up the runtime. The installer wires them together for the local install; you can also use any of them independently.

Tests exercise the real system (steerable handles, CodeAct, manager composition, nested tool loops) against simulated backends with cached LLM responses:

uv sync --all-groups
source .venv/bin/activate

tests/parallel_run.sh tests/                    # everything
tests/parallel_run.sh tests/actor/              # one module
tests/parallel_run.sh tests/contact_manager/    # another

See tests/README.md for the full philosophy — responses are cached, not mocked. Delete the cache and you're re-evaluating against live models.

unity/
├── unity/
│   ├── actor/                    # CodeActActor
│   ├── conversation_manager/     # Dual-brain orchestration
│   │   └── domains/              # Brain tools, action tracking, rendering
│   ├── common/
│   │   ├── async_tool_loop.py    # SteerableToolHandle
│   │   └── _async_tool/          # Tool loop internals
│   ├── contact_manager/
│   ├── knowledge_manager/
│   ├── task_scheduler/
│   ├── transcript_manager/
│   ├── guidance_manager/
│   ├── memory_manager/
│   ├── function_manager/
│   ├── file_manager/
│   ├── image_manager/
│   ├── web_searcher/
│   ├── secret_manager/
│   ├── events/
│   └── manager_registry.py
├── sandboxes/                    # Interactive playgrounds
│   └── conversation_manager/     # Full ConversationManager sandbox (start here)
├── tests/
├── agent-service/                # Node.js desktop/browser automation
└── deploy/                       # Dockerfile, Cloud Build, virtual desktop

MIT — see LICENSE.

Built by the team at Unify.