Skip to content

Epic: Google Meet Meeting Assistant — proactive join, notes, summaries & in-call agency (Phase 1 + 2) #3505

Closed
10 of 10 issues completed

Description

@graycyrus

Turn the existing Google Meet bot (backend Recall.ai/Camoufox bot driven via Socket.IO, core domain src/openhuman/agent_meetings/) into a proactive meeting assistant, then add in-call agency.

This is the tracking epic. Work is sliced into dependency-ordered sub-issues, each independently shippable behind a config flag.

Phase 1 — proactive meeting assistant

  1. Calendar-triggered auto-join — watch Google Calendar; when a meeting with a Meet link is ~30 min away (and a "happening now" check), notify the user: "You have a call — add OpenHuman?" with Yes / No / Always.
  2. Muted listener — bot joins mic-off, listen-only, captures the full transcript.
  3. One thread per meeting — a dedicated chat thread per meeting, tagged with a "Meetings" label, titled from the calendar event, with the transcript posted into it.
  4. Proactive summary — after the call, offer to summarize; on yes, generate + post a structured summary.
  5. Tool-aware actionables — extract action items, classify each against the user's connected tools: executable → approval card ("create Notion task — do it?"), non-executable → advice-only. All executable actions always require explicit approval.

Phase 2 — in-call live agency

  1. During a live call, the user can address the bot by voice ("Hey Tiny, schedule a follow-up in 40 minutes"). The bot detects the request in real time, routes it through the orchestrator with live meeting context, runs a clarification loop if needed, gets approval (spoken "approve" or thread card), executes, posts the result to the thread, and optionally confirms verbally.

Confirmed product decisions

  • Auto-join: default Ask each time; Always option available from day one (auto_join_policy: AskEachTime | Always | Never).
  • Which meetings: any calendar event with a Google Meet link (no organizer/acceptance filtering).
  • Threads: one new thread per meeting, tagged with a "Meetings" label (reusing the existing thread-label system).
  • Actionables: always require an explicit approval click (no silent execution, even for whitelisted tools) — in both phases.
  • Auto-summarize: Ask | Always | Never, default Ask.

Foundational design — the meeting_id

A single correlation ID threads the whole journey: calendar event → join request → returned transcript → meeting thread → (Phase 2) live in-call requests. This is the missing link today (the backend transcript can't currently be tied back to a specific meeting) and drives the two small backend asks below.

Roadmap (dependency order; each shippable behind a flag)

# Sub-issue Depends on Backend-blocked?
PR-0 #3506 — Config + types + meeting store + notification action-buttons — No
PR-1 #3507 — Calendar watch → auto-join prompt #3506 No
PR-3 #3509 — Muted join + correlation ID + per-meeting thread + "Meetings" label/tab + transcript (absorbed former PR-2) #3506, #3514 Yes
PR-4 #3510 — Post-call summary + tool-aware actionables #3509 No
PR-5 #3511 — Settings UI #3506 No
PR-6 #3512 — Phase 2: in-call request reception + orchestrator routing #3509, #3514 Yes
PR-7 #3513 — Phase 2: in-call approval + execution + speak #3512, #3514 Yes
— #3514 — Backend contract changes — tracking

Unblocked now (no backend needed): PR-0 (#3506), PR-1 (#3507), PR-4 (#3510), PR-5 (#3511).
Backend dependency: #3514 unblocks PR-3 (Phase 1) and PR-6/PR-7 (Phase 2).

Backend asks (see backend-contract sub-issue)

  • Phase 1: accept correlationId + listenOnly on join; echo correlationId back on every bot:* event.
  • Phase 2: a live mid-call command channel (bot:in_call_request) + an optional "speak into the call" capability (bot:speak).

Conventions

Canonical domain module shape, controller schemas wired via src/core/all.rs, ≥80% diff coverage, i18n parity across all 14 locales, verbose debug logging, no business logic in src/core.

Activity

  1. added
    featureNet-new user-facing capability or product behavior.
    agentBuilt-in agents, prompts, orchestration, and agent runtime in src/openhuman/agent/.
    on Jun 8, 2026
  2. moved this from Todo to In review in Team Openhumanon Jun 10, 2026
  3. moved this from In review to Testing in Team Openhumanon Jun 22, 2026
  4. Al629176 commented on Jun 26, 2026

    @Al629176
    Contributor

    Tracked bug: #4151 — "Mascot streaming capacity is exhausted" shown on first use (blocking the feature before any meeting is joined).

  5. Al629176 commented on Jun 26, 2026

    @Al629176
    Contributor

    Tracked bug: #4172 — Bot joins the call but does not respond when addressed (wake phrase not triggering orchestrator).

  6. self-assigned this
    on Jun 29, 2026
  7. moved this from Testing to Done in Team Openhumanon Jul 14, 2026
  8. closed this as completedby moving to Done in Team Openhumanon Jul 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

agentBuilt-in agents, prompts, orchestration, and agent runtime in src/openhuman/agent/.featureNet-new user-facing capability or product behavior.

Type

No type

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions