Feature Waves

Wave 3 — Voice Usability

Six connected improvements to make the voice canvas genuinely hands-free.

Dated 2026-06-12Programme Feature Waves

Situation

The voice canvas required a tap after every reply, long replies could push controls off-screen or shrink the font, and there was no way to interrupt Mallory mid-reply. This wave makes voice genuinely hands-free for real replies.

Six changes land together across voice/+page.svelte, session.svelte.ts, and settings/+page.svelte: pause moves to a corner icon, auto-listen loops replies, long replies auto-scroll without shrinking, a "clearing my throat" cue masks the TTS gap, barge-in lets the user interrupt, and mic failures fail safe instead of retrying forever.

Key decisions

Auto-listen model

A continuous hands-free loop (reply → listen → reply…), self-terminating on the existing silence auto-end, a mic tap, or leaving the voice view.

Auto-listen default

ON (opt-out), scoped to the voice canvas only — never re-opens the mic on the main thread.

Works with TTS off

The loop re-arms after the text reply finishes too, regardless of whether replies are spoken aloud.

Long replies

Auto-scroll follows the newest text without shrinking the font; manual scroll-up pauses auto-follow until the user returns to the bottom.

Barge-in trigger

The pinned bottom mic stays live during speaking (not tap-anywhere, which would fight scrolling a long reply).

Architecture

State and pure logic added to session.svelte.ts, with the voice view restructured into three fixed bands so controls never run off-screen.

session.svelte.ts — new state

autoListen (persisted, default true), voiceActive, micError, speakSeq (invalidates in-flight spoken replies on skip/barge-in).

voiceLogic.ts (pure helpers)

shouldRearm({autoListen, voiceActive, paused, recording, thinking}) and the speakSeq skip guard — unit-tested independent of the audio stack.

voice/+page.svelte — three bands

Top bar (close + corner pause/play icon) · scrollable middle (auto-follow reply text) · pinned bottom mic (context-aware: Reply/Speak vs Talk now).

interruptAndListen()

Composes stopSpeaking/skipSpeaking + toggleMic into a single barge-in action.

settings/+page.svelte

New "Auto-listen after replies" toggle in the Voice section, mirroring "Speak replies aloud".

Risks & guardrails

Runaway retry on mic failure

captureSpeech sets micError and returns '' on getUserMedia failure; a failed capture yields no reply, so the auto-listen loop stops after one attempt rather than retrying infinitely.

Font shrink on long replies

Explicitly rejected — font size is fixed; the scroll region auto-follows instead, with manual scroll-up pausing follow until the user returns to bottom.

Real mic/TTS paths need a device

The hands-free loop, barge-in and the cue are verified on deploy with the Access-gated API; layout (bands, scroll-follow, pinned controls) is verifiable in local preview.

Delivery

Task 1

Pure re-arm predicate (voiceLogic.ts).

Task 2

Session state + settings plumbing (session.svelte.ts).

Task 3

Session voice logic — skip, barge-in, re-arm, mic-error (session.svelte.ts).

Task 4

Auto-listen setting toggle (settings/+page.svelte).

Task 5

Voice view rework (voice/+page.svelte) — the three-band layout and all six behaviours.

Out of scope

Wake-word / always-listening without entering the voice viewHaptics or a sound cue on re-armWord-level karaoke scroll synced to TTSChanging the TTS engine or reducing its latencyBarge-in via tap-anywhere

spec · docs/superpowers/specs/2026-06-12-wave-3-voice-usability-design.md
plan · docs/superpowers/plans/2026-06-12-wave-3-voice-usability.md

Mallory · Estate briefing