ML — Transcription Recovery

Owner: ML Engineer Domain: ml Complexity: S Prerequisite: Milestone 04 merged

When this slice is complete, the ML streaming pipeline accepts non-contiguous audio frame sequences without crashing, logs gap events for observability, emits BackpressureClearedEvent when caught up after a gap, and reports session-level gap diagnostics on StreamDrainedEvent so Core can flag the transcription appropriately. The post-meeting batch transcription path (which operates on the fully composed audio.webm) is unaffected — this slice addresses only the live streaming path.

Terminology: In this slice, segment always refers to a transcript segment (a speaker-attributed text fragment). For GCS storage segments (aggregated audio frames), see the Core Audio Durability slice.

Required Capabilities

Gap-Tolerant Audio Input

• The ML WebSocket audio handler accepts audio frames with non-contiguous sequence numbers without raising an exception or closing the WebSocket.
• When ML detects a sequence gap (received sequence N+k when expecting N+1), it logs a structured warning: {"event": "audio_sequence_gap", "meeting_id": "...", "expected_sequence": N+1, "received_sequence": N+k, "gap_size": k-1}.
• ML updates its internal last_sequence_received to the latest sequence regardless of gaps — it does not attempt to re-request or buffer missing frames.
• ML forwards the received (post-gap) audio to AssemblyAI immediately — AssemblyAI's streaming protocol handles discontinuities internally.

Session-Level Gap Diagnostics

ML tracks gap events at the session level for observability and downstream degradation signalling. Individual transcript segments are not tagged with quality metadata — a segment produced from post-gap audio is a valid transcription of the audio it received. Gaps manifest as missing time coverage (seconds of audio that were never transcribed live), not as degraded existing segments.

• ML tracks a gap_count and total_gap_frames counter per live session, incremented on each detected sequence gap.
• The StreamDrainedEvent includes gap_count and total_gap_frames in its payload so Core can set transcriptions.is_degraded = true when gaps occurred during the live session.
• Core — not ML — is responsible for setting the is_degraded flag on the transcription resource. ML only reports the diagnostic data.
• The post-meeting batch transcription from the fully composed audio.webm creates a new transcription with is_degraded = false (assuming gap recovery filled the audio). This is handled by the existing TranscriptionJob pipeline — no ML changes needed.

Backpressure Recovery Signal

• After a gap, ML's audio processing queue may temporarily grow as post-gap frames arrive while AssemblyAI is still catching up. ML emits BackpressureEvent if its internal audio queue exceeds the configured depth threshold.
• When the queue depth returns to zero (AssemblyAI has caught up), ML emits BackpressureClearedEvent — matching the contract in contracts/audio.mdx:

  {
    "specversion": "1.0",
    "type": "com.wordloop.ml.backpressure_cleared.v1",
    "data": {
      "meeting_id": "meeting-uuid",
      "queue_depth": 0
    }
  }

• Core uses BackpressureClearedEvent to transition from degraded ML health to recovered, clearing the client banner.

Insight Pipeline Resilience

• The rolling transcript buffer (used for LLM insight extraction) continues appending segments produced from post-gap audio. The LLM context simply has a time discontinuity — no special handling is needed because the prompt already provides timestamp context for each segment.
• If a gap exceeds 30 seconds of audio (based on total_gap_frames * chunk_duration_ms), ML emits RecordingHealthEvent to Core with code: insight_warning and message: "Extended audio gap may affect insight quality." This is a pipeline health signal (Layer 1), not a transcript quality signal — insight extraction continues.

Session Recovery (ML Pod Restart)

• On ML WebSocket reconnect (Core sends a new StreamStartEvent), ML reconstructs its session state from the event payload: speaker_states, voice_profiles, last_audio_sequence, last_ml_event_id.
• ML resets its gap_count and total_gap_frames counters on reconnect — the reconnect itself is a known discontinuity and is not counted as a gap.
• ML requests recent transcript segments from Core (GET /transcriptions/{id}/segments?after_ms=...) to rebuild its LLM context buffer before processing new audio.

Dependencies

• Milestone 04 (live-transcript-to-final-rebuild) must be merged — specifically the ML streaming session handler, AssemblyAI proxy, and StreamStartEvent/StreamDrainedEvent contracts.
• The BackpressureClearedEvent contract from contracts/audio.mdx must be implemented.

Contract Extensions

This slice adds new fields to existing event contracts. The following contracts must be updated:

Domain Notes

Pipeline Sequencing

Audio Frame (seq N+k, gap detected)
    │
    ├──► Log structured warning (audio_sequence_gap)
    ├──► Increment gap_count, total_gap_frames
    ├──► Update last_sequence_received = N+k
    │
    ├──► Forward audio to AssemblyAI (immediate, no buffering)
    │       └──► AssemblyAI handles discontinuity internally
    │
    └──► If queue_depth > threshold:
            Emit BackpressureEvent
            └──► When queue_depth returns to 0:
                    Emit BackpressureClearedEvent

Degradation Model (ML's Role)

ML participates in two of the three degradation layers but does not own either:

Failure Modes

Test Cases

Gap tolerance

Backpressure

Insight resilience

Session recovery

Completion Checklist

• Code merged and deployed
• Bet progress tests pass (./dev test bet meeting-recording)
• Permanent service tests implemented per testing strategy
• Code review completed
• API review completed (if new or modified API surfaces)
• Testing review completed
• System documentation updated (architecture, data flows, API reference, database reference, runbooks — as applicable)