Patient Voice Bot

Problem

Testing conversational AI voice agents against realistic patient scenarios — scheduling, insurance questions, edge cases — is normally a slow, manual process requiring a human to place and evaluate every test call.

Challenges

• Streaming low-latency audio between Twilio Media Streams and the OpenAI Realtime API over WebSockets
• Keeping call transcripts synchronized with audio in real time
• Designing prompt-driven scenarios that reliably cover scheduling, insurance, and edge-case conversations

Solution

A real-time voice agent integrating Twilio Media Streams, the OpenAI Realtime API, and WebSockets to run low-latency conversational calls, paired with 10+ prompt-driven patient scenarios that capture end-to-end call flows for automated QA review.

Architecture

A FastAPI service bridges Twilio Media Streams and the OpenAI Realtime API over WebSockets, handling bidirectional audio streaming and resolving transcript synchronization between the two. Prompt-driven scenario definitions drive simulated patient calls end-to-end, capturing full call flows for QA review.

Stack

Highlights

• Real-time, low-latency conversational calls over Twilio + OpenAI Realtime API
• 10+ prompt-driven patient scenarios covering scheduling, insurance, and edge cases
• ~80% reduction in manual QA validation effort
• Resolved transcript synchronization issues between audio and text streams