Real-time Call Monitoring and Coaching

A supervisor silently monitors a live call between a customer and a human agent, with the ability to whisper coaching instructions that only the agent hears.

Overview

This use case demonstrates:

• 3-way conference with connect()
• Silent monitoring with spy()
• Private coaching with whisper()
• Real-time AI-powered conversation analysis
• Live sentiment and topic logging

State flow: PENDING -> ANSWERED -> CONNECTED (optionally followed by close() if the AI later leaves the conference)

Example

1
@client.on(events.INCOMING_CALL)
2
async def monitored_call(call: ActiveCall):
3
    """Connect caller to an agent with supervisor monitoring."""
4
    await call.answer()
5
6
    # Brief AI greeting before connecting
7
    await call.send_audio(
8
        await tts.synthesize("Please hold while I connect you to an agent.")
9
    )
10
11
    # Connect to the original callee (creates a 3-way conference)
12
    await call.connect()
13
14
    # Start in spy mode to silently monitor
15
    await call.spy()
16
17
    # AI analyzes the conversation in real-time
18
    async for chunk in call.audio_stream():
19
        analysis = await ai_model.analyze_conversation(chunk)
20
21
        if analysis.coaching_suggestion:
22
            # Switch to whisper mode to coach the agent privately
23
            await call.whisper()
24
            await call.send_audio(
25
                await tts.synthesize(analysis.coaching_suggestion)
26
            )
27
            # Switch back to spy mode
28
            await call.spy()
29
30
        # Log call analytics
31
        await db.log_call_analytics(
32
            call_id=call.call_id,
33
            sentiment=analysis.sentiment,
34
            topics=analysis.topics,
35
        )

How It Works

1. The AI answers the call with a brief greeting
2. The call is connected to the original callee using connect(), creating a 3-way conference
3. The AI switches to spy() mode, silently listening to both sides
4. In real-time, the AI analyzes the conversation for sentiment, topics, and coaching opportunities
5. When the AI has a coaching suggestion (e.g., “The customer seems frustrated, try offering a discount”), it:
   • Switches to whisper() mode
   • Speaks the coaching suggestion (only the human agent hears it, not the caller)
   • Switches back to spy() mode
6. Throughout the call, analytics (sentiment scores, topics discussed) are logged to a database

Conference Audio Modes

Key Commands Used

• answer() - Answer the incoming call
• connect() - Create the 3-way conference
• spy() - Silent monitoring
• whisper() - Private coaching audio
• send_audio() - Deliver coaching to the agent

Related

• AI Agent with Human Escalation - AI-to-human handoff pattern
• Call States - Understanding the CONNECTED state