AI Agent with Human Escalation

The AI handles the conversation but can escalate to a human agent when it detects it cannot help, with full context handoff.

Overview

This use case demonstrates:

• AI-first handling with continuous conversation
• Intelligent escalation detection
• Saving full conversation context to a database for the human agent
• Seamless handoff via connect() and close()

State flow: PENDING -> ANSWERED -> CONNECTED -> DISCONNECTED

Example

1
@client.on(events.INCOMING_CALL)
2
async def ai_with_escalation(call: ActiveCall):
3
    await call.answer()
4
5
    escalate = False
6
    conversation_summary = []
7
8
    async for chunk in call.audio_stream():
9
        transcript = await stt.transcribe(chunk)
10
        conversation_summary.append({"role": "caller", "text": transcript})
11
12
        # Check if the AI decides to escalate
13
        ai_response = await ai_model.generate(chunk)
14
        response_text = await stt.transcribe(ai_response)
15
        conversation_summary.append({"role": "ai", "text": response_text})
16
17
        if await ai_model.should_escalate(conversation_summary):
18
            escalate = True
19
            break
20
21
        await call.send_audio(ai_response)
22
23
    if escalate:
24
        # Save conversation context for the human agent
25
        await db.save_call_context(
26
            call_id=call.call_id,
27
            caller=call.caller_number,
28
            summary=conversation_summary,
29
        )
30
31
        await call.send_audio(
32
            await tts.synthesize(
33
                "Let me connect you with a specialist who can help."
34
            )
35
        )
36
37
        # Connect to the original callee, then leave the call
38
        await call.connect()
39
        await call.close()
40
    else:
41
        await call.disconnect()

How It Works

1. The AI answers the call and begins a normal conversation
2. Each turn is transcribed and added to a running conversation summary
3. After each AI response, the model evaluates whether it should escalate (e.g., the caller is frustrated, the request is outside the AI’s capabilities, or the caller explicitly asks for a human)
4. If escalation is triggered:
   • The full conversation summary is saved to a database, keyed by call.call_id
   • The caller hears a brief transition message
   • The call is connected to the original callee using connect(), then the agent leaves with close()
5. The human agent can retrieve the conversation context from the database to continue seamlessly
6. If the AI handles the call successfully, it ends the call with disconnect()

Key Commands Used

• answer() - Answer the incoming call
• audio_stream() - Receive caller audio
• send_audio() - Play AI responses
• connect() - Connect to the original callee
• close() - Leave the call after connecting the caller to the callee
• disconnect() - End the call when the AI fully handled it

Related

• AI Receptionist with Database Lookup - Route based on caller identity
• Call Monitoring and Coaching - Human-AI collaboration during calls