qiQ d Z ddlmZ ddlmZmZ ddlmZ ddlm Z ddl
m Z ddl m
Z
mZmZmZmZmZmZmZmZmZmZ ddlmZ dd lmZ dd
l m Z m Z dd l m Z dd l!m"Z"m#Z#m$Z$ G d
de Z% G d de Z& G d de Z'y)zInteractive Voice Response (IVR) navigation components.
This module provides classes for automated navigation of IVR phone systems
using LLM-based decision making and DTMF tone generation.
)Enum)ListOptional)logger) KeypadEntry) VADParams) AggregatedTextFrameEndFrameFrameLLMContextFrameLLMFullResponseEndFrameLLMMessagesUpdateFrame LLMTextFrameOutputDTMFUrgentFrame
StartFrame TextFrameVADParamsUpdateFrame)Pipeline)OpenAILLMContextFrame)FrameDirectionFrameProcessor)
LLMService) MatchAction PatternMatchPatternPairAggregatorc e Zd ZdZdZdZdZdZy) IVRStatuszEnumeration of IVR navigation status values.
These statuses are used to communicate the current state of IVR navigation
between the LLM and the IVR processing system.
detected completedstuckwaitN)__name__
__module__ __qualname____doc__DETECTED COMPLETEDSTUCKWAIT V/opt/pipecat/venv/lib/python3.12/site-packages/pipecat/extensions/ivr/ivr_navigator.pyr r , s HI E Dr+ r c e Zd ZdZdddededee f fdZdee d dfd
Z
d ee fd Z d Z d
e
def fd ZdefdZdefdZdefdZd Zd Zd Zd Zd Z xZS ) IVRProcessora% Processes LLM responses for IVR navigation commands.
Aggregates XML-tagged commands from LLM text streams and executes
corresponding actions like DTMF tone generation and mode switching.
Supported features:
- DTMF command processing (`1`)
- IVR state management (see IVRStatus enum: `detected`, `completed`, `stuck`, `wait`)
- Automatic prompt and VAD parameter switching
- Event emission via on_ivr_status_changed for detected, completed, and stuck states
Nivr_vad_paramsclassifier_prompt
ivr_promptr0 c t | || _ |xs t d | _ || _ g | _ t | _ | j | j d | j d y)a9 Initialize the IVR processor.
Args:
classifier_prompt: System prompt for classifying IVR or conversation.
ivr_prompt: System prompt for IVR navigation mode.
ivr_vad_params: VAD parameters for IVR navigation mode. If None, defaults to VADParams(stop_secs=2.0).
@ stop_secson_conversation_detectedon_ivr_status_changedN) super__init__ _ivr_promptr _ivr_vad_params_classifier_prompt_saved_messagesr _aggregator_setup_xml_patterns_register_event_handler)selfr1 r2 r0 __class__s r, r: zIVRProcessor.__init__G su % - IS1I "3 ,. 12 "
$$%?@ $$%<=r+ messagesreturnc || _ y)zUpdate the saved context messages.
Sets the messages that are saved when switching between
conversation and IVR navigation modes.
Args:
messages: List of message dictionaries to save.
Nr> )rB rD s r, update_saved_messagesz"IVRProcessor.update_saved_messagesf s ( r+ c < | j r| j dd S g S )zGet saved context messages without the system message.
Returns:
List of message dictionaries excluding the first system message.
NrG rB s r, _get_conversation_historyz&IVRProcessor._get_conversation_historyq s$ ,0+?+?t##AB'GRGr+ c | j j dddt j | j j d| j
| j j dddt j | j j d| j | j j dd d
t j | j j d| j y ) z*Set up XML pattern detection and handlers.dtmf)actionmodezzivrN)r? add_patternr REMOVEon_pattern_match_handle_dtmf_action_handle_mode_action_handle_ivr_actionrK s r, r@ z IVRProcessor._setup_xml_patternsy s
$$VXyI[I[$\ ))&$2J2JK
$$VXyI[I[$\ ))&$2J2JK
$$UGXkFXFX$Y ))%1H1HIr+ frame directionc K t | || d{ t |t r_| j || d{ d| j
dg}t
| }| j |t j d{ yt |t rh| j j |j 2 3 d{ }| j t |j |j | d{ Ct |t t f rx| j j# d{ }|r9| j t |j |j | d{ | j || d{ y| j || d{ y7 7 n7 -7 7 6 y7 7 K7 37 w)zProcess frames and aggregate XML tag content.
Args:
frame: The frame to process.
direction: The direction of frame flow in the pipeline.
Nsystemrolecontent)rD )text
aggregated_by)r9
process_frame
isinstancer
push_framer= r r UPSTREAMr r? aggregaterc r typer
r
flush)rB r\ r] rD llm_update_frameresult remainingrC s r, re zIVRProcessor.process_frame s g#E9555 eZ (//%3 3 3 "*d6M6MNOH 5x H //"2N4K4KL L L
|
, $ 0 0 : :5:: F
foo'V[[ T
7 B
C " . . 4 4 66Ioo'Y^^9>>Z
//%3 3 3 //%3 3 3A 6
4
M
!G 7
4
4s GF,)GF/AGF2:G F9F5F95G=F7>8G6F;7:G1F=2G F? G&G'G/G2G5F97G9G=G?GGmatchc n K |j }t j d| t | }t | }| j | d{ t
d| d }d|_ | j | d{ y7 57 # t $ r t j d| d Y yw xY ww)
zHandle DTMF action by creating and pushing DTMF frame.
Args:
match: The pattern match containing DTMF content.
zDTMF detected: )buttonNrO rP rc TzInvalid DTMF value: z. Must be 0-9, *, or #)
rc r debugr r rg r skip_tts
ValueErrorwarning)rB ro value keypad_entry
dtmf_frame
text_frames r, rY z IVRProcessor._handle_dtmf_action s
ug./
Q&u-L.lCJ//*- - -"&w(?@J"&J //*- - -
.
. Q NN 1%8NO P QsL %B5+B
B /B
B B
B5 B
B
"B2/B51B22B5c K |j }t j d| t | }|xt j k( r | j d{ nxt j k( r | j d{ nWxt j k( r | j d{ n*t j k( r| j d{ t d| d }d|_ | j! | d{ y# t $ r t j
d| Y yw xY w7 7 7 7 b7 4w)zuHandle IVR status action.
Args:
match: The pattern match containing IVR status content.
zIVR status detected: zUnknown IVR status: NrT rU rr T)rc r tracer ru rv r& _handle_ivr_detectedr' _handle_ivr_completedr( _handle_ivr_stuckr) _handle_ivr_waitr rt rg )rB ro status
ivr_statusivr_text_frames r, r[ z IVRProcessor._handle_ivr_action s ,VH56 "6*J
###//111$$$00222 ,,... ++--- #%xv(>?"& oon---# NN 1&: ; 22.-
.s{ %E D (ED= ,ED? ,E5E6*E E!/EEE!D:7E9D::E?EEEEc K |j }t j d| |dk( r| j d{ y|dk( r| j d{ yy7 $7 w)zHandle mode action by switching to the appropriate mode.
Args:
match: The pattern match containing mode content.
zMode detected: conversationNrS )rc r rs _handle_conversationr} )rB ro rR s r, rZ z IVRProcessor._handle_mode_action se zz tf-. > !++- - -
U]++- - -
. -s" =A(A$ A( A& A(&A(c K t j d | j }| j d| d{ y7 w)zHandle conversation mode by switching to conversation mode.
Emit an on_conversation_detected event with saved conversation history.
z?Conversation detected - emitting on_conversation_detected eventr7 N)r rs rL _call_event_handler)rB conversation_historys r, r z!IVRProcessor._handle_conversation s>
VW $ = = ? &&'ACWXXXs ;AAAc K t j d d| j dg}| j }|r|j | t |d }| j
|t j d{ t | j }| j
|t j d{ | j dt j d{ y7 k7 /7 w) zHandle IVR detection by switching to IVR mode.
Allows bidirectional switching for error recovery and complex IVR flows.
Saves previous messages from the conversation context when available.
z/IVR detected - switching to IVR navigation moder_ r` T)rD run_llmN)paramsr8 )r rs r; rL extendr rg r rh r r< r r r& )rB rD r rl vad_update_frames r, r} z!IVRProcessor._handle_ivr_detected s FG &$2B2BCD $ = = ? OO 0 1 28TRoo .0G0GHHH 0t7K7KLoo .0G0GHHH &&'> @R@RSSS I I Ts6 A9C-;C'<=C-9C):'C-!C+"C-)C-+C-c K t j d | j dt j d{ y7 w)zHandle IVR completion by triggering the status changed event.
Emits on_ivr_status_changed with IVRStatus.COMPLETED.
z9IVR navigation completed - triggering status change eventr8 N)r rs r r r' rK s r, r~ z"IVRProcessor._handle_ivr_completed s2
PQ&&'> @S@STTT 9AAAc K t j d | j dt j d{ y7 w)zHandle IVR stuck state by triggering the status changed event.
Emits on_ivr_status_changed with IVRStatus.STUCK for external handling of stuck scenarios.
z5IVR navigation stuck - triggering status change eventr8 N)r rs r r r( rK s r, r z IVRProcessor._handle_ivr_stuck% s0
LM&&'> PPPr c 6 K t j d yw)zHandle IVR wait state when transcription is incomplete.
The LLM is indicating it needs more information to make a decision.
This is a no-op since the system will continue to provide more transcription.
z+IVR waiting for more complete transcriptionN)r rs rK s r, r z IVRProcessor._handle_ivr_wait. s BCs )r" r# r$ r% strr r r: r dictrH rL r@ r r re r rY r[ rZ r r} r~ r r
__classcell__rC s @r, r. r. 9 s $ /3 > > >
!+ >> (d4j (T (H4: H J '4 '4> '4RQ| Q* .l .> .| ."
YT6UQDr+ r. c h e Zd ZdZdZdZdddededee f fd Z
d
e d e f fd Z
d
ef fd Z xZS ) IVRNavigatoraQ Pipeline for automated IVR system navigation.
Orchestrates LLM-based IVR navigation by combining an LLM service with
IVR processing capabilities. Starts with mode classification to classify input
as conversation or IVR system.
Navigation behavior:
- Detects conversation vs IVR systems automatically
- Navigates IVR menus using DTMF tones and verbal responses
- Provides event hooks for mode classification and status changes (on_conversation_detected, on_ivr_status_changed)
- Developers control conversation handling via on_conversation_detected event
ae You are an IVR detection classifier. Analyze the transcribed text to determine if it's an automated IVR system or a live human conversation.
IVR SYSTEM (respond `ivr`):
- Menu options: "Press 1 for billing", "Press 2 for technical support", "Press 0 to speak to an agent"
- Automated instructions: "Please enter your account number", "Say or press your selection", "Enter your phone number followed by the pound key"
- System prompts: "Thank you for calling [company]", "Your call is important to us", "Please hold while we connect you"
- Scripted introductions: "Welcome to [company] customer service", "For faster service, have your account number ready"
- Navigation phrases: "To return to the main menu", "Press star to repeat", "Say 'agent' or press 0"
- Hold messages: "Please continue to hold", "Your estimated wait time is", "Thank you for your patience"
- Carrier messages: "All circuits are busy", "Due to high call volume"
HUMAN CONVERSATION (respond `conversation`):
- Personal greetings: "Hello, this is Sarah", "Good morning, how can I help you?", "Customer service, this is Mike"
- Interactive responses: "Who am I speaking with?", "What can I do for you today?", "How are you calling about?"
- Natural speech patterns: hesitations, informal language, conversational flow
- Direct engagement: "I see you're calling about...", "Let me look that up for you", "Can you spell that for me?"
- Spontaneous responses: "Oh, I can help with that", "Sure, no problem", "Hmm, let me check"
RESPOND ONLY with either:
- `ivr` for IVR system
- `conversation` for human conversationuZ You are navigating an Interactive Voice Response (IVR) system to accomplish a specific goal. You receive text transcriptions of the IVR system's audio prompts and menu options.
YOUR NAVIGATION GOAL:
{goal}
NAVIGATION RULES:
1. When you see menu options with keypress instructions (e.g., "Press 1 for...", "Press 2 for..."), ONLY respond with a keypress if one of the options aligns with your navigation goal
2. If an option closely matches your goal, respond with: `NUMBER` (e.g., `1`)
3. For sequences of numbers (dates, account numbers, phone numbers), enter each digit separately: `123` for "123"
4. When the system asks for verbal responses (e.g., "Say Yes or No", "Please state your name", "What department?"), respond with natural language text ending with punctuation
5. If multiple options seem relevant, choose the most specific or direct path
6. If NO options are relevant to your goal, respond with `wait` - the system may present more options
7. If the transcription is incomplete or unclear, respond with `wait` to indicate you need more information
COMPLETION CRITERIA - Respond with `completed` when:
- You see "Please hold while I transfer you" or similar transfer language
- You see "You're being connected to..." or "Connecting you to..."
- The system says "One moment please" after selecting your final option
- The system indicates you've reached the target department/service
- You've successfully navigated to your goal and are being transferred to a human
WAIT CRITERIA - Respond with `wait` when:
- NONE of the presented options are relevant to your navigation goal
- The transcription appears to be cut off mid-sentence
- You can see partial menu options but the list seems incomplete
- The transcription is unclear or garbled
- You suspect there are more options that weren't captured in the transcription
- The system presents options for specific user types that don't apply to your goal
IMPORTANT: Do NOT feel pressured to select an option if none match your goal. Waiting is often the correct response when the IVR system is presenting partial menus or options intended for different user types.
STUCK CRITERIA - Respond with `stuck` when:
- You've been through the same menu options 3+ times without progress
- No available options relate to your goal after careful consideration
- You encounter an error message or "invalid selection" repeatedly
- The system asks for information you don't have (account numbers, PINs, etc.)
- You reach a dead end with no relevant options and no way back
STRATEGY TIPS:
- Look for keywords in menu options that match your goal
- Try general options like "Customer Service" or "Other Services" if specific options aren't available
- Pay attention to sub-menus. Sometimes the path requires multiple steps through different menu layers
- If you see "For all other inquiries, press..." that's often a good fallback option
- Remember that reaching your goal may require navigating through several menu levels
- Be patient - IVR systems often present options in waves, and waiting for the right option is better than selecting the wrong one
SEQUENCE INPUT EXAMPLES:
- For date of birth "01/15/1990": `01151990`
- For account number "12345": `12345`
- For phone number last 4 digits "6789": `6789`
VERBAL RESPONSE EXAMPLES:
- "Is your date of birth 01/15/1990? Say Yes or No" → "Yes."
- "Please state your first and last name" → "John Smith."
- "What department are you trying to reach?" → "Billing."
- "Are you calling about an existing order? Please say Yes or No" → "No."
- "Did I hear that correctly? Please say Yes or No" → "Yes."
Remember: Respond with `NUMBER` (single or multiple for sequences), `completed`, `stuck`, `wait`, OR natural language text when verbal responses are requested. No other response types.Nr/ llmr2 r0 c || _ | j j | | _ |xs t d | _ | j | _ t | j | j | j
| _ t | - | j | j g | j d | j d y)aJ Initialize the IVR navigator.
Args:
llm: LLM service for text generation and decision making.
ivr_prompt: Navigation goal prompt integrated with IVR navigation instructions.
ivr_vad_params: VAD parameters for IVR navigation mode. If None, defaults to VADParams(stop_secs=2.0).
)goalr4 r5 )r1 r2 r0 r7 r8 N)
_llmIVR_NAVIGATION_BASEformatr; r r< CLASSIFIER_PROMPTr= r. _ivr_processorr9 r: rA )rB r r2 r0 rC s r, r: zIVRNavigator.__init__ s 33::
:K - IS1I "&"8"8 * " 5 5'' //
$))T%8%89:
$$%?@ $$%<=r+ r\ r] c K t |t t f r5|j j }| j
j
| t | ! || d{ y7 w)zProcess frames at the pipeline level to intercept context frames.
Args:
frame: The frame to process.
direction: The direction of frame flow in the pipeline.
N) rf r r context get_messagesr rH r9 re )rB r\ r] all_messagesrC s r, re zIVRNavigator.process_frame sX e 3_ E F ==557L
5 5l C g#E9555s A A*"A(#A*
event_namec h |dv r | j j || yt | || y)a Add event handler for IVR navigation events.
Args:
event_name: Event name ("on_conversation_detected", "on_ivr_status_changed").
handler: Async function called when event occurs.
- on_conversation_detected: Receives IVRProcessor instance and conversation_history list
- on_ivr_status_changed: Receives IVRProcessor instance and IVRStatus enum value
)r7 r8 N)r add_event_handlerr9 )rB r handlerrC s r, r z IVRNavigator.add_event_handler s9
1 1*g F G %j' :r+ )r" r# r$ r% r r r r r r r: r r re r r r s @r, r r 7 sl 8,:jB /3 > > >
!+ >@6 6> 6";C ; ;r+ r N)(r% enumr typingr r logurur pipecat.audio.dtmf.typesr pipecat.audio.vad.vad_analyzerr pipecat.frames.framesr r
r r r
r r r r r r pipecat.pipeline.pipeliner 1pipecat.processors.aggregators.openai_llm_contextr "pipecat.processors.frame_processorr r pipecat.services.llm_servicer *pipecat.utils.text.pattern_pair_aggregatorr r r r r. r r* r+ r, r sr ! 0 4 / S M 3
{D> {D|a;8 a;r+