qidZddlZddlZddlmZddlmZddlmZm Z m Z m Z ddl m Z ddlmZddlmZdd lmZmZdd lmZmZmZmZmZmZmZmZmZmZdd l m!Z!Gd d e"eZ#Gdde"eZ$Gdde"eZ%Gdde!Z&y)uGenesys AudioHook Serializer for Pipecat. This module provides a serializer for integrating Pipecat pipelines with Genesys Cloud Contact Center via the AudioHook protocol. Features: - Bidirectional audio streaming (PCMU μ-law at 8kHz) - Automatic protocol handshake handling (open/opened, close/closed, ping/pong) - Input/output variables for Architect flow integration - DTMF event support - Barge-in (interruption) events - Pause/resume support for hold scenarios (optional) Protocol Reference: - https://developer.genesys.cloud/devapps/audiohook Audio Format: - PCMU (μ-law) at 8kHz sample rate (preferred) - L16 (16-bit linear PCM) at 8kHz also supported - Mono (external channel) or Stereo (external on left, internal on right) N) timedelta)Enum)AnyDictListOptional)logger) KeypadEntry)SOXRStreamAudioResampler) pcm_to_ulaw ulaw_to_pcm) AudioRawFrame CancelFrameEndFrameFrameInputAudioRawFrameInputDTMFFrameInterruptionFrameOutputTransportMessageFrame!OutputTransportMessageUrgentFrame StartFrame)FrameSerializerc@eZdZdZdZdZdZdZdZdZ dZ d Z d Z d Z d Zd Zy)AudioHookMessageTypez!AudioHook protocol message types.openopenedcloseclosedpauseresumedpingpongupdateeventerror disconnectN)__name__ __module__ __qualname____doc__OPENOPENEDCLOSECLOSEDPAUSERESUMEDPINGPONGUPDATEEVENTERROR DISCONNECTM/opt/pipecat/venv/lib/python3.12/site-packages/pipecat/serializers/genesys.pyrr1sA+ D F E F EG D D F E EJr8rceZdZdZdZdZdZy)AudioHookChannelz&AudioHook audio channel configuration.externalinternalbothN)r'r(r)r*EXTERNALINTERNALBOTHr7r8r9r;r;Bs0HH Dr8r;ceZdZdZdZdZy)AudioHookMediaFormatzSupported audio formats.PCMUL16N)r'r(r)r*rDrEr7r8r9rCrCJs" D Cr8rCc eZdZdZdZGddej Z d:deeffd Ze de fd Z e dee fd Z e de fd Ze de fd Ze deee effd Ze deee effdZe deee effdZdee efddfdZdefdZdede fdZde defdZdefdZ d;dedeee efde dee effdZ dd*ed+e d,e dee effd-Z&de'de e(zdzfd.Z)d/e e(zde'dzfd0Z*d/e(de'dzfd1Z+d+ee efde'dzfd2Z,d+ee efde'dzfd3Z-d+ee efde'dzfd4Z.d+ee efde'dzfd5Z/d+ee efde'dzfd6Z0d+ee efde'dzfd7Z1d+ee efde'dzfd8Z2d+ee efde'dzfd9Z3xZ4S)?GenesysAudioHookSerializeraSerializer for Genesys AudioHook WebSocket protocol. This serializer handles converting between Pipecat frames and Genesys AudioHook protocol messages. It supports: - Bidirectional audio streaming (PCMU at 8kHz) - Automatic protocol handshake (open/opened, close/closed, ping/pong) - Session lifecycle management with pause/resume support - Custom input/output variables for Architect flow integration - DTMF event handling - Barge-in events for interruption support The AudioHook protocol uses: - Text WebSocket frames for JSON control messages - Binary WebSocket frames for audio data Example usage: ```python serializer = GenesysAudioHookSerializer( params=GenesysAudioHookSerializer.InputParams( channel=AudioHookChannel.EXTERNAL, supported_languages=["en-US", "es-ES"], selected_language="en-US", ) ) # Use with FastAPI WebSocket transport transport = FastAPIWebsocketTransport( websocket=websocket, params=FastAPIWebsocketParams( audio_in_enabled=True, audio_out_enabled=True, serializer=serializer, audio_out_fixed_packet_size=1600, # Important: prevents 429 rate limiting from Genesys ), ) # Access call information after connection participant = serializer.participant # ani, dnis, etc. input_vars = serializer.input_variables # Custom vars from Architect # Set output variables to return to Architect serializer.set_output_variables({"intent": "billing", "resolved": True}) ``` Attributes: PROTOCOL_VERSION: The AudioHook protocol version (currently "2"). 2ceZdZUdZdZeed<dZeeed<e jZ e ed<e jZe ed<dZeed <d Zeed <dZeeeed <dZeeed <d Zeed<y)&GenesysAudioHookSerializer.InputParamsaQConfiguration parameters for GenesysAudioHookSerializer. Attributes: genesys_sample_rate: Sample rate used by Genesys (default: 8000 Hz). sample_rate: Optional override for pipeline input sample rate. channel: Which audio channels to process (external, internal, both). media_format: Audio format (PCMU or L16). process_external: Whether to process external (customer) audio. process_internal: Whether to process internal (agent) audio. supported_languages: List of language codes the bot supports (e.g., ["en-US", "es-ES"]). selected_language: Default language code to use. start_paused: Whether to start the session in paused state. ignore_rtvi_messages: Inherited from base FrameSerializer, defaults to True. i@genesys_sample_rateN sample_ratechannel media_formatTprocess_externalFprocess_internalsupported_languagesselected_language start_paused)r'r(r)r*rKint__annotations__rLrr;r?rMrCrDrNrOboolrPrQrstrrRrSr7r8r9 InputParamsrJs $(S'%) Xc])$4$=$=!=-A-F-F *F!%$%!&$&37Xd3i07+/8C=/" d"r8rXNparamsc t||xstjfi||jj |_d|_ttj|_ t|_ t|_d|_d|_d|_d|_t'd|_d|_d|_d|_d|_d|_d|_|j7d|j7d|j7d|j7d|j7d|j7d |j7d y) zInitialize the GenesysAudioHookSerializer. Args: params: Configuration parameters. **kwargs: Additional arguments passed to BaseObject (e.g., name). rFNon_openon_closeon_pingon_pause on_updateon_erroron_dtmf)super__init__rGrX_paramsrK_genesys_sample_rate _sample_raterWuuiduuid4 _session_idr _input_resampler_output_resampler _client_seq _server_seq_is_open _is_pausedr _position_conversation_id _participant_custom_config _media_info_input_variables_output_variables_register_event_handler)selfrYkwargs __class__s r9rcz#GenesysAudioHookSerializer.__init__s, K#=#I#I#KVvV$(LL$D$D!tzz|,!9 :!9!; "1046:8<;?:>;? $$Y/ $$Z0 $$Y/ $$Z0 $$[1 $$Z0 $$Y/r8returnc|jS)zAGet the Genesys AudioHook session ID generated by the serializer.)rirxs r9 session_idz%GenesysAudioHookSerializer.session_idsr8c|jS)z Get the Genesys conversation ID.)rqr}s r9conversation_idz*GenesysAudioHookSerializer.conversation_id$$$r8c|jS)z'Check if the AudioHook session is open.)rnr}s r9is_openz"GenesysAudioHookSerializer.is_opens}}r8c|jS)z#Check if audio streaming is paused.)ror}s r9 is_pausedz$GenesysAudioHookSerializer.is_pausedsr8c|jS)z=Get participant info (ani, dnis, etc.) from the open message.)rrr}s r9 participantz&GenesysAudioHookSerializer.participants   r8c|jS)z1Get custom input variables from the open message.)rur}s r9input_variablesz*GenesysAudioHookSerializer.input_variablesrr8c|jS)z4Get custom output variables to send back to Genesys.)rvr}s r9output_variablesz+GenesysAudioHookSerializer.output_variabless%%%r8 variablescB||_tjd|y)aSet custom output variables to send back to Genesys on close. These variables will be included in the 'closed' response when Genesys closes the connection, making them available in the Architect flow. Args: variables: Dictionary of custom variables to send to Genesys. Example: ```python # During the conversation, collect data and set it serializer.set_output_variables({ "intent": "billing_inquiry", "customer_verified": True, "summary": "Customer asked about their bill", "transfer_to": "billing_queue" }) ``` zOutput variables set: N)rvr debug)rxrs r9set_output_variablesz/GenesysAudioHookSerializer.set_output_variabless ("+ -i[9:r8framecK|jjxs |j|_t j d|jyw)zSets up the serializer with pipeline configuration. Args: frame: The StartFrame containing pipeline configuration. z2GenesysAudioHookSerializer setup with sample_rate=N)rdrLaudio_in_sample_raterfr r)rxrs r9setupz GenesysAudioHookSerializer.setups@ !LL44R8R8R I$J[J[I\]^sA Apositionc0|j}d|ddS)zFormat a timedelta as ISO 8601 duration string. Args: position: The timedelta to format. Returns: ISO 8601 duration string (e.g., "PT1.5S"). PTz.3fS) total_seconds)rxrrs r9_format_positionz+GenesysAudioHookSerializer._format_positions$!..0 M#&a((r8 position_strc|jdr,|jdr t|dd}t|StdS#t$r YtdSwxYw)zParse an ISO 8601 duration string to timedelta. Args: position_str: ISO 8601 duration string (e.g., "PT1.5S"). Returns: Corresponding timedelta. rr)secondsr) startswithendswithfloatr ValueError)rxrrs r9_parse_positionz*GenesysAudioHookSerializer._parse_positionsj  " "4 (\-B-B3-G  Qr 23 11| | sA AAcD|xjdz c_|jS)z$Get the next server sequence number.)rmr}s r9_next_server_seqz+GenesysAudioHookSerializer._next_server_seq.s Ar8msg_type parametersinclude_positionc|j}|j|j||j|jd}|r|j |j |d<|r||d<|S)aCreate a protocol message with common fields. Based on the Genesys AudioHook protocol, responses include: - seq: Server's sequence number (incremented per message) - clientseq: Echo of the client's last sequence number Args: msg_type: The message type. parameters: Optional parameters object. include_position: Whether to include position field. Returns: The message dictionary. )versiontypeseq clientseqidrr)rPROTOCOL_VERSIONvaluerlrirrp)rxrrrrmsgs r9_create_messagez*GenesysAudioHookSerializer._create_message3sm(##%,,NN))""   "33DNNCC O  *C  r8rSrQrRc$g}|jjtjk(rdg}nV|jjtjk(rdg}n+|jjtj k(rddg}|d|jj j||jdgd}|r||d<|r||d<|jtj|d }d |_ tjd |j|S) aCreate an 'opened' response message for the client. This should be sent in response to an 'open' message from Genesys. Args: start_paused: Whether to start the session paused. supported_languages: List of supported language codes. selected_language: The selected language code. Returns: Dictionary of the opened response message. r<r=audio)rformatchannelsrate) startPausedmediasupportedLanguagesselectedLanguageF)rrTzAudioHook session opened: )rdrMr;r?r@rArNrrerrr,rnr rri)rxrSrQrRrrrs r9create_opened_responsez1GenesysAudioHookSerializer.create_opened_responseXs&! <<  #3#<#< <"|H \\ ! !%5%>%> >"|H \\ ! !%5%:%: :"J/H($"ll77== ( 55    /BJ+ , ->J) *"" ' '!"#    1$2B2B1CDE r8rcd}|rd|i}|jtj|}d|_t j d|j |S)aCreate a 'closed' response message. This should be sent in response to a 'close' message from Genesys. Args: output_variables: Optional custom variables to pass back to Genesys. These will be available in the Architect flow after the AudioHook action completes. Returns: Dictionary of the closed response message. Example: ```python # Pass custom data back to Genesys serializer.create_closed_response( output_variables={ "intent": "billing_inquiry", "customer_verified": True, "summary": "Customer asked about their bill" } ) ``` NoutputVariablesrFzAudioHook session closed: )rrr.rnr rri)rxrrrs r9create_closed_responsez1GenesysAudioHookSerializer.create_closed_responsesa804 +-=>J"" ' '!#    1$2B2B1CDE r8cD|jtj}|S)zCreate a 'pong' response message. This should be sent in response to a 'ping' message from Genesys. Returns: Dictionary of the pong response message. )rrr2rxrs r9create_pong_responsez/GenesysAudioHookSerializer.create_pong_responses ""#7#<#<= r8c|jtj}d|_t j d|j |S)zCreate a 'resumed' response message. This should be sent in response to a 'pause' message when ready to resume. Returns: Dictionary of the resumed response message. FzAudioHook session resumed: )rrr0ror rrirs r9create_resumed_responsez2GenesysAudioHookSerializer.create_resumed_responsesB""#7#?#?@ 243C3C2DEF r8c~|jtjddidgi}tjd|S)aCreate a barge-in event message. This notifies Genesys Cloud that the user has interrupted the bot's audio output. Genesys will stop any queued audio playback. Returns: Dictionary of the barge-in event message. entitiesbarge_in)rdataru#🔇 Barge-in event sent to Genesys)rrr4r rrs r9create_barge_in_eventz0GenesysAudioHookSerializer.create_barge_in_eventsF"" & &"j"%E$FG#   :; r8reasonactioninfocd|i}d|i}|r|j|||d<|r||d<|jtj|}t j d|d||S)aCreate a 'disconnect' message to initiate session termination. Args: reason: Disconnect reason (e.g., "completed", "error"). action: Action to take ("transfer" to agent, "finished" if completed). output_variables: Custom output variables to pass back to Genesys. info: Optional additional information. Returns: Dictionary of the disconnect message. rrrrrzAudioHook disconnect: reason=z , action=)r#rrr6r r)rxrrrrrout_varsrs r9create_disconnect_messagez4GenesysAudioHookSerializer.create_disconnect_messages$'/%7 f%  OO, -(0 $% !%Jv "" + +!#   4VHIfXNO r8codemessage retryablec|||d}|jtj|}tjd|d||S)zCreate an 'error' message. Args: code: Error code. message: Error message. retryable: Whether the operation can be retried. Returns: Dictionary of the error message. )rrrrzAudioHook error:  - )rrr5r r%)rxrrrrrs r9create_error_messagez/GenesysAudioHookSerializer.create_error_message sX"" "" & &!#   (c';< r8chKt|ttfr0tj|j |j dSt|tr|jr |jry|j}|jjtjk(r5t||j |j"|j$d{}nt'j(dy|t+|dk(ryt-|St|t.r#tj|j1St|t2t4frZ|j7|ryt|j8t:r-d|j8vrtj|j8Syy7ۭw)a\Serializes a Pipecat frame to Genesys AudioHook format. Handles conversion of various frame types to AudioHook messages: - AudioRawFrame -> Binary PCMU audio data (resampled to 8kHz) - EndFrame/CancelFrame -> Disconnect message (JSON) - InterruptionFrame -> Barge-in event (JSON) - OutputTransportMessageFrame -> Pass-through JSON Args: frame: The Pipecat frame to serialize. Returns: Serialized data as string (JSON) or bytes (audio), or None if the frame type is not handled or session is not open. completed)rrN$L16 format not yet fully implementedrr) isinstancerrjsondumpsrrrrnrorrdrNrCrDr rLrerkr warninglenbytesrrrrshould_ignore_framerdict)rxrrserialized_datas r9 serializez$GenesysAudioHookSerializer.serialize+sh eh 4 5::..%)%:%:;/  } -==DOO;;D||((,@,E,EE(3%%--** )#EF&#o*>!*C) ) 0 1::d88:; ;  ;=^_ `''.%--.9 3Mzz%--00A#sCF2F0CF2rc2Kt|tr|j|d{S tj|}|j|d{S73#tj $r"}t jd|Yd}~yd}~wwxYw7>w)aDeserializes Genesys AudioHook data to Pipecat frames. Handles: - Binary data -> InputAudioRawFrame (converted from PCMU to PCM) - JSON 'open' -> OutputTransportMessageUrgentFrame with 'opened' response - JSON 'close' -> OutputTransportMessageUrgentFrame with 'closed' response - JSON 'ping' -> OutputTransportMessageUrgentFrame with 'pong' response - JSON 'pause' -> Sets is_paused=True, returns None - JSON 'dtmf' -> InputDTMFFrame - JSON 'update' -> Updates participant info, returns None - JSON 'error' -> Logs error, returns None Protocol responses (opened, closed, pong) are returned as urgent frames to be sent immediately through the transport. Args: data: The raw WebSocket data from Genesys (binary audio or JSON text). Returns: A Pipecat frame to process, or None if handled internally. Nz#Failed to parse AudioHook message: ) rr_deserialize_audiorloadsJSONDecodeErrorr r%_handle_control_message)rxrres r9 deserializez&GenesysAudioHookSerializer.deserializels. dE "0066 6 jj&G 11':::7 ##  LL>qcB C ;sD%BABABBBB0B B BBc K|jr |jry}t}|jjt j k(r[tdkDrMtfdtdtdD}tjd|dt|d|jjtjk(r5t||j|j |j"d{}ntj$dy|t|dk(ryd }t'|||j }|S7Hw) zDeserialize binary audio data to an InputAudioRawFrame. Args: data: Raw audio bytes (PCMU or L16). Returns: InputAudioRawFrame with PCM audio at pipeline sample rate. Nrc3(K|] }| ywNr7).0irs r9 z@GenesysAudioHookSerializer._deserialize_audio..sG1tAwGsru🔊 Stereo audio: u bytes → z bytes (external channel)rr)r num_channelsrL)rnrorrdrMr;rArranger rrNrCrDr rerfrjrr)rxr audio_data original_lendeserialized_datar audio_frames ` r9rz-GenesysAudioHookSerializer._deserialize_audios-}} 4y  <<  #3#8#8 8SY]GaTA0FGGJ LL%l^;s:>OOhi  << $ $(<(A(A A&1))!!%% '!  NNA B  $,=(>!(C (#%)) /!sC?E E A E cK|jdd}|jdd|_d|vr|j|d|_|tj j k(r|j|d{S|tjj k(r|j|d{S|tjj k(r|j|d{S|tjj k(r|j|d{S|tjj k(r|j|d{S|tj j k(r|j#|d{S|dk(r|j%|d{S|dk(rt'j(d y|d k(rt'j(d yt'j*d |y77M7777~7bw) zHandle a JSON control message from Genesys. Args: message: Parsed JSON message. Returns: Frame if the message should be passed to the pipeline, None otherwise. rrrrNdtmfplayback_startedzPlayback started (from Genesys)playback_completedz!Playback completed (from Genesys)z Unknown AudioHook message type: )getrlrrprr+r _handle_openr- _handle_closer1 _handle_pingr/ _handle_pauser3_handle_updater5 _handle_error _handle_dtmfr rr)rxrrs r9rz2GenesysAudioHookSerializer._handle_control_messages;;vr*";;ua0  !11'*2EFDN +0066 6**733 3 -3399 9++G44 4 -2288 8**733 3 -3399 9++G44 4 -44:: :,,W55 5 -3399 9++G44 4  **733 3 + + LL: ; - - LL< = NN=hZH I;4545654sA8H :G:;5H 0G=15H &H'5H H5H H5H H H &H 'AH =H H H H H  H c K|jdttj|_|jdi}|jd|_|jd|_|jd|_|jd|_|jd|_ |j}|r1t|tr t|dkDr|d}|jd g}tjd |jd d |d |jdt|trd|vr9d|vr5tj |j"_tjdnqd|vr5tj&|j"_tjdn8d|vr4tj(|j"_tjd|j r|j jddnd}tj*d|jd|j d||j-d|d{t/|j1|j"j2|j"j4|j"j6S7^w)aHandle an 'open' message from Genesys. This initializes the session with metadata from Genesys Cloud and automatically responds with an 'opened' message using the configured InputParams (supported_languages, selected_language, start_paused). Extracts and stores: - session_id: The AudioHook session identifier - conversation_id: The Genesys conversation ID - participant: Caller info (ani, dnis, etc.) - input_variables: Custom variables from Architect flow - media_info: Audio configuration from Genesys Args: message: The open message from Genesys. Returns: OutputTransportMessageUrgentFrame with the 'opened' response. rrconversationIdr customConfigrinputVariablesrru"📡 Genesys audio config: format=rz , channels=z, rate=rr<r=u-📡 Stereo mode: extracting external channelu 📡 Mono mode: external channelu 📡 Mono mode: internal channelaniunknownz AudioHook open request: session=z, conversation=z, ani=r[N)rSrQrRr)rrWrgrhrirqrrrsrtrurlistrr rr;rArdrMr?r@r_call_event_handlerrrrSrQrR)rxrrY media_list audio_mediarr s r9rz'GenesysAudioHookSerializer._handle_opens^(#;;tS->?\2. & +; <"JJ}5$jj8!::g. & +; <%% *Z63z?Q;N*4Q-K"z26H LL4[__X5N4O{[cZddklwl{l{}CmDlEF (D))jH.D+;+@+@DLL(LL!PQ8++;+D+DDLL(LL!CD8++;+D+DDLL(LL!CD:>9J9Jd##E95PY .t/?/?.@A 112& ? &&y':::0//!\\66$(LL$D$D"&,,"@"@0   ;sI$K&K'AKc>K|jdi}|jdd}tjd|d|_tjd|j d|d{t |j |j  S7*w) aHandle a 'close' message from Genesys. Automatically responds with a 'closed' message. If output_variables were set via set_output_variables(), they will be included in the response and made available in the Architect flow. Args: message: The close message from Genesys. Returns: OutputTransportMessageUrgentFrame with the closed response (includes outputVariables if set). rrr u$🔴 Genesys closed the connection: Fz%Sending closed response to Genesys...r\N)rr )rr rrnr rrrvrxrrYrs r9rz(GenesysAudioHookSerializer._handle_close6s\2.Hi0 :6(CD  ;=&&z7;;;1//AWAW/X  ":&&;d38n;;.__ ) )s )CI$ # 04!% #&#T#s(^,# # c3h #N#37+/ 77&d3i07$C= 7 c3h 7v6:)"4S>2) c3h)V d38n  c3h tCH~(" 59" ####4S>2 # sm # c3h #R   c3h  edl >V$sCx.VUT\V$4S>edl0DcNut|,4S>edl&$sCx.UT\r8rG)'r*rrgdatetimerenumrtypingrrrrlogurur pipecat.audio.dtmf.typesr .pipecat.audio.resamplers.soxr_stream_resamplerr pipecat.audio.utilsr r pipecat.frames.framesrrrrrrrrrr#pipecat.serializers.base_serializerrrWrr;rCrGr7r8r9r%sw, ,,0S8   @3"sD3s s r8