qis~dZddlZddlmZddlmZmZmZmZm Z ddl Z ddl m Z ddl mZddlmZddlmZmZmZmZmZmZmZmZmZmZmZdd lmZmZm Z dd l!m"Z"dd l#m$Z$dd l%m&Z&m'Z'dd l(m)Z)m*Z*m+Z+GddZ,GddeZ-Gdde*Z.GddZ/Gdde"Z0Gdde$Z1Gdde&Z2y)aTavus transport implementation for Pipecat. This module provides integration with the Tavus platform for creating conversational AI applications with avatars. It manages conversation sessions and provides real-time audio/video streaming capabilities through the Tavus API. N)partial)Any AwaitableCallableMappingOptional) AudioData)logger) BaseModel) BotConnectedFrame CancelFrameClientConnectedFrameEndFrameFrameInputAudioRawFrameInterruptionFrameOutputAudioRawFrameOutputTransportMessageFrame!OutputTransportMessageUrgentFrame StartFrame)FrameDirectionFrameProcessorFrameProcessorSetup)BaseInputTransport)BaseOutputTransport) BaseTransportTransportParams)DailyCallbacks DailyParamsDailyTransportClientcpeZdZdZdZdZdZdedejfdZ ded ed e fd Z d efd Z d ed efdZy)TavusApizHelper class for interacting with the Tavus API (v2). Provides methods for creating and managing conversations with Tavus avatars, including conversation lifecycle management and persona information retrieval. zhttps://tavusapi.com/v2zdev-conversationTestTavusTransportapi_keysessionc|||_||_d|jd|_tjd|_y)zInitialize the TavusApi client. Args: api_key: Tavus API key for authentication. session: An aiohttp session for making HTTP requests. zapplication/json)z Content-Typez x-api-keyTAVUS_SAMPLE_ROOM_URLN)_api_key_session_headersosgetenv _dev_room_url)selfr$r%s T/opt/pipecat/venv/lib/python3.12/site-packages/pipecat/transports/tavus/transport.py__init__zTavusApi.__init__:s4   );$--X YY'>? replica_id persona_idreturncK|jr|j|jdStjd|d||jd}||d}|j j ||j|4d{}|j|jd{}tjd||cdddd{S7W717 #1d{7swYyxYww) a?Create a new conversation with the specified replica and persona. Args: replica_id: ID of the replica to use in the conversation. persona_id: ID of the persona to use in the conversation. Returns: Dictionary containing conversation_id and conversation_url. )conversation_idconversation_urlz%Creating Tavus conversation: replica=z , persona=z/conversations)r2r3)headersjsonNzCreated Tavus conversation: ) r-MOCK_CONVERSATION_IDr debugBASE_URLr)postr*raise_for_statusr9)r.r2r3urlpayloadrresponses r/create_conversationzTavusApi.create_conversationGs   #'#<#<$($6$6   r r;)r.r6r?rAs r/end_conversationzTavusApi.end_conversationcs  "o9R9R&R .?tD==%%c4==%A H HQ   LL4_4EF G H H H H H H HsHAB)BB))B? B) B B)B)B&B B&"B)cK|j |jS|jd|}|jj ||j 4d{}|j |jd{}tjd||dcdddd{S7Z747 #1d{7swYyxYww)zGet the name of a persona by ID. Args: persona_id: ID of the persona to retrieve. Returns: The name of the persona. Nz /personas/rEzFetched Tavus persona: persona_name) r-MOCK_PERSONA_NAMEr<r)getr*r>r9r r;)r.r3r?rArBs r/get_persona_namezTavusApi.get_persona_nameqs    ))) )z*6==$$S$--$@ , ,A   VVX~H LL28*= >N+  , , ,% , , , ,sZAC B2C $B8?B4 B8 C ,B6-C 4B86C 8C >C ?C C N)__name__ __module__ __qualname____doc__r<r:rIstraiohttp ClientSessionr0dictrCrFrKr1r/r"r"/sq )H-, @ @g.C.C @CST8 Hc H,,,r1r"ceZdZUdZeeeefgedfe d<eeeefgedfe d<eeeefegedfe d<y)TavusCallbacksaCallback handlers for Tavus events. Parameters: on_joined: Called when the bot joins the Daily room. on_participant_joined: Called when a participant joins the conversation. on_participant_left: Called when a participant leaves the conversation. N on_joinedon_participant_joinedon_participant_left) rLrMrNrOrrrPrr__annotations__rTr1r/rVrVskc*+Yt_<==#WS#X%6$74$HII!738#4c":IdO"KLLr1rVc<eZdZUdZdZeed<dZeed<dZeed<y) TavusParamsa(Configuration parameters for the Tavus transport. Parameters: audio_in_enabled: Whether to enable audio input from participants. audio_out_enabled: Whether to enable audio output to participants. microphone_out_enabled: Whether to enable microphone output track. Taudio_in_enabledaudio_out_enabledFmicrophone_out_enabledN) rLrMrNrOr]boolrZr^r_rTr1r/r\r\s*"d!"t"#(D(r1r\c^eZdZdZedddededededed ed ejd d fd Z d efdZ de fdZ dZ dZdZdZd efdZdefdZdZ d+dededededef dZ d,dededed ed!ef d"Zdeezfd#Zed efd$Zed efd%Zd-d&Zd.d'Z de!d e"fd(Z#d)efd*Z$y )/TavusTransportClientaTransport client that integrates Pipecat with the Tavus platform. A transport client that integrates a Pipecat Bot with the Tavus platform by managing conversation sessions using the Tavus API. This client uses `TavusApi` to interact with the Tavus backend services. When a conversation is started via `TavusApi`, Tavus provides a `roomURL` that can be used to connect the Pipecat Bot into the same virtual room where the TavusBot is operating. pipecat-stream)paramsr3bot_namerd callbacksr$r2r3r%r4Nc||_t|||_||_||_d|_d|_||_||_y)aInitialize the Tavus transport client. Args: bot_name: The name of the Pipecat bot instance. params: Optional parameters for Tavus operation. callbacks: Callback handlers for Tavus-related events. api_key: API key for authenticating with Tavus API. replica_id: ID of the replica to use in the Tavus conversation. persona_id: ID of the Tavus persona. Defaults to "pipecat-stream", which signals Tavus to use the TTS voice of the Pipecat bot instead of a Tavus persona voice. session: The aiohttp session for making async HTTP requests. N) _bot_namer"_api _replica_id _persona_id_conversation_id_client _callbacks_params)r.rerdrfr$r2r3r%s r/r0zTavusTransportClient.__init__sF0"Wg. %%/37; # r1cK|jj|j|jd{}|d|_|dS7w)z4Initialize the conversation and return the room URL.Nr6r7)rirCrjrkrl)r.rBs r/ _initializez TavusTransportClient._initializesK66t7G7GIYIYZZ (): ;*++[s4A A A setupc K|j#tjd|jy |jd{}t d!idt |j dd|jd|jdt |j ddt |j ddt |j dd t |j d d t |j d d t |j d d t |j d d t |j d dt |j ddt |j ddt |j ddt |j ddt |j ddt |j ddt |j ddt |j dd|jjd|jjdt |j ddt |j ddt |j ddt |j ddt |j ddt |j ddt |j d}t|dd|j||j|_|jj!|d{y77#t"$rW}tj$d ||j&j)|jd{7d|_Yd}~yd}~wwxYww)"zSetup the client and initialize the conversation. Args: setup: The frame processor setup configuration. Nz!Conversation ID already defined: on_active_speaker_changedrWon_lefton_before_leaveon_erroron_app_messageon_call_state_updatedon_client_connectedon_client_disconnectedon_dialin_connectedon_dialin_readyon_dialin_stoppedon_dialin_erroron_dialin_warningon_dialout_answeredon_dialout_connectedon_dialout_stoppedon_dialout_erroron_dialout_warningrXrYon_participant_updatedon_transcription_messageon_recording_startedon_recording_stoppedon_recording_erroron_transcription_stoppedon_transcription_errorPipecatz&Failed to setup TavusTransportClient: rT)rlr r;rqrr_on_handle_callback _on_joined_on_leftrnrXrYr rorhrmrr ExceptionerrorrirF)r.rrroom_urldaily_callbackses r/rrzTavusTransportClient.setupsu  , LL*1,,.H*?#D(/t/G/GIa'bE#OH0$ 4<<$..DL,,$$U+ + +Q0P , ) LLA!E F)),,T-B-BC C C$(D ! ! )s`0M9LLKL L LM9LL M6AM1M" M1,M91M66M9cK |jjd{y7#t$r"}tjd|Yd}~yd}~wwxYww)zCleanup client resources.NzException during cleanup: )rmcleanuprr r)r.rs r/rzTavusTransportClient.cleanup sG ;,,&&( ( ( ; LL5aS9 : : ;s6A)')A) AA AAAcKtjd|jj|d{y7w)zHandle joined event.zTavusTransportClient joined!N)r r;rnrWr.datas r/rzTavusTransportClient._on_joineds+ 34oo''---s 4><>c6Ktjdyw)zHandle left event.zTavusTransportClient left!N)r r;r.s r/rzTavusTransportClient._on_lefts 12scHKtjd|d|d|yw)zHandle generic callback events.z [Callback] z called with args=z , kwargs=N)r trace)r. event_nameargskwargss r/rz(TavusTransportClient._on_handle_callbacks' {:,.@iPVxXYs "chK|jj|jd{S7w)zjGet the persona name from the API. Returns: The name of the current persona. N)rirKrkrs r/rKz%TavusTransportClient.get_persona_names) YY//0@0@AAAAs )202framecKtjd|jj|d{|jj d{y7'7w)zStart the client and join the room. Args: frame: The start frame containing initialization parameters. z#TavusTransportClient start invoked!N)r r;rmstartjoinr.rs r/rzTavusTransportClient.start'sL  :;ll  '''ll!!! (!s!4A"A!A"A A" A"cK|jjd{|jj|jd{d|_y797w)z)Stop the client and end the conversation.N)rmleaverirFrlrs r/stopzTavusTransportClient.stop1sKll  """ii(()>)>??? $ #?s!AA,A A AAparticipant_idcallback framerate video_source color_formatc^K|jj|||||d{y7w)aoCapture video from a participant. Args: participant_id: ID of the participant to capture video from. callback: Callback function to handle video frames. framerate: Desired framerate for video capture. video_source: Video source to capture from. color_format: Color format for video frames. N)rmcapture_participant_video)r.rrrrrs r/rz.TavusTransportClient.capture_participant_video7s/"ll44 Hi|   #-+- audio_source sample_ratecallback_interval_msc^K|jj|||||d{y7w)aCapture audio from a participant. Args: participant_id: ID of the participant to capture audio from. callback: Callback function to handle audio data. audio_source: Audio source to capture from. sample_rate: Desired sample rate for audio capture. callback_interval_ms: Interval between audio callbacks in milliseconds. N)rmcapture_participant_audio)r.rrrrrs r/rz.TavusTransportClient.capture_participant_audioLs0"ll44 HlKAU   rcVK|jj|d{y7w)eSend a message to participants. Args: frame: The message frame to send. N)rm send_messagers r/rz!TavusTransportClient.send_messageas ll''... )')c.|jjS)z`Get the output sample rate. Returns: The output sample rate in Hz. )rmout_sample_raters r/rz$TavusTransportClient.out_sample_rateks||+++r1c.|jjS)z^Get the input sample rate. Returns: The input sample rate in Hz. )rmin_sample_raters r/rz#TavusTransportClient.in_sample_ratets||***r1cvKtdd|jd}|j|d{y7w)z.Send an interrupt message to the conversation. conversationzconversation.interrupt) message_type event_typer6)messageN)rrlr)r.transport_frames r/send_interrupt_messagez+TavusTransportClient.send_interrupt_message}s:; .6#'#8#8 000s /979ctK|jsy|jj||d{y7w)Update subscription settings for participants. Args: participant_settings: Per-participant subscription settings. profile_settings: Global subscription profile settings. Nparticipant_settingsprofile_settingsrmupdate_subscriptionsr.rrs r/rz)TavusTransportClient.update_subscriptionss9|| ll//!5HX0   s .868cnK|jsy|jj|d{S7w)zWrite an audio frame to the transport. Args: frame: The audio frame to write. Returns: True if the audio frame was written successfully, False otherwise. FN)rmwrite_audio_framers r/rz&TavusTransportClient.write_audio_frames-||\\33E::::s ,535 destinationcpK|jsy|jj|d{y7w)zRegister an audio destination for output. Args: destination: The destination identifier to register. Nrmregister_audio_destinationr.rs r/rz/TavusTransportClient.register_audio_destinations* || ll55kBBBs ,646)cameraRGB) microphonei>)r4NNN)%rLrMrNrOr\rPrVrQrRr0rqrrrrrrrrKrrrrintrrrrrpropertyrrrrrr`rrrTr1r/rbrbs*m+  "   && B,3, 6)!46)p;. 3ZBB""%$!          2) $&        "  */03TT/,,,+++ 1   ;-@ ;T ; CC Cr1rbceZdZdZdedeffd Zdeffd ZfdZ de ffd Z de ffd Z deffd Zd Zd ededefdZxZS)TavusInputTransportzInput transport for receiving audio and events from Tavus conversations. Handles incoming audio streams from participants and manages audio capture from the Daily room connected to the Tavus conversation. clientrdc Pt||fi|||_||_d|_y)zInitialize the Tavus input transport. Args: client: The Tavus transport client instance. params: Transport configuration parameters. **kwargs: Additional arguments passed to parent class. FN)superr0rmro _initializedr.rrdr __class__s r/r0zTavusInputTransport.__init__s- *6*  !r1rrcKt||d{|jj|d{y7(7w)znSetup the input transport. Args: setup: The frame processor setup configuration. Nrrrrmr.rrrs r/rrzTavusInputTransport.setup< gmE"""ll  ''' #'A?"AAAAcKt|d{|jjd{y7'7w)z"Cleanup input transport resources.Nrrrmr.rs r/rzTavusInputTransport.cleanup6goll""$$$ $A=!A?AArcKt||d{|jryd|_|jj|d{|j |d{y7U7 7 w)z{Start the input transport. Args: frame: The start frame containing initialization parameters. NT)rrrrmset_transport_readyr.rrs r/rzTavusInputTransport.startsj gmE"""     ll  '''&&u--- # (-s3A2A,6A2 A.A2&A0'A2.A20A2cKt||d{|jjd{y7'7w)zpStop the input transport. Args: frame: The end frame signaling transport shutdown. Nrrrmrs r/rzTavusInputTransport.stop: gl5!!!ll!!! "!A>!AAAAcKt||d{|jjd{y7'7w)zyCancel the input transport. Args: frame: The cancel frame signaling immediate cancellation. Nrcancelrmrrs r/rzTavusInputTransport.cancel: gnU###ll!!! $!rcK|jjrctjd|d|jj |d|j |jjd{yy7w)zStart capturing audio from a participant. Args: participant: The participant to capture audio from. z;TavusTransportClient start capturing audio for participant id)rrrN)ror]r informr_on_participant_audio_datarr. participants r/start_capturing_audioz)TavusInputTransport.start_capturing_audios} << ( ( KKMkZ^N_M`a ,,88*4088 LL779   ) sA4A?6A=7A?raudiorcKt|j|j|j}||_|j |d{y7w)z'Handle received participant audio data.)rr num_channelsN)r audio_framesrrtransport_sourcepush_audio_frame)r.rrrrs r/rz.TavusInputTransport._on_participant_audio_data sK#$$))++  ".##E***sAA A A)rLrMrNrOrbrr0rrrrrrrrr rrrPr r __classcell__rs@r/rrsw "$" "&(!4(% .. """+" +! +*3 +CF +r1rceZdZdZdedeffd Zdeffd ZfdZ de ffd Z de ffd Z deffd Zdeezfd Zded effd ZdZdedefdZdefdZxZS)TavusOutputTransportzOutput transport for sending audio and events to Tavus conversations. Handles outgoing audio streams to participants and manages the custom audio track expected by the Tavus platform. rrdc ^t||fi|||_||_d|_d|_y)zInitialize the Tavus output transport. Args: client: The Tavus transport client instance. params: Transport configuration parameters. **kwargs: Additional arguments passed to parent class. FstreamN)rr0rmror_transport_destinationrs r/r0zTavusOutputTransport.__init__s7 *6*  "5=#r1rrcKt||d{|jj|d{y7(7w)zoSetup the output transport. Args: setup: The frame processor setup configuration. Nrrs r/rrzTavusOutputTransport.setup3rrcKt|d{|jjd{y7'7w)z#Cleanup output transport resources.Nrrs r/rzTavusOutputTransport.cleanup<rrrc^Kt||d{|jryd|_|jj|d{|jr-|jj |jd{|j |d{y77Y7"7 w)z|Start the output transport. Args: frame: The start frame containing initialization parameters. NT)rrrrmrrrrs r/rzTavusOutputTransport.startAs gmE"""     ll  '''  & &,,99$:U:UV V V&&u--- # ( W-sEB-B%6B- B'8B-B)B-B+ B-'B-)B-+B-cKt||d{|jjd{y7'7w)zqStop the output transport. Args: frame: The end frame signaling transport shutdown. Nrrs r/rzTavusOutputTransport.stopUrrcKt||d{|jjd{y7'7w)zzCancel the output transport. Args: frame: The cancel frame signaling immediate cancellation. Nrrs r/rzTavusOutputTransport.cancel^rrcKtjd||jj|d{y7w)rz%TavusOutputTransport sending message N)r rrmrrs r/rz!TavusOutputTransport.send_messagegs4  ;E7CDll''...s7A?A directioncKt|||d{t|tr|j d{yy7.7w)zProcess frames and handle interruptions. Args: frame: The frame to process. direction: The direction of frame flow in the pipeline. N)r process_frame isinstancer_handle_interruptions)r.rrrs r/rz"TavusOutputTransport.process_framersIg#E9555 e. /,,. . . 0 6 .s A A'A AA A cTK|jjd{y7w)z8Handle interruption events by sending interrupt message.N)rmrrs r/rz*TavusOutputTransport._handle_interruptions}sll11333s (&(r4cvK|j|_|jj|d{S7w)zWrite an audio frame to the Tavus transport. Args: frame: The audio frame to write. Returns: True if the audio frame was written successfully, False otherwise. N)rtransport_destinationrmrrs r/rz&TavusOutputTransport.write_audio_frames1'+&A&A#\\33E::::s 0979rcVK|jj|d{y7w)zwRegister an audio destination. Args: destination: The destination identifier to register. Nrrs r/rz/TavusOutputTransport.register_audio_destinations ll55kBBBr)rLrMrNrOrbrr0rrrrrrrrr rrrrrrrrrr`rrPrrr s@r/r r s >$> >,(!4(% ..("""+" /03TT / / /> /4 ;-@ ;T ;CCCr1r ceZdZdZdeddfdedejdededed ed eed eeffd Z d Z dZ dZ ddZ defdZdefdZdefdZdefdZxZS)TavusTransportaTransport implementation for Tavus video calls. When used, the Pipecat bot joins the same virtual room as the Tavus Avatar and the user. This is achieved by using `TavusTransportClient`, which initiates the conversation via `TavusApi` and obtains a room URL that all participants connect to. Event handlers available: - on_connected(transport, data): Bot connected to the room - on_client_connected(transport, participant): Participant connected to the session - on_client_disconnected(transport, participant): Participant disconnected from the session Example:: @transport.event_handler("on_client_connected") async def on_client_connected(transport, participant): ... rcNrer%r$r2r3rd input_name output_namec Lt |||||_t|j|j |j } td| ||||||_d|_ d|_ d|_ |jd|jd|jdy) aqInitialize the Tavus transport. Args: bot_name: The name of the Pipecat bot. session: aiohttp session used for async HTTP requests. api_key: Tavus API key for authentication. replica_id: ID of the replica model used for voice generation. persona_id: ID of the Tavus persona. Defaults to "pipecat-stream" to use the Pipecat TTS voice. params: Optional Tavus-specific configuration parameters. input_name: Optional name for the input transport. output_name: Optional name for the output transport. )r r!)rWrXrYr)rerfr$r2r3r%rdN on_connectedrzr{) rr0rorVr_on_participant_joined_on_participant_leftrbrm_input_output_tavus_participant_id_register_event_handler) r.rer%r$r2r3rdr r!rfrs r/r0zTavusTransport.__init__s0 JKH "oo"&"="= $ 9 9 ,!!  6: 7; %)" $$^4 $$%:; $$%=>r1cK|jd|d{|jr,|jjtd{yy7=7w)zHandle bot joined room event.r#N)_call_event_handlerr& push_framer rs r/rzTavusTransport._on_joinedsM&&~t<<< ;;++(():)<= = =  = =!AA6AAAAcK|jjd{}|jdijdd|k7r|j|d{yy7D7w)zHandle participant left events.NruserName)rmrKrJ_on_client_disconnected)r.rreasonrHs r/r%z#TavusTransport._on_participant_lefts^!\\::<< ??62 & * *:r :l J..{; ; ; K= ;s!A)A%=A)A'A)'A)cK|jjd{}|jdijdd|k(r |d|_y|j |d{|jrMt j d|jd|j|jdd d iii d{|jr$|jj|d{yy77797 w) z!Handle participant joined events.Nrr/r0rz Ignoring z 's microphonemediar unsubscribed)r) rmrKrJr(_on_client_connectedr r;rr&r)r.rrHs r/r$z%TavusTransport._on_participant_joineds"\\::<<  ??62 & * *:r :l J)4T):D &++K8 8 8)) y)C)C(DMRS//22#lN%C5*0{{kk77 DDD!= 9EsGDC8AD)C:*ADC<.D1C>2D:D<D>DcZK|jj||d{y7w)rrNrrs r/rz#TavusTransport.update_subscriptionss/ll//!5-0   s !+)+r4c~|js&t|j|j|_|jS)zGet the input transport for receiving media and events. Returns: The Tavus input transport instance. rrd)r&rrmrors r/inputzTavusTransport.input s+ {{-T\\$,,WDK{{r1c~|js&t|j|j|_|jS)zGet the output transport for sending media and events. Returns: The Tavus output transport instance. r9)r'r rmrors r/outputzTavusTransport.outputs+ ||/t||DLLYDL||r1rcK|jd|d{|jr,|jjtd{yy7=7w)zHandle client connected events.rzN)r+r&r,rrs r/r6z#TavusTransport._on_client_connected sO&&' < E,  ~AcA NNr1r)3rOr+ functoolsrtypingrrrrrrQ daily.dailyr logurur pydanticr pipecat.frames.framesr r rrrrrrrrr"pipecat.processors.frame_processorrrrpipecat.transports.base_inputrpipecat.transports.base_outputr!pipecat.transports.base_transportrr"pipecat.transports.daily.transportrrr r"rVr\rbrr rrTr1r/rJs >>!    cb<>LS,S,l MY M )+ )KCKC\d+,d+N~C.~CBQN]QNr1