qitdZddlmZddlmZmZmZmZmZddl Z ddl m Z ddl m Z ddlmZddlmZmZmZmZmZmZmZmZmZmZmZdd lmZmZmZdd l m!Z!dd l"m#Z#dd l$m%Z%m&Z&dd l'm(Z(m)Z)m*Z*ddl+m,Z,GddeZ-GddeZ.Gdde)Z/GddZ0Gdde!Z1Gdde#Z2Gdde%Z3y)zLemonSlice transport for Pipecat. This module adds LemonSlice avatars to Daily rooms, enabling real-time voice conversations with synchronized avatars. )partial)Any AwaitableCallableMappingOptionalN) AudioData)logger) BaseModel) BotStartedSpeakingFrameBotStoppedSpeakingFrame CancelFrameEndFrameFrameInputAudioRawFrameInterruptionFrameOutputAudioRawFrameOutputTransportMessageFrame!OutputTransportMessageUrgentFrame StartFrame)FrameDirectionFrameProcessorFrameProcessorSetup)BaseInputTransport)BaseOutputTransport) BaseTransportTransportParams)DailyCallbacks DailyParamsDailyTransportClient) LemonSliceApiceZdZUdZdZeeed<dZeeed<dZ eeed<dZ ee ed<dZ eeed<dZ eeed<dZeeed <y) LemonSliceNewSessionRequestaRequest model for creating a new LemonSlice session. Parameters: agent_image_url: URL to an agent image. Provide either agent_id or agent_image_url. agent_id: ID of a LemonSlice agent. Provide either agent_id or agent_image_url. agent_prompt: A high-level system prompt that subtly influences the avatar's movements, expressions, and emotional demeanor. idle_timeout: Idle timeout in seconds. daily_room_url: Daily room URL to use for the session. daily_token: Daily token for authenticating with the room. lemonslice_properties: Additional properties to pass to the session. Nagent_image_urlagent_id agent_prompt idle_timeoutdaily_room_url daily_tokenlemonslice_properties)__name__ __module__ __qualname____doc__r$rstr__annotations__r%r&r'intr(r)r*dictY/opt/pipecat/venv/lib/python3.12/site-packages/pipecat/transports/lemonslice/transport.pyr#r#.sl &*OXc])"Hhsm""&L(3-&"&L(3-&$(NHSM(!%K#%,08D>0r4r#c`eZdZUdZeeeefgedfe d<eeeefegedfe d<y)LemonSliceCallbackszCallback handlers for LemonSlice events. Parameters: on_participant_joined: Called when a participant joins the conversation. on_participant_left: Called when a participant leaves the conversation. Non_participant_joinedon_participant_left) r+r,r-r.rrr/rrr0r3r4r5r7r7EsK$WS#X%6$74$HII!738#4c":IdO"KLLr4r7c<eZdZUdZdZeed<dZeed<dZeed<y)LemonSliceParamsa-Configuration parameters for the LemonSlice transport. Parameters: audio_in_enabled: Whether to enable audio input from participants. audio_out_enabled: Whether to enable audio output to participants. microphone_out_enabled: Whether to enable microphone output track. Taudio_in_enabledaudio_out_enabledFmicrophone_out_enabledN) r+r,r-r.r<boolr0r=r>r3r4r5r;r;Qs*"d!"t"#(D(r4r;cpeZdZdZedddededededeed e jd dfd Z d efd Z d e fdZdZdZdZdZd efdZdefdZdZ d+dededededef dZ d,dededededef d Zdeezfd!Zed efd"Zed efd#Z d-d$Z!d-d%Z"d-d&Z#d.d'Z$de%d e&fd(Z'd)efd*Z(y)/LemonSliceTransportClientaTransport client that integrates Pipecat with the LemonSlice platform. A transport client that integrates a Pipecat Bot with the LemonSlice platform by managing conversation sessions using the LemonSlice API. This client uses `LemonSliceApi` to interact with the LemonSlice backend. LemonSlice either provides a room URL where the avatar is already present, or adds the LemonSlice avatar to a Daily room the user supplies. N)paramssession_requestbot_namerB callbacksapi_keyrCsessionreturnc||_t|||_|xs t|_d|_d|_d|_||_||_ y)aInitialize the LemonSlice transport client. Args: bot_name: The name of the Pipecat bot instance. params: Optional parameters for LemonSlice operation. callbacks: Callback handlers for LemonSlice-related events. api_key: API key for authenticating with LemonSlice API. session_request: Optional session creation parameters. If not provided, a default agent will be used. session: The aiohttp session for making async HTTP requests. N) _bot_namer!_apir#_session_request _session_id _control_url_daily_transport_client _callbacks_params)selfrDrBrErFrCrGs r5__init__z"LemonSliceTransportClient.__init__jsP*"!'73 / P3N3P*.+/GK$# r4c K|jj|jj|jj|jj |jj |jj|jj|jjd{}|d|_ |d|_ |dS7w)z4Initialize the conversation and return the room URL.)r$r%r&r'r(r) propertiesN session_id control_urlroom_url) rKcreate_sessionrLr$r%r&r'r(r)r*rMrN)rRresponses r5 _initializez%LemonSliceTransportClient._initializes11 11AA**33..;;..;;00??--99,,BB2  $L1$]3 ## sB2C4C5Csetupc BK|j#tjd|jy |jd{}t d!idt |j dd|jd|jdt |j ddt |j ddt |j dd t |j d d t |j d d t |j d d t |j d d t |j d dt |j ddt |j ddt |j ddt |j ddt |j ddt |j ddt |j ddt |j dd|jjd|jjdt |j ddt |j ddt |j ddt |j ddt |j ddt |j ddt |j d}t|d|j|j|d|_|jj!|d{y77#t"$r}}tj$d ||jrE|j&r9|j(j+|j|j&d{7d|_d|_d}~wwxYww)"zSetup the client and initialize the conversation. Args: setup: The frame processor setup configuration. NzSession ID already defined: on_active_speaker_changed on_joinedon_lefton_before_leaveon_erroron_app_messageon_call_state_updatedon_client_connectedon_client_disconnectedon_dialin_connectedon_dialin_readyon_dialin_stoppedon_dialin_erroron_dialin_warningon_dialout_answeredon_dialout_connectedon_dialout_stoppedon_dialout_erroron_dialout_warningr8r9on_participant_updatedon_transcription_messageon_recording_startedon_recording_stoppedon_recording_erroron_transcription_stoppedon_transcription_errorLemonSlicePipecatz+Failed to setup LemonSliceTransportClient: r3)rMr debugr[rr_on_handle_callback _on_joined_on_leftrPr8r9r rJrQrOr\ ExceptionerrorrNrK end_session)rRr\rXdaily_callbackses r5r\zLemonSliceTransportClient.setups    ' LL78H8H7IJ K 0 !--//H,#*1,,.I+#// #  # !((@(@BS T #!!9!9:F# 't'?'?AQR#'.d.F.FH_&`#%,D,D,DF[$\#(/t/G/GIa'b#%,D,D,DF[$\#!((@(@BS T##*$*B*BDW"X#!((@(@BS T# #*$*B*BDW"X!#"%,D,D,DF[$\##$&-T-E-EG]%^%#&$+4+C+CEY#Z'#("))A)ACU!V)#*$+4+C+CEY#Z+#,'+oo&K&K-#.%)OO$G$G/#0(/t/G/GIa'b1#2*1,,.H*3#8&-T-E-EG]%^9#:&-T-E-EG]%^;#<$+4+C+CEY#Z=#>*1,,.H*?#D(/t/G/GIa'bE#OH,@$ oOb,D (..44U; ; ;Q0P <  LLFqcJ KD$5$5ii++D,<,O>OPPP#D  $D    sZ0NLLKL L LNLL NA#NNNNNcK |jr#|jjd{yy7#t$r"}tjd|Yd}~yd}~wwxYww)zCleanup client resources.NzException during cleanup: )rOcleanupr}r r~)rRrs r5rz!LemonSliceTransportClient.cleanupsZ ;++22::<<<,< ; LL5aS9 : : ;s6A$)646A$6 A!AA$A!!A$c6Ktjdyw)zHandle joined event.z!LemonSliceTransportClient joined!Nr ry)rRdatas r5r{z$LemonSliceTransportClient._on_joineds 89c6Ktjdyw)zHandle left event.zLemonSliceTransportClient left!NrrRs r5r|z"LemonSliceTransportClient._on_lefts 67rcHKtjd|d|d|yw)zHandle generic callback events.z [Callback] z called with args=z , kwargs=N)r trace)rR event_nameargskwargss r5rzz-LemonSliceTransportClient._on_handle_callbacks' {:,.@iPVxXYs "c Kyw)zzGet the name of the LemonSlice participant. Returns: The name of the LemonSlice participant. LemonSlicer3rs r5 get_bot_namez&LemonSliceTransportClient.get_bot_names sframecK|jj|d{|jjd{y7'7w)zStart the client and join the room. Args: frame: The start frame containing initialization parameters. N)rOstartjoinrRrs r5rzLemonSliceTransportClient.startsC **00777**//111 81s!A A !A A A  A cK|jjd{|jrD|jr8|jj |j|jd{d|_d|_y7c7w)z)Stop the client and end the conversation.N)rOleaverMrNrKrrs r5stopzLemonSliceTransportClient.stopsn**00222    1 1))''(8(8$:K:KL L L  3 Ls"BBAB0B1BBparticipant_idcallback framerate video_source color_formatc^K|jj|||||d{y7w)aoCapture video from a participant. Args: participant_id: ID of the participant to capture video from. callback: Callback function to handle video frames. framerate: Desired framerate for video capture. video_source: Video source to capture from. color_format: Color format for video frames. N)rOcapture_participant_video)rRrrrrrs r5rz3LemonSliceTransportClient.capture_participant_videos1"**DD Hi|   #-+- audio_source sample_ratecallback_interval_msc^K|jj|||||d{y7w)aCapture audio from a participant. Args: participant_id: ID of the participant to capture audio from. callback: Callback function to handle audio data. audio_source: Audio source to capture from. sample_rate: Desired sample rate for audio capture. callback_interval_ms: Interval between audio callbacks in milliseconds. N)rOcapture_participant_audio)rRrrrrrs r5rz3LemonSliceTransportClient.capture_participant_audios2"**DD HlKAU   rcVK|jj|d{y7w)eSend a message to participants. Args: frame: The message frame to send. N)rO send_messagers r5rz&LemonSliceTransportClient.send_message)s"**77>>> )')c.|jjS)z`Get the output sample rate. Returns: The output sample rate in Hz. )rOout_sample_raters r5rz)LemonSliceTransportClient.out_sample_rate3s++;;;r4c.|jjS)z^Get the input sample rate. Returns: The input sample rate in Hz. )rOin_sample_raters r5rz(LemonSliceTransportClient.in_sample_rate<s++:::r4cKtjdtd|jd}|j |d{y7w)z4Send an interrupt message to the LemonSlice session.zSending interrupt message interrupteventrVmessageN)r ryrrMrrRtransport_frames r5send_interrupt_messagez0LemonSliceTransportClient.send_interrupt_messageEsD 01;$"..  000AA A A cKtjdtd|jd}|j |d{y7w)z:Send a response_started message to the LemonSlice session.z Sending response_started messageresponse_startedrrNr rrrMrrs r5send_response_started_messagez7LemonSliceTransportClient.send_response_started_messagePsD 78;+"..  000rcKtjdtd|jd}|j |d{y7w)z;Send a response_finished message to the LemonSlice session.z!Sending response_finished messageresponse_finishedrrNrrs r5send_response_finished_messagez8LemonSliceTransportClient.send_response_finished_message[sD 89;,"..  000rctK|jsy|jj||d{y7w)Update subscription settings for participants. Args: participant_settings: Per-participant subscription settings. profile_settings: Global subscription profile settings. Nparticipant_settingsprofile_settings)rOupdate_subscriptionsrRrrs r5rz.LemonSliceTransportClient.update_subscriptionsfs>++ **??!5HX@   s .868cnK|jsy|jj|d{S7w)zWrite an audio frame to the transport. Args: frame: The audio frame to write. Returns: True if the audio frame was written successfully, False otherwise. FN)rOwrite_audio_framers r5rz+LemonSliceTransportClient.write_audio_framets1++11CCEJJJJs ,535 destinationcpK|jsy|jj|d{y7w)zRegister an audio destination for output. Args: destination: The destination identifier to register. N)rOregister_audio_destinationrRrs r5rz4LemonSliceTransportClient.register_audio_destinations. ++ **EEkRRRs ,646)cameraRGB) microphonei>)rHNNN))r+r,r-r.r;r/r7rr#aiohttp ClientSessionrSr[rr\rr{r|rzrrrrrr1rrrrrpropertyrrrrrrrr?rrr3r4r5rArA_s$4#5BF!  '   ""=>&& < $3 $9!49v;:8ZC22!$!          2) $&        "  *?03TT?<<<;;; 1 1 1   K-@ KT K SC Sr4rAceZdZdZdedeffd Zdeffd ZfdZ de ffd Z de ffd Z deffd Zd Zd ededefdZxZS)LemonSliceInputTransportzInput transport for receiving audio and events from LemonSlice. Handles incoming audio streams from participants and manages audio capture from the Daily room connected to LemonSlice. clientrBc Pt||fi|||_||_d|_y)zInitialize the LemonSlice input transport. Args: client: The LemonSlice transport client instance. params: Transport configuration parameters. **kwargs: Additional arguments passed to parent class. FN)superrS_clientrQ _initializedrRrrBr __class__s r5rSz!LemonSliceInputTransport.__init__s- *6*  !r4r\cKt||d{|jj|d{y7(7w)znSetup the input transport. Args: setup: The frame processor setup configuration. Nrr\rrRr\rs r5r\zLemonSliceInputTransport.setup< gmE"""ll  ''' #'A?"AAAAcKt|d{|jjd{y7'7w)z"Cleanup input transport resources.NrrrrRrs r5rz LemonSliceInputTransport.cleanup6goll""$$$ $A=!A?AArcKt||d{|jryd|_|jj|d{|j |d{y7U7 7 w)z{Start the input transport. Args: frame: The start frame containing initialization parameters. NT)rrrrset_transport_readyrRrrs r5rzLemonSliceInputTransport.startsj gmE"""     ll  '''&&u--- # (-s3A2A,6A2 A.A2&A0'A2.A20A2cKt||d{|jjd{y7'7w)zpStop the input transport. Args: frame: The end frame signaling transport shutdown. Nrrrrs r5rzLemonSliceInputTransport.stop: gl5!!!ll!!! "!A>!AAAAcKt||d{|jjd{y7'7w)zyCancel the input transport. Args: frame: The cancel frame signaling immediate cancellation. Nrcancelrrrs r5rzLemonSliceInputTransport.cancel: gnU###ll!!! $!rcK|jjrctjd|d|jj |d|j |jjd{yy7w)zStart capturing audio from a participant. Args: participant: The participant to capture audio from. z@LemonSliceTransportClient start capturing audio for participant id)rrrN)rQr<r ryrr_on_participant_audio_datarrR participants r5start_capturing_audioz.LemonSliceInputTransport.start_capturing_audios~ << ( ( LLRS^_cSdRef ,,88*4088 LL779   ) sA4A?6A=7A?raudiorcKt|j|j|j}||_|j |d{y7w)aHandle received participant audio data. Args: participant_id: ID of the participant who sent the audio. audio: The audio data from the participant. audio_source: The source of the audio (e.g., microphone). )rr num_channelsN)r audio_framesrrtransport_sourcepush_audio_frame)rRrrrrs r5rz3LemonSliceInputTransport._on_participant_audio_datasK#$$))++  ".##E***sAA A A)r+r,r-r.rArrSrr\rrrrrrrrr/r r __classcell__rs@r5rrsw ")" "&(!4(% .. """+" +!+*3+CF+r4rceZdZdZdedeffd Zdeffd ZfdZ de ffd Z de ffd Z deffd Zdeezfd Zej(fded effd Zded effd ZdZdZdZdedefdZdefdZxZ S)LemonSliceOutputTransportzOutput transport for sending audio and events to LemonSlice. Handles outgoing audio streams to participants and manages the custom audio track expected by the LemonSlice platform. rrBc ^t||fi|||_||_d|_d|_y)zInitialize the LemonSlice output transport. Args: client: The LemonSlice transport client instance. params: Transport configuration parameters. **kwargs: Additional arguments passed to parent class. FstreamN)rrSrrQr_transport_destinationrs r5rSz"LemonSliceOutputTransport.__init__s7 *6*  "5=#r4r\cKt||d{|jj|d{y7(7w)zoSetup the output transport. Args: setup: The frame processor setup configuration. Nrrs r5r\zLemonSliceOutputTransport.setuprrcKt|d{|jjd{y7'7w)z#Cleanup output transport resources.Nrrs r5rz!LemonSliceOutputTransport.cleanup!rrrc^Kt||d{|jryd|_|jj|d{|jr-|jj |jd{|j |d{y77Y7"7 w)z|Start the output transport. Args: frame: The start frame containing initialization parameters. NT)rrrrrrrrs r5rzLemonSliceOutputTransport.start&s gmE"""     ll  '''  & &,,99$:U:UV V V&&u--- # ( W-sEB-B%6B- B'8B-B)B-B+ B-'B-)B-+B-cKt||d{|jjd{y7'7w)zqStop the output transport. Args: frame: The end frame signaling transport shutdown. Nrrs r5rzLemonSliceOutputTransport.stop:rrcKt||d{|jjd{y7'7w)zzCancel the output transport. Args: frame: The cancel frame signaling immediate cancellation. Nrrs r5rz LemonSliceOutputTransport.cancelCrrcKtjd||jj|d{y7w)rz$LemonSliceTransport sending message N)r rrrrs r5rz&LemonSliceOutputTransport.send_messageLs4  ;E7CDll''...s7A?A directioncK|tjk(rPt|tr|j d{t|t r|j d{t|!||d{y7E77 w)zPush a frame to the next processor in the pipeline. Args: frame: The frame to push. direction: The direction to push the frame. N) r DOWNSTREAM isinstancer _handle_response_startedr _handle_response_finishedr push_framerRrr rs r5rz$LemonSliceOutputTransport.push_frameWss 11 1%!8933555%!8944666g  222662s37BB'B"B#B:B;BBBcKt|||d{t|tr|j d{yy7.7w)zProcess frames and handle interruptions. Args: frame: The frame to process. direction: The direction of frame flow in the pipeline. N)r process_framerr_handle_interruptionsrs r5rz'LemonSliceOutputTransport.process_framegsIg#E9555 e. /,,. . . 0 6 .s A A'A AA A cTK|jjd{y7w)z8Handle interruption events by sending interrupt message.N)rrrs r5rz/LemonSliceOutputTransport._handle_interruptionsrsll11333 (&(cTK|jjd{y7w)zGHandle bot started speaking events by sending response_started message.N)rrrs r5rz2LemonSliceOutputTransport._handle_response_startedvsll88:::rcTK|jjd{y7w)zHHandle tts response stopped events by sending response_finished message.N)rrrs r5rz3LemonSliceOutputTransport._handle_response_finishedzsll99;;;rrHcvK|j|_|jj|d{S7w)zWrite an audio frame to the LemonSlice transport. Args: frame: The audio frame to write. Returns: True if the audio frame was written successfully, False otherwise. N)rtransport_destinationrrrs r5rz+LemonSliceOutputTransport.write_audio_frame~s1'+&A&A#\\33E::::s 0979rcVK|jj|d{y7w)zwRegister an audio destination. Args: destination: The destination identifier to register. N)rrrs r5rz4LemonSliceOutputTransport.register_audio_destinations ll55kBBBr)!r+r,r-r.rArrSrr\rrrrrrrrrrrrrrrrrrrr?rr/rrrs@r5rrs >)> >,(!4(% ..("""+" /03TT /JXIbIb3e33 / /> /4;< ;-@ ;T ;CCCr4rceZdZdZdeddfdedejdedee dedeed eeffd Z d Z d Z dd Z defdZdefdZdefdZdefdZxZS)LemonSliceTransportaTransport implementation to add a LemonSlice avatar to Daily calls. When used, the Pipecat bot joins the same virtual room as the LemonSlice Avatar and the user. This is achieved by using `LemonSliceTransportClient`, which initiates the conversation via `LemonSliceApi` and obtains a room URL that all participants connect to. Event handlers available: - on_client_connected(transport, participant): Participant connected to the session - on_client_disconnected(transport, participant): Participant disconnected from the session Example:: @transport.event_handler("on_client_connected") async def on_client_connected(transport, participant): ... NrDrGrFrCrB input_name output_namect |||||_t|j|j }t |||||||_d|_d|_ d|_ |jd|jdy)a;Initialize the LemonSlice transport. Args: bot_name: The name of the Pipecat bot. session: aiohttp session used for async HTTP requests. api_key: LemonSlice API key for authentication. session_request: Optional session creation parameters. If not provided, a default agent will be used. params: Optional LemonSlice-specific configuration parameters. input_name: Optional name for the input transport. output_name: Optional name for the output transport. )r!r")r8r9)rDrErFrCrGrBNrerf) rrSrQr7_on_participant_joined_on_participant_leftrAr_input_output_lemonslice_participant_id_register_event_handler) rRrDrGrFrCrBr!r"rErs r5rSzLemonSliceTransport.__init__s, JKH '"&"="= $ 9 9 1+   ;? <@ *.' $$%:; $$%=>r4cK|jjd{}|jdijdd|k7r|j|d{yy7D7w)zHandle participant left events.NinfouserName)rrget_on_client_disconnected)rRrreason ls_bot_names r5r%z(LemonSliceTransport._on_participant_lefts^ LL5577 ??62 & * *:r :k I..{; ; ; J8 ;s!A)A%=A)A'A)'A)cK|jjd{}|jdijdd|k(r |d|_y|j |d{|jrMt j d|jd|j|jdd d iii d{|jr$|jj|d{yy77797 w) z!Handle participant joined events.Nr+r,r-rz Ignoring z 's microphonemediar unsubscribed)r) rrr.r(_on_client_connectedr ryrr&r)rRrr1s r5r$z*LemonSliceTransport._on_participant_joineds LL5577  ??62 & * *:r :k I.9$.?D +++K8 8 8.. y)H)H(IWX//77#lN%C:*0{{kk77 DDD!8 9EsGDC8AD)C:*ADC<.D1C>2D:D<D>DcZK|jj||d{y7w)rrN)rrrs r5rz(LemonSliceTransport.update_subscriptionss/ll//!5-0   s !+)+rHc~|js&t|j|j|_|jS)zGet the input transport for receiving media and events. Returns: The LemonSlice input transport instance. rrB)r&rrrQrs r5inputzLemonSliceTransport.inputs+ {{2$,,t||\DK{{r4c~|js&t|j|j|_|jS)zGet the output transport for sending media and events. Returns: The LemonSlice output transport instance. r8)r'rrrQrs r5outputzLemonSliceTransport.outputs. ||4DLLQUQ]Q]^DL||r4rcDK|jd|d{y7w)zHandle client connected events.reN_call_event_handlerrs r5r5z(LemonSliceTransport._on_client_connecteds&&' ,? ! ,?SM,?c],?\< E*  ~KcKNNr4r )4r. functoolsrtypingrrrrrr daily.dailyr logurur pydanticr pipecat.frames.framesr r rrrrrrrrr"pipecat.processors.frame_processorrrrpipecat.transports.base_inputrpipecat.transports.base_outputr!pipecat.transports.base_transportrr"pipecat.transports.daily.transportrrr !pipecat.transports.lemonslice.apir!r#r7r;rArrr r3r4r5rMs >>!    cb<>L <1)1. M) M ){ )lSlS^ j+1j+ZVC 3VCrBN-BNr4