qi?~dZddlZddlZddlZddlZddlmZmZmZm Z m Z ddl m Z ddl mZmZddlmZ ddlmZmZmZmZddlmZdd lmZd ZdZ dZ!dZ"dZ#dZ$GddeZ%GddeZ&GddeZ'GddZ(GddZ)eZ*GddeZ+y#e$r7Ze j:d ee j:d ed edZ[wwxYw)zSmall WebRTC connection implementation for Pipecat. This module provides a WebRTC connection implementation using aiortc, with support for audio/video tracks, data channels, and signaling for real-time communication applications. N)AnyListLiteralOptionalUnion)logger) BaseModel TypeAdapter) BaseObject)RTCConfiguration RTCIceServerRTCPeerConnectionRTCSessionDescription)RemoteStreamTrack)Framez Exception: zNIn order to use the SmallWebRTC, you need to `pip install pipecat-ai[webrtc]`.zMissing module: signalling2 c6eZdZUdZeded<eed<eed<y)TrackStatusMessagezMessage for updating track enabled/disabled status. Parameters: type: Message type identifier. receiver_index: Index of the track receiver to update. enabled: Whether the track should be enabled or disabled. trackStatustypereceiver_indexenabledN)__name__ __module__ __qualname____doc__r__annotations__intbool[/opt/pipecat/venv/lib/python3.12/site-packages/pipecat/transports/smallwebrtc/connection.pyrr2s -   Mr%rc&eZdZUdZdZeded<y)RenegotiateMessagezMessage requesting WebRTC renegotiation. Parameters: type: Message type identifier for renegotiation requests. renegotiaterNrrrr rrr!r$r%r&r(r(@s $1D'- 0r%r(c&eZdZUdZdZeded<y)PeerLeftMessagezMessage indicating a peer has left the connection. Parameters: type: Message type identifier for peer departure. peerLeftrNr*r$r%r&r,r,Js !+D'* *r%r,c$eZdZdZeeZeeZy)SignallingMessagezUnion types for signaling message handling. Parameters: Inbound: Types of messages that can be received from peers. outbound: Types of messages that can be sent to peers. N) rrrr rrInboundr(outboundr$r%r&r/r/Ts&'G'(Hr%r/c\eZdZdZdZdeddfdZdefdZdZde e fd Z d Z d Z d Zy) SmallWebRTCTrackzWrapper for WebRTC media tracks with enabled/disabled state management. Provides additional functionality on top of aiortc MediaStreamTrack including enable/disable control and frame discarding for audio and video streams. c||_d|j_|j|_d|_d|_d|_d|_y)z{Initialize the WebRTC track wrapper. Args: receiver: The RemoteStreamTrack receiver instance. FTgNg@) _receiver_enabledtrack_track_last_recv_time _idle_task _idle_timeout)selfreceivers r&__init__zSmallWebRTCTrack.__init__gs@ ""'nn  &)26$'r%rreturnNc||_y)zEnable or disable the track. Args: enabled: Whether the track should be enabled for receiving frames. Nr6)r<rs r& set_enabledzSmallWebRTCTrack.set_enabledvs   r%c|jS)zCheck if the track is currently enabled. Returns: True if the track is enabled for receiving frames. rAr<s r& is_enabledzSmallWebRTCTrack.is_enabled~s }}r%cK|j}t|trt|dr$t|jt j s tdytjd|jjsP|jj|jj|jjsOyyyw)z:Discard old frames from the track queue to reduce latency._queuez8Warning: _queue does not exist or has changed in aiortc.NzDiscarding old frames) r8 isinstancerhasattrrGasyncioQueueprintrdebugempty get_nowait task_done)r< remote_tracks r&discard_old_framesz#SmallWebRTCTrack.discard_old_framess{{ l$5 6<2*##W]];PQ LL0 1"))//1##..0##--/#))//1 7s CCCcKd|j_tj|_|jr|jj r(t j|j|_|js|jjdk(ry|jjd{S7w)aReceive the next frame from the track. Enables the internal receiving state and starts idle watcher. Returns: The next frame, except for video tracks, where it returns the frame only if the track is enabled, otherwise, returns None. TvideoN) r5r6timer9r:donerJ create_task _idle_watcherr8kindrecvrDs r&rZzSmallWebRTCTrack.recvs#'#yy{$//"6"6"8%11$2D2D2FGDO}}!1!1W!<[[%%''''sBC?CcK|jjrtj|jd{t j |j z }||jk\rZtjd|jjd|dd|jd{d|j_|jjryy770w)zMDisable receiving if idle for more than _idle_timeout and monitor queue size.NzDisabling receiver for z track after z.2fzs idleF) r5r6rJsleepr;rUr9rrMr8rYrR)r< idle_durations r&rXzSmallWebRTCTrack._idle_watchersnn%%-- 2 23 3 3 IIK$*>*>>M 2 22 -dkk.>.>-?}][^L__ef--///*/'nn%% 30s(9C'C#A8C'4C%5+C'!C'%C'cd|j_|jr!|jjd|_|jr|jj yy)z%Stop receiving frames from the track.FN)r5r6r:cancelr8stoprDs r&r`zSmallWebRTCTrack.stopsH"' ?? OO " " $"DO ;; KK    r%c.t|j|S)zForward attribute access to the underlying track. Args: name: The attribute name to access. Returns: The attribute value from the underlying track. )getattrr8)r<names r& __getattr__zSmallWebRTCTrack.__getattr__st{{D))r%)rrrr r>r#rBrErRrrrZrXr`rdr$r%r&r3r3`sO ( 4 D D 0(HUO(& 0 *r%r3cTeZdZdZ d'deeeeeefde ffd Z e de fdZ e defdZdZd Zd ed efd Zd ed efd ZdZd(d ed edefdZdZdZdZdZdZdZdZd)dZd)dZd)dZd)dZ dZ!defdZ"dZ#dZ$d Z%d!e&fd"Z'd#Z(d$Z)d%Z*d&Z+xZ,S)*SmallWebRTCConnectionaWebRTC connection implementation using aiortc. Provides WebRTC peer connection functionality including ICE server configuration, track management, data channel communication, and connection state handling for real-time audio/video communication. ice_serversconnection_timeout_secsct||sg|_nWtd|Dr||_n=td|Dr |Dcgc]}t |c}|_n t dd|_i|_t|jt|jt|ji|_||_|j!|j#d|j#d|j#d|j#d |j#d |j#d |j#d |j#d |j#dycc}w)a:Initialize the WebRTC connection. Args: ice_servers: List of ICE servers as URLs or IceServer objects. connection_timeout_secs: Timeout in seconds for connecting to the peer. Raises: TypeError: If ice_servers contains mixed types or unsupported types. c3<K|]}t|tywN)rH IceServer.0ss r& z1SmallWebRTCConnection.__init__..s?aAy)?c3<K|]}t|tywrk)rHstrrms r&rpz1SmallWebRTCConnection.__init__..s9As#9rq)urlsz:ice_servers must be either List[str] or List[RTCIceServer]F app-message track-started track-ended connecting connected disconnectedclosedfailednewN)superr>rgallrl TypeError_connect_invoked _track_mapAUDIO_TRANSCEIVER_INDEXaudio_input_trackVIDEO_TRANSCEIVER_INDEXvideo_input_trackSCREEN_VIDEO_TRANSCEIVER_INDEXscreen_video_input_track_track_gettersrh _initialize_register_event_handler)r<rgrhro __class__s r&r>zSmallWebRTCConnection.__init__s9 02D  ?;? ?*D  9[9 9;FGa q 1GD XY Y % #T%;%; #T%;%; *D,I,I  (?$  $$]3 $$_5 $$]3 $$\2 $$[1 $$^4 $$X. $$X. $$U+3 Hs E'r?c|jS)zsGet the underlying RTCPeerConnection. Returns: The aiortc RTCPeerConnection instance. )_pcrDs r&pczSmallWebRTCConnection.pcs xxr%c|jS)zzGet the peer connection identifier. Returns: The unique identifier for this peer connection. )_pc_idrDs r&pc_idzSmallWebRTCConnection.pc_ids{{r%c|tjdt|j}d|_t ||_|jdtjj|_ |jd|_ d|_d|_g|_d|_g|_d|_d|_y)z9Initialize the peer connection and associated components.z Initializing new peer connection) iceServersN-FT)rrMr rg_answerrrrcuuiduuid4hexr_setup_listeners _data_channel_renegotiation_in_progress_last_received_time_outgoing_messages_queue_data_channel_enabled_pending_app_messages_connecting_timeout_task_data_channel_timeout_task)r< rtc_configs r&rz!SmallWebRTCConnection._initializes 78%1A1AB 8< $Z0 1TZZ\%5%5$67  !*/'#' (*%%)"%'"(,%*.'r%cdjjdfd}jjdfd}jjdfd}jjdfd}jjd fd }y ) z/Set up event listeners for the peer connection. datachannelcx|_|jdfd}|jdfd}y)NopencXKtjdjyw)NzData channel is open!)rrM_flush_message_queuerDsr&on_openzOSmallWebRTCConnection._setup_listeners..on_datachannel..on_open1s  45))+s'*messagec0K t|tr+|jdrtj_yt j |}|dtk(r&|jdrj|dyjrjd|d{ytjdjj|y76#t $r%}tj"d|d|Yd}~yd}~wwxYww)Npingrrruz*Client not connected. Queuing app-message.zError parsing JSON message z, )rHrs startswithrUrjsonloadsSIGNALLING_TYPEget_handle_signalling_message is_connected_call_event_handlerrrMrappend Exceptionerror)r json_messageer<s r& on_messagezRSmallWebRTCConnection._setup_listeners..on_datachannel..on_message6sO"'3/G4F4Fv4N3799;0'+zz'': '/?B|GWGWXaGb ;;L&>}l&[ [ [ & -Y Z $ : : A A, O!\!OLL#>wir!!MNNOsdD:C%DAC%D%C%,C#-C%1D20C%"D#C%% D.D DDD)ron)channelrrr<s r&on_datachannelz>SmallWebRTCConnection._setup_listeners..on_datachannel,sG!(D ZZ  ,  ,ZZ " O# Or%connectionstatechangecBKjd{y7wrk)_handle_new_connection_staterDsr&on_connectionstatechangezHSmallWebRTCConnection._setup_listeners..on_connectionstatechangeLs335 5 5s iceconnectionstatechangecKtjdjjdjjyw)NzICE connection state is z, connection is )rrMriceConnectionStateconnectionStaterDsr&on_iceconnectionstatechangezKSmallWebRTCConnection._setup_listeners..on_iceconnectionstatechangeRs= LL*488+F+F*GGWX\X`X`XpXpWqr sAAicegatheringstatechangecfKtjdjjyw)NzICE gathering state is )rrMriceGatheringStaterDsr&on_icegatheringstatechangezJSmallWebRTCConnection._setup_listeners..on_icegatheringstatechangeXs& LL24883M3M2NO Ps.1r7cKtjdjdjdd{j dfd}y7w)NTrack z receivedrvendedcKtjdjdjdd{y7w)Nrz endedrw)rrMrYrr<r7sr&on_endedzJSmallWebRTCConnection._setup_listeners..on_track..on_endedas6 vejj\89..}eDDDs9AAA)rrMrYrr)r7rr<s` r&on_trackz8SmallWebRTCConnection._setup_listeners..on_track\sY LL6%**Y7 8**?EB B B XXg  E E Cs9AA AN)rr)r<rrrrrs` r&rz&SmallWebRTCConnection._setup_listeners)s ] # O $ O> , - 6 . 6 / 0  1  . / Q 0 Q W  E  Er%sdprcKt||}|jj|d{|jt j d|jj d{}|jj|d{t j d|jj|_ y77Z79w)z)Create an SDP answer for the given offer.)rrNzCreating answerz9Setting the answer after the local description is created) rrsetRemoteDescriptionforce_transceivers_to_send_recvrrM createAnswersetLocalDescriptionlocalDescriptionr)r<rroffer local_answers r&_create_answerz$SmallWebRTCConnection._create_answerfs%#D9hh++E222 ,,.  (!XX2244 hh**<888 PRxx00  358s4,CCAC5C6"CC6CCCcDK|j||d{y7w)zInitialize the connection with an SDP offer. Args: sdp: The SDP offer string. type: The SDP type (usually "offer"). N)r)r<rrs r& initializez SmallWebRTCConnection.initializevs!!#t,,,s   cKd|_|jr|jdd{tjd|j D]}|jd|d{|j }|r&|j jd{|j}|r&|jjd{|s|r|jyyy777W7!w)z'7DDD<D>DD restart_pccKtjdj|rpjdd{tjdjj j d{jj||d{fd}tj|y77P7(w)zRenegotiate the WebRTC connection with new parameters. Args: sdp: The new SDP offer string. type: The SDP type (usually "offer"). restart_pc: Whether to restart the peer connection entirely. zRenegotiating rzNzClosing old peer connectioncZKtjdd{d_y7 w)NrF)rJr\rrDsr& delayed_taskz7SmallWebRTCConnection.renegotiate..delayed_tasks&--" " ".3D + #s +) +) rrMrrrremove_all_listeners_closerrrJrW)r<rrrrs` r&r)z!SmallWebRTCConnection.renegotiates  ~dkk]34 **>: : : LL6 7 HH ) ) +++-      !!#t,,,  4 LN+# ;  -s49CCACC)C-C.%CCCc|jj}t|D]\}}|dkrd|_d|_y)z:Force all transceivers to bidirectional send/receive mode.rsendrecvrecvonlyN)rgetTransceivers enumerate direction)r< transceiversi transceivers r&rz5SmallWebRTCConnection.force_transceivers_to_send_recvsCxx//1 ' 5 3NA{1u(2 %(2 %  3r%ctjd|j|jj }t |dkDr.|dj r|dj j|ytjdy)zReplace the audio track in the first transceiver. Args: track: The new audio track to use for sending. zReplacing audio track rz8Audio transceiver not found. Cannot replace audio track.N rrMrYrrlensender replaceTrackwarningr<r7rs r&replace_audio_trackz)SmallWebRTCConnection.replace_audio_trackn  -ejj\:;xx//1 | q \!_%;%; O " " / / 6 NNU Vr%ctjd|j|jj }t |dkDr.|dj r|dj j|ytjdy)zReplace the video track in the second transceiver. Args: track: The new video track to use for sending. zReplacing video track rz8Video transceiver not found. Cannot replace video track.Nrrs r&replace_video_trackz)SmallWebRTCConnection.replace_video_trackrr%ctjd|j|jj }t |dkDr.|dj r|dj j|ytjdy)zReplace the screen video track in the second transceiver. Args: track: The new screen video track to use for sending. zReplacing screen video track rzFScreen video transceiver not found. Cannot replace screen video track.Nrrs r&replace_screen_video_trackz0SmallWebRTCConnection.replace_screen_video_tracksn  4UZZLABxx//1 | q \!_%;%; O " " / / 6 NNc dr%cK|jttjd|j d{y7w)z+Disconnect from the WebRTC peer connection.rrN)send_app_messagerr, model_dumprrDs r& disconnectz SmallWebRTCConnection.disconnects4 ?CTC_C_CabckkmsAA A A cK|jjD]}|s|j|jj|jr"|jj d{|j jd|_|jji|_|j|jy7gw)z0Close the peer connection and cleanup resources.NT) rvaluesr`clearrcloserrr#_cancel_monitoring_connecting_state_cancel_data_channel_timeoutrs r&rzSmallWebRTCConnection._closes__++- E    88((.." " " %%++-%)" ""((* 002 ))+ #s C"AC"8C 9A(C"c|jsy|jj|jj|jdS)zGet the SDP answer for the current connection. Returns: Dictionary containing SDP answer, type, and peer connection ID, or None if no answer is available. N)rrr)rrrrrDs r& get_answerz SmallWebRTCConnection.get_answers;||<<##LL%%[[  r%cxtjdfd}tj|_y)a!Start monitoring the peer connection while it is in the *connecting* state. This method schedules a timeout task that will automatically close the connection if it remains in the connecting state for more than the specified timeout, default to 60 seconds. zMonitoring connecting statecKtjjd{tjdj d{y727w)Nz@Timeout establishing the connection to the remote peer. Closing.)rJr\rhrrrrDsr&timeout_handlerzKSmallWebRTCConnection._monitoring_connecting_state..timeout_handlersF-- < <= = = NN] ^++-   > s!#AA,AAAAN)rrMrJrWrr<r s` r&_monitoring_connecting_statez2SmallWebRTCConnection._monitoring_connecting_states/  23 )0(;(;O.timeout_handlerCs-- 9: : :%%););)F)F&)P:;T:UVXX--335-2* *Q ;sB B A*B N)rJrWrrs` r&_start_data_channel_timeoutz1SmallWebRTCConnection._start_data_channel_timeout5s  3+2*=*=o>O*P'r%c|jrI|jjs/tjd|jj d|_y)a)Cancel the data-channel open timeout task, if any. Should be called when the data channel opens successfully (the timeout is no longer needed) or when the connection is being torn down. If the task is still pending it will be cancelled and the reference cleared. z(Cancelling the data channel timeout taskN)rrVrrMr_rDs r&rz2SmallWebRTCConnection._cancel_data_channel_timeoutOsE  * *43R3R3W3W3Y LLC D  + + 2 2 4*.'r%cK|jj}|dk(r|jn|j|dk(r|js|j |dk(r |j sytjd||j|d{|dk(r.tjd|jd{yy787w)z,Handle changes in the peer connection state.rxryNzConnection state changed to: r|z+Connection failed, closing peer connection.) rrrrrrrrrMrrr)r<states r&rz2SmallWebRTCConnection._handle_new_connection_state[s(( L  - - /  4 4 6 K (G(G  , , . K (=(=  4UG<=&&u--- H  NNH I++-    . s$BCC1CCCCc|jsy|j|jjdk(St j|jz dkS)zCheck if the WebRTC connection is currently active. Returns: True if the connection is active and receiving data. Fry)rrrrrUrDs r&rz"SmallWebRTCConnection.is_connectedrsO$$  # # +88++{: : d666!;;r%cv|jjtr|jtS|jj }t |dk(s|tj stjdy|tj }|r t|nd}||jt<|S)zGet the audio input track wrapper. Returns: SmallWebRTCTrack wrapper for the audio track, or None if unavailable. rz!No audio transceiver is availableN) rrrrrrr=rrr3)r<rr= audio_tracks r&rz'SmallWebRTCConnection.audio_input_track ??  6 7??#:; ; xx//1 |  !6M)N)W)W NN> ? 78AA4<&x0$ 3>/0r%cv|jjtr|jtS|jj }t |dks|tj stjdy|tj }|r t|nd}||jt<|S)zGet the video input track wrapper. Returns: SmallWebRTCTrack wrapper for the video track, or None if unavailable. rz!No video transceiver is availableN) rrrrrrr=rrr3r<rr= video_tracks r&rz'SmallWebRTCConnection.video_input_trackrr%cv|jjtr|jtS|jj }t |dks|tj stjdy|tj }|r t|nd}||jt<|S)zGet the screen video input track wrapper. Returns: SmallWebRTCTrack wrapper for the screen video track, or None if unavailable. rz(No screen video transceiver is availableN) rrrrrrr=rrr3rs r&rz.SmallWebRTCConnection.screen_video_input_tracks ??  = >??#AB B xx//1 |  !6T)U)^)^ NNE F >?HH4<&x0$ :E67r%rctj|}|jr5|jjdk(r|jj |y|j rkt |jtkr1tjd|jj|ytjdtdytjdy)aSend an application message through the data channel. If the data channel is open the message is sent immediately. Otherwise, the message is placed in an in-memory queue so it can be flushed once the channel opens, subject to the following constraints: * Queueing is only attempted when ``_data_channel_enabled`` is ``True``. It is set to ``False`` when the data-channel open timeout fires (see :meth:`_start_data_channel_timeout`), after which messages are silently discarded. * The queue will not grow beyond ``MAX_MESSAGE_QUEUE_SIZE`` entries. Messages that arrive when the queue is full are discarded with a warning. Args: message: The message to send (will be JSON serialized). rz'Data channel not ready, queuing messagezMessage queue is full (z messages). Discarding message.zCData channel unavailable and queueing disabled. Discarding message.N)rdumpsrrsendrrrMAX_MESSAGE_QUEUE_SIZErrMrrtrace)r<rrs r&rz&SmallWebRTCConnection.send_app_messages"zz'*   $"4"4"?"?6"I    # #L 1  ' '40014JJ FG--44\B-.D-EEde LL^ _r%c|jtjd|jrD|jj d}|j j ||jrCyy)a)Flush all queued messages through the now-open data channel. Called when the data channel transitions to the *open* state. Cancels the data-channel open timeout (it is no longer needed) and sends every message that was buffered while the channel was unavailable. z.Data channel is open, flushing queued messagesrN)rrrMrpoprr$)r<rs r&rz*SmallWebRTCConnection._flush_message_queues[ ))+ EF++3377:G    # #G ,++r%c|jryd|_|jttj dy)z/Request renegotiation of the WebRTC connection.NTr)rrrr(rrDs r&rz(SmallWebRTCConnection.ask_to_renegotiates9  * * *.' $1C1E1P1P1R S r%c>tjd|ttj}|j |}|t dxP\|jj|jxsd}|r|j|jyyy)z#Handle incoming signaling messages.zSignalling message received: r$Ncyrkr$r$r%r&zBSmallWebRTCConnection._handle_signalling_message..sr%) rrMr r/r0validate_pythonrrrrrBr)r<rinbound_adaptersignalling_messager7s r&rz0SmallWebRTCConnection._handle_signalling_messages 4WI>?%&7&?&?@,<,M,MN`S_%%&8&@&@A &r%cKtjd||jj|d{y7w)zHandle incoming ICE candidates.zAdding remote candidate: N)rrMraddIceCandidate)r< candidates r&add_ice_candidatez'SmallWebRTCConnection.add_ice_candidate s2 0 <=gg%%i000s7A?A)N<)F)r?N)-rrrr rrrrsrlr"r>propertyrrrrrrrrr#r)rrrrrrr rrrrrrrrrrrrrrr3 __classcell__)rs@r&rfrfsIDH')-,eDItI$>?@-,"%-,^%s/$;Ez1131 -C-s-*.,S,,,<3W W e  , O& -Q4 / .r?s 66+0 , 8 !"  11+i+ ) )g*g*V  1J1Y,FLL;qc"#FLLab &qc* ++,sBC"2CC