qi#G^dZddlZddlZddlmZddlmZmZddlm Z ddl m Z m Z m Z mZmZmZmZmZddlmZmZddlmZdd lmZ dd lmZdd lmZeGddeZ#GddeZ$y#e$r7Z e jBd e e jBd e"de dZ [ wwxYw)z3Resemble AI text-to-speech service implementations.N) dataclass)AsyncGeneratorOptional)logger) CancelFrameEndFrame ErrorFrameFrame StartFrameTTSAudioRawFrameTTSStartedFrameTTSStoppedFrame) TTSSettings_warn_deprecated_param)WebsocketTTSService) traced_tts)connect)Statez Exception: zNIn order to use Resemble AI, you need to `pip install pipecat-ai[resembleai]`.zMissing module: ceZdZdZy)ResembleAITTSSettingsz"Settings for ResembleAITTSService.N)__name__ __module__ __qualname____doc__Q/opt/pipecat/venv/lib/python3.12/site-packages/pipecat/services/resembleai/tts.pyrr's,rrcJeZdZUdZeZeed<dddddddd ed eed ed eed eedee deeffdZ de fdZ d%dedefdZ deffd Zdeffd Zdeffd ZdZdZdZdZdZdefdZdefd Zd&deefd!Zd"Zd#Zedededee dffd$Z!xZ"S)'ResembleAITTSServicea0Resemble AI TTS service with WebSocket streaming and word timestamps. Provides text-to-speech using Resemble AI's streaming WebSocket API. Supports word-level timestamps and audio context management for handling multiple simultaneous synthesis requests with proper interruption support. _settingsNz*wss://websocket.cluster.resemble.ai/streamPCM_16wavi"V)voice_idurl precision output_format sample_ratesettingsapi_keyr#r$r%r&r'r(c htddd} |tdtd|| _|| j|t |d |d| d|||_||_|xsd|_|xsd|_ d |_ d|_ d |_ d|_ i|_i|_d |_d |_i|_y) a'Initialize the Resemble AI TTS service. Args: api_key: Resemble AI API key for authentication. voice_id: Voice UUID to use for synthesis. .. deprecated:: 0.0.105 Use ``settings=ResembleAITTSSettings(voice=...)`` instead. url: WebSocket URL for Resemble AI TTS API. precision: PCM bit depth (PCM_32, PCM_24, PCM_16, or MULAW). output_format: Audio format (wav or mp3). sample_rate: Audio sample rate (8000, 16000, 22050, 32000, or 44100). Defaults to 22050. settings: Runtime-updatable settings. When provided alongside deprecated parameters, ``settings`` values take precedence. **kwargs: Additional arguments passed to the parent service. N)modelvoicelanguager#r,F)r'reuse_context_id_within_turnr(r!r"riiDr)rrr, apply_updatesuper__init___api_key_url _precision_output_format_resemble_sample_rate _websocket_request_id_counter _receive_task_request_id_to_context_audio_buffers_buffer_threshold_bytes_jitter_buffer_bytes_playback_started) selfr)r#r$r%r&r'r(kwargsdefault_settings __class__s rr1zResembleAITTSService.__init__9s<1    ":/Dg N%-  "    ) )( 3  #).%      $/x+4u%&"#$ !79# 57'+$ %*!24rreturncy)zCheck if this service can generate processing metrics. Returns: True, as Resemble AI service supports metrics generation. Trr?s rcan_generate_metricsz)ResembleAITTSService.can_generate_metricss rtextc |jj|d|j|j|j|j dd}|xjdz c_t j|S)zBuild a JSON message for the Resemble AI WebSocket API. Args: text: The text or SSML to synthesize. Returns: JSON string containing the request payload. FT) voice_uuiddatabinary_response request_idr&r'r%no_audio_header)r r,r8r5r6r4jsondumps)r?rGmsgs r _build_msgzResembleAITTSService._build_msgsd....$22!0055#     A% zz#rframecKt||d{|j|_|j d{y7.7w)zStart the Resemble AI TTS service. Args: frame: The start frame containing initialization parameters. N)r0startr'r6_connectr?rSrBs rrUzResembleAITTSService.startsA gmE"""%)%5%5"mmo #s A A(A AA A ctKt||d{|jd{y77w)z[Stop the Resemble AI TTS service. Args: frame: The end frame. N)r0stop _disconnectrWs rrYzResembleAITTSService.stops6 gl5!!!    " 848688ctKt||d{|jd{y77w)z`Cancel the Resemble AI TTS service. Args: frame: The cancel frame. N)r0cancelrZrWs rr]zResembleAITTSService.cancels6 gnU###    $ r[cK|jd{|jr=|js0|j|j |j |_yyy7Nw)z%Connect to the Resemble AI WebSocket.N)_connect_websocketr7r9 create_task_receive_task_handler _report_errorrEs rrVzResembleAITTSService._connectsW%%''' ??4#5#5!%!1!1$2L2LTM_M_2`!aD $6? (sA'A%AA'cK|jr*|j|jd{d|_|jd{y7$7w)z*Disconnect from the Resemble AI WebSocket.N)r9 cancel_task_disconnect_websocketrEs rrZz ResembleAITTSService._disconnectsN   ""4#5#56 6 6!%D ((*** 7 +s!+AAA A AAcK |jr'|jjtjuryt j ddd|j i}t|j|d{|_|jdd{y7#7#t$rL}|jd||d{7d|_|jd |d{7Yd}~yd}~wwxYww) z.Establish WebSocket connection to Resemble AI.NzConnecting to Resemble AI TTS AuthorizationzBearer )additional_headers on_connectedUnknown error occurred:  error_msg exceptionon_connection_error) r7staterOPENrdebugr2websocket_connectr3_call_event_handler Exception push_error)r?headerses rr_z'ResembleAITTSService._connect_websockets J4??#8#8EJJ#F LL8 9&'$--(ABG$5diiT[$\\DO**>: : :] : J//.Fqc,JVW/X X X"DO**+@QCI I I JsuC72BC7AB7B8BBBC7BB C4(C/C"C/$C'%C/*C7/C44C7c K |jd{|jrHtjdd|j_|jj d{d|_|jj|jj|jj|jdd{y77u#t $r)}|jd||d{7Yd}~d}~wwxYw7>#d|_|jj|jj|jj|jdd{7wxYww)z*Close WebSocket connection to Resemble AI.NzDisconnecting from Resemble AIrrjrkon_disconnected) stop_all_metricsr7rrq close_timeoutclosertrur;clearr>r:rsr?rws rrez*ResembleAITTSService._disconnect_websockets= >'') ) ) =>01-oo++---#DO    % % '  " " ( ( *  ' ' - - /**+<= = = * . Y//.Fqc,JVW/X X X Y > #DO    % % '  " " ( ( *  ' ' - - /**+<= = =sFC#CAC#+C!,C#0A)FDFC#!C## D,DDD DDDFA*F FF  FcH|jr |jStd)zGet the current WebSocket connection. Returns: The active WebSocket connection. Raises: Exception: If websocket is not connected. zWebsocket not connected)r7rtrEs r_get_websocketz#ResembleAITTSService._get_websockets! ???? "122r context_idc@K|jd{y7w)z)Stop metrics when the bot is interrupted.N)rzr?rs ron_audio_context_interruptedz1ResembleAITTSService.on_audio_context_interrupteds##%%%s c Kyw)aStop metrics after the Resemble AI context finishes playing. No close message is needed: Resemble AI signals completion with an ``audio_end`` message (handled in ``_process_messages``), after which the server-side context is already closed. Nrrs ron_audio_context_completedz/ResembleAITTSService.on_audio_context_completeds  sc<Ktj|dyw)z9Flush any pending audio and finalize the current context.z: flushing audioN)rtracers r flush_audioz ResembleAITTSService.flush_audios v-./sc  K|j23d{} tj|}|s"|j d}|j d}|j j |t|}|j|s{|dk(r'|j dd}|stj|}t|d k(r||jvr&t|j|<d |j|<|j|}|j||jj |d s)t||j krDd |j|<t||j"k\r|j"} | d zd k7r| d z} t%|d| } || d|j|<|j|}t| d k(rot'| |j(d |} |j+|| d{t||j"k\r|j di} | s$| j dg} | j dg}g}t-| |D]/\}}|s t|d k\s|d }|j/||f1|s|j1||d{|dk(r|j3d{|jj |t}|rVt%|}t|d zd k7r|dd}|r3t'||j(d |} |j+|| d{||jvr |j|=||jvr(|j|=||j vr |j |=|j1ddg|d{|j5|d{|dk(r-|j dd}|j dd}|j dd }|j d|d|d|d{||jvr |j|=||jvr |j|=|j7t9|d{|j;d{|j t=|d |d!|"d{|d#vs|j?d{|jAd{tCjD|d$|7#tj$r"|j d|d{7YPwxYw7d777 777B77777z6yw)%z5Process incoming WebSocket messages from Resemble AI.NzReceived invalid JSON: )rltyperLaudio audio_contentrFTrN)rr' num_channelsraudio_timestamps graph_chars graph_times audio_end)rr)Resetrerror error_nameUnknownmessagez Unknown error status_codezError: z (status z): rz error: z - r)iiz unknown message type: )#rrOloadsJSONDecodeErrorrugetr:straudio_context_availablebase64 b64decodelenr; bytearrayr>extendr=r<bytesr r'append_to_audio_contextzipappendadd_word_timestampsstop_ttfb_metricsremove_audio_context push_framerrzr rer_rwarning)r?rrQmsg_typerLrr audio_bytesbuffer chunk_size chunk_to_sendrS timestampsrr word_timeschartimes start_time remainingrrlrs r_process_messagesz&ResembleAITTSService._process_messagess!002J KJ K' jj) wwvH.J4488S_UJ// ;7" # < $$..}= {#q(T%8%886?kD'' 39>D**:6,,Z8 k*--11*eD6{T%>%>> 9=D**:6&kT%A%AA!%!=!=J!A~*"a $)&**=$>M6xjIJUJ K'' oo2I'0SoTTT |J O/ Nc; N-d 73Q3sNVVT,VVT/GVU'V!V7:V2VVV4U*5VU-A;VU0A,V=U3>VU6A V7U98AVU<V(U>)*VVVV1V2V V "V,V/+U$UU$ V#U$$V*V-V0V3V6V9V<V>VVVVVcK |jd{7#t$rZ}|jd||d{7tj|d|j d{7Yd}~ad}~wwxYww)z2Main loop for receiving messages from Resemble AI.NzError in receive loop: rkz* Resemble AI connection lost, reconnecting)rrtrurrqr_r~s r_receive_messagesz&ResembleAITTSService._receive_messagess| 0,,.... 0oo2I!0MYZo[[[ v%OPQ--///  0sNB  B BA>A0A>3A64A>9B>BBcKtj|d|d |jr&|jjtj ur|j d{|j|s@|j|d{|jd{t|||j|j<|j|} |jj|d{|j!|d{dy7777)7#t"$r]}t%d|t'||j)d{7|j d{7Yd}~yd}~wwxYw#t"$r}t%d|Yd}~yd}~wwxYww)aGenerate speech from text using Resemble AI's streaming API. Args: text: The text to synthesize into speech. context_id: Unique identifier for this TTS context. Yields: Frame: Audio frames containing the synthesized speech. z: Generating TTS []Nr)rGrjr)rrqr7rorCLOSEDrVrcreate_audio_contextstart_ttfb_metricsr r:r8rRrsendstart_tts_usage_metricsrtr rrZ)r?rGrrQrws rrun_ttszResembleAITTSService.run_ttss  v/vQ78 C??doo&;&;u||&Kmmo%%// ;// ;;;--///%<<EOD ' '(@(@ A//t/,C ))+00555224888J+&</68  )A!'EFF%<<&&(((mmo%%   C%=aS#AB B B CsGAF$D,%)FD.F&D0'>F&"D6D2 D6!D4"D6&F+G,F.F0F2D64D66 F?4F3E64F F FFGFF G(F?:G?GG)r)N)#rrrrrSettings__annotations__rrintr1boolrFrRr rUrrYrr]rVrZr_rerrrrrrrrr r __classcell__)rBs@rrr.s^%H$$ #'?#+',%*48O5O53- O5  O5 C= O5 }O5c]O501O5bdsC.!!!+!b+ J>& 3&S& 3 0HSM0 LK\ 0%C#%C3%C>%QU+;V%C%Crr)%rrrO dataclassesrtypingrrlogururpipecat.frames.framesrrr r r r r rpipecat.services.settingsrrpipecat.services.tts_servicer(pipecat.utils.tracing.service_decoratorsrwebsockets.asyncio.clientrrrwebsockets.protocolrModuleNotFoundErrorrwrrtrrrrrrs: !+   J<?,F)  K   fC.fC,FLL;qc"#FLLab &qc* ++,s A00B,52B''B,