qiXZdZddlZddlZddlmZmZddlmZmZm Z m Z ddl m Z ddl mZddlmZmZmZmZmZmZmZddlmZdd lmZmZmZmZdd lmZdd l m!Z!dd l"m#Z#dd l$m%Z%ddl&m'Z' ddl(m)Z*ddl+m,Z,dZ1dZ2dZ3dZ4GddeZ5GddeZ6GddeZ7GddeZ8d e9d!e:fd"Z;d#e#d!eeGd'd(eZ?Gd)d*e!Z@y#e-$r7Z.e j^de.e j^de0de.dZ.[.wwxYw)+z-Soniox speech-to-text service implementation.N) dataclassfield)AnyAsyncGeneratorListOptional)logger) BaseModel) CancelFrameEndFrameFrameInterimTranscriptionFrame StartFrameTranscriptionFrameVADUserStoppedSpeakingFrame)FrameDirection) NOT_GIVEN STTSettings _NotGiven_warn_deprecated_param)SONIOX_TTFS_P99)WebsocketSTTService)Language)time_now_iso8601) traced_stt)connect)Statez Exception: zEIn order to use Soniox, you need to `pip install pipecat-ai[soniox]`.zMissing module: z{"type": "keepalive"}z{"type": "finalize"}zzc&eZdZUdZeed<eed<y)SonioxContextGeneralItemzGRepresents a key-value pair for structured general context information.keyvalueN__name__ __module__ __qualname____doc__str__annotations__M/opt/pipecat/venv/lib/python3.12/site-packages/pipecat/services/soniox/stt.pyrr4sQ H Jr*rc&eZdZUdZeed<eed<y)SonioxContextTranslationTermzORepresents a custom translation mapping for ambiguous or domain-specific terms.sourcetargetNr"r)r*r+r-r-;sY K Kr*r-cteZdZUdZdZeeeed<dZ ee ed<dZ eee ed<dZ eee ed<y)SonioxContextObjectzContext object for models with context_version 2, for Soniox stt-rt-v3-preview and higher. Learn more about context in the documentation: https://soniox.com/docs/stt/concepts/context Ngeneraltexttermstranslation_terms)r#r$r%r&r2rrrr(r3r'r4r5r-r)r*r+r1r1BsU 9=GXd34 5<D(3-!%E8DI %FJx%A BCJr*r1ceZdZUdZdZeed<dZeeed<dZ ee ed<dZ ee e ed <dZeeed <dZeeezed <d Zeeed <d Zeeed<dZeeed<y)SonioxInputParamsa\Real-time transcription settings. .. deprecated:: 0.0.105 Use ``settings=SonioxSTTSettings(...)`` instead. See Soniox WebSocket API documentation for more details: https://soniox.com/docs/speech-to-text/api-reference/websocket-api#configuration-parameters Parameters: model: Model to use for transcription. audio_format: Audio format to use for transcription. num_channels: Number of channels to use for transcription. language_hints: List of language hints to use for transcription. language_hints_strict: If true, strictly enforce language hints (only transcribe in provided languages). context: Customization for transcription. String for models with context_version 1 and ContextObject for models with context_version 2. enable_speaker_diarization: Whether to enable speaker diarization. Tokens are annotated with speaker IDs. enable_language_identification: Whether to enable language identification. Tokens are annotated with language IDs. client_reference_id: Client reference ID to use for transcription. stt-rt-v4model pcm_s16le audio_format num_channelsNlanguage_hintslanguage_hints_strictcontextFenable_speaker_diarizationenable_language_identificationclient_reference_id)r#r$r%r&r9r'r(r;rr=intr>rrr?boolr@r1rArBrCr)r*r+r7r7Os(E3"-L(3--"#L(3-#/3NHT(^,3,08D>037GX)C/ 071665:"HTN:)-#-r*r7tokenreturnc6|dtk(xs |dtk(S)z%Determine if a token is an end token.r3) END_TOKENFINALIZED_TOKENrFs r+ is_end_tokenrLss =I % Iv/)IIr*languagec|t|jj}d|vr|jddS|S)zPipecat Language enum uses same ISO 2-letter codes as Soniox, except with added regional variants. For a list of all supported languages, see: https://soniox.com/docs/speech-to-text/core-concepts/supported-languages -r)r'r!lowersplit)rMlang_strs r+language_to_soniox_languagerSxs; 8>>"((*H h~~c"1%% Or*r>cj|y|Dcgc] }t|}}tt|Scc}wN)rSlistset)r>langprepared_languagess r+_prepare_language_hintsrZs>HVW5d;WW &' ((Xs0ceZdZUdZedZeedzeze d<edZ e dzeze d<edZ e ezdzeze d <ed Ze dzeze d <ed Ze dzeze d <edZedzeze d<y)SonioxSTTSettingsa]Settings for SonioxSTTService. Parameters: language_hints: List of language hints to use for transcription. language_hints_strict: If true, strictly enforce language hints. context: Customization for transcription. String for models with context_version 1 and SonioxContextObject for models with context_version 2. enable_speaker_diarization: Whether to enable speaker diarization. enable_language_identification: Whether to enable language identification. client_reference_id: Client reference ID to use for transcription. ctSrUrr)r*r+zSonioxSTTSettings.sV_r*)default_factoryNr>ctSrUr^r)r*r+r_zSonioxSTTSettings.sS\r*r?ctSrUr^r)r*r+r_zSonioxSTTSettings.sZcr*r@ctSrUr^r)r*r+r_zSonioxSTTSettings.sXar*rActSrUr^r)r*r+r_zSonioxSTTSettings.s r*rBctSrUr^r)r*r+r_zSonioxSTTSettings.sPYr*rC)r#r$r%r&rr>rrrr(r?rEr@r1r'rArBrCr)r*r+r\r\s 9>N_8`NDNT)I5`5:K\5]4$;2]C)?"D4K)$;38HY2Zti/Zr*r\c~eZdZUdZeZeed<dddddddded d ed ed e e d e ed ede de e de de ede e ffdZde fdZdeffd Zdedeeefffd Zdeffd Zdeffd ZdedeedffdZe d-dede d e efd!Zded"e ffd# Z!d$Z"fd%Z#fd&Z$d'Z%d(Z&d)Z'd*Z(d+efd,Z)xZ*S).SonioxSTTServiceaSSpeech-to-Text service using Soniox's WebSocket API. This service connects to Soniox's WebSocket API for real-time transcription with support for multiple languages, custom context, speaker diarization, and more. For complete API documentation, see: https://soniox.com/docs/speech-to-text/api-reference/websocket-api _settingsz,wss://stt-rt.soniox.com/transcribe-websocketNr:r<T) url sample_rater9r;r=paramsvad_force_turn_endpointsettingsttfs_p99_latencyapi_keyrirjr9r;r=rkrlrmrnc tdddddddd} |tdtd|| _|tdt| s|j| _|j |j}|j |j}|j | _|j | _|j| _|j| _|j| _ |j| _ | | j| t |4d || dd| d | ||_||_||_||_||_g|_d|_d|_y) aMInitialize the Soniox STT service. Args: api_key: Soniox API key. url: Soniox WebSocket API URL. sample_rate: Audio sample rate. model: Soniox model to use for transcription. .. deprecated:: 0.0.105 Use ``settings=SonioxSTTSettings(model=...)`` instead. audio_format: Audio format for transcription. Defaults to ``"pcm_s16le"``. num_channels: Number of audio channels. Defaults to 1. params: Additional configuration parameters, such as language hints, context and speaker diarization. .. deprecated:: 0.0.105 Use ``settings=SonioxSTTSettings(...)`` instead. vad_force_turn_endpoint: Listen to `VADUserStoppedSpeakingFrame` to send finalize message to Soniox. If disabled, Soniox will detect the end of the speech. Defaults to True. settings: Runtime-updatable settings. When provided alongside deprecated parameters, ``settings`` values take precedence. ttfs_p99_latency: P99 latency from speech end to final transcript in seconds. Override for your deployment. See https://github.com/pipecat-ai/stt-benchmark **kwargs: Additional arguments passed to the STTService. r8NF)r9rMr>r?r@rArBrCr9rkr<)rjrnkeepalive_timeoutkeepalive_intervalrmr))r\rr9r;r=r>r?r@rArBrC apply_updatesuper__init___api_key_url_vad_force_turn_endpoint _audio_format _num_channels_final_transcription_buffer_last_tokens_received _receive_task)selfrorirjr9r;r=rkrlrmrnkwargsdefault_settings __class__s r+rvzSonioxSTTService.__init__swV-"&',+0 $     "7,=w G%*  "   "8-> ?)/ &&&2#)#6#6L&&2#)#6#6L282G2G /9?9U9U 6+1>> (>D>_>_ ;99!?8>7Q7Q 4    ) )( 3  #- %       (?%*)+-(6:"!r*rGcy)zCheck if this service can generate processing metrics. Returns: True, as Soniox STT supports metrics generation. Tr)rs r+can_generate_metricsz%SonioxSTTService.can_generate_metricss r*framectKt||d{|jd{y77w)zStart the Soniox STT websocket connection. Args: frame: The start frame containing initialization parameters. N)rustart_connectrrrs r+rzSonioxSTTService.start#s3 gmE"""mmo #848688deltacKt||d{}|r0|jd{|jd{|S787 7 w)zApply settings delta and reconnect if anything changed. Args: delta: A settings delta. Returns: Dict mapping changed field names to their previous values. N)ru_update_settings _disconnectr)rrchangedrs r+rz!SonioxSTTService._update_settings,sS077 ""$ $ $--/ ! ! 8 % !s1AAAAAA AAAcKt||d{|jd{|jd{y7577 w)aStop the Soniox STT websocket connection. Stopping waits for the server to close the connection as we might receive additional final tokens after sending the stop recording message. Args: frame: The end frame. N)rustop_send_stop_recordingrrs r+rzSonioxSTTService.stop=sLgl5!!!'')))    ") s1AA AAAAAAActKt||d{|jd{y77w)aeCancel the Soniox STT websocket connection. Compared to stop, this method closes the connection immediately without waiting for the server to close it. This is useful when we want to stop the connection immediately without waiting for the server to send any final tokens. Args: frame: The cancel frame. N)rucancelrrs r+rzSonioxSTTService.cancelJs6gnU###    $ raudiocK|jrI|jjtjur#|jj |d{dy7 w)zSend audio data to Soniox STT Service. Args: audio: Raw audio bytes to transcribe. Yields: Frame: None (transcription results come via WebSocket callbacks). N _websocketstaterOPENsend)rrs r+run_sttzSonioxSTTService.run_sttWsF ??t44 B//&&u- - -  .sAA A A  transcriptis_finalrMc Kyw)z+Handle a transcription result with tracing.Nr))rrrrMs r+_handle_transcriptionz&SonioxSTTService._handle_transcriptiones s directioncKt|||d{t|tr|jr|j rt|j j tjurM|j jtd{tjd|jd|yyyyy770w)zProcesses a frame of audio data, either buffering or transcribing it. Args: frame: The frame to process. direction: The direction of frame processing. Nz(Triggered finalize event on: frame.name=z , direction=)ru process_frame isinstancerryrrrrrFINALIZE_MESSAGEr debugname)rrrrs r+rzSonioxSTTService.process_framelsg#E9555 e8 9d>[>[4??#8#8EJJ#Foo**+;<<< HUZZMI<XY$G?\ 9 6 =s"B?B;A4B? B= /B?=B?cK|jrK|jjtjur$|jj dd{yyy7w)z&Send stop recording message to Soniox.Nrrs r+rz%SonioxSTTService._send_stop_recording{sE ??t44 B//&&r* * * C? *sAAAAcK|jd{t| d{|jr=|js0|j |j |j|_yyy7d7Pw)zyConnect to the Soniox service. Establishes websocket connection and starts receive and keepalive tasks. N)_connect_websocketrurrr~ create_task_receive_task_handler _report_errorrrs r+rzSonioxSTTService._connectso %%'''g   ??4#5#5!%!1!1$2L2LTM_M_2`!aD $6? ( s BA<BA>AB>BcKt|d{|jr*|j|jd{d|_|j d{y7S7&7 w)zfDisconnect from the Soniox service. Cleans up tasks and closes websocket connection. N)rurr~ cancel_task_disconnect_websocketrs r+rzSonioxSTTService._disconnectsh g!###   ""4#5#56 6 6!%D ((*** $ 7 +s3A/A).A/A+A/#A-$A/+A/-A/c 8K |jr'|jjtjuryt j dt |jd{|_|js?|jd|jd{td|j|j }|j}|j}t|tr|j}|j |j"|j$|j&||j(t+|j,|j.||j0|j2|j4d }|jj7t9j:|d{|j=dd{t j dy7{7F797"#t$r%}|jd|| d{7d}~wwxYww) z-Establish the websocket connection to Soniox.NzConnecting to Soniox STTz#Unable to connect to Soniox API at  error_msg) ror9r;r=enable_endpoint_detectionrjr>r?r@rArBrC on_connectedzConnected to Soniox STTzUnable to connect to Soniox: r exception)rrrrr rwebsocket_connectrx push_error Exceptionryrhr@rr1 model_dumprwr9rzr{rjrZr>r?rArBrCrjsondumps_call_event_handler)rrsr@configes r+rz#SonioxSTTService._connect_websockets- 4??#8#8EJJ#F LL3 4$5dii$@@DO??oo2UVZV_V_U`0aobbb"Edii[ QRR-1,I,I(I %AiiG'#67!,,. == $ 2 2 $ 2 2-F#//"9!:J:J"K)*)@)@"./.J.J232R2R'('<'< F //&&tzz&'9: : :**>: : : LL2 3IAc< ; : //.KA3,O[\/] ] ]  sH2G)H-G)$G%7G)G"DG)+G%,G)G'G)HG)"G)%G)'G)) H2H H HHHcK |jr7tjd|jjd{d|_|j dd{y7%#t$r)}|j d||d{7Yd}~Pd}~wwxYw7<#d|_|j dd{7wxYww)z)Close the websocket connection to Soniox.zDisconnecting from Soniox STTNzError closing websocket: ron_disconnected)rr rcloserrr)rrs r+rz&SonioxSTTService._disconnect_websockets > <=oo++---#DO**+<= = = . Z//.Gs,KWX/Y Y Y Z >#DO**+<= = =svC>A*A(A*C"B#C(A** B3B B BB!BB!C!C=C>CCcH|jr |jStd)zGet the current WebSocket connection. Returns: The WebSocket connection. Raises: Exception: If WebSocket is not connected. zWebsocket not connected)rrrs r+_get_websocketzSonioxSTTService._get_websockets! ???? "122r*c Kg_fd}j23d{} tj|}|d}|r7t |dk(r|ddt k(rnt j _g}|D]s}|dr[t|r|d{#jsjd{jj|c|j|ujs|rdjtd j}djtd |}jt||zjt!j|z d{|j#d } |j#d } | s| r/|d{j%d| d| d{|j#d} | r&|d{t'j(dy777b77j7L7,#tj*$rt'j,d|Y>t.$r#} t'j,d| Yd} ~ dd} ~ wwxYw6yw)zxReceive and process websocket messages. Continuously processes messages from the websocket connection. c zKjrdjtdj}jt |j t jdd{j|dd{jd{g_yy7@7'7w)Nrc |dSNr3r)rKs r+r_zVSonioxSTTService._receive_messages..send_endpoint_transcript..s vr*T)r3user_id timestampresult finalized)r) r|joinmap push_framer_user_idrrstop_processing_metrics)r3rs r+send_endpoint_transcriptzDSonioxSTTService._receive_messages..send_endpoint_transcripts//wws#>@`@`aboo&! $ "2"4#??"& 000EEE224443500F4s6A1B;4B55B;B7B;'B9(B;7B;9B;Ntokensr<rr3rrc |dSrr)rKs r+r_z4SonioxSTTService._receive_messages.. %-r*c |dSrr)rKs r+r_z4SonioxSTTService._receive_messages..!rr*)r3rrr error_code error_messagezError: z (_receive_messages) - rfinishedzTranscription finished.zReceived non-JSON message: zError processing message: )r|rrloadslenrJtimer}rLstart_processing_metricsappendrrrrrrgetrr rJSONDecodeErrorwarningr) rrmessagecontentrnon_final_transcriptionrF final_textnon_final_textrrrrs ` r+_receive_messagesz"SonioxSTTService._receive_messagess ,.( 6$"002D AD A'C A**W- *6{a'F1If,=,P6:YY[2+-'# >EZ('.#;"<<<#'#C#C&*&C&C&E E E <<CCEJ/66u= >337N!#79Y9YZ"J&(WW79PQ&N//1",n!<$(MM&6&8#'#C#CF]#]    %[[6 ' O < 2444//$+J<7N}o"^*#;;z22444LL!:; wD A,=!F 55'' H!JJ  Jsilencec^K|jjtd{y7w)zSend a Soniox protocol-level keepalive message. Args: silence: Silent PCM audio bytes (unused, Soniox uses a protocol message). N)rrKEEPALIVE_MESSAGE)rrs r+_send_keepalivez SonioxSTTService._send_keepaliveDs! oo""#4555s #-+-rU)+r#r$r%r&r\Settingsr(rr'rrDr7rEfloatrvrrrdictrrr rr rbytesrr rrrrrrrrrrrrrr __classcell__)rs@r+rgrgs!H  B%)#'.2(,04,;e"e" e" c] e" } e"e"e"*+e""&e",-e"#5/e"Nd,=$sCx." ! ! !+ ! 5 ^E4K-H NR  )- 9A(9K  Z Z> Z+ b +/b > 3^A@6U6r*rg)Ar&rr dataclassesrrtypingrrrrlogurur pydanticr pipecat.frames.framesr r r rrrr"pipecat.processors.frame_processorrpipecat.services.settingsrrrrpipecat.services.stt_latencyrpipecat.services.stt_servicerpipecat.transcriptions.languagerpipecat.utils.timer(pipecat.utils.tracing.service_decoratorsrwebsockets.asyncio.clientrrwebsockets.protocolrModuleNotFoundErrorrerrorrrrrIrJrr-r1r7rrErLr'rSrZr\rgr)r*r+rse4 (66>__8<4/?,F),)  y9 K) K!. !.HJJJ (s)T(^,) d3i) [ [ [0c6*c6E,FLL;qc"#FLLXY &qc* ++,s. C..D*32D%%D*