qiofdZddlZddlmZddlmZmZddlm Z m Z m Z m Z m Z ddlmZddlmZddlmZmZmZmZmZmZmZdd lmZmZmZmZdd lm Z dd l!m"Z"m#Z#dd l$m%Z%m&Z&dd l'm(Z(ddl)m*Z* ddl+Z,de%de e1fdZ2eGddeZ3eGddeZ4Gdde#Z5Gdde"Z6y#e-$r7Z.ej^de.ej^de0de.dZ.[.wwxYw)zYNVIDIA Riva Speech-to-Text service implementations for real-time and batch transcription.N)CancelledError) dataclassfield)AnyAsyncGeneratorListMappingOptional)logger) BaseModel) CancelFrameEndFrame ErrorFrameFrameInterimTranscriptionFrame StartFrameTranscriptionFrame) NOT_GIVEN STTSettings _NotGiven_warn_deprecated_param)NVIDIA_TTFS_P99)SegmentedSTTService STTService)Languageresolve_language)time_now_iso8601) traced_sttz Exception: zNIn order to use NVIDIA Riva STT, you need to `pip install pipecat-ai[nvidia]`.zMissing module: languagereturnc (itjdtjdtjdtjdtj dtj dtjdtjdtjdtjdtjdtjdtjdtjdtjd tj d tj"d tj$d tj&d tj(d tj*d tj,d tj.d i}t1||dS)a@Maps Language enum to NVIDIA Riva ASR language codes. Source: https://docs.nvidia.com/deeplearning/riva/user-guide/docs/asr/asr-riva-build-table.html?highlight=fr%20fr Args: language: Language enum value. Returns: Optional[str]: NVIDIA Riva language code or None if not supported. zar-ARen-USzen-GBzfr-FRzde-DEzhi-INzit-ITzja-JPzko-KRzpt-BRzru-RUzes-ESzes-USF) use_base_code)rARENEN_USEN_GBFRFR_FRDEDE_DEHIHI_INITIT_ITJAJA_JPKOKO_KRPTPT_BRRURU_RUESES_ESES_USr)r LANGUAGE_MAPs M/opt/pipecat/venv/lib/python3.12/site-packages/pipecat/services/nvidia/stt.py language_to_nvidia_riva_languager=*sg# W#  W #  #  #  W# #  W# #  W# #"  W##$ %#(  W)#* +#.  W/#0 1#4  W5#6  W WE#LJ Hl% HHceZdZdZy)NvidiaSTTSettingszSettings for NvidiaSTTService.N)__name__ __module__ __qualname____doc__r>r<r@r@^s(r>r@ceZdZUdZedZeezed<edZ eezed<edZ eezed<ed Z e e d zezed <ed Zeezed <y )NvidiaSegmentedSTTSettingsaSettings for NvidiaSegmentedSTTService. Parameters: profanity_filter: Whether to filter profanity from results. automatic_punctuation: Whether to add automatic punctuation. verbatim_transcripts: Whether to return verbatim transcripts. boosted_lm_words: List of words to boost in language model. boosted_lm_score: Score boost for specified words. ctSNrrEr>r<z#NvidiaSegmentedSTTSettings.qsyr>)default_factoryprofanity_filterctSrIrJrEr>r<rKz#NvidiaSegmentedSTTSettings.rsIr>automatic_punctuationctSrIrJrEr>r<rKz#NvidiaSegmentedSTTSettings.ss9r>verbatim_transcriptsctSrIrJrEr>r<rKz#NvidiaSegmentedSTTSettings.tsS\r>Nboosted_lm_wordsctSrIrJrEr>r<rKz#NvidiaSegmentedSTTSettings.us r>boosted_lm_score)rArBrCrDrrMboolr__annotations__rOrQrSrstrrUfloatrEr>r<rGrGes*/?P)QdY&Q.3DU.V4)+V-2CT-U$*U5:K\5]d3i$&2]*/@Q*Rei'Rr>rGcdeZdZUdZeZeed<GddeZddddd d d d e d d e d e de e e fde e de edede ede effdZdZdZdefdZde fdZdeffd Zdeffd Zdeffd ZdZd Zd!Ze d+d"e d#ed$e efd%Z d&Z!d'e"de#e$d ffd(Z%de"fd)Z&d*Z'xZ(S),NvidiaSTTServiceaReal-time speech-to-text service using NVIDIA Riva streaming ASR. Provides real-time transcription capabilities using NVIDIA's Riva ASR models through streaming recognition. Supports interim results and continuous audio processing for low-latency applications. _settingsc:eZdZUdZej Zeeed<y)NvidiaSTTService.InputParamszConfiguration parameters for NVIDIA Riva STT service. .. deprecated:: 0.0.105 Use ``settings=NvidiaSTTSettings(...)`` instead. Parameters: language: Target language for transcription. Defaults to EN_US. rN) rArBrCrDrr&rr rWrEr>r< InputParamsr^s (0~~(8$5r>r_grpc.nvcf.nvidia.com:443z$1598d209-5e27-4d3c-8079-4751568b1081zparakeet-ctc-1.1b-asr function_id model_nameNTservermodel_function_map sample_rateparamsuse_sslsettingsttfs_p99_latencyapi_keyrerfrgrhrirjrkc t|jdtj} |#t dt|s|j | _|| j |t | d ||| d| ||_ ||_ ||_ d|_ d|_ d|_d|_d|_d|_d|_|jd |_d|_d|_d|_d|_y) a'Initialize the NVIDIA Riva STT service. Args: api_key: NVIDIA API key for authentication. server: NVIDIA Riva server address. Defaults to NVIDIA Cloud Function endpoint. model_function_map: Mapping containing 'function_id' and 'model_name' for the ASR model. sample_rate: Audio sample rate in Hz. If None, uses pipeline default. params: Additional configuration parameters for NVIDIA Riva. .. deprecated:: 0.0.105 Use ``settings=NvidiaSTTSettings(...)`` instead. use_ssl: Whether to use SSL for the NVIDIA Riva server. Defaults to True. settings: Runtime-updatable settings. When provided alongside deprecated parameters, ``settings`` values take precedence. ttfs_p99_latency: P99 latency from speech end to final transcript in seconds. Override for your deployment. See https://github.com/pipecat-ai/stt-benchmark **kwargs: Additional arguments passed to STTService. rc)modelrNrhrgrkrjrbrE)r@getrr&rr apply_updatesuper__init___server_api_key_use_ssl_start_history_start_threshold _stop_history_stop_threshold_stop_history_eou_stop_threshold_eou_custom_configuration _function_id _asr_service_queue_config _thread_task selfrlrerfrgrhrirjrkkwargsdefault_settings __class__s r<rvzNvidiaSTTService.__init__sH-$((6^^    "8-> ?,2OO )    ) )( 3  #-%        $#!##' %'".22=A    r>cd|jgdd|jgg}tjj d|j |j |}tjj||_y)N function-id authorizationBearer ) rrxrivaclientAuthryrw ASRServicerrmetadataauths r<_initialize_clientz#NvidiaSTTService._initialize_clientse D-- .  7 8 {{dmmT\\8L KK2248r>c *tjjtjjtjjj |j jddddd|jd d}tjj||j|j|j|j|j|jtjj!||j"|S)5Create the NVIDIA Riva ASR recognition configuration.rrFT) encoding language_codernmax_alternativesrMenable_automatic_punctuationrQsample_rate_hertzaudio_channel_count)configinterim_results)rrStreamingRecognitionConfigRecognitionConfig AudioEncoding LINEAR_PCMr\rrg!add_endpoint_parameters_to_configrzr{r|r~r}r"add_custom_configuration_to_configr)rrs r<_create_recognition_configz+NvidiaSTTService._create_recognition_configs77;;0022=="nn55!"!&-1%)"&"2"2$%1 !8   55      ! !     " "   $ $  66vt?Y?YZ r>r cy)zCheck if this service can generate processing metrics. Returns: False - this service does not support metrics generation. FrErs r<can_generate_metricsz%NvidiaSTTService.can_generate_metricss r>rncKddl}|j5|jd|jdtddddy#1swYyxYww)aSet the ASR model for transcription. .. deprecated:: 0.0.104 Model cannot be changed after initialization for NVIDIA Riva streaming STT. Set model and function id in the constructor instead, e.g.:: NvidiaSTTService( api_key=..., model_function_map={"function_id": "", "model_name": ""}, ) Args: model: Model name to set. rNalwaysa 'set_model' is deprecated. Model cannot be changed after initialization for NVIDIA Riva streaming STT. Set model and function id in the constructor instead, e.g.: NvidiaSTTService(api_key=..., model_function_map={'function_id': '', 'model_name': ''})) stacklevel)warningscatch_warnings simplefilterwarnDeprecationWarning)rrnrs r< set_modelzNvidiaSTTService.set_model sX   $ $ &   ! !( + MMK #     sA*A  A AAframectKt||d{|j|j|_t j |_|js$|j|j|_tjd|jjy7w)zStart the NVIDIA Riva STT service and initialize streaming configuration. Args: frame: StartFrame indicating pipeline start. Nz)Initialized NvidiaSTTService with model: )rustartrrrasyncioQueuerr create_task_thread_task_handlerr debugr\rnrrrs r<rzNvidiaSTTService.start(s gmE""" !668 mmo   $ 0 01J1J1L MD  @AUAU@VWX #sB8B6B B8ctKt||d{|jd{y77w)zStop the NVIDIA Riva STT service and clean up resources. Args: frame: EndFrame indicating pipeline stop. N)rustop _stop_tasksrs r<rzNvidiaSTTService.stop9s6 gl5!!!    " 848688ctKt||d{|jd{y77w)zCancel the NVIDIA Riva STT service operation. Args: frame: CancelFrame indicating operation cancellation. N)rucancelrrs r<rzNvidiaSTTService.cancelBs6 gnU###    $ rc~K|jr+|j|jd{d|_yy7 wrI)r cancel_taskrs r<rzNvidiaSTTService._stop_tasksKs9   ""4#4#45 5 5 $D   5s +=;=c|jj||j}|D]B}|jst j |j ||jDy)N) audio_chunksstreaming_config)rstreaming_response_generatorrresultsrrun_coroutine_threadsafe_handle_responseget_event_loop)r responsesresponses r<_response_handlerz"NvidiaSTTService._response_handlerPsk%%BB!\\C " eH##  , ,T-B-B8-LdNaNaNc d er>cK d|_tj|jd{y7#tj$r d|_wxYww)NTF)_thread_runningr to_threadrrrs r<rz%NvidiaSTTService._thread_task_handlerZsL #'D ##D$:$:; ; ;%% #(D  s%A)535A5AA transcriptis_finalrc Kywz+Handle a transcription result with tracing.NrErrrrs r<_handle_transcriptionz&NvidiaSTTService._handle_transcriptionb c K|jD]6}|r |js|jdj}|s/t|dkDs>|jr|j d{|j t||jt|jj|d{|j||j|jjd{|j t||jt|jj|d{9y777Z7w)Nr)result)rrr)r alternativesrlenrstop_processing_metrics push_framer_user_idrr\rrr)rrrrs r<rz!NvidiaSTTService._handle_responseis1&& Ff11,,Q/::Jc*o1??66888//*& MM,. NN33#) 44#-!'!%!8!85 //1& MM,. NN33#) / 9 sT;EE E,E -A E9E :9E3E4A EE E EEEaudiocK|jd{|jj|d{dy7-7 w)zProcess audio data for speech-to-text transcription. Args: audio: Raw audio bytes to transcribe. Yields: None - transcription results are pushed to the pipeline via frames. N)start_processing_metricsrput)rrs r<run_sttzNvidiaSTTService.run_stts>++---kkooe$$$  .$sAA"AA AAc|jst tj|jj |j }|jS#t$rtwxYw)zGet the next audio chunk for NVIDIA Riva processing. Returns: Audio bytes from the queue. Raises: StopIteration: When the thread is no longer running. ) r StopIterationrrrrsrrFuturesCancelledError)rfutures r<__next__zNvidiaSTTService.__next__s_##  55dkkoo6GI\I\I^_F==? "$   s A A A0c|S)zdReturn iterator for audio chunk processing. Returns: Self as iterator. rErs r<__iter__zNvidiaSTTService.__iter__s  r>rI))rArBrCrDr@SettingsrWr r_rrXr r intrVrYrvrrrrrrrrr rrrrrrrrbytesrrrrr __classcell__rs@r<r[r[xs|!H   6i 6 1A11 &*(,04,;K!K! K! $CH- K!c]K!%K!K!,-K!#5/K!Z9<dS:YY"!!!+!% eNR  )- 9A(9K  D 5 ^E4K-H  % $r>r[cNeZdZUdZeZeed<GddeZddddd d d d e d d e d e de e e fde e de edede ede effdZdede e fdZdZde fdZdZdefdZdeffd Zdedee efffd Ze d%d e d!ede efd"Zd#ede e!d ffd$Z"xZ#S)&NvidiaSegmentedSTTServiceaSpeech-to-text service using NVIDIA Riva's offline/batch models. By default, his service uses NVIDIA's Riva Canary ASR API to perform speech-to-text transcription on audio segments. It inherits from SegmentedSTTService to handle audio buffering and speech detection. r\ceZdZUdZej Zeeed<dZ e ed<dZ e ed<dZ e ed<dZ eeeed <d Zeed <y) %NvidiaSegmentedSTTService.InputParamsarConfiguration parameters for NVIDIA Riva segmented STT service. .. deprecated:: 0.0.105 Use ``settings=NvidiaSegmentedSTTSettings(...)`` instead. Parameters: language: Target language for transcription. Defaults to EN_US. profanity_filter: Whether to filter profanity from results. automatic_punctuation: Whether to add automatic punctuation. verbatim_transcripts: Whether to return verbatim transcripts. boosted_lm_words: List of words to boost in language model. boosted_lm_score: Score boost for specified words. rFrMTrOrQNrS@rU)rArBrCrDrr&rr rWrMrVrOrQrSrrXrUrYrEr>r<r_rs\ (0~~(8$5!&$&&*t*%*d*04(49-4"%%%r>r_r`z$ee8dc628-76de-4acc-8595-1836e7e857bdz canary-1b-asrraNTrdrlrerfrgrhrirjrkc t|jdttjxsdddddd} |t dt|st|j xstjxsd| _|j| _|j| _|j| _ |j| _ |j| _ || j|t |8d||| d | ||_||_||_|jd |_d |_d |_d |_d |_d |_d |_d |_d|_d|_y)aJInitialize the NVIDIA Riva segmented STT service. Args: api_key: NVIDIA API key for authentication server: NVIDIA Riva server address (defaults to NVIDIA Cloud Function endpoint) model_function_map: Mapping of model name and its corresponding NVIDIA Cloud Function ID sample_rate: Audio sample rate in Hz. If not provided, uses the pipeline's rate params: Additional configuration parameters for NVIDIA Riva .. deprecated:: 0.0.105 Use ``settings=NvidiaSegmentedSTTSettings(...)`` instead. use_ssl: Whether to use SSL for the NVIDIA Riva server. Defaults to True. settings: Runtime-updatable settings. When provided alongside deprecated parameters, ``settings`` values take precedence. ttfs_p99_latency: P99 latency from speech end to final transcript in seconds. Override for your deployment. See https://github.com/pipecat-ai/stt-benchmark **kwargs: Additional arguments passed to SegmentedSTTService rcr"FTNr)rnrrMrOrQrSrUrhrorbrprqrrrE)rGrsr=rr&rrrMrOrQrSrUrtrurvrxrwryrrzr{r|r}r~rrrrrs r<rvz"NvidiaSegmentedSTTService.__init__ssH6$((65hnnEP""&!&!     "8-G H4V__5VWb[b!)5;4K4K 19?9U9U 68>8S8S 54:4K4K 14:4K4K 1    ) )( 3  #-%       .22=A! $#!##' %'"  r>rr ct|S)zConvert pipecat Language enum to NVIDIA Riva's language code. Args: language: Language enum value. Returns: NVIDIA Riva language code or None if not supported. )r=)rrs r<language_to_service_languagez6NvidiaSegmentedSTTService.language_to_service_language.s099r>c|jyd|jgdd|jgg}tjj d|j |j|}tjj||_y)zCInitialize the NVIDIA Riva ASR client with authentication metadata.Nrrr) rrrxrrrryrwrrs r<rz,NvidiaSegmentedSTTService._initialize_client9sv    ( D-- .  7 8  {{dmmT\\8L KK2248r>c6|jjxsdS)z1Get the current NVIDIA Riva language code string.r")r\rrs r<_get_language_codez,NvidiaSegmentedSTTService._get_language_codeHs~~&&1'1r>c n|j}tjj|j d|j |j |j}|jr5tjj||j|jtjj||j|j|j|j|j |j"|j$r*tjj'||j$|S)rr)rrrMrrQ)r\rrrrrMrOrQrSadd_word_boosting_to_configrUrrzr{r|r~r}rrr)rsrs r<rz4NvidiaSegmentedSTTService._create_recognition_configLs NN..113//)*)@)@!"!7!7 /    KK 3 3FAcy)zCheck if this service can generate processing metrics. Returns: True - this service supports metrics generation. TrErs r<rz.NvidiaSegmentedSTTService.can_generate_metricsms r>rcKt||d{|j|j|_t j d|jjy7Vw)zInitialize the service when the pipeline starts. Args: frame: StartFrame indicating pipeline start. Nz2Initialized NvidiaSegmentedSTTService with model: ) rurrrrr rr\rnrs r<rzNvidiaSegmentedSTTService.startus[ gmE""" !668  I$..J^J^I_`a #sA/A-AA/deltacpKt||d{}|r|j|_|S7w)zApply a settings delta and sync internal state. Args: delta: A :class:`STTSettings` (or ``NvidiaSegmentedSTTSettings``) delta. Returns: Dict mapping changed field names to their previous values. N)ru_update_settingsrr)rrchangedrs r<rz*NvidiaSegmentedSTTService._update_settingss8077 ::>udll[`>aL..0 0 0|X.'..0'#( h 26G! ^&v~rB '?55;;=D '7vQ%?@0  MM,. NN33  /3+"88tT^^E\E\]]] ^ ' v%]^_'K 2 1<^ [ LLD6!STVSWX Yv%OPSTVPWyYZ Z Z 9 LLD6aS1 2dV8A3#78 8 8 9sH9F F?F =F>AF F %Ar)7rDrconcurrent.futuresrr dataclassesrrtypingrrrr r logurur pydanticr pipecat.frames.framesr rrrrrrpipecat.services.settingsrrrrpipecat.services.stt_latencyrpipecat.services.stt_servicerrpipecat.transcriptions.languagerrpipecat.utils.timer(pipecat.utils.tracing.service_decoratorsr riva.clientrModuleNotFoundErrorrr rrXr=r@rGr[rrEr>r<r's `F(??`_8HF/?,1Ix1IHSM1Ih      SS S$xzxv ]9 3]9_ ,FLL;qc"#FLLab &qc* ++,s0B66C2;2C--C2