qi/3 d dl mZmZmZ ddlmZ ddlmZ ddlm Z ddl
m Z m Z ddl
mZ e rd d lZdd
lmZ dd lmZ d Z G d
ded Z G d de Zy ) )Any TypedDictoverload )
AudioInput)GenerationConfig)is_torch_available)ChatChatType )PipelineN)%MODEL_FOR_TEXT_TO_SPECTROGRAM_MAPPING)SpeechT5HifiGanzmicrosoft/speecht5_hifiganc & e Zd ZU dZeed< eed< y) AudioOutputz
audio (`AudioInput`):
The generated audio waveform.
sampling_rate (`int`):
The sampling rate of the generated audio waveform.
audio
sampling_rateN)__name__
__module__ __qualname____doc__r __annotations__int V/opt/pipecat/venv/lib/python3.12/site-packages/transformers/pipelines/text_to_audio.pyr r ! s r r F)totalc
e Zd ZdZdZdZdZdZdZ e d Z
ddd fd
Z d Z d
Z
ed ed ed
efd Zed ee d ed
ee fd Zed ed ed
efd Zed ee d ed
ee fd Z fdZ ddZd Z xZS )TextToAudioPipelinea
Text-to-audio generation pipeline using any `AutoModelForTextToWaveform` or `AutoModelForTextToSpectrogram`. This
pipeline generates an audio file from an input text and optional other conditional inputs.
Unless the model you're using explicitly sets these generation parameters in its configuration files
(`generation_config.json`), the following default values will be used:
- max_new_tokens: 256
Example:
```python
>>> from transformers import pipeline
>>> pipe = pipeline(model="suno/bark-small")
>>> output = pipe("Hey it's HuggingFace on the phone!")
>>> audio = output["audio"]
>>> sampling_rate = output["sampling_rate"]
```
Learn more about the basics of using a pipeline in the [pipeline tutorial](../pipeline_tutorial)
You can specify parameters passed to the model by using [`TextToAudioPipeline.__call__.forward_params`] or
[`TextToAudioPipeline.__call__.generate_kwargs`].
Example:
```python
>>> from transformers import pipeline
>>> music_generator = pipeline(task="text-to-audio", model="facebook/musicgen-small")
>>> # diversify the music generation by adding randomness with a high temperature and set a maximum music length
>>> generate_kwargs = {
... "do_sample": True,
... "temperature": 0.7,
... "max_new_tokens": 35,
... }
>>> outputs = music_generator("Techno music with high melodic riffs", generate_kwargs=generate_kwargs)
```
This pipeline can currently be loaded from [`pipeline`] using the following task identifiers: `"text-to-speech"` or
`"text-to-audio"`.
See the list of available models on [huggingface.co/models](https://huggingface.co/models?filter=text-to-speech).
TNF )max_new_tokens)vocoderr c t | |i | d | _ | j j t j v rE| rE
new_kwargs preprocessoroutputts r
preprocessz TextToAudioPipeline.preprocess s dC 6D :: ' '6 1 Jt--/@A $T%;%;%K%KMhjm n
(&+)-). J
f % F)- )Ct~~ dD !5\55
F
zz ++u4KOPa c):#aS APPzz ++u4MQR1<<+<%s !CRR!$F&FFF
QRs E? Fc z | j || j }|d }|d }| j j r| j || j }d|vr| j |d< |j | |j ddi | j j j dv r d|vrd|d< | j j d i ||}n>t | r t d |j | j d i ||d
}| j | j | }|S ) N)r2 forward_paramsgenerate_kwargsr& return_dict_in_generateT)rK output_audiozYou're using the `TextToAudioPipeline` with a forward-only model, but `generate_kwargs` is non empty. For forward-only TTA models, please use `forward_params` instead of `generate_kwargs`. For reference, the `generate_kwargs` used here are: r r )
_ensure_tensor_on_devicer2 r, can_generater& r8 r3 r4 generatelen
ValueErrorkeysr" )r< model_inputsr> r] r^ rY s r _forwardz TextToAudioPipeline._forward sT ..vdkk.J 01 !23 :: " " $ " ; ;OTXT_T_ ; `O #/97;7M7M 34
! !/ 2
! !#2(TZZ((J<J>JF?# KKZK_K_KaJbd
TZZA,A.A!DF << #\\&)F
r text_inputsr] returnc y Nr r< ri r] s r __call__z TextToAudioPipeline.__call__ s PSr c y rl r rm s r rn z TextToAudioPipeline.__call__ s \_r c y rl r rm s r rn z TextToAudioPipeline.__call__ s UXr c y rl r rm s r rn z TextToAudioPipeline.__call__ s adr c $ t | |fi |S )aL
Generates speech/audio from the inputs. See the [`TextToAudioPipeline`] documentation for more information.
Args:
text_inputs (`str`, `list[str]`, `ChatType`, or `list[ChatType]`):
One or several texts to generate. If strings or a list of string are passed, this pipeline will
generate the corresponding text. Alternatively, a "chat", in the form of a list of dicts with "role"
and "content" keys, can be passed, or a list of such chats. When chats are passed, the model's chat
template will be used to format them before passing them to the model.
forward_params (`dict`, *optional*):
Parameters passed to the model generation/forward method. `forward_params` are always passed to the
underlying model.
generate_kwargs (`dict`, *optional*):
The dictionary of ad-hoc parametrization of `generate_config` to be used for the generation call. For a
complete overview of generate, check the [following
guide](https://huggingface.co/docs/transformers/en/main_classes/text_generation). `generate_kwargs` are
only passed to the underlying model if the latter is a generative model.
Return:
`AudioOutput` or a list of `AudioOutput`, which is a `TypedDict` with two keys:
- **audio** (`np.ndarray` of shape `(nb_channels, audio_length)`) -- The generated audio waveform.
- **sampling_rate** (`int`) -- The sampling rate of the generated audio waveform.
)r* rn )r< ri r] r- s r rn z TextToAudioPipeline.__call__ s 2 w >~>>r c t | dd | j |d< t | dd | j |d< | j |d< |r|ni |r|ni d}|i }i }|||fS )Nassistant_modelassistant_tokenizerrR )r] r^ )r: rt rR ru )r< preprocess_paramsr] r^ paramspostprocess_paramss r _sanitize_parametersz(TextToAudioPipeline._sanitize_parameters
s 4*D 1 =151E1EO - . 4. 5 A+/>>OK (595M5MO 1 2 1?nB2Ar
$ " &*<<r sO , + $ ) & 6 QC1 )5 N
( N
r