qiI d Z ddlZddlmZmZ ddlmZmZ ddl m
Z
m Z m Z m
Z
mZ e
j e Z G d de Zy) zZ
Sequence feature extraction class for common feature extractors to preprocess sequences.
N )is_valid_audio
load_audio) BatchFeatureFeatureExtractionMixin)PaddingStrategy
TensorTypeis_torch_tensorloggingto_numpyc e Zd ZdZdededef fd Z ddeee z e e
ef z e e
ee f z ee e
ef z de e
z e z d edz d
e d edz d e dz d
e
e
z dz defdZde j ddfde e
ej" f ez d edz de d edz d e dz de f dZ dde e
ej" f ez d edz d edz d
e dz fdZddZde
ee
z eee
z fdZ xZS )SequenceFeatureExtractora
This is a general feature extraction class for speech recognition.
Args:
feature_size (`int`):
The feature dimension of the extracted features.
sampling_rate (`int`):
The sampling rate at which the audio files should be digitalized expressed in hertz (Hz).
padding_value (`float`):
The value that is used to fill the padding values / vectors.
feature_size
sampling_rate
padding_valuec || _ || _ || _ |j dd | _ |j dd | _ t
| di | y )N padding_siderightreturn_attention_maskT )r r r popr r super__init__)selfr r r kwargs __class__s `/opt/pipecat/venv/lib/python3.12/site-packages/transformers/feature_extraction_sequence_utils.pyr z!SequenceFeatureExtractor.__init__) sR ( * * "JJ~w ?%+ZZ0G%N"
"6" Nprocessed_featurespadding
max_length
truncationpad_to_multiple_ofr return_tensorsreturnc
x t |t t f rMt |d t t f r4|d j D ci c] }||D cg c] } | | c} }}} j d |vr5t d j d dt |j | j d }
||n j }t |
dk( r |rg |d< |S |
d } t | t t f r@d} t |
| dk( r| dz
} t |
| dk( r| t |
k r|
| d } |[t | rd}nMt | t t t t t j f rd}nt d| d t | d
|j! D ]I \ }}
t |
d t t f rt# |
||< .|
D cg c]
}t# | c}||< K j% || }| j d }
t |
t' fd |j) D s t d
g }t+ D ]N }|j! D ci c] \ }}|||
}}} j- |||| }|j/ | P |t0 j2 k( r$t5 fd|D }t0 j6 }i }t+ D ] } j9 || |||| }|j! D ]p \ }}
||vrg ||< |
j: t j: t j< u r |
j? t j@ }
|| j/ |
r t || S c c} w c c} }w c c}w c c}}w )a2
Pad input values / input vectors or a batch of input values / input vectors up to predefined length or to the
max sequence length in the batch.
Padding side (left/right) padding values are defined at the feature extractor level (with `self.padding_side`,
`self.padding_value`)
If the `processed_features` passed are dictionary of numpy arrays or PyTorch tensors the
result will use the same type unless you provide a different tensor type with `return_tensors`. In the case of
PyTorch tensors, you will lose the specific device of your tensors however.
Args:
processed_features ([`BatchFeature`], list of [`BatchFeature`], `dict[str, list[float]]`, `dict[str, list[list[float]]` or `list[dict[str, list[float]]]`):
Processed inputs. Can represent one input ([`BatchFeature`] or `dict[str, list[float]]`) or a batch of
input values / vectors (list of [`BatchFeature`], *dict[str, list[list[float]]]* or *list[dict[str,
list[float]]]*) so you can use this method during preprocessing as well as in a PyTorch Dataloader
collate function.
Instead of `list[float]` you can have tensors (numpy arrays or PyTorch tensors),
see the note above for the return type.
padding (`bool`, `str` or [`~utils.PaddingStrategy`], *optional*, defaults to `True`):
Select a strategy to pad the returned sequences (according to the model's padding side and padding
index) among:
- `True` or `'longest'`: Pad to the longest sequence in the batch (or no padding if only a single
sequence if provided).
- `'max_length'`: Pad to a maximum length specified with the argument `max_length` or to the maximum
acceptable input length for the model if that argument is not provided.
- `False` or `'do_not_pad'` (default): No padding (i.e., can output a batch with sequences of different
lengths).
max_length (`int`, *optional*):
Maximum length of the returned list and optionally padding length (see above).
truncation (`bool`):
Activates truncation to cut input sequences longer than `max_length` to `max_length`.
pad_to_multiple_of (`int`, *optional*):
If set will pad the sequence to a multiple of the provided value.
This is especially useful to enable the use of Tensor Cores on NVIDIA hardware with compute capability
`>= 7.5` (Volta), or on TPUs which benefit from having sequence lengths be a multiple of 128.
return_attention_mask (`bool`, *optional*):
Whether to return the attention mask. If left to the default, will return the attention mask according
to the specific feature_extractor's default.
[What are attention masks?](../glossary#attention-mask)
return_tensors (`str` or [`~utils.TensorType`], *optional*):
If set, will return tensors instead of list of python integers. Acceptable values are:
- `'pt'`: Return PyTorch `torch.Tensor` objects.
- `'np'`: Return Numpy `np.ndarray` objects.
r zYou should supply an instance of `transformers.BatchFeature` or list of `transformers.BatchFeature` to this method that includes z, but you provided attention_maskr ptnpztype of z
unknown: z6. Should be one of a python, numpy, or pytorch object.)r r! c 3 : K | ] }t | k( y w)N)len).0v
batch_sizes r z/SequenceFeatureExtractor.pad.. s MA3q6Z'Ms zLSome items in the output dictionary have a different batch size than others.)r! r# r" c 3 T K | ] }t |j d ! yw)r N)r+ model_input_names)r, input_slicer s r r/ z/SequenceFeatureExtractor.pad.. s& m[ST-C-CA-F!G H ms %()r! padding_strategyr# r ) tensor_type)!
isinstancelisttupledictr keysr1
ValueErrorr r+ r
intfloatr) ndarraytypeitemsr _get_padding_strategiesallvaluesrange _truncateappendr LONGESTmax
MAX_LENGTH_paddtypefloat64astypefloat32)r r r r! r" r# r r$ keyexamplerequired_input
first_elementindexvaluer- r3 truncated_inputsikinputs inputs_slice
batch_outputsoutputsr. s ` @r padz SequenceFeatureExtractor.pad3 s L (4- 8ZHZ[\H]`dfr_s=t RddeQfQkQkQm"JM2DEwgclEE" "
! !! $,> > 1151G1G1J0K L+002346
,D,B,B1,EF%:%F !DLfLf ~ ! #$79"#34% % 'q)
mdE] 3EnU+,1
nU+,1s>** .u 5a 8
! }-!%MCeRZZ+PQ!% }oZ]8K7L MK K
-224 GJC%(S%L1*25/"3'@E*F18A;*F"3' G 77T^7_+D,B,B1,EF (
M1C1J1J1LMM kl l z" 2A*<*B*B*DE$!Qa1gEFE >>%#5% * L
# #L 1 2 6 6 6 m\l mmJ . 9 9
z" 1Aii #%!1#5&; G &mmo
1
Um+)+M#&;;"((2::"66 !LL 4E c"))%0
1 1" M~FF F"d +G Fs$
N+ N& N+ N19N6&N+r3 c || j d }|t j k( r t | }||||z dk7 r ||z dz |z }|t j k7 xr t | |k }|r5d|vr1t j t | t
j |d< |r)|t | z
}| j dk( rn|r t j |d d|f |d< | j dkD rd|fdfnd|f} t j || d| j || j d < |S | j d k( rn|r t j |d |df |d< | j dkD r|dfdfn|df} t j || d| j || j d < |S t d
t | j z |S ) a
Pad inputs (on left/right and up to predefined length or max length in the batch)
Args:
processed_features (`Union[dict[str, np.ndarray], BatchFeature]`):
Dictionary of input values (`np.ndarray[float]`) / input vectors (`list[np.ndarray[float]]`) or batch
of inputs values (`list[np.ndarray[int]]`) / input vectors (`list[np.ndarray[int]]`)
max_length (`int`, *optional*):
Maximum length of the returned list and optionally padding length (see below)
padding_strategy (`PaddingStrategy`, *optional*, default to `PaddingStrategy.DO_NOT_PAD`):
PaddingStrategy to use for padding.
- PaddingStrategy.LONGEST Pad to the longest sequence in the batch
- PaddingStrategy.MAX_LENGTH: Pad to the max length (default)
- PaddingStrategy.DO_NOT_PAD: Do not pad
The feature_extractor padding sides are defined in self.padding_side:
- 'left': pads on the left of the sequences
- 'right': pads on the right of the sequences
pad_to_multiple_of (`int`, *optional*):
Integer if set will pad the sequence to a multiple of the provided value. This is especially useful to
enable the use of Tensor Core on NVIDIA hardware with compute capability `>= 7.5` (Volta), or on TPUs
which benefit from having sequence lengths be a multiple of 128.
return_attention_mask (`bool`, *optional*):
Set to False to avoid returning attention mask (default: set to model specifics)
r r r' )rJ r )r r constant)constant_valuesleftzInvalid padding strategy:)r1 r rF r+
DO_NOT_PADr) onesint32r r[ r r r: str)
r r r! r3 r# r rP needs_to_be_padded
difference
padding_shapes
r rI z SequenceFeatureExtractor._pad s D ,D,B,B1,EF 6 6 6 ^,J !&8&D*WiJimnJn%);;q@DVVJ - 1K1K K pPSTbPcfpPp %5=O%O35773~;NVXV^V^3_ / 0 #c.&99J G+(;=66*+;B=N=NQR=R!Z & 9YZ\fXg
@B"M:tOaOaA"4#9#9!#<= "! ""f,(;=66*+;B=N=NQR=R*a & 9YcefXg
@B"M:tOaOaA"4#9#9!#<= "! !!= 7.5` (Volta), or on TPUs
which benefit from having sequence lengths be a multiple of 128.
truncation (`bool`, *optional*):
Activates truncation to cut input sequences longer than `max_length` to `max_length`.
NzKWhen setting ``truncation=True``, make sure that ``max_length`` is defined.r r r' )r: r1 r+ )r r r! r# r" rP needs_to_be_truncateds r rD z"SequenceFeatureExtractor._truncate% s . % %
J . jk k+D,B,B1,EF !&8&D*WiJimnJn%);;q@DVVJ #N 3j @ )r rj xs r rl z$SequenceFeatureExtractor.fetch_audiop sr ' .1BCAD$$Q'C C
)3
/ /0 0
-
.$ $ [\`ar\s[t uv v
Ds A/)TNFNNN)NNN)FN)__name__
__module__ __qualname____doc__r; r< r r r6 r8 rc boolr r r[ r` r) r= rI rD r@ rl
__classcell__)r s @r r r s
#S # #U #" 15!% )--126hG (
|
sL
! " sD &&
' ( tC %&
' (hG o-hG $JhG hG $JhG $d{hG j(4/hG
hGZ "&,;,F,F)--1
F" bjj 1L @F" $JF" * F"
$J F" $d{
F"
F"V "&)-"& )" bjj 1L @)" $J)" $J )"
4K )"V @wS49_tDI-N wr r )rr numpyr) audio_utilsr r feature_extraction_utilsr r utilsr r r
r r
get_loggerro loggerr r r r r{ s@ 3 J R R
H %bw 5 bwr