qi:ddlZddlmZddlZddlmZddlmZddl m Z m Z m Z e jeZ ddeded ed ed ed ed edzdej&fdZdej&dedededej&f dZGddeZdgZy)N)Sequence)SequenceFeatureExtractor) BatchFeature)PaddingStrategy TensorTypeloggingn_freqsf_minf_maxn_mels sample_rate fft_lengthnormreturnc||dk7r tdtj|tj||z z}dt j d|dz zz}dt j d|dz zz} tj || |dz} dd | dz zdz z} | d d| dd z } tj| d tj|d z } tjd tj}d | ddddfz| dd z }| ddddf| d dz }tj|tj||}|/|dk(r*d| d|dz| d|z z }|tj|d z}|S)aCreate a frequency bin conversion matrix (NumPy version). Args: n_freqs (int): Number of frequencies to highlight/apply f_min (float): Minimum frequency (Hz) f_max (float): Maximum frequency (Hz) n_mels (int): Number of mel filterbanks sample_rate (int): Sample rate of the audio waveform fft_length (int): FFT length norm (Optional[str]): If 'slaney', divide the triangular mel weights by the width of the mel band (area normalization). (Default: ``None``) Returns: np.ndarray: Triangular filter banks (fb matrix) of size (``n_freqs``, ``n_mels``) meaning number of frequencies to highlight/apply to x the number of filterbanks. Each column is a filterbank so that assuming there is a matrix A of size (..., ``n_freqs``), the applied result would be ``A @ create_fb_matrix_numpy(A.shape[-1], ...)``. Nslaneyz$norm must be one of None or 'slaney'dtypegF@?g@ rgg@) ValueErrornparangefloat32mathlog10linspace expand_dimszerosmaximumminimum)r r r r rrr all_freqsm_minm_maxm_ptsf_ptsf_diffslopeszero down_slopes up_slopesfbenorms h/opt/pipecat/venv/lib/python3.12/site-packages/transformers/models/gemma3n/feature_extraction_gemma3n.pycreate_fb_matrixr4s> DH,?@@ '4 j8PQI TZZuu} 56 6E TZZuu} 56 6E KKufqj 1E REFN+c1 2E 12Ys #F ^^E1 %y!(D DF 88ARZZ (D&CRC.(F3BK7Kq!"u qr *I D"**[)< =B DH,uQ!,uWf~=> bnnUA&& Iarray dimensionsizestepc|jdk7r td|dk7r||jdz k7r td|j\}}||z |zdz}|dkr$tj|d|f|j S|||f}|j d|j d|z|j df}tjjj|||S) zNA basic NumPy equivalent of PyTorch's unfold for 2D arrays along the last dim.rzFThis unfold implementation currently supports 2D arrays (batch, time).rrzFThis unfold implementation only supports unfolding the last dimension.rr)shapestrides) ndimrr;rr$rr<lib stride_tricks as_strided) r6r7r8r9 batch_sizeoriginal_length num_frames output_shapeoutput_stridess r3_unfoldrFYs zzQabbB9 Q6abb"'++J!D(T1A5JQxxQ-U[[AA D1LmmA& a(84(?qAQRN 66   * *5 n * ]]r5c"eZdZdZddgZ d!dedededed ed ed ed ed ededededededeedzdeedzf fd Z de jde jde e je jffdZ d"de jeezee jzeeezdeezezdedzdededzdeezdzdedzdefd ZxZS)#Gemma3nAudioFeatureExtractoraT An audio feature extractor Universal Speech Models https://huggingface.co/papers/2303.01037. Args: feature_size (`int`, *optional*, defaults to 128): The feature dimension of the extracted features. sampling_rate (`int`, *optional*, defaults to 16000): The sampling rate at which the audio files should be digitalized expressed in hertz (Hz). padding_value (`float`, *optional*, defaults to 0.0): Padding value used to pad the audio. Should correspond to silences. return_attention_mask (`bool`, *optional*, defaults to `True`): Whether to return the attention mask for the generated MEL spectrograms. frame_length_ms (`float`, *optional*, defaults to 32.0): The length of a frame in milliseconds. hop_length_ms (`float`, *optional*, defaults to 10.0): Length of the overlapping windows for the STFT used to obtain the Mel Frequency coefficients. min_frequency (`float`, *optional*, defaults to 125.0): The minimum frequency (in Hz) for the Mel filterbank. max_frequency (`float`, *optional*, defaults to 7600.0): The maximum frequency (in Hz) for the Mel filterbank. preemphasis (`float`, *optional*, defaults to 0.97): The preemphasis coefficient. preemphasis_htk_flavor (`bool`, *optional*, defaults to `True`): Whether to use HTK-style preemphasis. fft_overdrive (`bool`, *optional*, defaults to `True`): Whether to use FFT overdrive. dither (`float`, *optional*, defaults to 0.0): Adds dithering. In other words, adds a small Gaussian noise to each frame. E.g. use 0.0001 to add dithering with a normal distribution centered around 0.0 with standard deviation 0.0001 (assuming [-1,+1] range of raw_speech). The value 0.0 means no dithering. Dithering has similar effect as `spectrogram(mel_floor=...)`. It reduces the high log_mel_fbank values for signals with hard-zero sections, when VAD cutoff is present in the signal. input_scale_factor (`float`, *optional*, defaults to 1.0): Scaling factor applied to the input waveform. mel_floor (`float`, *optional*, defaults to 1e-05): Minimum value for Mel spectrograms to avoid log(0). per_bin_mean (`Optional[Sequence[float]]`, *optional*): Mean values for per-bin normalization. per_bin_stddev (`Optional[Sequence[float]]`, *optional*): Standard deviation values for per-bin normalization. input_featuresinput_features_maskN feature_size sampling_rate padding_valuereturn_attention_maskframe_length_ms hop_length_ms min_frequency max_frequency preemphasispreemphasis_htk_flavor fft_overdriveditherinput_scale_factor mel_floor per_bin_meanper_bin_stddevc Nt|d||||d|||_||_| |_| |_| |_| |_| |_tt||zdz |_ tt||zdz |_ tj|tj|_dt#j$t#j&|jz}|j r|dz}||_tj*|jtj,}ddtj.dtj0z|z|jz z z}|j3tj,|_t7|j(dzdz||||j8d||_|,tj|j=dd||_nd|_|,tj|j=dd||_ yd|_ y) N)rKrLrMrNg@@rrg?r)r r r r rrr)!super__init__rQrRrSrTrUrVrWintround frame_length hop_lengthrr6float64rXr ceillog2rrrcospiastypewindowr4rL mel_filtersreshaperYrZ)selfrKrLrMrNrOrPrQrRrSrTrUrVrWrXrYrZkwargsr hann_arangeri __class__s r3r^z%Gemma3nAudioFeatureExtractor.__init__s(  %''"7    +*&&<#* "4mo&E&N OPeMM$AF$JKL)2::>$))DIId.?.?$@AA    !OJ$ii 1 1D BFF1ruu9{#:T=N=N#NOOPmmBJJ/ +OOq(1,**!   # " 6 > >q!\ RD  $D   %"$((>":"B"B1a"VD "&D r5waveformattention_maskrc |jdk(rtj|d}|jdkDrO||jtjj |j j|jzz}|jdk7r||jz}|jdz}t|d||j}|jdkDr|jrS|dd dfd|jz z}|dddf|j|dd d fzz }tj||gd}n*|ddd f|j|dd dfzz }n |dd df}||j z}tj"j%||j&d }tj(|} tj*| |j,} tj.tj0| |j2} |j4| |j4z } |j6| |j6z } | j9d} |d d |jjt:} | | d | j dfS) rr)axisrr)r7r8r9.Nr)nrt)r=rr#rVrandomrandnr;rhrrWrarFrbrSrT concatenaterifftrfftrabsmatmulrjlogr%rXrYrZsqueezebool)rlrprqframe_size_for_unfoldframes_to_processfirst_in_frame rest_in_frameframesstftmagnitude_specmel_spec log_mel_specmel_spectrogrammasks r3_extract_spectrogramz1Gemma3nAudioFeatureExtractor._extract_spectrogramsa ==A ~~hQ7H ;; $++ 0P0W0WX`XfXf0g"ggH  " "c )$"9"99H $ 1 1A 5$HAV]a]l]lm   c !**!237!;sTEUEU?U!V 1#qt) 99^T-=-=>vvbjj4>>BC    ('$*;*;;L    *'$*=*==L&..q100188>%?'<'t:jYZm^`^h^hjr]sCt%<)< $J3=>Rbjj"&((> > *J7 8!!1"7 " ! = =~?\?\] .LFD44VXXtDLFD  " "6==#< = ' ' - . .G[ \&  %?s9'E)i>ruTg@@g$@g@_@g@g ףp= ?TTrurgh㈵>NN)longestiSTrNT)__name__ __module__ __qualname____doc__model_input_namesr_floatrrr^rrtuplerliststrrrrr __classcell__)ros@r3rHrHls)V*+@A #"&*!%#$%!'+"$'/315#B'B'B' B' $ B'  B'B'B'B'B'!%B'B'B'"B'B' uo,!B'"!$.#B'H+ARZZ+A+AX]^`^h^hjljtjt^tXu+A`1:!(),26-1? JJe,tBJJ/??$tE{BSS? o-? $J ?  ?  $J ? j(4/?  $d{?  ? r5rH)N)r collections.abcrnumpyr!feature_extraction_sequence_utilsrfeature_extraction_utilsrutilsrrr get_loggerrloggerr_rrrr4rFrH__all__r\r5r3rs $I499   H %: : : :  :  :  : *:ZZ:z^2::^#^S^^ ^&^ #;^ B * *r5