qiY(ddlmZddlZddlmZddlmZmZmZm Z ddl Z ddl Z erddl ZddlmZddlmZddlmZdd lmZdd lmZdd lmZ d ZdZd ZdZGddZddZ GddZ!eGddZ"GddZ#GddZ$y)) annotationsN) dataclass) TYPE_CHECKINGAny AsyncIteratorOptional) AudioSource) AudioFrame)AudioProcessingModule) AudioMixer) AudioStream)Trackiiic0eZdZdZddZddZd dZd dZy) _AudioStreamIteratora Adapter to convert AudioStream (AsyncIterator[AudioFrameEvent]) to AsyncIterator[AudioFrame]. This adapter wraps an AudioStream and extracts the frame from each AudioFrameEvent, making it compatible with AudioMixer which expects AsyncIterator[AudioFrame]. c||_yN) _audio_stream)self audio_streams K/opt/pipecat/venv/lib/python3.12/site-packages/livekit/rtc/media_devices.py__init__z_AudioStreamIterator.__init__Es )c|Srrs r __aiter__z_AudioStreamIterator.__aiter__Hs rcjK|jjd{}|jS7wr)r __anext__frame)revents rrz_AudioStreamIterator.__anext__Ks,((2244{{5s 313cTK|jjd{y7wr)raclosers rr#z_AudioStreamIterator.acloseOs  '')))s (&(N)rrreturnNone)r$zAsyncIterator[AudioFrame])r$r r$r%)__name__ __module__ __qualname____doc__rrrr#rrrrr>s **rrc2|xstjSr)asyncioget_event_looploops r _ensure_loopr0Ss  +7))++rc(eZdZdZddZddZddZy) _APMDelayEstimatora'Thread-safe store for last known output (render) delay in seconds. The sounddevice callbacks are invoked on PortAudio's threads. This helper allows sharing the latest output delay measurement with the input callback so we can set APM's combined stream delay (render + capture). cDtj|_d|_y)N) threadingLock_lock_output_delay_secrs rrz_APMDelayEstimator.__init___s^^% (+rcf|j5t||_dddy#1swYyxYwr)r7floatr8)r delay_secs rset_output_delayz#_APMDelayEstimator.set_output_delaycs+ ZZ 6%*9%5D " 6 6 6s'0c^|j5|jcdddS#1swYyxYwr)r7r8rs rget_output_delayz#_APMDelayEstimator.get_output_delaygs' ZZ *)) * * *s #,Nr&)r;r:r$r%)r$r:)r'r(r)r*rr<r>rrrr2r2Ws,6*rr2cLeZdZUdZded<ded<ded<ded <d ed <dd Zy ) InputCaptureaHolds resources for an active audio input capture. Attributes: source: `rtc.AudioSource` that receives captured frames. This can be published as a `LocalAudioTrack`. input_stream: Underlying `sounddevice.InputStream`. task: Async task that drains a queue and calls `source.capture_frame`. apm: Optional `rtc.AudioProcessingModule` used to process 10 ms frames (AEC, NS, HPF, AGC). When performing echo cancellation, pass this instance to `open_output_player` so reverse frames are provided. delay_estimator: Internal helper used to combine capture and render delays. r sourcez'sd.InputStream' input_streamz asyncio.TasktaskOptional[AudioProcessingModule]apmOptional[_APMDelayEstimator]delay_estimatorctK|jrI|jjs/|jj |jd{ |j j |j jy7:#tj$rYMwxYw#t$rYywxYww)z,Stop capture and close underlying resources.N) rCdonecancelr,CancelledErrorrBstopclose Exceptionrs rr#zInputCapture.acloses 99TYY^^- II    ii     " " $    # # %  ))      sZAB8BBB4B) B8BB&#B8%B&&B8) B52B84B55B8Nr&)r'r(r)r*__annotations__r#rrrr@r@ls,  ""  ((11 rr@cbeZdZdZeeedddd d dZd dZd dZ d dZ d dZ y) OutputPlayeraSimple audio output helper using `sounddevice.OutputStream`. When `apm_for_reverse` is provided, this player will feed the same PCM it renders (in 10 ms frames) into the APM reverse path so that echo cancellation can correlate mic input with speaker output. The OutputPlayer includes an internal `AudioMixer` for convenient multi-track playback. Use `add_track()` and `remove_track()` to dynamically manage tracks, then call `start()` to begin playback. N sample_rate num_channels blocksizeapm_for_reverse output_devicerGc"ddl}|_|_|_|_t _tj_ d_ d_ |_ d_ i_dfd }|j|d||||_y)NrFc|dz}tj}||k\rLjd|}tj|tj||dddf<jd|=nr|dkDr\tjjd|tj|dz|d|dzdf<d||dzddf<jd|=n|j d t |j|jz }jjj|j|tz}t|D]j} | tz} | tz} | |kDry|| | df} t| j!j"dt} jj%| lyy#t$rYwxYw#t$rYwxYw)N)dtypecountrr )len_buffernp frombufferint16fillr:outputBufferDacTime currentTime_delay_estimatorr<rN_apm FRAME_SAMPLESranger tobytes _sample_rateprocess_reverse_stream)outdata frame_count time_infostatus bytes_needed availablechunkoutput_delay_sec num_chunksistartend render_chunk render_framers r _callbackz(OutputPlayer.__init__.._callbacks&?LDLL)IL( ]l3 " e288; W1 LL,/Q/1}}LL),BHHIQRN0()q.(!+,01 Q(!+,LL), Q #()F)FI^I^)^#_ ((4))::;KLyy$(M9 z* A -E-/C[(#*59a<#8L#-$,,.0A0A1m$L 88F %  "%s% A F8G8 GG GGracallbackr[channelsdevice sampleraterU) rl np.ndarrayrmintrnrrorr$r%) sounddevicerj _num_channels _blocksizerf bytearrayr^r,r6 _buffer_lock _play_task_runningre_mixer_track_streams OutputStream_stream) rrSrTrUrVrWrGsdrzs ` rrzOutputPlayer.__init__s !')##  { #LLN26 /-1   + Z'r! "   rcK|j|jvrtd|jd|j&t |j |j |_t||j |j }t|}||f|j|j<|jj|yw)aAdd an audio track to the internal mixer for playback. This creates an `AudioStream` from the track and adds it to the internal mixer. The mixer is created lazily on first track addition. Call `start()` to begin playback of all added tracks. Args: track: The audio track to add (typically from a remote participant). Raises: ValueError: If the track is not an audio track or has already been added. zTrack z already added to playerNrSrT) sidr ValueErrorrr rjrrr add_stream)rtrackstreamstream_iterators r add_trackzOutputPlayer.add_tracks 99++ +veii[0HIJ J ;; $1B1BQUQcQcdDKU0A0APTPbPbc.v6*0/)BEII& /sCC c*K|jj|jd}|y|\}}|j |jj | |j d{y#t $rY%wxYw7#t $rYywxYww)zRemove an audio track from the internal mixer. This removes the track's stream from the mixer and closes it. Args: track: The audio track to remove. N)rpoprr remove_streamrNr#)rrentryrrs r remove_trackzOutputPlayer.remove_track s##'' 48 = "' ;; "  ))/: --/ ! !    "   sX;BA3B-B.B2B3 A?<B>A??BB B BBBcKj%jjs tdj&t j j _fd}tj|_yw)a~Start playback of all tracks in the internal mixer. This begins a background task that consumes frames from the internal mixer and sends them to the output device. Tracks can be added or removed dynamically using `add_track()` and `remove_track()`. Raises: RuntimeError: If playback is already started or no mixer is available. NzPlayback already startedrc>Kd_jj j23d{}jsn4jj |j jKd_ jjjjy76A#t$rYywxYw#d_ jjjjw#t$rYwwxYwxYww)z;Internal playback loop that consumes frames from the mixer.TNF) rrrvrr^extenddatarirLrMrN)r rs r_playback_loopz*OutputPlayer.start.._playback_loop2s DM LL    #';;>>%==LL'' (:(:(<= % LL%%'LL&&(>;! !& LL%%'LL&&( s"D C B<B:B<AC =D4B>9D:B<<C > C D C  D D4D  D DDDDD) rrI RuntimeErrorrr rjrr, create_task)rrs` rrvzOutputPlayer.start"so ?? &t/C/C/E9: : ;; $1B1BQUQcQcdDK $"--n.>?sB B cKd|_|jI|jjs/|jj |jd{t |jjD]\}} |jd{ |jj|j* |jjd{d|_ |jj|jjy7#tj $rYwxYw7#t$rYwxYw7m#t$rYvwxYw#t$rYywxYww)zyStop playback and close the output stream. This also cleans up all added tracks and the internal mixer. FN)rrrIrJr,rKlistrvaluesr#rNclearrrrLrM)rr_s rr#zOutputPlayer.acloseFsO   ?? &t/C/C/E OO " " $ oo%% d1188:; IFA mmo%%  !!# ;; " kk((***DK  LL    LL   /&))   &  +     sAE5 D+D)D+)E5 EEE!(E5 E'E(E,E544E&(E5)D++E>E5EE5E EE5EE5E E# E5"E##E5& E2/E51E22E5)rSrrTrrUrrVrDrW Optional[int]rGrFr$r%)rrr$r%r&) r'r(r)r*DEFAULT_SAMPLE_RATEDEFAULT_CHANNELS BLOCKSIZErrrrvr#rrrrQrQs /,";?'+8<P P  P  P 9 P %P 6P  P d080"@H$rrQceZdZdZdeeeed ddZddZddZ ddZ ddZ d d d d dd dd  dd Z dd  ddZ y) MediaDevicesaHigh-level interface to native audio devices. - Device enumeration helpers. - Audio input capture into `rtc.AudioSource` with optional APM processing. - Audio output player that can feed APM reverse stream for AEC. Design notes: - APM operates on 10 ms frames; this module slices input/output audio into `FRAME_SAMPLES` for processing calls. - For AEC to be effective, render audio that could leak back into the mic should be played through `OutputPlayer` with the same `apm` instance. - Timing alignment: this helper does not attempt to set device latency on APM; for most setups the default behavior is acceptable. N)r/input_sample_rateoutput_sample_raterTrUcxt||_||_||_||_||_d|_d|_yr)r0_loop_in_sr_out_sr _channelsrrerf)rr/rrrTrUs rrzMediaDevices.__init__}s;"$' ' ) %#>B59 rcddl}|j}g}t|D]0\}}|jdddkDs|j d|i|2|S)zList available input devices. Returns a list of dictionaries with the `sounddevice` metadata and an added `index` key corresponding to the device index. rNmax_input_channelsindexr query_devices enumerategetappendrrdevicesresultidxdevs rlist_input_deviceszMediaDevices.list_input_devicesse !""""$')!'* 5HCww+Q/!3 w3s34 5 rcddl}|j}g}t|D]0\}}|jdddkDs|j d|i|2|S)z+List available output devices with indices.rNmax_output_channelsrrrs rlist_output_devicesz MediaDevices.list_output_devicessc """"$')!'* 5HCww,a014 w3s34 5 rcpddl}|jj}t|tt fr|dSdS)z0Return the default input device index (or None).rNrdefaultr~ isinstancertuplerrrs rdefault_input_devicez!MediaDevices.default_input_device0 jj#C$7s1vATArcpddl}|jj}t|tt fr|dSdS)z1Return the default output device index (or None).rNr rrs rdefault_output_devicez"MediaDevices.default_output_devicerrT2) enable_aecnoise_suppressionhigh_pass_filterauto_gain_control input_devicequeue_capacityinput_channel_indexc ddl}jtjjd |s|s|s|rt |||| t nd_ _tj| d fd } |j| dj|jj} | jd fd } tj| } t| |  S) aOpen the default (or chosen) audio input device and start capture. Frames are sliced into 10 ms chunks. If any processing option is enabled, an `AudioProcessingModule` is created and applied to each frame before it is queued for `AudioSource.capture_frame`. To enable AEC end-to-end, call `open_output()` after opening the input device. The output player will automatically use the input's APM for reverse stream processing, enabling echo cancellation. Args: enable_aec: Enable acoustic echo cancellation. noise_suppression: Enable noise suppression. high_pass_filter: Enable high-pass filtering. auto_gain_control: Enable automatic gain control. input_device: Optional input device index (default system device if None). queue_capacity: Max queued frames between callback and async pump. input_channel_index: Optional zero-based device channel to capture. If provided, only that channel is opened (via sounddevice mapping) and used as mono input. Returns: InputCapture: Holder with `source`, `apm`, and `aclose()`. rNr.)echo_cancellationrrr)maxsizec m t|j|jz }rtjnd}t t ||zdzd} j ||tz}t|D]}|tz} | tz} | |kDry|| | df} t| jtjj}  j|  jsj!j"| y#t$rYwxYw#t$rYwxYw#t$rYYwxYw#t$rYwxYw)Nr4g@@r)rsamples_per_channelrSrT)r:rdinputBufferAdcTimer>rmaxset_stream_delay_msrNrgrhr rirrprocess_streamfullcall_soon_threadsafe put_nowait)indatarmrnroinput_delay_secrstotal_delay_ms num_framesrurvrwrrr rErGr/qrs r_input_callbackz0MediaDevices.open_input.._input_callbacksu  &+I,A,AID`D`,`&aOETo>>@AZ]%&)o@P.PTZ-Z\_)`%aN//? % 5J:& M)m+$uSy!|,"(5 $ !%  ?**5166811!,,F+  % $%!sTAD2 D#"E4,E# D/,D2.D//D22 D>=D> E  E  EErar{cK jd{} j|d{47#tj$rYywxYw7 #t$rY)wxYwwr)rr,rK capture_framerN)r rrAs r_pumpz&MediaDevices.open_input.._pump)sk"#%%'ME ..u555 )--6 s^A%;9;AAAA%;AA%AA%A A"A%!A""A%)rArBrCrErG) rrrmrrnrrorr$r%r&)rrr rrr r2rerfr,Queue InputStreamrrvrr@)rrrrrrrrrrrBrrCrErGr/rrAs` @@@@@r open_inputzMediaDevices.open_inputs*D !zzT[[$..tD/3 *.>BS'","3!1"3 C%(O   !0 (/}}^'L+ + -0+ =@+ JM+  + + ^&r~~$^^{{oo    ""57+%+   r)rWct|j|j|j|j||j S)aPCreate an `OutputPlayer` for rendering and (optionally) AEC reverse. If an input device was opened with AEC enabled, the output player will automatically feed the APM's reverse stream for echo cancellation. Args: output_device: Optional output device index (default system device if None). rR)rQrrrrfre)rrWs r open_outputzMediaDevices.open_output?s9 oo II' 11   r) r/#Optional[asyncio.AbstractEventLoop]rrrrrTrrUrr$r%)r$zlist[dict[str, Any]])r$r)rboolrrrrrrrrrrrrr$r@)rWrr$rQ)r'r(r)r*rrrrrrrrrrrrrrrms $59!4"5,":2: :  :  :: :$  BB "&!%"&&* -1E E  E  E  E $E E +E  E T(, %    rr)r/rr$zasyncio.AbstractEventLoop)% __future__rr, dataclassesrtypingrrrrnumpyr_r5rrr audio_framer rEr audio_mixerr rrrrrrrgrrr0r2r@rQrrrrrs#!>>#&#%,  ***,***     FZZzf f r