o Si*@stddlmZddlmZmZddlZddlZddlm Z m Z m Z GdddZ dej defd d ZGd d d ZdS) )sqrt)ListOptionalN)DecibelsSecondscompute_num_samplesc @seZdZdZ  ddejdedeede fdd Z dd ejd ed eed ejfddZ e d efddZ e d eejfddZe d ejfddZe d ejfddZ  dd ejdeed e fddZdS) AudioMixera Utility class to mix multiple waveforms into a single one. It should be instantiated separately for each mixing session (i.e. each ``MixedCut`` will create a separate ``AudioMixer`` to mix its tracks). It is initialized with a numpy array of audio samples (typically float32 in [-1, 1] range) that represents the "reference" signal for the mix. Other signals can be mixed to it with different time offsets and SNRs using the ``add_to_mix`` method. The time offset is relative to the start of the reference signal (only positive values are supported). The SNR is relative to the energy of the signal used to initialize the ``AudioMixer``. .. note:: Both single-channel and multi-channel signals are supported as reference and added signals. The only requirement is that the when mixing 2 multi-channel signals, they must have the same number of channels. .. note:: When the AudioMixer contains multi-channel tracks, 2 types of mixed signals can be generated: - `mixed_audio` mixes each channel independently, and returns a multi-channel signal. If there is a mono track, it is added to all the channels. - `mixed_mono_audio` mixes all channels together, and returns a single-channel signal. N base_audio sampling_ratereference_energy base_offsetcCsV|g|_t||g|_||_|jd|_|jdj|_|dur&t||_dS||_dS)a AudioMixer's constructor. :param base_audio: A numpy array with the audio samples for the base signal (all the other signals will be mixed to it). :param sampling_rate: Sampling rate of the audio. :param reference_energy: Optionally pass a reference energy value to compute SNRs against. This might be required when ``base_audio`` corresponds to zero-padding. :param base_offset: Optionally pass a time offset for the base signal. rN) tracksroffsetsr shape num_channelsdtype audio_energyr )selfr r r r rF/home/ubuntu/.local/lib/python3.10/site-packages/lhotse/audio/mixer.py__init__"s  zAudioMixer.__init__audiooffsettotalreturncCs|jdks Jd|j|dur|jd|}|jd||ks0J|jdd|d|tj|d|||jd|ffdS)Nz audio.ndim= +  <= )rr) pad_width)ndimrnppad)rrrrrrr _pad_track@szAudioMixer._pad_trackcCs4d}t|j|jD]\}}t|||jd}q |S)Nrrziprrmaxr)rlongestrrrrrnum_samples_totalMzAudioMixer.num_samples_totalc$jfddtjjDS) Return a list of numpy arrays with the shape (C, num_samples), where each track is zero padded and scaled adequately to the offsets and SNR used in ``add_to_mix`` call. c g|] \}}j||dqS)rrr$.0rtrackrrrr [z,AudioMixer.unmixed_audio..)r)r&rrrrr3r unmixed_audioT  zAudioMixer.unmixed_audiocCs|j}tj|j|f|jd}t|j|jD],\}}|jddkr.|jdkr.t ||jdf}|dd|||jdf|7<q|S) Return a numpy ndarray with the shape (num_channels, num_samples) - a mix of the tracks supplied with ``add_to_mix`` calls. rrrN) r)r"zerosrrr&rrrtilerrmixedrr2rrr mixed_audio`s(zAudioMixer.mixed_audiocCsz|j}tjd|f|jd}t|j|jD]&\}}|jddkr'tj|ddd}|dd|||jdf|7<q|S)z Return a numpy ndarray with the shape (1, num_samples) - a mix of the tracks supplied with ``add_to_mix`` calls. rr:rT)axiskeepdimsN) r)r"r;rr&rrrsumr=rrrmixed_mono_audiors(zAudioMixer.mixed_mono_audiosnrcCs|jdkrdS|dksJdt||j}d}|dur8|jdkr8t|}|dkr8|jd| d}t||}|j|||j||j d|j kri|j dkri|j ddkrit d |j dd |j d t |j |j d|_ dS) a Add audio of a new track into the mix. :param audio: An array of audio samples to be mixed in. :param snr: Signal-to-noise ratio, assuming `audio` represents noise (positive SNR - lower `audio` energy, negative SNR - higher `audio` energy) :param offset: How many seconds to shift `audio` in time. For mixing, the signal will be padded before the start with low energy values. :return: rNr +Negative offset in mixing is not supported.g?g$@ rzCannot mix audios with z and z channels.) sizerr r rrrappendrrr ValueErrorr')rrrDrnum_samples_offsetgainadded_audio_energy target_energyrrr add_to_mixs&     zAudioMixer.add_to_mix)Nr N)__name__ __module__ __qualname____doc__r"ndarrayintrfloatrrr$propertyr)rr7r?rCrrNrrrrr sP    rrrcCstt|dS)Nr)rVr"average)rrrrrsrc @seZdZdZ ddejdedefddZ dd ejd e d e e d ejfd dZ e d e fddZ e d eejfddZe d ejfddZ dd ejd efddZdS) VideoMixerze Simple video "mixing" class that actually does not mix anything but supports concatenation. r base_videofpsr cCsfddlm}|g|_t||g|_||_|jdj|_||_|j|jd|jd|j ddS)Nr) IntervalTree) intervaltreer\rrrr[rtreeaddir)rrZr[r r\rrrrs (zVideoMixer.__init__Nvideorrrc Csz|dur |jd|}|jd||ks#J|jdd|d|tjjj|dddddd|||jd|fdddS)Nrrrconstant)modevalue)rtorchnn functionalr#)rr`rrrrrr$s zVideoMixer._pad_trackcCs4d}t|j|jD]\}}t|||jd}q |S)Nrr%)rr(rr`rrrnum_frames_totalr*zVideoMixer.num_frames_totalcr+)r,cr-r.r/r0r3rrr4r5z,VideoMixer.unmixed_video..)rgr&rrr6rr3r unmixed_videor8zVideoMixer.unmixed_videocCs^|j}|jd|f|jdjdd}t|j|jD]\}}|||||jd<q|S)r9rrN)rgr new_zerosrr&rr=rrr mixed_videos &zVideoMixer.mixed_videocCs|jdkrdS|dksJdt||j}ddlm}||||jd}|j|r8Jd|d|j|j ||j ||j |dS)Nrr rE)Intervalz%Cannot add an overlapping video. Got z( while we have the following intervals: ) rGrr[r]rkrr^overlaps all_intervalsrrHradd)rr`r frame_offsetrkintervalrrrrNs     zVideoMixer.add_to_mix)r rO)rPrQrRrSrdTensorrVrrrUrr$rWrgrrhrjrNrrrrrYs@   rY)mathrtypingrrnumpyr"rd lhotse.utilsrrrrrTrVrrYrrrrs &