o SiB;@sddlZddlZddlmZddlmZmZddlmZddl m Z m Z m Z m Z mZmZmZmZddlZddlZddlmZddlmZddlmZdd lmZdd lmZmZm Z m!Z!m"Z"m#Z#m$Z$m%Z%eGd d d eZ&dS) N) dataclass)partialreduce)add)AnyCallableIterableListOptionalSequenceTupleUnion) Recording)DataCut)Features)SupervisionSegment) add_durationsfastcopyhash_str_to_intis_equal_or_containsmerge_items_with_delimiteroverlapsrich_exception_infouuid4c@s@eZdZUdZeed<edefddZede e j fddZ ede e j fdd Z e d%d ede eeje ejffd d ZdeeeefdefddZ     d&de eeefdedededeede ede edefddZ  d'dede eeeegefddfd d!Zed"e ddfd#d$Z!dS)(MonoCuta :class:`~lhotse.cut.MonoCut` is a :class:`~lhotse.cut.Cut` of a single channel of a :class:`~lhotse.audio.Recording`. In addition to Cut, it has a specified channel attribute. This is the most commonly used type of cut. Please refer to the documentation of :class:`~lhotse.cut.Cut` to learn more about using cuts. See also: - :class:`lhotse.cut.Cut` - :class:`lhotse.cut.MixedCut` - :class:`lhotse.cut.CutSet` channelreturncCsdS)NselfrrC/home/ubuntu/.local/lib/python3.10/site-packages/lhotse/cut/mono.py num_channels,szMonoCut.num_channelscCs|jrA|jj|j|jd}|jd|jdkr$|d|jddf}|S|jd|jdkr?tj||ddddffdd}|SdS)z Load the features from the underlying storage and cut them to the relevant [begin, duration] region of the current MonoCut. )startdurationrrN)axis) has_featuresfeaturesloadr#r$shape num_framesnp concatenate)r featsrrr! load_features1s"zMonoCut.load_featurescCs"|jr|jj|j|j|jdSdS)a Load the audio by locating the appropriate recording in the supplied RecordingSet. The audio is trimmed to the [begin, end] range specified by the MonoCut. :return: a numpy ndarray with audio samples, with shape (1 , N ) )channelsoffsetr$N) has_recording recording load_audiorr#r$rrrr!r4DszMonoCut.load_audioT with_audiocCs$|jr|jj|j|j|j|dSdS)a Load the subset of video (and audio) from attached recording. The data is trimmed to the [begin, end] range specified by the MonoCut. :param with_audio: bool, whether to load and return audio alongside video. True by default. :return: a tuple of video tensor and optionally audio tensor (or ``None``), or ``None`` if this cut has no video. )r0r1r$r5N) has_videor3 load_videorr#r$)r r5rrr!r7Ts zMonoCut.load_videor0c stt}t|r gnt|jjs!Jdd|jj|p(tdk}|rM|s0\t|jd|j|j |j fdd|j D|j dSdd l m}||jdtd |j |j fd d|j D|j d S) a- Select specified channels from this cut. Supports extending to other channels available in the underlying :class:`Recording`. If a single channel is provided, we'll return a :class:`~lhotse.cut.MonoCut`, otherwise we'll return a :class:`~lhotse.cut.MultiCut`. zCannot select channels=z= because they are not a subset of self.recording.channel_ids=r-cs$g|]}t|jrt|dqS))r)rrr.0sr0rr! s   z)MonoCut.with_channels..)idr3r#r$r supervisionscustomrMultiCutchancsg|] }t|jr|qSr)rrr9r<rr!r=s  )r>r#r$rr?r@) isinstanceintsetissubsetr3 channel_idslenrr>r#r$r?r@lhotserB)r r0channel_is_intmonorBrr<r! with_channelsjs@     zMonoCut.with_channelsNFr rir_recordingnormalize_output early_onlyaffix_id rir_channels room_rng_seedsource_rng_seedc s>|jsJd|jrtdd|_dus%tfdd|Ds%Jddur@dg}|dur:ttt|j }|dur@|}t |dkrn|j j |||||d }fd d |j D} t|rg|j d n|j || d Sddlm} ttt ||j j |||||d }fdd |j D} t| ||| dS)a Return a new ``DataCut`` that will convolve the audio with the provided impulse response. If the `rir_recording` is multi-channel, the `rir_channels` argument determines which channels will be used. By default, we use the first channel and return a MonoCut. If we reverberate with a multi-channel RIR, we return a MultiCut. If no ``rir_recording`` is provided, we will generate an impulse response using a fast random generator (https://arxiv.org/abs/2208.04101). Note that the generator only supports simulating reverberation with a single microphone, so we will return a MonoCut in this case. :param rir_recording: The impulse response to use for convolving. :param normalize_output: When true, output will be normalized to have energy as input. :param early_only: When true, only the early reflections (first 50 ms) will be used. :param affix_id: When true, we will modify the ``MonoCut.id`` field by affixing it with "_rvb". :param rir_channels: The channels of the impulse response to use. First channel is used by default. If multiple channels are specified, this will produce a MultiCut instead of a MonoCut. :param room_rng_seed: The seed for the room configuration. :param source_rng_seed: The seed for the source position. :return: a modified copy of the current ``MonoCut``. z:Cannot apply reverberation on a MonoCut without Recording.zAttempting to reverberate a MonoCut that references pre-computed features. The feature manifest will be detached, as we do not support feature-domain reverberation.Nc3s|]}|jkVqdSN)r")r:c)rOrr!   z%MonoCut.reverb_rir..z(Invalid channel index in `rir_channels`.rr)rOrPrQrRrSrTrUcsg|]}|jdqS)rR reverb_rirr9rZrr!r=s z&MonoCut.reverb_rir.._rvb)r>r3r?rAcsg|] }|jdqS))rRrr[r9)rRr0rr!r=s)r3r?r)r2r'loggingwarningr(allrstrrr>rIr3r\r?rmultirBlistrange from_mono) r rOrPrQrRrSrTrU recording_rvbsupervisions_rvbrBr)rRr0rOr!r\sv!     zMonoCut.reverb_rir delimiter merge_policycustom_merge_fnc s|ttd|dkd|dur|nfddt|jdddtd kr'|Sd j}d j}t|| |jd }t d dD}t ddD}t ddt d dDrot ddDrot d|jdtddDd j||d jdddDddDddDddDfdd|Dfdd|Dd }t||gdS)a0 Return a copy of the cut that has all of its supervisions merged into a single segment. The new start is the start of the earliest superivion, and the new duration is a minimum spanning duration for all the supervisions. The text fields of all segments are concatenated with a whitespace. :param merge_policy: one of "keep_first" or "delimiter". If "keep_first", we keep only the first segment's field value, otherwise all string fields (including IDs) are prefixed with "cat#" and concatenated with a hash symbol "#". This is also applied to ``custom`` fields. Fields with a ``None`` value are omitted. :param custom_merge_fn: a function that will be called to merge custom fields values. We expect ``custom_merge_fn`` to handle all possible custom keys. When not provided, we will treat all custom values as strings. It will be called roughly like: ``custom_merge_fn(custom_key, [s.custom[custom_key] for s in sups])`` # keep_first)rh return_firstNcstt|SrV)mapra)kvs) merge_func_rr!,sz,MonoCut.merge_supervisions..cSs|jSrV)r#)r;rrr!rr.s)keyrrr%) sampling_ratecs.|]}|jdur|jD]}|VqqdSrV)r@keysr:r;rorrr!rX8$z-MonoCut.merge_supervisions..csrurV) alignmentrvrwrrr!rX;rxcss|] \}}t||VqdSrV)r)r:s1s2rrr!rX?scss|]}|jduVqdSrVtextr9rrr!rX?rYzYou are merging overlapping supervisions that have text transcripts. The result is likely to be unusable if you are going to train speech recognition models (cut id: z).css|]}|jVqdSrV)r>r9rrr!rXIs cs|] }|jr|jVqdSrVr|r9rrr!rXNcsrrV)speakerr9rrr!rXOrcsrrV)languager9rrr!rXPrcsrrV)genderr9rrr!rXQrcs&i|]fddDqS)c3.|]}|jdur|jvr|jVqdSrV)r@r9rorr!rXU8MonoCut.merge_supervisions...rr:) merge_customsupsrr! Rs  z.MonoCut.merge_supervisions..cs&i|]ttfddDqS)c3rrV)ryr9rrr!rXarr)rrr)rrr!r]s  ) r> recording_idr#r$rr}rrrr@ry)r?)rrsortedr?rIr#endrrtrFanyzipwarningswarnr>rrrjoinr) r rirjmstartmend mduration custom_keysalignment_keysmsupr)rrqrr!merge_supervisionss\    (    #zMonoCut.merge_supervisionsdatacCsddlm}|ddd|vrt|dnd}d|vr&t|dnd}d|vr1|dng}d|vr=||dd|vrF|dtd i|||dd |Dd S) Nr)deserialize_custom_fieldtyper(r3r?r@cSsg|]}t|qSr)r from_dictr9rrr!r=sz%MonoCut.from_dict..)r(r3r?r)lhotse.serializationrpoprrrr)rrr(r3supervision_infosrrr!rms$      zMonoCut.from_dict)T)NTFTrNNN)rhN)"__name__ __module__ __qualname____doc__rE__annotations__propertyr"rr r,ndarrayr/r4boolr torchTensorr7r r rrMrr r\rarrrr staticmethoddictrrrrr!rsh   / s er)'r^r dataclassesr functoolsrroperatorrtypingrrrr r r r r numpyr,r lhotse.audiorlhotse.cut.datarlhotse.featuresrlhotse.supervisionr lhotse.utilsrrrrrrrrrrrrr!s  (    (