o Si'@sddlmZddlmZmZmZmZmZmZm Z ddl Z ddl Z ddl mZmZddlmZmZmZddlmZddlmZddlmZdd lmZdd lmZmZmZm Z m!Z!m"Z"m#Z#m$Z$m%Z%m&Z&m'Z'm(Z(m)Z)Gd d d Z*dS) )Path)CallableDictListLiteralOptionalSetTupleN)Interval IntervalTree) AudioSource Recording VideoInfo) save_audio) AugmentFn)FeatureExtractor)SupervisionSegment) DecibelsPathlikeSecondsSetContainingAnything add_durations asdict_nonullcompute_num_samplescompute_num_windows'compute_start_duration_for_extended_cutfastcopyifnoneoverlaps to_hashablec @sdeZdZUdZeed<eed<eed<eed<ee ed<e eed<e eed<e eed <e eed <e eed <e ed <e ed <e ed<e e ed<e egdfed<e gejfed<e geeje ejffed<e gejfed<e ed<e ed<e ed<e ed<e ed<e ed<e ed<e ed<e ed<e ed<e ed<e ed<e ed <e ed!<e ed"<e ed#<e ed$<e ed%<e ed&<e ed'<e ed(<e ed)<ed*efd+d,Zd*efd-d.Zd/d0Zed*e fd1d2Zed*ee fd3d4Zd5ed*ed6fd7d8Z 9 : ; ;dde d?e ed@e ed*df dAdBZ ; ;dd>> from lhotse import Recording, SupervisionSegment, MonoCut >>> rec = Recording(id='rec1', duration=10.0, sampling_rate=8000, num_samples=80000, sources=[...]) >>> sups = [ ... SupervisionSegment(id='sup1', recording_id='rec1', start=0, duration=3.37, text='Hey, Matt!'), ... SupervisionSegment(id='sup2', recording_id='rec1', start=4.5, duration=0.9, text='Yes?'), ... SupervisionSegment(id='sup3', recording_id='rec1', start=6.9, duration=2.9, text='Oh, nothing'), ... ] >>> cut = MonoCut(id='rec1-cut1', start=0.0, duration=6.0, channel=0, recording=rec, ... supervisions=[sups[0], sups[1]]) .. note:: All Cut classes assume that the :class:`~lhotse.supervision.SupervisionSegment` time boundaries are relative to the beginning of the cut. E.g. if the underlying :class:`~lhotse.audio.Recording` starts at 0s (always true), the cut starts at 100s, and the SupervisionSegment inside the cut starts at 3s, it really did start at 103rd second of the recording. In some cases, the supervision might have a negative start, or a duration exceeding the duration of the cut; this means that the supervision in the recording extends beyond the cut. Cut allows to check and read audio data or features data:: >>> assert cut.has_recording >>> samples = cut.load_audio() >>> if cut.has_features: ... feats = cut.load_features() It can be visualized, and listened to, inside Jupyter Notebooks:: >>> cut.plot_audio() >>> cut.play_audio() >>> cut.plot_features() Cuts can be used with Lhotse's :class:`~lhotse.features.base.FeatureExtractor` to compute features. >>> from lhotse import Fbank >>> feats = cut.compute_features(extractor=Fbank()) It is also possible to use a :class:`~lhotse.features.io.FeaturesWriter` to store the features and attach their manifest to a copy of the cut:: >>> from lhotse import LilcomChunkyWriter >>> with LilcomChunkyWriter('feats.lca') as storage: ... cut_with_feats = cut.compute_and_store_features( ... extractor=Fbank(), ... storage=storage ... ) Cuts have several methods that allow their manipulation, transformation, and mixing. Some examples (see the respective methods documentation for details):: >>> cut_2_to_4s = cut.truncate(offset=2, duration=2) >>> cut_padded = cut.pad(duration=10.0) >>> cut_extended = cut.extend_by(duration=5.0, direction='both') >>> cut_mixed = cut.mix(other_cut, offset_other_by=5.0, snr=20) >>> cut_append = cut.append(other_cut) >>> cut_24k = cut.resample(24000) >>> cut_sp = cut.perturb_speed(1.1) >>> cut_vp = cut.perturb_volume(2.) >>> cut_rvb = cut.reverb_rir(rir_recording) .. note:: All cut transformations are performed lazily, on-the-fly, upon calling ``load_audio`` or ``load_features``. The stored waveforms and features are untouched. .. caution:: Operations on cuts are not mutating -- they return modified copies of :class:`.Cut` objects, leaving the original object unmodified. A :class:`.Cut` that contains multiple segments (:class:`SupervisionSegment`) can be decayed into smaller cuts that correspond directly to supervisions:: >>> smaller_cuts = cut.trim_to_supervisions() Cuts can be detached from parts of their metadata:: >>> cut_no_feat = cut.drop_features() >>> cut_no_rec = cut.drop_recording() >>> cut_no_sup = cut.drop_supervisions() Finally, cuts provide convenience methods to compute feature frame and audio sample masks for supervised regions:: >>> sup_frames = cut.supervisions_feature_mask() >>> sup_samples = cut.supervisions_audio_mask() See also: - :class:`lhotse.cut.MonoCut` - :class:`lhotse.cut.MixedCut` - :class:`lhotse.cut.CutSet` idstartduration sampling_rate supervisions num_samples num_frames num_features frame_shift features_type has_recording has_features has_videovideo from_dict load_audio load_video load_featurescompute_and_store_features drop_featuresdrop_recordingdrop_supervisionsdrop_alignmentsdrop_in_memory_data iter_datatruncatepad extend_byresample perturb_speed perturb_tempoperturb_volumephone reverb_rirmap_supervisionsmerge_supervisionsfilter_supervisionsfill_supervisionwith_features_path_prefixwith_recording_path_prefixreturncCst|j|j|jdS)Nr$)rr"r#r$selfrMC/home/ubuntu/.local/lib/python3.10/site-packages/lhotse/cut/base.pyends zCut.endcCst|}i|dt|jiS)Ntype)rrP__name__)rLdrMrMrNto_dictsz Cut.to_dictcKst|dii|j|S)a Returns a shallow copy of self, with specified attributes overwritten. Example: >>> cut = MonoCut(id="old-id", ...) ... cut2 = cut.copy(id="new-id") ... assert cut.id == "old-id" ... assert cut2.id == "new-id" NrM)rP__dict__)rL replace_attrsrMrMrNcopys zCut.copycCsTt|jdkr dSddlm}|dt|jdddD] \}}t||r'dSqdS) NFr)sliding_windowcS|jSNr"srMrMrNz2Cut.has_overlapping_supervisions..keyT)lenr%cytoolzrXsortedr)rLrXleftrightrMrMrNhas_overlapping_supervisionss  z Cut.has_overlapping_supervisionscsfddjDS)a Return the supervisions in this Cut that have modified time boundaries so as not to exceed the Cut's start or end. Note that when ``cut.supervisions`` is called, the supervisions may have negative ``start`` values that indicate the supervision actually begins before the cut, or ``end`` values that exceed the Cut's duration (it means the supervision continued in the original recording after the Cut's ending). .. caution:: For some tasks such as speech recognition (ASR), trimmed supervisions could result in corrupted training data. This is because a part of the transcript might actually reside outside of the cut. csg|]}|jqSrM)trimr#.0r]rKrMrN sz,Cut.trimmed_supervisions..r%rKrMrKrNtrimmed_supervisionsszCut.trimmed_supervisions timestamp)r r cCsNd|kr |jksnJd|d|j|j|d}|j|d}||fS)z Split a cut into two cuts at ``timestamp``, which is measured from the start of the cut. For example, a [0s - 10s] cut split at 4s yields: - left cut [0s - 4s] - right cut [4s - 10s] rz0 < z < )r#)offset)r#r:)rLrnrerfrMrMrNsplits.  z Cut.splitFNotheroffset_other_by allow_paddingsnr preserve_idcCs ddlm}|||||||dS)z3Refer to :function:`~lhotse.cut.mix` documentation.mix)rortrurv)setry)rLrrrsrtrurvryrMrMrNry s zCut.mixcCs ddlm}||||j||dS)a^ Append the ``other`` Cut after the current Cut. Conceptually the same as ``mix`` but with an offset matching the current cuts length. Optionally scale down (positive SNR) or scale up (negative SNR) the ``other`` cut. Returns a MixedCut, which only keeps the information about the mix; actual mixing is performed during the call to ``load_features``. :param preserve_id: optional string ("left", "right"). When specified, append will preserve the cut ID of the left- or right-hand side argument. Otherwise, a new random ID is generated. rwrx)rorurv)rzryr#)rLrrrurvryrMrMrNappends z Cut.append extractor augment_fncCs*|}|dur|||j}|||jS)aH Compute the features from this cut. This cut has to be able to load audio. :param extractor: a ``FeatureExtractor`` instance used to compute the features. :param augment_fn: optional ``WavAugmenter`` instance for audio augmentation. :return: a numpy ndarray with the computed features. N)r0r$extract)rLr|r}samplesrMrMrNcompute_features1s  zCut.compute_featurescKs~ddlm}|jdd}|dur|\}}|td|jt |||j D]}| |j}|j |j |jdddq)|S)zV Display a plot of the waveform. Requires matplotlib to be installed. rN)axisgreeng?)coloralpha)matplotlib.pyplotpyplotr0sumsubplotsplotnplinspacer#rbr%rhaxvspanr"rO)rLaxkwargspltrfig supervisionrMrMrN plot_audioBs    zCut.plot_audiocCs&ddlm}|}|||jdS)z Display a Jupyter widget that allows to listen to the waveform. Works only in Jupyter notebook/lab or similar (e.g. Colab). r)Audio)rate)IPython.displayrr0squeezer$)rLrrrMrMrN play_audioQs  zCut.play_audiocCs.ddlm}t|ddd}||S)z^ Display the feature matrix as an image. Requires matplotlib to be installed. rNrw)rrrflipr2 transposematshow)rLrfeaturesrMrMrN plot_features[s  zCut.plot_featureswordalignment_typec Cs^ddlm}ddlm}ddlm}t|jdksJd|jd}|jdur,||jvs4Jd|d|}|j j }| | |}|j pGd } |jpLd } |t|ddd} |d |jd | d | |jd ddddddt|j|D],\} } t| d}|| j|j|d}|j|d|rdnd| jdddd|j|ddq||dS)ze Display the alignment on top of a spectrogram. Requires matplotlib to be installed. rN)Fbank)compute_num_framesrwzHCannot plot alignment: there has to be exactly one supervision in a Cut.zBCannot plot alignment: missing alignment field or alignment type ''z zCut ID:z , Speaker:z , Language:bothmajorTF)rwhich labelbottomlabeltopbottomtoprW)r)r$F- wvertical)fontsizerrotationk)r)rrlhotser lhotse.utilsrrbr% alignmentr|r$r=rspeakerlanguagerrrrtitler! tick_params enumerateboolrOr)textsymbolaxvlineshow)rLrrrrsupfbankr$featsrrridxitemis_even end_framerMrMrNplot_alignmentdsX          zCut.plot_alignmentTcenterkeep_overlapping min_durationcontext_direction)rrerfrandomkeep_all_channelsCutSetc s"ddlm}ddlm}ddlm}g}|jdd} |jD]n|dur+jj } } n t jj ||d\} } |j | | || d } |sK| fd d } |s{t | |s{ttd d | jDdksdJd| jdj| _t | |r{| jdkr{| d} t| jdkrj| _|| q||S)a> Splits the current :class:`.Cut` into as many cuts as there are supervisions (:class:`.SupervisionSegment`). These cuts have identical start times and durations as the supervisions. When there are overlapping supervisions, they can be kept or discarded via ``keep_overlapping`` flag. For example, the following cut:: Cut |-----------------| Sup1 |----| Sup2 |-----------| is transformed into two cuts:: Cut1 |----| Sup1 |----| Sup2 |-| Cut2 |-----------| Sup1 |-| Sup2 |-----------| For the case of a multi-channel cut with multiple supervisions, we can either trim while respecting the supervision channels (in which case output cut has the same channels as the supervision) or ignore the channels (in which case output cut has the same channels as the input cut). .. hint:: If the resulting trimmed cut contains a single supervision, we set the cut id to the ``id`` of this supervision, for better compatibility with downstream tools, e.g. comparing the hypothesis of ASR with the reference in icefall. .. hint:: If a MultiCut is trimmed and the resulting trimmed cut contains a single channel, we convert it to a MonoCut. :param keep_overlapping: when ``False``, it will discard parts of other supervisions that overlap with the main supervision. In the illustration above, it would discard ``Sup2`` in ``Cut1`` and ``Sup1`` in ``Cut2``. In this mode, we guarantee that there will always be exactly one supervision per cut. :param min_duration: An optional duration in seconds; specifying this argument will extend the cuts that would have been shorter than ``min_duration`` with actual acoustic context in the recording/features. If there are supervisions present in the context, they are kept when ``keep_overlapping`` is true. If there is not enough context, the returned cut will be shorter than ``min_duration``. If the supervision segment is longer than ``min_duration``, the return cut will be longer. :param context_direction: Which direction should the cut be expanded towards to include context. The value of "center" implies equal expansion to left and right; random uniformly samples a value between "left" and "right". :param keep_all_channels: If ``True``, the output cut will have the same channels as the input cut. By default, the trimmed cut will have the same channels as the supervision. :return: a list of cuts. rwMixedCut)MultiCutrTindex_mixed_tracksN)r"r# new_duration directionror#keep_excessive_supervisions_supervisions_indexcs |jjkSrZr!r\segmentrMrNr^s z*Cut.trim_to_supervisions..cs|]}t|jVqdSrZ)rchannelrirMrMrN z+Cut.trim_to_supervisions..zTrimmed cut has supervisions with different channels. Either set `keep_all_channels=True` to keep original channels or `keep_overlapping=False` to retain only 1 supervision per trimmed cut.r)mixedrmultirrzrindex_supervisionsr%r"r#rr:rE isinstancerbr num_channelsto_monor!r{ from_cuts) rLrrrrrrrcutssupervisions_index new_startrtrimmedrMrrNtrim_to_supervisionssB >        zCut.trim_to_supervisions rP max_pausemax_segment_duration delimitercsddlm}|dur d}|dur|j}g}|jD]}|jdus)||jvs)|j|s*qt|j|dddddgfg} tddD]K\} } | jd krQqE| d \} } | j | j |kr| j | j |kr|| | j| jg| j | j | j d }|| | dgf| d <qE| | | dgfqEt| D]1\} \} }| t |jd | |j| j |j | j|j| j|j|j|j|fd d|Did qqt||d}|jd|dS)a Splits the current :class:`.Cut` into its constituent alignment items (:class:`.AlignmentItem`). These cuts have identical start times and durations as the alignment item. Additionally, the `max_pause` option can be used to merge alignment items that are separated by a pause shorter than `max_pause`. If `max_segment_duration` is specified, we will keep merging consecutive segments until the duration of the merged segment exceeds `max_segment_duration`. For the case of a multi-channel cut with multiple alignments, we can either trim while respecting the supervision channels (in which case output cut has the same channels as the supervision) or ignore the channels (in which case output cut has the same channels as the input cut). .. hint:: If the resulting trimmed cut contains a single supervision, we set the cut id to the ``id`` of this supervision, for better compatibility with downstream tools, e.g. comparing the hypothesis of ASR with the reference in icefall. .. hint:: If a MultiCut is trimmed and the resulting trimmed cut contains a single channel, we convert it to a MonoCut. .. hint:: If you have a Cut with multiple supervision segments and you want to trim it to the word-level alignment, you can use the :meth:`.Cut.merge_supervisions` method first to merge the supervisions into a single one, followed by the :meth:`.Cut.trim_to_alignments` method. For example:: >>> cut = cut.merge_supervisions(type='word', delimiter=' ') >>> cut = cut.trim_to_alignments(type='word', max_pause=1.0) .. hint:: The above technique can also be used to segment long cuts into roughly equal duration segments, while respecting alignment boundaries. For example, to split a Cut into 10s segments, you can do:: >>> cut = cut.merge_supervisions(type='word', delimiter=' ') >>> cut = cut.trim_to_alignments(type='word', max_pause=10.0, max_segment_duration=10.0) :param type: The type of the alignment to trim to (e.g. "word"). :param max_pause: The maximum pause allowed between the alignments to merge them. If ``None``, no merging will be performed. [default: None] :param delimiter: The delimiter to use when joining the alignment items. :param keep_all_channels: If ``True``, the output cut will have the same channels as the input cut. By default, the trimmed cut will have the same channels as the supervision. :param num_jobs: Number of parallel workers to process the cuts. :return: a CutSet object. r) AlignmentItemNgcSrYrZr[)arMrMrNr^Wr_z(Cut.trim_to_alignments..r`rw)rr"r#-csg|]}|qSrMrM)rjj alignmentsrMrNrksz*Cut.trim_to_alignments..) r! recording_idr"r#rrrrgenderrrlF)rr)lhotse.supervisionrr#r%rrdrrstripr"rOjoinr{rr!rrrrrrr)rLrPrrrrrnew_supervisionsrmerged_alignmentsir prev_item prev_indicesnew_itemindicesnew_cutsrMrrNtrim_to_alignmentss` 3       zCut.trim_to_alignmentsc Csddlm}|js||gSt|jddd}|dg}|dj}g}d}|ddD]D}|j||kr@||t||j}q+|dj} t|| |j d} ||j | | d d  |j d |d ||d7}|g}|j}q+t |dkr|dj} t|| |j d} ||j | | d d  |j d |d |td d |Dt |jksJdtdd |Ddt |jd|d|||S)uO Return a new CutSet with Cuts based on supervision groups. A supervision group is a set of supervisions with no gaps between them (or gaps shorter than ``max_pause``). This is similar to the concept of an `utterance group` as described in this paper: https://arxiv.org/abs/2211.00482 For example, the following cut:: Cut ╔═════════════════════════════════════════════════════════════════════════════════╗ ║┌──────────────────────┐ ┌────────┐ ║ ║│ Hello this is John. │ │ Hi │ ║ ║└──────────────────────┘ └────────┘ ║ ║ ┌──────────────────────────────────┐ ┌───────────────────┐║ ║ │ Hey, John. How are you? │ │ What do you do? │║ ║ └──────────────────────────────────┘ └───────────────────┘║ ╚═════════════════════════════════════════════════════════════════════════════════╝ is transformed into two cuts:: Cut 1 Cut 2 ╔════════════════════════════════════════════════╗ ╔═══════════════════════════╗ ║┌──────────────────────┐ ║ ║┌────────┐ ║ ║│ Hello this is John. │ ║ ║│ Hi │ ║ ║└──────────────────────┘ ║ ║└────────┘ ║ ║ ┌──────────────────────────────────┐║ ║ ┌───────────────────┐║ ║ │ Hey, John. How are you? │║ ║ │ What do you do? │║ ║ └──────────────────────────────────┘║ ║ └───────────────────┘║ ╚════════════════════════════════════════════════╝ ╚═══════════════════════════╝ For the case of a multi-channel cut with multiple supervisions, we keep all the channels in the recording. :param max_pause: An optional duration in seconds; if the gap between two supervisions is longer than this, they will be treated as separate groups. By default, this is set to 0.0, which means that no gaps are allowed between supervisions. :return: a ``CutSet``. rwrcSrYrZr[r\rMrMrNr^r_z0Cut.trim_to_supervision_groups..r`rNrJF)ror#rrcsrrZrbr%rjcrMrMrNrrz1Cut.trim_to_supervision_groups..zQThe total number of supervisions decreased after trimming to supervision groups. csrrZrrrMrMrNrrz != z This is likely a bug. Please report it here: https://github.com/lhotse-speech/lhotse/issues/new, and provide the following information: original cut: z , new cuts: )rzrr%rdrOr"r{maxrr$r:with_idr!rbrr) rLrrr%supervision_groupcur_endr group_idxrror#rMrMrNtrim_to_supervision_groupssd *         zCut.trim_to_supervision_groupshoprc Csddlm}|s |}|jrI||jjs-Jd|jd|d|jjd||jjd ||jjsIJd|d|jjd||jjdg}t|j||}|j d d }t |D]}| |j |||||d  |jd |q\||S) a Return a list of shorter cuts, made by traversing this cut in windows of ``duration`` seconds by ``hop`` seconds. The last window might have a shorter duration if there was not enough audio, so you might want to use either filter or pad the results. :param duration: Desired duration of the new cuts in seconds. :param hop: Shift between the windows in the new cuts in seconds. :param keep_excessive_supervisions: bool. When a cut is truncated in the middle of a supervision segment, should the supervision be kept. :return: a list of cuts made from shorter duration windows. rwrz[cut.id=z[] Window duration must be defined to result in an integer number of video frames (duration=z * fps=z = z).zb[cut.id={self.id}] Window hop must be defined to result in an integer number of video frames (hop=Trrr)rzrr-r.fps is_integerr!rr#rranger{r:r r) rLr#rrrr n_windowsrrrMrMrNcut_into_windowssH       zCut.cut_into_windowsrkeep_idscsvddlm}tt|jtfdd|jDi}|r9t||r9|jD]}tfdd|j jD||j j<q%|S)a Create a two-level index of supervision segments. It is a mapping from a Cut's ID to an interval tree that contains the supervisions of that Cut. The interval tree can be efficiently queried for overlapping and/or enveloping segments. It helps speed up some operations on Cuts of very long recordings (1h+) that contain many supervisions. :param index_mixed_tracks: Should the tracks of MixedCut's be indexed as additional, separate entries. :param keep_ids: If specified, we will only index the supervisions with the specified IDs. :return: a mapping from Cut ID to an interval tree of SupervisionSegments. rwrc3*|]}|jvrt|j|j|VqdSrZr!r r"rOrirrMrNr0 z)Cut.index_supervisions..c3rrZrrirrMrNr9r) rrrrr!r r%rtrackscut)rLrrrindexedtrackrMrrNrs     zCut.index_supervisions storage_pathformatencodingc Kst|}|jd i|}|dur|||j}t|||j||dt|j|j|jd|jd|jtdtt |j t |dgd}t | |j|jt|drQ|jnd|jrZ|jdSddS) a Store this cut's waveform as audio recording to disk. :param storage_path: The path to location where we will store the audio recordings. :param format: Audio format argument supported by ``torchaudio.save`` or ``soundfile.write``. Please refer to the relevant library's documentation depending on which audio backend you're using. :param encoding: Audio encoding argument supported by ``torchaudio.save`` or ``soundfile.write``. Please refer to the relevant library's documentation depending on which audio backend you're using. :param augment_fn: an optional callable used for audio augmentation. Be careful with the types of augmentations used: if they modify the start/end/duration times of the cut and its supervisions, you will end up with incorrect supervision information when using this API. E.g. for speed perturbation, use ``CutSet.perturb_speed()`` instead. :param kwargs: additional arguments passed to ``Cut.load_audio()``. Example, if saving a MixedCut, we can specify `mono_downmix=True` to downmix the tracks to mono before saving. :return: a new Cut instance. N)r$rr rwfile)rPchannelssource)r!r$r&r#sourcescustom)r!r%r%rrM)rr0r$rr stemshaper listrrstrrto_cutr!r%hasattrr%r,r)rLrrr r}rr recordingrMrMrNr@s@    zCut.save_audiomin_speaker_dimspeaker_to_idx_mapuse_alignment_if_existsc CsP|js Jd|jd|dur#ddtttdd|jDD}t|}|dur0t||}t ||j f}|jD]j}||j }|r}|j r}||j vr}|j |D]*}|j dkr`t|j |jnd} |j|jkrpt|j|jn|j } d ||| | f<qQq;|j dkrt|j |jnd} |j|jkrt|j|jn|j } d ||| | f<q;|S) a Return a matrix of per-speaker activity in a cut. The matrix shape is (num_speakers, num_frames), and its values are 0 for nonspeech **frames** and 1 for speech **frames** for each respective speaker. This is somewhat inspired by the TS-VAD setup: https://arxiv.org/abs/2005.07272 :param min_speaker_dim: optional int, when specified it will enforce that the matrix shape is at least that value (useful for datasets like CHiME 6 where the number of speakers is always 4, but some cuts might have less speakers than that). :param speaker_to_idx_map: optional dict mapping speaker names (strings) to their global indices (ints). Useful when you want to preserve the order of the speakers (e.g. speaker XYZ is always mapped to index 2) :param use_alignment_if_exists: optional str, key for alignment type to use for generating the mask. If not exists, fall back on supervision time spans. zPNo features available. Can't compute supervisions feature mask for cut with ID: .NcSi|]\}}||qSrMrMrjrspkrMrMrN z-Cut.speakers_feature_mask..cs|]}|jVqdSrZrrirMrMrNrz,Cut.speakers_feature_mask..rrw)r,r!rrdrzr%rbminrzerosr'rrr"roundr)rOr# rLr-r.r/ num_speakersmaskr speaker_idxalistetrMrMrNspeakers_feature_masksP       zCut.speakers_feature_maskc CsX|js Jd|jd|dur#ddtttdd|jDD}t|}|dur0t||}t ||j f}|jD]n}||j }|r|j r||j vr|j |D],}|j dkr_t|j |jnd} |j|jkrnt|j|jnt|j|j} d ||| | f<qQq;|j dkrt|j |jnd} |j|jkrt|j|jnt|j|j} d ||| | f<q;|S) a Return a matrix of per-speaker activity in a cut. The matrix shape is (num_speakers, num_samples), and its values are 0 for nonspeech **samples** and 1 for speech **samples** for each respective speaker. This is somewhat inspired by the TS-VAD setup: https://arxiv.org/abs/2005.07272 :param min_speaker_dim: optional int, when specified it will enforce that the matrix shape is at least that value (useful for datasets like CHiME 6 where the number of speakers is always 4, but some cuts might have less speakers than that). :param speaker_to_idx_map: optional dict mapping speaker names (strings) to their global indices (ints). Useful when you want to preserve the order of the speakers (e.g. speaker XYZ is always mapped to index 2) :param use_alignment_if_exists: optional str, key for alignment type to use for generating the mask. If not exists, fall back on supervision time spans. ONo recording available. Can't compute supervisions audio mask for cut with ID: r0NcSr1rMrMr2rMrMrNr4r5z+Cut.speakers_audio_mask..csr6rZr7rirMrMrNrr8z*Cut.speakers_audio_mask..rrw)r+r!rrdrzr%rbr9rr:r&rrr"rr$rOr#r<rMrMrNspeakers_audio_masksV          zCut.speakers_audio_maskcCsddlm}|||dS)aU Return a 1D numpy array with value 1 for **frames** covered by at least one supervision, and 0 for **frames** not covered by any supervision. :param use_alignment_if_exists: optional str, key for alignment type to use for generating the mask. If not exists, fall back on supervision time spans. rw)compute_supervisions_frame_mask)r/)rzrF)rLr/rFrMrMrNsupervisions_feature_masks zCut.supervisions_feature_maskcCs |js Jd|jdtj|jtjd}|jD]k}|rX|jrX||jvrX|j|D]-}|jdkr8t |j|j nd}|j |j krHt |j |j nt |j |j }d|||<q)q|jdkret |j|j nd}|j |j krut |j |j nt |j |j }d|||<q|S)aW Return a 1D numpy array with value 1 for **samples** covered by at least one supervision, and 0 for **samples** not covered by any supervision. :param use_alignment_if_exists: optional str, key for alignment type to use for generating the mask. If not exists, fall back on supervision time spans. rDr0)dtyperg?) r+r!rr:r&float32r%rr"r;r$rOr#)rLr/r>rr@rArBrMrMrNsupervisions_audio_masks<      zCut.supervisions_audio_maskid_cCs t||dS)z'Return a copy of the Cut with a new ID.r)r)rLrKrMrMrNr ?s z Cut.with_id)rqFNN)NNrZ)r)TNrF)NNrF)rq)NT)FN)NNN)5rQ __module__ __qualname____doc__r)__annotations__rintrrrrrrrrndarrayr torchTensorpropertyrOdictrSrVrgrmrprryr{rrrrrrrrrrrrrr rrrrCrErGrJr rMrMrMrNr "s w               : w } ` 6   & B B E  +r )+pathlibrtypingrrrrrrr numpyrrR intervaltreer r lhotse.audior r rlhotse.audio.backendrlhotse.augmentationrlhotse.featuresrrrrrrrrrrrrrrrrrr rMrMrMrNs $    <