o Si)@sddlmZmZddlZddlmZddlmZddlm Z ddl m Z ddl m Z ddlmZmZdd lmZGd d d eZdS) )DictOptionalN)CrossEntropyLoss)Dataset)validate) RecordingSet)CutSet)collate_featurescollate_matrices)SupervisionSetc sbeZdZdZ   d dedeedeededdf fd d Z dede e e j ffd d ZZS)DiarizationDatasetaB A PyTorch Dataset for the speaker diarization task. Our assumptions about speaker diarization are the following: * we assume a single channel input (for now), which could be either a true mono signal or a beamforming result from a microphone array. * we assume that the supervision used for model training is a speech activity matrix, with one row dedicated to each speaker (either in the current cut or the whole dataset, depending on the settings). The columns correspond to feature frames. Each row is effectively a Voice Activity Detection supervision for a single speaker. This setup is somewhat inspired by the TS-VAD paper: https://arxiv.org/abs/2005.07272 Each item in this dataset is a dict of: .. code-block:: { 'features': (B x T x F) tensor 'features_lens': (B, ) tensor 'speaker_activity': (B x num_speaker x T) tensor } .. note: In cases when padding needs to be performed during collation, the cuts are silence-padded, and the speaker activity tensor is padded with CrossEntropyLoss().ignore_index. Constructor arguments: :param cuts: a ``CutSet`` used to create the dataset object. :param uem: a ``SupervisionSet`` used to set regions for diarization :param min_speaker_dim: optional int, when specified it will enforce that the matrix shape is at least that value (useful for datasets like CHiME 6 where the number of speakers is always 4, but some cuts might have less speakers than that). :param global_speaker_ids: a bool, indicates whether the same speaker should always retain the same row index in the speaker activity matrix (useful for speaker-dependent systems) :param root_dir: a prefix path to be attached to the feature files paths. NFcutsuemmin_speaker_dimglobal_speaker_idsreturnc stt||s||_nEtdd|D}tj||d}g}|D]\}||vr;|ddD7}q)|fdd||D7}q)tj|t |d|_|raddt |jj Dnd|_ ||_ dS) NcSsi|] }|jr|jj|jqS) has_recording recordingid).0crrN/home/ubuntu/.local/lib/python3.10/site-packages/lhotse/dataset/diarization.py Dsz/DiarizationDataset.__init__..) recordings supervisionscSsg|]}|jqSr)data)ritrrr Msz/DiarizationDataset.__init__..cs8h|]}j|j|jdD] }|jj|j|jdq qS))beginend)start)overlaprr rtrim)ruem_itrtreerr Osz.DiarizationDataset.__init__..cSsi|]\}}||qSrr)ridxspkrrrrYs)super__init__rr rrfrom_manifestsindex_supervisionsitemsr from_segments enumeratespeakersr) selfr rrrr uem_intervalsrcut_id __class__r%rr+5s<     zDiarizationDataset.__init__cs2t|\}}||tfdd|DtjddS)Nc3s"|] }|jjjdVqdS))rspeaker_to_idx_mapN)speakers_feature_maskrr1)rcutr2rr es z1DiarizationDataset.__getitem__..) padding_value)features features_lensspeaker_activity)r r r ignore_index)r2r r=r>rr:r __getitem___s   zDiarizationDataset.__getitem__)NNF)__name__ __module__ __qualname____doc__rrr intboolr+rstrtorchTensorrA __classcell__rrr5rr s")$*r )typingrrrItorch.nnrtorch.utils.datarlhotser lhotse.audior lhotse.cutrlhotse.dataset.collationr r lhotse.supervisionr r rrrrs