o 2wiP<@sddlZddlmZddlmZddlmZmZmZm Z ddl Z ddl m Z ddl mZmZmZddlmZmZmZddlmZdd lmZdd lmZmZdd lmZmZmZm Z Gd d d eZ!de"de!fddZ#dS)N) defaultdict)partial)AnyListOptionalUnion)tqdm) RecordingSetSupervisionSet dill_enabled)CutSetMixedCutMixTrack)mix) parallel_map) add_durationsuuid4)MAX_TASKS_WAITINGBaseMeetingSimulatorMeetingSamplerreverberate_cutscs,eZdZdZ    d0dedededeffd d Zd efd d ZddZde j d e fddZ e dd1deed dfddZ d2deded efddZe d        !d3d"ed#eed$eed%eeeefd&eeed'eed(eeded)ed*ed efd+d,Zd"ed-ed efd.d/ZZS)4ConversationalMeetingSimulatora This simulator uses the method described in https://arxiv.org/abs/2204.00890 and implemented in https://github.com/BUTSpeechFIT/EEND_dataprep. Note that a similar method of meeting simulation is also implemented in https://github.com/jsalt2020-asrdiar/jsalt2020_simulate. The basic idea is to sample the silence and overlap durations collectively for all speakers so that we get similar speech/silence/overlap characteristics as the original distribution. In general, this method produces more realistic overlap durations than the `SpeakerIndependentMeetingSimulator`. This is done by learning the histogram of 3 distributions: same speaker pause, different speaker pause, and different speaker overlap. In this implementation, we learn the histograms from provided data, otherwise we use the initialization values as shift of a Gamma distribution with scale 1.0. ?@?same_spk_pausediff_spk_pausediff_spk_overlapprob_diff_spk_overlapcsNt|||fD]}|dus|dksJdq ||_||_||_||_dS)a :param same_spk_pause: the mean pause duration between utterances of the same speaker. [Default: 2.0] :param diff_spk_pause: the mean pause duration between utterances of different speakers. [Default: 3.0] :param diff_spk_overlap: the mean overlap duration between utterances of different speakers. [Default: 2.0] :param prob_diff_spk_overlap: the probability of overlap between utterances of different speakers. [Default: 0.5] NrzDurations must be > 0.)super__init__rrrr)selfrrrrduration __class__o/home/ubuntu/sommelier/.venv/lib/python3.10/site-packages/lhotse/workflows/meeting_simulation/conversational.pyr 's  z'ConversationalMeetingSimulator.__init__returnc Cs6|jjd|jdd|jdd|jdd|jdd S)Nz (same_spk_pause=z.2fz, diff_spk_pause=z, diff_spk_overlap=z, prob_diff_spk_overlap=))r$__name__rrrr)r!r%r%r&__repr__As z'ConversationalMeetingSimulator.__repr__cCsFddlm}|dd|jd|_|dd|jd|_|dd|jd|_dS)Nr)gammar)ascaleloc) scipy.statsr+rsame_spk_pause_distrdiff_spk_pause_distrdiff_spk_overlap_dist)r!r+r%r%r&_init_defaultsJs z-ConversationalMeetingSimulator._init_defaultsvaluescCs,ddlm}tj|ddd\}}|||fS)z Compute the histogram of the given values and return the bin edges and the corresponding probabilities. r) rv_histogramdT)binsdensity)r/r5np histogram)r!r4r5hist bin_edgesr%r%r&_compute_histogram_distQs  z6ConversationalMeetingSimulator._compute_histogram_distTNmeetingsc Cs|durtd|dSt|tsJdddlm}g}g}g}|ddt|ddd }|D]S}t d t |D]I}||j ||d j kr\| ||j ||d jq>||j ||d jkrx| ||j ||d jq>| ||d j||j q>q5|t||_|t||_|t||_t |t |dkrt |t |t |nd |_|j|_|j|_|j|_td |dS) z Learn the distribution of the meeting parameters from a given dataset. :param meetings: a SupervisionSet containing the meetings to be used for Nz/No meetings provided, using default parameters.z2The meetings must be provided as a SupervisionSet.r)groupbycS|jSN) recording_idsr%r%r&qz4ConversationalMeetingSimulator.fit..cSs |j|jfSrA)rBstartrCr%r%r&rErs keyrzLearned parameters: )logginginfor3 isinstancer cytoolz.itertoolzr?sortedr4rangelenspeakerappendrGendr=r9arrayr0r1r2rmeanrrrprint) r!r>r?same_spk_pause_valuesdiff_spk_pause_valuesdiff_spk_overlap_valuesrecording_segmentssegmentsir%r%r&fit[sf      z"ConversationalMeetingSimulator.fitF utterancesallow_3fold_overlapc Cs|j}t|}dd|jj|dD}dd|jj|dD}dd|jj|dD}|jj|j|d}t|}dg} |dj } |dj dj } d d |D} | | | <t t| d d } |dj}td t|D]}||j dj }||d j dj }||kr||}n>||dkr||}n3||}t| d kr|st|t| | | |dt| | d  |d}n t|t| | | |d}| }t| ||d} | | t| ||j |d} | | |<t t| d d } | d} qitt t|| ddd\}} tt}t|| D]\}}||j dj ||fqg}|D]3\}}|d\}}|d dD]\}}t||t|| |dd d}q.t|t||d}||qt |ddd}ttt|dS)a Create a MixedCut object from a list of MonoCuts (utterances). We sample pauses and/or overlaps from the initilized or learned distributions. Then, we create a MixedCut where each track represents a different speaker. :param utterances: a CutSet containing the utterances to be mixed. :param allow_3fold_overlap: if True, allow 3-fold overlaps between speakers. [Default: False] :return: a MixedCut object. cSg|]}t|dqSround.0xr%r%r& zBConversationalMeetingSimulator._create_mixture..)sizecSrarbrdrfr%r%r&rirjcSrarbrdrfr%r%r&ris )prkrcSsi|]}|dqS)rmr%)rgspkrr%r%r& szBConversationalMeetingSimulator._create_mixture..T)reverserJ) sampling_ratecSs|dS)NrJr%rhr%r%r&rEsz@ConversationalMeetingSimulator._create_mixture..rHN)offset allow_padding)cuttyperscSr@rA)rsrrr%r%r&rErF)idtracks)speakersrQr0rvsr1r2 bernoullirlistr" supervisionsrRrOr4rqrPminrrSzipritemsrrrvr strr)r!r_r`ryNsame_spk_pausesdiff_spk_pausesdiff_spk_overlapsdiff_spk_bernoullioffsets cur_offset first_spk last_utt_endlast_utt_end_timessrr]cur_spkprev_spkot spk_tracksuttrsrxspkspk_uttstrackrGr%r%r&_create_mixtures                z.ConversationalMeetingSimulator._create_mixturerc4@rrJcuts num_meetings num_repeatsnum_speakers_per_meetingspeaker_count_probsmax_duration_per_speakermax_utterances_per_speakerseednum_jobsc  Csddlm} |dur|durtd|durd}t|tr |g}|dur/dt|gt|}t|dddur;|| |_t|||||||| d} t | } t t ||d}g}| d krnt t || |d d D]}||qent t|| | | td |d d D]}||q}t|S) a8 Simulate the desired number of multi-speaker meetings. :param cuts: CutSet containing the MonoCut objects to be used for simulation. :param num_meetings: the number of meetings to simulate. :param num_repeats: the number of times to repeat the provided cuts. This means that the number of simulated meetings depends on how many cuts are available. :param num_speakers_per_meeting: the number of speakers per meeting. If a list is provided, the number of speakers per meeting is sampled from this list. [Default: 2] :param speaker_count_probs: the probability of each number of speakers per meeting. [Default: None] :param max_duration_per_speaker: the maximum duration of each speaker in a meeting. [Default: 20.0] :param max_utterances_per_speaker: the maximum number of utterances per speaker in a meeting. [Default: 5] :param allow_3fold_overlap: if True, allow 3-fold overlaps between speakers. [Default: False] :param seed: the random seed to be used for simulation. [Default: 0] :param num_jobs: the number of jobs to use for simulation. Use more jobs to speed up simulation when you have large number of source utterances. [Default: 1] r)r{Nz4Either num_meetings or num_repeats must be provided.rr0)rrrrrrr) simulatorr`rJzSimulating meetings)totaldesc)r queue_size)r/r{ ValueErrorrMintrQgetattrr3riterr_simulate_workerrmaprSrrr from_cuts)r!rrrrrrrr`rrr{sampler sampler_iterworkmixturesmixturer%r%r&simulate sb #    z'ConversationalMeetingSimulator.simulaterirscGst|g|RSrA)r)r!rrr%r%r& reverberatelsz*ConversationalMeetingSimulator.reverberate)rrrrrA)F) NNrcNrrFrrJ)r) __module__ __qualname____doc__floatr rr*r3r9ndarrayrr=r rr r^r boolr rrrrrr r __classcell__r%r%r#r&rs|  C k     brr`rcCs|j||dS)N)r`)r)r_r`rr%r%r&rpsr)$rK collectionsr functoolsrtypingrrrrnumpyr9rlhotser r r lhotse.cutr r rlhotse.cut.setrlhotse.parallelr lhotse.utilsrr(lhotse.workflows.meeting_simulation.baserrrrrrrr%r%r%r&s(     \