o Si@sddlZddlmZmZmZmZddlZddlmZm Z m Z m Z m Z ddl mZddlmZmZ    dd ee efd ed ed ed eedee ddff ddZ dde d ed ed ed eef ddZ dded ed ed ed eef ddZdee fddZdS)N) GeneratorListOptionalUnion)CutSetMonoCut RecordingSetSupervisionSegment add_durations)trim_supervisions_to_recordings)fastcopyis_module_availablebasecpuFmanifest model_namedeviceforce_nonoverlapping download_rootreturncksrtds Jdt|trt|||||fi|EdHdSt|tr5t|||||fi|EdHdStd)a  Use OpenAI Whisper model to annotate either RECORDINGS_MANIFEST, RECORDINGS_DIR, or CUTS_MANIFEST. It will perform automatic segmentation, transcription, and language identification. If the first argument is a CutSet, it will overwrite the supervisions with the results of the inference. Note: this is an experimental feature of Lhotse, and is not guaranteed to yield high quality of data. See the original repo for more details: https://github.com/openai/whisper :param manifest: a ``RecordingSet`` or ``CutSet`` object. :param language: specify the language if known upfront, otherwise it will be auto-detected. :param model_name: one of available Whisper variants (base, medium, large, etc.). :param device: Where to run the inference (cpu, cuda, etc.). :param force_nonoverlapping: if True, the Whisper segment time-stamps will be processed to make sure they are non-overlapping. :param download_root: if specified, the model will be downloaded to this directory. Otherwise, it will be downloaded to the default location specfied by whisper. :param decode_options: additional options to pass to the ``whisper.transcribe`` function. :return: a generator of cuts (use ``CutSet.open_writer()`` to write them). whisperzThis function expects OpenAI Whisper to be installed. You can install it via 'pip install git+https://github.com/openai/whisper.git' (see https://github.com/openai/whisper for details).Nz;The ``manifest`` must be either a RecordingSet or a CutSet.)r isinstancer_annotate_recordingsr_annotate_cuts ValueError)rrrrrdecode_optionsrL/home/ubuntu/.local/lib/python3.10/site-packages/lhotse/workflows/whisper.pyannotate_with_whisper s2    r recordingsc +sddl}|j|||d}|D]Ujdkr%tdjdjdqtd  d}|j d||d |fd d d D} } | ra|rUt | n| } tt| d d| _| VqdS)zE Helper function that annotates a RecordingSet with Whisper. rNrrzSkipping recording ' '. It has 4 channels, but we currently only support mono input.>modelaudioc stg|]6}|d|ddkrtjd|ddjt|dddt|d|d dd |d d d qS endstartr-id06d)ndigitsr$) sampling_ratetextlanguage)r, recording_idr*durationr1r2)r r,roundr strip.0segment recordingresultrr ]s   z(_annotate_recordings..segmentsF)r supervisionsverboser)r load_model num_channelsloggingwarningr,torch from_numpyresample load_audiosqueeze transcribeto_cut_postprocess_timestampslistr r?) rrrrrrrr&r'r?cutrr:rrDs6     rcutsc +sddl}|j|||d}|D]Ijdkr%tdjdjdqtd  d}|j d||d |fd d d D} t |rQt | n| d } | VqdS)z? Helper function that annotates a CutSet with Whisper. rNr r!zSkipping cut 'r"r#r$r%c s|g|]:}|d|ddkrtjd|ddjt|dddtt|dj|d dd |d d d qSr()r r,r3r5r minr4r6r7rNr<rrr=s    z"_annotate_cuts..r>)r?r)rrArBrCrDr,rErFrGrHrIrJr rL) rOrrrrrrr&r'r?new_cutrrQrrzs,     rr?cCsvddlm}t|ddd}t|dkr|Sg}|d|D]\}}|j|jkr.|j|jd}||q|||S)z Whisper tends to have a lot of overlapping segments due to inaccurate end timestamps. Under a strong assumption that the input speech is non-overlapping, we can fix that by always truncating to the start timestamp of the next segment. r)sliding_windowcSs|jSN)r*)srrrsz)_postprocess_timestamps..)key)r))cytoolzrSsortedlenr)r*trimappend)r?rSoutcurnxtrrrrLs     rL)rrFNrT)rCtypingrrrrrElhotserrrr r lhotse.qar lhotse.utilsr r strboolrrrrLrrrrs`    > ; 2