o :i4 @sDddlmZddlZddlZddlZddlmZddlm Z m Z hdZ d3dee de fd d Zd3dee de fd d Zdeee d ededefddZ d4deejdede dejfddZde fddZde de fdd Zdejd!efd"d#Z $ d5d%ejde d&ed'efd(d)Z *d6d%ed+e d,efd-d.Zd/d0Zd1d2ZdS)7)ListN) AudioSegment)detect_leading_silencesplit_on_silence> 。！，：；？,.?!:;d tokens_list max_tokenscCsg}g}|D].}t|dkr$t|dkr$|tvs|dkr$|d|q|||tvr4||g}qt|dkr@||g}g}|D]}t|t||krX||qFt|dkrc|||}qFt|dkrq|||S)aB Splits the input tokens list into chunks according to punctuations, each with a maximum number of tokens. Args: token_list (list of str): The list of tokens to be split. max_tokens (int): The maximum number of tokens per chunk. Returns: List[str]: A list of text chunks. r )len punctuationappendextend)rr sentencescurrent_sentencetokenchunks current_chunksentencer!+/home/ubuntu/LuxTTS/zipvoice/utils/infer.pychunk_tokens_punctuation s2           r#cCsg}g}|D]}|dkrt|dkr||g}||qt|dkr*||g}g}|D]}t|t||krB||q0t|dkrM|||}q0t|dkr[|||S)aN Splits the input tokens list into chunks according to speaker-turn symbol [S1], each with a maximum number of tokens. Args: token_list (list of str): The list of tokens to be split. max_tokens (int): The maximum number of tokens per chunk. Returns: List[str]: A list of text chunks. z[S1]r)rrr)rrdialogscurrent_dialogrrrdialogr!r!r"chunk_tokens_dialogAs*          r' max_durationprompt_durationtoken_durationc stt|}t|dddfddttD}fddttD}g}g}d} |D]2} | |t||t| ||krP|| | t| 7} q0t|dkr[||| g}t| } q0t|dkrn||||fS)a Sort and group the input list of token sequences into batches, where each batch's total duration does not exceed the maximum. Args: tokens_list (List[List[int]]): A list of token sequences, where each inner list represents a sequence of tokens. max_duration (float): The maximum allowed total duration for each batch. prompt_duration (float): The duration cost per prompt in the batch. token_duration (float): The duration cost per token. Returns: batches: List[List[List[int]]]: A list of batches, where each batch is a list of token sequences that fit within the max duration. index: List[int]: The original index of each sentence, used to recover the sequential order in the future. cSs t|dS)N)r)xr!r!r"s z!batchify_tokens..)keycg|]}|dqS)rr!.0iindexed_sorted_tokensr!r" sz#batchify_tokens..cr/)r+r!r0r3r!r"r5s r)list enumeratesortedrangerr) rr(r)r*indexed_tokensindex sorted_tokensbatchesbatch batch_sizetokensr!r3r"batchify_tokensls4           rA皙?]r fade_duration sample_ratereturncCst|dkr|r |dStgSt||}|dkr"tj|ddS|d}|ddD]T}t||jd|jd}|dkrHtj||gdd}q,tjdd||jdd}tj|dd| f|d| df||dd|fd||d|dfgdd}q,|S)a Concatenates audio chunks with cross-fading between consecutive chunks. Args: chunks: List of audio tensors, each with shape (C, T) where C = number of channel, T = time dimension (samples) fade_duration: Duration of cross-fade in seconds sample_rate: Audio sample rate in Hz Returns: Concatenated audio tensor with shape (N, T_total) r+rr)dimN)device.) rtorchtensorintcatminshapelinspacerH)rrDrE fade_samplesfinal next_chunkkfader!r!r"cross_fade_concats,   rUtextcCs |}|dtvr|d7}|S)z2Add punctuation if there is not in the end of textrr )stripr)rVr!r!r"add_punctuations rX prompt_wav sampling_ratecCs2t|\}}||krtjj||d}||}|S)a Load the waveform with torchaudio and resampling if needed. Parameters: prompt_wav: path of the prompt wav. sampling_rate: target sampling rate. Returns: Loaded prompt waveform with target sampling rate, PyTorch tensor of shape (C, T) ) orig_freqnew_freq) torchaudioload transformsResample)rYrZprompt_sampling_rate resamplerr!r!r"load_prompt_wavs rc target_rmscCs2ttt|}||kr|||}||fS)am Normalize the rms of prompt_wav is it is smaller than target rms. Parameters: prompt_wav: PyTorch tensor with shape (C, T). target_rms: target rms value Returns: prompt_wav: normalized prompt wav with shape (C, T). promt_rms: rms of original prompt wav. Will be used to re-normalize the generated wav. )rIsqrtmeansquare)rYrd prompt_rmsr!r!r"rms_norms  riFaudio only_edge trail_silcCsbt||}|st|ddddd}tjdd}|D]}||7}qt|dd}|tj|d}t|S)a Remove silences longer than 1 second, and edge silences longer than 0.1 seconds Parameters: audio: PyTorch tensor with shape (C, T). sampling_rate: sampling rate of the audio. only_edge: If true, only remove edge silences. trail_sil: the duration of added trailing silence in ms. Returns: PyTorch tensor with shape (C, T), where C is number of channels and T is number of audio samples i )min_silence_lensilence_thresh keep_silence seek_stepr)durationr)tensor_to_audiosegmentrrsilentremove_silence_edgesaudiosegment_to_tensor)rjrZrkrlwavenon_silent_segssegr!r!r"remove_silences    r{rmrqsilence_thresholdcCs`t||d}td||}||d}|}t||d}td||}||d}|}|S)a> Remove edge silences longer than `keep_silence` ms. Parameters: audio: an AudioSegment object. keep_silence: kept silence in the edge. only_edge: If true, only remove edge silences. silence_threshold: the threshold of silence. Returns: An AudioSegment object )r|rN)rmaxreverse)rjrqr| start_idxr!r!r"rvHs    rvcCsVt|}|tjd}|jdkrt|d}|St| d|jj }|S)z> Convert a pydub.AudioSegment to PyTorch audio tensor @r+rr) nparrayget_array_of_samplesastypefloat32channelsrI from_numpy unsqueezereshapeT)aseg audio_data tensor_datar!r!r"rwfs rwcCs|}|jdkr|tjddf}|dddtj}|jddkr/| dd }| }t |d||jdd}|S) z Convert a PyTorch audio tensor to pydub.AudioSegment Parameters: tensor: Tensor with shape (C, T), where C is the number of channels and T is the time steps sample_rate: Audio sample rate r+Nriir)data sample_width frame_rater) cpunumpyndimrnewaxiscliprint16rN transposeflattentobytesr)rJrEaudio_np audio_bytes audio_segmentr!r!r"rtzs rt)r)rBrC)Fr)rrm)typingrrrrIr]pydubr pydub.silencerrrstrrKr#r'floatrATensorrUrXrcriboolr{rvrwrtr!r!r!r"sh  5+  B ; 0