o ui9#@sDddlZddlZddlZddlmZGdddZGdddZdS)N)tqdmcs4eZdZdZd fdd ZddZd dd ZZS) ConvTDFNetzG ConvTDFNet - Convolutional Temporal Frequency Domain Network. cstt|d|_||_d||_||_||_|jdd|_||jd|_ t j |jdd|_ ||_ |dkr=|jdn|j}t d||j|j|jg|_|d|_dS)a Initialize ConvTDFNet. Args: target_name (str): The target name for separation. L (int): Number of layers. dim_f (int): Dimension in the frequency domain. dim_t (int): Dimension in the time domain (log2). n_fft (int): FFT size. hop (int, optional): Hop size. Defaults to 1024. Returns: None T) window_lengthperiodic*N)superr__init__dim_cdim_fdim_tn_ffthopn_bins chunk_sizetorch hann_windowwindow target_namezerosfreq_padn)selfrLrrrrout_c __class__?/home/ubuntu/sommelier/podcast-pipeline/models/separate_fast.pyr s zConvTDFNet.__init__cCs|d|jg}tj||j|j|jddd}t|}|gd}|ddd|j |j gd|j |j |j g}|ddddd|j fS)z Perform Short-Time Fourier Transform (STFT). Args: x (torch.Tensor): Input waveform. Returns: torch.Tensor: STFT of the input waveform. T)r hop_lengthrcenterreturn_complex)rrrrN) reshaperrstftrrr view_as_realpermuterrr r)rxr r r!r(1s  zConvTDFNet.stftNcCs|dur|j|jddddgn|}t||gd}|jdkr"dnd}|d|d|j|jgdd|j|jg}| gd }| }t |}tj ||j |j|jd d }|d||jgS) a Perform Inverse Short-Time Fourier Transform (ISTFT). Args: x (torch.Tensor): Input STFT. freq_pad (torch.Tensor, optional): Frequency padding. Defaults to None. Returns: torch.Tensor: Inverse STFT of the input. Nrrr rr")rrr&rT)rr#rr$)rrepeatshapercatrr'rrr* contiguousview_as_complexistftrrrr)rr+rcr r r!r3Ks   zConvTDFNet.istft)r)N)__name__ __module__ __qualname____doc__r r(r3 __classcell__r r rr!r s rc@s0eZdZdZddZddZddZdd Zd S) PredictorzR Predictor class for source separation using ConvTDFNet and ONNX Runtime. cCsr||_tdd|d|d|dd|_|dkr$tj|dd gd |_dS|d kr5tj|dd gd |_dStd )a, Initialize the Predictor. Args: args (dict): Configuration arguments. device (str): Device to run the model ('cuda' or 'cpu'). Returns: None Raises: ValueError: If the provided device is not 'cuda' or 'cpu'. vocals rrr)rrrrrcuda model_pathCUDAExecutionProvider) providerscpuCPUExecutionProviderz%Device must be either 'cuda' or 'cpu'N)argsrmodel_ortInferenceSessionmodel ValueError)rrCdevicer r r!r ns"    zPredictor.__init__c Cs|jd}|jd}|jdd}|dksJd||kr|}i}|jddks,||kr.|}d}td||D]/}|d7}|dkrBdn|}t||||} ||} |dd| | f||<| |krenq6|j||d } | S) z Separate the sources from the input mix. Args: mix (np.ndarray): Input mixture signal. Returns: np.ndarray: Separated sources. Raises: AssertionError: If margin is zero. r"marginchunksDrzMargin cannot be zero!rN) margin_size)r/rCrangemincopy demix_base) rmixsamplesrJr segmented_mixcounterskips_marginendstartsourcesr r r!demixs* zPredictor.demixc Csg}tt|d}|d|D],}||}g}|jd}|j} | jd} | jd| } | || } tt d| f|t d| ft d| ffd} g}d}||| krut | dd||| jf}| ||| 7}||| ksVt j t |t jd}t |j}| |}|jdr|dd | id d |dd |idd }| t |}n| t |dd |id}|dddd| | fdddd ddd| f}|dkrdn|}|t|ddd dkr dn| }|dkrd}| |dd||f|dWdn 1s3wY| |qtj|d d }||S) z Base function for source separation. Args: mixes (dict): Dictionary of segmented mixtures. margin_size (int): Size of the margin. Returns: np.ndarray: Separated sources. )totalzSource separationrrrN)dtypedenoiseinputg?r")axis)rlenset_descriptionr/rDrrnp concatenaterarrayappendrtensorfloat32no_gradrGr(rCrunrAnumpyr3 transposer'listkeysupdateclose)rmixesrMchunked_sources progress_barrRcmixrZn_samplerGtrimgen_sizepadmix_p mix_wavesiwaves_ortspek spec_pred tar_waves tar_signalrYrX_sourcesr r r!rQsf      *       " *   zPredictor.demix_basec Cs|jdkr t||g}|jd|jdd}|jd|jdddkrAt|dd|jdd|jd|jddff}|j}||j}|dj}|dkri||d|jdd| ddf|fS|||fS)z Predict the separated sources from the input mix. Args: mix (np.ndarray): Input mixture signal. Returns: tuple: Tuple containing the mixture minus the separated sources and the separated sources. rrKrLr)rrN)ndimrcasfortranarrayr/rCrxTr[)rrRtailrZoptr r r!predicts(    , zPredictor.predictN)r5r6r7r8r r[rQrr r r r!r:is "( Gr:)rrkrc onnxruntimerErrr:r r r r!s  \