o Siu@sddlmZddlmZmZmZmZmZddlZddl Z ddl m Z ddlm Z ddl mZmZddlmZddlmZdddZGddde jZdS)) annotations)AnyDictTupleUnionOptionalN)hf_hub_download)nn)FeatureExtractorEncodecFeatures) FourierHead)BackboneargsUnion[Any, Tuple[Any, ...]]initDict[str, Any]returnrcCsV|di}t|ts|f}|ddd\}}t||gd}t||}||i|S)zInstantiates a class with the given args and init. Args: args: Positional arguments required for instantiation. init: Dict of the form {"class_path":...,"init_args":...}. Returns: The instantiated class object. init_args class_path.)fromlist)get isinstancetuplersplit __import__getattr)rrkwargs class_module class_namemodule args_classr#D/home/ubuntu/.local/lib/python3.10/site-packages/vocos/pretrained.pyinstantiate_classs  r%csveZdZdZd!fdd Zed"d dZed#d$ddZe d%ddZ e d&ddZ e d'dd Z Z S)(Vocosa# The Vocos class represents a Fourier-based neural vocoder for audio synthesis. This class is primarily designed for inference, with support for loading from pretrained model checkpoints. It consists of three main components: a feature extractor, a backbone, and a head. feature_extractorr backboner headr cs t||_||_||_dSN)super__init__r'r(r))selfr'r(r) __class__r#r$r,)s  zVocos.__init__ config_pathstrrcCsvt|d }t|}Wdn1swYtd|dd}td|dd}td|dd}||||d}|S) z} Class method to create a new Vocos model instance from hyperparameters stored in a yaml configuration file. rNr#r')rrr(r))r'r(r))openyaml safe_loadr%)clsr0fconfigr'r(r)modelr#r#r$ from_hparams1s  zVocos.from_hparamsNrepo_idrevision Optional[str]cCszt|d|d}t|d|d}||}tj|dd}t|jtr2dd|jj D}| || || |S)z Class method to create a new Vocos model instance from a pre-trained model stored in the Hugging Face model hub. z config.yaml)r;filenamer<zpytorch_model.bincpu) map_locationcSsi|] \}}d||qS)zfeature_extractor.encodec.r#).0keyvaluer#r#r$ Hsz)Vocos.from_pretrained..) rr:torchloadrr'r encodec state_dictitemsupdateload_state_dicteval)r6r;r<r0 model_pathr9rHencodec_parametersr#r#r$from_pretrained>s    zVocos.from_pretrained audio_input torch.TensorrrcKs(|j|fi|}|j|fi|}|S)a' Method to run a copy-synthesis from audio waveform. The feature extractor first processes the audio input, which is then passed through the backbone and the head to reconstruct the audio output. Args: audio_input (Tensor): The input tensor representing the audio waveform of shape (B, T), where B is the batch size and L is the waveform length. Returns: Tensor: The output tensor representing the reconstructed audio waveform of shape (B, T). )r'decode)r-rPrfeatures audio_outputr#r#r$forwardQsz Vocos.forwardfeatures_inputcKs |j|fi|}||}|S)a Method to decode audio waveform from already calculated features. The features input is passed through the backbone and the head to reconstruct the audio output. Args: features_input (Tensor): The input tensor of features of shape (B, C, L), where B is the batch size, C denotes the feature dimension, and L is the sequence length. Returns: Tensor: The output tensor representing the reconstructed audio waveform of shape (B, T). )r(r))r-rVrxrTr#r#r$rRcs z Vocos.decodecodescCst|jts Jd|dkr|d}|jjjj}tj d|t |||j d}|| ddd}tj j||jjjdd}|dd}|S)a  Transforms an input sequence of discrete tokens (codes) into feature embeddings using the feature extractor's codebook weights. Args: codes (Tensor): The input tensor. Expected shape is (K, L) or (K, B, L), where K is the number of codebooks, B is the batch size and L is the sequence length. Returns: Tensor: Features of shape (B, C, L), where B is the batch size, C denotes the feature dimension, and L is the sequence length. z:Feature extractor should be an instance of EncodecFeaturesrr)device)dim)rr'r r\ unsqueezerG quantizerbinsrEarangelenrZviewr functional embeddingcodebook_weightssum transpose)r-rXn_binsoffsetsembeddings_idxsrSr#r#r$codes_to_featurests    zVocos.codes_to_features)r'r r(r r)r )r0r1rr&r*)r;r1r<r=rr&)rPrQrrrrQ)rVrQrrrrQ)rXrQrrQ)__name__ __module__ __qualname____doc__r, classmethodr:rOrEinference_moderUrRrk __classcell__r#r#r.r$r&!s    r&)rrrrrr) __future__rtypingrrrrrrEr4huggingface_hubrr vocos.feature_extractorsr r vocos.headsr vocos.modelsr r%Moduler&r#r#r#r$s