o ꁱi@sdZddlmZddlmZddlmZddlmZddl Z ddl m Z ddl m Z e eZeGdd d ZGd d d e jZd%ddZd&ddZd'ddZd'ddZd(d)d#d$ZdS)*zEUtilities for loading fine-tuned LoRA adapters and connector weights.) annotations) dataclass)Path)OptionalN)loggingc@sZeZdZUdZdZded<dZded<dZded<dZded<dZ ded<d Z d ed <d S) _LoadReportz3Simple structure capturing what assets were loaded.Fboollanguage_modeldiffusion_head_loradiffusion_head_fullacoustic_connectorsemantic_connectorNzOptional[Path] adapter_root) __name__ __module__ __qualname____doc__r __annotations__r r r r rrrB/home/ubuntu/vibevoice-community/vibevoice/modular/lora_loading.pyrs      rcs*eZdZdZdfdd ZddZZS) _DiffusionHeadForwardShimzBWraps the diffusion head to expose the signature expected by PEFT.base nn.Modulecst||_dSN)super__init__r)selfr __class__rrr s  z"_DiffusionHeadForwardShim.__init__cOsLt|dkr|dd\}}}n|d}|d}|d}||||S)N noisy_images timesteps condition)lengetr)rargskwargsr r!r"rrrforward$s    z!_DiffusionHeadForwardShim.forward)rr)rrrrrr' __classcell__rrrrrsrcheckpoint_pathrreturncCs&|r|j}|dr|dS|S)z8Return the directory that actually holds adapter assets.lora)is_fileparentexists)r)rrr_resolve_adapter_root.s  r/moduleOptional[nn.Module]pathdevice torch.devicercCsj|dus|s dStj||d}|j|dd\}}|r$td||r.td|||dS)NF map_locationstrictzConnector load missing keys: z Connector load unexpected keys: T)r.torchloadload_state_dictloggerwarningto)r0r2r3 state_dictmissing unexpectedrrr_load_connector9s rBrreportNonec CsH|d}|d}|d}|d}zddlm}Wnty)} ztd| d} ~ ww|rX|s6|rXtd|t|jj } | | |} | || |j_ d |_ dS|d } | sd|d } | rtd | t j| |d } |jj j| d d\}}|rtd||rtd||jj |d |_dSdS)Ndiffusion_headadapter_config.jsonadapter_model.binadapter_model.safetensorsr PeftModelzEpeft is required to load diffusion head adapters but is not installedz!Loading diffusion head LoRA from Tzdiffusion_head_full.binz)Loading full diffusion head weights from r5Fr7z"Diffusion head load missing keys: z%Diffusion head load unexpected keys: )peftrJ ImportError RuntimeErrorr.r<informodelprediction_headfrom_pretrainedr>r r9r:r;r=r )rOrr3rCdiff_diradapter_config adapter_modeladapter_model_safetensorsrJexcshim peft_head full_pathr?r@rArrr_load_diffusion_headGsH    rZc Cs|d}|d}|d}|r|s|sdSzddlm}Wnty3}ztd|d}~wwtd|||jj |} | || |j_ t |drsz| Wnt yr}ztd |WYd}~nd}~wwd |_ dS) NrFrGrHrrIzEpeft is required to load language model adapters but is not installedz!Loading language model LoRA from tie_weightsz5Failed to retie weights after loading language LoRA: T)r.rKrJrLrMr<rNrQrOr r>hasattrr[ Exceptionr=) rOrr3rC config_filebin_filesafe_tensors_filerJrVpeft_lmrrr_load_language_modelps6    rbcheckpoint_dirstrOptional[torch.device]cCstt|}|std||pt|j}t|d}t||||t |||||dd}t t |j dd||rAd|_ |dd}t t |j dd||rUd|_t|jsatd|S) aLoad LoRA adapters and connector weights onto an instantiated model. Args: model: The already instantiated `VibeVoiceForConditionalGenerationInference` model. checkpoint_dir: Directory produced during fine-tuning containing a `lora/` folder. device: Optional device to place loaded modules on. When ``None`` the function infers the device from the model parameters. Returns: `_LoadReport` summarizing which assets were successfully loaded. zAdapter directory not found: )rr zpytorch_model.binNTr zdNo adapter assets were loaded. Ensure the checkpoint directory is correct and contains LoRA weights.)r/rr.FileNotFoundErrornext parametersr3rrbrZrBgetattrrOr r any__dict__valuesr<r=)rOrcr3rinferred_devicerCac_pathse_pathrrrload_lora_assetss$    rp)r)rr*r)r0r1r2rr3r4r*r)rrr3r4rCrr*rDr)rcrdr3rer*r)r __future__r dataclassesrpathlibrtypingrr9torch.nnnntransformers.utilsr get_loggerrr<rModulerr/rBrZrbrprrrrs"          )