o -i!@sddlZddlZddlZddlmZddlmZddlmZddl m Z m Z m Z ddl mZddlmZddlmZeeZGd d d ZdS) N) init_logger)LoRALayerWeights) PEFTHelper) get_lora_idis_base_embeddding_weightsparse_fine_tuned_lora_name)TensorizerConfig) WeightsMapper)is_pin_memory_availablec@seZdZdZdededeeefddfddZdeddfd d Z d ededBfd d Z dede fddZ e    d dedeeejfdededejdBdedBdedBddfddZe ddddddddedeedededBdedejdBdedBdedBdedBddfddZdS)! LoRAModelzA LoRA fine-tuned model. lora_model_idranklorasreturnNcCs.||_|dksJd|j||_||_dS)z Args: lora_model_id: The integer id for the lora model. rank: lora rank. loras: module name -> weights for lora-replaced layers. rz.a valid lora id should be greater than 0, got N)idr r)selfr r rrQ/home/ubuntu/veenaModal/venv/lib/python3.10/site-packages/vllm/lora/lora_model.py__init__s    zLoRAModel.__init__cCs|j||j|jdS)z[Return a copy of the object with different ids. Will share the underlying tensors.)r r) __class__r rcopy)rr rrrclone0s zLoRAModel.clone module_namecCs|j|dS)z#Get LoRA for a given module by nameN)rget)rrrrrget_lora:szLoRAModel.get_lora lora_namecCs ||jvS)N)r)rrrrrcheck_lora_name>s zLoRAModel.check_lora_namecudatensors peft_helperdevicedtypemodel_vocab_sizeweights_mappercCst|dkot}i} |D]h\} } t| rqt| |\} } | | vr+t| || | <| rad| vrJ|durJ|| jdkrJtd| jdd|d| j ||d| | _ |r`| | j | | _ q| j ||d| | _ |rw| | j | | _ q|||j | S) z0Create a LoRAModel from a dictionary of tensors.cpulora_embedding_ANzThe embedding LoRA size(z;) must be consistent with the base model's vocabulary size(z).)r r!)strr itemsrrr from_configshape RuntimeErrortolora_a pin_memorylora_br)clsr rrr r!r"r#r.r tensor_nametensorr is_lora_arrrfrom_lora_tensorsAs< zLoRAModel.from_lora_tensors)r r r!r"r#tensorizer_config_dictlora_direxpected_lora_modulesr6c stjd} tjd} tjd} i} gdtffdd }| rQddlm}tdi| }tj|jd } |}|| fd |j i|j } || n\tj | rgt j | d d }|||D] }||| |<qiWd n1s}wYn*tj | stj | rtj | r| n| }tj||dd} || ntd|j|d urtn|| ||||dS)a8Create a LoRAModel from a local checkpoint. Args: lora_dir: The local path that has lora data. expected_lora_modules: Name of modules that are expected to be replaced by lora. peft_helper: Loaded lora configuration information. lora_model_id: LoRA model id. If not given, automatically set by a global counter. device: Device where the lora model is loaded. dtype: dtype of the lora model weights. Returns: Loaded LoRA Model. zadapter_model.safetensorszadapter_model.binzadapter_model.ptmodulescs|D]=}t|r qd|vrqt|\}}d|vr2|d}||dd}|vr1|q|dddvrA|qrRtdddd dS) N base_layerz.expertsr&.zWhile loading z, expected target modules in z but received z6. Please verify that the loaded LoRA module is correct)keysrrfindappendrsplit ValueError)r9 lora_moduler_ expert_idx expert_suffixr8r7unexpected_modulesr#rrcheck_unexpected_moduless0    zALoRAModel.from_local_checkpoint..check_unexpected_modulesr)TensorDeserializerzadapter_model.tensorsr!pt) frameworkNT) map_location weights_onlyz doesn't contain tensors)r rrr r!r"r#r)ospathjoindict tensorizerrIrtensorizer_dir_construct_tensorizer_argsr!deserialization_kwargsisfile safetensors safe_openr= get_tensortorchloadrAr5r)r1r7r8rr r r!r"r#r6lora_tensor_pathlora_bin_file_pathlora_pt_file_pathrrHrItensorizer_configtensorizer_argsfmodulelora_file_pathrrFrfrom_local_checkpointos^      zLoRAModel.from_local_checkpoint)rNNN)__name__ __module__ __qualname____doc__intrQr'rrrrboolr classmethodrZTensorrr!r r5setrdrrrrr s     -    r )rNrWrZ vllm.loggerrvllm.lora.lora_weightsrvllm.lora.peft_helperrvllm.lora.utilsrrr+vllm.model_executor.model_loader.tensorizerr vllm.model_executor.models.utilsr vllm.utils.platform_utilsr reloggerr rrrrs