o TiX+@sxddlZddlmZddlmZer!ddlmZddlmZm Z de de fdd Z Gd d d Z d e d e fddZ dS)N)HfApi)is_mergekit_available)MergeConfiguration) MergeOptions run_merge folder_pathrepo_idcCs,t}|j|dd}|j||j|jddS)Nmodel) repo_type)rrr )r create_repo upload_folderrr )rrapireporF/home/ubuntu/.local/lib/python3.10/site-packages/trl/mergekit_utils.pyupload_model_to_hfs rc@sReZdZdZddefddZddd Zdd d Zdd d ZdddZ dddZ dS) MergeConfiga Configuration class for merging two models using `mergekit`. This class provides a structured way to configure and generate merge configurations for various merge methods, such as `linear`, `ties`, `dare_ties`, and `slerp`. Args: method (`str`, *optional*, defaults to `"linear"`): Merge method to use. Supported methods include: - `"linear"`: Linearly combines two models with specified weights. - `"ties"`: Combines two models using the TIES method with density parameters. - `"dare_ties"`: A variant of TIES for domain adaptation. - `"slerp"`: Combines models using spherical linear interpolation. Note: For more details about the merge methods and how they are implemented, see the [MergeKit GitHub repository](https://github.com/arcee-ai/mergekit?tab=readme-ov-file#merge-methods). Attributes: method (`str`): The merge method to use. policy_model_path (`str` or `None`): Path to the policy model. target_model_path (`str` or `None`): Path to the target model. policy_model_weight (`float`): Weight for the policy model (for `linear` and `ties` methods). target_model_weight (`float`): Weight for the target model (for `linear` and `ties` methods). policy_model_density (`list[float]`): Density parameters for the policy model (for `ties` and `dare_ties`). target_model_density (`list[float]`): Density parameters for the target model (for `ties` and `dare_ties`). normalize (`float` or `None`): Normalization factor for the TIES method. t_values (`float` or `None`): Interpolation factor for the SLERP method. dtype (`str`): Data type to use for merging, e.g., `"float16"`. linearmethodcCststd||_d|_d|_|dkrd|_d|_d|_dS|dkr:d|_gd|_d|_dg|_ d|_ d|_dS|dkrUd|_gd|_d|_dg|_ d|_ d|_dS|d krad|_ d|_dSt d |) NzRMergeConfig requires the `mergekit` extra. To install, run `pip install mergekit`.rg?float16ties?)rgffffff?g? dare_tiesslerpzUnsupported merge method: ) r ImportErrorrpolicy_model_pathtarget_model_pathpolicy_model_weighttarget_model_weightdtypepolicy_model_densitytarget_model_density normalizet_values ValueError)selfrrrr__init__Is6      zMergeConfig.__init__returnrcCs<|jd|jd|jid|jd|jidgd}t|}|S)zh Creates a merge configuration for a linear merge of two models with specified weights. rweightr parameters)r merge_methodmodels)rrrrrrmodel_validater%merge_config_dict merge_configrrrcreate_merge_config_linearis z&MergeConfig.create_merge_config_linearc Cdd|jddddd|j|jdd|jddddd|j|jddgd|ji|jddddd|jddddd }t |}|S) zu Creates a merge configuration for a TIES merge of two models, with specified weights and densities. rNpathrevisionr loraoverride_architecturedensityr(r)r" r+slicesr,r* base_modelrtokenizer_source tokenizer chat_template out_dtype rr!rrr rr"rrr-r.rrrcreate_merge_config_ties|8      #z$MergeConfig.create_merge_config_tiesc Cr2) zz Creates a merge configuration for a DARE TIES merge of two models, with specified weights and densities. rNr3r6r9r)r"r;rBr.rrrcreate_merge_config_dare_tiesrDz)MergeConfig.create_merge_config_dare_tiesc CsXdd|jdddddddgd|ji|jddddd|jddddd }t|}|S)z_ Creates a merge configuration for a SLERP merge of a model with a base model. rNr3r6r)tr;)rr#rrrr-r.rrrcreate_merge_config_slerps.    z%MergeConfig.create_merge_config_slerpcCsL|jdkr |S|jdkr|S|jdkr|S|jdkr$|SdS)Nrrrr)rr1rCrErG)r%rrrcreates    zMergeConfig.createN)r)r'r) __name__ __module__ __qualname____doc__strr&r1rCrErGrHrrrrr's!  , ,'rconfigout_pathc Cs4tstdt||tdtjddddddS)z Merge two models using mergekit Args: config (`MergeConfig`): The merge configuration. out_path (`str`): The output path for the merged model. zSmerge_models requires the `mergekit` extra. To install, run `pip install mergekit`.autoTF)devicecudacopy_tokenizer lazy_unpicklelow_cpu_memory)rOoptionsN)rrrrtorchrR is_available)rNrOrrr merge_modelss rY)rWhuggingface_hubrtrl.import_utilsrmergekit.configrmergekit.mergerrrMrrrYrrrrs    `