o }oi @sddlmZddlmZddlmZddlZddlm m Z ddl m ZddlmZddlmZmZddlmZddlmZmZ  dd ejd ed eedededef ddZdedeeeejfeffddZddZ  dddZ!dS))Path)OptionalN)dist_checkpointing)FileSystemReader)ConnectorMixinModelConnector)ckpt_to_weights_subdir)logging model_utilsFTmodelsource output_path overwriteon_import_ckptreturncCs:t|ts td||}|||d}|r|||S)a Overriding nemo/lightning/io/api.py:import_ckpt to add on_import_ckpt flag, which is used to Imports a checkpoint into a model using the model's associated importer, typically for the purpose of fine-tuning a community model trained in an external framework, such as Hugging Face. This function leverages the ConnectorMixin interface to integrate external checkpoint data seamlessly into the specified model instance. The importer component of the model reads the checkpoint data from the specified source and transforms it into the right format. This is particularly useful for adapting models that have been pre-trained in different environments or frameworks to be fine-tuned or further developed within the current system. The function allows for specifying an output path for the imported checkpoint; if not provided, the importer's default path will be used. The 'overwrite' parameter enables the replacement of existing data at the output path, which is useful when updating models with new data and discarding old checkpoint files. For instance, using `import_ckpt(Mistral7BModel(), "hf")` initiates the import process by searching for a registered model importer tagged with "hf". In NeMo, `HFMistral7BImporter` is registered under this tag via: `@io.model_importer(Mistral7BModel, "hf", default_path="mistralai/Mistral-7B-v0.1")`. This links `Mistral7BModel` to `HFMistral7BImporter`, designed for HuggingFace checkpoints. The importer then processes and integrates these checkpoints into `Mistral7BModel` for further fine-tuning. Args: model (pl.LightningModule): The model into which the checkpoint will be imported. This model must implement the ConnectorMixin, which includes the necessary importer method for checkpoint integration. source (str): The source from which the checkpoint will be imported. This can be a file path, URL, or any other string identifier that the model's importer can recognize. output_path (Optional[Path]): The path where the imported checkpoint will be stored. If not specified, the importer's default path is used. overwrite (bool): If set to True, existing files at the output path will be overwritten. This is useful for model updates where retaining old checkpoint files is not required. on_import_ckpt (bool): If set to True, the importer will also call importer.on_import_ckpt(model), which imports the tokenizer associated with the model. This is set to False for multi-modal LLMs like SpeechLM, where the tokenizer is associated with the SpeechLM model instead of the internal LLM. Returns ------- Path: The path where the checkpoint has been saved after import. This path is determined by the importer, based on the provided output_path and its internal logic. Raises ------ ValueError: If the model does not implement ConnectorMixin, indicating a lack of necessary importer functionality. Example: model = Mistral7BModel() imported_path = import_ckpt(model, "hf://mistralai/Mistral-7B-v0.1") z+Model must be an instance of ConnectorMixin)rr ) isinstancer ValueErrorimporterr)r r r rrr ckpt_pathrV/home/ubuntu/.local/lib/python3.10/site-packages/nemo/collections/speechlm/utils/io.py import_ckpts <   rckpt_dircCsXt|ts t|}t|dd}t|}|}dd|jD}tj||d||fS)zW Load a distributed checkpoint from a directory, return as pytorch state dict. F) is_savingcSs4i|]\}}t|jdkr|tj|j|jjdqS)TensorStorageMetadata)dtype)type__name__torchemptysize propertiesr).0ktprrr ns z)load_distributed_ckpt..)storage_reader) rrrr read_metadatastate_dict_metadataitemsdcpload)r fs_readermetadata state_dictrrrload_distributed_ckptcs  r/cCsH|}|dD]}t||std|jd|d|t||}q|S)z, Get nested attribute of an object. .zObject z does not have attribute z , failed at )splithasattrAttributeError __class__getattr)objattr original_objkeyrrrget_nested_attr{s   r: model_configOnemo.collections.speechlm.models.speech_to_text_llm_model.SpeechToTextLLMConfigcCsZ|j}t|r |Std|jd|t|j}t||j |j |dd}|S)z5 Prepare distribute checkpoint for base LLM. z%Preparing distributed checkpoint for z from F)r) language_model_from_pretrained mcore_dcpcheck_is_distributed_checkpointr infolanguage_model_classr import_class_by_pathrlanguage_model_configlanguage_model_hub)r;checkpoint_path llm_model_clsrrrr prepare_pretrained_llm_dist_ckpts   rG)NFT)r;r<)"pathlibrtypingrlightning.pytorchpytorchplrtorch.distributed.checkpoint distributed checkpointr* megatron.corerr>rnemo.lightning.io.mixinrrnemo.lightning.io.plr nemo.utilsr r LightningModulestrboolrtupledictTensorr/r:rGrrrrs>       $F