o }oi[@sddlmZddlmZddlmZmZmZmZm Z m Z m Z m Z ddl ZddlmZddlmZddlmZddlmZmZddlmZddlmZmZmZerVdd l m!Z!e d ej"d Z#Gd d d ej$eZ$e Gdddee#Z%dS)deepcopy)Path) TYPE_CHECKINGOptionalProtocolSequenceTypeTypeVarUnionruntime_checkableN)nn)Selfoverride)ckpt_to_context_subdir)IOMixin serializationtrack_io)OptimizerConfigModelT)boundcseZdZdejefddZ ddeee fde e ddfdd Z deee fd e e ddfd d Zeddejdedeffdd ZddejdedejfddZZS)FabricreturncKsPdd|D}|D]}tt|stt|q tjt|fi|S)NcSsi|] \}}|t|qSr).0kvrrP/home/ubuntu/.local/lib/python3.10/site-packages/nemo/lightning/fabric/fabric.py %sz"Fabric.io_init..)itemsvaluesrfind_node_traversertyperfdlConfig)selfkwargs cfg_kwargsvalrrrio_init#s   zFabric.io_initNpathmodelzDistributedModel[ModelT]cCsT|ddlm}t|}|dur|t|}|j}||}||d|i|S)aZLoad and set up a model for distributed training. This method loads a model from the given path, sets it up for distributed training using the current Fabric instance, and returns a DistributedModel. Args: path (Union[str, Path]): The path to the saved model checkpoint. model (Optional[ModelT], optional): An optional pre-instantiated model. If not provided, the model will be loaded from the checkpoint. Defaults to None. Returns: DistributedModel[ModelT]: The loaded and distributed model. Example: >>> from nemo import lightning as nl >>> >>> trainer = nl.Trainer( ... devices=2, ... strategy=nl.MegatronStrategy(tensor_model_parallel_size=2), ... plugins=nl.MegatronMixedPrecision(precision='16-mixed') ... ) >>> fabric = trainer.to_fabric() >>> distributed_model = fabric.load_model("path/to/checkpoint/dir") >>> >>> # You can now interact with the parallel model r) load_contextN state_dict)launchnemo.lightning.ior,rrr+ setup_moduleload)r%r*r+r,context dist_modelrrr load_model-s   zFabric.load_model model_typecCs6ddlm}t||std||}||j|S)a8 Import a model from a given path and set it up for distributed training. This method imports a model of the specified type from the given path, loads it, and sets it up for distributed training using the current Fabric instance. Args: path (Union[str, Path]): The path to the model. Can be a local path or a Hugging Face model identifier. model_type (Type[ModelT]): The type of the model to import. Must be a subclass of ConnectorMixin. Returns: DistributedModel[ModelT]: The imported and distributed model. Raises: TypeError: If the provided model_type is not a subclass of ConnectorMixin. Example: >>> from nemo import lightning as nl >>> from nemo.collections.llm import MistralModel >>> >>> trainer = nl.Trainer( ... devices=2, ... strategy=nl.MegatronStrategy(tensor_model_parallel_size=2), ... plugins=nl.MegatronMixedPrecision(precision='16-mixed') ... ) >>> fabric = trainer.to_fabric() >>> model = fabric.import_model("hf://mistralai/Mistral-7B-v0.1", MistralModel) >>> >>> # You can now interact with the parallel model r)ConnectorMixinz=The provided model class must be a subclass of ConnectorMixin)r/r6 issubclass TypeError import_fromr4 ckpt_path)r%r*r5r6r+rrr import_modelZs %  zFabric.import_modelTmodulemove_to_device_reapply_compilecs4ddlm}tj|||d}t|j|r|jS|S)Nr)FabricMegatronStrategy)r=r>) nemo.lightning.fabric.strategiesr?superr0 isinstancestrategy_forward_module)r%r<r=r>r?out __class__rrr0s  zFabric.setup_module datamodulestagecCs&||t|jdr|j|}|S)Nprocess_datamodule)setuphasattrrCrK)r%rIrJrrrsetup_datamodules   zFabric.setup_datamodule)N)TT)rH)__name__ __module__ __qualname__r#r$rr)r strrrrr4r r;rr Moduleboolr0plLightningDataModulerN __classcell__rrrFrr"s(   -  . $ rc@seZdZUeed<dS)DistributedModelr<N)rOrPrQr__annotations__rrrrrXs rX)&copyrpathlibrtypingrrrrr r r r fiddler#lightning.fabricfabriclblightning.pytorchpytorchrUtorchr typing_extensionsrrnemo.lightning.ckpt_utilsrnemo.lightning.io.mixinrrrmegatron.core.optimizerrrSrrrXrrrrs  (     |