o cin@sddlZddlZddlZddlmZddlmZmZmZm Z ddl Z ddl m Z m Z ddlmZddlmZer>ddlmZdZed d Gd d d eZdS) N)Path) TYPE_CHECKINGAnyDictOptional)4consume_prefix_in_state_dict_if_present_not_in_placeload_torch_model)FrameworkCheckpoint) PublicAPI) Preprocessortorch_encoded_databeta) stabilityc@seZdZdZdZedddeeefde dddfd d Z eddd e j j de dddfd d Zdd e e j j de j j fddZdS)TorchCheckpointzCA :class:`~ray.train.Checkpoint` with Torch-specific functionality.zmodel.ptN) preprocessor state_dictrr returncCsJt}t||j}t|d}t||||}|r#| ||S)ahCreate a :class:`~ray.train.Checkpoint` that stores a model state dictionary. .. tip:: This is the recommended method for creating :class:`TorchCheckpoints`. Args: state_dict: The model state dictionary to store in the checkpoint. preprocessor: A fitted preprocessor to be applied before inference. Returns: A :class:`TorchCheckpoint` containing the specified state dictionary. Examples: .. testcode:: import torch import torch.nn as nn from ray.train.torch import TorchCheckpoint # Set manual seed torch.manual_seed(42) # Function to create a NN model def create_model() -> nn.Module: model = nn.Sequential(nn.Linear(1, 10), nn.ReLU(), nn.Linear(10,1)) return model # Create a TorchCheckpoint from our model's state_dict model = create_model() checkpoint = TorchCheckpoint.from_state_dict(model.state_dict()) # Now load the model from the TorchCheckpoint by providing the # model architecture model_from_chkpt = checkpoint.get_model(create_model()) # Assert they have the same state dict assert str(model.state_dict()) == str(model_from_chkpt.state_dict()) print("worked") .. testoutput:: :hide: ... zmodule.) tempfilemkdtemprMODEL_FILENAMEas_posixrtorchsavefrom_directoryset_preprocessor)clsrrtempdir model_pathstripped_state_dict checkpointr T/home/ubuntu/.local/lib/python3.10/site-packages/ray/train/torch/torch_checkpoint.pyfrom_state_dicts8   zTorchCheckpoint.from_state_dictmodelcCs@t}t||j}t||||}|r|||S)a Create a :class:`~ray.train.Checkpoint` that stores a Torch model. .. note:: PyTorch recommends storing state dictionaries. To create a :class:`TorchCheckpoint` from a state dictionary, call :meth:`~ray.train.torch.TorchCheckpoint.from_state_dict`. To learn more about state dictionaries, read `Saving and Loading Models `_. # noqa: E501 Args: model: The Torch model to store in the checkpoint. preprocessor: A fitted preprocessor to be applied before inference. Returns: A :class:`TorchCheckpoint` containing the specified model. Examples: .. testcode:: from ray.train.torch import TorchCheckpoint import torch # Create model identity and send a random tensor to it model = torch.nn.Identity() input = torch.randn(2, 2) output = model(input) # Create a checkpoint checkpoint = TorchCheckpoint.from_model(model) print(checkpoint) .. testoutput:: :hide: ... ) rrrrrrrrr)rr#rrrrr r r! from_modelas-   zTorchCheckpoint.from_modelcCs|!}t||j}tj|stdtj |dd}Wdn1s(wYt |tj j r;|r;t dt||d}|S)aRetrieve the model stored in this checkpoint. Args: model: If the checkpoint contains a model state dict, and not the model itself, then the state dict will be loaded to this ``model``. Otherwise, the model will be discarded. z`model.pt` not found within this checkpoint. Make sure you created this `TorchCheckpoint` from one of its public constructors (`from_state_dict` or `from_model`).cpu) map_locationNzTorchCheckpoint already contains all information needed. Discarding provided `model` argument. If you are using TorchPredictor directly, you should do `TorchPredictor.from_checkpoint(checkpoint)` by removing kwargs `model=`.) saved_modelmodel_definition) as_directoryrrrospathexists RuntimeErrorrload isinstancennModulewarningswarnr)selfr#rrmodel_or_state_dictr r r! get_models"   zTorchCheckpoint.get_model)N)__name__ __module__ __qualname____doc__r classmethodrstrrrr"rr0r1r$r6r r r r!rs,  D $6r)r*rr2pathlibrtypingrrrrrray.air._internal.torch_utilsrr(ray.train._internal.framework_checkpointr ray.util.annotationsr ray.data.preprocessorr ENCODED_DATA_KEYrr r r r!s