o oi @s>dZddlZddlZddlmZddlmZmZmZm Z ddl Z ddl Z ddl Z ddl mZddlmZddlmZddlmZmZeeZ  d d e eefd ed ed efddZdeded efddZdeeefde eefd dfddZded efddZd!dede eefded efddZded efddZ dS)"z)Utilities related to data saving/loading.N)Path)IOAnyDictUnion) url_to_fs)AbstractFileSystem)module_available)_MAP_LOCATION_TYPE_PATHF path_or_url map_location weights_onlyreturncCst|ttfstj|||dSt|dr!tjjt|||dSt|}| |d}tj|||dWdS1s>wYdS)zLoads a checkpoint. Args: path_or_url: Path or URL of the checkpoint. map_location: a function, ``torch.device``, string or a dict specifying how to remap storage locations. )r rhttprbN) isinstancestrrtorchload startswithhubload_state_dict_from_urlget_filesystemopen)r r rfsfrW/home/ubuntu/.local/lib/python3.10/site-packages/lightning/fabric/utilities/cloud_io.py_load!s( $rpathkwargscKstt|fi|\}}|S)N)rr)r r!r_rrrrCsr checkpointfilepathc Cst}td|t||tjt |\}}|j .| |d}| | Wdn1s8wYWddSWddS1sPwYdS)aSaves a checkpoint atomically, avoiding the creation of incomplete checkpoints. Args: checkpoint: The object to save. Built to be used with the ``dump_checkpoint`` method, but can deal with anything which ``torch.save`` accepts. filepath: The path to which the checkpoint will be saved. This points to the file that the checkpoint will be stored in. zSaving checkpoint: wbN)ioBytesIOlogdebugrsavefsspeccorerr transactionrwritegetvalue)r#r$ bytesbufferrurlpathrrrr _atomic_saveHs  Pr2rcCsjtdrddlm}t||rdStdr"ddlm}t||r"dStdr3ddlm}t||r3dSd S) Nadlfsr)AzureBlobFileSystemTgcsfs) GCSFileSystems3fs) S3FileSystemF)r r3r4rr5r6r7r8)rr4r6r8rrr_is_object_storage]s      r9strictcCs,t|r|r ||S|| S||S)aCheck if a path is directory-like. This function determines if a given path is considered directory-like, taking into account the behavior specific to object storage platforms. For other filesystems, it behaves similarly to the standard `fs.isdir` method. Args: fs: The filesystem to check the path against. path: The path or URL to be checked. strict: A flag specific to Object Storage platforms. If set to ``False``, any non-existing path is considered as a valid directory-like path. In such cases, the directory (and any non-existing parent directories) will be created on the fly. Defaults to False. )r9isdirisfile)rr r:rrr_is_dirss    r=cCstjt|dkS)Nfile)r+utils get_protocolr)r rrr_is_local_file_protocolsrA)NF)F)!__doc__r&loggingpathlibrtypingrrrrr+ fsspec.utilsr fsspec.corerfsspec.implementations.localr lightning_utilities.core.importsr lightning.fabric.utilities.typesr r getLogger__name__r(boolrrrr2r9r=rArrrrs:        "&$