o 8wi @sBdZddlZddlZddlZddlmZddlmZmZm Z ddl Z ddl Z ddl Z ddl mZddlmZddlmZddlmZmZeeZ  d d e eefd ed ed efddZdeded efddZdeeefde eefd dfddZded efddZd!dede eefded efddZ ded efddZ!dS)"z)Utilities related to data saving/loading.N)Path)IOAnyUnion) url_to_fs)AbstractFileSystem)module_available)_MAP_LOCATION_TYPE_PATHF path_or_url map_location weights_onlyreturncCst|ttfstj|||dSt|dr!tjjt|||dSt|}| |d}tj|||dWdS1s>wYdS)zLoads a checkpoint. Args: path_or_url: Path or URL of the checkpoint. map_location: a function, ``torch.device``, string or a dict specifying how to remap storage locations. )r r httprbN) isinstancestrrtorchload startswithhubload_state_dict_from_urlget_filesystemopen)r r r fsfr`/home/ubuntu/sommelier/.venv/lib/python3.10/site-packages/lightning_fabric/utilities/cloud_io.py_load"s( $rpathkwargscKstt|fi|\}}|S)N)rr)rr r_rrrrDsr checkpointfilepathc Cst}td|t||zHtjt |\}}|j 0| |d}| | Wdn1s9wYWdWdSWdWdS1sSwYWdSty}z#t|jtrwt|jddtjkr~td|WYd}~dSWYd}~dSd}~ww)aSaves a checkpoint atomically, avoiding the creation of incomplete checkpoints. Args: checkpoint: The object to save. Built to be used with the ``dump_checkpoint`` method, but can deal with anything which ``torch.save`` accepts. filepath: The path to which the checkpoint will be saved. This points to the file that the checkpoint will be stored in. zSaving checkpoint: wbNerrnoz]Upgrade fsspec to enable cross-device local checkpoints: pip install "fsspec[http]>=2025.5.0")ioBytesIOlogdebugrsavefsspeccorerr transactionrwritegetvaluePermissionErrorr __context__OSErrorgetattrr%EXDEV RuntimeError)r"r# bytesbufferrurlpathrerrr _atomic_saveIs$  V r9rcCsjtdrddlm}t||rdStdr"ddlm}t||r"dStdr3ddlm}t||r3dSd S) Nadlfsr)AzureBlobFileSystemTgcsfs) GCSFileSystems3fs) S3FileSystemF)rr:r;rr<r=r>r?)rr;r=r?rrr_is_object_storageds      r@strictcCs,t|r|r ||S|| S||S)aCheck if a path is directory-like. This function determines if a given path is considered directory-like, taking into account the behavior specific to object storage platforms. For other filesystems, it behaves similarly to the standard `fs.isdir` method. Args: fs: The filesystem to check the path against. path: The path or URL to be checked. strict: A flag specific to Object Storage platforms. If set to ``False``, any non-existing path is considered as a valid directory-like path. In such cases, the directory (and any non-existing parent directories) will be created on the fly. Defaults to False. )r@isdirisfile)rrrArrr_is_dirzs    rDcCstjt|dkS)Nfile)r+utils get_protocolr)rrrr_is_local_file_protocolsrH)NF)F)"__doc__r%r&loggingpathlibrtypingrrrr+ fsspec.utilsr fsspec.corerfsspec.implementations.localr lightning_utilities.core.importsr lightning_fabric.utilities.typesr r getLogger__name__r(boolrrdictrr9r@rDrHrrrrs<        "&$