o
ߥi+ @ st d dl Z d dlZd dlZd dlZd dlmZmZ d dlmZ d dl m
Z
e
Z G dd de Z
G dd de
ZdS ) N)movermtree)Dict)
get_loggerc @ st e Zd ZdZdZdZdZ defddZdd Z d
d Z
d d
Z dd Z dd Z
dd Zdd Zdd Zdd ZdS )FileSystemCachez.mscz.mdlidz.mvcache_root_locationc K s t j|dd || _| dS )zBase file system cache interface.
Args:
cache_root_location (str): The root location to store files.
kwargs(dict): The keyword arguments.
Texist_okN)osmakedirsr
load_cache)selfr kwargs r P/home/ubuntu/.local/lib/python3.10/site-packages/modelscope/hub/utils/caching.py__init__ s zFileSystemCache.__init__c C s | j S N)r r r r r get_root_location( s z!FileSystemCache.get_root_locationc C sd g | _ tj| jtj}tj|r0t|d}t
|| _ W d d S 1 s)w Y d S d S )Nrb) cached_filesr pathjoinr r
KEY_FILE_NAMEexistsopenpickleload)r cache_keys_file_pathfr r r r
+ s
"zFileSystemCache.load_cachec C sd t j| jtj}t \}}t|d}t
| j | W d n1 s&w Y t || dS )zSave cache metadata.wbN)
r r r r r r tempfilemkstempr r dumpr r )r r fdfnr r r r save_cached_files3 s
z!FileSystemCache.save_cached_filesc C dS )zCheck the key is in the cache, if exist, return the file, otherwise return None.
Args:
key(str): The cache key.
Raises:
None
Nr r keyr r r get_file> s zFileSystemCache.get_filec C r( )zPut file to the cache.
Args:
key (str): The cache key
location (str): Location of the file, we will move the file to cache.
Raises:
None
Nr )r r* locationr r r put_fileI s
zFileSystemCache.put_filec C s& || j v r| j | | dS dS )zvRemove cache key in index, The file is removed manually
Args:
key (dict): The cache key.
N)r remover' r) r r r
remove_keyU s
zFileSystemCache.remove_keyc C s | j D ] }||kr dS qdS )NTF)r )r r*
cache_filer r r r _ s
zFileSystemCache.existsc C s t | j | dS )zRemove all files and metadata from the cache
In the case of multiple cache locations, this clears only the last one,
which is assumed to be the read/write one.
N)r r r
r r r r clear_cachef s
zFileSystemCache.clear_cachec C s t | S r )hashlibsha256encode hexdigestr) r r r hash_namen s zFileSystemCache.hash_nameN)__name__
__module__ __qualname__r MODEL_META_FILE_NAMEMODEL_META_MODEL_IDMODEL_VERSION_FILE_NAMEstrr r r
r' r+ r- r/ r r1 r6 r r r r r s$
r c s e Zd ZdZd fdd Zdd Zdd Zd efd
d Zd d
Z dd Z
dd Z dd Z dd Z
dd Zdd Zdd Zd d Z ZS ) ModelFileSystemCachezLocal cache file layout
cache_root/owner/model_name/individual cached files and cache index file '.mcs'
Save only one version for each file.
Nc sp |du s|du rt tj| | nt tj||| tjd||f i| _| |
| _ dS )aY Put file to the cache
Args:
cache_root(`str`): The modelscope local cache root(default: ~/.cache/modelscope/)
owner(`str`): The model owner.
name('str'): The name of the model
Returns:
Raises:
None
model_id = {owner}/{name}
Nz%s/%s) superr r r r load_model_metar r;
model_metasave_model_metaload_model_versioncached_model_revision)r
cache_rootownername __class__r r r x s
z ModelFileSystemCache.__init__c C sj t j| jtj}t j|r-t|d}t || _
W d d S 1 s&w Y d S tj di| _
d S )Nr unknown) r r r r r r: r r r r rA r; r meta_file_pathr r r r r@ s
"z$ModelFileSystemCache.load_model_metac C s\ t j| jtj}t j|r,t|d}| W d S 1 s%w Y d S d S )Nr)
r r r r r r< r r readstrip)r model_version_file_pathr r r r rC s
$z'ModelFileSystemCache.load_model_version
revision_infoc C s t j| jtj}t|d+}t|tr$d|d |d f }| | n
| | W d d S W d d S 1 s r s r> )r2 r r r" shutilr r typingr modelscope.utils.loggerr loggerobjectr r> r r r r s a