o ߥi(@sddlZddlZddlZddlmZmZddlmZddlm Z m Z ddl Z GdddedZ Gdd d e Z Gd d d e ZGd d d e ZiZGdddeZdS)N)ABCMetaabstractmethod)Path) GeneratorUnionc @seZdZdZedefddZedefddZedede ee fdd fd d Z e ddede ee fd edd fddZ d S)StoragezAbstract class of storage. All backends need to implement two apis: ``read()`` and ``read_text()``. ``read()`` reads the file as a byte stream and ``read_text()`` reads the file as texts. filepathcCdSNselfrr r J/home/ubuntu/.local/lib/python3.10/site-packages/modelscope/fileio/file.pyreadz Storage.readcCr r r r r r r read_textrzStorage.read_textobjreturnNcCr r r r rrr r rwriterz Storage.writeutf-8encodingcCr r r r rrrr r r write_text!szStorage.write_textr) __name__ __module__ __qualname____doc__rstrrrbytesrrrrr r r rr s&  r) metaclassc @seZdZdZdeeefdefddZ ddeeefdedefdd Z d edeeefdd fd d Z dd edeeefdedd fddZ e j deeefdeeeefd d ffddZd S) LocalStoragezLocal hard disk storagerrcCs8t|d }|}Wd|S1swY|S)Read data from a given ``filepath`` with 'rb' mode. Args: filepath (str or Path): Path to read data. Returns: bytes: Expected bytes object. rbNopenr)r rfcontentr r rr,s  zLocalStorage.readrrcCs<t|d|d }|}Wd|S1swY|S)>Read data from a given ``filepath`` with 'r' mode. Args: filepath (str or Path): Path to read data. encoding (str): The encoding format used to open the ``filepath``. Default: 'utf-8'. Returns: str: Expected text reading from ``filepath``. rrNr%)r rrr' value_bufr r rr9s   zLocalStorage.read_textrNcCsdtj|}|rtj|stj|ddt|d}||WddS1s+wYdS))Write data to a given ``filepath`` with 'wb' mode. Note: ``write`` will create a directory if the directory of ``filepath`` does not exist. Args: obj (bytes): Data to be written. filepath (str or Path): Path to write data. Texist_okwbNospathdirnameexistsmakedirsr&r)r rrr4r'r r rrJs   "zLocalStorage.writecCshtj|}|rtj|stj|ddt|d|d}||WddS1s-wYdS)Write data to a given ``filepath`` with 'w' mode. Note: ``write_text`` will create a directory if the directory of ``filepath`` does not exist. Args: obj (str): Data to be written. filepath (str or Path): Path to write data. encoding (str): The encoding format used to open the ``filepath``. Default: 'utf-8'. Tr.wr+Nr1)r rrrr4r'r r rr\s  "zLocalStorage.write_textccs |VdSz$Only for unified API and do nothing.Nr r r r r as_local_pathss zLocalStorage.as_local_pathr)rrrrrrrr rrrr contextlibcontextmanagerrr:r r r rr")s<    r"c @seZdZdZddZddZejdede e ee fddffd d Z d e d e ee fddfd dZ dd ed e ee fdeddfddZdS) HTTPStoragezHTTP and HTTPS storage.cCt|}||jSr )requestsgetraise_for_statusr(r urlr*r r rrs zHTTPStorage.readcCr>r )r?r@rAtextrBr r rrs zHTTPStorage.read_textrrNc cPztjdd}|||||jVWt|jdSt|jw)ahDownload a file from ``filepath``. ``as_local_path`` is decorated by :meth:`contextlib.contextmanager`. It can be called with ``with`` statement, and when exists from the ``with`` statement, the temporary path will be released. Args: filepath (str): Download a file from ``filepath``. Examples: >>> storage = HTTPStorage() >>> # After existing from the ``with`` clause, >>> # the path will be removed >>> with storage.get_local_path('http://path/to/file') as path: ... # do something here FdeleteNtempfileNamedTemporaryFilerrclosenamer2remover rr'r r rr:  zHTTPStorage.as_local_pathrrCcCtd)Nz&write is not supported by HTTP StorageNotImplementedError)r rrCr r rrszHTTPStorage.writerrcCrP)Nz+write_text is not supported by HTTP StoragerQ)r rrCrr r rrzHTTPStorage.write_textr)rrrrrrr;r<rrrrr:r rrr r r rr=|s*  r=c @seZdZdZdddZddZddd Zejd e d e e e e fddffd d Z ded e e e fd dfddZ dde d e e e fde d dfddZdS) OSSStoragez OSS storage.NcCrP)Nz3OSSStorage.__init__ to be implemented in the futurerQ)r oss_config_filer r r__init__szOSSStorage.__init__cCrP)Nz/OSSStorage.read to be implemented in the futurerQr r r rrzOSSStorage.readrcCrP)Nz4OSSStorage.read_text to be implemented in the futurerQ)r rrr r rrrWzOSSStorage.read_textrrc crE)agDownload a file from ``filepath``. ``as_local_path`` is decorated by :meth:`contextlib.contextmanager`. It can be called with ``with`` statement, and when exists from the ``with`` statement, the temporary path will be released. Args: filepath (str): Download a file from ``filepath``. Examples: >>> storage = OSSStorage() >>> # After existing from the ``with`` clause, >>> # the path will be removed >>> with storage.get_local_path('http://path/to/file') as path: ... # do something here FrFNrHrNr r rr:rOzOSSStorage.as_local_pathrcCrP)Nz0OSSStorage.write to be implemented in the futurerQrr r rrrWzOSSStorage.writercCrP)Nz5OSSStorage.write_text to be implemented in the futurerQrr r rrrSzOSSStorage.write_textr r)rrrrrVrrr;r<rrrrr:r rrr r r rrTs,    rTc @seZdZUeeeedZeed<e ddZ e de de fddZ e ddee efd e de fd d Ze d e dee efddfddZe dd e de d e ddfddZejde deee efddffddZdS)File)osshttphttpslocal_prefix_to_storagecCst|tsJdt|d|vrd}n |d\}}|}|tjvs2Jd|dttj|tvr>tj|t|<t|S)Nz uri should be str type, but got z://r\zUnsupported uri z, valid prefixs: ) isinstancertypesplitrXr]listkeys G_STORAGES)uri storage_typeprefix_r r r _get_storages"   zFile._get_storagerdrcCst|}||S)r#)rXrhr)rdstorager r rrs z File.readrrcCst|}||S)r))rXrhr)rdrrir r rrs zFile.read_textrNcCst|}|||S)r-)rXrhr)rrdrir r rr#s z File.writecCst|}|||S)r7)rXrhr)rrdrrir r rr2s  zFile.write_textccsBt|}|| }|VWddS1swYdSr9)rXrhr:)rdri local_pathr r rr:Cs   "zFile.as_local_pathr)rrrrTr=r"r]dict__annotations__ staticmethodrhrr rrrrrrr;r<rr:r r r rrXs$   " *rX)r;r2rIabcrrpathlibrtypingrrr?rr"r=rTrcobjectrXr r r rs S47