o i(@sddlZddlZddlZddlmZmZddlmZddlm Z m Z ddl Z ddl m Z ddZGdd d ed ZGd d d eZGd ddeZGdddeZiZGdddeZdS)N)ABCMetaabstractmethod)Path) GeneratorUnion)urlparsecCst|}d}|jdurOt|jdkrOt}||}tj}tj |s*t |tj |tj |}t|d }||Wdn1sJwY|dusZJd||S)Nrwbzfailed to download: )rschemelen HTTPStoragereadtempfileTemporaryDirectorynameospathexistsmakedirsjoinbasenameopenwrite)urlresult file_pathstoragedatawork_dirfbrH/home/ubuntu/.local/lib/python3.10/site-packages/funasr/download/file.pydownload_from_urls      r!c @s~eZdZdZedefddZedefddZedede ee fdd fd d Z eddede ee fd edd fddZ d S)StoragezAbstract class of storage. All backends need to implement two apis: ``read()`` and ``read_text()``. ``read()`` reads the file as a byte stream and ``read_text()`` reads the file as texts. filepathcCdSNrselfr#rrr r 'z Storage.readcCr$r%rr&rrr read_text+r(zStorage.read_textobjreturnNcCr$r%rr'r*r#rrr r/r(z Storage.writeutf-8encodingcCr$r%rr'r*r#r.rrr write_text3r(zStorage.write_textr-) __name__ __module__ __qualname____doc__rstrr r)bytesrrrr0rrrr r"s *r") metaclassc @seZdZdZdeeefdefddZddeeefdedefdd Z d edeeefdd fd d Z dd edeeefdedd fddZ e j deeefdeeeefd d ffddZd S) LocalStoragezLocal hard disk storager#r+cCs8t|d }|}Wd|S1swY|S)Read data from a given ``filepath`` with 'rb' mode. Args: filepath (str or Path): Path to read data. Returns: bytes: Expected bytes object. rbNrr )r'r#fcontentrrr r ;s  zLocalStorage.readr-r.cCs<t|d|d }|}Wd|S1swY|S)>Read data from a given ``filepath`` with 'r' mode. Args: filepath (str or Path): Path to read data. encoding (str): The encoding format used to open the ``filepath``. Default: 'utf-8'. Returns: str: Expected text reading from ``filepath``. rr.Nr<)r'r#r.r= value_bufrrr r)Hs   zLocalStorage.read_textr*NcCsdtj|}|rtj|stj|ddt|d}||WddS1s+wYdS))Write data to a given ``filepath`` with 'wb' mode. Note: ``write`` will create a directory if the directory of ``filepath`` does not exist. Args: obj (bytes): Data to be written. filepath (str or Path): Path to write data. Texist_okrNrrdirnamerrrr)r'r*r#rGr=rrr rWs   "zLocalStorage.writecCshtj|}|rtj|stj|ddt|d|d}||WddS1s-wYdS)Write data to a given ``filepath`` with 'w' mode. Note: ``write_text`` will create a directory if the directory of ``filepath`` does not exist. Args: obj (str): Data to be written. filepath (str or Path): Path to write data. encoding (str): The encoding format used to open the ``filepath``. Default: 'utf-8'. TrDwrANrF)r'r*r#r.rGr=rrr r0is  "zLocalStorage.write_textccs |VdSz$Only for unified API and do nothing.Nrr&rrr as_local_path}s zLocalStorage.as_local_pathr1)r2r3r4r5rr6rr7r r)rr0 contextlibcontextmanagerrrKrrrr r98s $2r9c @seZdZdZddZddZejdede e ee fddffd d Z d e d e ee fddfd dZdd ed e ee fdeddfddZdS)r zHTTP and HTTPS storage.cCt|}||jSr%)requestsgetraise_for_statusr>r'rr@rrr r s zHTTPStorage.readcCrNr%)rOrPrQtextrRrrr r)s zHTTPStorage.read_textr#r+Nc cPztjdd}|||||jVWt|jdSt|jw)ahDownload a file from ``filepath``. ``as_local_path`` is decorated by :meth:`contextlib.contextmanager`. It can be called with ``with`` statement, and when exists from the ``with`` statement, the temporary path will be released. Args: filepath (str): Download a file from ``filepath``. Examples: >>> storage = HTTPStorage() >>> # After existing from the ``with`` clause, >>> # the path will be removed >>> with storage.get_local_path('http://path/to/file') as path: ... # do something here FdeleteNr NamedTemporaryFilerr closerrremover'r#r=rrr rK  zHTTPStorage.as_local_pathr*rcCtd)Nz&write is not supported by HTTP StorageNotImplementedError)r'r*rrrr rzHTTPStorage.writer-r.cCr])Nz+write_text is not supported by HTTP Storager^)r'r*rr.rrr r0r`zHTTPStorage.write_textr1)r2r3r4r5r r)rLrMr6rrrrKr7rr0rrrr r s&(r c @seZdZdZdddZddZddd Zejd e d e e e e fddffd d Z ded e e e fd dfddZdde d e e e fde d dfddZdS) OSSStoragez OSS storage.NcCr])Nz3OSSStorage.__init__ to be implemented in the futurer^)r'oss_config_filerrr __init__szOSSStorage.__init__cCr])Nz/OSSStorage.read to be implemented in the futurer^r&rrr r r`zOSSStorage.readr-cCr])Nz4OSSStorage.read_text to be implemented in the futurer^)r'r#r.rrr r)r`zOSSStorage.read_textr#r+c crT)agDownload a file from ``filepath``. ``as_local_path`` is decorated by :meth:`contextlib.contextmanager`. It can be called with ``with`` statement, and when exists from the ``with`` statement, the temporary path will be released. Args: filepath (str): Download a file from ``filepath``. Examples: >>> storage = OSSStorage() >>> # After existing from the ``with`` clause, >>> # the path will be removed >>> with storage.get_local_path('http://path/to/file') as path: ... # do something here FrUNrWr[rrr rKr\zOSSStorage.as_local_pathr*cCr])Nz0OSSStorage.write to be implemented in the futurer^r,rrr rr`zOSSStorage.writer.cCr])Nz5OSSStorage.write_text to be implemented in the futurer^r/rrr r0r`zOSSStorage.write_textr%r1)r2r3r4r5rcr r)rLrMr6rrrrKr7rr0rrrr ras  &(rac @seZdZUeeeedZeed<e ddZ e de de fddZ e ddee efd e de fd d Ze d e dee efddfddZe dd e de d e ddfddZejde deee efddffddZdS)File)osshttphttpslocal_prefix_to_storagecCst|tsJdt|d|vrd}n |d\}}|}|tjvs2Jd|dttj|tvr>tj|t|<t|S)Nz uri should be str type, but got z://rhzUnsupported uri z, valid prefixs: ) isinstancer6typesplitrdrilistkeys G_STORAGES)uri storage_typeprefix_rrr _get_storages zFile._get_storagerpr+cCst|}||S)r:)rdrtr )rprrrr r s z File.readr-r.cCst|}||S)r?)rdrtr))rpr.rrrr r) s zFile.read_textr*NcCst|}|||S)rC)rdrtr)r*rprrrr rs z File.writecCst|}|||S)rH)rdrtr0)r*rpr.rrrr r0*s  zFile.write_textccsBt|}|| }|VWddS1swYdSrJ)rdrtrK)rpr local_pathrrr rK;s   "zFile.as_local_pathr1)r2r3r4rar r9ridict__annotations__ staticmethodrtr6r7r rrr)rr0rLrMrrKrrrr rds$   " *rd)rLrr abcrrpathlibrtypingrrrO urllib.parserr!r"r9r raroobjectrdrrrr s  K/.