o ei6@sdZddlZddlZddlZddlZddlZddlZddlm Z ddl m Z ddl m Z mZddlZddlmZddlmZeeZGdd d e Ze d d d gZd e_d de_dde_Gddde ZdejdejdedejfddZdeeefdefddZddddddddejf deeefde eeejfde de d e d!e ed"e d#e ed$e eeejfde efd%d&Z!dS)'zDownloads or otherwise fetches pretrained models Authors: * Aku Rouhe 2021 * Samuele Cornell 2021 * Andreas Nautsch 2022, 2023 * Sylvain de Langen 2024 * Peter Plantinga 2024 N) namedtuple)Enum)OptionalUnion) HTTPError) get_loggerc@seZdZdZdZdZdZdS) FetchFromzDesignator where to fetch models/audios from. Note: HuggingFace repository sources and local folder sources may be confused if their source type is undefined. N)__name__ __module__ __qualname____doc__LOCAL HUGGING_FACEURIrrX/home/ubuntu/transcripts/venv/lib/python3.10/site-packages/speechbrain/utils/fetching.pyrs r FetchSourcepathz7NamedTuple describing a source path and how to fetch itcCs t|jS)N)hashr)selfrrr-s rcOs&dt|jt|jfj|i|S)N_)joinstrrrencode)rargskwargsrrrr.sc@s&eZdZdZdZ dZ dZ dZdS) LocalStrategyzoDescribes what strategy should be chosen for fetching and linking to local files when using :func:`~fetch`.r r r N)r r rrSYMLINKCOPYCOPY_SKIP_CACHENO_LINKrrrrr 4sr srcdstlocal_strategyreturncCs|tjkr|S|}|}||kr'|rtd|dtd||S|tjkrKt dkr7t dtd|||j dd| ||S|tjtjfvrltd |||j ddtt|t||Std |d ) a?If using `LocalStrategy.COPY` or `LocalStrategy.COPY_SKIP_CACHE`, destroy the file or symlink at `dst` if present and creates a copy from `src` to `dst`. If using `LocalStrategy.SYMLINK`, destroy the file or symlink at `dst` if present and creates a symlink from `src` to `dst`. If `LocalStrategy.NO_LINK` is passed, the src path is returned. Arguments --------- src : pathlib.Path Path to the source file to link to. Must be a valid path. dst : pathlib.Path Path of the final destination file. The file might not already exist, but the directory leading up to it must exist. local_strategy : LocalStrategy Strategy to adopt for linking. Returns ------- pathlib.Path Path to the final file on disk, after linking/copying (if any). zFetch: Found local symlink 'z' pointing to itself. This may require manual removal to recover. Did you maybe incorrectly call fetch() with `source==savedir`?zUFetch: Source and destination '%s' are identical, returning assuming this is intendedWindowszUsing SYMLINK strategy on Windows for fetching potentially requires elevated privileges and is not recommended. See `LocalStrategy` documentation.z6Fetch: Local file found, creating symlink '%s' -> '%s'T) missing_okz-Fetch: Local file found, copying '%s' -> '%s'zIllegal local strategy z passed for linking)r r%absolute is_symlink ValueErrorloggerdebugr"platformsystemwarningswarnunlink symlink_tor#r$infoshutilcopyr)r&r'r(rrrlink_with_strategydsB        r:sourcecCsHt|tr|St|rtj|fSd}||rtj|fStj |fS)ajFrom a given `FetchSource` or string source identifier, attempts to guess the matching :class:`~FetchFrom` (e.g. local or URI). If `source` is already a `FetchSource`, it is returned as-is. Arguments --------- source : str or FetchSource Where to look for the file. :func:`~fetch` interprets this path using the following logic: - First, if the source begins with "http://" or "https://", it is interpreted as a web address and the file is downloaded. - Second, if the source is a valid directory path, the file is either linked, copied or directly returned depending on the local strategy. - Otherwise, the source is interpreted as a HuggingFace model hub ID, and the file is downloaded from there (potentially with caching). Returns ------- tuple of (FetchFrom, str))zhttp:zhttps:) isinstancerpathlibPathis_dirrr startswithrr)r;uri_supported_schemesrrr guess_sources     rBFTsavedir overwrite allow_updates allow_network save_filenameuse_auth_tokenrevisionhuggingface_cache_dirc  Cs4|dur|}t|\} }|d|} |dur*t|}|jddd||} nd} tj} |p7| tjko7|}| durV| rV|sV| rHdnd}t d||t | | S| tjkrht| } t| | | S| tjkr| durytd|d |std|d t d |t | z tj| | W| Stjjy}z td |d |d}~ww| tjksJzD||||| d}t dt |t || tjkrtjdi||d|d}t|WStjdi|d| i}t|}Wnty}z dt |vrtd|d}~wwt|| | S)ad Fetches a local path, remote URL or remote HuggingFace file, downloading it locally if necessary and returns the local path. When a `savedir` is specified, but the file already exists locally elsewhere, the specified :class:`~LocalStrategy` chooses whether to copy or symlink it. If `/` exists locally, it is returned as is (unless using `overwrite` or `allow_updates`). The `HF_HOME` environment (default: `~/.cache/huggingface`) `selects the cache directory for HF `__. To prefer directly downloading to `savedir`, specify `local_strategy=LocalStrategy.COPY_SKIP_CACHE`. **HF cache is always looked up first if possible.** Arguments --------- filename : str Name of the file including extensions. source : str or FetchSource Local or remote root path for the filename. The final path is determined by `/`. See :func:`~guess_source` for how the path kind is deduced. savedir : str, optional If specified, directory under which the files will be available (possibly as a copy or symlink depending on `local_strategy`). Must be specified when downloading from an URL. overwrite : bool, defaults to `False` Allows the destination to be recreated by copy/symlink/fetch. This does **not** skip the HuggingFace cache (see `allow_updates`). allow_updates : bool, defaults to `True` If `True`, for a remote file on HF, check for updates and download newer revisions if available. If `False`, when the requested files are available locally, load them without fetching from HF. allow_network : bool, defaults to `True` If `True`, network accesses are allowed. If `False`, then remote URLs or HF won't be fetched, regardless of any other parameter. save_filename : str, optional, defaults to `None` The filename to use for saving this file. Defaults to the `filename` argument if not given or `None`. use_auth_token : bool, defaults to `False` If `True`, use HuggingFace's `auth_token` to enable loading private models from the Hub. revision : str, optional, defaults to `None` HuggingFace Hub model revision (Git branch name/tag/commit hash) to pin to a specific version. When changing the revision while local files might still exist, `allow_updates` must be `True`. huggingface_cache_dir: str, optional, defaults to `None` Path to HuggingFace cache; if `None`, `assumes the default cache location `__. Ignored if using `LocalStrategy.COPY_SKIP_CACHE`. Please prefer to let the user specify the cache directory themselves through the environment. local_strategy : LocalStrategy, optional Which strategy to use for local file storage -- see `LocalStrategy` for options. Ignored unless `savedir` is provided, default is `LocalStrategy.SYMLINK` which adds a link to the downloaded/cached file in the `savedir`. Returns ------- pathlib.Path Path to file on local file system. Raises ------ ValueError If file is not found N/T)parentsexist_oksymlinkfilez Fetch %s: Using %s found at '%s'zFetch z-: `savedir` must be specified for URI fetchesz>: File was not found locally and `allow_network` was disabled.zFetch %s: Downloading '%s'z Interpreted 'z)' as web address, but could not download.)repo_idfilenamerHrIlocal_files_onlyz:Fetch %s: Fetching from HuggingFace Hub '%s' if not cachedF) local_dirlocal_dir_use_symlinksforce_filename cache_dirz404 Client ErrorzFile not found on HF hubr)rBr=r>mkdirr,r r%rrexistsr-r/r7rrr:rr.urllibrequest urlretrieveerrorURLErrorr$huggingface_hubhf_hub_downloadr)rQr;rCrDrErFrGrHrIrJr( fetch_from source_path destinationshould_try_update file_kinder fetched_filerrrfetchsQ             rg)"rr=r1r8 urllib.errorrYurllib.requestr3 collectionsrenumrtypingrrr^requests.exceptionsrspeechbrain.utils.loggerrr r/rr__hash__rr r>r:rrBr"boolrgrrrrsz        0 M*