o piL@sdZddlZddlZddlZddlZddlZddlZddlZddlm Z ddl m Z m Z m Z ddlmZddlmZmZddlmZmZddlmZd d lmZd d lmZmZmZeeZd Z ddZ!ddZ"de e#ej$ffddZ%ddZ&ddZ'ddZ(ddZ)ddZ*e      d-d e e#ej$fd!e#d"e e e#ej$fd#e+d$e e e#e#fd%e e e+e#fd&e e#d'e+fd(d)Z,e       d.d e e#ej$fd!e#d*e e#d"e e e#ej$fd#e+d$e e e#e#fd%e e e+e#fd&e e#d'e+fd+d,Z-dS)/z3Utilities to dynamically load objects from the Hub.N)Path)DictOptionalUnion)request)hf_hub_download model_info)RevisionNotFoundErrorvalidate_hf_hub_args)version) __version__)DIFFUSERS_DYNAMIC_MODULE_NAMEHF_MODULES_CACHEloggingz$diffusers/community-pipelines-mirrorcCs0d}tt|d}t|dddS)Nz$https://pypi.org/pypi/diffusers/jsonreleasescSs t|S)N)r Version)xrc/home/ubuntu/SoloSpeech/.venv/lib/python3.10/site-packages/diffusers/utils/dynamic_modules_utils.py-s z(get_diffusers_versions..)key)jsonloadsrurlopenreadkeyssorted)urlrrrrget_diffusers_versions*sr cCsLttjvrdStjttjtddttd}|s$|dSdS)z_ Creates the cache directory for modules with an init, and adds it to the Python path. NTexist_ok __init__.py) rsyspathappendosmakedirsrexiststouch) init_pathrrrinit_hf_modules0s    r,namecCsTttt|}|jst|jtj|dd|d}|s(|dSdS)zF Creates a dynamic module in the cache directory for modules. Tr!r#N) r,rrparentr)create_dynamic_moduler'r(r*)r-dynamic_module_pathr+rrrr/?s    r/cCsjt|ddd }|}Wdn1swYtjd|tjd}|tjd|tjd7}tt|S)z Get the list of modules that are relatively imported in a module file. Args: module_file (`str` or `os.PathLike`): The module file to inspect. rutf-8encodingNz^\s*import\s+\.(\S+)\s*$flagsz^\s*from\s+\.(\S+)\s+import)openrrefindall MULTILINElistset) module_filefcontentrelative_importsrrrget_relative_importsNs   rAcsd}|g}g|sBg}|D] }|t|q t|jfdd|D}fdd|D}dd|D}t|dk}||r S)a) Get the list of all files that are needed for a given module. Note that this function recurses through the relative imports (if a imports b and b imports c, it will return module files for b and c). Args: module_file (`str` or `os.PathLike`): The module file to inspect. Fcsg|]}t|qSr)str).0m) module_pathrr sz-get_relative_import_files..csg|]}|vr|qSrrrCr>)all_relative_importsrrrFtrGcSsg|]}|dqS).pyrrHrrrrFusr)extendrArr.len)r= no_changefiles_to_check new_importsr>new_import_filesr)rIrErget_relative_import_files`s    rQc Cst|ddd }|}Wdn1swYtjd|tjd}|tjd|tjd7}dd |D}tt|}g}|D]}zt|Wq@t yW| |Yq@wt |d krot d d |d d |dt |S)zi Check if the current Python environment contains all the libraries that are imported in a file. r1r2r3Nz^\s*import\s+(\S+)\s*$r5z^\s*from\s+(\S+)\s+importcSs$g|]}|ds|ddqS).r) startswithsplit)rCimprrrrFs$z!check_imports..rz\This modeling file requires the following packages that were not found in your environment: , z. Run `pip install  `)r7rr8r9r:r;r< importlib import_module ImportErrorr&rLjoinrA)filenamer>r?importsmissing_packagesrUrrr check_imports}s.    r`cCs4|tjjd}t|}|durt|St||S)zY Import a module on the cache directory for modules and extract a class from it. rRN)replacer'r%seprYrZfind_pipeline_classgetattr) class_namerEmodulerrrget_class_in_modules   rgc Csddlm}tt|tj}d}|D]1\}}||jkrFt||rF|j dddkrF|durDt d|jd|jd |d |d |}q|S) z Retrieve pipeline class that inherits from `DiffusionPipeline`. Note that there has to be exactly one class inheriting from `DiffusionPipeline`. r )DiffusionPipelineNrRr diffusersz#Multiple classes that inherit from z have been found: z, and z). Please make sure to define only one in ) pipelinesrhdictinspect getmembersisclassitems__name__ issubclass __module__rT ValueError) loaded_modulerh cls_memberspipeline_classcls_nameclsrrrrcs*   rcFpretrained_model_name_or_pathr= cache_dirforce_downloadproxiestokenrevisionlocal_files_onlyc Cst|}tj||}tj|r|} d} n|ddkrt} ddtddd} |durH| dd| vr<| nd }t d |dn"|| vrRd|}n|d krY|}nt d |d d | d gdzt t d|d|d||||d} d} |d}WnWty} z td|d| d} ~ wtyt d|d|dwzt |||||||d} tjdd|d} Wntyt d|d|dwt| }ttjj| }t|tt|}| dks| dkrDt| |||D]>}t|ddkr/d|d}|dd}tj||s/t|||d}ttj||||qnvt|||dj}||}|tjj|}t|||st|ddkr|dd}tj||st||t| |||D]+}t|ddkrd|d}||st||d||||||dqtj||S)a Prepares Downloads a module from a local folder or a distant repo and returns its path inside the cached Transformers module. Args: pretrained_model_name_or_path (`str` or `os.PathLike`): This can be either: - a string, the *model id* of a pretrained model configuration hosted inside a model repo on huggingface.co. Valid model ids can be located at the root-level, like `bert-base-uncased`, or namespaced under a user or organization name, like `dbmdz/bert-base-german-cased`. - a path to a *directory* containing a configuration file saved using the [`~PreTrainedTokenizer.save_pretrained`] method, e.g., `./my_model_directory/`. module_file (`str`): The name of the module file containing the class to look for. cache_dir (`str` or `os.PathLike`, *optional*): Path to a directory in which a downloaded pretrained model configuration should be cached if the standard cache should not be used. force_download (`bool`, *optional*, defaults to `False`): Whether or not to force to (re-)download the configuration files and override the cached versions if they exist. proxies (`Dict[str, str]`, *optional*): A dictionary of proxy servers to use by protocol or endpoint, e.g., `{'http': 'foo.bar:3128', 'http://hostname': 'foo.bar:4012'}.` The proxies are used on each request. token (`str` or *bool*, *optional*): The token to use as HTTP bearer authorization for remote files. If `True`, will use the token generated when running `transformers-cli login` (stored in `~/.huggingface`). revision (`str`, *optional*, defaults to `"main"`): The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a git-based system for storing models and other artifacts on huggingface.co, so `revision` can be any identifier allowed by git. local_files_only (`bool`, *optional*, defaults to `False`): If `True`, will only try to load the tokenizer configuration from local files. You may pass a token in `token` if you are not logged in (`huggingface-cli login`) and want to use private or [gated models](https://huggingface.co/docs/hub/models-gated#gated-models). Returns: `str`: The path to the module inside the cache. local/rvrRNrmainzDefaulting to latest_version: z`custom_revision`: z3 does not exist. Please make sure to choose one of rVdatasetrJ)repo_id repo_typer]rzr{r|rgitz Revision 'a' not found in the community pipelines mirror. Check available revisions on https://huggingface.co/datasets/diffusers/community-pipelines-mirror/tree/main. If you don't find the revision you are looking for, please open an issue on https://github.com/huggingface/diffusers/issues.zCould not locate the z inside )rzr{r|rr}z--r )r~r}rzr{r|r}r~r)rBr'r%r\isfilecountr r rTloggerinforsrCOMMUNITY_PIPELINES_MIRROR_IDr EnvironmentErrorerrorr`rrbr/rrshutilcopyrLr)r(rshaget_cached_module_file)ryr=rzr{r|r}r~rmodule_file_or_urlresolved_module_file submoduleavailable_versionslatest_versionemodules_neededfull_submodulesubmodule_path module_needed module_folder commit_hashrrrrs9          rrec Ks*t||||||||d} t|| ddS)a Extracts a class from a module file, present in the local folder or repository of a model. Calling this function will execute the code in the module file found locally or downloaded from the Hub. It should therefore only be called on trusted repos. Args: pretrained_model_name_or_path (`str` or `os.PathLike`): This can be either: - a string, the *model id* of a pretrained model configuration hosted inside a model repo on huggingface.co. Valid model ids can be located at the root-level, like `bert-base-uncased`, or namespaced under a user or organization name, like `dbmdz/bert-base-german-cased`. - a path to a *directory* containing a configuration file saved using the [`~PreTrainedTokenizer.save_pretrained`] method, e.g., `./my_model_directory/`. module_file (`str`): The name of the module file containing the class to look for. class_name (`str`): The name of the class to import in the module. cache_dir (`str` or `os.PathLike`, *optional*): Path to a directory in which a downloaded pretrained model configuration should be cached if the standard cache should not be used. force_download (`bool`, *optional*, defaults to `False`): Whether or not to force to (re-)download the configuration files and override the cached versions if they exist. proxies (`Dict[str, str]`, *optional*): A dictionary of proxy servers to use by protocol or endpoint, e.g., `{'http': 'foo.bar:3128', 'http://hostname': 'foo.bar:4012'}.` The proxies are used on each request. token (`str` or `bool`, *optional*): The token to use as HTTP bearer authorization for remote files. If `True`, will use the token generated when running `transformers-cli login` (stored in `~/.huggingface`). revision (`str`, *optional*, defaults to `"main"`): The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a git-based system for storing models and other artifacts on huggingface.co, so `revision` can be any identifier allowed by git. local_files_only (`bool`, *optional*, defaults to `False`): If `True`, will only try to load the tokenizer configuration from local files. You may pass a token in `token` if you are not logged in (`huggingface-cli login`) and want to use private or [gated models](https://huggingface.co/docs/hub/models-gated#gated-models). Returns: `type`: The class, dynamically imported from the module. Examples: ```python # Download module `modeling.py` from huggingface.co and cache then extract the class `MyBertModel` from this # module. cls = get_class_from_dynamic_module("sgugger/my-bert-model", "modeling.py", "MyBertModel") ```rrJ)rrgra) ryr=rerzr{r|r}r~rkwargs final_modulerrrget_class_from_dynamic_moduleusJ r)NFNNNF)NNFNNNF).__doc__rYrlrr'r8rr$pathlibrtypingrrrurllibrhuggingface_hubrrhuggingface_hub.utilsr r packagingr rr rrr get_loggerrprrr r,rBPathLiker/rArQr`rgrcboolrrrrrrs         1