o Gid@sddlZddlZddlZddlZddlZddlZddlmZddlm Z ddl m Z m Z m Z mZmZmZmZmZddlmZmZddlmZddlmZmZmZmZmZmZddlm Z d d l!m"Z"d d l#m$Z$m%Z%m&Z&m'Z'd d l(m)Z)m*Z*m+Z+m,Z,m-Z-m.Z.m/Z/m0Z0d dl1m2Z2e2e3Z4ee5j6dZ7e j8Z9dCde:e;BdBde;fddZ<            dDde;de;dBde=de=de;dBde;de;dBde;dBde>e:dBde=dBde=d e=de fd!d"Z?dCd#e d$e;e>e;BdBde fd%d&Z@dCd'e;dBd(e;dBfd)d*ZAdCd+e;d,e;dBde;fd-d.ZBeddddddddddd/ d0e;eBd+e;d1e;dBd2e;dBd3e=d4e:dBd5e=de;dBde:e;BdBd6e;dBd(e;dBd7e:e;e fdBfd8d9ZC       : dEd7e:e;e fdBfd;d<ZDdFd=e;d>e>e;d,e;fd?d@ZEGdAdBdBZFdS)GN)Path)uuid4) DDUFEntry ModelCard ModelCardData create_repohf_hub_download model_infosnapshot_download upload_folder)HF_HUB_DISABLE_TELEMETRYHF_HUB_OFFLINE)REGEX_COMMIT_HASH)EntryNotFoundErrorHfHubHTTPErrorRepositoryNotFoundErrorRevisionNotFoundErroris_jinja_availablevalidate_hf_hub_args)version) __version__)DEPRECATED_REVISION_ARGSHUGGINGFACE_CO_RESOLVE_ENDPOINTSAFETENSORS_WEIGHTS_NAME WEIGHTS_NAME)ENV_VARS_TRUE_VALUES _flax_version _jax_version_onnxruntime_version_torch_versionis_flax_availableis_onnx_availableis_torch_available) get_loggerzmodel_card_template.md user_agentreturncCsdtdtjddt}tstr|dStr"|dt7}t r3|dt 7}|dt 7}t r=|d t 7}tjd d tvrL|d 7}t|trc|d d dd|D7}|St|trn|d |7}|S)zF Formats a user-agent string with basic info about a request. z diffusers/z ; python/rz ; session_id/z; telemetry/offz; torch/z; jax/z; flax/z; onnxruntime/DIFFUSERS_IS_CIz ; is_ci/truez; css"|] \}}|d|VqdS)/N).0kvr+r+M/home/ubuntu/.local/lib/python3.10/site-packages/diffusers/utils/hub_utils.py \s z"http_user_agent..)rsysrsplit SESSION_IDr r r$r!r"rrr#r osenvirongetupperr isinstancedictjoinitemsstr)r&uar+r+r/http_user_agentJs$    r>Frepo_id_or_pathtoken is_pipeline from_trainingmodel_description base_modelpromptlicensewidget inference is_modularupdate_model_cardc  Cststd| r| stdz%tj||d} | r,| r/|dur2| j} t|} | | _W| SW| SW| SW| Sttfy|rRtjt|d| |||dt |d} Y| St}| re|duret|} || _Y| S|ridnd }|durud |d }tj||d } Y| Sw) a Loads or creates a model card. Args: repo_id_or_path (`str`): The repo id (e.g., "stable-diffusion-v1-5/stable-diffusion-v1-5") or local path where to look for the model card. token (`str`, *optional*): Authentication token. Will default to the stored token. See https://huggingface.co/settings/token for more details. is_pipeline (`bool`): Boolean to indicate if we're adding tag to a [`DiffusionPipeline`]. from_training: (`bool`): Boolean flag to denote if the model card is being created from a training script. model_description (`str`, *optional*): Model description to add to the model card. Helpful when using `load_or_create_model_card` from a training script. base_model (`str`): Base model identifier (e.g., "stabilityai/stable-diffusion-xl-base-1.0"). Useful for DreamBooth-like training. prompt (`str`, *optional*): Prompt used for training. Useful for DreamBooth-like training. license: (`str`, *optional*): License of the output artifact. Helpful when using `load_or_create_model_card` from a training script. widget (`list[dict]`, *optional*): Widget to accompany a gallery template. inference: (`bool`, optional): Whether to turn on inference widget. Helpful when using `load_or_create_model_card` from a training script. is_modular: (`bool`, optional): Boolean flag to denote if the model card is for a modular pipeline. When True, uses model_description as-is without additional template formatting. update_model_card: (`bool`, optional): When True, regenerates the model card content even if one already exists on the remote repo. Existing card metadata (tags, license, etc.) is preserved. Only supported for modular pipelines (i.e., `is_modular=True`). zModelcard rendering is based on Jinja templates. Please make sure to have `jinja` installed before using `load_or_create_model_card`. To install it, please run `pip install Jinja2`.zU`update_model_card=True` is only supported for modular pipelines (`is_modular=True`).r@N diffusers)rF library_namerHrDinstance_promptrG) card_data template_pathrCpipelinemodelu+This is the model card of a 🧨 diffusers zS that has been pushed on the Hub. This model card has been automatically generated.)rC) r ValueErrorrloaddatarr from_templaterMODEL_CARD_TEMPLATE_PATH)r?r@rArBrCrDrErFrGrHrIrJ model_card existing_datarO componentr+r+r/load_or_create_model_cardbsZ+   r[rXtagscCs\|jjdur d|j_|dur,t|tr|g}|jjdur g|j_|D] }|jj|q"|S)z?Populates the `model_card` with library name and optional tags.NrL)rUrMr8r<r\append)rXr\tagr+r+r/populate_model_cards   r_ resolved_file commit_hashcCsZ|dus|dur |Stt|}td|}|durdS|d}t|r+|SdS)zP Extracts the commit hash from a resolved filename toward a cache file. Nzsnapshots/([^/]+)/r)r<ras_posixresearchgroupsrmatch)r`rardr+r+r/extract_commit_hashs  rg weights_namevariantcCs>|dur|d}|dd|g|dd}d|}|S)N.)r2r:)rhrisplitsr+r+r/ _add_variants   rm) subfolder cache_dirforce_downloadproxieslocal_files_onlyr@r&revisionra dduf_entriespretrained_model_name_or_pathrnrorprqrrrsrtc Cst|}| r.|durtd|dkr|nd||g} | | vr!| Std|d| dtj|r6|Stj|rutjtj||rPtj||} | S|durjtjtj|||rjtj|||} | Std|d|d| t vr|t ks|t krt t tjt dkrz/t|t|| |||||||| p| d } td | d |d | d | d| d| d t| WStd| d |d | d| d|dt|| d|d|dt|| dtYzt|||||||||| p| d } | WSty} zt|d| d} ~ wty*} z t| d|d| d} ~ wtyA} z t|d|d| d} ~ wtyX} z td|d| | d} ~ wtys} ztdtd |d!|d"| d} ~ wty} ztd#|d$|d%|| d} ~ ww)&NzDDUF file only allow for 1 level of directory (e.g transformer/model1/model.safetentors is not allowed). Please check the DDUF structurer)r*zError no file named z found in archive rjz found in directory z0.22.0) filenamerorprqrrr@r&rnrszLoading the variant z from z via `revision='zG'` is deprecated. Loading instead from `revision='main'` with `variant=z)`. Loading model variants via `revision='z9'` will be removed in diffusers v1. Please use `variant='z '` instead.zYou are loading the variant z^'`. This behavior is deprecated and will be removed in diffusers v1. One should use `variant='z%'` instead. However, it appears that z currently does not have a z file in the 'main' branch of z. The Diffusers team and community would be very grateful if you could open an issue: https://github.com/huggingface/diffusers/issues/new with the title 'z is missing z0' so that the correct variant file can be added.z is not a local folder and is not a valid model identifier listed on 'https://huggingface.co/models' If this is a private repository, make sure to pass a token having permission to this repo with `token` or log in with `hf auth login`.z is not a valid git identifier (branch name, tag name or commit id) that exists for this model name. Check the model page at 'https://huggingface.co/z' for available revisions.& does not appear to have a file named z:There was a specific connection error when trying to load z: We couldn't connect to 'zM' to load this model, couldn't find it in the cached files and it looks like z8 is not the path to a directory containing a file named z or Checkout your internet connection or see how to run the library in offline mode at 'https://huggingface.co/docs/diffusers/installation#offline-mode'.zCan't load the model for 'z'. If you were trying to load it from 'https://huggingface.co/models', make sure you don't have a local directory with the same name. Otherwise, make sure 'z=' is the correct path to a directory containing a file named )r<rSr:EnvironmentErrorkeysr4pathisfileisdirrrrrparser base_versionrrmwarningswarn FutureWarningrrrrr)rurhrnrorprqrrr@r&rsrart model_fileer+r+r/_get_model_files    (F rr)c  s|| r|| vrtd|d|dntj|s#td|d|d| r/t| |} nt|d} t| } Wdn1sFwYt t | d } | d} t | d | d<| d| d<tj|tj|sz| rfd d | D}|D]#| r| vrtd d qtjstd d q|| fS| }durfd d |D}ddg}|st|||d}| D]tfdd|jD}|std d qzt|||||||||d durtjWnty}z tdtd|d|d}~wwfdd | D}|D]}tj|s8td|d q%|| fS)a For a given model: - download and cache all the shards of a sharded checkpoint if `pretrained_model_name_or_path` is a model ID on the Hub - returns the list of paths to all the shards, as well as some metadata. For the description of each arg, see [`PreTrainedModel.from_pretrained`]. `index_filename` is the full path to the index (downloaded and cached if `pretrained_model_name_or_path` is a model ID on the Hub). zCan't find a checkpoint index (z) in rjrN weight_mapmetadataall_checkpoint_keyscg|] }tj|qSr+r4r{r:r,f) shards_pathr+r/ z/_get_checkpoint_shard_files..rwz5 which is required according to the checkpoint index.crr+r)r,p)rnr+r/rrz*.jsonz*.md)rsr@c3s|]}|jvVqdSN) rfilename)r,r-) shard_filer+r/r0sz._get_checkpoint_shard_files..)rorqrrr@rsallow_patternsignore_patternsr&rxz ' to load z?. You should try again after checking your internet connection.crr+rr) cached_folderr+r/rrz does not have a file named )rSr4r{r|jsonloads read_textopenreadsortedsetvalueslistrzcopyr:r}FileNotFoundErrorexistsr anysiblingsryr rr)ruindex_filenamerorqrrr@r&rsrnrtindexroriginal_shard_filenamessharded_metadatashard_filenamesrrmodel_files_infoshard_file_presentrcached_filenames cached_filer+)rrrrnr/_get_checkpoint_shard_filesgs    rfolder filenamescs||r|rtd|s%g}t|D]\}}}|D] }|tj|qqd}td|d|dtfdd|DS)Nz1Both `filenames` and `folder` cannot be provided.z\d{5}-of-\d{5}z.*-z\.z \.[a-z]+$c3s|] }|duVqdSr)rfrvariant_file_rer+r/r0sz8_check_legacy_sharding_variant_format..) rSr4walkr]r{basenamerccompiler)rrri_filesfiletransformers_index_formatr+rr/%_check_legacy_sharding_variant_formatsrc@seZdZdZ    ddeejBdededBdedBded edBf d d Z      ddededBd edBdedBdedededBd edBdefddZ dS)PushToHubMixinzR A Mixin to push a model, scheduler, or pipeline to the Hugging Face Hub. NF working_dirrepo_idr@commit_message create_prrncCsf|durd|jjvr d}nd|jjvrd}nd|jj}td|d|d t||||||d S) zB Uploads all files in `working_dir` to `repo_id`. NModelz Upload model SchedulerzUpload schedulerzUpload zUploading the files of z to rj)r folder_pathr@rr path_in_repo) __class____name__loggerinfor )selfrrr@rrrnr+r+r/_upload_folders  zPushToHubMixin._upload_folderTprivatesafe_serializationrir'c Cst|||ddj}|st||d} t| } d|i} d|jjvr&| d|it(} |j | fi| |s@| t j | d|j| |||||dWd S1sUwYd S) uy Upload model, scheduler, or pipeline files to the 🤗 Hugging Face Hub. Parameters: repo_id (`str`): The name of the repository you want to push your model, scheduler, or pipeline files to. It should contain your organization name when pushing to an organization. `repo_id` can also be a path to a local directory. commit_message (`str`, *optional*): Message to commit while pushing. Default to `"Upload {object}"`. private (`bool`, *optional*): Whether to make the repo private. If `None` (default), the repo will be public unless the organization's default is private. This value is ignored if the repo already exists. token (`str`, *optional*): The token to use as HTTP bearer authorization for remote files. The token generated when running `hf auth login` (stored in `~/.huggingface`). create_pr (`bool`, *optional*, defaults to `False`): Whether or not to create a PR with the uploaded files or directly commit. safe_serialization (`bool`, *optional*, defaults to `True`): Whether or not to convert the model weights to the `safetensors` format. variant (`str`, *optional*): If specified, weights are saved in the format `pytorch_model..bin`. Examples: ```python from diffusers import UNet2DConditionModel unet = UNet2DConditionModel.from_pretrained("stabilityai/stable-diffusion-2", subfolder="unet") # Push the `unet` to your namespace with the name "my-finetuned-unet". unet.push_to_hub("my-finetuned-unet") # Push the `unet` to an organization with the name "my-finetuned-unet". unet.push_to_hub("your-org/my-finetuned-unet") ``` T)rr@exist_okrKrrriz README.md)r@rrrnN)rrr[r_rrupdatetempfileTemporaryDirectorysave_pretrainedsaver4r{r:r) rrrrr@rrrirnrX save_kwargstmpdirr+r+r/ push_to_hubs(0   $zPushToHubMixin.push_to_hub)NNFN)NNNFTNN) r __module__ __qualname____doc__r<r4PathLikeboolrrr+r+r+r/rsZ !  rr) NNFFNNNNNNFF)NNFNNNr)N)NNN)Grr4rcr1rrpathlibruuidrhuggingface_hubrrrrrr r r huggingface_hub.constantsr r huggingface_hub.file_downloadrhuggingface_hub.utilsrrrrrr packagingrr)r constantsrrrr import_utilsrrrr r!r"r#r$loggingr%rr__file__parentrWhexr3r9r<r>rrr[r_rgrmrrrrr+r+r+r/s  (    (      $Z       o