o 8wi@sZddlZddlmZmZddlmZddlmZmZm Z ddl m Z eGdddZ dS) N) dataclassfield)Path)AnyOptionalUnion)configcs.eZdZUdZdZeeeefe d<dZ e e d<dZ e e d<dZ e e d<dZeee d<dZeee d <dZe e d <dZe e d <dZe e d <dZe e d <dZe e d<dZeee d<dZee d<dZeeee fe d<eedZeeefe d<dZeee d<dZe e d<dddZfddZ Z!S)DownloadConfiga\ Configuration for our cached path manager. Attributes: cache_dir (`str` or `Path`, *optional*): Specify a cache directory to save the file to (overwrite the default cache dir). force_download (`bool`, defaults to `False`): If `True`, re-download the file even if it's already cached in the cache dir. resume_download (`bool`, defaults to `False`): If `True`, resume the download if an incompletely received file is found. proxies (`dict`, *optional*): user_agent (`str`, *optional*): Optional string or dict that will be appended to the user-agent on remote requests. extract_compressed_file (`bool`, defaults to `False`): If `True` and the path point to a zip or tar file, extract the compressed file in a folder along the archive. force_extract (`bool`, defaults to `False`): If `True` when `extract_compressed_file` is `True` and the archive was already extracted, re-extract the archive and override the folder where it was extracted. delete_extracted (`bool`, defaults to `False`): Whether to delete (or keep) the extracted files. extract_on_the_fly (`bool`, defaults to `False`): If `True`, extract compressed files while they are being read. use_etag (`bool`, defaults to `True`): Whether to use the ETag HTTP response header to validate the cached files. num_proc (`int`, *optional*): The number of processes to launch to download the files in parallel. max_retries (`int`, default to `1`): The number of times to retry an HTTP request if it fails. token (`str` or `bool`, *optional*): Optional string or boolean to use as Bearer token for remote files on the Datasets Hub. If `True`, or not specified, will get token from `~/.huggingface`. storage_options (`dict`, *optional*): Key/value pairs to be passed on to the dataset file-system backend, if any. download_desc (`str`, *optional*): A description to be displayed alongside with the progress bar while downloading the files. disable_tqdm (`bool`, defaults to `False`): Whether to disable the individual files download progress bar N cache_dirFforce_downloadresume_downloadlocal_files_onlyproxies user_agentextract_compressed_file force_extractdelete_extractedextract_on_the_flyTuse_etagnum_proc max_retriestoken)default_factorystorage_options download_desc disable_tqdmreturncCs |jdidd|jDS)NcSsi|] \}}|t|qS)copydeepcopy).0kvrr^/home/ubuntu/sommelier/.venv/lib/python3.10/site-packages/datasets/download/download_config.py Isz'DownloadConfig.copy..r) __class____dict__items)selfrrr%r Hs zDownloadConfig.copycsl|dkr-t|dddur-d|jvr|tjd|jd<nt|jddddur-||jdd<t||dS)Nrrhf)rendpoint)getattrrr HF_ENDPOINTsuper __setattr__)r*namevaluer'rr%r0Ks  zDownloadConfig.__setattr__)rr )"__name__ __module__ __qualname____doc__r rrstrr__annotations__r boolr rrdictrrrrrrrintrrrrrrrr r0 __classcell__rrr3r%r s* +           r ) r dataclassesrrpathlibrtypingrrrr r rrrr%s