o ̳i!@sddlZddlZddlmZddlmZddlmZddlm Z m Z ddl m Z ddl mZmZddlmZdd lmZeeZgd Ze jd e jd e jd iZe jhde jddhe jhdiZ d.ddddededBdedBdedBdef ddZ  d/dededede eeBdBddf dd Z!dddddd!d"e dededededBd#ed$e dBd%edefd&d'Z"dddd(d"e dededededBd#edefd)d*Z#d"e d+eddfd,d-Z$dS)0N)Any) ApiBlobType)registry)KaggleDatasetAdapterPolarsFrameType)create_dataset_or_version)normalize_patternsupload_files_and_directories)parse_dataset_handle)EXTRA_CONSOLE_BLOCK)z.git/z*/.git/z.cache/z .huggingface/z hf-datasetszpandas-datasetszpolars-datasets> hf_kwargs sql_query pandas_kwargsrr >r polars_kwargspolars_frame_typeFforce_download output_dirhandlepathrrreturncCs@t|}tjd|ditdtj||||d\}}|S)aDownload dataset files Args: handle: (string) the dataset handle path: (string) Optional path to a file within a dataset force_download: (bool) Optional flag to force download a dataset, even if it's cached or already in output_dir. output_dir: (string) Optional output directory for direct download, bypassing the default cache. Returns: A string requesting the path to the requested dataset files. zDownloading Dataset:  ...)extrar)r loggerinfoto_urlr rdataset_resolver)rrrrh resolved_path_r F/home/ubuntu/.local/lib/python3.10/site-packages/kagglehub/datasets.pydataset_download"s r"local_dataset_dir version_notesignore_patternscCsZt|}td|d|rd}t|t|tjt t |dd}t |||dS)aUpload dataset files. Args: handle: (string) the dataset handle. local_dataset_dir: (string) path to a file in a local directory. version_notes: (string) Optional to write dataset versions. ignore_patterns (str or list[str], optional): Additional ignore patterns to DEFAULT_IGNORE_PATTERNS. Files matching any of the patterns are not uploaded. Patterns are standard wildcards that can be matched by https://docs.python.org/3/library/fnmatch.html. Use a pattern ending with "/" to ignore the whole dir, e.g., ".git/" is equivalent to ".git/*". zUploading Dataset rz1The dataset handle should not include the version)default additional) item_typer&N) r rrr is_versioned ValueErrorr rDATASETrDEFAULT_IGNORE_PATTERNSr)rr$r%r&ris_versioned_exceptiontokensr r r!dataset_upload=s r0rr r rradapterr rrc Cst||||||d|dur|ntj}zG|tjur)ddl}|jj|||||dWS|tjur=ddl }|j j ||||dWS|tj urRddl }|jj|||||dWS|d} t| tyst|} d| d | d } t| dw) aLoad a Kaggle Dataset into a python object based on the selected adapter Args: adapter: (KaggleDatasetAdapter) The adapter used to load the dataset handle: (string) The dataset handle path: (string) Path to a file within the dataset pandas_kwargs: (dict) Optional set of kwargs to pass to the pandas `read_*` method while constructing the DataFrame(s) sql_query: (string) Argument to be used for SQLite files. Required when reading a SQLite file. See pandas documentation for details: https://pandas.pydata.org/docs/reference/api/pandas.read_sql_query.html hf_kwargs: (dict) Optional set of kwargs to pass to Dataset.from_pandas() while constructing the Dataset polars_frame_type: (PolarsFrameType) Optional control for which Frame to return: LazyFrame or DataFrame. The default is PolarsFrameType.LAZY_FRAME. polars_kwargs: (dict) Optional set of kwargs to pass to the polars `read_*` method while constructing the DataFrame(s) Returns: A python object based on the selected adapter: - 'pandas': A DataFrame (or dict[int | str, DataFrame] for Excel-like files with multiple sheets) - 'hugging_face': A Huggingface Dataset (via pandas) - 'polars': A LazyFrame or DataFrame (or dict[int | str, LazyFrame] / dict[int | str, DataFrame] for Excel-like files with multiple sheets) r1Nrrr r )rr )rrr z is not yet implementedz*The 'dataset_load' function requires the 'z3' extras. Install them with 'pip install kagglehub[z]')validate_dataset_load_argsr LAZY_FRAMEr HUGGING_FACEkagglehub.hf_datasets hf_datasetsload_hf_datasetPANDASkagglehub.pandas_datasetspandas_datasetsload_pandas_datasetPOLARSkagglehub.polars_datasetspolars_datasetsload_polars_datasetNotImplementedError ImportError.DATASET_LOAD_ADAPTER_OPTIONAL_DEPENDENCIES_MAP) r2rrrr r rr kagglehubnot_implemented_error_messageadapter_optional_dependencyimport_warning_messager r r! dataset_load_sN%       rIr3cCs$tjdtddt||||||dS)NzaUse dataset_load() instead of load_dataset(). load_dataset() will be removed in a future version.) stacklevelr3)warningswarnDeprecationWarningrI)r2rrrr r r r r! load_datasets  rOkwargscKslt|}g}|D]\}}||vr|dur||q t|dkr$dSd|}t|d|ddS)Nrz, z are invalid for z and will be ignored)_DATASET_LOAD_VALID_KWARGS_MAPitemsappendlenjoinrwarning)r2rP valid_kwargsinvalid_kwargs_listkeyvalueinvalid_kwargsr r r!r4s   r4)N)r#N)%loggingrLtypingr&kagglesdk.blobs.types.blob_api_servicerrErkagglehub.datasets_enumsrrkagglehub.datasets_helpersrkagglehub.gcs_uploadrr kagglehub.handler kagglehub.loggerr getLogger__name__rr-r6r:r>rDrQstrboolr"listr0rIrOr4r r r r!s             (   X