o .i* @s4ddlZddlmZmZddlmZmZmZmZm Z m Z ddl Z ddl ZddlmZddlmZddlmZddlmZmZdd lmZerRdd lmZd d lmZGd dde ZeGdddZddde fddZ!dddefddZ"de j#defddZ$ d de%dee&e%e e'e%ffde%ddfddZ(dS)!N) dataclassfield) TYPE_CHECKINGAnyClassVarOptional TypedDictUnion)config)DownloadConfig) array_cast) is_local_pathxopen)string_to_dict VideoReader) FeatureTypec@s&eZdZUeeed<eeed<dS)ExamplepathbytesN)__name__ __module__ __qualname__rstr__annotations__rrrK/home/ubuntu/.local/lib/python3.10/site-packages/datasets/features/video.pyrs rc @seZdZUdZdZeed<dZee ed<dZ e e ed<e e e dZe eed <edd d d Ze ed <d dZdee eeejdfdefddZ ddee efdeee eee ffddfddZdedee dfffddZdee je je j fde jfddZ!dS)Videoa% **Experimental.** Video [`Feature`] to read video data from a video file. Input: The Video feature accepts as input: - A `str`: Absolute path to the video file (i.e. random access is allowed). - A `dict` with the keys: - `path`: String with relative path of the video file in a dataset repository. - `bytes`: Bytes of the video file. This is useful for archived files with sequential access. - A `torchvision.io.VideoReader`: torchvision video reader object. Args: mode (`str`, *optional*): The mode to convert the video to. If `None`, the native mode of the video is used. decode (`bool`, defaults to `True`): Whether to decode the video data. If `False`, returns the underlying dictionary in the format `{"path": video_path, "bytes": video_bytes}`. Examples: ```py >>> from datasets import Dataset, Video >>> ds = Dataset.from_dict({"video":["path/to/Screen Recording.mov"]}).cast_column("video", Video()) >>> ds.features["video"] Video(decode=True, id=None) >>> ds[0]["video"] >>> ds = ds.cast_column('video', Video(decode=False)) {'bytes': None, 'path': 'path/to/Screen Recording.mov'} ``` TdecodeNidztorchvision.io.VideoReaderdtyperrpa_typeF)defaultinitrepr_typecCs|jSN)r$)selfrrr__call__GzVideo.__call__valuerreturncCstjr ddlm}nd}t|trt|}t|tr |ddSt|t r*d|dSt|tj r4t |S|durAt||rAt |St|t ru|d|d}}|dur`tj|r`d|dS|dush|durm||dStd|d td t|) zEncode example into a format for Arrow. Args: value (`str`, `np.ndarray`, `VideoReader` or `dict`): Data passed as input to Video feature. Returns: `dict` with "path" and "bytes" fields rrNrrrrr#zTA video sample should have one of 'path' or 'bytes' but they are missing or None in .z!Unsupported encode_example type: )r TORCHVISION_AVAILABLEtorchvision.ior isinstancelistnparrayrrndarrayencode_np_arrayencode_torchvision_videodictgetosrisfile ValueError TypeErrortype)r*r-rrbytes_rrrencode_exampleJs.            zVideo.encode_exampletoken_per_repo_idcCs|jstdtjrddlm}ntd|duri}t|tr&|d}}n |d|d}}|durO|dur?t d|d t |rH||}n t ||d }n||}||d |_ |S) a|Decode example video file into video data. Args: value (`str` or `dict`): A string with the absolute video file path, a dictionary with keys: - `path`: String with absolute or relative video file path. - `bytes`: The bytes of the video file. token_per_repo_id (`dict`, *optional*): To access and decode video files from private repositories on the Hub, you can pass a dictionary repo_id (`str`) -> token (`bool` or `str`). Returns: `torchvision.io.VideoReader` zMDecoding is disabled for this feature. Please use Video(decode=True) instead.rrz9To support decoding videos, please install 'torchvision'.NrrzBA video should have one of 'path' or 'bytes' but both are None in r0)rCr/) r RuntimeErrorr r1r2r ImportErrorr3rr>rhf_video_reader _hf_encoded)r*r-rCrrrAvideorrrdecode_examplevs&    zVideo.decode_examplercCs(ddlm}|jr |S|d|ddS)zfIf in the decodable state, return the feature itself, otherwise flatten the feature into a dictionary.r)Valuebinarystringr#)featuresrJr )r*rJrrrflattens z Video.flattenstoragecCstj|jr%tjdgt|td}tjj||gddg| d}ntj |jrJtjdgt|t d}tjj||gddg| d}ntj |jr|j ddkr_|d}ntjdgt|td}|j ddkr{|d}ntjdgt|t d}tjj||gddg| d}n4tj|jrtjdd|Dtd}tjdgt|t d}tjj||gddg| d}t||jS) a'Cast an Arrow array to the Video arrow storage type. The Arrow types that can be converted to the Video pyarrow storage type are: - `pa.string()` - it must contain the "path" data - `pa.binary()` - it must contain the video bytes - `pa.struct({"bytes": pa.binary()})` - `pa.struct({"path": pa.string()})` - `pa.struct({"bytes": pa.binary(), "path": pa.string()})` - order doesn't matter - `pa.list(*)` - it must contain the video array data Args: storage (`Union[pa.StringArray, pa.StructArray, pa.ListArray]`): PyArrow array to cast. Returns: `pa.StructArray`: Array in the Video arrow storage type, that is `pa.struct({"bytes": pa.binary(), "path": pa.string()})`. N)r@rr)maskrcSs*g|]}|durtt|dndqS)Nr)r8r5r6).0arrrrr s*z&Video.cast_storage..)patypes is_stringr@r6lenrK StructArray from_arraysis_null is_binaryrL is_structget_field_indexris_list to_pylistr r$)r*rO bytes_array path_arrayrrr cast_storages0      zVideo.cast_storager))"rrr__doc__r boolrr!rrr"rrTstructrKrLr$rrr(r+r rrr5r7rBr:rIrN StringArrayrX ListArrayrbrrrrrs&  $$"/  3( rrHrr.cCt)zPConvert a torchvision Video object to bytes using native compression if possibleNotImplementedErrorrHrrrvideo_to_bytessrlcCst|dr|jStd)NrGzXEncoding a VideoReader that doesn't come from datasets.Video.decode() is not implemented)hasattrrGrjrkrrrr9s r9r6cCrhr)ri)r6rrrr8r,r8rrCstreamcCs$ddl}ddlm}ddlm}|duri}|dd}|tjr&tj ntj }t ||}|dur9| |dnd} t | d} t|d| d } t|} || _| jd kr]td | j|j| d d | _|dd} t|ddkrwdnt|dd}| |i| _| jjdi| j| _| S)Nr)get_video_backendrz::repo_id)tokenrb)download_configpyavz9Unsupported video backend for VideoReader from HF files: ignore)metadata_errors:rr)av torchvisionror2rsplit startswithr HF_ENDPOINTHUB_DATASETS_URLHUB_DATASETS_HFFS_URLrr;r robject__new__backendrDopen containerrWint pyav_streamr _c)rrCrnryror source_urlpatternsource_url_fieldsrrrtfvr stream_type stream_idrrrrFs*      ( rF)NrH))r< dataclassesrrtypingrrrrrr numpyr5pyarrowrTr download.download_configr tabler utils.file_utilsrrutils.py_utilsrr2rrMrrrrrlr9r7r8rr:rdrFrrrrs<       K