o ϯi'@s ddlmZddlZddlZddlZddlZddlZddlmZddl m Z m Z m ZddlmZmZmZmZmZmZmZddlZddlZddlmZeeZd4d d Z d5d6ddZ d7ddZ!Gdddej"j#j$Z%d8d!d"Z&Gd#d$d$Z' %d9d:d,d-Z(d;d2d3Z)dS)<) annotationsN) defaultdict)Fieldfieldfields)AnyCallableDictIterableListTupleUnion) g_pathmgrdata torch.TensorreturncCs|ddddS)zd Permute tensor from (time, height, weight, channel) to (channel, height, width, time). r)permute)rrK/home/ubuntu/.local/lib/python3.10/site-packages/pytorchvideo/data/utils.py thwc_to_cthwsrfloortime_in_secondsfloat time_base start_ptsint round_modestrcCsT|tjkrtjS|dvsJd|d|dkr!t|||St|||S)z Converts a time (in seconds) to the given time base and start_pts offset presentation time. Round_mode specifies the mode of rounding when converting time. Returns: pts (int): The time in the given time base. )rceilz round_mode=z is not supported!r)mathinfrr!)rrrrrrr secs_to_ptss r$ptscCs |tjkrtjSt|||S)z Converts a present time with the given time base and start_pts offset to seconds. Returns: time_in_seconds (float): The corresponding time in seconds. )r"r#r)r%rrrrr pts_to_secs6s r&c@s"eZdZdZd ddZdd Zd S) MultiProcessSamplerz MultiProcessSampler splits sample indices from a PyTorch Sampler evenly across workers spawned by a PyTorch DataLoader. samplertorch.utils.data.SamplerrNonecCs ||_dSN)_sampler)selfr(rrr__init__Is zMultiProcessSampler.__init__c Cstjj}|durT|jdkrTtt|j}t ||j}|j }||}t|dkr>t d|jdt|jdt dS|d}|dd}tt |j||}|St |j}|S) zj Returns: Iterator for underlying PyTorch Sampler indices split by worker id. NrzMore data workers(z) than videos(z3). For optimal use of processes reduce num_workers.rr)torchutilsrget_worker_info num_workersrangelenr,np array_splitidloggerwarningiter itertoolsislice) r- worker_info video_indexes worker_splits worker_id worker_split iter_startiter_endworker_samplerrrr__iter__Ls&     zMultiProcessSampler.__iter__N)r(r)rr*)__name__ __module__ __qualname____doc__r.rFrrrrr'Cs  r'targetr args_iterableIterable[Tuple] multithreadedboolcCs^|r$g}|D]}tj||d}|||q|D]}|qdS|D]}||q&dS)a Applies 'target' function to each Tuple args in 'args_iterable'. If 'multithreaded' a thread is spawned for each function application. Args: target (Callable): A function that takes as input the parameters in each args_iterable Tuple. args_iterable (Iterable[Tuple]): An iterable of the tuples each containing a set of parameters to pass to target. multithreaded (bool): Whether or not the target applications are parallelized by thread. )rKargsN) threadingThreadstartappendjoin)rKrLrNthreadsrPthreadtrrroptional_threaded_foreachls   rYc@s,eZdZdZdZd ddZedd d Zd S)DataclassFieldCasterz Class to allow subclasses wrapped in @dataclass to automatically cast fields to their relevant type by default. Also allows for an arbitrary intialization function to be applied for a given field. )DataclassFieldCaster__complex_initializerrr*c Csd|jdt|D]Q}t||j}t||jrtj|jvr\tj|jvrRt||j|jtj|tt||j|jsQJd|jd|jdtt||jq t||j||q dS)Na This function is run by the dataclass library after '__init__'. Here we use this to ensure all fields are casted to their declared types and to apply any complex field_initializer functions that have been declared via the 'complex_initialized_dataclass_field' method of this class. A complex field_initializer for a given field would be stored in the field.metadata dictionary at: key = 'z&' (self.COMPLEX_INITIALIZER) z 'field_initializer' function of z must return type z but returned type ) COMPLEX_INITIALIZERdataclass_fieldsgetattrname isinstancetyperZmetadatasetattr)r-rvaluerrr __post_init__s0      z"DataclassFieldCaster.__post_init__field_initializerrrcKs<|dpi}tj|vsJ||tj<||d<tdi|S)u Allows for the setting of a function to be called on the named parameter associated with a field during initialization, after __init__() completes. Args: field_initializer (Callable): The function to be called on the field **kwargs: To be passed downstream to the dataclasses.field method Returns: (dataclasses.Field) that contains the field_initializer and kwargs infoÎ rbNr)getrZr\dataclass_field)rfkwargsrbrrr#complex_initialized_dataclass_fields  z8DataclassFieldCaster.complex_initialized_dataclass_fieldN)rr*)rfrrr)rGrHrIrJr\re staticmethodrjrrrrrZs  &rZFinput_csv_file_pathdataclass_classradict_key_field list_per_key Dict[Any, Union[Any, List[Any]]]c s|rttni}t|P}tj|ddd}ddtt|D|D]1|fddt|D}t ||}|rB|| |q#||vsPJd|d ||||<q#Wd |S1s`wY|S) a Args: input_csv_file_path (str): File path of the csv to read from dataclass_class (type): The dataclass to read each row into. dict_key_field (str): The field of 'dataclass_class' to use as the dictionary key. list_per_key (bool) = False: If the output data structure contains a list of dataclass objects per key, rather than a single unique dataclass object. Returns: Dict[Any, Union[Any, List[Any]] mapping from the dataclass value at attr = dict_key_field to either: if 'list_per_key', a list of all dataclass objects that have equal values at attr = dict_key_field, equal to the key if not 'list_per_key', the unique dataclass object for which the value at attr = dict_key_field is equal to the key Raises: AssertionError: if not 'list_per_key' and there are dataclass obejcts with equal values at attr = dict_key_field ," delimiter quotecharcSsi|]\}}||qSrr).0iheaderrrr z0load_dataclass_dict_from_csv..c3s|] }|jVqdSr+r_)rvr column_indexlinerr s  z/load_dataclass_dict_from_csv..zMultiple entries for z in N) rlistropencsvreader enumeratenextr]r^rT) rlrmrnro output_dictdataclass_filerdatumdict_keyrr|rload_dataclass_dict_from_csvs,      rdataclass_objs List[Any] file_namer*cst|d}ddt|D}t|d'}tj|ddd}|||D]|fdd|Dq%Wd d S1s?wYd S) z Saves a list of @dataclass objects to the specified csv file. Args: dataclass_objs (List[Any]): A list of @dataclass objects to be saved. file_name (str): file_name to save csv data to. rcSsg|]}|jqSrr{rvfrrr sz7save_dataclass_objs_to_headered_csv..wrqrrrscsg|]}t|qSr)r^robjrrrrzN)rar]rrrwriterwriterow)rrdataclass_type field_namesrrrrr#save_dataclass_objs_to_headered_csv s  "r)rrrr)r) rrrrrrrr rr)r%rrrrrrr)rKrrLrMrNrO)F) rlr rmrarnr rorOrrp)rrrr rr*)* __future__rrr<loggingr"rQ collectionsr dataclassesrrrhrr]typingrrr r r r r numpyr6r0iopath.common.file_ior getLoggerrGr9rr$r&r1rSamplerr'rYrZrrrrrrs.  $     )!N 5