o پi@sLddlZddlZddlmZddlmZGdddeZGdddeZdS)N)Samplerc@s*eZdZdZd ddZddZddZdS) OrderedDistributedSampleraSampler that restricts data loading to a subset of the dataset. It is especially useful in conjunction with :class:`torch.nn.parallel.DistributedDataParallel`. In such case, each process can pass a DistributedSampler instance as a DataLoader sampler, and load a subset of the original dataset that is exclusive to it. .. note:: Dataset is assumed to be of constant size. Arguments: dataset: Dataset used for sampling. num_replicas (optional): Number of processes participating in distributed training. rank (optional): Rank of the current process within num_replicas. NcCs|durts tdt}|dur tstdt}||_||_||_tt t |jd|j|_ |j |j|_ dS)N,Requires distributed package to be availableg?)dist is_available RuntimeErrorget_world_sizeget_rankdataset num_replicasrankintmathceillen num_samples total_size)selfr r r rQ/home/ubuntu/.local/lib/python3.10/site-packages/timm/data/distributed_sampler.py__init__s z"OrderedDistributedSampler.__init__cCslttt|j}||d|jt|7}t||jksJ||j|j|j}t||jks2Jt|SN) listrangerr rr r riter)rindicesrrr__iter__%s z"OrderedDistributedSampler.__iter__cC|jSr)rrrrr__len__2z!OrderedDistributedSampler.__len__)NN)__name__ __module__ __qualname____doc__rrrrrrrrs   rc@s>eZdZdZ      dddZd d Zd d Zd dZdS)RepeatAugSampleraSampler that restricts data loading to a subset of the dataset for distributed, with repeated augmentation. It ensures that different each augmented version of a sample will be visible to a different process (GPU). Heavily based on torch.utils.data.DistributedSampler This sampler was taken from https://github.com/facebookresearch/deit/blob/0c4b8f60/samplers.py Used in Copyright (c) 2015-present, Facebook, Inc. NTrcCs|durts tdt}|dur tstdt}||_||_||_||_||_ d|_ t t t|j||j|_|j|j|_|pL|}|rbt t t|j||||_dSt t t|j||_dS)Nrr)rrrrr r r r shuffle num_repeatsepochr rrrrrfloornum_selected_samples)rr r r r(r)selected_roundselected_ratiorrrrAs,   zRepeatAugSampler.__init__cs t}|jjrtjtj|d}n tjdtjd}t j t rIj sIt j tj}|tfddt|D}n tj|tj dd}|}jt|}|dkrk||d|7}t|jkstJ|jjj}t|jksJt|djS)N) generatorr)startendcsg|] }t|jqSr)r r)).0irrr rsz-RepeatAugSampler.__iter__..)repeatsdim)torch Generator manual_seedr*r(randpermrr arange isinstancer)float is_integerrrtensorrrepeat_interleaver tolistrr r rrr,)rgr repeat_size padding_sizerrrres" "zRepeatAugSampler.__iter__cCrr)r,rrrrrr zRepeatAugSampler.__len__cCs ||_dSr)r*)rr*rrr set_epochs zRepeatAugSampler.set_epoch)NNTr&r'r)r!r"r#r$rrrrErrrrr%6s  $ r%) rr7torch.utils.datartorch.distributed distributedrrr%rrrrs   /