o پi?O@s|dZddlZddlZddlmZddlmZddlmZddl m Z m Z m Z m Z ddlZddlZddlZddlmZmZdd lmZmZdd lmZmZdd lmZdd lmZdd lm Z e!e"Z#ddZ$ddZ%GdddZ&dMddZ'dddddddddddddddddddeeddddddddej(e)ddddddf$de eefd e e*e e*e*fe e*e*e*ffd!e*d"e+d#e+d$e,d%e-d&e*d'e+d(e e-d)e e e,e,fd*e e e,e,fd+e,d,e,d-e,d.e e,d/e,d0e,d1e e-d2e*d3e*d4e-d5e e,d6fd7e e,d6fd8e*d9e+d:e e,d;e e-de+d?e+d@ej.dAej)dBe+dCe+dDe+dEe-dFe+fNdGdHZ/GdIdJdJej0j1j2Z3GdKdLdLe4Z5dS)Na Loader Factory, Fast Collate, CUDA Prefetcher Prefetcher and Fast Collate inspired by NVIDIA APEX example at https://github.com/NVIDIA/apex/commit/d5e2bb4bdeedd27b1dfaf5bb2b24d6c000dee9be#diff-cf86c282ff7fba81fad27a559379d5bf Hacked together by / Copyright 2019, Ross Wightman N)suppress)partial)repeat)CallableOptionalTupleUnion)IMAGENET_DEFAULT_MEANIMAGENET_DEFAULT_STD)IterableImageDataset ImageDataset)OrderedDistributedSamplerRepeatAugSampler) RandomErasing)FastCollateMixup)create_transformc Cs`t|dts Jt|}t|ddtrt|ddtj}t|dd}||}tj|tjd}tj|g|dddjRtj d}t |D]K}t||d|ksYJt |D]8}||d||||<|r||||t ||d|7<q]||||||d|7<q]qK||fSt|ddtjrtj dd|Dtjd}t||ksJtj|g|ddjRtj d}t |D]}||t ||d7<q||fSt|ddtj r.tj dd|Dtjd}t||ksJtj|g|ddjRtj d}t |D]}||||dq||fSJ)zd A fast collation function optimized for uint8 images (np array or torch) and int64 targets (labels)r)dtyper cSg|]}|dqSr .0brrD/home/ubuntu/.local/lib/python3.10/site-packages/timm/data/loader.py 3z fast_collate..cSrrrrrrrr:r) isinstancetuplelennpndarraytorchzerosint64shapeuint8range from_numpytensorTensorcopy_) batch batch_sizeis_npinner_tuple_sizeflattened_batch_sizetargetsr)ijrrr fast_collates@&  ,&"  " r4cCsrt|ttfstt||}|St||kr-t|}|f|}t d|d|St||ks7Jd|S)Nz@Pretrained mean/std different shape than model, using avg value .z-normalization stats must match image channels) rrlistrrr meanitem_loggerwarning)xnx_meanrrr adapt_to_chsDs  r>c@seZdZeededddddddf d ejjj d e e d fd e e d fd e dejde ejdede dede de fddZddZddZeddZeddZedd Zejd!d ZdS)"PrefetchLoadercudaNFconstr rloaderr7.stdchannelsdevice img_dtypefp16re_probre_modere_count re_num_splitsc Cst||}t||}d|ddf} ||_||_|rtj}|ptj|_tjdd|D||d| |_ tjdd|D||d| |_ |dkrRt || | | |d|_ nd|_ |j dko^tj|_|j d koitj|_dS) Nr cSg|]}|dqSrrr;rrrrkrz+PrefetchLoader.__init__..)rGrcSrNrOrrQrrrrmrrB) probabilitymode max_count num_splitsrGrAnpu)r>rDrGr"float16float32rHr)viewr7rErrandom_erasingtyperA is_availableis_cudarVis_npu) selfrDr7rErFrGrHrIrJrKrLrMnormalization_shaperrr__init__Rs:     zPrefetchLoader.__init__c csPd}|jrtjj|jd}ttjj|d}n|jr+tjj|jd}ttjj|d}nd}t }|j D]n\}}|0|j |jdd}|j |jdd}| |j  |j|j}|jdurb||}Wdn1slwY|sy||fVnd}|dur|jrtjj|jd|n|jrtjj|jd||}|}q2||fVdS)NT)rG)stream)rG non_blockingF)r]r"rAStreamrGrrbr^rVrrDtorHsub_r7div_rErZcurrent_stream wait_stream)r_firstrbstream_context next_input next_targetinputtargetrrr__iter__{s<   zPrefetchLoader.__iter__cCs t|jSN)rrDr_rrr__len__ zPrefetchLoader.__len__cC|jjSrq)rDsamplerrrrrrrvzPrefetchLoader.samplercCrurq)rDdatasetrrrrrrxrwzPrefetchLoader.datasetcCst|jjtr |jjjSdS)NFrrD collate_fnr mixup_enabledrrrrrr{s zPrefetchLoader.mixup_enabledcCs t|jjtr||jj_dSdSrqry)r_r;rrrr{s)__name__ __module__ __qualname__r r r"rGutilsdata DataLoaderrfloatintrrboolstrrarprspropertyrvrxr{setterrrrrr?PsX      )%   r?allcCstjj}|j|ks Jt|tr*||}t|t |t j|ddS|dvs0J|dkr?t j|jddSdS)Nl)rpartr) r"rrget_worker_infoidrrrandomseed manual_seedr ) worker_idworker_seeding worker_inforrrr _worker_inits     rFrBrCg?g?bilinearrATrx input_sizer- is_trainingno_augrJrKrLre_splittrain_crop_modescaleratiohflipvflip color_jittercolor_jitter_probgrayscale_probgaussian_blur_prob auto_augmentnum_aug_repeatsnum_aug_splits interpolationr7.rE num_workers distributedcrop_pct crop_modecrop_border_pixelsrz pin_memoryrIrHrGuse_prefetcheruse_multi_epochs_loaderpersistent_workersrtf_preprocessingc'. Cs"d}'|r|pd}'t|fid|d|d| d| d| d| d | d |d |d |d |d|d|d|d|d|d|d|d|d|d|d|'d|&d|"d|dk|_t|trg|j|dd}(|rt|tjjjs|r|r~t ||d}(ntjjj |}(n t |}(n|dksJd|dur|"rt ntjjjj}tjjj})|#rt})t|t|tjjj o|(duo|||(|||tt|%d |$d! }*z |)|fi|*}+Wnty},z|*d"|)|fi|*}+WYd},~,nd},~,ww|"r|r|s|nd#}-t|+|||d|!|| |-|||'d$ }+|+S)%a Args: dataset: The image dataset to load. input_size: Target input size (channels, height, width) tuple or size scalar. batch_size: Number of samples in a batch. is_training: Return training (random) transforms. no_aug: Disable augmentation for training (useful for debug). re_prob: Random erasing probability. re_mode: Random erasing fill mode. re_count: Number of random erasing regions. re_split: Control split of random erasing across batch size. scale: Random resize scale range (crop area, < 1.0 => zoom in). ratio: Random aspect ratio range (crop ratio for RRC, ratio adjustment factor for RKR). hflip: Horizontal flip probability. vflip: Vertical flip probability. color_jitter: Random color jitter component factors (brightness, contrast, saturation, hue). Scalar is applied as (scalar,) * 3 (no hue). color_jitter_prob: Apply color jitter with this probability if not None (for SimlCLR-like aug grayscale_prob: Probability of converting image to grayscale (for SimCLR-like aug). gaussian_blur_prob: Probability of applying gaussian blur (for SimCLR-like aug). auto_augment: Auto augment configuration string (see auto_augment.py). num_aug_repeats: Enable special sampler to repeat same augmentation across distributed GPUs. num_aug_splits: Enable mode where augmentations can be split across the batch. interpolation: Image interpolation mode. mean: Image normalization mean. std: Image normalization standard deviation. num_workers: Num worker processes per DataLoader. distributed: Enable dataloading for distributed training. crop_pct: Inference crop percentage (output size / resize size). crop_mode: Inference crop mode. One of ['squash', 'border', 'center']. Defaults to 'center' when None. crop_border_pixels: Inference crop border of specified # pixels around edge of original image. collate_fn: Override default collate_fn. pin_memory: Pin memory for device transfer. fp16: Deprecated argument for half-precision input dtype. Use img_dtype. img_dtype: Data type for input image. device: Device to transfer inputs and targets to. use_prefetcher: Use efficient pre-fetcher to load samples onto device. use_multi_epochs_loader: persistent_workers: Enable persistent worker processes. worker_seeding: Control worker random seeding at init. tf_preprocessing: Use TF 1.0 inference preprocessing for testing model ports. Returns: DataLoader rrrrrrrrrrrrrrr7rErrrrJrKrLrMrrseparate)rN) num_repeatszORepeatAugment not currently supported in non-distributed or IterableDataset use)r) r-shufflerrvrzr drop_lastworker_init_fnrrrB) r7rErFrGrIrHrJrKrLrM)r transformrr set_loader_cfgr"rrIterableDatasetrrDistributedSamplerrr4 dataloaderdefault_collaterMultiEpochsDataLoaderdictrr TypeErrorpopr?).rxrr-rrrJrKrLrrrrrrrrrrrrrrr7rErrrrrrzrrIrHrGrrrrrrMrv loader_class loader_argsrDeprefetch_re_probrrr create_loadersW            rcs,eZdZfddZddZddZZS)rcsRtj|i|d|_|jdurt|j|_nt|j|_d|_t|_dS)NFT)superra_DataLoader__initialized batch_sampler_RepeatSamplerrvrpiterator)r_argskwargs __class__rrras  zMultiEpochsDataLoader.__init__cCs |jdur t|jSt|jjSrq)rrrvrrrrrrss zMultiEpochsDataLoader.__len__ccs$tt|D]}t|jVqdSrq)r'rnextr)r_r2rrrrpszMultiEpochsDataLoader.__iter__)r|r}r~rarsrp __classcell__rrrrr}s  rc@s eZdZdZddZddZdS)rzH Sampler that repeats forever. Args: sampler (Sampler) cCs ||_dSrq)rv)r_rvrrrrartz_RepeatSampler.__init__ccs t|jEdHqrq)iterrvrrrrrrpsz_RepeatSampler.__iter__N)r|r}r~__doc__rarprrrrrs r)r)6rloggingr contextlibr functoolsr itertoolsrtypingrrrrr"torch.utils.datanumpyr constantsr r rxr r distributed_samplerrrrZrmixuprtransforms_factoryr getLoggerr|r9r4r>r?rrXrGrrrrrrrrrrobjectrrrrrs       ' h         !"#$%&' 6