o ॵiR+@sddlZddlZddlZddlZddlmZddlZddlmZddl m Z ddl m Z ddl m Z mZmZmZmZ   dddZ    dd d Zd d ZddZdddZdddZddZdS)N)Mapping) distributed)tqdm) to_device) broadcast get_dist_infois_dist is_master make_tmp_dirc Cs^|j}d}|dur(zt|}Wnty$} z t| tdd} ~ wwd} nd}|}d} t|| do} t|D]U\} } t| |} t || |||rNd}n+t | t rud | vr\| d }nz tt t | }Wntyt|j}Ynwt| }t|D]}| q}|r| d|krn q9Wdt|SWdt|S1swYt|S) aBTest model in EpochBasedTrainer with a single gpu. Args: trainer (modelscope.trainers.EpochBasedTrainer): Trainer to be tested. data_loader (nn.Dataloader): Pytorch data loader. device (str | torch.device): The target device for the data. metric_classes (List): List of Metric class that uses to collect metrics. vis_closure (Callable): Collect data for TensorboardHook. data_loader_iters (int): Used when dataset has no attribute __len__ or only load part of dataset. Returns: list: The prediction results. FNzVPlease implement ``__len__`` method for your dataset, or provide ``data_loader_iters``zTotal test samplesTzTest iterationstotaldesc nsentences)datasetlen Exceptionloggingerror ValueErrorr enumeraterevaluate_batch isinstancernextitervalues batch_sizerangeupdateget_metric_values)trainer data_loaderdevicemetric_classes vis_closuredata_loader_itersrprogress_with_itersdata_lener pbaridatar_r-W/home/ubuntu/.local/lib/python3.10/site-packages/modelscope/trainers/utils/inference.pysingle_gpu_testsX           r/Fc Cs|j}t|j\} } d} |dur1zt|} | } Wnty-}z t|tdd}~wwd}n d} d} || } d}d}t| |d}t |D]\}}t ||}t ||||t |t rqd |vrf|d }nttt|}nt|}|| | d krt|g|jj} tj| tjjd | } n|| } | r| }n| }||7}| dkr|| kr| ||}t|D]}|q| r|d | krnqHWdn1swY|rt||j}n|durt}t||t j!"|d }t#|}t$|S) aTest model in EpochBasedTrainer with multiple gpus. This method tests model with multiple gpus and collects the results under two different modes: gpu and cpu modes. By setting ``gpu_collect=True``, it encodes results to gpu tensors and use gpu communication for results collection. On cpu mode it saves the results on different gpus to ``tmpdir`` and collects them by the rank 0 worker. Args: trainer (modelscope.trainers.EpochBasedTrainer): Trainer to be tested. data_loader (nn.Dataloader): Pytorch data loader. device: (str | torch.device): The target device for the data. tmpdir (str): Path of directory to save the temporary results from different gpus under cpu mode. gpu_collect (bool): Option to use either gpu or cpu to collect results. data_loader_iters_per_gpu (int): Used when dataset has no attribute __len__ or only load part of dataset. Returns: list: The prediction results. FNz^Please implement ``__len__`` method for your dataset, or provide ``data_loader_iters_per_gpu``z"Total test samples with multi gpusrTz%Total test iterations with multi gpusr rr)opmetrics)%rrdp_grouprrrrrrrrrrrrrrtorch LongTensortomodelr"dist all_reduce reduce_opSUMitemrrcollect_results_gpur collect_results_cpuospathjoin merge_metricsr)r r!r"r#r$tmpdir gpu_collectdata_loader_iters_per_gpurrank world_sizer&r' total_samplesr(r countr)r*r+r iter_cnt_allr,metric_classes_listr-r-r.multi_gpu_testPsx          $rKcCs@||}|dur|D]}|||q |dur||dSdSN)evaluation_stepadd)r r+r#r$ batch_result metric_clsr-r-r.rs  rcCs6i}tr|D] }||qtrt|d}|S)Nr)r revaluaterr)r# metric_valuesrPr-r-r.rs rc Cs4t|j\}}|durt}tj|stj|ddt| r(t |j rU| r1t |j rUttj|d|dd}t||Wdn1sPwYtt s^dSg}t|D].}tj|d|d}t|d }t|} Wdn1swY| r|| qdt||S)aBCollect results under cpu mode. On cpu mode, this function will save the results on different gpus to ``tmpdir`` and collect them by the rank 0 worker. Args: result_part (list): Result list containing result parts to be collected. trainer(`EpochBasedTrainer`): The trainer instance to get the parallel groups. tmpdir (str | None): temporal directory for collected results to store. If set to None, it will create a random temporal directory for it. Returns: list: The collected results. NT)exist_okpart_z.pklwbrb)rr2r r>r?existsmakedirsr7barrieris_tp_group_availabler tp_groupis_pp_group_availablepp_groupopenr@pickledumprloadappendshutilrmtree) result_partr rBrErFf part_listr* part_file part_resultr-r-r.r=s8      r=c st|\}}tjtt|tjddtjjddfddt|D}t ||t| tj tjdd}|dd<fddt|D}t |||t rg}t||D]\}} t|d| d} | r|| qc|SdS) aCollect results under gpu mode. On gpu mode, this function will encode results to gpu tensors and use gpu communication for results collection. Args: result_part (list): Result list containing result parts to be collected. dp_group(`ProcessGroup` or None): The data parallel group, default None for global group. Returns: list: The collected results. cuda)dtyper")r"csg|]}qSr-)clone.0r,) shape_tensorr-r. sz'collect_results_gpu..Nrcsg|]}qSr-) new_zerosrm) part_tensor shape_maxr-r.rps )rr3tensor bytearrayr_dumpsuint8shaperr7 all_gathermaxzerosr ziploadscpunumpytobytesrb) rer2r,rF shape_list part_sendpart_recv_listrgrecvrxrir-)rrrsror.r<s.  " r<cCsH|durdS|d}|ddD]}t||D] \}}||qq|S)Nrr)r|merge)rJmetric_classes_0metric_classes_icls_0cls_ir-r-r.rA's rA)NNN)NNNFNrL)rr>r_rccollections.abcrr3rr7rmodelscope.utils.data_utilsrmodelscope.utils.torch_utilsrrrr r r/rKrrr=r<rAr-r-r-r.s2     A b 2 -