o ۷iE @sUddlZddlZddlZddlZddlZddlmZddlmZddl m Z ddl Z ddl m ZddlmZddlmZmZddlmZdd lmZdd lmZdd lmZd d lmZd dlmZddlmZe dee!Z"e Z#Gddde jj$j%Z&     d+ddZ' d,ddZ(     d+ddZ)GdddZ*da+e*e,d <d!ej-e.Bd"ej-e.Bd#eee/e0fed$ffd%d&Z1d'e.d(e.d#eee/e0fed$ffd)d*Z2dS)-N)Image)tqdm)linalg)adaptive_avg_pool2d)TupleUnion) InceptionV3)_IMAGE_EXTENSIONS)_VIDEO_EXTENSIONS)get_metrics_verbose)current_platform) disable_print) init_loggerignorec@s&eZdZdddZddZddZdS) ImagePathDatasetNcCs||_||_dSN) files_or_imgs transforms)selfrrrK/home/ubuntu/vllm_env/lib/python3.10/site-packages/cache_dit/metrics/fid.py__init__ s zImagePathDataset.__init__cCs t|jSr)lenr)rrrr__len__$s zImagePathDataset.__len__cCsr|j|}t|ttjfrt|d}nt|tj r)t |t j }t |}ntd|jdur7||}|S)NRGBz3file_or_img must be a file path or an OpenCV image.)r isinstancestrpathlibPathropenconvertnpndarraycv2cvtColor COLOR_BGR2RGB fromarray ValueErrorr)ri file_or_imgimgrrr __getitem__'s     zImagePathDataset.__getitem__r)__name__ __module__ __qualname__rrr-rrrrrs  r2cpuTc Cs||t|krtdt|}t|td}tjj j ||dd|d}t t||f} d} t ||dD]S} | |} t|| d} Wdn1sTwY| dd ksg| d d krmt| d d } | d d} | | | | | jd<| | jd} q8| S) aCalculates the activations of the pool_3 layer for all images. Params: -- files_or_imgs : List of image files paths or OpenCV image -- model : Instance of inception model -- batch_size : Batch size of images for the model to process at once. Make sure that the number of samples is a multiple of the batch size, otherwise some samples are ignored. This behavior is retained to match the original FID score implementation. -- dims : Dimensionality of features returned by Inception -- device : Device to run calculations -- num_workers : Number of parallel dataloader workers Returns: -- A numpy array of dimension (num images, dims) that contains the activations of the given tensor when feeding inception with the query tensor. zQWarning: batch size is bigger than the data size. Setting batch size to data size)rF) batch_sizeshuffle drop_last num_workersr)disableNr r)rr) output_size)evalrloggerinforTFToTensortorchutilsdata DataLoaderr#emptyrtono_gradsizersqueezer3numpyshape) rmodelr4dimsdevicer7 disable_tqdmdataset dataloaderpred_arr start_idxbatchpredrrrget_activations6s6    rUư>c Cs2t|}t|}t|}t|}|j|jksJd|j|jks(Jd||}tj||dd\}}t|s]d|}t |t |jd|} t|| || }t |rtj t |jddds~tt|j} td | |j}t|} ||t|t|d | S) a9Numpy implementation of the Frechet Distance. The Frechet distance between two multivariate Gaussians X_1 ~ N(mu_1, C_1) and X_2 ~ N(mu_2, C_2) is d^2 = ||mu_1 - mu_2||^2 + Tr(C_1 + C_2 - 2*sqrt(C_1*C_2)). Stable version by Dougal J. Sutherland. Params: -- mu1 : Numpy array containing the activations of a layer of the inception net (like returned by the function 'get_predictions') for generated samples. -- mu2 : The sample mean over activations, precalculated on an representative data set. -- sigma1: The covariance matrix over activations for generated samples. -- sigma2: The covariance matrix over activations, precalculated on an representative data set. Returns: -- : The Frechet Distance. z5Training and test mean vectors have different lengthsz7Training and test covariances have different dimensionsF)dispzQfid calculation produces singular product; adding %s to diagonal of cov estimatesrgMbP?)atolzImaginary component {}r )r# atleast_1d atleast_2drJrsqrtmdotisfiniteallprinteye iscomplexobjallclosediagonalimagmaxabsr)formatrealtrace) mu1sigma1mu2sigma2epsdiffcovmean_msgoffsetm tr_covmeanrrrcalculate_frechet_distance{s,      &rvc Cs8t|||||||}tj|dd}tj|dd} || fS)aCalculation of the statistics used by the FID. Params: -- files_or_imgs : List of image files paths or OpenCV image -- model : Instance of inception model -- batch_size : Batch size of images for the model to process at once. Make sure that the number of samples is a multiple of the batch size, otherwise some samples are ignored. This behavior is retained to match the original FID score implementation. -- dims : Dimensionality of features returned by Inception -- device : Device to run calculations -- num_workers : Number of parallel dataloader workers Returns: -- mu : The mean over samples of the activations of the pool_3 layer of the inception model. -- sigma : The covariance matrix of the activations of the pool_3 layer of the inception model. r)axisF)rowvar)rUr#meancov) rrKr4rLrMr7rNactmusigmarrrcalculate_activation_statisticss r~c @seZdZer ejndddddfdedededefd d Zd e j e Bd e j e Bfd dZ de de fddZ de de fddZdS)FrechetInceptionDistancer3r2rTrLr7r4rNcCsR||_||_||_||_||_tj|j|_t|jg|j|_ |j |_ dSr) rLrMr7r4rNr BLOCK_INDEX_BY_DIM block_idxrErKr;)rrMrLr7r4rNrrrrs z!FrechetInceptionDistance.__init__ image_true image_testcs2t|ts t|trtj|stj|rCtj|sJtj|s&J|ddtvs1J|ddtvsz8FrechetInceptionDistance.compute_fid..crrrr)image_test_dirrrrrcSg|]}|qSras_posixrrrrrrcSrrrrrrrrrrzONo valid Image pairs, please note that Image pairs must have the same basename.NNzimage_true_files: zimage_test_files: )rrospathisfileexistssplitr isdirrr sortedrangeminrbasenameappendcopyr<errordebugr4r~rKrLrMr7rNrv)rrrimage_true_filesimage_test_filesimage_true_files_selectedimage_test_files_selectedr*selected_image_trueselected_image_testr4m1s1m2s2 fid_valuer)rrr compute_fids            z$FrechetInceptionDistance.compute_fid video_true video_testcs"tj|rtj|r|j||d\}}}ntj|rtj|rt|tfddtD}t|tfddtD}dd|D}dd|D}g}g} t t t |t |D] } || } || } tj | tj | kr| | | | q`|}| }t |dkrtdd Std |td |g}g}d}t||D]\} }|j| |d\}}}||||||7}qntd |dkrtd d St d|j}t||j||j|j|j|j\}}t||j||j|j|j|j\}}t||||}||fS)Nrrcrrrr)video_true_dirrrrhrz>FrechetInceptionDistance.compute_video_fid..crrrr)video_test_dirrrrprcSrrrrrrrrvrcSrrrrrrrrwrrzONo valid Video pairs, please note that Video pairs must have the same basename.rzvideo_true_files: zvideo_test_files: z0video_true and video_test must be files or dirs.zNo valid frames to comparer)rrr_fetch_video_framesrrr rr rrrrrrr<rrzipextendr)r4r~rKrLrMr7rNrv)rrrvideo_true_framesvideo_test_frames valid_framesvideo_true_filesvideo_test_filesvideo_true_files_selectedvideo_test_files_selectedr*selected_video_trueselected_video_test video_true_ video_test_video_true_frames_video_test_frames_ valid_frames_r4rrrrrr)rrrcompute_video_fidYs              z*FrechetInceptionDistance.compute_video_fidc Cst|}t|}|r|stdggdfStt|tjt|tj}d}g}g}t d| | \} } | \} } | rL| sMn| | | | |d7}q<| | |dkrmggdfS|||fS)NzCould not open video filesrzTotal frames: Tr) r% VideoCaptureisOpenedr<rrintgetCAP_PROP_FRAME_COUNTrreadrrelease) rrrcap1cap2 frame_countrrrret1frame1ret2frame2rrrrs6           z,FrechetInceptionDistance._fetch_video_framesN)r.r/r0ris_accelerator_available device_typerboolrr#r$rrrrrrrrrs<  b er fid_instancerrreturnrcCTtdurttt daWdn1swYtdus$Jt||SN)rNrrrr r)rrrrrrs  rrrcCrrrrrrrrs  r)r1r2r3rT)rV)3rr%rwarningsrIr#PILrrscipyrr@torchvision.transformsrr>torch.nn.functionalrtypingrr inceptionr configr r r platformsrrArcache_dit.loggerrfilterwarningsr.r<DISABLE_VERBOSErBDatasetrrUrvr~rr__annotations__r$rfloatrrrrrrrsn               J @ *