o xi5u @sddlZddlmZddlmZddlZddlmZddlm Z m Z m Z m Z m Z eeZidgdgd ejddddgd d gd gd ejejddddgd dgdgdejddgd dgdgdejejddgd dgdgdejejejddgd dgdgdejddgd dgdgdejddgd dgdgdejejejgd dgd d!d"gejejgd d#gd$d%d&gddgd d'gd$d%d&gddgd d(gd$d%d&gddgd d)d*d+gd%gdgd d,d*d+gd%gdgd d-gd$d%d&gddgd d.gd$d%d&gddgd d/gd$d%d&gddgd id0gd$d%d&gddgd d1gd$d%d&gddgd d2gd$d%d&gddgd d3gd$d%d&gddgd d4gd$d%d&gddgd d5gdgdejddgd d6gdgdejddgd d7gdgdejddgd d8gdgdejddgd d9gd:gd;gd<d=d>d?gd@d!dAgejdgd=d>dBgdCgdDgd<dEd>dFgdCgdDgd<dEd>dGgdCgdDgd<dEd>dHgd$d%d&gddgd dIgdJdKdLgddgd dMd*dNgd%gdgd=d>idOgdgdejddgd dPgdQgdRejejddgd dSgdgdejddgd dTdUd+gd!gejgd dVgd$d%d&gddgd dWgdXgdejddgd dYgdgdejddgd dZgd[gd\dddejdgd d]gd$d%d&gddgd d^gd_d!d&gejdgd d`gdgdejddgd dagdJdKdLgddgd dbgd:gd;d=dcddgd:gdedfdggdhd%digddgd djgdkgdlddejddgd dmgdngdoddejejddgd gd$d%d&gddgd gdpgdqddejgd gdrgdsgdtd gdugdvddddejgd dwZGdxdydyZdS)zN)Sequence)Any)Response)chunkifydecode_sdxl_t2i_latentsget_updated_kwargspostprocess_np_arrays_for_videopostprocess_pils_to_npBlipDiffusionPipeline)Reference-ImagePromptNegative-PromptSource-Subject-CategoryTarget-Subject-CategoryGenerated-Image)reference_imageprompt neg_promptsource_subject_categorytarget_subject_category) table-schema kwarg-logging kwarg-actionsBlipDiffusionControlNetPipeline)r Control-Imager rrrr)rcondtioning_imagerrrr!StableDiffusionControlNetPipeline)rr rr)imagernegative_prompt(StableDiffusionControlNetImg2ImgPipeline) Source-Imagerr rr)r control_imagerr(StableDiffusionControlNetInpaintPipeline)r! Mask-Imagerr rr)r mask_imager"rrCycleDiffusionPipeline)r!r z Source-Promptr)rr source_prompt&StableDiffusionInstructPix2PixPipeline)r!r rrPaintByExamplePipeline)r!z Example-Image Mask-Promptr)r example_imager%RePaintPipeline)r!r*rrr%StableDiffusionPipeline)r rrrrKandinskyCombinedPipelineKandinskyV22CombinedPipelineLatentConsistencyModelPipeliner rLDMTextToImagePipelineStableDiffusionPanoramaPipelinePixArtAlphaPipelineStableDiffusionSAGPipelineSemanticStableDiffusionPipelineWuerstchenCombinedPipeline IFPipelineAltDiffusionPipeline&StableDiffusionAttendAndExcitePipeline KandinskyImg2ImgCombinedPipeline KandinskyInpaintCombinedPipeline#KandinskyV22Img2ImgCombinedPipeline#KandinskyV22InpaintCombinedPipelineAnimateDiffPipeline)r rzNumber-of-FramesGenerated-Video)rr num_frames)NNNvideo)rrr output-typeStableVideoDiffusionPipeline) Input-ImagezFrames-Per-Secondr?fpsAudioLDMPipeline)r rzAudio-Length-in-SecondszGenerated-Audio)rraudio_length_in_saudioAudioLDM2PipelineMusicLDMPipeline"StableDiffusionPix2PixZeroPipeline PNDMPipeline)z Batch-SizezNumber-of-Inference-Stepsr batch_sizenum_inference_steps ShapEPipeliner?StableDiffusionImg2ImgPipelineStableDiffusionInpaintPipeline)r!r$r rr)rr%rr StableDiffusionDepth2ImgPipeline%StableDiffusionImageVariationPipeliner!StableDiffusionPipelineSafeStableDiffusionUpscalePipeline)r!r rzUpscaled-ImageStableDiffusionAdapterPipelineStableDiffusionGLIGENPipeline)r zGLIGEN-Phrasesz GLIGEN-BoxeszGLIGEN-Inpaint-Imagerr)rgligen_phrases gligen_boxesgligen_inpaint_imager%VersatileDiffusionTextToImagePipeline(VersatileDiffusionImageVariationPipeline)r!rr$VersatileDiffusionDualGuidedPipeline LDMPipelineTextToVideoSDPipeline)rrrBTextToVideoZeroPipeline)rr video_length)rrAmusedPipeline)r zGuidance Scalerguidance_scale#StableDiffusionXLControlNetPipeline)Prompt-1Prompt-2rNegative-Prompt-1Negative-Prompt-2r)rprompt_2rrnegative_prompt_2*StableDiffusionXLControlNetImg2ImgPipeline)rerfrDrrgrhr)rrirr"rrj)r rrDr)rrr)r rrfrhr)rrrirj)NNNN)r rrfrhrDr)rrrirjr)Kandinsky3PipelineKandinsky3Img2ImgPipelineStableDiffusionXLPipeline StableDiffusionXLImg2ImgPipelinec @seZdZdZdededdfddZdeed e eefd e d e d e def d dZ d e de fddZdede deddfddZdede deddfddZded e d e eefde eeffddZdS)#DiffusersMultiModalPipelineResolvera Resolver for request and responses from [HuggingFace Diffusers](https://huggingface.co/docs/diffusers/index) multi-modal Diffusion Pipelines, providing necessary data transformations, formatting, and logging. This resolver is internally involved in the `__call__` for `wandb.integration.diffusers.pipeline_resolver.DiffusersPipelineResolver`. This is based on `wandb.sdk.integration_utils.auto_logging.RequestResponseResolver`. Args: pipeline_name: (str) The name of the Diffusion Pipeline. pipeline_namepipeline_call_countreturnNcCsF||_||_g}|tvr|t|d7}ntdtj|d|_dS)Nrz"Pipeline not supported for logging)columns)rqrrSUPPORTED_MULTIMODAL_PIPELINESwandbErrorTable wandb_table)selfrqrrrtr{d/home/ubuntu/.local/lib/python3.10/site-packages/wandb/integration/diffusers/resolvers/multimodal.py__init__]s z,DiffusersMultiModalPipelineResolver.__init__argskwargsresponse start_time time_elapsedc CszU|d|dd}}t|||}t|j}|j|d<dtjvr3tjd||d|jdgintjj}|||d|jdg} tjjd| idd ||||WSt ym} z t | WYd} ~ dSd} ~ ww) a1Main call method for the `DiffusersPipelineResolver` class. Args: args: (Sequence[Any]) List of arguments. kwargs: (Dict[str, Any]) Dictionary of keyword arguments. response: (wandb.sdk.integration_utils.auto_logging.Response) The response from the request. start_time: (float) Time when request started. time_elapsed: (float) Time elapsed for the request. Returns: Packed data as a dictionary for logging to wandb, None if an exception occurred. rrNz pipeline-nameworkflowzPipeline-Call-)pipelineparamsstageT)allow_val_change) rdictconfigrqrvupdaterrrprepare_loggable_dict Exceptionloggerwarning) rzr~rrrrrpipeline_configsexisting_workflowupdated_workflower{r{r|__call__gs>       z,DiffusersMultiModalPipelineResolver.__call__cCsXdt|jvr |jSt|jddkr|jdvr|jS|jSt|jddkr*|jSdS)a3Unpack the generated images, audio, video, etc. from the Diffusion Pipeline's response. Args: response: (wandb.sdk.integration_utils.auto_logging.Response) The response from the request. Returns: List of generated images, audio, video, etc. rBrA)rOrHN)rurqimagesframesaudios)rzrr{r{r|get_output_imagess    z5DiffusersMultiModalPipelineResolver.get_output_imagesrloggable_kwarg_chunksidxcCsdt|jvrcz>d}|jdvr4t|jdd}t|jdd}d|||d|||}nt|jdd}|||}Wn tyPd }Ynwtd |jtj||d id St|jdd krzt|jdd}|||}Wn tyd }Ynwtd |jtjt |d|did St|jddkrzt|jdd}|||}Wn tyd }Ynwtd|jtj |d|did Sd S)afLog the generated images, audio, video, etc. from the Diffusion Pipeline's response along with an optional caption to a media panel in the run. Args: image: (Any) The generated images, audio, video, etc. from the Diffusion Pipeline's response. loggable_kwarg_chunks: (List) Loggable chunks of kwargs. rB)rnrorrriz Prompt-1: z Prompt-2: NzGenerated-Image/Pipeline-Call-)captionrAGenerated-Video/Pipeline-Call-rErrHzGenerated-Audio/Pipeline-Call->) sample_rater) rurqindex ValueErrorrvlogrrImageVideor Audio)rzrrrr prompt_index prompt2_indexr{r{r| log_medias "       z-DiffusersMultiModalPipelineResolver.log_mediacCsg}t|jd}t|D]%\}}||dur'|||dur#||ndq |||||q dt|jvrC|t|n)t|jddkrY|tjt|ddnt|jddkrl|tj|d d |j j |dS) a!Populate the row of the `wandb.Table`. Args: image: (Any) The generated images, audio, video, etc. from the Diffusion Pipeline's response. loggable_kwarg_chunks: (List) Loggable chunks of kwargs. idx: (int) Chunk index. rNrrBrArrErHr)r) rurq enumerateappendrvrrr rryadd_data)rzrrr table_row kwarg_actions column_idxloggable_kwarg_chunkr{r{r|add_data_to_table s.     z5DiffusersMultiModalPipelineResolver.add_data_to_tablerc s`||}|jdkrddkrt||j}|jdvr]t||jdkd}td|jtj|dd d it |jd fd d t t D}| tj|dd|j j|nJt |jd g}D]}| t|trw|n|gqht|t |d}t t |dD]} || D]} || || || || qqd|j|j iS)aPrepare the loggable dictionary, which is the packed data as a dictionary for logging to wandb, None if an exception occurred. Args: pipeline: (Any) The Diffusion Pipeline. response: (wandb.sdk.integration_utils.auto_logging.Response) The response from the request. kwargs: (Dict[str, Any]) Dictionary of keyword arguments. Returns: Packed data as a dictionary for logging to wandb, None if an exception occurred. rn output_typelatent)r_r`r`) normalizerrrrrcsg|]}|qSr{r{).0rrloggable_kwarg_idsr{r| Vs zMDiffusersMultiModalPipelineResolver.prepare_loggable_dict..rrzResult-Table/Pipeline-Call-)rrqrrr rvrrrrrurangelenrryr isinstancelistrrr) rzrrrrrArrloggable_kwarg_idrrr{rr|r1sP            z9DiffusersMultiModalPipelineResolver.prepare_loggable_dict)__name__ __module__ __qualname____doc__strintr}rrrrfloatrrrrrrr{r{r{r|rpRsH    ?J '  rp)loggingcollections.abcrtypingrrv(wandb.sdk.integration_utils.auto_loggingrutilsrrrr r getLoggerrrrrurpr{r{r{r|s     ' 1<N \ jx  #(-27<AF P Z d ny%.7@ JU _ir | "+ 5>HQ]o   C