o }oi/@sZddlZddlZddlmZmZddlZddlZddlmZddlm Z GdddZ dS)N)DictList) AutoencoderKL)nnc@seZdZdZd&dededdfdd Zd d Zd'd ejfd dZ ddZ ddZ ddZ ddZ ddZdeededdfddZddZd(d d!Zedefd"d#Zedefd$d%ZdS)) VAEGeneratoras A class for generating and searching different Variational Autoencoder (VAE) configurations. This class provides functionality to generate various VAE architecture configurations given a specific input resolution and compression ratio. It allows searching through a design space to find configurations that match given parameter and memory budgets. input_resolutioncompression_ratioreturnNcCsD|dkr |dvs Jn|dkr|dvsJntd||_||_dS)Nr)r)r r z3Higher resolution than 2028 is not implemented yet!)NotImplementedError_input_resolution_compression_ratio)selfr r rZ/home/ubuntu/.local/lib/python3.10/site-packages/nemo/collections/diffusion/vae/autovae.py__init__"s zVAEGenerator.__init__cCs(tdd|j|j}|jtjdd}|S)z Generate a random input tensor with the specified input resolution. The tensor is placed on the GPU in half-precision (float16). cudadtypedevice)torchrandr tofloat16)r random_tensorrrr_generate_input-szVAEGenerator._generate_inputmodelcCs&|dusJdtdd|DS)z Count the number of trainable parameters in a given model. Args: model (nn.Module): The model for which to count parameters. Returns: int: The number of trainable parameters. Nz3Please provide a nn.Module to count the parameters.css|] }|jr|VqdSN) requires_gradnumel).0prrr Bsz1VAEGenerator._count_parameters..)sum parameters)rr"rrr_count_parameters7s zVAEGenerator._count_parameterscCs(ddddggddddddd d gd }|S) z Load a base configuration skeleton for the VAE. Returns: dict: A dictionary representing the base configuration JSON skeleton. rz 0.20.0.dev0z ../sdxl-vae/siluFrrrgy&1?) _class_name_diffusers_version _name_or_pathact_fnblock_out_channelsdown_block_types force_upcast in_channelslatent_channelslayers_per_blocknorm_num_groups out_channels sample_sizescaling_factorup_block_typesr)rskeletonrrr_load_base_json_skeletonDs"z%VAEGenerator._load_base_json_skeletoncs\t}fdd|D}ttj|}g}|D]}ddt||D}||q|S)a0 Generates all possible combinations from a search space dictionary. Args: attr (dict): A dictionary where each key has a list of possible values. Returns: List[Dict]: A list of dictionaries, each representing a unique combination of attributes. csg|]}|qSrr)r&keyattrrr isz;VAEGenerator._generate_all_combinations..cSsi|]\}}||qSrr)r&r?valuerrr nsz;VAEGenerator._generate_all_combinations..)listkeys itertoolsproductzipappend)rrArFchoicesall_combinationscombination_dicts combinationcombination_dictrr@r_generate_all_combinations^s  z'VAEGenerator._generate_all_combinationscCsH|}|d|d<|d|d<|d|d<|d|d<|d|d<|S)a Assign a chosen set of attributes to the base VAE configuration skeleton. Args: choice (dict): A dictionary of attributes to assign to the skeleton. Returns: dict: A dictionary representing the updated VAE configuration. r3r<r2r7r6)r>)rchoicesearch_space_skletonrrr_assign_attributesss     zVAEGenerator._assign_attributescCs`i}dgdg|d<dgdg|d<gdgdgdgd g|d <gd |d <gd |d<|S)z Define the search space for a 16x compression ratio at 1024 resolution. Returns: dict: A dictionary defining lists of possible attribute values. DownEncoderBlock2Dr3UpDecoderBlock2Dr<)rYrY)rWrXrYrYr)rWrXrYrr )@rWrXrYrYr2rrr7rrrZr6rrrArrr_search_space_16x1024s  z"VAEGenerator._search_space_16x1024cCsZi}dgdg|d<dgdg|d<gdgdgdg|d <gd |d <gd |d <|S)z Define the search space for an 8x compression ratio at 1024 resolution. Returns: dict: A dictionary defining lists of possible attribute values. rTr^r3rVr<)rWrXrYrY)rWrXrYr)rZrWrXrYr2r[r7r]r6rr_rrr_search_space_8x1024s  z!VAEGenerator._search_space_8x1024datamodecCs\|dkr|jdddd S|dkr|jdddd S|dkr*|jdddd Std ) a) Sort the list of design configurations in place based on a chosen mode. Args: data (List[Dict]): A list of dictionaries representing design configurations. mode (str): The sorting criterion. Can be 'abs_param_diff', 'abs_cuda_mem_diff', or 'mse'. abs_param_diffcS t|dS)N param_diffabsxrrr z2VAEGenerator._sort_data_in_place..)r?abs_cuda_mem_diffcSre)N cuda_mem_diffrgrirrrrkrlmsecSs|dd|dddS)Nrfr\rnrrirrrrkszGInvalid mode. Choose from 'abs_param_diff', 'abs_cuda_mem_diff', 'mse'.N)sort ValueError)rrbrcrrr_sort_data_in_placesz VAEGenerator._sort_data_in_placec Csd}t||D] \}}||d|7}qt|tdt||D]}|dd|d}||dd|d7}t|q#dS) a Print a formatted table of the design choices. Args: data (List[Dict]): The data to print, each entry a design configuration. headers (List[str]): Column headers. col_widths (List[int]): Widths for each column. <-rfrrnrN)rIprintr)) rrbheaders col_widths header_rowheaderwidthitemrowrrr _print_tables  zVAEGenerator._print_tablerc Cs|dkr |dkr tdg}|jdkr"|jdkr"|}||}n|jdkr5|jdkr5|}||}|}|jtj dd}g}|D]}d}d} | |} t dt |t | } | jtj dd} || } | d } tjtjtjjtjjjtjjjgd d d d 1} t}t| |j}Wd n1swYtjt}Wd n1swY||d }tj}|d}|| }||} ||| | dt d| t d|dt d|ddt | j dddt dqEd}|dkrd}n |dkrd}nd}t d|!||ddg}dd g}|"|||t |dd!} | S)"a Search through available VAE design choices to find one that best matches the given parameter and memory budgets. Args: parameters_budget (float, optional): The target number of parameters (in millions). cuda_max_mem (float, optional): The target maximum GPU memory usage (in MB). Returns: AutoencoderKL: The chosen VAE configuration that best matches the provided budgets. rzAPlease specify a valid parameter budget or cuda max memory budgetrr rrrz--------------------i@BT) activitiesprofile_memory record_shapes with_stackNii)rfrndesignz Total params: z Max GPU Memory Usage: z MBz Total Execution Time: z.2fz mscuda_time_total )sort_by row_limitrdrmroz####################zparam_diff (M)zcuda_mem_diff (MB) r)#rqr r rarPr`r!rrrrSrvr from_configr+rreset_peak_memory_stats synchronizeprofilerprofileProfilerActivityCPUCUDAtime perf_counterno_gradencode latent_distsamplemax_memory_allocatedrJ key_averagestablerrr~)rparameters_budget cuda_max_memsearch_space_choices search_space inp_tensordesign_choicesrQparameters_budget_diffcuda_max_mem_diffcurt_design_jsonvae total_paramsprof start_time_end_timetotal_execution_time_msr sort_moderwrxrrrsearch_for_target_vaes                  z"VAEGenerator.search_for_target_vaecC|jS)zp Get the input resolution for the VAE. Returns: int: The input resolution. )rrrrrr -zVAEGenerator.input_resolutioncCr)zt Get the compression ratio for the VAE. Returns: float: The compression ratio. )rrrrrr 7rzVAEGenerator.compression_ratio)rrr#)rr)__name__ __module__ __qualname____doc__intrr!rModuler+r>rPrSr`rarrstrrrr~rpropertyr floatr rrrrrs"    ^ r) rGrtypingrrrtorch.profiler diffusersrrrrrrrs