o ٷi7 @s$ddlZddlZddlmZmZmZddlmZmZm Z m Z ddl m Z m Z ddlmZeeZdd ed ed efd d Zdded ed efddZddeeBd ed efddZdeeBd eeee ffddZdeeBd ee eBfddZ ddeeBded eeeee e feffddZdS)N)TupleOptionalUnion) DBCacheConfigTaylorSeerCalibratorConfig DBPruneConfigCalibratorConfig)ParallelismConfigParallelismBackend) init_loggerF cache_kwargsresetreturnc Cs|zt|}i}|ddrB|s't|d|dd|dd|dddntj|d|dd|dd|ddd|d <d |vr_|sKtnt|d <|d jdi||WSt|d d }|d kr|sptnt|d <|d jdi||WS|dkr|stnt|d <|d jdi||WSt d|dt y}z t dt|d }~ww)a Load cache options from a dictionary. We keep this function for backward compatibility. Args: cache_kwargs (`dict`): A dictionary containing the cache configuration. reset (`bool`, *optional*, defaults to `False`): Whether to reset the configuration to default values to None before applying the loaded settings. This is useful when you want to ensure that only the settings specified in the dictionary are applied, without retaining any previous configurations (e.g., when using ParaModifier to modify existing configurations). Returns: `dict`: A dictionary containing the loaded cache options. enable_taylorseerFenable_encoder_taylorseertaylorseer_cache_typeresidualtaylorseer_orderr)enable_calibratorenable_encoder_calibratorcalibrator_cache_typercalibrator_config cache_type cache_configNDBCacheDBPrunezUnsupported cache_type: .z#Error parsing cache configuration. ) copydeepcopygetrrrupdatestrr ValueError Exception)rrkwargscache_context_kwargsrerrK/home/ubuntu/.local/lib/python3.10/site-packages/cache_dit/caching/utils.pyload_cache_options_from_dictsN         r+yaml_file_pathc Csz t|d }t|}Wdn1swYt||WSty.td|tjyD}z tdt|d}~ww)NrConfiguration file not found: YAML file parsing error: )openyaml safe_loadr+FileNotFoundError YAMLErrorr$)r,rfr'r)rrr*load_cache_options_from_yamlPs    r6 path_or_dictcCs:tdt|trt||St|trt||Std)a Load cache options from a YAML file or a dictionary. Args: path_or_dict (`str` or `dict`): The file path to the YAML configuration file or a dictionary containing the configuration. reset (`bool`, *optional*, defaults to `False`): Whether to reset the configuration to default values to None before applying the loaded settings. This is useful when you want to ensure that only the settings specified in the file or dictionary are applied, without retaining any previous configurations (e.g., when using ParaModifier to modify existing configurations). Returns: `dict`: A dictionary containing the loaded cache options. zg`load_options` is deprecated and will be removed in future versions. Please use `load_configs` instead.EInput must be a file path (str) or a configuration dictionary (dict).)loggerwarning isinstancer$r6dictr+r%)r7rrrr* load_options[s    r=c Kst|trFzt|d }t|}Wdn1swYWn3ty/td|tjyE}z tdt|d}~wwt|trQt |}nt dd|vrd|vr_dSt || d d }| dd}| d d}|dur|t d ||fS|d}d |vr|d } t| trd|vsJdddl m} |d} | | | d|d <t || d d }| dd}| d d}|durt d ||fS)an New APU that only load cache configuration from a YAML file or a dictionary. Assumes that the yaml contains a 'cache_config' section, and returns only that section. Raise ValueError if not found. Args: path_or_dict (`str` or `dict`): The file path to the YAML configuration file or a dictionary containing the configuration. reset (`bool`, *optional*, defaults to `False`): Whether to reset the configuration to default values to None before applying the loaded settings. This is useful when you want to ensure that only the settings specified in the file or dictionary are applied, without retaining any previous configurations (e.g., when using ParaModifier to modify existing configurations). Returns: `dict`: A dictionary containing the loaded cache configuration. r-Nr.r/r8rparallelism_config)NNrFrz(Failed to load 'cache_config'. Got None.steps_computation_masknum_inference_stepszUTo parse steps_mask from str, 'num_inference_steps' must be provided in cache_config.r) steps_mask) total_steps mask_policy)r;r$r0r1r2r3r4r<r r!r%r+r"cache_interfacerA) r7r'r5rr)r(rrcache_config_kwargsr?rAr@rrr*load_cache_configvs\                rFc Kstt|trFzt|d }t|}Wdn1swYWn3ty/td|tjyE}z tdt|d}~wwt|trQt |}nt d| ddr_d|vSd|vredS|d}d |vrx|d }t ||d <d ttBdBd ttfd d }| ddrd|vr||d|d<d|vr||d|d<d|vr||d|d<tdi|}|S)a Load parallelism configuration from a YAML file or a dictionary. Assumes that the yaml contains a 'parallelism_config' section, and returns only that section. Raise ValueError if not found. Args: path_or_dict (`str` or `dict`): The file path to the YAML configuration file or a dictionary containing the configuration. Returns: `ParallelismConfig`: An instance of ParallelismConfig containing the loaded parallelism configuration. r-Nr.r/r8 check_onlyFr>backendsizercSs|durdSt|tr |St|tr>|dkr>ddlm}d}|r(|}|dkr3t d|St d|d|St d|d) NautorrzvAuto parallel size selected as 1. Make sure to run with torch.distributed to utilize multiple devices for parallelism.zAuto selected parallel size to rzInvalid parallel size value: z. Must be int or 'auto'.) r;intr$lowertorch.distributed distributedis_initializedget_world_sizer9r:infor%)rIdistrrr*_maybe_auto_parallel_sizes"  z:load_parallelism_config.._maybe_auto_parallel_sizeauto_parallel_sizeT ulysses_size ring_sizetp_sizer)r;r$r0r1r2r3r4r<r r!r%r"r from_strrKrr ) r7r'r5parallel_kwargsr)parallelism_config_kwargs backend_strrSr>rrr*load_parallelism_configsN        r\T return_dictcKsLt|fi|\}}t|fi|}t|trd}|r!|||dS|||fS)a Load both cache and parallelism configurations from a YAML file or a dictionary. For example, the YAML file can be structured as follows: ```yaml cache_config: max_warmup_steps: 8 warmup_interval: 2 max_cached_steps: -1 max_continuous_cached_steps: 2 Fn_compute_blocks: 1 Bn_compute_blocks: 0 residual_diff_threshold: 0.12 enable_taylorseer: true taylorseer_order: 1 parallelism_config: ulysses_size: 4 parallel_kwargs: attention_backend: native experimental_ulysses_anything: true experimental_ulysses_float8: true extra_parallel_modules: ["text_encoder", "vae"] ``` Args: path_or_dict (`str` or `dict`): The file path to the YAML configuration file or a dictionary containing the configuration. Returns: `Tuple[DBCacheConfig, Optional[CalibratorConfig], ParallelismConfig]`: A tuple containing the loaded cache configuration, optional calibrator configuration, and parallelism configuration. If `return_dict` is set to `True`, returns a dictionary with keys "cache_config", "calibrator_config", and "parallelism_config". N)rrr>)rFr\r;bool)r7r]r'rrr>rrr* load_configs s#  r_)F)T)r1r typingrrrcache_contextsrrrr parallelismr r cache_dit.loggerr __name__r9r<r^r+r$r6r=rFr\r_rrrr*s8 @  H  P