o Ti!@sdZddlZddlZddlZddlmZddlmZmZm Z ddl m Z GdddeZ dd Z Gd d d eZGd d d ejZGdddeZddZddZddZddZdS)z1 Collection of DeepSpeed configuration utilities N)reduce) BaseModel ConfigDictfield_serializer)loggerc sheZdZdZdfdd ZddZddZed d d d d d d d Ze d ddd e j de fddZ ZS)DeepSpeedConfigModela This class should be used as a base for all DeepSpeed configs. It extends pydantic.BaseModel to allow for deprecated fields. To enable this feature, add deprecated=True to pydantic.Field: my_dep_field: int = Field(0, deprecated=True) Deprecated Field kwargs: - deprecated: [True|False], default False Enables / Disables deprecated fields - deprecated_msg: str, default "" Message to include with deprecation warning - new_param: str, default "" Name of the field replacing the deprecated field - set_new_param: [True|False], default True If new_param is provided, enables setting the value of that param with deprecated field value - new_param_fn: callable, default (lambda x: x) If new_param is provided and set_new_param is True, this function will modify the value of the deprecated field before placing that value in the new_param field Example: my_new_field is replacing a deprecated my_old_field. The expected type for my_new_field is int while the expected type for my_old_field is str. We want to maintain backward compatibility with our configs, so we define the fields with: class MyExampleConfig(DeepSpeedConfigModel): my_new_field: int = 0 my_old_field: str = Field('0', deprecated=True, new_param='my_new_field', new_param_fn=(lambda x: int(x))) Fc s4|s dd|D}tjdi||dS)NcSs&i|]\}}|dks|dkr||qS)autoreplace_method.0kvr r R/home/ubuntu/.local/lib/python3.10/site-packages/deepspeed/runtime/config_utils.py 8s&z1DeepSpeedConfigModel.__init__..r )itemssuper__init___deprecated_fields_check)selfstrictdata __class__r rr6s zDeepSpeedConfigModel.__init__c Cs|}|j}t|j|j}|ddd}|t||}|dd}|dd}||vrtd|d|r;d |d nd|rDd |nd|r|d d rzt||Wnt yn} z t d|d| d} ~ ww| d} t | dkrt t| dd|}|j}| d} | |vsJd|d|dz t|| |WdSt y} zt d|d|d| d} ~ wwdSdSdS)N new_param_fncSs|SNr )xr r rAsz@DeepSpeedConfigModel._process_deprecated_field.. new_paramdeprecated_msgzConfig parameter z is deprecatedz use z insteadz. set_new_paramTzTried removing deprecated 'z ' from config.z%Cannot provide deprecated parameter 'z' and replacing parameter 'z ' togetherzTried setting value for 'z' with value from deprecated '')model_fields_settype model_fieldsjson_schema_extragetgetattrrwarningdelattr Exceptionerrorsplitlenrsetattr) r dep_fieldpydantic_config fields_setkwargsr param_value new_fielddep_msgenew_param_nestednew_param_namer r r_process_deprecated_field<sL     z.DeepSpeedConfigModel._process_deprecated_fieldcCs:|j}|D]\}}|jr|jddr||qdS)N deprecatedF)r(rr)r*r=)rfields field_name field_infor r rrcs  z-DeepSpeedConfigModel._deprecated_fields_checkTforbidr )validate_defaultvalidate_assignmentuse_enum_valuespopulate_by_nameextraarbitrary_types_allowedprotected_namespacesdtype) check_fieldsreturncCst|Sr)str)rJr r rserialize_torch_dtypessz*DeepSpeedConfigModel.serialize_torch_dtype)F)__name__ __module__ __qualname____doc__rr=rr model_configrtorchrJrMrN __classcell__r r rrrs $' rcCsN||jvsJd|d||j|rJd|d|j|S)Nr%z' is not a field in z7' is a required field and does not have a default value)r(r* is_required get_default)configr@r r rget_config_defaultxs rYcs*eZdZdZdfdd ZddZZS)pp_intz A wrapper for integers that will return a custom string or comma-formatted string of the integer. For example, print(pp_int(1e5)) will return "10,000". This is useful mainly for auto-generated documentation purposes. Ncst||}||_|Sr)r__new__custom_print_str)clsvalr\instrr rr[szpp_int.__new__cCs|jr|jS|jdS)N,)r\realrr r r__repr__s zpp_int.__repr__r)rOrPrQrRr[rcrUr r rrrZsrZcs"eZdZdZdfdd ZZS)ScientificNotationEncodera This class overrides ``json.dumps`` default formatter. This version keeps everything as normal except formats numbers bigger than 1e3 using scientific notation. Just pass ``cls=ScientificNotationEncoder`` to ``json.dumps`` to activate it Frcsjdurjnd}d|}d7d|t|tr%|r#dSdSt|ts/t|tr:|dkr7|dS|St|tjjr\fdd |D}d d |d |d St|tjj rut|t sudd t j |dSd t ||S)N r#truefalseg@@r:c s0g|]\}}dd|dj|dqS) "z": )level) iterencoder rkprefixrr r s0z8ScientificNotationEncoder.iterencode..{z, ri}[]z , )indent isinstanceboolfloatint collectionsabcMappingrjoinSequencerMmaprlr)ro _one_shotrkrt prefix_closerrrmrrls     z$ScientificNotationEncoder.iterencode)Fr)rOrPrQrRrlrUr r rrrds rdc@s eZdZdZddZddZdS)DeepSpeedConfigObjectz For json serialization cCs|jSr)__dict__rbr r rreprszDeepSpeedConfigObject.reprcCstj|jddtdS)NTre) sort_keysrtr])jsondumpsrrdrbr r rrcs zDeepSpeedConfigObject.__repr__N)rOrPrQrRrrcr r r rrs rcC |||Srr* param_dict param_nameparam_default_valuer r rget_scalar_param rcCrrrrr r rget_list_paramrrcCrrrrr r rget_dict_paramrrcCsZtdd|D}t|t|kr+tdd|D}dd|D}td||S)zReject duplicate keys.css|] \}}||fVqdSrr r r r r sz5dict_raise_error_on_duplicate_keys..cSsg|]}|dqS)rr )r pairr r rrosz6dict_raise_error_on_duplicate_keys..cSsg|] \}}|dkr|qS)r#r )r keyvaluer r rrosz&Duplicate keys in DeepSpeed config: {})dictr1ryCounterr ValueErrorformat) ordered_pairsdcounterkeysr r r"dict_raise_error_on_duplicate_keyss r)rRryrrT functoolsrpydanticrrrdeepspeed.utilsrrrYrxrZ JSONEncoderrdobjectrrrrrr r r rs   g