o ]i1 @sdZddlZddlZddlZddlZddlZddlmZmZm Z m Z m Z m Z m Z ddlmZedZejddejddejddejd dejd dedd diZd d Zdede efddZ d6de efddZdedede eeffddZdedefddZdedede efddZdede efdd Zdede ee ed!fffd"d#Z dede!fd$d%Z"dede efd&d'Z#dedefd(d)Z$ * d7deded+e e efd,d-Z%dedede!fd.d/Z&d0d1deded2ed3e!fd4d5Z'dS)8z"Module for spliting flag prefixes.N)AnyMutableSequenceOptionalSequenceTupleUnionType) config_dictcCsgt|j|jRSN) _split_nodevalueattrnr[/home/ubuntu/.local/lib/python3.10/site-packages/ml_collections/config_flags/config_path.pysrcCs t|jSr )r r irrrr s cCs|jfSr )idrrrrr!scCsttt|j|j|jfSr )slicemapr loweruppersteprrrrr"cCsgt|jt|jRSr )r r rrrrrr#rcCsdSr rrrrrr$scCstt|tj|Sr )_AST_SPLIT_CONFIG_PATHgettypeast literal_eval)noderrrr (sr config_pathreturnc Csnz tj|dd}Wnty }z td|d|dd}~wwt|tjr3t|j}t|tr3|St|)aReturns config_path split into a tuple of parts. Example usage: >>> assert config_path.split('a.b.cc') == ('a', 'b', 'cc') >>> assert config_path.split('a.b["cc.d"]') == ('a', 'b', 'cc.d') >>> assert config_path.split('a.b[10]') == ('a', 'b', 10) >>> assert config_path.split('a[(1, 2)]') == ('a', (1, 2)) >>> assert config_path.split('a[:]') == ('a', slice(None)) Args: config_path: Input path to be split - see example usage. Returns: Tuple of config_path split into parts. Parts are attributes or subscripts. Attrributes are treated as strings and subscripts are parsed using ast.literal_eval. It is up to the caller to ensure all returned types are valid. Raises: ValueError: Failed to parse config_path. eval)modezCould not parse z: N) rparse SyntaxError ValueError isinstance Expressionr bodytuple)r"r!eresultrrrsplit,s   r/ field_pathcCstt|trt||rt||St|dr||St|tr+tt|d|d|tdt|d|d|)z2Returns attribute of member failing that the item. __getitem__z$ does not support integer indexing [z]]. Attempting to lookup: z Attribute .zN does not exist and the type does not support indexing. Attempting to lookup: )r)strhasattrgetattrint IndexErrorrKeyError)configfieldr0rrr_get_item_or_attributeMs   r;r9cCsDt|}|s tdtjt|d}t||dd|}||dfS)aReturns the last part config_path and config to allow assignment. Example usage: >>> config = {'a': {'b', {'c', 10}}} >>> holder, lastfield = _get_holder_field('a.b.c', config) >>> assert lastfield == 'c' >>> assert holder is config['a']['b'] >>> assert holder[lastfield] == 10 Args: config_path: Any string that `split` can process. config: A nested datastructure that can be accessed via _get_item_or_attribute Returns: The penultimate object when walking config with config_path. And the final part of the config path. Raises: IndexError: Integer field not found in nested structure. KeyError: Non-integer field not found in nested structure. ValueError: Empty/invalid config_path after parsing. zPath cannot be emptyr0N)r/r( functoolspartialr;reduce)r"r9fieldsget_itemholderrrr_get_holder_field^s  rDcCs tjt|d}t|t||S)aGets value of a single field. Example usage: >>> config = {'a': {'b', {'c', 10}}} >>> assert config_path.get_value('a.b.c', config) == 10 Args: config_path: Any string that `split` can process. config: A nested datastructure Returns: The last object when walking config with config_path. Raises: IndexError: Integer field not found in nested structure. KeyError: Non-integer field not found in nested structure. ValueError: Empty/invalid config_path after parsing. r<)r>r?r;r@r/)r"r9rBrrr get_value~srEoverrideallowed_missingc st}fdd|D}|}t|dddD]l\}}|}t||}|dur)qt||} t| sFtdd|d| d t|d ||vr[td d|d |d |d z| }Wnty}} ztdd|d| d| | d} ~ wwt |||qdS)a{Adds some missing nested holder fields for a particular override. For example if override is 'config.a.b.c' and config.a is None, it will default initialize config.a, and if config.a.b is None will default initialize it as well. Only overrides present in allowed_missing will be initialized. Args: config: config object (typically dataclass) override: dot joined override name. allowed_missing: list of overrides that are allowed to be set. For example, if override is 'a.b.c.d', allowed_missing could be ['a.b.c', 'a', 'foo.bar']. Raises: ValueError: if parent field is not of dataclass type. cs&h|]}|drtt|qS)r2) startswithlenr/).0xrFrr s z3initialize_missing_parent_fields..Nr=z Override z# can not be applied because field "z" is None, and its type "z," is not a dataclass in the parent of type "z".zFlag zv" is None by default and it is not explicitly provided in flags (it can be default intialized by providing --.z =build flagz " of type z! can not be default instantiated:) r/ enumerater;get_typedc is_dataclassr(r Exception set_value) r9rFrGrAallowed_levelschildlevelfparent field_typer-rrLr initialize_missing_parent_fieldss^    r[ type_speccCs ttdr t|St|ddS)zACall typing.get_origin, with a fallback for Python 3.7 and below. get_origin __origin__N)r4typingr]r5r\rrrr]   r].cCs ttdr t|St|dtS)z=Call typing.get_args, with fallback for Python 3.7 and below.get_args__args__)r4r_rbr5NoneTyper`rrrrbrarbcCst|tttdtfvS)z-Cheeck if a type_spec is a Union type or not. UnionType)r]rr5typesr`rrr_is_union_typesrgcCs6t|sdSddt|D}t|dkrdS|dS)zEIf type_spec is of type Optional[T], returns T object, otherwise NoneNcSsg|]}|tur|qSr)rd)rJtrrr sz.extract_type_from_optional..rNr)rgrbrI)r\non_nonerrrextract_type_from_optionals  rkcCs.t|rt|}|durtd||S|S)aNormalizes a type object. Strips all None types from the type specification and returns the remaining single type. This is primarily useful for Optional type annotations in which case it will strip out the NoneType and return the inner type. Args: type_spec: The type to normalize. Raises: TypeError: If there is not exactly 1 non-None type in the union. Returns: The normalized type. Nz$Unable to normalize ambiguous type: )rgrk TypeError)r\subtyperrrnormalize_types rnT default_typecst||\}t|tjtjfr|dur|vr|S|St|rJfddt|D}|s>t ddt ||rFt |dS|dSt t ||S)aFGets type of field in config described by a config_path. Example usage: >>> config = {'a': {'b', {'c', 10}}} >>> assert config_path.get_type('a.b.c', config) is int Args: config_path: Any string that `split` can process. config: A nested datastructure normalize: whether to normalize the type (in particular strip Optional annotations on dataclass fields) default_type: If the `config_path` is not found and `default_type` is set, the `default_type` is returned. Returns: The type of last object when walking config with config_path. Raises: IndexError: Integer field not found in nested structure. KeyError: Non-integer field not found in nested structure. ValueError: Empty/invalid config_path after parsing. TypeError: Ambiguous type annotation on dataclass field. Ncsg|] }|jkr|jqSr)namer)rJrXr:rrri-szget_type..zField z not found on dataclass r) rDr)r ConfigDictFieldReferencerPrQrRrAr8rrnr;)r"r9 normalizerorCmatchesrrqrrPs   rPcCst||dd}t|duS)NF)rt)rPrk)r"r9raw_typerrr is_optional5s rwF)accept_new_attributesr rxcCst||\}}t|trt|tr|||<dSt|dr(||vs"|r(|||<dSt|t|r9t|t||dSt|trMt|dt|d|dt |dt|d|d)aeSets value of field described by config_path. Example usage: >>> config = {'a': {'b', {'c', 10}}} >>> config_path.set_value('a.b.c', config, 20) >>> assert config['a']['b']['c'] == 20 Args: config_path: Any string that `split` can process. config: A nested datastructure value: A value to assign to final field. accept_new_attributes: If `True`, the new config attributes can be added Raises: IndexError: Integer field not found in nested structure. KeyError: Non-integer field not found in nested structure. ValueError: Empty/invalid config_path after parsing. __setitem__z is not a valid index for z (in: )z$ is not a valid key or attribute of N) rDr)r6rr4r3setattrr7rr8)r"r9r rxrCr:rrrrT:s"      rTr )TN)(__doc__r dataclassesrQr>rfr_rrrrrrrml_collectionsr rrd AttributeIndexNameSlice Subscriptrr r3r/r;rDrEr[r]rbboolrgrkrnrPrwrTrrrrsp$        "   ;"   .