o wiL+ @sHddlZddlmZmZmZmZmZmZddlm Z eGdddeZ ede j dZ ee j ge j fZ ee j geee ffZ d d e d e d ed e fd dZ d d e d e d ed e fddZd d e j d eded efddZ d!d e d e d e fddZ d!de d e d e fddZ   d!de d e ded ed e f ddZddZddZdS)"N)CallableIterableProtocolTypeVarUnionruntime_checkable)nnc@seZdZdefddZdS)HasBoolreturncCsdSN)selfr r Y/home/ubuntu/sommelier/.venv/lib/python3.10/site-packages/nemo/collections/llm/fn/base.py__bool__szHasBool.__bool__N)__name__ __module__ __qualname__boolrr r r rr sr _TModule)boundFmodulefunc leaf_onlyr c s|ddstdrj|fd|i|Sttr>> import torch >>> import torch.nn as nn >>> from nemo.collections.llm import fn # Example: Doubling the weights of all Linear layers in a model model = nn.Sequential(nn.Linear(10, 20), nn.ReLU(), nn.Linear(20, 10)) def double_weights(m): if isinstance(m, nn.Linear): m.weight.data *= 2 return m model = fn.map(model, double_weights) print(model) _skip_mapFmaprc3s|]}t|VqdSr )hasattr).0keyrr r Lszmap..)itemsvalueskeys) poprr isinstancerall_map_module_dict_map_module_list _map_modulerrrkwargsr rrrs) rcKst||fd|d|S)a/Recursively apply a function to a module or collection. This function is similar to `map`, but it applies the function recursively to all child modules as well. This is useful for applying transformations that need to consider the module hierarchy. Args: module: The module or collection to recursively apply to. func: The function to apply. leaf_only: If True, only apply to modules without parameters. Defaults to False. **kwargs: Additional kwargs to pass to the function. Returns ------- The transformed module or collection. Examples -------- >>> import torch >>> import torch.nn as nn >>> from nemo.collections.llm import fn # Example: Setting the bias of all Conv2d layers to False model = nn.Sequential(nn.Conv2d(1, 20, 5), nn.ReLU(), nn.Conv2d(20, 10, 5)) def remove_bias(m): if isinstance(m, nn.Conv2d): m.bias = None return m model = fn.walk(model, remove_bias) print(model) T)recurser)rr)r r rwalkTs%r,r+cs@fdd|rfdd|D}t|S|g}t|S)ao Checks if a predicate holds for all modules in a given module or its children, optionally recursively. This function iterates over all modules and applies a predicate function to determine if all modules satisfy a certain condition. If `recurse` is True, it checks all child modules recursively. Args: module (nn.Module): The root module to check. func (ModulePredicate): A predicate function that takes a module as input and returns a boolean or an object that can be evaluated as a boolean. recurse (bool): If True, applies the predicate recursively to all child modules. Defaults to False. Returns ------- bool: True if all modules satisfy the predicate, False otherwise. Examples -------- >>> import torch.nn as nn >>> model = nn.Sequential(nn.Linear(10, 20), nn.ReLU(), nn.Linear(20, 10)) >>> predicate = lambda m: isinstance(m, nn.Linear) >>> print(forall(model, predicate)) False >>> print(forall(model, predicate, recurse=True)) True cs|}t|Sr )r)mresult)rr rapply_predicateszforall..apply_predicatecsg|]}|qSr r )rr-)r/r r szforall..)modulesr%)rrr+resultsr )r/rrforalls  r3c Ks|durt}t||vr|S|}t|fi|}|r#t|jddr+||fi|}|dds7|ddn |dd|d}|dd|dd|ddt|D]\} \} } t || t | |f|||| | |d |qZ| t||S) a  Applies a transformation function to a module and optionally to its child modules. Parameters ---------- module : nn.Module The module to which the function will be applied. func : ModuleFunc The function that will be applied to the module. recurse : bool, optional Whether to apply the function recursively to child modules. leaf_only : bool, optional Whether to apply the function only to modules without parameters. transformed_modules : set, optional A set to keep track of modules that have already been transformed. **kwargs : dict Additional keyword arguments that will be passed to the transformation function. Returns ------- nn.Module The transformed module. NF)r+prefixname.ir+rtransformed_modulesr8r6r4) setid_get_func_kwargslist parametersgetr# enumeratenamed_childrensetattrradd) rrr+rr:r* new_modulef_kwargsr4r8r6childr r rr(s@ .    r( module_listc Ks|durt}t|fi|}|s||fi|}g}|dds'|ddn |dd|d}|dd|dd|ddt|D]\} } t| |f|||| t| |d|} || qHt||S)Nr4r5r6r7r8r9) r;r=r@r#rArstrappend_create_list_wrapper) rHrr+rr:r*rFmapped_modulesr4r8rrEr r rr's4.     r' module_dictc Ks|durt}t|fi|}|s||fi|}i}t|D]\}\} } ||d<| |d<t| |f|||d||| <q!t||S)a Applies a transformation function to a ModuleDict of modules. Parameters ---------- module_dict : nn.ModuleDict The ModuleDict of modules to which the function will be applied. func : ModuleFunc The function that will be applied to the modules. recurse : bool, optional Whether to apply the function recursively to child modules. parameterless_modules_only : bool, optional Whether to apply the function only to modules without parameters. **kwargs : dict Additional keyword arguments that will be passed to the transformation function. Returns ------- nn.ModuleDict The ModuleDict of transformed modules. Nr8r6)r+rr:)r;r=rAr rtype) rMrr+rr:r*rFrLr8r6rr r rr&s( r&cCs2tt|j}d|jvrt||St||S)Nargs)inspect signaturerN__init__r?)rHto_addsigr r rrKFs   rKc s t|fdd|DS)Ncs i|] \}}|jvr||qSr )r?)rkwargvaluerTr r Qs z$_get_func_kwargs..)rPrQr )rr*r rWrr=Os r=)F)FFN)rPtypingrrrrrrtorchrr Moduler ModuleFuncrModulePredicaterr,r3r(r'r&rKr=r r r rsz   8 ./ B # 5