o ߗid%@sddlZddlmZddlmZddlmZmZmZm Z m Z m Z m Z ddl Z ddlmZddlmZddlmZddd ZeZeZGd d d Zefd e efd dZdejde eeeffddZdS)N) OrderedDictwraps)CallableDictListOptionalSequenceTypeUnion)_State)_get_root_modules__composable_api_state_keycCs|dttS)N_)struuiduuid4)stringrd/home/ubuntu/transcripts/venv/lib/python3.10/site-packages/torch/distributed/_composable/contract.pygenerate_state_key src@s eZdZdS) RegistryItemN)__name__ __module__ __qualname__rrrrrsr state_clscstfdd}|S)a% Decorate a function as a composable distributed API, where the first argument of the function must be an :class:`nn.Module` instance or sequence of :class:`nn.Module` instances. The decorator verifies that the decorated function does not modify fully-qualified names (FQNs) for parameters, buffers, or modules. The decorated function can return different module instances than the input modules; the FQN invariant will be enforced following the input order. When a function ``func`` is decorated by ``@contract()``, a ``.state(module: nn.Module)`` method will be installed to the decorated function. Then you can retrieve and modify the state on a module by calling ``func.state(module)``. Example:: >>> # xdoctest: +SKIP >>> import torch.nn as nn >>> >>> class MyModel(nn.Module): >>> def __init__(self) -> None: >>> super().__init__() >>> self.l1 = nn.Linear(10, 10) >>> self.l2 = nn.Linear(10, 10) >>> >>> def forward(self, x): >>> return self.l2(self.l1(x)) >>> >>> @contract() >>> def my_feature(module: nn.Module) -> nn.Module: >>> my_feature.state(module).some_state = "any value" >>> return module >>> >>> model = MyModel() >>> my_feature(model.l1) >>> assert my_feature.state(model.l1).some_state == "any value" >>> my_feature(model.l2) >>> model(torch.randn(2, 10)).sum().backward() cs\tdttjttjfdttjffdd }dtjdttffdd }||_|S)Nmodulereturncs|}t|tjr |g}ntt|}}t}g}g}g} |D]j}t} t} |jt | } t| t s;t d| |jt | } t| t sNt d| | vsWj | vrbt dj d|| || j ||t||t|| t|q |g|Ri|}|dur|}t|tjr|g}ntt|}g}g}g}|D]}|t||t||t|qt| }t|}||krt j d|d|ddttd ttd tfd d }t||D]\}}|t|t|d qt||D]\}}|t|t|dqt| |D]\}}|t|t|dq2|S)Nz-Distributed composable API states corrupted: z/Distributed composable API registry corrupted: zOEach distinct composable distributed API can only be applied to a module once. z3 has already been applied to the following module: zB should return the same number of modules as input modulesInputs: z modules Outputs: z modules orig_fqnsnew_fqns check_keycSsn||krdSt|t|}}||}||}t|st|r+t|d|d|t|d|d|)NzVComposable distributed API implementations cannot modify FQNs. FQNs only in original: z FQNs only in new: z[Composable distributed API implementations cannot modify the order of FQNs. Original FQNs: z New FQNs: )setlen RuntimeError)rrr orig_fqn_set new_fqn_set orig_onlynew_onlyrrr check_fqns(z;contract..inner..wrapper..check_fqnzChecking parameters: zChecking buffers: zChecking modules: ) isinstancennModuler listrr__dict__ setdefault STATE_KEYdictAssertionError REGISTRY_KEYrappendnamed_parameters named_buffers named_modulesr"rrzipkeys)rargskwargs inp_modulemodulesstate registry_itemall_orig_named_paramsall_orig_named_buffersall_orig_named_modulesdefault_all_statedefault_registry all_stateregistryupdatedupdated_modulesall_new_named_paramsall_new_named_buffersall_new_named_modulesnum_orig_modulesnum_new_modulesr(orig_named_paramsnew_named_paramsorig_named_buffersnew_named_buffersorig_named_modulesnew_named_modules)funcrrrwrapperIs                 z(contract..inner..wrappercs|jtiS)N)r-r.r/getrrSrr get_statesz*contract..inner..get_state)rr r*r+r rr r=)rSrTrXrrWrinnerGszcontract..innerr)rrZrrYrcontracts*r[rrcCs t|tdS)z Get an ``OrderedDict`` of composable APIs that have been applied to the ``module``, indexed by the API name. If no API has been applied, then this returns ``None``. N)getattrr2rVrrr _get_registrys r])r)r collectionsr functoolsrtypingrrrrr r r torchtorch.nnr*#torch.distributed._composable_stater torch.distributed.utilsr rr/r2rr[r+rr]rrrrs  $    $>