o -i @sdZddlZddlmZddlZddlmZdejdee efdBfddZ d ej j d e d ejfd d Z dej j dee ee ffddZdej j deee e ee ffddZde dee effddZdS)zUtils for model executor.N)Anyis_torch_equal_or_newerweight weight_attrscCsl|durdS|D])\}}t||rJd|ddlm}|r-|dkr-||}t|||q dS)aSet attributes on a weight tensor. This method is used to set attributes on a weight tensor. This method will not overwrite existing attributes. Args: weight: The weight tensor. weight_attrs: A dictionary of attributes to set on the weight tensor. Nz'Overwriting existing tensor attribute: r)current_platform weight_loader)itemshasattrvllm.platformsruse_sync_weight_loadermake_synced_weight_loadersetattr)rrkeyvaluerrV/home/ubuntu/veenaModal/venv/lib/python3.10/site-packages/vllm/model_executor/utils.pyset_weight_attrs s   rlayer param_namenew_datacCsft|tjjr |j}tjj|dd}t||d}|dur+t|dr+|j}t|d|it |||dS)a Replace a parameter of a layer while maintaining the ability to reload the weight. Called within implementations of the `process_weights_after_loading` method. This function should not be called on weights which are tied/shared Args: layer: Layer containing parameter to replace param_name: Name of parameter to replace new_data: New data of the new parameter F) requires_gradNr) isinstancetorchnn Parameterdatagetattrr rrr)rrr new_param old_paramrrrrreplace_parameter/s  r modelreturncst|dddurtnirS|D]6}t|dd}|dur*t|ni}tfdd|DrJtdt|jdt|jd |qS)Npacked_modules_mappingc3s(|]\}}|vo||kVqdS)Nr).0kv parent_maprr Us&z-get_packed_modules_mapping..z Can't update z;'s packed_modules_mapping safely because of conflicts from .) rcopydeepcopychildrenanyr ValueErrortype__name__update)r!child child_maprr'rget_packed_modules_mappingHs    r5cCsFt|dd}r |S|D]}t|dd}|dur |SqgS)Nget_expert_mapping)rr-)r!r(r3r4rrrget_moe_expert_mapping_s   r7current_backendcCs|dkr tdr ddiSiS)Ninductorz 2.9.0.devgraph_partitionFr)r8rrrmaybe_disable_graph_partitionmsr;)__doc__r+typingrrvllm.utils.torch_utilsrTensordictstrrrModuler listr5tupleintr7boolr;rrrrs$   ""