o پi @sdZddlmZddlZdejdeeefdBfddZdefd d Zd ede fd d Z  ddejdejdBdejdBdejfddZ dejdejdejdedejf ddZ dS)zUtils for model executor.)AnyNweight weight_attrscCsj|durdS|D](\}}t||rJd|ddlm}|r,|dkr,t|}t|||q dS)aSet attributes on a weight tensor. This method is used to set attributes on a weight tensor. This method will not overwrite existing attributes. Args: weight: The weight tensor. weight_attrs: A dictionary of attributes to set on the weight tensor. Nz'Overwriting existing tensor attribute: r)current_platform weight_loader)itemshasattr'sglang.multimodal_gen.runtime.platformsris_tpu_make_synced_weight_loadersetattr)rrkeyvaluerr^/home/ubuntu/.local/lib/python3.10/site-packages/sglang/multimodal_gen/runtime/models/utils.pyset_weight_attrs s  rreturncsfdd}|S)Ncs$|g|Ri|t|dS)N)torch_sync)paramargskwargsoriginal_weight_loaderrr_synced_weight_loader0sz9_make_synced_weight_loader.._synced_weight_loaderr)rrrrrr .s r layer_namec Cs`|d}g}|D]}z |t|Wq tyYq wt|dks,Jd|d|dS)z Extract the layer index from the module name. Examples: - "encoder.layers.0" -> 0 - "encoder.layers.1.self_attn" -> 1 - "2.self_attn" -> 2 - "model.encoder.layers.0.sub.1" -> ValueError .z layer name z should only contain one integerr)splitappendint ValueErrorlen)rsubnamesint_valssubnamerrrextract_layer_index7s   r&xshiftscalecCs`|dur |dur |S|dur|d|dS|dur"||dS|d|d|dS)zmodulate by shift and scaleNr) unsqueeze)r'r(r)rrrmodulateMsr+ pred_noisenoise_input_latenttimestep schedulerc Cs|jdkr|dd}||jdksJn)|jdkr8|jddkr,||jd}n||jdks7Jntd|j|j}|j}| |}| |}|j  |}|j  |}t j |d|ddd}||dddd} || |} | |S)a Convert predicted noise to clean latent. Args: pred_noise: the predicted noise with shape [B, C, H, W] where B is batch_size or batch_size * num_frames noise_input_latent: the noisy latent with shape [B, C, H, W], timestep: the timestep with shape [1] or [bs * num_frames] or [bs, num_frames] scheduler: the scheduler Returns: the predicted video with shape [B, C, H, W] rrz3[pred_noise_to_pred_video] Invalid timestep shape: )dim)ndimflattennumelshapeexpandr!dtypedevicedoubletosigmas timestepsrargminr*absreshape) r,r-r.r/r8r9r<r= timestep_idsigma_t pred_videorrrpred_noise_to_pred_video_s,      rD)NN) __doc__typingrrTensordictstrrr r r&r+rDrrrrs@  "