o ̳i @sddlZddlmZddlZddlmZddlmZmZmZddl m Z m Z m Z m Z ddlmZddlmZedZ dd eeefd ed dfd dZddejd ed ejfddZdS)N)Union)nn)TransformerCrossAttentionLayerTransformerDecoderTransformerSelfAttentionLayer)CEWithChunkedOutputLossForwardKLWithChunkedOutputLossReverseKLWithChunkedOutputLoss SymmetricKLWithChunkedOutputLoss)DeepFusionModel) get_loggerINFOTmodelverbosereturncCsftjdd}t|tr|j}|rtdtt | D]}t|t s*t|t r0|j |dqdS)a Utility to compile a transformer model inplace. On PyTorch nightlies we use per-layer compile to reduce compile times. Otherwise we compile the full model, which takes longer. Args: model (Union[TransformerDecoder, DeepFusionModel]): A model to compile. Can be a TransformerDecoder or DeepFusionModel; in the latter case only the model's decoder will be compiled. verbose (bool): Whether to log compile info. Default: True Returns: None TORCH_COMPILE_BACKENDinductorz,Compiling model layers with torch.compile...backendN)osenvironget isinstancer decoderloginforeversedlistmodulesrrcompile)rrrmr!O/home/ubuntu/.local/lib/python3.10/site-packages/torchtune/training/_compile.py compile_models    r#losscCstjdd}|rtdt|trtj|j |d|_ |St|t r.tj|j |d|_ |St|t r>tj|j |d|_ |St|trNtj|j|d|_|Stj||d}|S)a0 Utility to compile and return loss function. If the loss function is chunked cross-entropy, we only compile the upcast + cross-entropy calculation, not the chunking. For other losses we compile the entire loss function. Args: loss (nn.Module): A loss function to compile. verbose (bool): Whether to log compile info. Default: True Returns: loss (nn.Module): loss with either entire module compiled or (in the case of CEWithChunkedOutputLoss) only the upcast and cross-entropy calculation compiled. rrz$Compiling loss with torch.compile...r)rrrrrrrtorchrcompute_cross_entropyrfkl_lossr rkl_lossr sym_kl_loss)r$rrr!r!r" compile_loss<s&      r*)T)rtypingrr%rtorchtune.modulesrrrtorchtune.modules.lossrrr r torchtune.modules.model_fusionr torchtune.utilsr rboolr#Moduler*r!r!r!r"s$