o }oi@sddlZddlmZedd\ZZer'ddlmZddZddZee_ ee_  dd ej d ej dej de de de dede fddZdS)N)safe_import_fromcut_cross_entropylinear_cross_entropycCsfddl}z|dj}||}||}td|d|||kWS|jy2tdYdSw)zCheck if pytorch-triton version is greater than or equal to the specified version. Args: version_str: Version string to check Returns: bool: True if pytorch-triton version >= specified version rNzpytorch-tritonz Current pytorch-triton version: z, Required triton version: zpytorch-triton not foundF) pkg_resourcesget_distributionversion parse_versionprintDistributionNotFound) version_strrpytorch_triton_versioncurrentrequiredrQ/home/ubuntu/.local/lib/python3.10/site-packages/nemo/automodel/loss/linear_ce.pynew_is_triton_greater_or_equalLs     rcCstdS)zCheck if pytorch-triton version is greater than or equal to 3.1.0 Returns: bool: True if pytorch-triton version >= 3.1.0 z3.1.0)rrrrr$new_is_triton_greater_or_equal_3_2_0asrmeanauto hidden_states lm_weightlabelsnum_items_in_batch ignore_index reductionlogit_softcappingaccuracy_thresholdc CsR|dkrd}t|||||dddd}|dkr'|dur#t||k}||}|S)aC Compute fused linear cross entropy loss that matches PyTorch's cross_entropy behavior. Args: hidden_states: Input hidden states lm_weight: Weight matrix for linear transformation labels: Target labels num_items_in_batch: Number of valid tokens (where labels != ignore_index) ignore_index: Value to ignore in labels (default: -100) reduction: Reduction method ('mean' or 'sum') logit_softcapping: Value for softcapping logits (0 means no capping) accuracy_threshold: Threshold for accuracy computation rNsumF)targetsrsoftcaprshift filter_epsr)rtorchritem) rrrrrrrrlossrrrfused_linear_cross_entropyns" r&)Nrrrr)r#nemo.utils.import_utilsrrHAVE_LINEAR_LOSS_CEcut_cross_entropy.tl_utilstl_utilsrris_triton_greater_or_equal is_triton_greater_or_equal_3_2_0Tensorintstrfloatr&rrrrsD>