o wil@sbddlmZmZmZmZddlZddlmZddl m Z ddl m Z m Z dgZGdddeZdS))AnyDictListOptionalN)Callback) read_tb_log)flops_formulasloggingFLOPsMeasurementCallbackc@sjeZdZdZdZ  ddeeefdeedeefddZ d d Z d e e Be Bd e fd dZddZdS)r a Calculate FLOPs per second after last train step for a given job run. Args: model_config (Dict[str, Any]): params for running the experiment/job. Expects a nested dictionary with parent keys 1. run- for assessing model name (Eg. 'gpt3', 'llama2', etc.) from sub-key 'name'. 'name' usually has value like- train_gpt3_5b_*, which is matched to model name 'gpt3'. 2. exp_manager- for accessing 'explicit_log_dir'. tensorboard log file is stored here, used for accessing step time needed for calculating TFLOPs per sec per GPU 3. trainer- for accessing 'num_nodes' and 'devices' needed for calculating TFLOPs per sec per GPU 4. model- Hyperparams for the model. Specifically- global batch size, sequence length, hidden size, ffn hidden size, num_layers, num_attention_heads, num_query_groups, moe_router_topk. (list might increase with new models as required) log_dir (Optional[str]): Directory with tenbsorboard log file. If present, will overrride 'explicit_log_dir' in model_config. Defaults to None. model_name (Optional[str]): If present, will override 'name' under 'run' in model_config. Defaults to None. TN model_configlog_dir model_namec Cs`||_|jdi|_|jdipi|_|jdi|_|jdi|_|dur0|jddn||_|dur>|jddn||_|jdd|_|jd d|_ |jd d}|jd d}|jd d}|jd d}|jdd}|jdd} |jdd} |jdd} | dur| } t j |||||| | | d|_ |jdur|j |_dS|j|_dS)Nrun exp_managertrainermodelnameexplicit_log_dir num_nodesdevicesglobal_batch_sizeencoder_seq_length hidden_size num_layersffn_hidden_sizenum_attention_headsmoe_router_topknum_query_groups)gbs enc_seq_lenhslayersffn_hsattention_headsr query_groups)cfggetrun_cfgexp_cfg train_cfg model_cfgrr rnum_gpus_per_noder FLOPSConfig flops_configlower) selfr r r rr r!r"r#r$rr%r1i/home/ubuntu/sommelier/.venv/lib/python3.10/site-packages/nemo/collections/common/metrics/perf_metrics.py__init__2s<& z!FLOPsMeasurementCallback.__init__c Csd}zd|jdvrtdt|jd}||}Wnty4}ztd|WYd}~nd}~wwtd|d |j rK|j j d |dSdS) zd PyTorch Lightning callback hook to calculate TFLOPs per sec per GPU after training peftrz3FLOPs measurement not supported for finetuning jobsztrain_step_timing in sz,Failed to calculate TFLOPs per sec per GPU. NzTFLOPs per sec per GPU=z.2ftflops_per_sec_per_gpu) r&NotImplementedErrorrr eval_tflops_per_sec_per_gpu Exceptionr errorinfologger experiment add_scalar)r0r pl_moduler6step_time_listexcr1r1r2 on_train_endas z%FLOPsMeasurementCallback.on_train_endtrain_step_timereturncCsL|\}}t|ts|g}t|}t|t|dd}|d|S)ae Args: train_step_time (Any[List, float, int]): Train step time (in seconds). Step time will be less stable for initial steps (~10 steps)- less accurate measurement Use average step time over several steps for higher accuracy Returns: (float): Model TFLOPs per sec per gpu NgmB)eval_model_flops isinstancelistnparraymeanlen)r0rC total_flops flops_per_gpu step_time_arrr1r1r2r8ts   z4FLOPsMeasurementCallback.eval_tflops_per_sec_per_gpucstjtjtjtjtjtjd}jdur+fdd|D}t|dkr'|dnj_j|vrDt dt | t dj|jj}|jj}||fS)z9 Calculate model FLOPs for a given model )gpt3llama2llama3nemotronmixtralbertNcsg|] }|jvr|qSr1)r).0rr0r1r2 sz=FLOPsMeasurementCallback.eval_model_flops..rz FLOPs measurement supported for zJFailed to extract valid model name from or missing FLOPs calculations for )rrPrQrRrSrTrUrrLr r;rHkeysKeyErrorr.rr,)r0model_flops_map model_matchesrMrNr1rWr2rFs   z)FLOPsMeasurementCallback.eval_model_flops)NN)__name__ __module__ __qualname____doc__higher_is_betterrstrrrr3rBrfloatintr8rFr1r1r1r2r s  / )typingrrrrnumpyrIlightning.pytorch.callbacksr0nemo.collections.common.parts.perf_metrics_utilsr nemo.utilsrr __all__r r1r1r1r2s