o wi| @sddlZddlmZddlmZddZedkrKeZzejej ej ej ddWne yCZ zed e WYdZ [ ndZ [ wwed dSdS) N)api)loggingcCsDtjdd}|jdtdd|jdtdd|jdtd d|S) NzHTest evaluation with lm-eval-harness on nemo2 model deployed on PyTriton) descriptionz--nemo2_ckpt_pathzNeMo 2.0 ckpt path)typehelpz--max_batch_sizezMax BS for the modelz --trtllm_dirzjFolder for the trt-llm conversion, trt-llm engine gets saved in this specified dir)argparseArgumentParser add_argumentstrint parse_args)parserr[/home/ubuntu/sommelier/.venv/lib/python3.10/site-packages/tests/evaluation/deploy_script.pyget_argssr__main__trtllm)nemo_checkpointmax_batch_sizetriton_model_repositorybackendz%Deploy process encountered an error: zDeploy process terminated.)rnemo.collections.llmr nemo.utilsrr__name__argsdeploynemo2_ckpt_pathr trtllm_dir Exceptioneerrorinforrrrs&