o wi! @sddlZddlmZddlmZddZedkrKeZzejej ej dej dWne yCZ zed e WYdZ [ ndZ [ wwed dSdS) N)api)loggingcCsDtjdd}|jdtdd|jdtdd|jdd d d |S) NzHTest evaluation with lm-eval-harness on nemo2 model deployed on PyTriton) descriptionz--nemo2_ckpt_pathzNeMo 2.0 ckpt path)typehelpz--max_batch_sizezMax BS for the modelz --legacy_ckpt store_truez/Whether the nemo checkpoint is in legacy format)actionr)argparseArgumentParser add_argumentstrint parse_args)parserra/home/ubuntu/sommelier/.venv/lib/python3.10/site-packages/tests/evaluation/deploy_in_fw_script.pyget_argssr__main__i")nemo_checkpointmax_batch_size fastapi_port legacy_ckptz%Deploy process encountered an error: zDeploy process terminated.)r nemo.collections.llmr nemo.utilsrr__name__argsdeploynemo2_ckpt_pathrr Exceptioneerrorinforrrrs&