o }oi @sddlmZddlmZddlZddlmZddl m Z ddl m Z ddl mZdZejjedd ejejfd d Zejje ed         ddeedededededeedededed ejfddZdS) )OptionalN)llm)finetune)bert_embedding_model)default_finetune_recipee5_340m)namereturncCs ttdS)av Factory function to create a E5-Large (340 million) model configuration. Returns: run.Config[pl.LightningModule]: Configuration for the E5-Large (340 million) model. Examples: CLI usage: $ nemo llm pretrain model=e5_340m ... Python API usage: >>> model_config = model() >>> print(model_config) )version)rNAMEr r X/home/ubuntu/.local/lib/python3.10/site-packages/nemo/collections/llm/recipes/e5_340m.pymodels r)targetrintfloat/e5-large-v2default dir resume_pathr num_nodesnum_gpus_per_node peft_scheme seq_lengthmicro_batch_sizeglobal_batch_sizec CsNtt|||||} tjtj|||d} | | _|dus%|dks%Jd| S)a Create a fine-tuning recipe for E5-large (340 million) model. This function sets up a complete configuration for fine-tuning, including model, trainer, data, logging, optimization, and resumption settings. Only SFT is currently supported for E5 model. Args: dir (Optional[str]): Directory for saving logs and checkpoints. name (str): Name of the fine-tuning run. num_nodes (int): Number of compute nodes to use. num_gpus_per_node (int): Number of GPUs per node. peft_scheme (Optional[str]): Name of the peft scheme to use for fine-tuning. Allowed values: 'none'/None. resume_path (str): Path to the NeMo checkpoint seq_length (int): Maximum number of tokens per microbatch. micro_batch_size (int): Micro batch size. global_batch_size (int): Global batch size. Returns: run.Partial: Partial configuration for fine-tuning. Examples: CLI usage: $ nemo llm finetune --factory e5_340m Python API usage: >>> recipe = finetune_recipe(name="e5_340m_finetune", num_nodes=1) >>> print(recipe) Note: This recipe uses the Specter dataset for fine-tuning. )rrrNnonezE5 only supports SFT.)rrrunConfigrSpecterDataModuledatalower) rrrrrrrrrrecipe datamoduler r r finetune_recipe.s.r') NrrrrNrrr)typingrlightning.pytorchpytorchplnemo_runr nemo.collectionsrnemo.collections.llm.apir+nemo.collections.llm.recipes.bert_embeddingr-nemo.collections.llm.recipes.finetune_defaultrr clifactoryr!LightningModulerstrintPartialr'r r r r sR