o }oi @sRddlZddlmZddlmZdZejjej edde de dej fd d Z dS) N)llm) PruningConfigdefault)targetnamenemo_checkpoint save_pathreturncCs<tjtj||ttdddddtjtjdddddd }|S)aCreate a generic pruning recipe. This function sets up a complete configuration for pruning, including trainer and data. Args: nemo_checkpoint (str): The path to the NeMo checkpoint to be pruned. save_path (str): The path to save the pruned NeMo checkpoint. Returns: run.Partial: Partial configuration for pruning. Python recipe usage: >>> import nemo_run as run >>> from nemo.collections import llm >>> from nemo.collections.llm.modelopt.recipes import prune_recipe >>> recipe = prune_recipe( ... nemo_checkpoint="/path/to/llama3.1-8b/nemo-ckpt/", ... save_path="/path/to/pruned/llama3.1-8b/nemo-ckpt/", ... ) >>> recipe.devices = 8 >>> recipe.pp_size = 8 >>> recipe.data = run.Config( ... llm.PreTrainingDataModule, ... paths=["1.0", "path/to/tokenized/data"], ... seq_length=8192, ... micro_batch_size=1, ... global_batch_size=1, # should be equal to micro_batch_size ... ) >>> recipe.pruning_config.target_ffn_hidden_size = 9216 >>> recipe.pruning_config.target_hidden_size = 3072 >>> ... >>> run.run(recipe) i) seq_lengthmicro_batch_sizeglobal_batch_sizeF) rrpruning_configdevices num_nodestp_sizepp_sizenum_train_samplesdata legacy_ckpt)runPartialrpruneConfigrMockDataModule)rrreciperf/home/ubuntu/.local/lib/python3.10/site-packages/nemo/collections/llm/modelopt/recipes/prune_recipe.py prune_recipes$r) nemo_runrnemo.collectionsrnemo.collections.llm.modeloptrNAMEclifactoryrstrrrrrrrs