o …wÖi ã@s<ddlmZddlmZddlmZeGdd„deƒƒZdS)é)Ú dataclass)Únn)ÚLoRAc@s>eZdZUdZdZeed<dZeed<dej ddfdd „Z dS) raÁ Built on top of llm.LoRA, vlm.LoRA additionally allows the user to specify whether the language or vision models should be frozen. For example, a common finetuning workload for multimodal models is to apply adapters to language model and fully finetune the vision model. For detailed usage of the LoRA api, see llm.LoRA docstrings. Example: -------- >>> from nemo.collections import vlm >>> lora = vlm.peft.LoRA(target_modules=["*.language_model.*.linear_qkv"], freeze_vision_model=False, dim=32) >>> model = vlm.MLlamaModel(model_transform=lora) >>> # (set up trainer and data) >>> trainer.fit(model, data) References: ----------- Hu, E. J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., & Chen, W. (2021). LoRA: Low-Rank Adaptation of Large Language Models. arXiv preprint arXiv:2106.09685. https://arxiv.org/abs/2106.09685 ) TÚfreeze_language_modelÚfreeze_vision_modelÚmodelÚreturnNcCsng}|jr|jjjdur| |jjj¡|jr&|jjjdur&| |jjj¡|D]}| ¡D]}d|_q.q(dS)NF)rÚmoduleÚlanguage_modelÚappendrÚvision_modelÚ parametersÚ requires_grad)ÚselfrÚmodulesr Úparam©rú[/home/ubuntu/sommelier/.venv/lib/python3.10/site-packages/nemo/collections/vlm/peft/lora.pyÚfreeze_model5sÿÿzLoRA.freeze_model)Ú__name__Ú __module__Ú__qualname__Ú__doc__rÚboolÚ__annotations__rrÚModulerrrrrrs rN)ÚdataclassesrÚtorchrÚnemo.collections.llm.peft.lorarÚLLMLoRArrrrÚs