o 8wiF@s\ddlZddlmZddlmZddlZddlmZddlm Z m Z Gdddej e Z dS)N)deepcopy)Optional)BaseTunerLayercheck_adapters_to_mergecseZdZdZdZdejdeffdd Zdejdefdd Z d e d d fd dZ dde e ede fddZddZdejd ejfddZd effdd ZZS) LNTuningLayerz) Selects a layer from the model. )ln_tuning_layers base_layer adapter_namecs:t||_ti|_||j|||_g|_dSN) super__init__rnn ModuleDictr update_layer_active_adaptermerged_adapters)selfrr  __class__X/home/ubuntu/sommelier/.venv/lib/python3.10/site-packages/peft/tuners/ln_tuning/layer.pyr s   zLNTuningLayer.__init__layercCst||j|<dSr )rr)rrr rrrr(szLNTuningLayer.update_layerenabledreturnNcCsR|r ||jd|_dS|jr||jD] }t||}|dqd|_dS)zToggle the enabling and disabling of adapters Takes care of setting the requires_grad flag for the adapter weights. Args: enabled (bool): True to enable adapters, False to disable adapters FTN) set_adapteractive_adapters_disable_adaptersmergedunmergeadapter_layer_namesgetattrrequires_grad_)rr layer_namerrrrenable_adapters+s      zLNTuningLayer.enable_adaptersF adapter_names safe_mergecCst||}|s dSt|dkrtdt|dt|j}|r-td|d||j|d|j |_ |j|d<|j |ddS)NzTrying to merge zO adapters, but LN tuning does not allow merging more than one adapter at a timezAlready merged with z. Unmerging first.r) rlen ValueErrorsetrwarningswarnrrrappend)rr$r%rrrrmerge?s    zLNTuningLayer.mergecCs<|js tddS|j}|j||j|_|j|<dS)Nz Already unmerged. Nothing to do.)rr*r+rpoprr)r merged_namerrrrVs  zLNTuningLayer.unmergexcOs|jr|jr ||j|g|Ri|}|S|jr)|j|g|Ri|}|St|jdkr;tdt|jd|jd}|j||g|Ri|}|S)Nr&zTrying to run forward with z] active adapters, but LN tuning does not allow inference with more than one adapter at a timer)disable_adaptersrrrr'rr(r)rr0argskwargsresultactive_adapterrrrforwardbs   zLNTuningLayer.forwardcst}d|S)Nz ln_tuning.)r __repr__)rreprrrr7ts zLNTuningLayer.__repr__)NF)__name__ __module__ __qualname____doc__rr Modulestrr rboolr#rlistr-rtorchTensorr6r7 __classcell__rrrrrs r) r*copyrtypingrrAtorch.nnr peft.tuners.tuners_utilsrrr=rrrrrs