o 5ti& @sddlmZddlmZddlmZddlmZmZm Z m Z m Z ddl Z ddl mZddl mZmZddlmZdd lmZdd lmZed e eefd eeefd e dedffddZedGdddeZdS))contextmanager)partial)Path)AnyCallable GeneratorOptionalUnionN) PeftModel)Tensornn)PreTrainedModel)register_model)HFLMmodel hook_to_steerreturnc #sdtffdd }g}t}|jD]\}}||vr+|||}||qt|t|kr;td|zdVW|D]}| qBdS|D]}| qMw)z Context manager that temporarily hooks models and steers them. Args: model: The transformer model to hook hook_to_steer: Dictionary mapping hookpoints to steering functions Yields: None hookpointcs"dtjdtdtffdd }|S)Nmoduleinputoutputcs>t|tr|dg|ddR}|S|}|S)Nr) isinstancetuple)rrr)rrM/home/ubuntu/.local/lib/python3.10/site-packages/lm_eval/models/hf_steered.pyhook_fns   z+steer..create_hook..hook_fn)r Modulerr )rrr)rr create_hooks zsteer..create_hookz&Not all hookpoints could be resolved: N) strlistkeys base_model named_modulesregister_forward_hookappendlen ValueErrorremove)rrrhandles hookpointsnamerhandlerrrsteers$    r.steeredc seZdZUeeefed< ddededeeffdd Ze defdd Z e d e d e d ee fd dZ e dd e de ded ee dee f ddZddZfddZfddZZS) SteeredModelrN pretrained steer_pathdevicec s|tjd||d||ds|dr3t|d}tj|dd}Wdn1s-wYn|dr>||}ntd |i}|D]m\}} | d } | d  |j  |j j } t | d d } | dd} | dd}|dur| |j  |j j }| dkr|dusJdt|j| | | d||<qK| dkrt|j| t| | || d||<qKtd| ||_dS)a HFLM with a steered forward pass. To load steering vectors directly, provide the path to a pytorch (.pt) file with content in the following format: { hookpoint: { "steering_vector": , "steering_coefficient": , "action": , "bias": , "head_index": , }, ... } To derive steering vectors from a sparse model loadable with sparsify or sae_lens, provide the path to a CSV file with the following columns (example rows are provided below): loader,action,sparse_model,hookpoint,feature_index,steering_coefficient,head_index,sae_id,description, sparsify,add,EleutherAI/sae-pythia-70m-32k,layers.3,30,10.0,,,, sae_lens,add,gemma-scope-2b-pt-res-canonical,layers.20,12082,240.0,,layer_20/width_16k/canonical,increase dogs, )r1r3z.ptz.pthrbT) weights_onlyNz.csvzUnknown steer file type: actionsteering_vectorsteering_coefficientg? head_indexbiasaddz+Bias is not supported for the `add` action.)vectorr9clamp) directionvaluer:r9zUnknown hook type: r)super__init__endswithopentorchloadderive_steer_configr(itemstor3rdtypefloatgetrr;r=normr)selfr1r2r3kwargsf steer_configrr steer_infor6r7r8r9r: __class__rrrA@sL         zSteeredModel.__init__csvddl}||}i}t|ddkrddlm}t|ddkr5ddlmidtd tffd d }|D]\}}| d d } |d} |d} t |d} t |d} | dd}|dkrt | }| rp||| n|| | }|jdus}J|j| }|j}n)|dkr|| |d d}|j| }|j}| dks|| r|jj} ntd|| | ||d|| <q9|S)zVDerive a dictionary of steering vectors from sparse model(/s) specified in a CSV file.rNloadersparsify) SparseCodersae_lens)SAE sae_releasesae_idcs,||f}|vr||d|<|S)Nr)from_pretrained)rYrZ cache_keyrX sae_cacherrload_from_sae_lenssz.load_from_sae_lensr6r; sparse_modelr feature_indexr8)rYrZzUnknown loader: )r6r8r7r:)pandasread_csvanyrUrVrWrXr iterrowsrKintrJrexistsload_from_disk load_from_hubW_decb_decisnacfg hook_namer()clsr2pddf steer_datarVr__rowr6 sparse_namerrar8rT name_path sparse_coderr7r:rr]rrFsR           z SteeredModel.derive_steer_configactsr<r9cCsP|dur"|dddd|ddf||dddd|ddf<|S||}|S)a.Adds the given vector to the activations. Args: acts (Tensor): The activations tensor to edit of shape [batch, pos, ..., features] vector (Tensor): A vector to add of shape [features] head_index (int | None): Optional attention head index to add to Nr)rpryr<r9rrrr;s 8zSteeredModel.addr>r?r:c Cs|dur||}|durD|dddd|ddf}||jddd}|||ks+J|}|||||dddd|ddf<ntj||ddd}||||}|dur^||S|S)acClamps the activations to a given value in a specified direction. The direction must be a unit vector. Args: acts (Tensor): The activations tensor to edit of shape [batch, pos, ..., features] direction (Tensor): A direction to clamp of shape [features] value (float): Value to clamp the direction to head_index (int | None): Optional attention head index to clamp bias (Tensor | None): Optional bias to add to the activations Returns: Tensor: The modified activations with the specified direction clamped NT)dimkeepdim)sumclonerD) rpryr>r?r9r:xprojclampedrrrr=s*zSteeredModel.clampc Os|t0t|j|j|jj|i|WdWdS1s'wYWddS1s7wYdSN)rDno_gradr.rrforwardrMargsrNrrrr s "zSteeredModel.forwardcDt|j|jtj|i|WdS1swYdSr)r.rrr@ _model_callrrRrrr$zSteeredModel._model_callcrr)r.rrr@_model_generaterrRrrrrzSteeredModel._model_generater)__name__ __module__ __qualname__dictr r__annotations__rrA classmethodrFr rgr;rJr=rrr __classcell__rrrRrr0<sH L>  ( r0) contextlibr functoolsrpathlibrtypingrrrrr rDpeft.peft_modelr r r transformersr lm_eval.api.registryrlm_eval.models.huggingfacerrr r.r0rrrrs(          ,