o ̳i @sFddlmZddlZddlmZddlmZmZGdddejZ dS))OptionalN) linear_nf4to_nf4c s`eZdZdZ   ddedededeejdeej f fd d Z d ej d ej fd dZ Z S)FrozenNF4LinearaN A linear layer similar to ``torch.nn.Linear`` but uses a quantized NF4Tensor as its weight. This class also freezes its ``weight`` parameter and is meant to be used as the base Linear layer for modeling use cases such as QLoRA where base model parameters are frozen. Args: in_dim (int): input dimension out_dim (int): output dimension bias (bool): whether to include bias in the linear layer. Default: False device (Optional[torch.device]): device to use for the underlying weight. If ``None``, uses the default device given by `torch.get_default_device()`. dtype (Optional[torch.dtype]): dtype to use for the underlying weight. If ``None``, uses the default **quantization_kwargs: Keyword arguments to pass to `to_nf4` when quantizing the base linear weight. Examples of valid arguments are `block_size` and `scaler_block_size`, which control the granularity of weight quantization and scaler quantization respectively. This is only used if `quantize_base` is True. Default None FNin_dimout_dimbiasdevicedtypec sjtj|||||d|jd|jdur|jdt|jfi|}tj|jtj j |dddS)N)rr r F) requires_grad) super__init__weightrequires_grad_rrtorchutils swap_tensorsnn Parameter)selfrrrr r quantization_kwargs nf4_weight __class__^/home/ubuntu/.local/lib/python3.10/site-packages/torchtune/modules/low_precision/nf4_linear.pyr #s   zFrozenNF4Linear.__init__inputreturncCs&t||jd}|jdur||j}|S)ak Runs linear operation with input tensor as given by `input`. Computation happens in higher precision, though only the nf4 weight is saved for backward for gradient computation to ensure additional memory is not used. Args: input (torch.Tensor): input tensor Returns: Tensor: output tensor )rrN)rrr)rroutrrrforward7s  zFrozenNF4Linear.forward)FNN)__name__ __module__ __qualname____doc__intboolrrr r r Tensorr __classcell__rrrrrs"r) typingrrtorch.nnrtorchao.dtypes.nf4tensorrrLinearrrrrrs