o *i@sddlZddlmZmZddlZddlmZddlmZm Z m Z ddl m Z ddl mZddlmZddlmZdd lmZdd lmZGd d d eZdS) N)DictTuple)BaseCompressor)QuantizationSchemeQuantizationStatus"initialize_module_for_quantization)register_offload_parameter)get_execution_device)Tensor) Parameter)linear)LinearcsVeZdZdZdfdd Zeedede de fd d Z d e de fd d Z ZS)CompressedLinearz Wrapper module for running a compressed forward pass of a quantized Linear module. The wrapped layer will decompressed on each forward call. returnNcs"tj|i|tdtdS)NzXCompressedLinear should not be initialized directly. Use the from_linear method instead.)super__init__warningswarn UserWarning)selfargskwargs __class__h/home/ubuntu/veenaModal/venv/lib/python3.10/site-packages/compressed_tensors/linear/compressed_linear.pyr(s zCompressedLinear.__init__modulequantization_schemequantization_formatc Cst|_t||_t|}t||dd|j|jj |j }t |d| D]\}\}}t tj|||ddd} t||| q'tj|_t|drQtj|t|_|S)a :param module: dense linear module to replace :param quantization_scheme: quantization config for the module to wrap :param quantization_format: compression format module is stored as :return: CompressedLinear module wrapping the input module F)force_zero_pointweight)devicedtype requires_grad _old_forward)rrrload_from_registry compressorr rcompression_param_infor shapeweightsdelattritemsr torchemptyrr COMPRESSEDquantization_statushasattrforward__get__r%) clsrrr init_devicecompression_paramsnamer)r"paramrrr from_linear0s*    zCompressedLinear.from_linearinputcCsH|jtjkr|j|}t|dd}t|d|tj|_t||j |j S)zM Decompresses the weight, then runs the wrapped forward pass Fr#r ) r0rr/r'decompress_moduler rFROZENr r bias)rr: weight_datar8rrrr2bs    zCompressedLinear.forward)rN)__name__ __module__ __qualname____doc__r classmethodr-no_gradr rstrr9r r2 __classcell__rrrrr!s0r)rtypingrrr-#compressed_tensors.compressors.basercompressed_tensors.quantizationrrrcompressed_tensors.utilsr compressed_tensors.utils.offloadr r torch.nnr torch.nn.functionalr torch.nn.modulesr rrrrrs