o d۷i$@sddlmZmZddlmZmZmZmZmZddl Z ddl m Z ddl m Z mZddlmZddlmZddl mZdd lmZd gZGd d d eeZdS) )ABCabstractmethod)Dict GeneratorOptionalTupleUnionN)SparsityCompressionConfig)QuantizationArgsQuantizationConfig) RegistryMixin)has_offloaded_params)Tensor)ModuleBaseCompressorc @s6eZdZdZ d deeedffddZ d dej de e de e eej ejfffd d Zeedee fd d Zed e e efde e effddZe d!de de deee efddffddZdede e e ejffddZdede e ejffddZdefddZde e efdejfddZdS)"ra Base class representing a model compression algorithm. Each child class should implement compression_param_info, compress_weight and decompress_weight. Compressors support compressing/decompressing a full module state dict or a single quantized PyTorch leaf module. Model Load Lifecycle (run_compressed=False): - ModelCompressor.decompress() - apply_quantization_config() - BaseCompressor.decompress() Model Save Lifecycle: - ModelCompressor.compress() - BaseCompressor.compress() Module Lifecycle (run_compressed=True): - apply_quantization_config() - compressed_module = CompressedLinear(module) - initialize_module_for_quantization() - BaseCompressor.compression_param_info() - register_parameters() - compressed_module.forward() -compressed_module.decompress() :param config: config specifying compression parameters NconfigcCs ||_dSN)r)selfrrY/home/ubuntu/vllm_env/lib/python3.10/site-packages/compressed_tensors/compressors/base.py__init__=s zBaseCompressor.__init__ weight_shapequantization_argsreturncCt)aY Creates a dictionary of expected shapes and dtypes for each compression parameter used by the compressor :param weight_shape: uncompressed weight shape :param quantization_args: quantization parameters for the weight :return: dictionary mapping compressed parameter names to shape and dtype NotImplementedError)rrrrrrcompression_param_infoB z%BaseCompressor.compression_param_infocCr)zx Returns a tuple of compression parameter names introduced by the compressor during compression r)rrrrcompression_param_namesQsz&BaseCompressor.compression_param_names model_statecKr)z Compresses a dense state dict :param model_state: state dict of uncompressed model :param kwargs: additional arguments for compression :return: compressed state dict r)rr kwargsrrrcompressZrzBaseCompressor.compresscpupath_to_model_or_tensorsdevicecKr)a Reads a compressed state dict located at path_to_model_or_tensors and returns a generator for sequentially decompressing back to a dense state dict :param path_to_model_or_tensors: path to compressed safetensors model (directory with one or more safetensors files) or compressed tensors file :param names_to_scheme: quantization args for each quantized weight :param device: optional device to load intermediate weights into :return: compressed state dict r)rr$r%r!rrr decompressiszBaseCompressor.decompressmodulecCs^t|dsdS|j}t|dsdS|j}t|dd}t|dd}t|dd}|j||||dS)a" Compresses a single quantized leaf PyTorch module. If the module is not quantized, this function has no effect. :param module: PyTorch module to compress :return: dictionary of compressed weight data, or None if module is not quantized quantization_schemeNweightsweight weight_scaleweight_zero_point)r*scale zero_pointr)hasattrr(r)getattrcompress_weight)rr'r(rr*r+r,rrrcompress_module}s     zBaseCompressor.compress_moduler*cKr)z Compresses a single uncompressed weight :param weight: uncompressed weight tensor :param kwargs: additional arguments for compression r)rr*r!rrrr1s zBaseCompressor.compress_weightc Csxt|j}t|r dn|}t|dsdS|j}t|ds dS|j}i}|D]\}}|||<q)|j||d |S)a Decompresses a single compressed leaf PyTorch module. If the module is not quantized, this function has no effect. :param module: PyTorch module to decompress :return: tensor of the decompressed weight, or None if module is not quantized r#r(Nr))compressed_datar) next parametersr%r r/r(r)named_parametersdecompress_weightto) rr' params_devicer%r(rr3name parameterrrrdecompress_modules     z BaseCompressor.decompress_moduler3cKr)z Decompresses a single compressed weight :param compressed_data: dictionary of data needed for decompression :param kwargs: additional arguments for decompression :return: tensor of the decompressed weight r)rr3r!rrrr7s z BaseCompressor.decompress_weightr)r#)__name__ __module__ __qualname____doc__rr r rtorchSizerr rstrrdtyperpropertyrrrr"rr&rr2r1r<r7rrrrrsX           )abcrrtypingrrrrrrAcompressed_tensors.configr compressed_tensors.quantizationr r compressed_tensors.registryr compressed_tensors.utilsr rtorch.nnr__all__rrrrrs