o iz @s6ddlZddlmZddlmZmZddlZddlmZddl m Z ddl m Z ddl mZmZmZddlmZmZdd lmZd d ZeGd d d e ZeZeedejjdefddZGddde ZeZeedejjdedejjfddZ ddejjde deeejjege fdejjfddZ!dS)N) dataclass)CallableOptional)to_sparse_semi_structured) AOBaseConfig)WeightNormSparsifier) _is_linear_linear_extra_repr)_replace_with_custom_fn_if_matches_filter)_QUANTIZE_CONFIG_HANDLER register_quantize_module_handler)BlockSparseTensorcKsn|dt}g}|D]\}}|||r|d|diq tdddd}|||||dS) zO This function simulates 2:4 sparsity on all linear layers in a model. filter_fn tensor_fqnz.weightg?))sparsity_levelsparse_block_shapezeros_per_blockN)popr named_modulesappendrpreparestep squash_mask)modelkwargsr sparse_confignamemod sparsifierr"O/home/ubuntu/.local/lib/python3.10/site-packages/torchao/sparsity/sparse_api.pyapply_fake_sparsitys    r$c@s"eZdZUdZeed<ddZdS)BlockSparseWeightConfig@ blocksizecCtjddS)Nz(torchao.sparsity.BlockSparseWeightConfigtorch_C_log_api_usage_onceselfr"r"r# __post_init__5z%BlockSparseWeightConfig.__post_init__N)__name__ __module__ __qualname__r'int__annotations__r/r"r"r"r#r%1s  r%moduleconfigcCs8|j}t|j|}tjj|dd|_tt ||_ |SNF) requires_grad) r'r from_denseweightr*nn Parametertypes MethodTyper extra_repr)r6r7r' new_weightr"r"r#_block_sparse_weight_transform=s rBc@seZdZdZddZdS)SemiSparseWeightConfigze Configuration for converting the weight of linear modules to semi-structured (2:4) sparsity cCr()Nz'torchao.sparsity.SemiSparseWeightConfigr)r-r"r"r#r/Nr0z$SemiSparseWeightConfig.__post_init__N)r1r2r3__doc__r/r"r"r"r#rCIs rCreturncCs.t|j}tjj|dd|_tt||_|Sr8) rr;r*r<r=r>r?r r@)r6r7rAr"r"r#_semi_sparse_weight_transformVs rFrrcCs:tjdtt|}t|||durtn||fddS)aConvert the weight of linear modules in the model with `apply_tensor_subclass`. This function is essentially the same as quantize, put for sparsity subclasses. Currently, we support three options for sparsity: - semi-structured (2:4) sparsity with `semi_sparse_weight` - int8 dynamic quantization + 2:4 sparsity with `layout=SemiSparseLayout` Args: model (torch.nn.Module): input model config (AOBaseConfig): a workflow configuration object filter_fn (Optional[Callable[[torch.nn.Module, str], bool]]): function that takes a nn.Module instance and fully qualified name of the module, returns True if we want to apply the specified workflow to this module. **Example:** :: import torch import torch.nn as nn from torchao.sparsity import sparsify_ def filter_fn(module: nn.Module, fqn: str) -> bool: return isinstance(module, nn.Linear) m = nn.Sequential(nn.Linear(32, 1024), nn.Linear(1024, 32)) # for 2:4 sparsity from torchao.sparse_api import semi_sparse_weight m = sparsify_(m, semi_sparse_weight(), filter_fn) # for int8 dynamic quantization + 2:4 sparsity from torchao.dtypes import SemiSparseLayout m = quantize_(m, Int8DynamicActivationInt8WeightConfig(layout=SemiSparseLayout), filter_fn) ztorchao.sparsity.sparsify_N) extra_args)r*r+r,r typer r)rr7rhandlerr"r"r# sparsify_as $  rJ)N)"r> dataclassesrtypingrrr* torch.sparsertorchao.core.configrsP