o ÔÙ¾iç ã@spddlmZmZddlmZmZddlZddlmZer"ddl m Z ddgZ Gdd„deƒZ Gd d„deƒZ dS) é)ÚABCÚabstractmethod)Ú TYPE_CHECKINGÚOptionalN)ÚMoeRunnerConfig)ÚStandardDispatchOutputÚBaseLinearSchemeÚ BaseMoESchemec@sXeZdZdZedd„ƒZedejjfdd„ƒZ edejjdej de ej fd d „ƒZ d S) rúu Abstract class used to describe the weight creation and forward pass of different quantization schemes. cOót‚©zU Weight creation for the particular scheme. Inputs to this function ©ÚNotImplementedError©ÚselfÚargsÚkwargs©rú^/home/ubuntu/.local/lib/python3.10/site-packages/sglang/srt/layers/quantization/base_scheme.pyÚcreate_weightsózBaseLinearScheme.create_weightsÚlayercCr ©zf Called after weight loading is complete for any cleanup that needs to occur. r ©rrrrrÚprocess_weights_after_loadingrz.BaseLinearScheme.process_weights_after_loadingÚxÚbiascCr ©ag Run the forward pass for the particular scheme. This is where scheme-specific dequant/quant steps/kernels should be applied. :param layer: torch.nn.Module with the registered weights and other parameters relevant to the particular scheme. :param x: input to the layer :param bias: bias parameter r )rrrrrrrÚ apply_weights&szBaseLinearScheme.apply_weightsN) Ú__name__Ú __module__Ú __qualname__Ú__doc__rrÚtorchÚnnÚModulerÚTensorrrrrrrrs ÿÿÿc@sfeZdZdZedd„ƒZedejjde fdd„ƒZ edejjfdd „ƒZ edejjd d fd d „ƒZ dS)r r cOr r r rrrrr=rzBaseMoEScheme.create_weightsrÚmoe_runner_configcCr )Nr )rrr'rrrÚcreate_moe_runnerEszBaseMoEScheme.create_moe_runnercCr rr rrrrrKrz+BaseMoEScheme.process_weights_after_loadingÚdispatch_outputrcCr rr )rrr)rrrrSszBaseMoEScheme.apply_weightsN) rr r!r"rrr#r$r%rr(rrrrrrr 7s$ ÿ ÿþý)ÚabcrrÚtypingrrr#Úsglang.srt.layers.moerÚ&sglang.srt.layers.moe.token_dispatcherrÚ__all__rr rrrrÚs  '