o ‚o™i=ã@s@ddlmZGdd„deƒZGdd„deƒZGdd„deƒZdS) é)ÚOptimizer1Statecs0eZdZ         d ‡fdd „ Z‡ZS) ÚSGDrFé NéédTc s8|dkrtdƒ‚tƒ d||||fd|||| | | ¡ dS)a• Base SGD optimizer. Arguments: params (`torch.tensor`): The input parameters to optimize. lr (`float`): The learning rate. momentum (`float`, defaults to 0): The momentum value speeds up the optimizer by taking bigger steps. dampening (`float`, defaults to 0): The dampening value reduces the momentum of the optimizer. weight_decay (`float`, defaults to 0.0): The weight decay value for the optimizer. nesterov (`bool`, defaults to `False`): Whether to use Nesterov momentum. optim_bits (`int`, defaults to 32): The number of bits of the optimizer state. args (`object`, defaults to `None`): An object with additional arguments. min_8bit_size (`int`, defaults to 4096): The minimum number of elements of the parameter tensors for 8-bit optimization. percentile_clipping (`int`, defaults to 100): Adapts clipping threshold automatically by tracking the last 100 gradient norms and clipping the gradient at a certain percentile to improve stability. block_wise (`bool`, defaults to `True`): Whether to independently quantize each block of tensors to reduce outlier effects and improve stability. rú&SGD without momentum is not supported!ÚmomentumçN©ÚNotImplementedErrorÚsuperÚ__init__) ÚselfÚparamsÚlrrÚ dampeningÚ weight_decayÚnesterovÚ optim_bitsÚargsÚ min_8bit_sizeÚpercentile_clippingÚ block_wise©Ú __class__©úJ/home/ubuntu/.local/lib/python3.10/site-packages/bitsandbytes/optim/sgd.pyr s)õz SGD.__init__) rrrFrNrrT©Ú__name__Ú __module__Ú __qualname__r Ú __classcell__rrrrrsôrcó.eZdZ        d ‡fdd„ Z‡ZS) ÚSGD8bitrFNrrTc ó8|dkrtdƒ‚tƒ d||||fd|d||| | ¡ dS)a+ 8-bit SGD optimizer. Arguments: params (`torch.tensor`): The input parameters to optimize. lr (`float`): The learning rate. momentum (`float`, defaults to 0): The momentum value speeds up the optimizer by taking bigger steps. dampening (`float`, defaults to 0): The dampening value reduces the momentum of the optimizer. weight_decay (`float`, defaults to 0.0): The weight decay value for the optimizer. nesterov (`bool`, defaults to `False`): Whether to use Nesterov momentum. args (`object`, defaults to `None`): An object with additional arguments. min_8bit_size (`int`, defaults to 4096): The minimum number of elements of the parameter tensors for 8-bit optimization. percentile_clipping (`int`, defaults to 100): Adapts clipping threshold automatically by tracking the last 100 gradient norms and clipping the gradient at a certain percentile to improve stability. block_wise (`bool`, defaults to `True`): Whether to independently quantize each block of tensors to reduce outlier effects and improve stability. rrrr éNr © rrrrrrrrrrrrrrr Dó&õzSGD8bit.__init__©rrrFNrrTrrrrrr#Cóõr#cr") ÚSGD32bitrFNrrTc r$)a, 32-bit SGD optimizer. Arguments: params (`torch.tensor`): The input parameters to optimize. lr (`float`): The learning rate. momentum (`float`, defaults to 0): The momentum value speeds up the optimizer by taking bigger steps. dampening (`float`, defaults to 0): The dampening value reduces the momentum of the optimizer. weight_decay (`float`, defaults to 0.0): The weight decay value for the optimizer. nesterov (`bool`, defaults to `False`): Whether to use Nesterov momentum. args (`object`, defaults to `None`): An object with additional arguments. min_8bit_size (`int`, defaults to 4096): The minimum number of elements of the parameter tensors for 8-bit optimization. percentile_clipping (`int`, defaults to 100): Adapts clipping threshold automatically by tracking the last 100 gradient norms and clipping the gradient at a certain percentile to improve stability. block_wise (`bool`, defaults to `True`): Whether to independently quantize each block of tensors to reduce outlier effects and improve stability. rrrr rNr r&rrrr |r'zSGD32bit.__init__r(rrrrrr*{r)r*N)Úbitsandbytes.optim.optimizerrrr#r*rrrrÚs ;8