o 3wÖijã@s@ddlmZGdd„deƒZGdd„deƒZGdd„deƒZdS) é)ÚOptimizer1Statecs4eZdZ          d ‡fd d „ Z‡ZS)ÚRMSpropç{®Gáz„?ç®Gáz®ï?ç:Œ0âŽyE>rFé NéédTc sD|dkrtdƒ‚|rtdƒ‚tƒ d||||f|||| | | | ¡ dS)a‘ Base RMSprop optimizer. Arguments: params (`torch.tensor`): The input parameters to optimize. lr (`float`, defaults to 1e-2): The learning rate. alpha (`float`, defaults to 0.99): The alpha value is the decay rate of the squared gradients of the optimizer. eps (`float`, defaults to 1e-8): The epsilon value prevents division by zero in the optimizer. weight_decay (`float`, defaults to 0.0): The weight decay value for the optimizer. momentum (`float`, defaults to 0): The momentum value speeds up the optimizer by taking bigger steps. centered (`bool`, defaults to `False`): Whether the gradients are normalized by the variance. If `True`, it can help training at the expense of additional compute. optim_bits (`int`, defaults to 32): The number of bits of the optimizer state. args (`object`, defaults to `None`): An object with additional arguments. min_8bit_size (`int`, defaults to 4096): The minimum number of elements of the parameter tensors for 8-bit optimization. percentile_clipping (`int`, defaults to 100): Adapts clipping threshold automatically by tracking the last 100 gradient norms and clipping the gradient at a certain percentile to improve stability. block_wise (`bool`, defaults to `True`): Whether to independently quantize each block of tensors to reduce outlier effects and improve stability. rú)RMSprop with alpha==0.0 is not supported!ú"Centered RMSprop is not supported!ÚrmspropN©ÚNotImplementedErrorÚsuperÚ__init__) ÚselfÚparamsÚlrÚalphaÚepsÚ weight_decayÚmomentumÚcenteredÚ optim_bitsÚargsÚ min_8bit_sizeÚpercentile_clippingÚ block_wise©Ú __class__©úW/home/ubuntu/sommelier/.venv/lib/python3.10/site-packages/bitsandbytes/optim/rmsprop.pyr ó",õzRMSprop.__init__) rrrrrFrNrr T©Ú__name__Ú __module__Ú __qualname__rÚ __classcell__r r rr!rsórcó2eZdZ          d ‡fd d „ Z‡ZS) Ú RMSprop8bitrrrrFNrr Tc óD|dkrtdƒ‚|rtdƒ‚tƒ d||||f||d|| | | ¡ dS)a’ 8-bit RMSprop optimizer. Arguments: params (`torch.tensor`): The input parameters to optimize. lr (`float`, defaults to 1e-2): The learning rate. alpha (`float`, defaults to 0.99): The alpha value is the decay rate of the squared gradients of the optimizer. eps (`float`, defaults to 1e-8): The epsilon value prevents division by zero in the optimizer. weight_decay (`float`, defaults to 0.0): The weight decay value for the optimizer. momentum (`float`, defaults to 0): The momentum value speeds up the optimizer by taking bigger steps. centered (`bool`, defaults to `False`): Whether the gradients are normalized by the variance. If `True`, it can help training at the expense of additional compute. optim_bits (`int`, defaults to 32): The number of bits of the optimizer state. args (`object`, defaults to `None`): An object with additional arguments. min_8bit_size (`int`, defaults to 4096): The minimum number of elements of the parameter tensors for 8-bit optimization. percentile_clipping (`int`, defaults to 100): Adapts clipping threshold automatically by tracking the last 100 gradient norms and clipping the gradient at a certain percentile to improve stability. block_wise (`bool`, defaults to `True`): Whether to independently quantize each block of tensors to reduce outlier effects and improve stability. rr r r éNr © rrrrrrrrrrrrrr r!rIs"+õzRMSprop8bit.__init__© rrrrrFNrr Tr#r r rr!r)Hóôr)cr() Ú RMSprop32bitrrrrFNrr Tc r*)a“ 32-bit RMSprop optimizer. Arguments: params (`torch.tensor`): The input parameters to optimize. lr (`float`, defaults to 1e-2): The learning rate. alpha (`float`, defaults to 0.99): The alpha value is the decay rate of the squared gradients of the optimizer. eps (`float`, defaults to 1e-8): The epsilon value prevents division by zero in the optimizer. weight_decay (`float`, defaults to 0.0): The weight decay value for the optimizer. momentum (`float`, defaults to 0): The momentum value speeds up the optimizer by taking bigger steps. centered (`bool`, defaults to `False`): Whether the gradients are normalized by the variance. If `True`, it can help training at the expense of additional compute. optim_bits (`int`, defaults to 32): The number of bits of the optimizer state. args (`object`, defaults to `None`): An object with additional arguments. min_8bit_size (`int`, defaults to 4096): The minimum number of elements of the parameter tensors for 8-bit optimization. percentile_clipping (`int`, defaults to 100): Adapts clipping threshold automatically by tracking the last 100 gradient norms and clipping the gradient at a certain percentile to improve stability. block_wise (`bool`, defaults to `True`): Whether to independently quantize each block of tensors to reduce outlier effects and improve stability. rr r r rNr r,rr r!rˆr"zRMSprop32bit.__init__r-r#r r rr!r/‡r.r/N)Úbitsandbytes.optim.optimizerrrr)r/r r r r!Ús @?