o ãÊiáã@sÖdZddlmZddlZddlmZddlmZddlm Z m Z m Z m Z ddl mZde jdejddfd ejd ejd e d ed e dedeedede fdd„Zdededeefdd„ZejjGdd„dejjƒƒZdS)z9 Utilities for scaling high precision tensors to float8. é)ÚOptionalN)ÚScalingGranularity)Útensor_already_casted_to_fp8)ÚFloat8TrainingTensorÚ GemmInputRoleÚLinearMMConfigÚhp_tensor_and_scale_to_float8)Útensor_to_scaleFÚ hp_tensorÚ float8_dtypeÚlinear_mm_configÚ reduce_amaxÚgemm_input_roleÚscaling_granularityÚ axiswise_dimÚround_scales_to_power_of_2Úreturnc Cs&t|||||||ƒ} t|| ||||ƒS)a6 Given a high precision tensor `hp_tensor`, scales `hp_tensor` dynamically and returns a `Float8TrainingTensor` of the result. Args: hp_tensor: the tensor to convert float8_dtype: the float8 dtype to use linear_mm_config: Defines the configuration for the scaled_mm for the 3 fwd/bwd gemms of linear reduce_amax: whether to reduce the max(abs(hp_tensor)) value across distributed ranks gemm_input_role: Defines the role of this tensor (input, weight or grad_output) in the 3 fwd/bwd gemms of linear scaling_granularity: Defines the scaling granularity axiswise_dim: if axiswise granularity is used, defines the dim to scale across round_scales_to_power_of_2: if true, round scaling factor down to the nearest power of 2. )r r) r r r r rÚ device_meshrrrÚscale©rúW/home/ubuntu/.local/lib/python3.10/site-packages/torchao/float8/float8_scaling_utils.pyÚhp_tensor_to_float8_dynamics"ù úrcCs|tjur|SdS)a2 Convenience function which takes in an axiswise dim which is only relevant for axiswise scaing, and a scaling type. The output is pass-through if scaling type is axiswise, and None otherwise. This is done to keep the logic from choosing the axiswise dim out of the scaling function. N)rÚAXISWISE)rrrrrÚget_maybe_axiswise_dimKs rc@s4eZdZdZededejfdd„ƒZedd„ƒZ dS) ÚNoopFwToFloat8BwDynamiczR Forward: no-op Backward: convert to float8_e5m2 with dynamic scaling r Ú target_dtypecCs||_||_|S©N)r r)ÚctxÚtensorr rrrrÚforwardaszNoopFwToFloat8BwDynamic.forwardcCs>t|ƒr |ddfSt||jƒ}t|||j|jtjƒ}|ddfSr)rr rrr rÚ GRAD_OUTPUT)rÚgradYÚ gradY_scaleÚ fp8_tensorrrrÚbackwardls  û z NoopFwToFloat8BwDynamic.backwardN) Ú__name__Ú __module__Ú __qualname__Ú__doc__Ú staticmethodrÚtorchÚdtyperr$rrrrrZsý ü r)r(Útypingrr*Útorchao.float8.configrÚ torchao.float8.distributed_utilsrÚ%torchao.float8.float8_training_tensorrrrrÚtorchao.float8.float8_utilsr ÚINPUTÚ TENSORWISEÚTensorr+ÚboolÚintrrÚ_dynamoÚallow_in_graphÚautogradÚFunctionrrrrrÚsR     ÷ÿþýüûùø ÷ ö.ÿþ ý