o Xi @sddlZddlmZeeZdejddfddZdejddfddZdejddfd d Z dejddfd d Z d ej ddfddZ dS)N)irvaluereturncCs(|jtjjkr dStjj|_t|dSN)dtyperDataTypeBFLOAT16FLOAT164_insert_cast_nodes_for_float16_to_bfloat16_to_inputsrr u/home/ubuntu/.local/lib/python3.10/site-packages/onnxscript/rewriter/onnxruntime/bfloat16_utils/bfloat16_converter.py(_convert_inputs_from_bfloat16_to_float16 s  rcCs|jtjjkr dSt|dSr)rrrr5_insert_cast_nodes_for_bfloat16_to_float16_to_outputsr r r r )_convert_outputs_from_bfloat16_to_float16s rcCst|}tjdtjjd}tjdd|gd|gd}tjj|jd_|j |jd_ t|D]\}}||ur9q0| ||jdq0|ddj dusPJd|ddj d |dS) NtonamerCastdomainop_typeinputs num_outputs attributesrz!The node should belong to a graph) tupleusesr AttrInt64rrNodeoutputsrshapereplace_input_withgraphprepend)ruser_nodes_and_indicesattrcastnodeindexr r r r s" r cCs|}|}|dus|durtd|dStjdtjjd}tjdd|j |gd|gd}tjj|j d_ |j |j |j d_ | ||j dusPJd t|j j D]\}}||j |krj|j d|j j |<qV|j dj|j |j|j |_|j d_dS) Nz(Output value %s has no producer or indexrrrrrrrzNode graph should not be None)producerr*loggerwarningrrrr r r!rr"appendr$ enumerater)rr)r*r'r(idxgraph_or_function_outputr r r r0s0     rmodelcCs0|jjD]}t|q|jjD]}t|qdS)a Adapt the model datatype if it's bfloat16. Because onnxruntime does not support bfloat16 as input/output datatype, we need to convert the bfloat16 datatype to float16. This function will convert the bfloat16 datatype to float16 and insert Cast nodes to convert float16 to bfloat16. Model: inputs(float16) -> Cast(bfloat16) -> nodes(bfloat16) -> Cast(float16) -> outputs(float16) TODO: Delete this function after onnxruntime supports bfloat16. Args: model: The model to adapt. N)r$rrr!r)r2inputoutputr r r dtype_adapter_for_bfloat16_modelPs    r5) logging onnxscriptr getLogger__name__r,Valuerrr rModelr5r r r r s