o XiEJ@sDdZddlmZddlmZgdZddlZddlZddlZddl m Z m Z ddl m Z mZmZddl mZdd lmZd Zd Zd ZeeZejGd ddZejGdddZ dIdJddZdKddZdLd!d"ZeeefdMd)d*Z dNd+d,Z! dOdPd4d5Z"dQd9d:Z#dRd   r:base_dirstr | os.PathLikeNonecCs(t|ddD] }t|tjr||_qdS)zSet the base directory for external data in a graph (including all of its subgraphs). Args: graph: The graph to traverse tensors on. base_dir: The base directory. This is the directory where the ONNX file is. T)r$N)r: isinstancer ExternalTensorr;)r"r;r6rrrrms  rr6_protocols.TensorProtocolcCsFt|tjstdt||}|tj||j |j dS)aCConvert an external tensor to an in memory tensor. Args: tensor: An external tensor to load. base_dir: Path of base directory. relative_path: Path to which external data is to be stored, relative to the ONNX file. Returns: An ir.Tensor object with the data loaded into memory. zExpected ExternalTensor, got )rdtype) r>r r? TypeErrorr/numpycopyreleaseTensorrrA)r6 tensor_datarrr!_external_tensor_to_memory_tensorys  rHcurrent_offsetr tensor_size align_offsetalign_thresholdallocation_granularitycCs.|r||krtd|}||d||S|S)aiCompute the offset to align the tensor data based on the current offset. Args: current_offset: Current location in the file at which tensor data will be written to. tensor_size: Size of the tensor data to be written to file. align_offset: Offset will always be page aligned and alloction granularity aligned for mmap support. This is done by padding previous tensor data with zeros keeping same length. Tensor data will be aligned if > align_threshold align_threshold: Alignment threshold for size of data. Having a low threshold will waste file space for small initializers. Only when tensor's data is > the page_align_threshold it will be force aligned. allocation_granularity: The allocation Granularity for mmap() support. Typically 64KB for Windows & 4KB for other OSes. Returns: The updated offset value. i)max)rIrJrKrLrMalignment_factorrrr_compute_new_offsets  rQcCs"|j}t||}t|j||}|S)zICapture information about a tensor that is to be stored as external data.)nbytesrQrr)r6rIrJexternal_data_inforrr_compute_external_data_infos rTtensors#Sequence[_protocols.TensorProtocol]external_data_infosSequence[_ExternalDataInfo] file_pathcallback@Callable[[_protocols.TensorProtocol, CallbackInfo], None] | Nonec Cst|}|t|ksJdt|de}tt||ddD]R\}\}}|dur7||t|||jtj|d|j} |dus@J| } | | krQ| d| | t |dr\| |q| } t|tjrj|| | qWddS1s{wYdS) aWrite tensor data to an external file according to information stored in ExternalDataInfo objects. Args: tensors: Tensors to be written as external data. external_data_infos: External data information stored for each tensor to be written as external data. file_path: Location to which external data is to be stored. callback: A callback function that is called for each tensor that is saved to external data for debugging or logging purposes. z6Number of tensors and external data infos should matchwbTstrictN)rrrr!tofile)lenopen enumeraterr rospathbasenametellwritehasattrr`tobytesr>r r?rE) rUrWrYrZ tensors_count data_fileir6 tensor_inforI file_sizeraw_datarrr_write_external_datas>        "rqrS relative_path_core.ExternalTensorc Cs2tjtj||j|j|j|j|j tj|dS)aCreate external tensors from external data information. Args: tensor: Tensor to be converted to external tensor. external_data_info: External data information stored for the tensor to be written as external data. base_dir: Path of base directory. relative_path: Path to which external data is to be stored, relative to the ONNX file. Returns: External tensor created from the information. )shaperr;) r r?rdrenormpathrrrArtr)r6rSr;rrrrr_create_external_tensors  rvlist[_protocols.TensorProtocol]cCsdd|DS)zConvert a sequence of external tensors to in-memory tensors. Args: tensors: External tensors to be converted to in-memory tensors. Returns: A list of in-memory tensors derived from a list of external tensors. cSsg|]}t|qSr)rH).0r6rrr z1convert_tensors_from_external..rrUrrrrs rlist[_core.ExternalTensor]c s@tj}tj|rBg}D].}t|tjr:tj|jr:tj||jr:|t || t d|q||q|g}t ttfdddfddD}d} |D]}t|| } || | j| j} q_t||||dfd dt||d d Dfd dt ttfd ddDS)aConvert a sequence of any TensorProtocol tensors to external tensors. Existing external tensors are loaded to memory if they are referring to the same file path as the destination path. Args: tensors: Tensors to be converted to external tensors. They can be external tensors themselves. base_dir: Path of base directory. relative_path: Path to which external data is to be stored, relative to the ONNX file. callback: A callback function that is called for each tensor that is saved to external data for debugging or logging purposes. Returns: A list of external tensors derived from a list of input tensors. The order should match the input tensor order. zExternal tensor %s is referring to the same file as the destination path. It has been invalidated because the data file is changed. To avoid this, save the external data to a different path or load the newly saved model back with ir.load().cs |jSN)rRrmr{rrMs z-convert_tensors_to_external..)keycg|]}|qSrrrxrmr{rrryNrzz/convert_tensors_to_external..r)rZcsg|] \}}t||qSr)rv)rxr6 external_info)r;rrrrryYs Tr]crrrr)external_tensorsrrry_scs|Sr}rr~)sorted_indicesrrras)rdrejoinexistsr>r r?samefileappendrH invalidateloggerwarningsortedrangerarTrrrqr) rUr;rrrZre new_tensorsr6rWsorted_tensorsrIrr)r;rrrrrUrrsD         rmodel _core.ModelcCszg}|D]}|jD]}|jdurq t|jtjr!||q qtdd|D}t ||ddD]\}}||_q3|S)zConvert all external model initializers to memory tensors in-place. All initializers in the main graph and subgraphs are handled. Args: model: Model to process. NcSg|]}|jqSrr*rxvrrrryyz!load_to_model..Tr]) graphsr(r)r*r>r r?rrr)rvalues_to_convertr"r2loaded_tensorsr6rrrrgs    r)size_threshold_bytesrZrc Csg}g}|D](}|jD] }|jdurq|jj|kr#||qt|jtjr/||qqt dd|D} t dd|D|||d} t || ddD]\}} | |_qNt || ddD]\}} | |_q]|S)aConvert all initializers equal or above size_threshold_bytes to external tensors in-place and save data to a single data file. It should only replace the initializers in the model with external tensors and not make any other modifications to the model. If any existing external tensor references the provided ``external_data`` path, it will be invalidated after the external data is overwritten. To obtain a valid model, use :func:`load` to load the newly saved model, or provide a different external data path that is not currently referenced by any tensors in the model. All initializers in the main graph and subgraphs are handled. Args: model: Model to process. base_dir: Path the directory where the ONNX model file is. relative_path: Path to which external data is to be stored, relative to the ONNX file. E.g. "model.data" size_threshold_bytes: Save to external data if the tensor size in bytes is larger than this threshold. callback: A callback function that is called for each tensor that is saved to external data for debugging or logging purposes. Returns: An ir.Model with all initializer data equal or above ``size_threshold_bytes`` converted to external tensors. NcSrrrrrrrryrz%unload_from_model..cSrrrrrrrryr)r;rrrZTr]) rr(r)r*rRrr>r r?rrr) rr;rrrrZinitializers_to_become_externalinitializers_to_load_to_memoryr"r2memory_tensorsrexternal_tensor memory_tensorrrrrs<#       r)F)r"r#r$r%r&r')r"r#r;r<r&r=)r6r@r&r@) rIrrJrrKr%rLrrMrr&r)r6r@rIrr&rr}) rUrVrWrXrYr<rZr[r&r=) r6r@rSrr;r<rrr<r&rs)rUrVr&rw) rUrVr;r<rrr<rZr[r&r|)rrr&r) rrr;r<rrr<rrrZr[r&r)(r __future__rtypingr__all__ dataclassesloggingrdcollections.abcr r onnx_irr r rrr+onnx_ir._polyfillr _ALIGN_OFFSET_ALIGN_THRESHOLD_ALLOCATION_GRANULARITY getLoggerrr dataclassrr r:rrHrQrTrqrvrrrrrrrrsL       &   3  Q!