o oi@sddlmZddlmZddlZddlmZmZddlm Z m Z m Z ddl m Z mZz ddlZddlZWn eyAdZdZYnwdgZGd ddZdS) ) annotations)ClassVarN)DeviceTensor) KORNIA_CHECKKORNIA_CHECK_SAME_DEVICESKORNIA_CHECK_SHAPE)download_onnx_from_urlnormalize_keypoints OnnxLightGluec@sXeZdZUdZdddddZded<d d gZd ed <ddddZd ddZd ddZ d S)!r aWrapper for loading LightGlue-ONNX models and running inference via ONNXRuntime. LightGlue :cite:`LightGlue2023` performs fast descriptor-based deep keypoint matching. This module requires `onnxruntime` to be installed. If you have trained your own LightGlue model, see https://github.com/fabio-sim/LightGlue-ONNX for how to export the model to ONNX and optimize it. Args: weights: Pretrained weights, or a path to your own exported ONNX model. Available pretrained weights are ``'disk'``, ``'superpoint'``, ``'disk_fp16'``, and ``'superpoint_fp16'``. `Note that FP16 requires CUDA.` Defaults to ``'disk_fp16'`` if ``device`` is CUDA, and ``'disk'`` if CPU. device: Device to run inference on. z^https://github.com/fabio-sim/LightGlue-ONNX/releases/download/v1.0.0/disk_lightglue_fused.onnxzdhttps://github.com/fabio-sim/LightGlue-ONNX/releases/download/v1.0.0/superpoint_lightglue_fused.onnxzchttps://github.com/fabio-sim/LightGlue-ONNX/releases/download/v1.0.0/disk_lightglue_fused_fp16.onnxzihttps://github.com/fabio-sim/LightGlue-ONNX/releases/download/v1.0.0/superpoint_lightglue_fused_fp16.onnx)disk superpoint disk_fp16superpoint_fp16zClassVar[dict[str, str]] MODEL_URLSimage0image1zClassVar[list[str]]required_data_keysNcpuweights str | NonedevicerreturnNonecCsttdudttdudt|}||_|jdkrdg}n|jdkr)ddg}ntd||dur=|jdkr;dnd }||jvrbd |vrNt|jdkd |j|}|jdkr^|d d }t |}tj ||d|_ dS)Nzonnxruntime is not installed.znumpy is not installed.rCPUExecutionProvidercudaCUDAExecutionProviderzUnsupported device rr fp16zFP16 requires CUDA.z.onnxz _cpu.onnx) providers) rortnptorchrtype ValueErrorrreplacer InferenceSessionsession)selfrrrurlr*[/home/ubuntu/.local/lib/python3.10/site-packages/kornia/feature/lightglue_onnx/lightglue.py__init__As&        zOnnxLightGlue.__init__datadict[str, dict[str, Tensor]]dict[str, Tensor]cCs ||S)N)forward)r(r-r*r*r+__call__^s zOnnxLightGlue.__call__c Cs|jD] }t||vd|dq|d|d}}|d|d}}|d|d}}t||||gdt|jj|jjkdttj|jkoa|jkoa|jkoa|jkndt |gd t |gd t |gd t |gd t|j d |j d kdt|j d |j d kdt|j d|j dkd| d| d} } | dur| n |dj ddddd} | dur| n |dj ddddd} t || d} t || d} tt | dkot | d kdtt | dkot | d kd| | ||d} ddg}|j}| D]\}}|j||jjdtjt|j |dq'|D] }|j||jjddqB|j||\}}t||jt||jd}|S)aVMatch keypoints and descriptors between two images. The output contains the matches (the indices of the matching keypoint pairs between the first and second image) and the corresponding confidence scores. Only a batch size of 1 is supported. Args: data: Dictionary containing both images and the keypoints and descriptors thereof. Returns: Dictionary containing the matches and scores. ``data`` (``dict``): ``image0`` (``dict``): ``keypoints`` (`float32`): :math:`(1, M, 2)` ``descriptors`` (`float32`): :math:`(1, M, D)` ``image``: :math:`(1, C, H, W)` or ``image_size``: :math:`(1, 2)` ``image1`` (``dict``): ``keypoints`` (`float32`): :math:`(1, N, 2)` ``descriptors`` (`float32`): :math:`(1, N, D)` ``image``: :math:`(1, C, H, W)` or ``image_size``: :math:`(1, 2)` ``output`` (``dict``): ``matches`` (`int64`): :math:`(S, 2)` ``scores`` (`float32`): :math:`(S)` z Missing key z in datarr keypoints descriptorsz Wrong devicez Wrong dtype)1M2)r4Nr6)r4r5D)r4r7r8r z8Number of keypoints does not match number of descriptorsz$Descriptors' dimensions do not match image_sizeNimage)size)kpts0kpts1desc0desc1matches0mscores0r) device_type device_id element_typeshape buffer_ptr)rFrG)matchesscores)rr contiguousrrr#r"float32dtyperrIgetr allitemr' io_bindingitems bind_inputr!tupledata_ptr bind_outputrun_with_iobinding get_outputs from_dlpacknumpyto)r(r-keydata0data1kpts0_kpts1_rBrCsize0size1r@rAlightglue_inputslightglue_outputsbindingnametensorrKmscoresoutputsr*r*r+r0asT #<((  *,     zOnnxLightGlue.forward)Nr)rrrrrr)r-r.rr/) __name__ __module__ __qualname____doc__r__annotations__rr,r1r0r*r*r*r+r 's   ) __future__rtypingrr" kornia.corerrkornia.core.checkrrrutilsr r r\r! onnxruntimer ImportError__all__r r*r*r*r+s