o oi[@sddlmZmZmZmZmZmZddlZddlm Z m Z m Z ddl m Z mZddlmZddlmZddlmZdd lmZmZd e d e d e fd dZd eeeffddZd2de de dee d e fddZde d ee e ffddZd2de de dee d ee e ffddZd2de de dee d ee e ffddZd3de de dedee d ee e ff dd Z d4de de dedee d ee e ff d"d#Z!  $ % d5de de d&e d'e ded(ed)e"dee d ee e ffd*d+Z#Gd,d-d-e Z$Gd.d/d/e Z%Gd0d1d1e Z&dS)6)AnyClassVarDictListOptionalTupleN)ModuleTensor concatenate)KORNIA_CHECK_DM_DESCKORNIA_CHECK_SHAPE)get_laf_center)DiscreteSteerer)is_mps_tensor_safe)get_adalam_default_config match_adalamd1d2returncCst|st|st||S|djddd}|djddd}|d|d|d|dd||}|jdd}|S) zManual `torch.cdist` for M1.rT)dimkeepdimrg@g)min) rtorchcdistsumrepeatsizetclampsqrt)rrd1_sqd2_sqdmr%K/home/ubuntu/.local/lib/python3.10/site-packages/kornia/feature/matching.py_cdists 8r'cCsdddd}|S)Ng333333?F$@)thmutual spatial_thr%configr%r%r&_get_default_fginn_params*s r.desc1desc2dm_cCs*|dur t||}|St||||}|S)aCheck validity of provided distance matrix, or calculates L2-distance matrix if dm is not provided. Args: desc1: Batch of descriptors of a shape :math:`(B1, D)`. desc2: Batch of descriptors of a shape :math:`(B2, D)`. dm_: Tensor containing the distances from each descriptor in desc1 to each descriptor in desc2, shape of :math:`(B1, B2)`. N)r'r )r/r0r1r$r%r%r&_get_lazy_distance_matrix/s   r2r$cCs4tjdd|j|jd}tjdd|jtjd}||fS)zOutput empty tensors. Returns: - Descriptor distance of matching descriptors, shape of :math:`(0, 1)`. - Long tensor indexes of matching descriptors in desc1 and desc2, shape of :math:`(0, 2)`. rr)devicedtyper)remptyr3r4long)r$distsidxsr%r%r& _no_matchAsr9cCst|ddgt|ddgt|dkst|dkrt|St|||}tj|dd\}}tjd|d|jd}t | dd| ddgd}| dd| ddfS) aiFind nearest neighbors in desc2 for each vector in desc1. If the distance matrix dm is not provided, :py:func:`torch.cdist` is used. Args: desc1: Batch of descriptors of a shape :math:`(B1, D)`. desc2: Batch of descriptors of a shape :math:`(B2, D)`. dm: Tensor containing the distances from each descriptor in desc1 to each descriptor in desc2, shape of :math:`(B1, B2)`. Returns: - Descriptor distance of matching descriptors, shape of :math:`(B1, 1)`. - Long tensor indexes of matching descriptors in desc1 and desc2, shape of :math:`(B1, 2)`. BDIMrrrr3r) r lenr9r2rrarangerr3r view)r/r0r$distance_matrix match_dists idxs_in_2idxs_in1 matches_idxsr%r%r&match_nnNs rGc Cs<t|ddgt|ddgt|dkst|dkrt|St|||}t|d|d}tj|dd\}}tj|dd\}}tj||jd} |d|dkrs| ||d|k} t | dd| ddgd| } || }n| ||d|k} t | dd| ddgd| } || }| dd| dd fS) aFind mutual nearest neighbors in desc2 for each vector in desc1. If the distance matrix dm is not provided, :py:func:`torch.cdist` is used. Args: desc1: Batch of descriptors of a shape :math:`(B1, D)`. desc2: Batch of descriptors of a shape :math:`(B2, D)`. dm: Tensor containing the distances from each descriptor in desc1 to each descriptor in desc2, shape of :math:`(B1, B2)`. Return: - Descriptor distance of matching descriptors, shape of. :math:`(B3, 1)`. - Long tensor indexes of matching descriptors in desc1 and desc2, shape of :math:`(B3, 2)`, where 0 <= B3 <= min(B1, B2) r:r;rrr<r=Nr>r) r r?r9r2rrrr@r3r rA) r/r0r$rBmsrCrD match_dists2 idxs_in_1 minsize_idxs mutual_nnsrFr%r%r& match_mnnis" " "rM皙?r)c Cst|ddgt|ddg|jddkrt|St|||}tj|dddd\}}|dddf|dddf}||k}||} t| dkrLt|Stjd|d|j d |} |dddf|}t | d d| d dgd} | d d| d dfS) aFind nearest neighbors in desc2 for each vector in desc1. The method satisfies first to second nearest neighbor distance <= th. If the distance matrix dm is not provided, :py:func:`torch.cdist` is used. Args: desc1: Batch of descriptors of a shape :math:`(B1, D)`. desc2: Batch of descriptors of a shape :math:`(B2, D)`. th: distance ratio threshold. dm: Tensor containing the distances from each descriptor in desc1 to each descriptor in desc2, shape of :math:`(B1, B2)`. Return: - Descriptor distance of matching descriptors, shape of :math:`(B3, 1)`. - Long tensor indexes of matching descriptors in desc1 and desc2. Shape: :math:`(B3, 2)`, where 0 <= B3 <= B1. r:r;rrrFrlargestNr=r>) r shaper9r2rtopkr?r@rr3r rA) r/r0r)r$rBvalsrDratiomaskrCrErFr%r%r& match_snns   rVffffff?cCst|ddgt|ddg|jddks|jddkr t|St|||}t||||\}}t||||\}}t|dkrt|dkr|d}t|s\t j | | dd} n-| |j |ddd} | ||dd| jdd} | |d|d} | jdddd k} | jdddd k} || d } || d }|| d }|| d }t | d d df\}}t |d d df\}}| |} t ||||}| }|d d|d d}}||fSt|\}}||fS) aFind mutual nearest neighbors in desc2 for each vector in desc1. the method satisfies first to second nearest neighbor distance <= th. If the distance matrix dm is not provided, :py:func:`torch.cdist` is used. Args: desc1: Batch of descriptors of a shape :math:`(B1, D)`. desc2: Batch of descriptors of a shape :math:`(B2, D)`. th: distance ratio threshold. dm: Tensor containing the distances from each descriptor in desc1 to each descriptor in desc2, shape of :math:`(B1, B2)`. Return: - Descriptor distance of matching descriptors, shape of. :math:`(B3, 1)`. - Long tensor indexes of matching descriptors in desc1 and desc2, shape of :math:`(B3, 2)` where 0 <= B3 <= B1. r:r;rrrg?)pr<g:0yE>r>N)r rQr9r2rVrr?fliprrrfloattorepeat_interleaverrabsrreshaperrAsortmax)r/r0r)r$rBdists1idx1dists2idx2idxs_dm idxs1_rep mutual_idxs1 mutual_idxs2 good_idxs1 good_idxs2 dists1_good dists2_good_idx_upl1idx_upl2rCrFr%r%r& match_smnns:  ( rpr(Flafs1lafs2r+r*cCst|ddgt|ddgd}t|||} | j} |jddkr$t| Stdtd|jd} tj| | ddd \} } | d d df}t | d d}|| }tj ||ddddd }| d d dd f|d d dd f|k | |}|jdd \}}|}| d d df} ||}||k}||}t |dkrt| Stjd| d| jd|}| |} t| d d| d dgd}| d d| d d}}|s||fStj| dd \}}|d d df||d d dfk}||||fS)afFind nearest neighbors in desc2 for each vector in desc1. The method satisfies first to second nearest neighbor distance <= th, and assures 2nd nearest neighbor is geometrically inconsistent with the 1st one (see :cite:`MODS2015` for more details) If the distance matrix dm is not provided, :py:func:`torch.cdist` is used. Args: desc1: Batch of descriptors of a shape :math:`(B1, D)`. desc2: Batch of descriptors of a shape :math:`(B2, D)`. lafs1: LAFs of a shape :math:`(1, B1, 2, 3)`. lafs2: LAFs of a shape :math:`(1, B2, 2, 3)`. th: distance ratio threshold. spatial_th: minimal distance in pixels to 2nd nearest neighbor. mutual: also perform mutual nearest neighbor check dm: Tensor containing the distances from each descriptor in desc1 to each descriptor in desc2, shape of :math:`(B1, B2)`. Return: - Descriptor distance of matching descriptors, shape of :math:`(B3, 1)`. - Long tensor indexes of matching descriptors in desc1 and desc2. Shape: :math:`(B3, 2)`, where 0 <= B3 <= B1. r:r;g.Arr rFrONr>)rXrr<r=)r r2r4rQr9r`rrrRr rAnormr[r?r@rr3r )r/r0rqrrr)r+r*r$ BIG_NUMBERrBr4num_candidates vals_candrDrSxy2 candidates_xykdist fginn_valsfginn_vals_best_fginn_idxs_bestvals_2ndrTrUrCrErFrm idxs_in_1_mut good_maskr%r%r& match_fginns>$ 6 $rcsNeZdZdZddededdffdd Zd ed edeeeffd d Z Z S)DescriptorMatcherarModule version of matching functions. See :func:`~kornia.feature.match_nn`, :func:`~kornia.feature.match_snn`, :func:`~kornia.feature.match_mnn` or :func:`~kornia.feature.match_smnn` for more details. Args: match_mode: type of matching, can be `nn`, `snn`, `mnn`, `smnn`. th: threshold on distance ratio, or other quality measure. snnrN match_moder)rNcsJt|}gd|_||jvrt|d|j||_||_dS)Nnnmnnrsmnn is not supported. Try one of )super__init__lower known_modesNotImplementedErrorrr))selfrr) _match_mode __class__r%r&rFs    zDescriptorMatcher.__init__r/r0cCsl|jdkr t||}|S|jdkrt||}|S|jdkr&t|||j}|S|jdkr4t|||j}|St)aRun forward. Args: desc1: Batch of descriptors of a shape :math:`(B1, D)`. desc2: Batch of descriptors of a shape :math:`(B2, D)`. Returns: - Descriptor distance of matching descriptors, shape of :math:`(B3, 1)`. - Long tensor indexes of matching descriptors in desc1 and desc2, shape of :math:`(B3, 2)` where :math:`0 <= B3 <= B1`. rrrrrrGrMrVr)rpr)rr/r0outr%r%r&forwardOs    zDescriptorMatcher.forward)rrN) __name__ __module__ __qualname____doc__strrZrr rr __classcell__r%r%rr&r:s & rcseZdZdZ   ddedededed ed d f fd d Zdde de de e d e e e ffddZ  dde de de de ed e e e e eff ddZZS)DescriptorMatcherWithSteereraMatching that is invariant under rotations, using Steerers. Args: steerer: An instance of :func:`kornia.feature.steerers.DiscreteSteerer`. steerer_order: order of discretisation of rotation angles, e.g. 4 leads to quarter rotations. steer_mode: can be `global`, `local`. `global` means that the we output matches from the global rotation with most matches. `local` means that we output matches from a distance matrix where the distance between each descriptor pair is the minimal over rotations. match_mode: type of matching, can be `nn`, `snn`, `mnn`, `smnn`. WARNING: using steer_mode `global` with match_mode `nn` will lead to bad results since `nn` doesn't generate different amount of matches depending on goodness of fit. th: threshold on distance ratio, or other quality measure. Example: >>> import kornia as K >>> import kornia.feature as KF >>> device = K.utils.get_cuda_or_mps_device_if_available() >>> img1 = torch.randn([1, 3, 768, 768], device=device) >>> img2 = torch.randn([1, 3, 768, 768], device=device) >>> dedode = KF.DeDoDe.from_pretrained(detector_weights="L-C4-v2", descriptor_weights="B-SO2").to(device) >>> steerer_order = 8 # discretisation order of rotation angles >>> steerer = KF.steerers.DiscreteSteerer.create_dedode_default( ... generator_type="SO2", steerer_order=steerer_order ... ) >>> steerer = steerer.to(device) >>> matcher = KF.matching.DescriptorMatcherWithSteerer( ... steerer=steerer, steerer_order=steerer_order, steer_mode="global", match_mode="smnn", th=0.98 ... ) >>> with torch.inference_mode(): ... kps1, scores1, descs1 = dedode(img1, n=20_000) ... kps2, scores2, descs2 = dedode(img2, n=20_000) ... kps1, kps2, descs1, descs2 = kps1[0], kps2[0], descs1[0], descs2[0] ... dists, idxs, num_rot = matcher( ... descs1, descs2, normalize=True, subset_size=1000, ... ) >>> # print(f"{idxs.shape[0]} tentative matches with steered DeDoDe") >>> # print(f"at rotation of {num_rot * 360 / steerer_order} degrees") globalrrNsteerer steerer_order steer_moderr)rNcst||_||_|}ddg|_||jvr#t|d|j||_|}gd|_||jvr>t|d|j||_ ||_ dS)Nrlocalrr) rrrrrknown_steer_modesrrrrr))rrrrrr) _steer_moderrr%r&rs      z%DescriptorMatcherWithSteerer.__init__rrr$cCsl|jdkr t|||dS|jdkrt|||dS|jdkr&t|||j|dS|jdkr4t|||j|dSt)Nr)r$rrrr)rrrr$r%r%r&matching_functions    z.DescriptorMatcherWithSteerer.matching_functionFr/r0 normalize subset_sizecCsd}|rtjjj|dd}tjjj|dd}|jdkr|dur\t|jdd|}t|jdd|}||||||d\}}}|jj|||d}| ||d\} } | | |fS| ||d\} } d}t d|j D]%} |jj||d}| ||d\} } | jd| jdkr| | | } } }qmn2|jd krt ||}t d|j D]}|jj||d}t ||}t ||}q| |||\} } nt| | |fS) aRun forward. Args: desc1: Batch of descriptors of a shape :math:`(B1, D)`. desc2: Batch of descriptors of a shape :math:`(B2, D)`. normalize: bool to decide whether to normalize descriptors to unit norm. subset_size: If set, the subset size to use for determining optimal number of rotations. Smaller subset size leads to faster but less accurate matching. Only used when `self.steer_mode` is `"global"`. Returns: - Descriptor distance of matching descriptors, shape of :math:`(B3, 1)`. - Long tensor indexes of matching descriptors in desc1 and desc2, shape of :math:`(B3, 2)` where :math:`0 <= B3 <= B1`. - Number of global rotations from desc1 to desc2, in terms of `self.steerer_order` (will be `None` if `self.steer_mode` is `local`). Nr>r<rr)r) steerer_powerrrr)rr functionalrrrandpermrQrsteer_descriptionsrrangerr'minimumr)rr/r0rrrot1to2 subsample1 subsample2rmdistidxrdist_newidx_newr$dm_newr%r%r&rsL       z$DescriptorMatcherWithSteerer.forward)rrrNN)FN)rrrrrintrrZrr rrrboolrrr%r%rr&ris@-(rc s|eZdZUdZddgZeeeed<ddede e ee fddffd d Z d e d e d e de de e e ff ddZZS)GeometryAwareDescriptorMatchera_Module version of matching functions. See :func:`~kornia.feature.match_nn`, :func:`~kornia.feature.match_snn`, :func:`~kornia.feature.match_mnn` or :func:`~kornia.feature.match_smnn` for more details. Args: match_mode: type of matching, can be `fginn`. th: threshold on distance ratio, or other quality measure. fginnadalamrNrparamsrcsDt|}||jvrt|d|j||_|pi|_dS)Nr)rrrrrrr)rrrrrr%r&rs  z'GeometryAwareDescriptorMatcher.__init__r/r0rqrrc Csv|jdkr t}||jt|||||d|d|d}|S|jdkr9t}||jt|||||d}|St)aRun forward. Args: desc1: Batch of descriptors of a shape :math:`(B1, D)`. desc2: Batch of descriptors of a shape :math:`(B2, D)`. lafs1: LAFs of a shape :math:`(1, B1, 2, 3)`. lafs2: LAFs of a shape :math:`(1, B2, 2, 3)`. Returns: - Descriptor distance of matching descriptors, shape of :math:`(B3, 1)`. - Long tensor indexes of matching descriptors in desc1 and desc2, shape of :math:`(B3, 2)` where :math:`0 <= B3 <= B1`. rr)r+r*rr,)rr.updaterrrrr)rr/r0rqrrrr_paramsr%r%r&rs     z&GeometryAwareDescriptorMatcher.forward)rN)rrrrrrrr__annotations__rrr rrrrr%r%rr&rs  (.rr)rNN)rWN)rNr(FN)'typingrrrrrrr kornia.corerr r kornia.core.checkr r kornia.feature.lafr kornia.feature.steerersrkornia.utils.helpersrrrrr'rr.r2r9rGrMrZrVrprrrrrr%r%r%r&sV     ( (,&,&< N/