o oiO @sddlZddlmZmZmZmZddlZddlmm Z ddl m Z m Z mZmZmZmZmZddlmZmZddlmZmZmZddlmZddlmZde d e fd d Zde d e fd d Zde d e fddZ de de d e fddZ!de de d e fddZ"dHde dee dee d e fddZ#de dee$e fd e fddZ%dIde de$d e fd d!Z&d"e d e fd#d$Z'dJde d&e(d e fd'd(Z)dKde d)e(d eee(ee(ffd*d+Z*de d,e d e fd-d.Z+de d,e d e fd/d0Z,dLd2e de d3e(d e fd4d5Z- 6dMd2e de d3e(d7e.d e f d8d9Z/ 6dMd2e de d3e(d7e.d e f d:d;Z0dKde d,e dZ1de d e fd?d@Z2dAe d e fdBdCZ3dDe dEe d e fdFdGZ4dS)NN)ListOptionalTupleUnion)Tensor concatenatecossinstacktensorzeros)KORNIA_CHECK_LAFKORNIA_CHECK_SHAPE)angle_to_rotation_matrixconvert_points_from_homogeneousrad2deg)transform_points)pyrdownLAFreturncCstt|d}|dddddf|dddddf|dddddf|dddddf|}|S)zReturn a scale of the LAFs. Args: LAF: :math:`(B, N, 2, 3)` Returns: scale :math:`(B, N, 1, 1)` Example: >>> input = torch.ones(1, 5, 2, 3) # BxNx2x3 >>> output = get_laf_scale(input) # BxNx1x1 g|=.r)r abssqrt)repsoutrF/home/ubuntu/.local/lib/python3.10/site-packages/kornia/feature/laf.py get_laf_scales\ rcCst||d}|S)a^Return a center (keypoint) of the LAFs. The convention is that center of 5-pixel image (coordinates from 0 to 4) is 2, and not 2.5. Args: LAF: :math:`(B, N, 2, 3)` Returns: xy :math:`(B, N, 2)` Example: >>> input = torch.ones(1, 5, 2, 3) # BxNx2x3 >>> output = get_laf_center(input) # BxNx2 .r)r )rrrrrget_laf_center3sr cCs*t|t|d|d}t|dS)aReturn orientation of the LAFs, in degrees. Args: LAF: :math:`(B, N, 2, 3)` Returns: angle in degrees :math:`(B, N, 1)` Example: >>> input = torch.ones(1, 5, 2, 3) # BxNx2x3 >>> output = get_laf_orientation(input) # BxNx1 ).rr).rr)r torchatan2r unsqueeze)r angle_radrrrget_laf_orientationHsr&angles_degreescCst||jdd\}}t|||dd}|}t|ddddddddf||dd|||dd|ddddddddf<|S)aCApply additional rotation to the LAFs. Compared to `set_laf_orientation`, the resulting rotation is original LAF orientation plus angles_degrees. Args: LAF: :math:`(B, N, 2, 3)` angles_degrees: :math:`(B, N, 1)` in degrees. Returns: LAF oriented with angles :math:`(B, N, 2, 3)` Nr)r shaperviewcloner"bmmreshape)rr'BNrotmatout_lafrrr rotate_laf[s  ^r1cCs6t||jdd\}}t||}t|||S)zChange the orientation of the LAFs. Args: LAF: :math:`(B, N, 2, 3)` angles_degrees: :math:`(B, N, 1)` in degrees. Returns: LAF oriented with angles :math:`(B, N, 2, 3)` Nr)r r(r& reshape_asr1)rr'_B_Norirrrset_laf_orientationps r6xyscaler5c Cst|gd|j}|j}|jdd\}}|dur%tj||dd||d}|dur2t||d||d}t|gdt|gdtt| d| dgdd }t ||}|S) azCreate a LAF from keypoint center, scale and orientation. Useful to create kornia LAFs from OpenCV keypoints. Args: xy: :math:`(B, N, 2)`. scale: :math:`(B, N, 1, 1)`. If not provided, scale = 1.0 is assumed ori: angle in degrees :math:`(B, N, 1)`. If not provided orientation = 0 is assumed Returns: LAF :math:`(B, N, 2, 3)` )r-r.2Nrr)devicedtype)r-r.1r<)r-r.r<r!dim) rr:r;r(r"onesr rrsqueezer$ scale_laf) r7r8r5r:r;r-r. unscaled_laflafrrrlaf_from_center_scale_oris  rDrC scale_coefcCsxt|ttfstdt|t||ddddddddf}t|||ddddddddfgddS)aMultiplies region part of LAF ([:, :, :2, :2]) by a scale_coefficient. So the center, shape and orientation of the local feature stays the same, but the region area changes. Args: laf: :math:`(B, N, 2, 3)` scale_coef: broadcastable tensor or float. Returns: LAF :math:`(B, N, 2, 3)` Example: >>> input = torch.ones(1, 5, 2, 3) # BxNx2x3 >>> scale = 0.5 >>> output = scale_laf(input, scale) # BxNx2x3 z*scale_coef should be float or Tensor. Got Nrr=) isinstancefloatr TypeErrortyper r)rCrEcenterless_lafrrrrAs  0rA& .>rcCst|t|}|}t|dddddfd|dddddfd|}t||t|gdd}t|dddddf|dddddf|dddddf|dddddf||||gdd}tt||gdd|dddddfgdd}t||S)a2Rectify the affine matrix, so that it becomes upright. Args: laf: :math:`(B, N, 2, 3)` eps: for safe division. Returns: laf: :math:`(B, N, 2, 3)` Example: >>> input = torch.ones(1, 5, 2, 3) # BxNx2x3 >>> output = make_upright(input) # BxNx2x3 .rrrrFr=N)r rr"rr contiguous zeros_likerA)rCrdetr8b2a2laf1_elllaf2_elllaf_unit_scalerrr make_uprights>^ . rTellsc Cst|gd|j\}}}|dddf}t|}|dddf}|dddf||d}t||||gdd ||dd }t ||dd df||dd gdd } | S) aConvert ellipse regions to LAF format. Ellipse (a, b, c) and upright covariance matrix [a11 a12; 0 a22] are connected by inverse matrix square root: A = invsqrt([a b; b c]). See also https://github.com/vlfeat/vlfeat/blob/master/toolbox/sift/vl_frame2oell.m Args: ells: tensor :math:`(B, N, 5)` of ellipses in Oxford format [x y a b c]. Returns: LAF :math:`(B, N, 2, 3)` Example: >>> input = torch.ones(1, 10, 5) # BxNx5 >>> output = ellipse_to_laf(input) # BxNx2x3 )r-r.5.rrFrLr!r=Nr) rr(rrr"rNclampr r)inverser) rUr-r._a11a12a22a21Arrrrellipse_to_lafs  $(ra2n_ptsc Cs<t||\}}}}tttddtj|ddt tddtj|ddt |ddgdd}tt gd dd|gddd |||d}||j|j}t gd ddd ||dd}t| ddd||j|jgdd}t||dddddd}t| |||dS)a,Convert LAFs to boundary points of the regions + center. Used for local features visualization, see visualize_laf function. Args: LAF: :math:`(B, N, 2, 3)` n_pts: number of points to output. Returns: tensor of boundary points LAF: :math:`(B, N, n_pts, 2)` rrrr!r=)rd?rF)r sizerr r"linspacemathpir$rr?r r)expandtor:r;r+permuter) rrcr-r.r[ptsauxHLAFpts_hrrrlaf_to_boundary_pointss   4 $* rqimg_idxcCsNt|t|||dd}|ddd}|d|dfS)aReturn list for drawing LAFs (local features). Args: LAF: :math:`(B, N, 2, 3)` img_idx: which points to output. Returns: List of boundary points x, y` Examples: x, y = get_laf_pts_to_draw(LAF, img_idx) plt.figure() plt.imshow(kornia.utils.tensor_to_image(img[img_idx])) plt.plot(x, y, 'r') plt.show() rrr.r.r)r rqdetachrlcputolist)rrrrmpts_nprrrget_laf_pts_to_draw"sryimagesc Csvt||\}}}}t|d}t|d}t||}tjdddd|j|jd|}||d<||d<|||S)a De-normalize LAFs from scale to image scale. The convention is that center of 5-pixel image (coordinates from 0 to 4) is 2, and not 2.5. B,N,H,W = images.size() MIN_SIZE = min(H - 1, W -1) [a11 a21 x] [a21 a22 y] becomes [a11*MIN_SIZE a21*MIN_SIZE x*(W-1)] [a21*MIN_SIZE a22*MIN_SIZE y*(W-1)] Args: LAF: :math:`(B, N, 2, 3)` images: :math:`(B, CH, H, W)` Returns: the denormalized LAF: :math:`(B, N, 2, 3)`, scale in pixels rrrFr;r:rrrrrrrr r rfrHminr"r?r;r: expand_as rrzr[hwwfhfmin_sizecoefrrrdenormalize_laf;s   rc Cs~t||\}}}}t|d}t|d}t||}tjdddd|j|jd|}d||d<d||d<|||S)aNormalize LAFs to [0,1] scale from pixel scale. See below: B,N,H,W = images.size() MIN_SIZE = min(H - 1, W -1) [a11 a21 x] [a21 a22 y] becomes: [a11/MIN_SIZE a21/MIN_SIZE x/(W-1)] [a21/MIN_SIZE a22/MIN_SIZE y/(H-1)] Args: LAF: :math:`(B, N, 2, 3)` images: :math:`(B, CH, H, W)` Returns: the denormalized LAF: :math:`(B, N, 2, 3)`, scale in image percentage (0, 1) rrrFr{rer|r}r~rrrr normalize_laf[s     r imgPSc Cst||\}}}}|\}}}}t||} tj| ||dd|||||gdd} d| ddddft|d d | ddddf<d| dddd ft|d d | dddd f<| S) zGenerate affine grid. Args: img: image tensor of shape :math:`(B, CH, H, W)`. LAF: laf with shape :math:`(B, N, 2, 3)`. PS: patch size to be extracted. Returns: grid :math:`(B*N, PS, PS, 2)` rrFF) align_corners@.Nrrre)r rfrF affine_gridr)r*rH) rrrr-r.r[chrr LAF_renormgridrrr'generate_patch_grid_from_normalized_LAFzs  *88rT normalize_lafs_before_extractionc Cst||r t||}n|}|\}}}}|\} } }}g} t| D]3} t|| | d|| | d||j} | tj || | d | d|||| dddq$t | dd | | |||S)aExtract patches defined by LAFs from image tensor. No smoothing applied, huge aliasing (better use extract_patches_from_pyramid). Args: img: images, LAFs are detected in :math:`(B, CH, H, W)`. laf: :math:`(B, N, 2, 3)`. PS: patch size. normalize_lafs_before_extraction: if True, lafs are normalized to image size. Returns: patches with shape :math:`(B, N, CH, PS,PS)`. rrborderF padding_moderr=) r rrfrangerrkr:appendr grid_samplerjrr))rrCrrnlafr[rrrr-r.rirrrrextract_patches_simples  ,&rc Cst||r t||}n|}|\}}}}|\}}} } dtt||t|} t|d|d|} | jdt d| dd } |}d}t ||||| |j |j}d}|r|\}}} } t|D]`}| ||k}|dkrqm|dkd }t|||d|||d|d d d d f|}tj|||d|jd|| | |d d d }|||d ddd| |jqmt|d|d|k}|s |St|}|d7}|sa|S)aExtract patches defined by LAFs from image tensor. Patches are extracted from appropriate pyramid level. Args: img: images, LAFs are detected in :math:`(B, CH, H, W)`. laf: :math:`(B, N, 2, 3)`. PS: patch size. normalize_lafs_before_extraction: if True, lafs are normalized to image size. Returns: patches with shape :math:`(B, N, CH, PS,PS)`. rrrFrdrr)rmaxTr!NrFr)r rrfrrrHrlog2rYrlongr"r rkr;r:rr@sumitemr)rrrrjr(masked_scatter_r)rrCrrrr-r.r[rrrr8 max_levelpyr_idxcur_img cur_pyr_levelrwe_are_in_businessr scale_maskrpatchesrrrextract_patches_from_pyramidsB  " 4&&rrcCsnt||\}}}}t|d}|d|k|d||k|d|k|d||k}|jddd}|S)aaCheck if the LAF is touching or partly outside the image boundary. Returns the mask of LAFs, which are fully inside the image, i.e. valid. Args: laf: :math:`(B, N, 2, 3)`. images: images, lafs are detected in :math:`(B, CH, H, W)`. border: additional border. Returns: mask with shape :math:`(B, N)`. rsrtrr=r)r rfrqr)rCrzrr[rrrmgood_lafs_maskrrrlaf_is_inside_images 6rcCs:t|t|d|d|d|d|dgdd}|S)zConvert local affine frame(LAF) to alternative representation: coordinates of LAF center, LAF-x unit vector, LAF-y unit vector. Args: laf: :math:`(B, N, 2, 3)`. Returns: threepts :math:`(B, N, 2, 3)`. rrsrtr!r=)r r )rC three_ptsrrrlaf_to_three_pointss .rthreeptscCs2t|d|d|d|d|dgdd}|S)zConvert three points to local affine frame. Order is (0,0), (0, 1), (1, 0). Args: threepts: :math:`(B, N, 2, 3)`. Returns: laf :math:`(B, N, 2, 3)`. rsrrtr!r=)r )rrCrrrlaf_from_three_pointss. rtrans_01lafs_1c Cst|t|s td|j|jkstd|jd|jdks%td|jddkr3|jddks7td|\}}}}t|}| dd dd  ||dd }t ||}| ||dd  dd dd }t |S) aApply perspective transformations to a set of local affine frames (LAFs). Args: trans_01: tensor for perspective transformations of shape :math:`(B, 3, 3)`. lafs_1: tensor of lafs of shape :math:`(B, N, 2, 3)`. Returns: tensor of N-dimensional points of shape :math:`(B, N, 2, 3)`. Examples: >>> rng = torch.manual_seed(0) >>> lafs_1 = torch.rand(2, 4, 2, 3) # BxNx2x3 >>> lafs_1 tensor([[[[0.4963, 0.7682, 0.0885], [0.1320, 0.3074, 0.6341]], [[0.4901, 0.8964, 0.4556], [0.6323, 0.3489, 0.4017]], [[0.0223, 0.1689, 0.2939], [0.5185, 0.6977, 0.8000]], [[0.1610, 0.2823, 0.6816], [0.9152, 0.3971, 0.8742]]], [[[0.4194, 0.5529, 0.9527], [0.0362, 0.1852, 0.3734]], [[0.3051, 0.9320, 0.1759], [0.2698, 0.1507, 0.0317]], [[0.2081, 0.9298, 0.7231], [0.7423, 0.5263, 0.2437]], [[0.5846, 0.0332, 0.1387], [0.2422, 0.8155, 0.7932]]]]) >>> trans_01 = torch.eye(3).repeat(2, 1, 1) # Bx3x3 >>> trans_01.shape torch.Size([2, 3, 3]) >>> lafs_0 = perspective_transform_lafs(trans_01, lafs_1) # BxNx2x3 zInput type is not a Tensorz!Tensor must be in the same devicerz2Input batch size must be the same for both tensorsr!rFz#Transformation should be homographyrr)r r" is_tensorrIr:r( ValueErrorrfrrlr,rr)r) rrbsnr[ threepts_1points_1points_0 threepts_0rrrperspective_transform_lafs$s,   r)NN)rL)rb)r)r)rT)5rhtypingrrrrr"torch.nn.functionalnn functionalr kornia.corerrrr r r r kornia.core.checkr rkornia.geometry.conversionsrrrkornia.geometry.linalgrkornia.geometry.transformrrr r&r1r6rDrHrArTraintrqryrrrboolrrrrrrrrrrsf$  $!*(   % 4