o oiJ@sXUddlZddlmZmZmZddlZddlmmZ ddl m Z ddl m Z mZmZmZmZmZmZmZddlmZddlmZmZddlmZmZmZdd lmZm Z dd l!m"Z"dd l#m$Z$d ed ede%defddZ&dedee%defddZ'GdddeZ(Gddde Z)de)fddZ*GdddeZ+ddde,d d!d"Z-e)e.d#<dS)$N)ListOptionalTuple) TypedDict)DeviceModuleTensor concatenateeyetensorwherezeros)KORNIA_CHECK_SHAPE)ConvSoftArgmax3dNonMaximaSuppression2d) ScalePyramidpyrdownresize)laf_from_center_scale_orilaf_is_inside_image)PassLAF) BlobHessian max_coordssigmas num_levelsreturncCs||j\}}}|dddddfdddd}t|dtd|t|||d|ddddddfgd}|S)acAuxiliary function for ScaleSpaceDetector. Converts scale level index from ConvSoftArgmax3d to the actual scale, using the sigmas from the ScalePyramid output. Args: max_coords: tensor [BxNx3]. sigmas: tensor [BxNxD], D >= 1 num_levels: number of levels in the scale index. Returns: tensor [BxNx3]. Nrr)rrg@)shape contiguousviewr torchpowfloat)rrrBN_ scale_coordsoutr*W/home/ubuntu/.local/lib/python3.10/site-packages/kornia/feature/scale_space_detector.py_scale_index_to_scale#s &@r,mask octave_shapecCs(|dd}tj||ddd}|dS)z2Downsample a mask based on the given octave shape.NbilinearF)mode align_cornersr)F interpolate unsqueeze)r-r. mask_shape mask_octaver*r*r+_create_octave_maskAs  r8cseZdZdZ         ddededeed eed eed eed eed ededdffdd Z de fddZ dde dedee de e e ffddZdde dee de e e ffddZZS)ScaleSpaceDetectoraModule for differentiable local feature detection. As close as possible to classical local feature detectors like Harris, Hessian-Affine or SIFT (DoG). It has 5 modules inside: scale pyramid generator, response ("cornerness") function, soft nms function, affine shape estimator and patch orientation estimator. Each of those modules could be replaced with learned custom one, as long, as they respect output shape. Args: num_features: Number of features to detect. In order to keep everything batchable, output would always have num_features output, even for completely homogeneous images. mr_size: multiplier for local feature scale compared to the detection scale. 6.0 is matching OpenCV 12.0 convention for SIFT. scale_pyr_module: generates scale pyramid. See :class:`~kornia.geometry.ScalePyramid` for details. Default: ScalePyramid(3, 1.6, 10). resp_module: calculates ``'cornerness'`` of the pixel. nms_module: outputs per-patch coordinates of the response maxima. See :class:`~kornia.geometry.ConvSoftArgmax3d` for details. ori_module: for local feature orientation estimation. Default:class:`~kornia.feature.PassLAF`, which does nothing. See :class:`~kornia.feature.LAFOrienter` for details. aff_module: for local feature affine shape estimation. Default: :class:`~kornia.feature.PassLAF`, which does nothing. See :class:`~kornia.feature.LAFAffineShapeEstimator` for details. minima_are_also_good: if True, then both response function minima and maxima are detected Useful for symmetric response functions like DoG or Hessian. Default is False @NF num_featuresmr_sizescale_pyr_module resp_module nms_module ori_module aff_moduleminima_are_also_goodscale_space_responserc st||_||_|durtddd}||_|durt}||_|dur/tdddddd}||_ |dur9t }||_ |durCt }||_ ||_ | |_dS) Ng?)rErErE)rrrFT)normalized_coordinates output_value)super__init__r=r<r scale_pyrrresprnmsroriaffrCrD) selfr<r=r>r?r@rArBrCrD __class__r*r+rJfs,    zScaleSpaceDetector.__init__cCsZ|jjd|jd|jd|jd|jd|jd|jd|j dS) Nz(num_features=z , mr_size=z , scale_pyr=z, resp=z, nms=z, ori=z, aff=)) rR__name__r<r=rK__repr__rLrMrNrO)rPr*r*r+rUs zScaleSpaceDetector.__repr__img num_featsr-c' Cs~|j}|j}||\}}}g} g} |jjrdnd} t|D]t\} } || }| \}}}}}|jr=|| |d}nD|| ddddd ||||||d|||||}| ddddd}t |jj t r|jj ddkr|ddddddf}|durt||j}||}||\}}|jr|| \}}||k|j}||d||}||dd|d|}||dd}||ddd ddd}|d|krtj||dd \}}t|d|dddd}n|}|}|\}} t |jjt rt|jj}!nt |jjtr$|jj}!n td t|jjt|||!}td||d dddd}"t|j |dddddf|| dd|"|ddddddf|| ddgd}#t!|#| dddf}$||$||}|#| 9}#| "|| "|#| d9} qt| d}%t| d}&tj|%|dd \}%}t|&d|dddddd}&|%|&fS) N??rrrrrEkdimzOExpected the scale pyramid module to have `n_levels` as a Tensor or int.Gotcha )dtypedevice)#r_r^rK double_image enumeratesizerDrLr!permutereshape isinstance extra_levelsrr8rrMrCtor5r"topkgatherrepeatn_levelsintitem TypeErrortyper,r r r=rappend)'rPrVrWr-devr^sprr' all_responsesall_lafspx_sizeoct_idxoctave sigmas_octr%CHLHWoct_respoct_mask coord_max response_max coord_min response_min take_min_maskresponses_flattenmax_coords_flattenresp_flat_bestidxsmax_coords_bestr&rrotmat current_lafs good_mask responseslafsr*r*r+detects|0        *$      &zScaleSpaceDetector.detectcCs4|||j|\}}|||}|||}||fS)a6Three stage local feature detection. First the location and scale of interest points are determined by detect function. Then affine shape and orientation. Args: img: image to extract features with shape [BxCxHxW] mask: a mask with weights where to apply the response function. The shape must be the same as the input image. Returns: lafs: shape [BxNx2x3]. Detected local affine frames. responses: shape [BxNx1]. Response function values for corresponding lafs )rr<rOrNrPrVr-rrr*r*r+forwards  zScaleSpaceDetector.forward) r:r;NNNNNFFN)rT __module__ __qualname____doc__rlr$rrboolrJstrrUrrrr __classcell__r*r*rQr+r9HsF   %( ,\r9c@s6eZdZUeed<eed<eed<eed<eed<dS)Detector_confignms_sizepyramid_levels up_levelsscale_factor_levelss_multN)rTrrrl__annotations__r$r*r*r*r+r s  rcCstS)zReturn default config.)_DEFAULT_DETECTOR_CONFIGcopyr*r*r*r+get_default_detector_configsrc seZdZdZ    ddededeedeedeed df fd d Zdd e ded e fddZ de dede e e fd e e e ffddZ dde dee d e e e ffddZdde dee d e e e ffddZZS)MultiResolutionDetectoraMulti-scale feature detector, based on code from KeyNet. Can be used with any response function. This is based on the original code from paper "Key.Net: Keypoint Detection by Handcrafted and Learned CNN Filters". See :cite:`KeyNet2019` for more details. Args: model: response function, such as KeyNet or BlobHessian num_features: Number of features to detect. conf: Dict with initialization parameters. Do not pass it, unless you know what you are doing`. ori_module: for local feature orientation estimation. Default: :class:`~kornia.feature.PassLAF`, which does nothing. See :class:`~kornia.feature.LAFOrienter` for details. aff_module: for local feature affine shape estimation. Default: :class:`~kornia.feature.PassLAF`, which does nothing. See :class:`~kornia.feature.LAFAffineShapeEstimator` for details. Nmodelr<configrArBrcst||_|durt}|d|_|d|_|d|_|d|_|d|_t |j|jf|_ ||_ |dur=t |_ n||_ |durJt |_dS||_dS)Nrrrrr)rIrJrrnum_pyramid_levelsnum_upscale_levelsrr=rrrMr<rrNrO)rPrr<rrArBrQr*r+rJ)s"         z MultiResolutionDetector.__init__rF score_mapborderscCs6t|}d|dddd|| || f<||S)zCRemove the borders of the image to avoid detections on the corners.rN)r" zeros_like)rPrrr-r*r*r+remove_bordersHs $z&MultiResolutionDetector.remove_borders level_imgnum_kpfactorc Cs||||}|j}|j}|ddddf}|dd|d|df}tj|dd\} } | t | dk} |dd| d|f}t |} | d| d dt |||d} d|d|d} | |jtjd| dd||d}t| |td| d||d}| d||fS) NrrrT) descending)r_r^rX)rMrrr_r^nonzerotr"sortr lenr!flipr r=onesrr )rPrrrdet_mapr_r^yxscores scores_sortedindicescurrent_kp_num xy_projected scale_factorscalerr*r*r+detect_features_on_single_levelNs" z7MultiResolutionDetector.detect_features_on_single_levelrVr-c sgd|jd}|j|jd}t|D]}|d||j7|j|d||j}|qfddD|j\}}}} |} |} g} g} t|jD]Z}t|jd|d}t|}|jd|}t||t| |}}t | t |t |t |f}t | ||fddd } | | ||\}}| | dd| |qQt|jdD]k}|d krt | |jd } | j\}}}}t | t |t |t |f}nd }t|}|d ks|jd krfd dtd |d|jD}t|}t|}| | ||\}}| | dd| |qt| d}t| d}|jd|jkrLtj||jdd\}}t|d|ddddd}||fS)Nrrrrcsg|]}t|qSr*)rl).0x)tmpr*r+ lsz2MultiResolutionDetector.detect..r0F) interpolationr2r)r)rYrYcsg|]}|qSr*r*)ra)num_features_per_levelr*r+rsr[).NNrE)rrrranger<rprrrlr$rrr!rr sumrmr r"rhriexpand)rPrVr- factor_pointslevels idx_levelnfr'hwimg_upcur_imgrsrtnum_points_level up_factornhnwup_factor_kpts cur_scorescur_lafsrnf2 res_pointsrrrr*)rrr+rbsX     " "   zMultiResolutionDetector.detectcCs>t|gd|||\}}|||}|||}||fS)aThree stage local feature detection. First the location and scale of interest points are determined by detect function. Then affine shape and orientation. Args: img: image to extract features with shape [1xCxHxW]. KeyNetDetector does not support batch processing, because the number of detections is different on each image. mask: a mask with weights where to apply the response function. The shape must be the same as the input image. Returns: lafs: shape [1xNx2x3]. Detected local affine frames. responses: shape [1xNx1]. Response function values for corresponding lafs )1Cr{r|)rrrOrNrr*r*r+rs   zMultiResolutionDetector.forward)rNNN)rFr)rTrrrrrlrrrJrrrr$rrrrr*r*rQr+rs>   $,:rrFrZrg6@)rrrrrr)/mathtypingrrrr"torch.nn.functionalnn functionalr3typing_extensionsr kornia.corerrrr r r r r kornia.core.checkrkornia.geometry.subpixrrkornia.geometry.transformrrrlafrr orientationrrrrlr,r8r9rrrsqrtrrr*r*r*r+s4  (   B