o €o™iôiã@sjddlmZddlZddlZddlmmZddlm Z m Z ddl m Z ddl mZmZmZmZddl mZddlmZmZmZddlmZdd lmZdd lmZmZmZd d gZ dDdd„Z!dDdd„Z"dEdd„Z#dFdd„Z$dEdd „Z%dEd!d"„Z&dGd$d%„Z'dHd(d)„Z(dHd*d+„Z)dId.d/„Z*dJd1d2„Z+dKd6d7„Z,dLd;d<„Z-dMd?d@„Z.e  dNdOdBd „ƒZ/GdCd „d eƒZ0dS)Pé)Ú annotationsN)Ú rgb_to_ycbcrÚ ycbcr_to_rgb)Úpi)ÚDeviceÚDtypeÚ ParameterÚTensor)Ú ImageModule)Ú KORNIA_CHECKÚKORNIA_CHECK_IS_TENSORÚKORNIA_CHECK_SHAPE©Úrescale)Úperform_keep_shape_image)Údifferentiable_clippingÚdifferentiable_polynomial_floorÚ"differentiable_polynomial_roundingÚJPEGCodecDifferentiableÚjpeg_codec_differentiableÚdevicerÚdtyperÚreturnr c Cs@tjgd¢gd¢gd¢gd¢gd¢gd¢gd¢gd¢g||d S) z1Generate default Quantization table of Y channel.)éé é réé(é3é=)é r éééé:é<é7)r!é rrré9éEé8)r!éééréWéPé>)ér,é%r*éDémégéM)ré#r&é@éQéhéqé\)é1r8éNr.r5éyéxée)éHr<é_ébépédr5éc©rr©ÚtorchÚtensorrH©rLúG/home/ubuntu/.local/lib/python3.10/site-packages/kornia/enhance/jpeg.pyÚ_get_default_qt_y-óø ôrNc Cs@tjgd¢gd¢gd¢gd¢gd¢gd¢gd¢gd¢g||dS)z2Generate default Quantization table of C channels.)r+r1ré/rGrGrGrG)r1ér#éBrGrGrGrG)rr#r*rGrGrGrGrG)rPrRrGrGrGrGrGrG)rGrGrGrGrGrGrGrGrHrIrHrLrLrMÚ_get_default_qt_c?rOrSÚinputcCsD|j\}}}| ||dd|dd¡ ddddd¡ |ddd¡}|S)zïExtract non-overlapping 8 x 8 patches from the given input image. Args: input (Tensor): Input image of the shape :math:`(B, H, W)`. Returns: output (Tensor): Image patchify of the shape :math:`(B, N, 8, 8)`. éréééééÿÿÿÿ©ÚshapeÚviewÚpermuteÚreshape)rTÚBÚHÚWÚoutputrLrLrMÚ _patchify_8x8Qs 4rdraÚintrbcCsH|jdd…\}}| ||d|ddd¡ ddddd¡ |||¡}|S)a"Reverse non-overlapping 8 x 8 patching. Args: input (Tensor): Input image of the shape :math:`(B, N, 8, 8)`. H: height of resulting tensor. W: width of resulting tensor. Returns: output (Tensor): Image patchify of the shape :math:`(B, H, W)`. NrXrUrrVrWrYr[)rTrarbr`Ú_NrcrLrLrMÚ_unpatchify_8x8bs 2rgc Cs²|j}|j}tjd||d}t ||||¡\}}}}d|d|td ¡d|d|td ¡}tjd||d} d| d<t d| | ¡d } | d t  |d |¡} | S) úèPerform an 8 x 8 discrete cosine transform. Args: input (Tensor): Patched input tensor of the shape :math:`(B, N, 8, 8)`. Returns: output (Tensor): DCT output tensor of the shape :math:`(B, N, 8, 8)`. rU©rrç@çð?ç0@çÌ;fž æ?rz i, j -> ijçÐ?©NNç`@) rrrJÚarangeÚmeshgridrÚcosÚonesÚeinsumÚ tensordot) rTrrÚindexÚxÚyÚuÚvÚ dct_tensorÚalphaÚ dct_scalercrLrLrMÚ_dct_8x8us 8rc Cs¸|j}|j}tjd||d}d|d<t ||¡}||d}tjd||d}t ||||¡\}}}} d|d|td ¡d| d|td ¡} d tj || d d d } | S) rhrUrirmrrorjrkrlrnrX)Údimsrp) rrrJrtÚouterrqrrrrsrv) rTrrr}r~rwrxryrzr{Ú idct_tensorrcrLrLrMÚ _idct_8x8Žs   8rƒÚcompression_strengthcCs&tt |dkd|dd|¡ƒ}|S)aConvert a given JPEG quality to the scaling factor. Args: compression_strength (Tensor): Compression strength ranging from 0 to 100. Any shape is supported. Returns: scale (Tensor): Scaling factor to be applied to quantization matrix. Same shape as input. é2gˆ³@gi@rj)rrJÚwhere)r„ÚscalerLrLrMÚ_jpeg_quality_to_scale©s  ýÿrˆÚ jpeg_qualityÚquantization_tablecCsT|dd…dft|ƒdd…dddf}tt|ddddƒƒ}||}t|ƒ}|S)a–Perform quantization. Args: input (Tensor): Input tensor of the shape :math:`(B, N, 8, 8)`. jpeg_quality (Tensor): Compression strength to be applied, shape is :math:`(B)`. quantization_table (Tensor): Quantization table of the shape :math:`(1, 8, 8)` or :math:`(B, 8, 8)`. Returns: output (Tensor): Quantized output tensor of the shape :math:`(B, N, 8, 8)`. NçI@çY@rVéÿ)rˆrrr©rTr‰rŠÚquantization_table_scaledrcrLrLrMÚ _quantizeÀs&ÿÿrcCsH|dd…dft|ƒdd…dddf}|tt|ddddƒƒ}|S)a˜Perform dequantization. Args: input (Tensor): Input tensor of the shape :math:`(B, N, 8, 8)`. jpeg_quality (Tensor): Compression strength to be applied, shape is :math:`(B)`. quantization_table (Tensor): Quantization table of the shape :math:`(1, 8, 8)` or :math:`(B, 8, 8)`. Returns: output (Tensor): Quantized output tensor of the shape :math:`(B, N, 8, 8)`. Nr‹rŒrVr)rˆrrrŽrLrLrMÚ _dequantizeÞs &ÿÿr‘Ú input_ycbcrútuple[Tensor, Tensor, Tensor]cCsŽ|dd…df}|dd…df}|dd…df}t|dd…dfddddd }t|dd…dfddddd }||dd…df|dd…dffS) aŸImplement chroma subsampling. Args: input_ycbcr (Tensor): YCbCr input tensor of the shape :math:`(B, 3, H, W)`. Returns: output_y (Tensor): Y component (not-subsampled), shape is :math:`(B, H, W)`. output_cb (Tensor): Cb component (subsampled), shape is :math:`(B, H // 2, W // 2)`. output_cr (Tensor): Cr component (subsampled), shape is :math:`(B, H // 2, W // 2)`. NrrVrXgà?ÚbilinearFT©ÚfactorÚ interpolationÚ align_cornersÚ antialiasr)r’Úoutput_yÚ output_cbÚ output_crrLrLrMÚ_chroma_subsamplingùs$ ûû"rÚinput_ccCs.t|dd…dfddddd}|dd…dfS)z÷Perform chroma upsampling. Args: input_c (Tensor): Cb or Cr component to be upsampled of the shape :math:`(B, H, W)`. Returns: output_c (Tensor): Upsampled C(b or r) component of the shape :math:`(B, H * 2, W * 2)`. Nrjr”Fr•rr)ržÚoutput_crLrLrMÚ_chroma_upsamplings ûr Ú image_rgbÚquantization_table_yÚquantization_table_cc Csˆt|ƒ}d|}t|ƒ\}}}t|ƒt|ƒt|ƒ}}}t|ƒ}ttj||fddƒ} t|||ƒ} t| ||ƒjddd\} } | | | fS)acPerform JPEG encoding. Args: image_rgb (Tensor): RGB input images of the shape :math:`(B, 3, H, W)`. jpeg_quality (Tensor): Compression strength of the shape :math:`(B)`. quantization_table_y (Tensor): Quantization table for Y channel. quantization_table_c (Tensor): Quantization table for C channels. Returns: y_encoded (Tensor): Encoded Y component of the shape :math:`(B, N, 8, 8)`. cb_encoded (Tensor): Encoded Cb component of the shape :math:`(B, N, 8, 8)`. cr_encoded (Tensor): Encoded Cr component of the shape :math:`(B, N, 8, 8)`. çào@rV©ÚdimrX)rrrdrrJÚcatrÚchunk) r¡r‰r¢r£Ú image_ycbcrÚinput_yÚinput_cbÚinput_crÚdct_yÚ dct_cb_crÚ y_encodedÚ cb_encodedÚ cr_encodedrLrLrMÚ _jpeg_encode0s, ýýý ü r²rªr«r¬cCs¬t|||ƒ}ttj||fdd||ƒ}t|ƒ} t|ƒjddd\} } t| ||ƒ} t| |d|dƒ} t| |d|dƒ}t| ƒ} t|ƒ}tj| | |fddd}t|ƒ}|S)a½Perform JPEG decoding. Args: input_y (Tensor): Compressed Y component of the shape :math:`(B, N, 8, 8)`. input_cb (Tensor): Compressed Cb component of the shape :math:`(B, N, 8, 8)`. input_cr (Tensor): Compressed Cr component of the shape :math:`(B, N, 8, 8)`. jpeg_quality (Tensor): Compression strength of the shape :math:`(B)`. H (int): Original image height. W (int): Original image width. quantization_table_y (Tensor): Quantization table for Y channel. quantization_table_c (Tensor): Quantization table for C channels. Returns: rgb_decoded (Tensor): Decompressed RGB image of the shape :math:`(B, 3, H, W)`. rVr¥rXr¤) r‘rJr§rƒr¨rgr Ústackr)rªr«r¬r‰rarbr¢r£Ú input_cb_crÚidct_yÚidct_cbÚidct_crÚimage_yÚimage_cbÚimage_crr©Ú rgb_decodedrLrLrMÚ _jpeg_decode_s(ýý r¼Úimageútuple[Tensor, int, int]cCs^|jdd…\}}t |d¡d|}t |d¡d|}t |d|d|fd¡}|||fS)a?Pad a given image to be dividable by 16. Args: image: Image of the shape :math:`(*, 3, H, W)`. Returns: image_padded: Padded image of the shape :math:`(*, 3, H_{new}, W_{new})`. h_pad: Padded pixels along the horizontal axis. w_pad: Padded pixels along the vertical axis. éþÿÿÿNrrÚ replicate)r\ÚmathÚceilÚFÚpad)r½rarbÚh_padÚw_padÚ image_paddedrLrLrMÚ_perform_padding•s   rÈú Tensor | Nonec Csft|ƒt|ƒ|j}|j}|durt||ƒn|}|dur"t||ƒn|}t|ƒt|ƒt|gd¢ƒt|dgƒ|jdkrD|jdd}|jdkrO|jdd}t|gd¢ƒt|gd¢ƒt|  ¡  ¡dkom|  ¡  ¡d kd |  ¡  ¡›d |  ¡  ¡›d ƒt |ƒ\}}}|j d d…\}} |j ddkr°t|j d|j dkd|j d›d|j d›dƒ|j ddkrÑt|j d|j dkd|j d›d|j d›dƒ|j ddkròt|j d|j dkd|j d›d|j d›dƒ| ||¡}| ||¡}| ||¡}t||||d\} } } t| | | ||| ||d} t| ddd} | dd||…d| |…f} | S)aDifferentiable JPEG encoding-decoding module. Based on :cite:`reich2024` :cite:`shin2017`, we perform differentiable JPEG encoding-decoding as follows: .. image:: _static/img/jpeg_codec_differentiable.png .. math:: \text{JPEG}_{\text{diff}}(I, q, QT_{y}, QT_{c}) = \hat{I} Where: - :math:`I` is the original image to be coded. - :math:`q` is the JPEG quality controlling the compression strength. - :math:`QT_{y}` is the luma quantization table. - :math:`QT_{c}` is the chroma quantization table. - :math:`\hat{I}` is the resulting JPEG encoded-decoded image. .. note::: The input (and output) pixel range is :math:`[0, 1]`. In case you want to handle normalized images you are required to first perform denormalization followed by normalizing the output images again. Note, that this implementation models the encoding-decoding mapping of JPEG in a differentiable setting, however, does not allow the excess of the JPEG-coded byte file itself. For more details please refer to :cite:`reich2024`. This implementation is not meant for data loading. For loading JPEG images please refer to `kornia.io`. There we provide an optimized Rust implementation for fast JPEG loading. Args: image_rgb: the RGB image to be coded. jpeg_quality: JPEG quality in the range :math:`[0, 100]` controlling the compression strength. quantization_table_y: quantization table for Y channel. Default: `None`, which will load the standard quantization table. quantization_table_c: quantization table for C channels. Default: `None`, which will load the standard quantization table. Shape: - image_rgb: :math:`(*, 3, H, W)`. - jpeg_quality: :math:`(1)` or :math:`(B)` (if used batch dim. needs to match w/ image_rgb). - quantization_table_y: :math:`(8, 8)` or :math:`(B, 8, 8)` (if used batch dim. needs to match w/ image_rgb). - quantization_table_c: :math:`(8, 8)` or :math:`(B, 8, 8)` (if used batch dim. needs to match w/ image_rgb). Return: JPEG coded image of the shape :math:`(B, 3, H, W)` Example: To perform JPEG coding with the standard quantization tables just provide a JPEG quality >>> img = torch.rand(3, 3, 64, 64, requires_grad=True, dtype=torch.float) >>> jpeg_quality = torch.tensor((99.0, 25.0, 1.0), requires_grad=True) >>> img_jpeg = jpeg_codec_differentiable(img, jpeg_quality) >>> img_jpeg.sum().backward() You also have the option to provide custom quantization tables >>> img = torch.rand(3, 3, 64, 64, requires_grad=True, dtype=torch.float) >>> jpeg_quality = torch.tensor((99.0, 25.0, 1.0), requires_grad=True) >>> quantization_table_y = torch.randint(1, 256, size=(3, 8, 8), dtype=torch.float) >>> quantization_table_c = torch.randint(1, 256, size=(3, 8, 8), dtype=torch.float) >>> img_jpeg = jpeg_codec_differentiable(img, jpeg_quality, quantization_table_y, quantization_table_c) >>> img_jpeg.sum().backward() In case you want to control the quantization purly base on the quantization tables use a JPEG quality of 99.5. Setting the JPEG quality to 99.5 leads to a QT scaling of 1, see Eq. 2 of :cite:`reich2024` for details. >>> img = torch.rand(3, 3, 64, 64, requires_grad=True, dtype=torch.float) >>> jpeg_quality = torch.ones(3) * 99.5 >>> quantization_table_y = torch.randint(1, 256, size=(3, 8, 8), dtype=torch.float) >>> quantization_table_c = torch.randint(1, 256, size=(3, 8, 8), dtype=torch.float) >>> img_jpeg = jpeg_codec_differentiable(img, jpeg_quality, quantization_table_y, quantization_table_c) >>> img_jpeg.sum().backward() N)Ú*Ú3rarbr`rXrr¥)r`Ú8rÌgrŒz?JPEG quality is out of range. Expected range is [0, 100], got [z, z"]. Consider clipping jpeg_quality.r¿rVz#Batch dimensions do not match. Got z images and z quantization tables (Y).z quantization tables (C).z JPEG qualities.)r¡r‰r£r¢)rªr«r¬r‰rarbr£r¢r¤)rTÚmin_valÚmax_val.)r rrrNrSr ÚndimÚ unsqueezer ÚaminÚitemÚamaxrÈr\Útor²r¼r)r¡r‰r¢r£rrrÅrÆrarbr¯r°r±Úimage_rgb_jpegrLrLrMr«s–Q      ÿ ÿþÿÿþÿÿþÿÿþ    üø cs2eZdZdZ  dd‡fdd „ Zdd d„Z‡ZS)ra_Differentiable JPEG encoding-decoding module. Based on :cite:`reich2024` :cite:`shin2017`, we perform differentiable JPEG encoding-decoding as follows: .. math:: \text{JPEG}_{\text{diff}}(I, q, QT_{y}, QT_{c}) = \hat{I} Where: - :math:`I` is the original image to be coded. - :math:`q` is the JPEG quality controlling the compression strength. - :math:`QT_{y}` is the luma quantization table. - :math:`QT_{c}` is the chroma quantization table. - :math:`\hat{I}` is the resulting JPEG encoded-decoded image. .. image:: _static/img/jpeg_codec_differentiable.png .. note:: The input (and output) pixel range is :math:`[0, 1]`. In case you want to handle normalized images you are required to first perform denormalization followed by normalizing the output images again. Note, that this implementation models the encoding-decoding mapping of JPEG in a differentiable setting, however, does not allow the excess of the JPEG-coded byte file itself. For more details please refer to :cite:`reich2024`. This implementation is not meant for data loading. For loading JPEG images please refer to `kornia.io`. There we provide an optimized Rust implementation for fast JPEG loading. Args: quantization_table_y: quantization table for Y channel. Default: `None`, which will load the standard quantization table. quantization_table_c: quantization table for C channels. Default: `None`, which will load the standard quantization table. Shape: - quantization_table_y: :math:`(8, 8)` or :math:`(B, 8, 8)` (if used batch dim. needs to match w/ image_rgb). - quantization_table_c: :math:`(8, 8)` or :math:`(B, 8, 8)` (if used batch dim. needs to match w/ image_rgb). - image_rgb: :math:`(*, 3, H, W)`. - jpeg_quality: :math:`(1)` or :math:`(B)` (if used batch dim. needs to match w/ image_rgb). Example: You can use the differentiable JPEG module with standard quantization tables by >>> diff_jpeg_module = JPEGCodecDifferentiable() >>> img = torch.rand(2, 3, 32, 32, requires_grad=True, dtype=torch.float) >>> jpeg_quality = torch.tensor((99.0, 1.0), requires_grad=True) >>> img_jpeg = diff_jpeg_module(img, jpeg_quality) >>> img_jpeg.sum().backward() You can also specify custom quantization tables to be used by >>> quantization_table_y = torch.randint(1, 256, size=(2, 8, 8), dtype=torch.float) >>> quantization_table_c = torch.randint(1, 256, size=(2, 8, 8), dtype=torch.float) >>> diff_jpeg_module = JPEGCodecDifferentiable(quantization_table_y, quantization_table_c) >>> img = torch.rand(2, 3, 32, 32, requires_grad=True, dtype=torch.float) >>> jpeg_quality = torch.tensor((99.0, 1.0), requires_grad=True) >>> img_jpeg = diff_jpeg_module(img, jpeg_quality) >>> img_jpeg.sum().backward() In case you want to learn the quantization tables just pass parameters `nn.Parameter` >>> quantization_table_y = torch.nn.Parameter(torch.randint(1, 256, size=(2, 8, 8), dtype=torch.float)) >>> quantization_table_c = torch.nn.Parameter(torch.randint(1, 256, size=(2, 8, 8), dtype=torch.float)) >>> diff_jpeg_module = JPEGCodecDifferentiable(quantization_table_y, quantization_table_c) >>> img = torch.rand(2, 3, 32, 32, requires_grad=True, dtype=torch.float) >>> jpeg_quality = torch.tensor((99.0, 1.0), requires_grad=True) >>> img_jpeg = diff_jpeg_module(img, jpeg_quality) >>> img_jpeg.sum().backward() Nr¢úTensor | Parameter | Noner£rÚNonecs„tƒ ¡|durtddƒn|}|durtddƒn|}t|tƒr'| d|¡n| d|¡t|tƒr:| d|¡dS| d|¡dS)Nr¢r£)ÚsuperÚ__init__rNrSÚ isinstancerÚregister_parameterÚregister_buffer)Úselfr¢r£©Ú __class__rLrMrÙ“s    z JPEGCodecDifferentiable.__init__r¡r r‰cCs<|j}|j}|j ||¡}|j ||¡}t||||d}|S)N)r‰r£r¢)rrr¢rÔr£r)rÝr¡r‰rrr¢r£rÕrLrLrMÚforward¥süzJPEGCodecDifferentiable.forwardro)r¢rÖr£rÖrr×)r¡r r‰r rr )Ú__name__Ú __module__Ú __qualname__Ú__doc__rÙràÚ __classcell__rLrLrÞrMrKs Iý)rrrrrr )rTr rr )rTr rarerbrerr )r„r rr )rTr r‰r rŠr rr )r’r rr“)ržr rr ) r¡r r‰r r¢r r£r rr“)rªr r«r r¬r r‰r rarerbrer¢r r£r rr )r½r rr¾ro) r¡r r‰r r¢rÉr£rÉrr )1Ú __future__rrÁrJÚtorch.nn.functionalÚnnÚ functionalrÃÚ kornia.colorrrÚkornia.constantsrÚ kornia.corerrrr r ÚModuleÚkornia.core.checkr r r Ú!kornia.geometry.transform.affwarprÚkornia.utils.imagerÚkornia.utils.miscrrrÚ__all__rNrSrdrgrrƒrˆrr‘rr r²r¼rÈrrrLrLrLrMÚsB                "  / 6ü