o oi,<@sddlmZddlmZmZmZmZddlZddlm Z ddl m Z ddl m Z ddlmZmZddlmZd gZGd d d e ZdS) ) annotations)AnyOptionalSequenceUnionN)MixAugmentationBaseV2)_validate_input_dtype)DataKey)Tensortensor) KORNIA_CHECKRandomTransplantationcsleZdZdZ    d+d,fdd Zd-ddZd.ddZd.ddZddd/d#d$Zddd%d0d)d*Z Z S)1r aRandomTransplantation augmentation. .. image:: _static/img/RandomTransplantation.png Randomly transplant (copy and paste) image features and corresponding segmentation masks between images in a batch. The transplantation transform works as follows: 1. Based on the parameter `p`, a certain number of images in the batch are selected as acceptor of a transplantation. 2. For each acceptor, the image below in the batch is selected as donor (via circling: :math:`i - 1 \mod B`). 3. From the donor, a random label is selected and the corresponding image features and segmentation mask are transplanted to the acceptor. The augmentation is described in `Semantic segmentation of surgical hyperspectral images under geometric domain shifts` :cite:`sellner2023semantic`. Args: excluded_labels: sequence of labels which should not be transplanted from a donor. This can be useful if only parts of the image are annotated and the non-annotated regions (with a specific label index) should be excluded from the augmentation. If no label is left in the donor image, nothing is transplanted. p: probability for applying an augmentation to an image. This parameter controls how many images in a batch receive a transplant. p_batch: probability for applying an augmentation to a batch. This param controls the augmentation probabilities batch-wise. data_keys: the input type sequential for applying augmentations. There must be at least one "mask" tensor. If no data keys are given, the first tensor is assumed to be `DataKey.INPUT` and the second tensor `DataKey.MASK`. Accepts "input", "mask". Note: - This augmentation requires that segmentation masks are available for all images in the batch and that at least some objects in the image are annotated. - When using this class directly (`RandomTransplantation()(...)`), it works for arbitrary spatial dimensions including 2D and 3D images. When wrapping in :class:`kornia.augmentation.AugmentationSequential`, use :class:`kornia.augmentation.RandomTransplantation` for 2D and :class:`kornia.augmentation.RandomTransplantation3D` for 3D images. Inputs: - Segmentation mask tensor which is used to determine the objects for transplantation: :math:`(B, *)`. - (optional) Additional image or mask tensors where the features are transplanted based on the first segmentation mask: :math:`(B, C, *)` (`DataKey.INPUT`) or :math:`(B, *)` (`DataKey.MASK`). Returns: Tensor | list[Tensor]: Tensor: - Augmented mask tensors: :math:`(B, *)`. list[Tensor]: - Augmented mask tensors: :math:`(B, *)`. - Additional augmented image or mask tensors: :math:`(B, C, *)` (`DataKey.INPUT`) or :math:`(B, *)` (`DataKey.MASK`). Examples: >>> import torch >>> rng = torch.manual_seed(0) >>> aug = RandomTransplantation(p=1.) >>> image = torch.randn(2, 3, 5, 5) >>> mask = torch.randint(0, 3, (2, 5, 5)) >>> mask tensor([[[0, 0, 1, 1, 0], [1, 2, 0, 0, 0], [1, 2, 1, 1, 0], [0, 0, 0, 0, 2], [2, 2, 2, 0, 2]], [[2, 0, 0, 2, 1], [2, 1, 0, 2, 1], [2, 0, 1, 0, 2], [2, 2, 2, 0, 2], [2, 1, 0, 0, 0]]]) >>> image_out, mask_out = aug(image, mask) >>> image_out.shape torch.Size([2, 3, 5, 5]) >>> mask_out.shape torch.Size([2, 5, 5]) >>> mask_out tensor([[[2, 0, 1, 2, 0], [2, 2, 0, 2, 0], [2, 2, 1, 1, 2], [2, 2, 2, 0, 2], [2, 2, 2, 0, 2]], [[0, 0, 0, 2, 0], [2, 1, 0, 0, 0], [2, 0, 1, 0, 0], [0, 0, 0, 0, 2], [2, 1, 0, 0, 0]]]) >>> aug._params["selected_labels"] # Image 0 received label 2 from image 1 and image 1 label 0 from image 0 tensor([2, 0]) You can apply the same augmentation again in which case the same objects get transplanted between the images: >>> aug._params["selection"] # The pixels (objects) which get transplanted tensor([[[ True, False, False, True, False], [ True, False, False, True, False], [ True, False, False, False, True], [ True, True, True, False, True], [ True, False, False, False, False]], [[ True, True, False, False, True], [False, False, True, True, True], [False, False, False, False, True], [ True, True, True, True, False], [False, False, False, True, False]]]) >>> image2 = torch.zeros(2, 3, 5, 5) >>> image2[1] = 1 >>> image2[:, 0] tensor([[[0., 0., 0., 0., 0.], [0., 0., 0., 0., 0.], [0., 0., 0., 0., 0.], [0., 0., 0., 0., 0.], [0., 0., 0., 0., 0.]], [[1., 1., 1., 1., 1.], [1., 1., 1., 1., 1.], [1., 1., 1., 1., 1.], [1., 1., 1., 1., 1.], [1., 1., 1., 1., 1.]]]) >>> image_out2, mask_out2 = aug(image2, mask, params=aug._params) >>> image_out2[:, 0] tensor([[[1., 0., 0., 1., 0.], [1., 0., 0., 1., 0.], [1., 0., 0., 0., 1.], [1., 1., 1., 0., 1.], [1., 0., 0., 0., 0.]], [[0., 0., 1., 1., 0.], [1., 1., 0., 0., 0.], [1., 1., 1., 1., 0.], [0., 0., 0., 0., 1.], [1., 1., 1., 0., 1.]]]) N??excluded_labels&Optional[Union[Sequence[int], Tensor]]pfloatp_batch data_keys#Optional[list[str | int | DataKey]]returnNonecstj||d|durg}t|tst|}||_t|jjdkd|jjd|dur3tj tj g}dd|D|_ d|_ dS)N)rrz:excluded_labels must be a 1-dimensional sequence, but got dimensions.cSg|]}t|qSr get.0inprr_/home/ubuntu/.local/lib/python3.10/site-packages/kornia/augmentation/_2d/mix/transplantation.py z2RandomTransplantation.__init__..) super__init__ isinstancer r rr ndimr INPUTMASKr _channel_dim)selfrrrr __class__rr"r&s    zRandomTransplantation.__init__inputr paramsdict[str, Tensor]flagsdict[str, Any]cCs|SNr)r,r/r0r2rrr"apply_non_transform_masksz.RandomTransplantation.apply_non_transform_maskacceptordonor selectioncCs$|j|jd|}||||<|S)Ndim) unsqueezer+ expand_asr,r6r7r8rrr"transform_inputs z%RandomTransplantation.transform_inputcCs||||<|Sr4rr=rrr"transform_masks z$RandomTransplantation.transform_mask) extra_args list[DataKey]r@'Optional[dict[DataKey, dict[str, Any]]]c stt|t|kdt|dt|d||tj}t||D]?\}}|tjkrbt|j|jdkd|jd|jdt|t fdd t |Dkd |d|dq#d |vrtt |d d kd|d <d|vr|d dt|d |d<d|vrj j|jkrj |j_ g}tt|dD]7} ||d| } | } | | ddj ddkjdd} t| dkr| t t| d} || qt|dkrt |nt d|d<d|vr[t jt|d g|jddRt j|jd} |d}t|jdkd|jdtt|t|d kdt|dt|d dttt|d|D]\} } ||d| } | | | | kdq?| |d<|S)aiCompute parameters for the transformation which are based on one or more input tensors. This function is, for example, called by :class:`kornia.augmentation.container.ops.AugmentationSequentialOps` before the augmentation is applied on the individual input tensors. Args: *input: All input tensors passed to the augmentation pipeline. data_keys: Associated data key for every input tensor. params: Dictionary of parameters computed so far by the augmentation pipeline (e.g. including the `batch_prob`). extra_args: Optional dictionary of extra arguments with specific options for different input types. Returns: Updated dictionary of parameters with the necessary information to apply the augmentation on all input tensors separately. zLength of keys (z#) does not match number of inputs (z).rzmEvery image input must have one additional dimension (channel dimension) than the segmentation mask, but got z for the input image and z for the segmentation mask.csg|] \}}|jkr|qSr)r+)r isr,rr"r#sz;RandomTransplantation.params_from_input..zmThe dimensions of the input image and segmentation mask must match except for the channel dimension, but got acceptor_indices batch_probrr donor_indicesselected_labelsr9r8N)dtypedevicez8selected_labels must be a 1-dimensional tensor, but got rz&There cannot be more selected labels (z9) than images where this augmentation should be applied (T)r lenindexr r*zipr)r(sizetorchSize enumeratewhererrLtorangeuniqueviewallrandpermappendstackemptyzerosshapebool masked_fill_)r,rr0r@r/mask_inputkey donor_labelsd current_masklabelsselected_labelr8rIrrEr"params_from_inputsz &&  $ $   z'RandomTransplantation.params_from_input)r0rOptional[dict[str, Tensor]]kwargsTensor | list[Tensor]c s|durj}ndd|D}|dur#||tj}|j_n|_tfdddDr?jj ||jdg}t ||D]{\}} | jd } | jd} |tj krt | tjtjtjgd | | jd } | jj} | jdf| jj} n,|tjkr| | jd } | jj} | jdf| jj} nt|| qFt|d kr|d S|S) NcSrrrrrrr"r#3r$z1RandomTransplantation.forward..c3s|]}|jvVqdSr4)_params)r krErr" ;sz0RandomTransplantation.forward..)rFrHr8)rr0rFrH)accepted_dtypesr8rr)rrNr r*forward_parametersr_rnanyupdaterjrOcloner)rrQfloat16float32float64r>apply_non_transformr2 index_putr5r?NotImplementedErrorr[rM)r,r0rr/rlkeysrboutputsdcatercr6r7appliedoutputrrEr"forward(sB      zRandomTransplantation.forward)NrrN) rrrrrrrrrr)r/r r0r1r2r3rr )r6r r7r r8r rr ) r/r rrAr0r1r@rBrr1) r/r r0rkrrrlr3rrm) __name__ __module__ __qualname____doc__r&r5r>r?rjr __classcell__rrr-r"r !s     _) __future__rtypingrrrrrQ kornia.augmentation._2d.mix.baserkornia.augmentation.utilsrkornia.constantsr kornia.corer r kornia.core.checkr __all__r rrrr"s