o ei. @sddlZddlmZddlmZddlmZddlmZm Z m Z m Z er+ddl m Z   dd ejd ejd ed ed ef ddZGdddeZGddde Z    dddZdS)N)center_to_corners_format)is_scipy_available)HungarianMatcher ImageLoss _set_aux_lossgeneralized_box_ioulinear_sum_assignment?inputstargets num_boxesalphagammac Csv|}tjj||dd}||d|d|}|d||}|dkr5||d|d|} | |}||S)aR Loss used in RetinaNet for dense detection: https://huggingface.co/papers/1708.02002. Args: inputs (`torch.FloatTensor` of arbitrary shape): The predictions for each example. targets (`torch.FloatTensor` with the same shape as `inputs`) A tensor storing the binary classification label for each element in the `inputs` (0 for the negative class and 1 for the positive class). num_boxes (`int`): The total number of boxes in the batch. alpha (`float`, *optional*, defaults to 0.25): Optional weighting factor in the range (0,1) to balance positive vs. negative examples. gamma (`int`, *optional*, defaults to 2): Exponent of the modulating factor (1 - p_t) to balance easy vs hard examples. Returns: Loss tensor none) reductionrr)sigmoidnn functional binary_cross_entropy_with_logitssum) r rrrrprobce_lossp_tlossalpha_trc/home/ubuntu/transcripts/venv/lib/python3.10/site-packages/transformers/loss/loss_grounding_dino.pysigmoid_focal_losss r c@seZdZeddZdS)GroundingDinoHungarianMatchercCsd|djdd\}}|ddd}|ddd}|d}tdd t||D}||jd d d }td d |D}d} d} d| || d|d } | d|| |d } | | |} tj ||dd}t t |t | }|j ||j | |j|}|||d }dd |D}dd t||d D}dd |DS)a Args: outputs (`dict`): A dictionary that contains at least these entries: * "logits": Tensor of dim [batch_size, num_queries, num_classes] with the classification logits * "pred_boxes": Tensor of dim [batch_size, num_queries, 4] with the predicted box coordinates. * "label_maps": Tuple of tensors of dim [num_classes, hidden_dim]. targets (`list[dict]`): A list of targets (len(targets) = batch_size), where each target is a dict containing: * "class_labels": Tensor of dim [num_target_boxes] (where num_target_boxes is the number of ground-truth objects in the target) containing the class labels * "boxes": Tensor of dim [num_target_boxes, 4] containing the target box coordinates. Returns: `list[Tuple]`: A list of size `batch_size`, containing tuples of (index_i, index_j) where: - index_i is the indices of the selected predictions (in order) - index_j is the indices of the corresponding selected targets (in order) For each batch element, it holds: len(index_i) = len(index_j) = min(num_queries, num_target_boxes) logitsNrrr pred_boxes label_mapscSsg|] \}}||dqS class_labelsr).0 label_maptargetrrr bz9GroundingDinoHungarianMatcher.forward..T)dimkeepdimcSsg|]}|dqSboxesrr'vrrrr*gsr @g:0yE>)pcSg|]}t|dqSr/lenr1rrrr*{cSsg|] \}}t||qSrr )r'icrrrr*|r+cSs0g|]\}}tj|tjdtj|tjdfqS))dtype)torch as_tensorint64)r'r9jrrrr*}s0)shapeflattenrr<catziprlogtcdistr r bbox_cost class_cost giou_costviewcpu enumeratesplit)selfoutputsr batch_size num_queriesout_probout_bboxr$ target_bboxrrneg_cost_classpos_cost_classrHrGrI cost_matrixsizesindicesrrrforwardDs&"z%GroundingDinoHungarianMatcher.forwardN)__name__ __module__ __qualname__r<no_gradrZrrrrr!Csr!c@s8eZdZdZddZeddZddZdd Z d S) GroundingDinoImageLossa This class computes the losses for `GroundingDinoForObjectDetection`. The process happens in two steps: 1) we compute hungarian assignment between ground truth boxes and the outputs of the model 2) we supervise each pair of matched ground-truth / prediction (supervise class and box). Args: matcher (`GroundingDinoHungarianMatcher`): Module able to compute a matching between targets and proposals. focal_alpha (`float`): Alpha parameter in focal loss. losses (`list[str]`): List of all the losses to be applied. See `get_loss` for a list of all available losses. cCs"tj|||_||_||_dSN)rModule__init__matcher focal_alphalosses)rNrcrdrerrrrbs  zGroundingDinoImageLoss.__init__c Csb|d}|j}tjdd|D|d}|djdkd}tj | | } d| i} | S) z Compute the cardinality error, i.e. the absolute error in the number of predicted non-empty boxes. This is not really a loss, it is intended for logging purposes only. It doesn't propagate gradients. r"cSr5r%r6r1rrrr*r8z;GroundingDinoImageLoss.loss_cardinality..)devicer,g?rcardinality_error) rfr<r=rmaxvaluesrrrl1_lossfloat) rNrOrrYrr"rftarget_lengths card_predcard_errrerrrloss_cardinalitysz'GroundingDinoImageLoss.loss_cardinalityc srd}tfddtt||D}tjddd}||}tj||jtjd}||tj||<|S)z> Create one_hot based on the matching indices r"csHg|] \}\}\}}|dkr|d|td|n|d|qS)rr&r$r6)r'r9r)_JrOrrr*s.zFGroundingDinoImageLoss._get_target_classes_one_hot..r$r)r-)rfr;) r<rBrLrC_get_source_permutation_idx zeros_likerflongto) rNrOrrYr"r&r$idxtarget_classes_onehotrrrr_get_target_classes_one_hots   z2GroundingDinoImageLoss._get_target_classes_one_hotc Cs~d|vrtdd|vrtd||||}|d}|d}t||}t||}|}t||||jdd}d|i} | S)z Classification loss (Binary focal loss) targets dicts must contain the key "class_labels" containing a tensor of dim [nb_target_boxes] r"z#No logits were found in the outputs text_maskz&No text_mask were found in the outputsr)r rrrrloss_ce)KeyErrorryr< masked_selectrkr rd) rNrOrrYrrx source_logitsrzr{rerrr loss_labelss&  z"GroundingDinoImageLoss.loss_labelsN) r[r\r]__doc__rbr<r^roryrrrrrr_s  r_c slt|j|j|jd} gd} t| |j| d} | |i}||d<||d<||d<||d<d}|jrHt||}|D] }||d<||d<q9||d<| |||j rj| | ||d }| ||}d d | D} |d |j |j d |j rdd  D} ||jri}t|jdD]| fdd  Dq |tfddD}||fS)N)rHrGrI)labelsr0 cardinality)rcrdrer"r#r$rzauxiliary_outputs)r"r#r$rzcSi|] \}}|d|qS_encrr'kr2rrr z7GroundingDinoForObjectDetectionLoss..r3)r{ loss_bbox loss_gioucSrrrrrrrrrrcs i|] \}}|d|qS)rprr)r9rrrs c3s(|]}|vr||VqdSr`r)r'r) loss_dict weight_dictrr s&z6GroundingDinoForObjectDetectionLoss..)r!rHrGrIr_rdrvauxiliary_lossr two_stageitemsupdatebbox_loss_coefficientgiou_loss_coefficientrangedecoder_layersr)r"rrfr#configr$rz outputs_class outputs_coordencoder_logitsencoder_pred_boxesrcre criterion outputs_lossr aux_outputencoder_outputs_lossencoder_loss_dictenc_weight_dictaux_weight_dictrr)r9rrr#GroundingDinoForObjectDetectionLosss\          r)r r)NNNN)r<torch.nnrimage_transformsrutilsrloss_for_object_detectionrrrr scipy.optimizer Tensorintrkr r!r_rrrrrs6      '=a