o ߥi@sddlmZddlZddlmZddlmZddlmZddlm Z ddl m Z m Z dd l mZmZmZd gZe je jejd Gd d d eZdS) )deepcopyN)nn)Models) TorchModel)MODELS) ModelFileTasks) BertEncoder BertModelBertPreTrainedModelSiameseUieModel) module_namecsXeZdZdZfddZdddZddZd d Z dd dZddZ ddZ Z S)r aSiameseUIE general information extraction model, based on the construction idea of prompt (Prompt) + text (Text), uses pointer network (Pointer Network) to realize segment extraction (Span Extraction), so as to realize named entity recognition (NER), relation extraction (RE), Extraction of various tasks such as event extraction (EE), attribute sentiment extraction (ABSA), etc. Different from the existing general information extraction tasks on the market: c s\t|||_t|jdd|_t|j|_t |j d|_ t |j d|_ | dS)NT)add_pooling_layerr )super__init__configr plmrDropouthidden_dropout_probdropoutLinear hidden_size head_clsf tail_clsfset_crossattention_layer)selfrkwargs __class__Z/home/ubuntu/.local/lib/python3.10/site-packages/modelscope/models/nlp/bert/siamese_uie.pyrs  zSiameseUieModel.__init__cCsdt|j}||_|jj|8_t||_|jjj|jjd|j_|jjjd|jj|jj_dS)N)rrnum_hidden_layersr crossattentionrencoderlayer)rr#crossattention_configr r r!r%s    z(SiameseUieModel.set_crossattention_layerc Cs|d}||d}||d}dd||}||d}|d|d}t|ddddf}tj||gdd}tj||gdd}tj|dd}tj|dd}||S)Nrr gmBdim)sizeviewtorch zeros_likecat logsumexpmean) ry_predy_true batch_size y_pred_neg y_pred_poszerosneg_losspos_lossr r r! circle_loss/s     zSiameseUieModel.circle_losscCsrtj||gdd}tj||gdd}|j||dd}|j||ddddd|dddf}|S)Nr r*r)) hidden_statesattention_maskr)r.r0rget_extended_attention_maskr,r$)rr<r=encoder_hidden_statesencoder_attention_maskcat_hidden_statescat_attention_maskr r r!get_cross_attention_output=s$ z*SiameseUieModel.get_cross_attention_outputNFcCs,|rt|nd}|j||||dd}|S)N)r=token_type_ids position_idsr)r. ones_liker)r input_idsr=rEis_hintrDsequence_outputr r r!get_plm_sequence_outputLsz'SiameseUieModel.get_plm_sequence_outputcCs|||}|d|ddksJt|dd|d}||j}|j|||dd} |||| |}|| d} | | d} |j } | | |} | | |}d| |iS)atrain forward Args: input_ids (Tensor): input token ids of text. attention_masks (Tensor): attention_masks of text. hint_ids (Tensor): input token ids of prompt. cross_attention_masks (Tensor): attention_masks of prompt. head_labels (Tensor): labels of start position. tail_labels (Tensor): labels of end position. Returns: Dict[str, float]: the loss Example: {"loss": 0.5091743} r ir r(TrHr(loss) rJr,r.arangeexpandtodevicerCrsqueezerr;)rrGattention_maskshint_idscross_attention_masks head_labels tail_labelsrIrEhint_sequence_output head_logits tail_logits loss_func head_loss tail_lossr r r!forwardYs.    zSiameseUieModel.forwardc Cst|dd|d}||j}|j|||dd}|||||}|| d}| | d}|d|d} |d|d} | | fS)ao Args: sequence_output(tensor): 3-dimension tensor (batch size, sequence length, hidden size) attention_masks(tensor): attention mask, 2-dimension tensor (batch size, sequence length) hint_ids(tensor): token ids of prompt 2-dimension tensor (batch size, sequence length) cross_attention_masks(tensor): cross attention mask, 2-dimension tensor (batch size, sequence length) Default Returns: head_probs(tensor): 2-dimension tensor(batch size, sequence length) tail_probs(tensor): 2-dimension tensor(batch size, sequence length) r rKTrLr(i) r.rNr,rOrPrQrJrCrrRr) rrIrSrTrUrErXrYrZ head_probs tail_probsr r r!fast_inference}s$  zSiameseUieModel.fast_inference)r")NF) __name__ __module__ __qualname____doc__rrr;rCrJr^ra __classcell__r r rr!r s    $)copyrr.rmodelscope.metainformodelscope.models.basermodelscope.models.builderrmodelscope.utils.constantrrbackboner r r __all__register_module siamese_uiebertr r r r r!s