o eiq@sddlmZddlZddlmZddlmZddlmZddl m Z m Z dd l m Z d d lmZe eZeGd d d eZe GdddeZdgZdS)) dataclassN)Cache)$ImageClassifierOutputWithNoAttention)PreTrainedModel)auto_docstringlogging)AutoModelForImageTextToText)ShieldGemma2Configc@s$eZdZUdZdZejdBed<dS)0ShieldGemma2ImageClassifierOutputWithNoAttentionz^ShieldGemma2 classifies imags as violative or not relative to a specific policy Args: N probabilities)__name__ __module__ __qualname____doc__rtorchTensor__annotations__rrt/home/ubuntu/transcripts/venv/lib/python3.10/site-packages/transformers/models/shieldgemma2/modeling_shieldgemma2.pyr !s r c seZdZUeed<dZdddddZdeffdd Zd d Zd d Z ddZ ddZ e              d%de jdBde jdBde jdBde jdBdedBde jdBde jdBde jdBde jdBdedBdedBdedBd edBd!ee jBd"efd#d$ZZS)&"ShieldGemma2ForImageClassificationconfig)imagetextzmodel.model.language_modelzmodel.model.vision_towerz!model.model.multi_modal_projectorz model.lm_head)zmodel.language_model.modelzmodel.vision_towerzmodel.multi_modal_projectorzmodel.language_model.lm_headcsDtj|dt|dd|_t|dd|_tj|d|_|dS)N)ryes_token_indexi *no_token_indexi) super__init__getattrrrr from_configmodel post_init)selfr __class__rrr5s  z+ShieldGemma2ForImageClassification.__init__cC|jSN)r" get_decoderget_input_embeddingsr$rrrr*<z7ShieldGemma2ForImageClassification.get_input_embeddingscC|j|dSr()r"r)set_input_embeddings)r$valuerrrr.?z7ShieldGemma2ForImageClassification.set_input_embeddingscCr'r()r"r)get_output_embeddingsr+rrrr1Br,z8ShieldGemma2ForImageClassification.get_output_embeddingscCr-r()r"r)set_output_embeddings)r$new_embeddingsrrrr2Er0z8ShieldGemma2ForImageClassification.set_output_embeddingsNr input_ids pixel_valuesattention_mask position_idspast_key_valuestoken_type_idscache_position inputs_embedslabels use_cacheoutput_attentionsoutput_hidden_states return_dictlogits_to_keepreturncKsh|jd||||||||| | | | | |d|}|j}|ddd|j|jgf}tj|dd}t||dS)aY Returns: A `ShieldGemma2ImageClassifierOutputWithNoAttention` instance containing the logits and probabilities associated with the model predicting the `Yes` or `No` token as the response to that prompt, captured in the following properties. * `logits` (`torch.Tensor` of shape `(batch_size, 2)`): The first position along dim=1 is the logits for the `Yes` token and the second position along dim=1 is the logits for the `No` token. * `probabilities` (`torch.Tensor` of shape `(batch_size, 2)`): The first position along dim=1 is the probability of predicting the `Yes` token and the second position along dim=1 is the probability of predicting the `No` token. ShieldGemma prompts are constructed such that predicting the `Yes` token means the content *does violate* the policy as described. If you are only interested in the violative condition, use `violated = outputs.probabilities[:, 1]` to extract that slice from the output tensors. When used with the `ShieldGemma2Processor`, the `batch_size` will be equal to `len(images) * len(policies)`, and the order within the batch will be img1_policy1, ... img1_policyN, ... imgM_policyN. )r4r5r6r7r8r9r:r;r<r=r>r?r@rAN)dim)logitsrr)r"rErrrsoftmaxr )r$r4r5r6r7r8r9r:r;r<r=r>r?r@rA lm_kwargsoutputsrEselected_logitsrrrrforwardHs2'z*ShieldGemma2ForImageClassification.forward)NNNNNNNNNNNNNr)rrrr rinput_modalities_checkpoint_conversion_mappingrr*r.r1r2rr LongTensor FloatTensorrrboolintr rJ __classcell__rrr%rr*sv      r) dataclassesrr cache_utilsrmodeling_outputsrmodeling_utilsrutilsrrautor configuration_shieldgemma2r get_loggerrloggerr r__all__rrrrs       _