o ۷i@sddlmZddlmZmZddlZddlmZddlm Z ddl m Z ddl m Z mZd d lmZd d lmZeeZeGd dde Ze Gddde ZdgZdS)) dataclass)OptionalUnionN)Cache)$ImageClassifierOutputWithNoAttention)PreTrainedModel)auto_docstringlogging)AutoModelForImageTextToText)ShieldGemma2Configc@s$eZdZUdZdZeejed<dS)0ShieldGemma2ImageClassifierOutputWithNoAttentionz^ShieldGemma2 classifies imags as violative or not relative to a specific policy Args: N probabilities) __name__ __module__ __qualname____doc__rrtorchTensor__annotations__rrl/home/ubuntu/vllm_env/lib/python3.10/site-packages/transformers/models/shieldgemma2/modeling_shieldgemma2.pyr#s rc s(eZdZUeed<dddddZdeffdd Zd d Zd d Zd dZ ddZ ddZ ddZ ddZ e              d*deejdeejdeejdeejdeedeejdeejd eejd!eejd"eed#eed$eed%eed&eeejfd'efd(d)ZZS)+"ShieldGemma2ForImageClassificationconfigzmodel.model.language_modelzmodel.model.vision_towerz!model.model.multi_modal_projectorz model.lm_head)zmodel.language_model.modelzmodel.vision_towerzmodel.multi_modal_projectorzmodel.language_model.lm_headcs<tj|dt|dd|_t|dd|_tj|d|_dS)N)ryes_token_indexi *no_token_indexi)super__init__getattrrrr from_configmodel)selfr __class__rrr6sz+ShieldGemma2ForImageClassification.__init__cC |jjSN)r"language_modelget_input_embeddingsr#rrrr)< z7ShieldGemma2ForImageClassification.get_input_embeddingscC|jj|dSr')r"r(set_input_embeddings)r#valuerrrr-?z7ShieldGemma2ForImageClassification.set_input_embeddingscCr&r')r"r(get_output_embeddingsr*rrrr0Br+z8ShieldGemma2ForImageClassification.get_output_embeddingscCr,r')r"r(set_output_embeddings)r#new_embeddingsrrrr1Er/z8ShieldGemma2ForImageClassification.set_output_embeddingscCr,r')r"r( set_decoder)r#decoderrrrr3Hr/z.ShieldGemma2ForImageClassification.set_decodercCr&r')r"r( get_decoderr*rrrr5Kr+z.ShieldGemma2ForImageClassification.get_decodercCr&r')r"r( tie_weightsr*rrrr6Nr+z.ShieldGemma2ForImageClassification.tie_weightsNr input_ids pixel_valuesattention_mask position_idspast_key_valuestoken_type_idscache_position inputs_embedslabels use_cacheoutput_attentionsoutput_hidden_states return_dictlogits_to_keepreturncKsh|jd||||||||| | | | | |d|}|j}|ddd|j|jgf}tj|dd}t||dS)aY Returns: A `ShieldGemma2ImageClassifierOutputWithNoAttention` instance containing the logits and probabilities associated with the model predicting the `Yes` or `No` token as the response to that prompt, captured in the following properties. * `logits` (`torch.Tensor` of shape `(batch_size, 2)`): The first position along dim=1 is the logits for the `Yes` token and the second position along dim=1 is the logits for the `No` token. * `probabilities` (`torch.Tensor` of shape `(batch_size, 2)`): The first position along dim=1 is the probability of predicting the `Yes` token and the second position along dim=1 is the probability of predicting the `No` token. ShieldGemma prompts are constructed such that predicting the `Yes` token means the content *does violate* the policy as described. If you are only interested in the violative condition, use `violated = outputs.probabilities[:, 1]` to extract that slice from the output tensors. When used with the `ShieldGemma2Processor`, the `batch_size` will be equal to `len(images) * len(policies)`, and the order within the batch will be img1_policy1, ... img1_policyN, ... imgM_policyN. )r7r8r9r:r;r<r=r>r?r@rArBrCrDN)dim)logitsrr)r"rHrrrsoftmaxr)r#r7r8r9r:r;r<r=r>r?r@rArBrCrD lm_kwargsoutputsrHselected_logitsrrrrforwardQs2'z*ShieldGemma2ForImageClassification.forward)NNNNNNNNNNNNNr)rrrrr_checkpoint_conversion_mappingrr)r-r0r1r3r5r6r rr LongTensor FloatTensorrrboolrintrrM __classcell__rrr$rr,sz       r) dataclassesrtypingrrr cache_utilsrmodeling_outputsrmodeling_utilsrutilsr r autor configuration_shieldgemma2r get_loggerrloggerrr__all__rrrrs       f