o iz)@sdZddlmZddlmZddlZddlmZddlmZddl m Z m Z dd l m Z d d lmZee d d Gddde Ze GdddeZGdddejZGdddejZGdddejZGdddejZGdddejZe dd GdddeZddgZdS)zPyTorch ViTMatte model.) dataclass)OptionalN)nn)PreTrainedModel) ModelOutputauto_docstring) load_backbone)VitMatteConfigz4 Class for outputs of image matting models. ) custom_introc@sbeZdZUdZdZeejed<dZ eejed<dZ ee ejed<dZ ee ejed<dS)ImageMattingOutputa loss (`torch.FloatTensor` of shape `(1,)`, *optional*, returned when `labels` is provided): Loss. alphas (`torch.FloatTensor` of shape `(batch_size, num_channels, height, width)`): Estimated alpha values. hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`): Tuple of `torch.FloatTensor` (one for the output of the embeddings, if the model has an embedding layer, + one for the output of each stage) of shape `(batch_size, sequence_length, hidden_size)`. Hidden-states (also called feature maps) of the model at the output of each stage. Nlossalphas hidden_states attentions) __name__ __module__ __qualname____doc__rrtorch FloatTensor__annotations__rrtuplerrrk/home/ubuntu/veenaModal/venv/lib/python3.10/site-packages/transformers/models/vitmatte/modeling_vitmatte.pyr s  r c@s2eZdZUeed<dZdZgZdej fddZ dS)VitMattePreTrainedModelconfig pixel_valuesTmodulecCsJt|tjtjfr!|jjjd|jjd|j dur#|j j dSdSdS)Ng)meanstd) isinstancerConv2d BatchNorm2dweightdatanormal_rinitializer_rangebiaszero_)selfrrrr _init_weights<s  z%VitMattePreTrainedModel._init_weightsN) rrrr rmain_input_namesupports_gradient_checkpointing_no_split_modulesrModuler,rrrrr5s rcs*eZdZdZdfdd ZddZZS) VitMatteBasicConv3x3zP Basic convolution layers including: Conv3x3, BatchNorm2d, ReLU layers. r csBttj||d||dd|_tj||jd|_t|_ dS)NrF) in_channels out_channels kernel_sizestridepaddingr))eps) super__init__rr#convr$batch_norm_eps batch_normReLUrelu)r+rr3r4r6r7 __class__rrr:Hs zVitMatteBasicConv3x3.__init__cCs"||}||}||}|SN)r;r=r?r+ hidden_staterrrforwardUs   zVitMatteBasicConv3x3.forward)r2r rrrrr:rE __classcell__rrr@rr1Cs r1c(eZdZdZfddZddZZS)VitMatteConvStreamzc Simple ConvStream containing a series of basic conv3x3 layers to extract detail features. cstd}|jdur|jj}|j}t|_|g||_t t |jdD]}|j|}|j|d}|j t |||q'dS)Nr ) r9r:backbone_config num_channelsconvstream_hidden_sizesr ModuleListconvs conv_chansrangelenappendr1)r+rr3r4iin_chan_ out_chan_r@rrr:bs     zVitMatteConvStream.__init__cCsJd|i}|}tt|jD]}|j||}dt|d}|||<q |S)Ndetailed_feature_map_0detailed_feature_map_r )rQrRrOstr)r+rout_dict embeddingsrTname_rrrrEus zVitMatteConvStream.forwardrFrrr@rrI]s rIcrH)VitMatteFusionBlockz\ Simple fusion block to fuse features from ConvStream and Plain Vision Transformer. cs"tt|||ddd|_dS)Nr )r6r7)r9r:r1r;)r+rr3r4r@rrr:s zVitMatteFusionBlock.__init__cCs4tjj|dddd}tj||gdd}||}|S)Nr2bilinearF) scale_factormode align_cornersr )dim)r functional interpolatercatr;)r+featuresdetailed_feature_mapupscaled_featuresoutrrrrEs zVitMatteFusionBlock.forwardrFrrr@rr]s r]crH) VitMatteHeadzJ Simple Matting Head, containing only conv3x3 and conv1x1 layers. c sZt|jd}d}ttj||ddddt|tdtj|ddddd|_dS)Nrr )r5r6r7Tr) r9r:fusion_hidden_sizesr Sequentialr#r$r> matting_convs)r+rr3 mid_channelsr@rrr:s   zVitMatteHead.__init__cCs||}|SrB)rorCrrrrEs zVitMatteHead.forwardrFrrr@rrjs  rjcrH)VitMatteDetailCaptureModulezG Simple and lightweight Detail Capture Module for ViT Matting. c stt|jt|jdkrtd||_t||_|jj |_ t |_ |j g|j|_tt|jdD]}|j t||j||j |d |j|ddq8t||_dS)Nr z_The length of fusion_hidden_sizes should be equal to the length of convstream_hidden_sizes + 1.)rr3r4)r9r:rRrmrM ValueErrorrrI convstreamrPrrN fusion_blocks hidden_sizefusion_channelsrQrSr]rj matting_head)r+rrTr@rrr:s&     z$VitMatteDetailCaptureModule.__init__cCs`||}tt|jD]}dtt|j|d}|j||||}q t||}|S)NrXr )rsrQrRrtrYrsigmoidrw)r+rfrdetail_featuresrTdetailed_feature_map_namerrrrrEs z#VitMatteDetailCaptureModule.forwardrFrrr@rrqs rqzX ViTMatte framework leveraging any vision backbone e.g. for ADE20k, CityScapes. c sbeZdZfddZe     d deejdeedeedeejdeef d d Z Z S) VitMatteForImageMattingcs2t|||_t||_t||_|dSrB)r9r:rr backbonerqdecoder post_init)r+rr@rrr:s    z VitMatteForImageMatting.__init__Nroutput_attentionsoutput_hidden_stateslabels return_dictc Cs|dur|n|jj}|dur|n|jj}|dur|n|jj}d}|dur(td|jj|||d}|jd}|||} |sR| f|dd} |durP|f| S| St || |j |j dS)a8 labels (`torch.LongTensor` of shape `(batch_size, height, width)`, *optional*): Ground truth image matting for computing the loss. Examples: ```python >>> from transformers import VitMatteImageProcessor, VitMatteForImageMatting >>> import torch >>> from PIL import Image >>> from huggingface_hub import hf_hub_download >>> processor = VitMatteImageProcessor.from_pretrained("hustvl/vitmatte-small-composition-1k") >>> model = VitMatteForImageMatting.from_pretrained("hustvl/vitmatte-small-composition-1k") >>> filepath = hf_hub_download( ... repo_id="hf-internal-testing/image-matting-fixtures", filename="image.png", repo_type="dataset" ... ) >>> image = Image.open(filepath).convert("RGB") >>> filepath = hf_hub_download( ... repo_id="hf-internal-testing/image-matting-fixtures", filename="trimap.png", repo_type="dataset" ... ) >>> trimap = Image.open(filepath).convert("L") >>> # prepare image + trimap for the model >>> inputs = processor(images=image, trimaps=trimap, return_tensors="pt") >>> with torch.no_grad(): ... alphas = model(**inputs).alphas >>> print(alphas.shape) torch.Size([1, 1, 640, 960]) ```NzTraining is not yet supported)rrrkr )rrrr) ruse_return_dictrrNotImplementedErrorr|forward_with_filtered_kwargs feature_mapsr}r rr) r+rrrrrroutputsrfroutputrrrrEs*)  zVitMatteForImageMatting.forward)NNNNN) rrrr:rrrTensorboolrErGrrr@rr{s&  r{)r dataclassesrtypingrrrmodeling_utilsrutilsrrutils.backbone_utilsr configuration_vitmatter r rr0r1rIr]rjrqr{__all__rrrrs4       #) Q