o }oi @sddlmZmZddlmZmZmZddlZddlm Z ddl m Z m Z eGddde Z eGdd d e ZeGd d d e ZeGd d d eZdS)) dataclassfield)ListOptionalUnionNPackedSeqParams)ImageTextRawBatchImageTextSamplec@sHeZdZUdZdZeed<edddZe j ed<dZ e e j ed <dS) LlavaNextTextSampleaz Sample type for LLaVA-Next, extending ImageTextSample to support tiled image data. This class adds additional attributes for handling high-resolution images processed as tiles, along with metadata about the tiled images. Attributes: num_media_tiles (int): The number of tiles used to represent the high-resolution image. image_sizes (torch.Tensor): A tensor representing the sizes of the tiled images. attention_mask (Optional[torch.Tensor]): An optional attention mask for the sample, used to determine which tokens or tiles are attended to during processing. Defaults to None. rnum_media_tilescC tgSNtorchtensorrr_/home/ubuntu/.local/lib/python3.10/site-packages/nemo/collections/vlm/llava_next/data/sample.py' zLlavaNextTextSample.default_factory image_sizesNattention_mask) __name__ __module__ __qualname____doc__r int__annotations__rrrrrrrrrrr s  r c@s^eZdZUdZdZeeeeefdfe d<e dddZ e j e d<e d ddZee d <d S) PackedLlavaNextTextSamplez(Sample type for packed image text sampler.__restore_key__cCtjdtjdSNr)dtyperemptyfloatrrrrr0z"PackedLlavaNextTextSample.r position_idscCtSrrrrrrr1packed_seq_paramsN)rrrrr!tuplerstrrrrr)rTensorr,rrrrrr +s r c@sReZdZUdZeedZeee d<edddZ e j e d<dZ ee j e d<dS) LlavaNextTextRawBatchaI Batch type for raw LLaVA-Next samples, supporting tiled image data. This class aggregates multiple `LlavaNextTextSample` instances into a batch for processing. It includes attributes for managing tiled images and associated metadata for each sample in the batch. Attributes: num_media_tiles (List[int]): A list containing the number of tiles for each image in the batch. image_sizes (torch.Tensor): A tensor containing the sizes of all tiled images in the batch. attention_mask (Optional[torch.Tensor]): Attention mask. Defaults to None. rr cCr rrrrrrrCrzLlavaNextTextRawBatch.rNr)rrrrrlistr rrrrrrrrrrrrr04s  r0c@s@eZdZUdZedddZejed<edddZ e ed<dS) PackedLlavaNextTextRawBatchz$Sample type for image text raw batchcCr"r#r%rrrrrKr(z$PackedLlavaNextTextRawBatch.rr)cCr*rrrrrrrLr+r,N) rrrrrr)rr/rr,rrrrrr2Gs r2) dataclassesrrtypingrrrrmegatron.core.packed_seq_paramsr/nemo.collections.multimodal.data.energon.configr r r r r0r2rrrrs