o -i@sddlmZmZddlmZddlmZmZddlm Z m Z ddl Z ddl mZddlmZddlmZmZmZmZddlmZd d lmZd d lmZe d edZeeZeGdddZGdddee eZ dS))ABCabstractmethod)Mapping) dataclassfield)GenericTypeVarN)Image)AudioDummyOptionsBaseDummyOptionsImageDummyOptionsVideoDummyOptions) init_logger)MultiModalDataDict)BaseProcessingInfo_I)boundc@s^eZdZUdZeeeBed<eed<e e dZ e ee fed<e e dZe ee fed<dS)ProcessorInputszq Represents the keyword arguments to [`vllm.multimodal.processing.BaseMultiModalProcessor.apply`][]. promptmm_data)default_factoryhf_processor_mm_kwargstokenization_kwargsN)__name__ __module__ __qualname____doc__strlistint__annotations__rrdictrrobjectrr%r%d/home/ubuntu/veenaModal/venv/lib/python3.10/site-packages/vllm/multimodal/processing/dummy_inputs.pyrs rcs:eZdZdZdeddffdd Zedeee fdefdd Z e dd e deee fd eee fdBde fd d Z dd e deee fd eee fdBdefddZddde de dedBdeejfddZddde de de dedBdeejf ddZddde de de de dedBdeejf ddZZS) BaseDummyInputsBuilderz_ Abstract base class that constructs the dummy data to profile multi-modal models. inforeturnNcst||_dSN)super__init__r()selfr( __class__r%r&r,/s  zBaseDummyInputsBuilder.__init__ mm_countscCt)zD Build the text input corresponding to `mm_counts`. NotImplementedError)r-r0r%r%r&get_dummy_text4sz%BaseDummyInputsBuilder.get_dummy_textseq_len mm_optionscCr1)a Build the multimodal input which, after processing, results in the maximum possible number of placeholder tokens. Args: seq_len: Sequence length mm_counts: Count of items per modality mm_options: Configurable options per modality (optional). If None, use model defaults for backward compatibility. If provided, models can use these to customize dummy data generation. r2)r-r5r0r6r%r%r&get_dummy_mm_data;sz(BaseDummyInputsBuilder.get_dummy_mm_datacCs.||}||||}ddi}t|||dS)a, Build the input which, after processing, results in the maximum possible number of placeholder tokens. Args: seq_len: Sequence length mm_counts: Count of items per modality mm_options: Configurable options per modality (optional) truncationF)rrr)r4r7r)r-r5r0r6 dummy_text dummy_mm_datarr%r%r&get_dummy_processor_inputsPs z1BaseDummyInputsBuilder.get_dummy_processor_inputs) overrideslength num_audiosr<cCsR|dkrgS|r|jr|j|krtd|j|t||j}t|f}|g|S)NrzOaudio.length override (%d) exceeds model's maximum length (%d), will be ignored)r=loggerwarningminnpzeros)r-r=r>r<audior%r%r&_get_dummy_audiosls     z(BaseDummyInputsBuilder._get_dummy_audioswidthheight num_imagescCs|dkrgS|r4|jr|j|krtd|j|t||j}|jr4|j|kr.td|j|t||j}tjd||fdd}|g|S)NrzMimage.width override (%d) exceeds model's maximum width (%d), will be ignoredzOimage.height override (%d) exceeds model's maximum height (%d), will be ignoredRGB)color)rFr?r@rArGr new)r-rFrGrHr<imager%r%r&_get_dummy_imagess*     z(BaseDummyInputsBuilder._get_dummy_images num_frames num_videoscCs|dkrgS|rJ|jr|j|krtd|j|t||j}|jr4|j|kr.td|j|t||j}|jrJ|j|krDtd|j|t||j}tj|||dfdtjd}|g|S)Nrz]video.num_frames override (%d) exceeds model's maximum number of frames (%d), will be ignoredzMvideo.width override (%d) exceeds model's maximum width (%d), will be ignoredzOvideo.height override (%d) exceeds model's maximum height (%d), will be ignoredrJ)dtype) rOr?r@rArFrGrBfulluint8)r-rFrGrOrPr<videor%r%r&_get_dummy_videoss:        z(BaseDummyInputsBuilder._get_dummy_videosr*)rrrrrr,rrrr!r4r rr7rr;r r nptNDArrayrEr r rNr rV __classcell__r%r%r.r&r')s|    !  'r')!abcrrcollections.abcr dataclassesrrtypingrrnumpyrB numpy.typingrWPILr vllm.config.multimodalr r r r vllm.loggerrinputsrcontextrrrr?rr'r%r%r%r&s