o -i60@sddlmZddlmZddlmZmZmZmZm Z ddl m Z m Z ddl mZddlmZddlmZddlmZdd lmZdd lmZerNdd lmZeeZed Zed ZeeGdddZdS))Callable)InitVar) TYPE_CHECKINGAnyClassVarLiteralcast)Fieldfield_validator) dataclass)Self)config) init_logger) safe_hash)resolve_obj_by_qualname)SchedulerInterface)generatepoolingdraft)fcfspriorityc@seZdZUdZeeed< eeed< dZe eed<dZ e eed<dZ e ed < e ed d Zeed < e e d d Zeed < e d d d Zeed< e d d d Zeed< dZeed< dZeed< dZeed< e ddZeed< e ddZeed< dZeed< dZeed< e ddZeeeBed< dZedBed< e ddZeed < e d d d Z eed!< e!d"d#Z"d$ed%fd&d'Z#d$efd(d)Z$e%dd d*d+e&d,e'd-e(d$e'fd.d/Z)deded$dfd0d1Z*ded$e+fd2d3Z,dS)4SchedulerConfigzScheduler configuration. max_model_lenis_encoder_decoderiDEFAULT_MAX_NUM_BATCHED_TOKENSDEFAULT_MAX_NUM_SEQSr runner_type)defaultgemax_num_batched_tokens max_num_seqsmax_num_partial_prefillsmax_long_partial_prefillsrlong_prefill_token_thresholdTenable_chunked_prefillFis_multimodal_model)initmax_num_encoder_input_tokensencoder_cache_sizerpolicydisable_chunked_mm_inputN)r scheduler_clsdisable_hybrid_kv_cache_managerasync_schedulingstream_intervalcKs.d|vrd|d<d|vrd|d<tdi|S)z` Factory method to create `SchedulerConfig` with default values for `InitVar`s. ri rFN)r)kwargsr1r1R/home/ubuntu/veenaModal/venv/lib/python3.10/site-packages/vllm/config/scheduler.pydefault_factorys zSchedulerConfig.default_factoryreturnrcCsd|jdur|jrddlm}|Sddlm}|Std|jt|jt s-t t d|jSt |jS)Nr)AsyncScheduler) SchedulerzpUsing custom scheduler class %s. This scheduler interface is not public and compatibility may not be maintained.r) r-r/"vllm.v1.core.sched.async_schedulerr6vllm.v1.core.sched.schedulerr7logger warning_once isinstancestrrtyper)selfr6r7r1r1r3get_scheduler_clss     z!SchedulerConfig.get_scheduler_clscCs,g}||jtt|dd}|S)a WARNING: Whenever a new field is added to this config, ensure that it is included in the factors list if it affects the computation graph. Provide a hash that uniquely identifies all the configs that affect the structure of the computation graph from input ids/embeddings to the final hidden states, excluding anything before input ids/embeddings and after the final hidden states. F)usedforsecurity)appendr!rr=encode hexdigest)r?factorshash_strr1r1r3 compute_hashs zSchedulerConfig.compute_hashwrap)modevaluehandlercCs|durdS||S)zFSkip validation if the value is `None` when initialisation is delayed.Nr1)clsrJrKr1r1r3_skip_none_validationsz%SchedulerConfig._skip_none_validationcCs|rd|_d|_d|_td|j|_|j|_|jr"td|j|jdkr>|jdkr3t |d|_td|j|j |j| |dS) NTFrzYEncoder-decoder models do not support chunked prefill nor prefix caching; disabling both.z:Chunked prefill is enabled with max_num_batched_tokens=%d.rg{Gz?zConcurrent partial prefills enabled with max_num_partial_prefills=%d, max_long_partial_prefills=%d, long_prefill_token_threshold=%d) r,r&r%r:infor!r)r*r#intr$verify_max_model_len)r?rrr1r1r3 __post_init__s0   zSchedulerConfig.__post_init__cCs|j|kr|jstd|jd|d|j|jkr'td|jd|jd|j|j|kr:td|j|j||jdkrW|jsFtd|j|krWtd |jd |d|j|jkrjtd |jd |jd |S)Nzmax_num_batched_tokens (z!) is smaller than max_model_len (z). This effectively limits the maximum sequence length to max_num_batched_tokens and makes vLLM reject longer sequences. Please increase max_num_batched_tokens or decrease max_model_len.z1) must be greater than or equal to max_num_seqs (z).zlmax_num_batched_tokens (%d) exceeds max_num_seqs * max_model_len (%d). This may lead to unexpected behavior.rzDChunked prefill must be enabled to set max_num_partial_prefills > 1.zlong_prefill_token_threshold (z,) cannot be greater than the max_model_len (zself.max_long_partial_prefills=z= must be less than or equal to self.max_num_partial_prefills=.) r!r& ValueErrorr"r:warningr#r%r$)r?rr1r1r3rPsP        z$SchedulerConfig.verify_max_model_len)-__name__ __module__ __qualname____doc__rrO__annotations__boolrrrr RunnerTyper r!r"r#r$r%r&r'r)r*r+SchedulerPolicyr,r-r=r>objectr.r/r0 staticmethodr4r@rGr classmethodrrrMrQr rPr1r1r1r3rsb             #rN) collections.abcr dataclassesrtypingrrrrrpydanticr r pydantic.dataclassesr typing_extensionsr vllm.config.utilsr vllm.loggerrvllm.utils.hashingrvllm.utils.import_utilsrvllm.v1.core.sched.interfacerrUr:r[r\rr1r1r1r3s$