o ٷiP @s4ddlZddlZddlmZddlmZmZmZddlm Z ddl m Z m Z m Z ddlmZddlmZddlmZmZdd lmZdd lmZdd lmZe ejjjZeeZd e d e e!e fde!de!ddf ddZ"de!de!dBfddZ#de e Bde fddZ$de de fddZ%de!de!fddZ&d._is_callable_valuez Filtered out zR callable object(s) from base_engine_args that are not compatible with OmegaConf: z. ) rboolrappendstr_convert_dataclasses_to_dictr"warninglen)r4r=result filtered_keyskvr(r(r)_filter_dict_like_objectHs  rHc Cst|dr|jjdkrzt|WSttfyiYSwt|tr't|St|tr0t |St |rsz/_convert_dataclasses_to_dict..keysvalues)rrI__name__r TypeError ValueErrorr0rsetlistrrrArHr7tupletyper@bytesAttributeError)r4rDr(r(r)rAvs4     " rAc Cs@z t|dd}|j}Wnettfypt|dddr,t|}|dur+td|dnBt|dddrfztd|dd}|rFd |vrF|d }ntd |Wntye}z td |d ||d}~wwtd |dYnwt}|d}t ||}t j |rt |Sd|d}t |} t j | sdSt | S)aEResolve the stage config file path from the model name. Resolves stage configuration path based on the model type and device type. First tries to find a device-specific YAML file from stage_configs/{device_type}/ directory. If not found, falls back to the default config file. Args: model: Model name or path (used to determine model_type) Returns: String path to the stage configuration file Raises: ValueError: If model_type cannot be determined FileNotFoundError: If no stage config file exists for the model type T)trust_remote_coder,Nr-z4Could not determine model_type for diffusers model: zY. Please ensure the model has 'model_type' in transformer/config.json or model_index.jsonz config.json model_typez6config.json found but missing 'model_type' for model: z&Failed to read config.json for model: z . Error: z*Could not determine model_type for model: z. Model is not in standard transformers format and does not have model_index.json. Please ensure the model has proper configuration files with 'model_type' fieldz.yamlz'vllm_omni/model_executor/stage_configs/)r rZrRr rr3r rget_default_stage_config_path PROJECT_ROOTospathexistsr@) r+ hf_configrZ config_dictr'default_config_pathmodel_type_strcomplete_config_pathstage_config_filestage_config_pathr(r(r)resolve_model_config_pathsJ           rgbase_engine_argscCs0|duri}t|}|durgSt||d}|S)aLoad stage configurations from model's default config file. Loads stage configurations based on the model type and device type. First tries to load a device-specific YAML file from stage_configs/{device_type}/ directory. If not found, falls back to the default config file. Args: model: Model name or path (used to determine model_type) Returns: List of stage configuration dictionaries Raises: FileNotFoundError: If no stage config file exists for the model type N) config_pathrh)rgload_stage_configs_from_yaml)r+rhrf stage_configsr(r(r)load_stage_configs_from_models rlric Cs|duri}t|}|j}|dd}t|}t|}|D]B}|}t|dr6|jdur6t ||j}t |dd}t|dr^|j dur^|dkr^|j }t |d d pUd } | |d <||_ ||_q|S) zLoad stage configurations from a YAML file. Args: config_path: Path to the YAML configuration file Returns: List of stage configuration dictionaries from the file's stage_args N async_chunkFr stage_typellmruntime diffusionmax_batch_size max_num_seqs)r load stage_argsrrAcreatecopyrrmergegetattrrpintrm) rirh config_datarvglobal_async_chunk stage_argbase_engine_args_tmprn runtime_cfgrrr(r(r)rjs&     rjstage_configs_pathkwargsdefault_stage_cfg_factorycCsd|dur&t|}t||d}|s"|dur |}t|}||fSg}||fS|}t||d}||fS)aLoad stage configurations from model or YAML file with fallback to defaults. Args: model: Model name or path stage_configs_path: Optional path to YAML file containing stage configurations kwargs: Engine arguments to merge with stage configs default_stage_cfg_factory: Optional callable that takes no args and returns default stage config list when no configs are found Returns: Tuple of (config_path, stage_configs) N)rh)rgrlr rwrj)r+rrrrirkdefault_stage_cfgr(r(r)load_and_resolve_stage_configs#s   routput_modalitiesdefault_modalities stage_listc Cst|d}|dur'g}|D]}||vrtd|dq||q|}n|}z(t|ddD]}t||ddrE||j|vrE|}nq0|dkrO|}W|SW|Styn}ztjd |d d |}WYd}~|Sd}~ww) zGet the final stage id for e2e. Args: stage_list: List of stage configurations Returns: Final stage id for e2e rsNzInvalid output modality: z , ignoring it final_outputFrz`[Orchestrator] Failed to determine final stage for E2E; falling back to last: %sT)exc_info) rCr"rBr?rangerzfinal_output_typer r1) rrr last_stage_idprompt_modalitiesmodality_sidfinal_stage_id_for_e2er'r(r(r)get_final_stage_id_for_e2eEsB    rclscst|s t|dt|tstddtdtdtffdd dd t|D}i}|D]\}}||vr9q0||}||j||<q0|S) zFilter kwargs to only include fields defined in the dataclass. Args: cls: Dataclass type kwargs: Keyword arguments to filter Returns: Filtered keyword arguments containing only valid dataclass fields z is not a dataclasszkwargs must be a dictionaryr5 annotationrcs^|dur|St|}|dur#t|tr!t|r!t|tr!t||S|S|tttfvrNt |}|r4|dndt|tttfrLt|fdd|DS|S|turut |}t |dkr`|dndt|trsfdd| DS|S|t j us|tt ddurt |D]$}t|trt|rt|trt||S||}||ur|Sq|S|S) zJRecursively filter nested dict/list values based on dataclass annotations.Nrc3s|]}|VqdSrJr()rKrG) _filter_valueinnerr(r)rMszAfilter_dataclass_kwargs.._filter_value..rscsi|] \}}||qSr(r()rKrFrG)rval_typer(r) szBfilter_dataclass_kwargs.._filter_value.. UnionType)r r0rVrrfilter_dataclass_kwargsrTrUrSrrCrr8rrz)r5roriginargsargfilteredr)rrr)rs:    z.filter_dataclass_kwargs.._filter_valuecSsi|] }|jr|j|qSr()initname)rKfr(r(r)rsz+filter_dataclass_kwargs..)rrRr0rrrrrV)rr valid_fieldsfiltered_kwargsrFrGfieldr(rr)rss  %rsourcecstdo ttd}|rdd}ntdd}|dur#d|ind}gd}|r5fdd|D}n fdd|D}|rI|pCi}|||S)aBuild base engine args with tokenizer and parallel configuration. Automatically detects whether source is a dict-like object or namespace object. Args: source: Source object (args namespace or kwargs dict) containing configuration. Returns: Dictionary containing tokenizer and parallel configuration overrides, or None if no configuration is present. r tokenizerN)tensor_parallel_sizepipeline_parallel_sizedata_parallel_sizedata_parallel_size_localdata_parallel_backenddistributed_executor_backendcs*i|]}|vr|dur||qSrJr(rKrFrr(r)rs*z*build_base_engine_args..cs0i|]}t|rt|dur|t|qSrJ)rrzrrr(r)rs )rr7rzrupdate)r is_dict_likerrh parallel_keysparallel_overridesr(rr)build_base_engine_argss     rr^c CsZzt|ddd }|WdWS1swYWdStttfy,YdSw)Nzutf-8replace)encodingerrors)openreadFileNotFoundErrorPermissionErrorOSError)r^rr(r(r) _read_texts(rcs6tjdrdStdp dd}tfdd|DS)Nz /.dockerenvTz/proc/1/cgroup)docker containerdkubepodslibpodpodmanc3s|]}|vVqdSrJr()rKmcgr(r)rMszin_container..)r]r^r_rany)markersr(rr) in_containers  rcCsjtd}|dur|}|dkrdSdSdD]\}}td|d}|dur2||kr/dSdSqdS) z Returns: True -> very likely running with --pid=host (host PID namespace) False -> very likely isolated PID namespace (default) None -> cannot determine z /proc/2/commNkthreaddTF))rcu_gp) rcu_par_gp) z ksoftirqd/0z/proc/z/comm)rstrip)comm2pidrcommr(r(r) has_pid_hosts  rcCst}|sdStSr6)rr)icr(r(r)detect_pid_hostsrrJ)5r]r8 collectionsr dataclassesrrrpathlibrtypingrrr omegaconfr vllm.loggerr vllm.transformers_utils.configr r "vllm.transformers_utils.repo_utilsr!vllm_omni.entrypoints.stage_utilsrvllm_omni.platformsr__file__parentr\rPr"rr@r*r3rHrArgrTrlrjrUrr{rrrrr>rrrr(r(r(r)s`       &!.6>$  "  .@4