Output:
Exit code: 1
Wall time: 54.5 seconds
Total output lines: 4268
Output:
/home/ubuntu/.local/lib/python3.10/site-packages/torch/cuda/__init__.py:63: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you.
  import pynvml  # type: ignore[import]
2026-01-14 23:16:35.211382: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
/home/ubuntu/.local/lib/python3.10/site-packages/google/api_core/_python_version_support.py:266: FutureWarning: You are using a Python version (3.10.12) which Google will stop supporting in new releases of google.api_core once it reaches its end of life (2026-10-04). Please upgrade to the latest Python version, or at least Python 3.11, to continue receiving updates for google.api_core past that date.
  warnings.warn(message, FutureWarning)
INFO 01-14 23:16:42 [utils.py:253] non-default args: {'trust_remote_code': True, 'max_model_len': 4096, 'gpu_memory_utilization': 0.15, 'disable_log_stats': True, 'model': 'mistralai/Voxtral-Mini-3B-2507'}
The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
INFO 01-14 23:16:43 [model.py:514] Resolved architecture: VoxtralForConditionalGeneration
INFO 01-14 23:16:43 [model.py:1661] Using max model len 4096
INFO 01-14 23:16:43 [scheduler.py:230] Chunked prefill is enabled with max_num_batched_tokens=8192.
[2026-01-14 23:16:44] INFO tekken.py:195: Non special vocabulary size is 130072 with 1000 special tokens.
[2026-01-14 23:16:44] INFO tekken.py:567: Cutting non special vocabulary to first 130072 tokens.
WARNING 01-14 23:16:46 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: CUDA is initialized
/home/ubuntu/.local/lib/python3.10/site-packages/torch/cuda/__init__.py:63: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you.
  import pynvml  # type: ignore[import]
2026-01-14 23:16:51.040318: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
/home/ubuntu/.local/lib/python3.10/site-packages/google/api_core/_python_version_support.py:266: FutureWarning: You are using a Python version (3.10.12) which Google will stop supporting in new releases of google.api_core once it reaches its end of life (2026-10-04). Please upgrade to the latest Python version, or at least Python 3.11, to continue receiving updates for google.api_core past that date.
  warnings.warn(message, FutureWarning)
(EngineCore_DP0 pid=1836041) INFO 01-14 23:16:54 [core.py:93] Initializing a V1 LLM engine (v0.13.0) with config: model='mistralai/Voxtral-Mini-3B-2507', speculative_config=None, tokenizer='mistralai/Voxtral-Mini-3B-2507', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, tokenizer_revision=None, trust_remote_code=True, dtype=torch.bfloat16, max_seq_len=4096, download_dir=None, load_format=auto, tensor_parallel_size=1, pipeline_parallel_size=1, data_parallel_size=1, disable_custom_all_reduce=False, quantization=None, enforce_eager=False, kv_cache_dtype=auto, device_config=cuda, structured_outputs_config=StructuredOutputsConfig(backend='auto', disable_fallback=False, disable_any_whitespace=False, disable_additional_properties=False, reasoning_parser='', reasoning_parser_plugin='', enable_in_reasoning=False), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None, kv_cache_metrics=False, kv_cache_metrics_sample=0.01, cudagraph_metrics=False, enable_layerwise_nvtx_tracing=False), seed=0, served_model_name=mistralai/Voxtral-Mini-3B-2507, enable_prefix_caching=True, enable_chunked_prefill=True, pooler_config=None, compilation_config={'level': None, 'mode': <CompilationMode.VLLM_COMPILE: 3>, 'debug_dump_path': None, 'cache_dir': '', 'compile_cache_save_format': 'binary', 'backend': 'inductor', 'custom_ops': ['none'], 'splitting_ops': ['vllm::unified_attention', 'vllm::unified_attention_with_output', 'vllm::unified_mla_attention', 'vllm::unified_mla_attention_with_output', 'vllm::mamba_mixer2', 'vllm::mamba_mixer', 'vllm::short_conv', 'vllm::linear_attention', 'vllm::plamo2_mamba_mixer', 'vllm::gdn_attention_core', 'vllm::kda_attention', 'vllm::sparse_attn_indexer'], 'compile_mm_encoder': False, 'compile_sizes': [], 'compile_ranges_split_points': [8192], 'inductor_compile_config': {'enable_auto_functionalized_v2': False, 'combo_kernels': True, 'benchmark_combo_kernel': True}, 'inductor_passes': {}, 'cudagraph_mode': <CUDAGraphMode.FULL_AND_PIECEWISE: (2, 1)>, 'cudagraph_num_of_warmups': 1, 'cudagraph_capture_sizes': [1, 2, 4, 8, 16, 24, 32, 40, 48, 56, 64, 72, 80, 88, 96, 104, 112, 120, 128, 136, 144, 152, 160, 168, 176, 184, 192, 200, 208, 216, 224, 232, 240, 248, 256, 272, 288, 304, 320, 336, 352, 368, 384, 400, 416, 432, 448, 464, 480, 496, 512], 'cudagraph_copy_inputs': False, 'cudagraph_specialize_lora': True, 'use_inductor_graph_partition': False, 'pass_config': {'fuse_norm_quant': False, 'fuse_act_quant': False, 'fuse_attn_quant': False, 'eliminate_noops': True, 'enable_sp': False, 'fuse_gemm_comms': False, 'fuse_allreduce_rms': False}, 'max_cudagraph_capture_size': 512, 'dynamic_shapes_config': {'type': <DynamicShapesType.BACKED: 'backed'>, 'evaluate_guards': False}, 'local_cache_dir': None}
(EngineCore_DP0 pid=1836041) [2026-01-14 23:16:56] INFO tekken.py:195: Non special vocabulary size is 130072 with 1000 special tokens.
(EngineCore_DP0 pid=1836041) [2026-01-14 23:16:56] INFO tekken.py:567: Cutting non special vocabulary to first 130072 tokens.
(EngineCore_DP0 pid=1836041) INFO 01-14 23:16:57 [parallel_state.py:1203] world_size=1 rank=0 local_rank=0 distributed_init_method=tcp://216.81.248.184:56965 backend=nccl
(EngineCore_DP0 pid=1836041) INFO 01-14 23:16:57 [parallel_state.py:1411] rank 0 in world size 1 is assigned as DP rank 0, PP rank 0, PCP rank 0, TP rank 0, EP rank 0
(EngineCore_DP0 pid=1836041) INFO 01-14 23:16:58 [gpu_model_runner.py:3562] Starting to load model mistralai/Voxtral-Mini-3B-2507...
(EngineCore_DP0 pid=1836041) INFO 01-14 23:16:59 [cuda.py:351] Using FLASH_ATTN attention backend out of potential backends: ('FLASH_ATTN', 'FLASHINFER', 'TRITON_ATTN', 'FLEX_ATTENTION')
(EngineCore_DP0 pid=1836041) INFO 01-14 23:16:59 [layer.py:477] Using AttentionBackendEnum.FLASH_ATTN for MultiHeadAttention in multimodal encoder.
(EngineCore_DP0 pid=1836041) INFO 01-14 23:17:13 [weight_utils.py:487] Time spent downloading weights for mistralai/Voxtral-Mini-3B-2507: 13.874908 seconds
(EngineCore_DP0 pid=1836041) INFO 01-14 23:17:13 [weight_utils.py:527] No consolidated.safetensors.index.json found in remote.
(EngineCore_DP0 pid=1836041)
Loading safetensors checkpoint shards:   0% Completed | 0/1 [00:00<?, ?it/s]
(EngineCore_DP0 pid=1836041)
Loading safetensors checkpoint shards: 100% Completed | 1/1 [00:01<00:00,  1.94s/it]
(EngineCore_DP0 pid=1836041)
Loading safetensors checkpoint shards: 100% Completed | 1/1 [00:01<00:00,  1.94s/it]
(EngineCore_DP0 pid=1836041)
(EngineCore_DP0 pid=1836041) INFO 01-14 23:17:15 [default_loader.py:308] Loading weights took 2.24 seconds
(EngineCore_DP0 pid=1836041) INFO 01-14 23:17:16 [gpu_model_runner.py:3659] Model loading took 8.7223 GiB memory and 16.770581 seconds
(EngineCore_DP0 pid=1836041) INFO 01-14 23:17:16 [gpu_model_runner.py:4446] Encoder cache will be initialized with a budget of 8192 tokens, and profiled with 2 audio items of the maximum feature size.
(EngineCore_DP0 pid=1836041) ERROR 01-14 23:17:17 [core.py:866] EngineCore failed to start.
(EngineCore_DP0 pid=1836041) ERROR 01-14 23:17:17 [core.py:866] Traceback (most recent call last):
(EngineCore_DP0 pid=1836041) ERROR 01-14 23:17:17 [core.py:866]   File "/home/ubuntu/.local/lib/python3.10/site-packages/vllm/multimodal/processing.py", line 1115, in call_hf_processor
(EngineCore_DP0 pid=1836041) ERROR 01-14 23:17:17 [core.py:866]     output = hf_processor(**data, **allowed_kwargs, return_tensors="pt")
(EngineCore_DP0 pid=1836041) ERROR 01-14 23:17:17 [core.py:866]   File "/home/ubuntu/.local/lib/python3.10/site-packages/vllm/model_executor/models/voxtral.py", line 161, in __call__
(EngineCore_DP0 pid=1836041) ERROR 01-14 23:17:17 [core.py:866]     audio = self._audio_processor.pad(audio, self.sampling_rate)
(EngineCore_DP0 pid=1836041) ERROR 01-14 23:17:17 [core.py:866] TypeError: AudioEncoder.pad() missing 1 required positional argument: 'is_online_streaming'
(EngineCore_DP0 pid=1836041) ERROR 01-14 23:17:17 [core.py:866]
(EngineCore_DP0 pid=1836041) ERROR 01-14 23:17:17 [core.py:866] The above exception was the direct cause of the following exception:
(EngineCore_DP0 pid=1836041) ERROR 01-14 23:17:17 [core.py:866]
(EngineCore_DP0 pid=1836041) ERROR 01-14 23:17:17 [core.py:866] Traceback (most recent call last):
(EngineCore_DP0 pid=1836041) ERROR 01-14 23:17:17 [core.py:866]   File "/home/ubuntu/.local/lib/python3.10/site-packages/vllm/v1/engine/core.py", line 857, in run_engine_core
(EngineCore_DP0 pid=1836041) ERROR 01-14 23:17:17 [core.py:866]     engine_core = EngineCoreProc(*args, **kwargs)
(EngineCore_DP0 pid=1836041) ERROR 01-14 23:17:17 [core.py:866]   File "/home/ubuntu/.local/lib/python3.10/site-packages/vllm/v1/engine/core.py", line 637, in __init__
(EngineCore_DP0 pid=1836041) ERROR 01-14 23:17:17 [core.py:866]     super().__init__(
(EngineCore_DP0 pid=1836041) ERROR 01-14 23:17:17 [core.py:866]   File "/home/ubuntu/.local/lib/python3.10/site-packages/vllm/v1/engine/core.py", line 109, in __init__
(EngineCore_DP0 pid=1836041) ERROR 01-14 23:17:17 [core.py:866]     num_gpu_blocks, num_cpu_blocks, kv_cache_config = self._initialize_kv_caches(
(EngineCore_DP0 pid=1836041) ERROR 01-14 23:17:17 [core.py:866]   File "/home/ubuntu/.local/lib/python3.10/site-packages/vllm/v1/engine/core.py", line 240, in _initialize_kv_caches
(EngineCore_DP0 pid=1836041) ERROR 01-14 23:17:17 [core.py:866]     available_gpu_memory = self.model_executor.determine_available_memory()
(EngineCore_DP0 pid=1836041) ERROR 01-14 23:17:17 [core.py:866]   File "/home/ubuntu/.local/lib/python3.10/site-packages/vllm/v1/executor/abstract.py", line 126, in determine_available_memory
(EngineCore_DP0 pid=1836041) ERROR 01-14 23:17:17 [core.py:866]     return self.collective_rpc("determine_available_memory")
(EngineCore_DP0 pid=1836041) ERROR 01-14 23:17:17 [core.py:866]   File "/home/ubuntu/.local/lib/python3.10/site-packages/vllm/v1/executor/uniproc_executor.py", line 75, in collective_rpc
(EngineCore_DP0 pid=1836041) ERROR 01-14 23:17:17 [core.py:866]     result = run_method(self.driver_worker, method, args, kwargs)
(EngineCore_DP0 pid=1836041) ERROR 01-14 23:17:17 [core.py:866]   File "/home/ubuntu/.local/lib/python3.10/site-packages/vllm/v1/serial_utils.py", line 461, in run_method
(EngineCore_DP0 pid=1836041) ERROR 01-14 23:17:17 [core.py:866]     return func(*args, **kwargs)
(EngineCore_DP0 pid=1836041) ERROR 01-14 23:17:17 [core.py:866]   File "/home/ubuntu/.local/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 120, in decorate_context
(EngineCore_DP0 pid=1836041) ERROR 01-14 23:17:17 [core.py:866]     return func(*args, **kwargs)
(EngineCore_DP0 pid=1836041) ERROR 01-14 23:17:17 [core.py:866]   File "/home/ubuntu/.local/lib/python3.10/site-packages/vllm/v1/worker/gpu_worker.py", line 340, in determine_available_memory
(EngineCore_DP0 pid=1836041) ERROR 01-14 23:17:17 [core.py:866]     self.model_runner.profile_run()
(EngineCore_DP0 pid=1836041) ERROR 01-14 23:17:17 [core.py:866]   File "/home/ubuntu/.local/lib/python3.10/site-packages/vllm/v1/worker/gpu_model_runner.py", line 4456, in profile_run
(EngineCore_DP0 pid=1836041) ERROR 01-14 23:17:17 [core.py:866]     batched_dummy_mm_inputs = self._get_mm_dummy_batch(
(EngineCore_DP0 pid=1836041) ERROR 01-14 23:17:17 [core.py:866]   File "/home/ubuntu/.local/lib/python3.10/site-packages/vllm/v1/worker/gpu_model_runner.py", line 3948, in _get_mm_dummy_batch
(EngineCore_DP0 pid=1836041) ERROR 01-14 23:17:17 [core.py:866]     dummy_decoder_data = self.mm_registry.get_decoder_dummy_data(
(EngineCore_DP0 pid=1836041) ERROR 01-14 23:17:17 [core.py:866]   File "/home/ubuntu/.local/lib/python3.10/site-packages/vllm/multimodal/registry.py", line 293, in get_decoder_dummy_data
(EngineCore_DP0 pid=1836041) ERROR 01-14 23:17:17 [core.py:866]     dummy_data = profiler.get_decoder_dummy_data(seq_len, mm_counts, mm_options)
(EngineCore_DP0 pid=1836041) ERROR 01-14 23:17:17 [core.py:866]   File "/home/ubuntu/.local/lib/python3.10/site-packages/vllm/multimodal/profiling.py", line 313, in get_decoder_dummy_data
(EngineCore_DP0 pid=1836041) ERROR 01-14 23:17:17 [core.py:866]     mm_inputs = self._get_dummy_mm_inputs(seq_len, mm_counts, mm_options)
(EngineCore_DP0 pid=1836041) ERROR 01-14 23:17:17 [core.py:866]   File "/home/ubuntu/.local/lib/python3.10/site-packages/vllm/multimodal/profiling.py", line 267, in _get_dummy_mm_inputs
(EngineCore_DP0 pid=1836041) ERROR 01-14 23:17:17 [core.py:866]     return self.processor.apply(
(EngineCore_DP0 pid=1836041) ERROR 01-14 23:17:17 [core.py:866]   File "/home/ubuntu/.local/lib/python3.10/site-packages/vllm/multimodal/processing.py", line 2133, in apply
(EngineCore_DP0 pid=1836041) ERROR 01-14 23:17:17 [core.py:866]     ) = self._cached_apply_hf_processor(
(EngineCore_DP0 pid=1836041) ERROR 01-14 23:17:17 [core.py:866]   File "/home/ubuntu/.local/lib/python3.10/site-packages/vllm/model_executor/models/voxtral.py", line 310, in _cached_apply_hf_processor
(EngineCore_DP0 pid=1836041) ERROR 01-14 23:17:17 [core.py:866]     prompt_ids, mm_info, _ = super()._cached_apply_hf_processor(
(EngineCore_DP0 pid=1836041) ERROR 01-14 23:17:17 [core.py:866]   File "/home/ubuntu/.local/lib/python3.10/site-packages/vllm/multimodal/processing.py", line 1915, in _cached_apply_hf_processor
(EngineCore_DP0 pid=1836041) ERROR 01-14 23:17:17 [core.py:866]     ) = self._apply_hf_processor_main(
(EngineCore_DP0 pid=1836041) ERROR 01-14 23:17:17 [core.py:866]   File "/home/ubuntu/.local/lib/python3.10/site-packages/vllm/multimodal/processing.py", line 1652, in _apply_hf_processor_main
(EngineCore_DP0 pid=1836041) ERROR 01-14 23:17:17 [core.py:866]     mm_processed_data = self._apply_hf_processor_mm_only(
(EngineCore_DP0 pid=1836041) ERROR 01-14 23:17:17 [core.py:866]   File "/home/ubuntu/.local/lib/python3.10/site-packages/vllm/multimodal/processing.py", line 1610, in _apply_hf_processor_mm_only
(EngineCore_DP0 pid=1836041) ERROR 01-14 23:17:17 [core.py:866]     _, mm_processed_data, _ = self._apply_hf_processor_text_mm(
(EngineCore_DP0 pid=1836041) ERROR 01-14 23:17:17 [core.py:866]   File "/home/ubuntu/.local/lib/python3.10/site-packages/vllm/multimodal/processing.py", line 1537, in _apply_hf_processor_text_mm
(EngineCore_DP0 pid=1836041) ERROR 01-14 23:17:17 [core.py:866]     processed_data = self._call_hf_processor(
(EngineCore_DP0 pid=1836041) ERROR 01-14 23:17:17 [core.py:866]   File "/home/ubuntu/.local/lib/python3.10/site-packages/vllm/multimodal/processing.py", line 1497, in _call_hf_processor
(EngineCore_DP0 pid=1836041) ERROR 01-14 23:17:17 [core.py:866]     return self.info.ctx.call_hf_processor(
(EngineCore_DP0 pid=1836041) ERROR 01-14 23:17:17 [core.py:866]   File "/home/ubuntu/.local/lib/python3.10/site-packages/vllm/multimodal/processing.py", line 1144, in call_hf_processor
(EngineCore_DP0 pid=1836041) ERROR 01-14 23:17:17 [core.py:866]     raise ValueError(msg) from exc
(EngineCore_DP0 pid=1836041) ERROR 01-14 23:17:17 [core.py:866] ValueError: Failed to apply VoxtralProcessorAdapter on data={'text': '', 'audios': [array([0., 0., 0., ..., 0., 0., 0.], dtype=float32)]} with kwargs={}
(EngineCore_DP0 pid=1836041) Process EngineCore_DP0:
(EngineCore_DP0 pid=1836041) Traceback (most recent call last):
(EngineCore_DP0 pid=1836041)   File "/home/ubuntu/.local/lib/python3.10/site-packages/vllm/multimodal/processing.py", line 1115, in call_hf_processor
(EngineCore_DP0 pid=1836041)     output = hf_processor(**data, **allowed_kwargs, return_tensors="pt")
(EngineCore_DP0 pid=1836041)   File "/home/ubuntu/.local/lib/python3.10/site-packages/vllm/model_executor/models/voxtral.py", line 161, in __call__
(EngineCore_DP0 pid=1836041)     audio = self._audio_processor.pad(audio, self.sampling_rate)
(EngineCore_DP0 pid=1836041) TypeError: AudioEncoder.pad() missing 1 required positional argument: 'is_online_streaming'
(EngineCore_DP0 pid=1836041)
(EngineCore_DP0 pid=1836041) The above exception was the direct cause of the following exception:
(EngineCore_DP0 pid=1836041)
(EngineCore_DP0 pid=1836041) Traceback (most recent call last):
(EngineCore_DP0 pid=1836041)   File "/usr/lib/python3.10/multiprocessing/process.py", line 314, in _bootstrap
(EngineCore_DP0 pid=1836041)     self.run()
(EngineCore_DP0 pid=1836041)   File "/usr/lib/python3.10/multiprocessing/process.py", line 108, in run
(EngineCore_DP0 pid=1836041)     self._target(*self._args, **self._kwargs)
(EngineCore_DP0 pid=1836041)   File "/home/ubuntu/.local/lib/python3.10/site-packages/vllm/v1/engine/core.py", line 870, in run_engine_core
(EngineCore_DP0 pid=1836041)     raise e
(EngineCore_DP0 pid=1836041)   File "/home/ubuntu/.local/lib/python3.10/site-packages/vllm/v1/engine/core.py", line 857, in run_engine_core
(EngineCore_DP0 pid=1836041)     engine_core = EngineCoreProc(*args, **kwargs)
(EngineCore_DP0 pid=1836041)   File "/home/ubuntu/.local/lib/python3.10/site-packages/vllm/v1/engine/core.py", line 637, in __init__
(EngineCore_DP0 pid=1836041)     super().__init__(
(EngineCore_DP0 pid=1836041)   File "/home/ubuntu/.local/lib/python3.10/site-packages/vllm/v1/engine/core.py", line 109, in __init__
(EngineCore_DP0 pid=1836041)     num_gpu_blocks, num_cpu_blocks, kv_cache_config = self._initialize_kv_caches(
(EngineCore_DP0 pid=1836041)   File "/home/ubuntu/.local/lib/python3.10/site-packages/vllm/v1/engine/core.py", line 240, in _initialize_kv_caches
(Engin…90919 tokens truncated…      │ │
│ │              eng_identity = b'\x00\x00'                                  │ │
│ │                 eng_index = 0                                            │ │
│ │                    engine = <vllm.v1.engine.utils.CoreEngine object at   │ │
│ │                             0x7f5ca21514b0>                              │ │
│ │                    events = [(64, 4)]                                    │ │
│ │                  finished = {}                                           │ │
│ │          handshake_socket = <zmq.Socket(zmq.ROUTER) at 0x7f5ca213ce40    │ │
│ │                             closed>                                      │ │
│ │                  headless = False                                        │ │
│ │              init_message = b'\x83\xa9addresses\x85\xa6inputs\x91\xd9/i… │ │
│ │                     local = True                                         │ │
│ │               local_count = 1                                            │ │
│ │                       msg = {                                            │ │
│ │                             │   'status': 'HELLO',                       │ │
│ │                             │   'local': True,                           │ │
│ │                             │   'headless': False                        │ │
│ │                             }                                            │ │
│ │           parallel_config = ParallelConfig(                              │ │
│ │                             │   pipeline_parallel_size=1,                │ │
│ │                             │   tensor_parallel_size=1,                  │ │
│ │                             │   prefill_context_parallel_size=1,         │ │
│ │                             │   data_parallel_size=1,                    │ │
│ │                             │   data_parallel_size_local=1,              │ │
│ │                             │   data_parallel_rank=0,                    │ │
│ │                             │   data_parallel_rank_local=0,              │ │
│ │                             │   data_parallel_master_ip='127.0.0.1',     │ │
│ │                             │   data_parallel_rpc_port=29550,            │ │
│ │                             │   data_parallel_master_port=0,             │ │
│ │                             │   data_parallel_backend='mp',              │ │
│ │                             │   data_parallel_external_lb=False,         │ │
│ │                             │   data_parallel_hybrid_lb=False,           │ │
│ │                             │   enable_expert_parallel=False,            │ │
│ │                             │   enable_eplb=False,                       │ │
│ │                             │   eplb_config=EPLBConfig(                  │ │
│ │                             │   │   window_size=1000,                    │ │
│ │                             │   │   step_interval=3000,                  │ │
│ │                             │   │   num_redundant_experts=0,             │ │
│ │                             │   │   log_balancedness=False,              │ │
│ │                             │   │   use_async=False,                     │ │
│ │                             │   │   policy='default'                     │ │
│ │                             │   ),                                       │ │
│ │                             │   expert_placement_strategy='linear',      │ │
│ │                             │                                            │ │
│ │                             all2all_backend='allgather_reducescatter',   │ │
│ │                             │   max_parallel_loading_workers=None,       │ │
│ │                             │   disable_custom_all_reduce=False,         │ │
│ │                             │   enable_dbo=False,                        │ │
│ │                             │   dbo_decode_token_threshold=32,           │ │
│ │                             │   dbo_prefill_token_threshold=512,         │ │
│ │                             │                                            │ │
│ │                             disable_nccl_for_dp_synchronization=False,   │ │
│ │                             │   ray_workers_use_nsight=False,            │ │
│ │                             │   ray_runtime_env=None,                    │ │
│ │                             │   placement_group=None,                    │ │
│ │                             │   distributed_executor_backend='uni',      │ │
│ │                             │                                            │ │
│ │                             worker_cls='vllm.v1.worker.gpu_worker.Worke… │ │
│ │                             │   sd_worker_cls='auto',                    │ │
│ │                             │   worker_extension_cls='',                 │ │
│ │                             │   master_addr='127.0.0.1',                 │ │
│ │                             │   master_port=29501,                       │ │
│ │                             │   node_rank=0,                             │ │
│ │                             │   nnodes=1,                                │ │
│ │                             │   world_size=1,                            │ │
│ │                             │   rank=0,                                  │ │
│ │                             │   _data_parallel_master_port_list=[],      │ │
│ │                             │   decode_context_parallel_size=1,          │ │
│ │                             │   dcp_kv_cache_interleave_size=1,          │ │
│ │                             │   cp_kv_cache_interleave_size=1,           │ │
│ │                             │   _api_process_count=1,                    │ │
│ │                             │   _api_process_rank=0                      │ │
│ │                             )                                            │ │
│ │                    poller = <zmq.sugar.poll.Poller object at             │ │
│ │                             0x7f5ca2151f30>                              │ │
│ │              proc_manager = <vllm.v1.engine.utils.CoreEngineProcManager  │ │
│ │                             object at 0x7f5ca2152050>                    │ │
│ │           ready_msg_bytes = b'\x83\xa6status\xa5HELLO\xa5local\xc3\xa8h… │ │
│ │              remote_count = 0                                            │ │
│ │ remote_should_be_headless = True                                         │ │
│ │                  sentinel = 64                                           │ │
│ │             start_pending = [1, 0]                                       │ │
│ │                    status = 'HELLO'                                      │ │
│ ╰──────────────────────────────────────────────────────────────────────────╯ │
╰──────────────────────────────────────────────────────────────────────────────╯
RuntimeError: Engine core initialization failed. See root cause above. Failed
core proc(s): {}

The above exception was the direct cause of the following exception:

╭───────────────────── Traceback (most recent call last) ──────────────────────╮
│ /home/ubuntu/maya-transcribe/maya_transcribe/cli.py:463 in bench_run         │
│                                                                              │
│   460 │   │   thresholds["require_ctc"] = require_ctc                        │
│   461 │   │   thresholds["min_ctc_margin"] = float(min_ctc_margin)           │
│   462 │                                                                      │
│ ❱ 463 │   results_path = run_pipeline(                                       │
│   464 │   │   segments_dir=audio_dir,                                        │
│   465 │   │   out_dir=out,                                                   │
│   466 │   │   lid_model_id=lid_model_id,                                     │
│                                                                              │
│ ╭───────────────────────────────── locals ─────────────────────────────────╮ │
│ │                      audio_dir = PosixPath('data/bench/maya_hf_hin_v2/a… │ │
│ │                      bench_dir = PosixPath('data/bench/maya_hf_hin_v2')  │ │
│ │                        ctc_map = None                                    │ │
│ │                      ctc_model = None                                    │ │
│ │                  ctc_model_map = None                                    │ │
│ │                     hypothesis = 'final'                                 │ │
│ │                         ic_map = None                                    │ │
│ │   indicconformer_default_model = None                                    │ │
│ │       indicconformer_model_map = None                                    │ │
│ │                   lid_model_id = 'facebook/mms-lid-256'                  │ │
│ │                          limit = None                                    │ │
│ │                 min_ctc_margin = 0.05                                    │ │
│ │                min_silence_sec = 0.15                                    │ │
│ │                  mms_asr_model = None                                    │ │
│ │                            out = PosixPath('outputs/bench/maya_hf_hin_v… │ │
│ │                      refs_path = PosixPath('data/bench/maya_hf_hin_v2/r… │ │
│ │                    require_ctc = True                                    │ │
│ │            require_lid_backend = True                                    │ │
│ │           silence_threshold_db = -40.0                                   │ │
│ │                   skip_whisper = True                                    │ │
│ │                         tf_map = None                                    │ │
│ │                     thresholds = {                                       │ │
│ │                                  │   'route_lang_from_path': True,       │ │
│ │                                  │   'route_lang_prefer_path': True      │ │
│ │                                  }                                       │ │
│ │                thresholds_json = None                                    │ │
│ │     transformers_whisper_model = []                                      │ │
│ │ transformers_whisper_model_map = None                                    │ │
│ │                   trim_silence = False                                   │ │
│ │                 vllm_asr_model = 'mistralai/Voxtral-Mini-3B-2507'        │ │
│ │    vllm_gpu_memory_utilization = 0.15                                    │ │
│ │             vllm_max_model_len = 4096                                    │ │
│ │                  whisper_model = 'large-v3'                              │ │
│ ╰──────────────────────────────────────────────────────────────────────────╯ │
│                                                                              │
│ /home/ubuntu/maya-transcribe/maya_transcribe/pipeline.py:185 in run_pipeline │
│                                                                              │
│   182 │   │   │   │   max_model_len=vllm_max_model_len,                      │
│   183 │   │   │   )                                                          │
│   184 │   │   except Exception as e:                                         │
│ ❱ 185 │   │   │   raise RuntimeError("Failed to initialize vLLM ASR backend. │
│   186 │                                                                      │
│   187 │   default_whisper_run_sampling = bool(whisper_backend is not None) a │
│   188 │   │   transformers_backends or transformers_router or gemini_backend │
│                                                                              │
│ ╭───────────────────────────────── locals ─────────────────────────────────╮ │
│ │                       audio_files = [                                    │ │
│ │                                     │                                    │ │
│ │                                     PosixPath('data/bench/maya_hf_hin_v… │ │
│ │                                     │                                    │ │
│ │                                     PosixPath('data/bench/maya_hf_hin_v… │ │
│ │                                     │                                    │ │
│ │                                     PosixPath('data/bench/maya_hf_hin_v… │ │
│ │                                     │                                    │ │
│ │                                     PosixPath('data/bench/maya_hf_hin_v… │ │
│ │                                     │                                    │ │
│ │                                     PosixPath('data/bench/maya_hf_hin_v… │ │
│ │                                     │                                    │ │
│ │                                     PosixPath('data/bench/maya_hf_hin_v… │ │
│ │                                     │                                    │ │
│ │                                     PosixPath('data/bench/maya_hf_hin_v… │ │
│ │                                     │                                    │ │
│ │                                     PosixPath('data/bench/maya_hf_hin_v… │ │
│ │                                     │                                    │ │
│ │                                     PosixPath('data/bench/maya_hf_hin_v… │ │
│ │                                     │                                    │ │
│ │                                     PosixPath('data/bench/maya_hf_hin_v… │ │
│ │                                     │   ... +190                         │ │
│ │                                     ]                                    │ │
│ │                         ctc_model = None                                 │ │
│ │                     ctc_model_map = None                                 │ │
│ │                        ctc_scorer = None                                 │ │
│ │ extra_transformers_whisper_models = []                                   │ │
│ │                    gemini_backend = None                                 │ │
│ │                  gemini_hard_only = True                                 │ │
│ │          gemini_max_output_tokens = 512                                  │ │
│ │                      gemini_model = None                                 │ │
│ │                gemini_prompt_path = None                                 │ │
│ │      indicconformer_default_model = None                                 │ │
│ │          indicconformer_model_map = None                                 │ │
│ │             indicconformer_router = None                                 │ │
│ │                       lid_backend = <maya_transcribe.lid.MMSLID object   │ │
│ │                                     at 0x7f5ec1adc3a0>                   │ │
│ │                      lid_model_id = 'facebook/mms-lid-256'               │ │
│ │                             limit = None                                 │ │
│ │                   min_silence_sec = 0.15                                 │ │
│ │                     mms_asr_model = None                                 │ │
│ │                       mms_backend = None                                 │ │
│ │                           out_dir = PosixPath('outputs/bench/maya_hf_hi… │ │
│ │                       rank_by_ctc = None                                 │ │
│ │               require_lid_backend = True                                 │ │
│ │                      segments_dir = PosixPath('data/bench/maya_hf_hin_v… │ │
│ │              silence_threshold_db = -40.0                                │ │
│ │                      skip_whisper = True                                 │ │
│ │                        thresholds = {                                    │ │
│ │                                     │   'route_lang_from_path': True,    │ │
│ │                                     │   'route_lang_prefer_path': True   │ │
│ │                                     }                                    │ │
│ │             transformers_backends = []                                   │ │
│ │               transformers_router = None                                 │ │
│ │  transformers_whisper_default_model None                                 │ │
│ │                                   =                                      │ │
│ │    transformers_whisper_model_map = None                                 │ │
│ │                      trim_silence = False                                │ │
│ │                    vllm_asr_model = 'mistralai/Voxtral-Mini-3B-2507'     │ │
│ │                      vllm_backend = None                                 │ │
│ │       vllm_gpu_memory_utilization = 0.15                                 │ │
│ │                vllm_max_model_len = 4096                                 │ │
│ │                   whisper_backend = None                                 │ │
│ │                     whisper_model = 'large-v3'                           │ │
│ ╰──────────────────────────────────────────────────────────────────────────╯ │
╰──────────────────────────────────────────────────────────────────────────────╯
RuntimeError: Failed to initialize vLLM ASR backend.


Summary: auto

## Skills
A skill is a set of local instructions to follow that is stored in a `SKILL.md` file. Below is the list of skills that can be used. Each entry includes a name, description, and file path so you can open the source for full instructions when using a specific skill.
### Available skills
- skill-creator: Guide for creating effective skills. This skill should be used when users want to create a new skill (or update an existing skill) that extends Codex's capabilities with specialized knowledge, workflows, or tool integrations. (file: /home/ubuntu/.codex/skills/.system/skill-creator/SKILL.md)
- skill-installer: Install Codex skills into $CODEX_HOME/skills from a curated list or a GitHub repo path. Use when a user asks to list installable skills, install a curated skill, or install a skill from another repo (including private repos). (file: /home/ubuntu/.codex/skills/.system/skill-installer/SKILL.md)
### How to use skills
- Discovery: The list above is the skills available in this session (name + description + file path). Skill bodies live on disk at the listed paths.
- Trigger rules: If the user names a skill (with `$SkillName` or plain text) OR the task clearly matches a skill's description shown above, you must use that skill for that turn. Multiple mentions mean use them all. Do not carry skills across turns unless re-mentioned.
- Missing/blocked: If a named skill isn't in the list or the path can't be read, say so briefly and continue with the best fallback.
- How to use a skill (progressive disclosure):
  1) After deciding to use a skill, open its `SKILL.md`. Read only enough to follow the workflow.
  2) If `SKILL.md` points to extra folders such as `references/`, load only the specific files needed for the request; don't bulk-load everything.
  3) If `scripts/` exist, prefer running or patching them instead of retyping large code blocks.
  4) If `assets/` or templates exist, reuse them instead of recreating from scratch.
- Coordination and sequencing:
  - If multiple skills apply, choose the minimal set that covers the request and state the order you'll use them.
  - Announce which skill(s) you're using and why (one short line). If you skip an obvious skill, say why.
- Context hygiene:
  - Keep context small: summarize long sections instead of pasting them; only load extra files when needed.
  - Avoid deep reference-chasing: prefer opening only files directly linked from `SKILL.md` unless you're blocked.
  - When variants exist (frameworks, providers, domains), pick only the relevant reference file(s) and note that choice.
- Safety and fallback: If a skill can't be applied cleanly (missing files, unclear instructions), state the issue, pick the next-best approach, and continue.


**Identifying version mismatch causing argument error**

The Voxtral processor's pad method now requires an `is_online_streaming` argument, indicating the current `mistral_common` or vLLM version is outdated; the next step is to verify and update these dependencies to meet the required `mistral_common >= 1.8.1`.


python3 - <<'PY'
import mistral_common
print(mistral_common.__version__)
PY


Output:
Exit code: 0
Wall time: 0.1 seconds
Output:
1.8.8


Summary: auto


**Assessing version mismatch and patch options**

The latest `mistral_common` version conflicts with vLLM's expected method signature, causing errors in Voxtral; fixing this requires aligning package versions or monkeypatching vLLM's code. A quick workaround is to try running Gemma3n first since it might avoid this dependency, but Voxtral still needs a proper fix.


**Considering patch versus version pinning**

Patching vLLM code directly in site-packages is possible but risky, so pinning `mistral_common` to a compatible older version (matching vLLM 0.13.0’s expected `pad` signature) looks safer and easier to implement next.


python3 - <<'PY'
from mistral_common.tokens.tokenizers.audio import AudioEncoder
import inspect
print(inspect.signature(AudioEncoder.pad))
PY


Output:
Exit code: 0
Wall time: 0.3 seconds
Output:
(self, audio_array: numpy.ndarray, sampling_rate: int, is_online_streaming: bool) -> numpy.ndarray


**Planning runtime patch for audio pad signature**

I'm considering adding a monkeypatch to AudioEncoder.pad to supply the missing is_online_streaming argument with a default, running this patch before vLLM imports or VoxtralProcessorAdapter calls; this can be triggered in VllmTranscriptionASR initialization when the model ID contains "Voxtral".


**Finalizing monkeypatch to add default param**

I'm preparing a wrapper for AudioEncoder.pad that adds a default for the third argument to maintain compatibility with vLLM calls using two arguments, to be applied in VllmTranscriptionASR.__init__ before LLM loads the model.


Input:
*** Begin Patch
*** Update File: maya_transcribe/asr.py
@@
         try:
             import torch
             from vllm import LLM, SamplingParams
             from vllm.model_executor.model_loader import get_model_cls
         except Exception as e:  # pragma: no cover
@@
         self.model_id = model_id
         self.dtype = dtype
         self.tensor_parallel_size = max(1, int(tensor_parallel_size))
         self.trust_remote_code = bool(trust_remote_code)

+        try:
+            from mistral_common.tokens.tokenizers.audio import AudioEncoder  # type: ignore
+            import inspect
+
+            sig = inspect.signature(AudioEncoder.pad)
+            needs_default = "is_online_streaming" in sig.parameters and sig.parameters["is_online_streaming"].default is inspect._empty
+            if needs_default:
+                orig_pad = AudioEncoder.pad
+
+                def _pad(self, audio_array, sampling_rate, is_online_streaming=False):
+                    return orig_pad(self, audio_array, sampling_rate, is_online_streaming)
+
+                AudioEncoder.pad = _pad  # type: ignore[assignment]
+        except Exception:
+            pass
+
         llm_kwargs = {
             "model": model_id,
             "tensor_parallel_size": self.tensor_parallel_size,
             "trust_remote_code": self.trust_remote_code,
         }
*** End Patch


Output: {"output":"Success. Updated the following files:\nM maya_transcribe/asr.py\n","metadata":{"exit_code":0,"duration_seconds":0.0}}


**Checking GPU processes before rerun**

I'm noting that the previous failed run might have left GPU processes active, so I'll verify with `nvidia-smi` before running again.


Output:
Exit code: 0
Wall time: 0.3 seconds
Output:
Wed Jan 14 23:18:15 2026
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 560.35.05              Driver Version: 560.35.05      CUDA Version: 12.6     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA A100-SXM4-80GB          On  |   00000000:01:00.0 Off |                    0 |
| N/A   29C    P0             57W /  400W |   61611MiB /  81920MiB |      0%      Default |
|                                         |                        |             Disabled |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI        PID   Type   Process name                              GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|    0   N/A  N/A    738802      C   /usr/bin/python3.12                         61602MiB |
+-----------------------------------------------------------------------------------------+


**Proceeding with limited GPU memory**

GPU memory remains blocked by a root process, so we'll work with the 14GB free. I'll rerun Voxtral-Mini after the monkeypatch, ensuring multiprocessing uses spawn to avoid memory issues.


python3 -m maya_transcribe.cli bench run --bench-dir data/bench/maya_hf_hin_v2 --out outputs/bench/maya_hf_hin_v2/voxtral_mini_vllm --skip-whisper --vllm-asr-model mistralai/Voxtral-Mini-3B-2507 --vllm-gpu-memory-utilization 0.15 --vllm-max-model-len 4096


Output:
Exit code: 1
Wall time: 37.6 seconds
Total output lines: 4273
Output:
/home/ubuntu/.local/lib/python3.10/site-packages/torch/cuda/__init__.py:63: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you.
  import pynvml  # type: ignore[import]
2026-01-14 23:18:26.943950: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
/home/ubuntu/.local/lib/python3.10/site-packages/google/api_core/_python_version_support.py:266: FutureWarning: You are using a Python version (3.10.12) which Google will stop supporting in new releases of google.api_core once it reaches its end of life (2026-10-04). Please upgrade to the latest Python version, or at least Python 3.11, to continue receiving updates for google.api_core past that date.
  warnings.warn(message, FutureWarning)
INFO 01-14 23:18:33 [utils.py:253] non-default args: {'trust_remote_code': True, 'max_model_len': 4096, 'gpu_memory_utilization': 0.15, 'disable_log_stats': True, 'model': 'mistralai/Voxtral-Mini-3B-2507'}
The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
INFO 01-14 23:18:34 [model.py:514] Resolved architecture: VoxtralForConditionalGeneration
INFO 01-14 23:18:34 [model.py:1661] Using max model len 4096
INFO 01-14 23:18:34 [scheduler.py:230] Chunked prefill is enabled with max_num_batched_tokens=8192.
[2026-01-14 23:18:34] INFO tekken.py:195: Non special vocabulary size is 130072 with 1000 special tokens.
[2026-01-14 23:18:34] INFO tekken.py:567: Cutting non special vocabulary to first 130072 tokens.
WARNING 01-14 23:18:35 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: CUDA is initialized
/home/ubuntu/.local/lib/python3.10/site-packages/torch/cuda/__init__.py:63: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you.
  import pynvml  # type: ignore[import]
2026-01-14 23:18:41.009882: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
/home/ubuntu/.local/lib/python3.10/site-packages/google/api_core/_python_version_support.py:266: FutureWarning: You are using a Python version (3.10.12) which Google will stop supporting in new releases of google.api_core once it reaches its end of life (2026-10-04). Please upgrade to the latest Python version, or at least Python 3.11, to continue receiving updates for google.api_core past that date.
  warnings.warn(message, FutureWarning)
(EngineCore_DP0 pid=1836852) INFO 01-14 23:18:44 [core.py:93] Initializing a V1 LLM engine (v0.13.0) with config: model='mistralai/Voxtral-Mini-3B-2507', speculative_config=None, tokenizer='mistralai/Voxtral-Mini-3B-2507', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, tokenizer_revision=None, trust_remote_code=True, dtype=torch.bfloat16, max_seq_len=4096, download_dir=None, load_format=auto, tensor_parallel_size=1, pipeline_parallel_size=1, data_parallel_size=1, disable_custom_all_reduce=False, quantization=None, enforce_eager=False, kv_cache_dtype=auto, device_config=cuda, structured_outputs_config=StructuredOutputsConfig(backend='auto', disable_fallback=False, disable_any_whitespace=False, disable_additional_properties=False, reasoning_parser='', reasoning_parser_plugin='', enable_in_reasoning=False), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None, kv_cache_metrics=False, kv_cache_metrics_sample=0.01, cudagraph_metrics=False, enable_layerwise_nvtx_tracing=False), seed=0, served_model_name=mistralai/Voxtral-Mini-3B-2507, enable_prefix_caching=True, enable_chunked_prefill=True, pooler_config=None, compilation_config={'level': None, 'mode': <CompilationMode.VLLM_COMPILE: 3>, 'debug_dump_path': None, 'cache_dir': '', 'compile_cache_save_format': 'binary', 'backend': 'inductor', 'custom_ops': ['none'], 'splitting_ops': ['vllm::unified_attention', 'vllm::unified_attention_with_output', 'vllm::unified_mla_attention', 'vllm::unified_mla_attention_with_output', 'vllm::mamba_mixer2', 'vllm::mamba_mixer', 'vllm::short_conv', 'vllm::linear_attention', 'vllm::plamo2_mamba_mixer', 'vllm::gdn_attention_core', 'vllm::kda_attention', 'vllm::sparse_attn_indexer'], 'compile_mm_encoder': False, 'compile_sizes': [], 'compile_ranges_split_points': [8192], 'inductor_compile_config': {'enable_auto_functionalized_v2': False, 'combo_kernels': True, 'benchmark_combo_kernel': True}, 'inductor_passes': {}, 'cudagraph_mode': <CUDAGraphMode.FULL_AND_PIECEWISE: (2, 1)>, 'cudagraph_num_of_warmups': 1, 'cudagraph_capture_sizes': [1, 2, 4, 8, 16, 24, 32, 40, 48, 56, 64, 72, 80, 88, 96, 104, 112, 120, 128, 136, 144, 152, 160, 168, 176, 184, 192, 200, 208, 216, 224, 232, 240, 248, 256, 272, 288, 304, 320, 336, 352, 368, 384, 400, 416, 432, 448, 464, 480, 496, 512], 'cudagraph_copy_inputs': False, 'cudagraph_specialize_lora': True, 'use_inductor_graph_partition': False, 'pass_config': {'fuse_norm_quant': False, 'fuse_act_quant': False, 'fuse_attn_quant': False, 'eliminate_noops': True, 'enable_sp': False, 'fuse_gemm_comms': False, 'fuse_allreduce_rms': False}, 'max_cudagraph_capture_size': 512, 'dynamic_shapes_config': {'type': <DynamicShapesType.BACKED: 'backed'>, 'evaluate_guards': False}, 'local_cache_dir': None}
(EngineCore_DP0 pid=1836852) [2026-01-14 23:18:46] INFO tekken.py:195: Non special vocabulary size is 130072 with 1000 special tokens.
(EngineCore_DP0 pid=1836852) [2026-01-14 23:18:46] INFO tekken.py:567: Cutting non special vocabulary to first 130072 tokens.
(EngineCore_DP0 pid=1836852) INFO 01-14 23:18:47 [parallel_state.py:1203] world_size=1 rank=0 local_rank=0 distributed_init_method=tcp://216.81.248.184:41023 backend=nccl
(EngineCore_DP0 pid=1836852) INFO 01-14 23:18:47 [parallel_state.py:1411] rank 0 in world size 1 is assigned as DP rank 0, PP rank 0, PCP rank 0, TP rank 0, EP rank 0
(EngineCore_DP0 pid=1836852) INFO 01-14 23:18:48 [gpu_model_runner.py:3562] Starting to load model mistralai/Voxtral-Mini-3B-2507...
(EngineCore_DP0 pid=1836852) INFO 01-14 23:18:48 [cuda.py:351] Using FLASH_ATTN attention backend out of potential backends: ('FLASH_ATTN', 'FLASHINFER', 'TRITON_ATTN', 'FLEX_ATTENTION')
(EngineCore_DP0 pid=1836852) INFO 01-14 23:18:48 [layer.py:477] Using AttentionBackendEnum.FLASH_ATTN for MultiHeadAttention in multimodal encoder.
(EngineCore_DP0 pid=1836852) INFO 01-14 23:18:49 [weight_utils.py:527] No consolidated.safetensors.index.json found in remote.
(EngineCore_DP0 pid=1836852)
Loading safetensors checkpoint shards:   0% Completed | 0/1 [00:00<?, ?it/s]
(EngineCore_DP0 pid=1836852)
Loading safetensors checkpoint shards: 100% Completed | 1/1 [00:01<00:00,  1.96s/it]
(EngineCore_DP0 pid=1836852)
Loading safetensors checkpoint shards: 100% Completed | 1/1 [00:01<00:00,  1.96s/it]
(EngineCore_DP0 pid=1836852)
(EngineCore_DP0 pid=1836852) INFO 01-14 23:18:51 [default_loader.py:308] Loading weights took 2.28 seconds
(EngineCore_DP0 pid=1836852) INFO 01-14 23:18:52 [gpu_model_runner.py:3659] Model loading took 8.7223 GiB memory and 2.870745 seconds
(EngineCore_DP0 pid=1836852) INFO 01-14 23:18:52 [gpu_model_runner.py:4446] Encoder cache will be initialized with a budget of 8192 tokens, and profiled with 2 audio items of the maximum feature size.
(EngineCore_DP0 pid=1836852) ERROR 01-14 23:18:53 [core.py:866] EngineCore failed to start.
(EngineCore_DP0 pid=1836852) ERROR 01-14 23:18:53 [core.py:866] Traceback (most recent call last):
(EngineCore_DP0 pid=1836852) ERROR 01-14 23:18:53 [core.py:866]   File "/home/ubuntu/.local/lib/python3.10/site-packages/vllm/multimodal/processing.py", line 1115, in call_hf_processor
(EngineCore_DP0 pid=1836852) ERROR 01-14 23:18:53 [core.py:866]     output = hf_processor(**data, **allowed_kwargs, return_tensors="pt")
(EngineCore_DP0 pid=1836852) ERROR 01-14 23:18:53 [core.py:866]   File "/home/ubuntu/.local/lib/python3.10/site-packages/vllm/model_executor/models/voxtral.py", line 161, in __call__
(EngineCore_DP0 pid=1836852) ERROR 01-14 23:18:53 [core.py:866]     audio = self._audio_processor.pad(audio, self.sampling_rate)
(EngineCore_DP0 pid=1836852) ERROR 01-14 23:18:53 [core.py:866] TypeError: AudioEncoder.pad() missing 1 required positional argument: 'is_online_streaming'
(EngineCore_DP0 pid=1836852) ERROR 01-14 23:18:53 [core.py:866]
(EngineCore_DP0 pid=1836852) ERROR 01-14 23:18:53 [core.py:866] The above exception was the direct cause of the following exception:
(EngineCore_DP0 pid=1836852) ERROR 01-14 23:18:53 [core.py:866]
(EngineCore_DP0 pid=1836852) ERROR 01-14 23:18:53 [core.py:866] Traceback (most recent call last):
(EngineCore_DP0 pid=1836852) ERROR 01-14 23:18:53 [core.py:866]   File "/home/ubuntu/.local/lib/python3.10/site-packages/vllm/v1/engine/core.py", line 857, in run_engine_core
(EngineCore_DP0 pid=1836852) ERROR 01-14 23:18:53 [core.py:866]     engine_core = EngineCoreProc(*args, **kwargs)
(EngineCore_DP0 pid=1836852) ERROR 01-14 23:18:53 [core.py:866]   File "/home/ubuntu/.local/lib/python3.10/site-packages/vllm/v1/engine/core.py", line 637, in __init__
(EngineCore_DP0 pid=1836852) ERROR 01-14 23:18:53 [core.py:866]     super().__init__(
(EngineCore_DP0 pid=1836852) ERROR 01-14 23:18:53 [core.py:866]   File "/home/ubuntu/.local/lib/python3.10/site-packages/vllm/v1/engine/core.py", line 109, in __init__
(EngineCore_DP0 pid=1836852) ERROR 01-14 23:18:53 [core.py:866]     num_gpu_blocks, num_cpu_blocks, kv_cache_config = self._initialize_kv_caches(
(EngineCore_DP0 pid=1836852) ERROR 01-14 23:18:53 [core.py:866]   File "/home/ubuntu/.local/lib/python3.10/site-packages/vllm/v1/engine/core.py", line 240, in _initialize_kv_caches
(EngineCore_DP0 pid=1836852) ERROR 01-14 23:18:53 [core.py:866]     available_gpu_memory = self.model_executor.determine_available_memory()
(EngineCore_DP0 pid=1836852) ERROR 01-14 23:18:53 [core.py:866]   File "/home/ubuntu/.local/lib/python3.10/site-packages/vllm/v1/executor/abstract.py", line 126, in determine_available_memory
(EngineCore_DP0 pid=1836852) ERROR 01-14 23:18:53 [core.py:866]     return self.collective_rpc("determine_available_memory")
(EngineCore_DP0 pid=1836852) ERROR 01-14 23:18:53 [core.py:866]   File "/home/ubuntu/.local/lib/python3.10/site-packages/vllm/v1/executor/uniproc_executor.py", line 75, in collective_rpc
(EngineCore_DP0 pid=1836852) ERROR 01-14 23:18:53 [core.py:866]     result = run_method(self.driver_worker, method, args, kwargs)
(EngineCore_DP0 pid=1836852) ERROR 01-14 23:18:53 [core.py:866]   File "/home/ubuntu/.local/lib/python3.10/site-packages/vllm/v1/serial_utils.py", line 461, in run_method
(EngineCore_DP0 pid=1836852) ERROR 01-14 23:18:53 [core.py:866]     return func(*args, **kwargs)
(EngineCore_DP0 pid=1836852) ERROR 01-14 23:18:53 [core.py:866]   File "/home/ubuntu/.local/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 120, in decorate_context
(EngineCore_DP0 pid=1836852) ERROR 01-14 23:18:53 [core.py:866]     return func(*args, **kwargs)
(EngineCore_DP0 pid=1836852) ERROR 01-14 23:18:53 [core.py:866]   File "/home/ubuntu/.local/lib/python3.10/site-packages/vllm/v1/worker/gpu_worker.py", line 340, in determine_available_memory
(EngineCore_DP0 pid=1836852) ERROR 01-14 23:18:53 [core.py:866]     self.model_runner.profile_run()
(EngineCore_DP0 pid=1836852) ERROR 01-14 23:18:53 [core.py:866]   File "/home/ubuntu/.local/lib/python3.10/site-packages/vllm/v1/worker/gpu_model_runner.py", line 4456, in profile_run
(EngineCore_DP0 pid=1836852) ERROR 01-14 23:18:53 [core.py:866]     batched_dummy_mm_inputs = self._get_mm_dummy_batch(
(EngineCore_DP0 pid=1836852) ERROR 01-14 23:18:53 [core.py:866]   File "/home/ubuntu/.local/lib/python3.10/site-packages/vllm/v1/worker/gpu_model_runner.py", line 3948, in _get_mm_dummy_batch
(EngineCore_DP0 pid=1836852) ERROR 01-14 23:18:53 [core.py:866]     dummy_decoder_data = self.mm_registry.get_decoder_dummy_data(
(EngineCore_DP0 pid=1836852) ERROR 01-14 23:18:53 [core.py:866]   File "/home/ubuntu/.local/lib/python3.10/site-packages/vllm/multimodal/registry.py", line 293, in get_decoder_dummy_data
(EngineCore_DP0 pid=1836852) ERROR 01-14 23:18:53 [core.py:866]     dummy_data = profiler.get_decoder_dummy_data(seq_len, mm_counts, mm_options)
(EngineCore_DP0 pid=1836852) ERROR 01-14 23:18:53 [core.py:866]   File "/home/ubuntu/.local/lib/python3.10/site-packages/vllm/multimodal/profiling.py", line 313, in get_decoder_dummy_data
(EngineCore_DP0 pid=1836852) ERROR 01-14 23:18:53 [core.py:866]     mm_inputs = self._get_dummy_mm_inputs(seq_len, mm_counts, mm_options)
(EngineCore_DP0 pid=1836852) ERROR 01-14 23:18:53 [core.py:866]   File "/home/ubuntu/.local/lib/python3.10/site-packages/vllm/multimodal/profiling.py", line 267, in _get_dummy_mm_inputs
(EngineCore_DP0 pid=1836852) ERROR 01-14 23:18:53 [core.py:866]     return self.processor.apply(
(EngineCore_DP0 pid=1836852) ERROR 01-14 23:18:53 [core.py:866]   File "/home/ubuntu/.local/lib/python3.10/site-packages/vllm/multimodal/processing.py", line 2133, in apply
(EngineCore_DP0 pid=1836852) ERROR 01-14 23:18:53 [core.py:866]     ) = self._cached_apply_hf_processor(
(EngineCore_DP0 pid=1836852) ERROR 01-14 23:18:53 [core.py:866]   File "/home/ubuntu/.local/lib/python3.10/site-packages/vllm/model_executor/models/voxtral.py", line 310, in _cached_apply_hf_processor
(EngineCore_DP0 pid=1836852) ERROR 01-14 23:18:53 [core.py:866]     prompt_ids, mm_info, _ = super()._cached_apply_hf_processor(
(EngineCore_DP0 pid=1836852) ERROR 01-14 23:18:53 [core.py:866]   File "/home/ubuntu/.local/lib/python3.10/site-packages/vllm/multimodal/processing.py", line 1915, in _cached_apply_hf_processor
(EngineCore_DP0 pid=1836852) ERROR 01-14 23:18:53 [core.py:866]     ) = self._apply_hf_processor_main(
(EngineCore_DP0 pid=1836852) ERROR 01-14 23:18:53 [core.py:866]   File "/home/ubuntu/.local/lib/python3.10/site-packages/vllm/multimodal/processing.py", line 1652, in _apply_hf_processor_main
(EngineCore_DP0 pid=1836852) ERROR 01-14 23:18:53 [core.py:866]     mm_processed_data = self._apply_hf_processor_mm_only(
(EngineCore_DP0 pid=1836852) ERROR 01-14 23:18:53 [core.py:866]   File "/home/ubuntu/.local/lib/python3.10/site-packages/vllm/multimodal/processing.py", line 1610, in _apply_hf_processor_mm_only
(EngineCore_DP0 pid=1836852) ERROR 01-14 23:18:53 [core.py:866]     _, mm_processed_data, _ = self._apply_hf_processor_text_mm(
(EngineCore_DP0 pid=1836852) ERROR 01-14 23:18:53 [core.py:866]   File "/home/ubuntu/.local/lib/python3.10/site-packages/vllm/multimodal/processing.py", line 1537, in _apply_hf_processor_text_mm
(EngineCore_DP0 pid=1836852) ERROR 01-14 23:18:53 [core.py:866]     processed_data = self._call_hf_processor(
(EngineCore_DP0 pid=1836852) ERROR 01-14 23:18:53 [core.py:866]   File "/home/ubuntu/.local/lib/python3.10/site-packages/vllm/multimodal/processing.py", line 1497, in _call_hf_processor
(EngineCore_DP0 pid=1836852) ERROR 01-14 23:18:53 [core.py:866]     return self.info.ctx.call_hf_processor(
(EngineCore_DP0 pid=1836852) ERROR 01-14 23:18:53 [core.py:866]   File "/home/ubuntu/.local/lib/python3.10/site-packages/vllm/multimodal/processing.py", line 1144, in call_hf_processor
(EngineCore_DP0 pid=1836852) ERROR 01-14 23:18:53 [core.py:866]     raise ValueError(msg) from exc
(EngineCore_DP0 pid=1836852) ERROR 01-14 23:18:53 [core.py:866] ValueError: Failed to apply VoxtralProcessorAdapter on data={'text': '', 'audios': [array([0., 0., 0., ..., 0., 0., 0.], dtype=float32)]} with kwargs={}
(EngineCore_DP0 pid=1836852) Process EngineCore_DP0:
(EngineCore_DP0 pid=1836852) Traceback (most recent call last):
(EngineCore_DP0 pid=1836852)   File "/home/ubuntu/.local/lib/python3.10/site-packages/vllm/multimodal/processing.py", line 1115, in call_hf_processor
(EngineCore_DP0 pid=1836852)     output = hf_processor(**data, **allowed_kwargs, return_tensors="pt")
(EngineCore_DP0 pid=1836852)   File "/home/ubuntu/.local/lib/python3.10/site-packages/vllm/model_executor/models/voxtral.py", line 161, in __call__
(EngineCore_DP0 pid=1836852)     audio = self._audio_processor.pad(audio, self.sampling_rate)
(EngineCore_DP0 pid=1836852) TypeError: AudioEncoder.pad() missing 1 required positional argument: 'is_online_streaming'
(EngineCore_DP0 pid=1836852)
(EngineCore_DP0 pid=1836852) The above exception was the direct cause of the following exception:
(EngineCore_DP0 pid=1836852)
(EngineCore_DP0 pid=1836852) Traceback (most recent call last):
(EngineCore_DP0 pid=1836852)   File "/usr/lib/python3.10/multiprocessing/process.py", line 314, in _bootstrap
(EngineCore_DP0 pid=1836852)     self.run()
(EngineCore_DP0 pid=1836852)   File "/usr/lib/python3.10/multiprocessing/process.py", line 108, in run
(EngineCore_DP0 pid=1836852)     self._target(*self._args, **self._kwargs)
(EngineCore_DP0 pid=1836852)   File "/home/ubuntu/.local/lib/python3.10/site-packages/vllm/v1/engine/core.py", line 870, in run_engine_core
(EngineCore_DP0 pid=1836852)     raise e
(EngineCore_DP0 pid=1836852)   File "/home/ubuntu/.local/lib/python3.10/site-packages/vllm/v1/engine/core.py", line 857, in run_engine_core
(EngineCore_DP0 pid=1836852)     engine_core = EngineCoreProc(*args, **kwargs)
(EngineCore_DP0 pid=1836852)   File "/home/ubuntu/.local/lib/python3.10/site-packages/vllm/v1/engine/core.py", line 637, in __init__
(EngineCore_DP0 pid=1836852)     super().__init__(
(EngineCore_DP0 pid=1836852)   File "/home/ubuntu/.local/lib/python3.10/site-packages/vllm/v1/engine/core.py", line 109, in __init__
(EngineCore_DP0 pid=1836852)     num_gpu_blocks, num_cpu_blocks, kv_cache_config = self._initialize_kv_caches(
(EngineCore_DP0 pid=1836852)   File "/home/ubuntu/.local/lib/python3.10/site-packages/vllm/v1/engine/core.py", line 240, in _initialize_kv_caches
(EngineCore_DP0 pid=1836852)     available_gpu_memory = self.model_executor.determine_available_memory()
(EngineCore_DP0 pid=1836852)   File "/home/ubuntu/.l…91010 tokens truncated…      │ │
│ │              eng_identity = b'\x00\x00'                                  │ │
│ │                 eng_index = 0                                            │ │
│ │                    engine = <vllm.v1.engine.utils.CoreEngine object at   │ │
│ │                             0x7cbf636d0eb0>                              │ │
│ │                    events = [(60, 4)]                                    │ │
│ │                  finished = {}                                           │ │
│ │          handshake_socket = <zmq.Socket(zmq.ROUTER) at 0x7cbf636ecd60    │ │
│ │                             closed>                                      │ │
│ │                  headless = False                                        │ │
│ │              init_message = b'\x83\xa9addresses\x85\xa6inputs\x91\xd9/i… │ │
│ │                     local = True                                         │ │
│ │               local_count = 1                                            │ │
│ │                       msg = {                                            │ │
│ │                             │   'status': 'HELLO',                       │ │
│ │                             │   'local': True,                           │ │
│ │                             │   'headless': False                        │ │
│ │                             }                                            │ │
│ │           parallel_config = ParallelConfig(                              │ │
│ │                             │   pipeline_parallel_size=1,                │ │
│ │                             │   tensor_parallel_size=1,                  │ │
│ │                             │   prefill_context_parallel_size=1,         │ │
│ │                             │   data_parallel_size=1,                    │ │
│ │                             │   data_parallel_size_local=1,              │ │
│ │                             │   data_parallel_rank=0,                    │ │
│ │                             │   data_parallel_rank_local=0,              │ │
│ │                             │   data_parallel_master_ip='127.0.0.1',     │ │
│ │                             │   data_parallel_rpc_port=29550,            │ │
│ │                             │   data_parallel_master_port=0,             │ │
│ │                             │   data_parallel_backend='mp',              │ │
│ │                             │   data_parallel_external_lb=False,         │ │
│ │                             │   data_parallel_hybrid_lb=False,           │ │
│ │                             │   enable_expert_parallel=False,            │ │
│ │                             │   enable_eplb=False,                       │ │
│ │                             │   eplb_config=EPLBConfig(                  │ │
│ │                             │   │   window_size=1000,                    │ │
│ │                             │   │   step_interval=3000,                  │ │
│ │                             │   │   num_redundant_experts=0,             │ │
│ │                             │   │   log_balancedness=False,              │ │
│ │                             │   │   use_async=False,                     │ │
│ │                             │   │   policy='default'                     │ │
│ │                             │   ),                                       │ │
│ │                             │   expert_placement_strategy='linear',      │ │
│ │                             │                                            │ │
│ │                             all2all_backend='allgather_reducescatter',   │ │
│ │                             │   max_parallel_loading_workers=None,       │ │
│ │                             │   disable_custom_all_reduce=False,         │ │
│ │                             │   enable_dbo=False,                        │ │
│ │                             │   dbo_decode_token_threshold=32,           │ │
│ │                             │   dbo_prefill_token_threshold=512,         │ │
│ │                             │                                            │ │
│ │                             disable_nccl_for_dp_synchronization=False,   │ │
│ │                             │   ray_workers_use_nsight=False,            │ │
│ │                             │   ray_runtime_env=None,                    │ │
│ │                             │   placement_group=None,                    │ │
│ │                             │   distributed_executor_backend='uni',      │ │
│ │                             │                                            │ │
│ │                             worker_cls='vllm.v1.worker.gpu_worker.Worke… │ │
│ │                             │   sd_worker_cls='auto',                    │ │
│ │                             │   worker_extension_cls='',                 │ │
│ │                             │   master_addr='127.0.0.1',                 │ │
│ │                             │   master_port=29501,                       │ │
│ │                             │   node_rank=0,                             │ │
│ │                             │   nnodes=1,                                │ │
│ │                             │   world_size=1,                            │ │
│ │                             │   rank=0,                                  │ │
│ │                             │   _data_parallel_master_port_list=[],      │ │
│ │                             │   decode_context_parallel_size=1,          │ │
│ │                             │   dcp_kv_cache_interleave_size=1,          │ │
│ │                             │   cp_kv_cache_interleave_size=1,           │ │
│ │                             │   _api_process_count=1,                    │ │
│ │                             │   _api_process_rank=0                      │ │
│ │                             )                                            │ │
│ │                    poller = <zmq.sugar.poll.Poller object at             │ │
│ │                             0x7cbf636d1930>                              │ │
│ │              proc_manager = <vllm.v1.engine.utils.CoreEngineProcManager  │ │
│ │                             object at 0x7cbf636d1a50>                    │ │
│ │           ready_msg_bytes = b'\x83\xa6status\xa5HELLO\xa5local\xc3\xa8h… │ │
│ │              remote_count = 0                                            │ │
│ │ remote_should_be_headless = True                                         │ │
│ │                  sentinel = 60                                           │ │
│ │             start_pending = [1, 0]                                       │ │
│ │                    status = 'HELLO'                                      │ │
│ ╰──────────────────────────────────────────────────────────────────────────╯ │
╰──────────────────────────────────────────────────────────────────────────────╯
RuntimeError: Engine core initialization failed. See root cause above. Failed
core proc(s): {}

The above exception was the direct cause of the following exception:

╭───────────────────── Traceback (most recent call last) ──────────────────────╮
│ /home/ubuntu/maya-transcribe/maya_transcribe/cli.py:463 in bench_run         │
│                                                                              │
│   460 │   │   thresholds["require_ctc"] = require_ctc                        │
│   461 │   │   thresholds["min_ctc_margin"] = float(min_ctc_margin)           │
│   462 │                                                                      │
│ ❱ 463 │   results_path = run_pipeline(                                       │
│   464 │   │   segments_dir=audio_dir,                                        │
│   465 │   │   out_dir=out,                                                   │
│   466 │   │   lid_model_id=lid_model_id,                                     │
│                                                                              │
│ ╭───────────────────────────────── locals ─────────────────────────────────╮ │
│ │                      audio_dir = PosixPath('data/bench/maya_hf_hin_v2/a… │ │
│ │                      bench_dir = PosixPath('data/bench/maya_hf_hin_v2')  │ │
│ │                        ctc_map = None                                    │ │
│ │                      ctc_model = None                                    │ │
│ │                  ctc_model_map = None                                    │ │
│ │                     hypothesis = 'final'                                 │ │
│ │                         ic_map = None                                    │ │
│ │   indicconformer_default_model = None                                    │ │
│ │       indicconformer_model_map = None                                    │ │
│ │                   lid_model_id = 'facebook/mms-lid-256'                  │ │
│ │                          limit = None                                    │ │
│ │                 min_ctc_margin = 0.05                                    │ │
│ │                min_silence_sec = 0.15                                    │ │
│ │                  mms_asr_model = None                                    │ │
│ │                            out = PosixPath('outputs/bench/maya_hf_hin_v… │ │
│ │                      refs_path = PosixPath('data/bench/maya_hf_hin_v2/r… │ │
│ │                    require_ctc = True                                    │ │
│ │            require_lid_backend = True                                    │ │
│ │           silence_threshold_db = -40.0                                   │ │
│ │                   skip_whisper = True                                    │ │
│ │                         tf_map = None                                    │ │
│ │                     thresholds = {                                       │ │
│ │                                  │   'route_lang_from_path': True,       │ │
│ │                                  │   'route_lang_prefer_path': True      │ │
│ │                                  }                                       │ │
│ │                thresholds_json = None                                    │ │
│ │     transformers_whisper_model = []                                      │ │
│ │ transformers_whisper_model_map = None                                    │ │
│ │                   trim_silence = False                                   │ │
│ │                 vllm_asr_model = 'mistralai/Voxtral-Mini-3B-2507'        │ │
│ │    vllm_gpu_memory_utilization = 0.15                                    │ │
│ │             vllm_max_model_len = 4096                                    │ │
│ │                  whisper_model = 'large-v3'                              │ │
│ ╰──────────────────────────────────────────────────────────────────────────╯ │
│                                                                              │
│ /home/ubuntu/maya-transcribe/maya_transcribe/pipeline.py:185 in run_pipeline │
│                                                                              │
│   182 │   │   │   │   max_model_len=vllm_max_model_len,                      │
│   183 │   │   │   )                                                          │
│   184 │   │   except Exception as e:                                         │
│ ❱ 185 │   │   │   raise RuntimeError("Failed to initialize vLLM ASR backend. │
│   186 │                                                                      │
│   187 │   default_whisper_run_sampling = bool(whisper_backend is not None) a │
│   188 │   │   transformers_backends or transformers_router or gemini_backend │
│                                                                              │
│ ╭───────────────────────────────── locals ─────────────────────────────────╮ │
│ │                       audio_files = [                                    │ │
│ │                                     │                                    │ │
│ │                                     PosixPath('data/bench/maya_hf_hin_v… │ │
│ │                                     │                                    │ │
│ │                                     PosixPath('data/bench/maya_hf_hin_v… │ │
│ │                                     │                                    │ │
│ │                                     PosixPath('data/bench/maya_hf_hin_v… │ │
│ │                                     │                                    │ │
│ │                                     PosixPath('data/bench/maya_hf_hin_v… │ │
│ │                                     │                                    │ │
│ │                                     PosixPath('data/bench/maya_hf_hin_v… │ │
│ │                                     │                                    │ │
│ │                                     PosixPath('data/bench/maya_hf_hin_v… │ │
│ │                                     │                                    │ │
│ │                                     PosixPath('data/bench/maya_hf_hin_v… │ │
│ │                                     │                                    │ │
│ │                                     PosixPath('data/bench/maya_hf_hin_v… │ │
│ │                                     │                                    │ │
│ │                                     PosixPath('data/bench/maya_hf_hin_v… │ │
│ │                                     │                                    │ │
│ │                                     PosixPath('data/bench/maya_hf_hin_v… │ │
│ │                                     │   ... +190                         │ │
│ │                                     ]                                    │ │
│ │                         ctc_model = None                                 │ │
│ │                     ctc_model_map = None                                 │ │
│ │                        ctc_scorer = None                                 │ │
│ │ extra_transformers_whisper_models = []                                   │ │
│ │                    gemini_backend = None                                 │ │
│ │                  gemini_hard_only = True                                 │ │
│ │          gemini_max_output_tokens = 512                                  │ │
│ │                      gemini_model = None                                 │ │
│ │                gemini_prompt_path = None                                 │ │
│ │      indicconformer_default_model = None                                 │ │
│ │          indicconformer_model_map = None                                 │ │
│ │             indicconformer_router = None                                 │ │
│ │                       lid_backend = <maya_transcribe.lid.MMSLID object   │ │
│ │                                     at 0x7cc17dae4a90>                   │ │
│ │                      lid_model_id = 'facebook/mms-lid-256'               │ │
│ │                             limit = None                                 │ │
│ │                   min_silence_sec = 0.15                                 │ │
│ │                     mms_asr_model = None                                 │ │
│ │                       mms_backend = None                                 │ │
│ │                           out_dir = PosixPath('outputs/bench/maya_hf_hi… │ │
│ │                       rank_by_ctc = None                                 │ │
│ │               require_lid_backend = True                                 │ │
│ │                      segments_dir = PosixPath('data/bench/maya_hf_hin_v… │ │
│ │              silence_threshold_db = -40.0                                │ │
│ │                      skip_whisper = True                                 │ │
│ │                        thresholds = {                                    │ │
│ │                                     │   'route_lang_from_path': True,    │ │
│ │                                     │   'route_lang_prefer_path': True   │ │
│ │                                     }                                    │ │
│ │             transformers_backends = []                                   │ │
│ │               transformers_router = None                                 │ │
│ │  transformers_whisper_default_model None                                 │ │
│ │                                   =                                      │ │
│ │    transformers_whisper_model_map = None                                 │ │
│ │                      trim_silence = False                                │ │
│ │                    vllm_asr_model = 'mistralai/Voxtral-Mini-3B-2507'     │ │
│ │                      vllm_backend = None                                 │ │
│ │       vllm_gpu_memory_utilization = 0.15                                 │ │
│ │                vllm_max_model_len = 4096                                 │ │
│ │                   whisper_backend = None                                 │ │
│ │                     whisper_model = 'large-v3'                           │ │
│ ╰──────────────────────────────────────────────────────────────────────────╯ │
╰──────────────────────────────────────────────────────────────────────────────╯
RuntimeError: Failed to initialize vLLM ASR backend.


**Identifying patch scope issue**

The patch doesn't propagate to vLLM's worker process because it uses spawn and loads modules independently, so patching the main process isn't enough; applying the patch inside the worker—likely by modifying vLLM's site-packages code—is necessary.


**Considering dependency version adjustment**

I'm weighing patching vLLM's local code versus downgrading mistral_common to a version matching vLLM requirements, which could naturally resolve the signature mismatch in both main and worker processes.


python3 - <<'PY'
import importlib.metadata as m
print(m.version('vllm'))
requires = m.requires('vllm') or []
for req in requires:
    if 'mistral' in req.lower():
        print(req)
PY


Output:
Exit code: 0
Wall time: 0.1 seconds
Output:
0.13.0
mistral_common[image]>=1.8.5
mistral_common[audio]; extra == "audio"


**Considering patch over upgrade**

vLLM 0.13.0 mismatches mistral_common's method signature due to extra argument in AudioEncoder.pad; upgrading vLLM might fix this but patching vLLM's voxtral.py to add the missing argument is likely simpler and more controlled.


**Locating target lines for patch**


python3 - <<'PY'
from pathlib import Path
path = Path('/home/ubuntu/.local/lib/python3.10/site-packages/vllm/model_executor/models/voxtral.py')
for i, line in enumerate(path.read_text(encoding='utf-8').splitlines(), 1):
    if 'pad(' in line and 'audio' in line and 'sampling_rate' in line:
        print(i, line)
PY


Output:
Exit code: 0
Wall time: 0 seconds
Output:
161             audio = self._audio_processor.pad(audio, self.sampling_rate)


**Deciding to patch external method**

I'm choosing to patch the external audio processing method via apply_patch to handle differing method signatures, modifying the installed package directly since it’s allowed and not part of the repo.


Input:
*** Begin Patch
*** Update File: /home/ubuntu/.local/lib/python3.10/site-packages/vllm/model_executor/models/voxtral.py
@@
-            audio = self._audio_processor.pad(audio, self.sampling_rate)
+            try:
+                audio = self._audio_processor.pad(audio, self.sampling_rate, False)
+            except TypeError:
+                audio = self._audio_processor.pad(audio, self.sampling_rate)
*** End Patch


Output: {"output":"Success. Updated the following files:\nM /home/ubuntu/.local/lib/python3.10/site-packages/vllm/model_executor/models/voxtral.py\n","metadata":{"exit_code":0,"duration_seconds":0.0}}


**Confirming patch applies in worker processes**


Output:
Exit code: 0
Wall time: 0.3 seconds
Output:
Wed Jan 14 23:19:35 2026
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 560.35.05              Driver Version: 560.35.05      CUDA Version: 12.6     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA A100-SXM4-80GB          On  |   00000000:01:00.0 Off |                    0 |
| N/A   29C    P0             57W /  400W |   61611MiB /  81920MiB |      0%      Default |
|                                         |                        |             Disabled |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI        PID   Type   Process name                              GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|    0   N/A  N/A    738802      C   /usr/bin/python3.12                         61602MiB |
+-----------------------------------------------------------------------------------------+


Summary: auto


**Retrying with patch applied**