Output:
Exit code: 0
Wall time: 342.4 seconds
Total output lines: 898
Output:
/home/ubuntu/.local/lib/python3.10/site-packages/torch/cuda/__init__.py:63: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you.
  import pynvml  # type: ignore[import]
2026-01-15 04:49:24.635189: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
/home/ubuntu/.local/lib/python3.10/site-packages/google/api_core/_python_version_support.py:266: FutureWarning: You are using a Python version (3.10.12) which Google will stop supporting in new releases of google.api_core once it reaches its end of life (2026-10-04). Please upgrade to the latest Python version, or at least Python 3.11, to continue receiving updates for google.api_core past that date.
  warnings.warn(message, FutureWarning)
INFO 01-15 04:49:32 [utils.py:253] non-default args: {'trust_remote_code': True, 'max_model_len': 4096, 'disable_log_stats': True, 'model': 'mistralai/Voxtral-Small-24B-2507'}
The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
INFO 01-15 04:49:32 [model.py:514] Resolved architecture: VoxtralForConditionalGeneration
INFO 01-15 04:49:32 [model.py:1661] Using max model len 4096
INFO 01-15 04:49:32 [scheduler.py:230] Chunked prefill is enabled with max_num_batched_tokens=8192.
[2026-01-15 04:49:33] INFO tekken.py:195: Non special vocabulary size is 130072 with 1000 special tokens.
[2026-01-15 04:49:33] INFO tekken.py:567: Cutting non special vocabulary to first 130072 tokens.
WARNING 01-15 04:49:34 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: CUDA is initialized
/home/ubuntu/.local/lib/python3.10/site-packages/torch/cuda/__init__.py:63: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you.
  import pynvml  # type: ignore[import]
2026-01-15 04:49:39.258034: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
/home/ubuntu/.local/lib/python3.10/site-packages/google/api_core/_python_version_support.py:266: FutureWarning: You are using a Python version (3.10.12) which Google will stop supporting in new releases of google.api_core once it reaches its end of life (2026-10-04). Please upgrade to the latest Python version, or at least Python 3.11, to continue receiving updates for google.api_core past that date.
  warnings.warn(message, FutureWarning)
(EngineCore_DP0 pid=2024739) INFO 01-15 04:49:43 [core.py:93] Initializing a V1 LLM engine (v0.13.0) with config: model='mistralai/Voxtral-Small-24B-2507', speculative_config=None, tokenizer='mistralai/Voxtral-Small-24B-2507', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, tokenizer_revision=None, trust_remote_code=True, dtype=torch.bfloat16, max_seq_len=4096, download_dir=None, load_format=auto, tensor_parallel_size=1, pipeline_parallel_size=1, data_parallel_size=1, disable_custom_all_reduce=False, quantization=None, enforce_eager=False, kv_cache_dtype=auto, device_config=cuda, structured_outputs_config=StructuredOutputsConfig(backend='auto', disable_fallback=False, disable_any_whitespace=False, disable_additional_properties=False, reasoning_parser='', reasoning_parser_plugin='', enable_in_reasoning=False), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None, kv_cache_metrics=False, kv_cache_metrics_sample=0.01, cudagraph_metrics=False, enable_layerwise_nvtx_tracing=False), seed=0, served_model_name=mistralai/Voxtral-Small-24B-2507, enable_prefix_caching=True, enable_chunked_prefill=True, pooler_config=None, compilation_config={'level': None, 'mode': <CompilationMode.VLLM_COMPILE: 3>, 'debug_dump_path': None, 'cache_dir': '', 'compile_cache_save_format': 'binary', 'backend': 'inductor', 'custom_ops': ['none'], 'splitting_ops': ['vllm::unified_attention', 'vllm::unified_attention_with_output', 'vllm::unified_mla_attention', 'vllm::unified_mla_attention_with_output', 'vllm::mamba_mixer2', 'vllm::mamba_mixer', 'vllm::short_conv', 'vllm::linear_attention', 'vllm::plamo2_mamba_mixer', 'vllm::gdn_attention_core', 'vllm::kda_attention', 'vllm::sparse_attn_indexer'], 'compile_mm_encoder': False, 'compile_sizes': [], 'compile_ranges_split_points': [8192], 'inductor_compile_config': {'enable_auto_functionalized_v2': False, 'combo_kernels': True, 'benchmark_combo_kernel': True}, 'inductor_passes': {}, 'cudagraph_mode': <CUDAGraphMode.FULL_AND_PIECEWISE: (2, 1)>, 'cudagraph_num_of_warmups': 1, 'cudagraph_capture_sizes': [1, 2, 4, 8, 16, 24, 32, 40, 48, 56, 64, 72, 80, 88, 96, 104, 112, 120, 128, 136, 144, 152, 160, 168, 176, 184, 192, 200, 208, 216, 224, 232, 240, 248, 256, 272, 288, 304, 320, 336, 352, 368, 384, 400, 416, 432, 448, 464, 480, 496, 512], 'cudagraph_copy_inputs': False, 'cudagraph_specialize_lora': True, 'use_inductor_graph_partition': False, 'pass_config': {'fuse_norm_quant': False, 'fuse_act_quant': False, 'fuse_attn_quant': False, 'eliminate_noops': True, 'enable_sp': False, 'fuse_gemm_comms': False, 'fuse_allreduce_rms': False}, 'max_cudagraph_capture_size': 512, 'dynamic_shapes_config': {'type': <DynamicShapesType.BACKED: 'backed'>, 'evaluate_guards': False}, 'local_cache_dir': None}
(EngineCore_DP0 pid=2024739) [2026-01-15 04:49:44] INFO tekken.py:195: Non special vocabulary size is 130072 with 1000 special tokens.
(EngineCore_DP0 pid=2024739) [2026-01-15 04:49:44] INFO tekken.py:567: Cutting non special vocabulary to first 130072 tokens.
(EngineCore_DP0 pid=2024739) INFO 01-15 04:49:45 [parallel_state.py:1203] world_size=1 rank=0 local_rank=0 distributed_init_method=tcp://216.81.248.184:56275 backend=nccl
(EngineCore_DP0 pid=2024739) INFO 01-15 04:49:45 [parallel_state.py:1411] rank 0 in world size 1 is assigned as DP rank 0, PP rank 0, PCP rank 0, TP rank 0, EP rank 0
(EngineCore_DP0 pid=2024739) INFO 01-15 04:49:46 [gpu_model_runner.py:3562] Starting to load model mistralai/Voxtral-Small-24B-2507...
(EngineCore_DP0 pid=2024739) INFO 01-15 04:49:47 [cuda.py:351] Using FLASH_ATTN attention backend out of potential backends: ('FLASH_ATTN', 'FLASHINFER', 'TRITON_ATTN', 'FLEX_ATTENTION')
(EngineCore_DP0 pid=2024739) INFO 01-15 04:49:47 [layer.py:477] Using AttentionBackendEnum.FLASH_ATTN for MultiHeadAttention in multimodal encoder.
(EngineCore_DP0 pid=2024739) INFO 01-15 04:49:47 [weight_utils.py:527] No consolidated.safetensors.index.json found in remote.
(EngineCore_DP0 pid=2024739)
Loading safetensors checkpoint shards:   0% Completed | 0/1 [00:00<?, ?it/s]
(EngineCore_DP0 pid=2024739)
Loading safetensors checkpoint shards: 100% Completed | 1/1 [00:07<00:00,  7.13s/it]
(EngineCore_DP0 pid=2024739)
Loading safetensors checkpoint shards: 100% Completed | 1/1 [00:07<00:00,  7.13s/it]
(EngineCore_DP0 pid=2024739)
(EngineCore_DP0 pid=2024739) INFO 01-15 04:49:56 [default_loader.py:308] Loading weights took 8.66 seconds
(EngineCore_DP0 pid=2024739) INFO 01-15 04:49:57 [gpu_model_runner.py:3659] Model loading took 45.2032 GiB memory and 9.396152 seconds
(EngineCore_DP0 pid=2024739) INFO 01-15 04:49:57 [gpu_model_runner.py:4446] Encoder cache will be initialized with a budget of 8192 tokens, and profiled with 2 audio items of the maximum feature size.
(EngineCore_DP0 pid=2024739) INFO 01-15 04:50:04 [backends.py:643] Using cache directory: /home/ubuntu/.cache/vllm/torch_compile_cache/936cd8900f/rank_0_0/backbone for vLLM's torch.compile
(EngineCore_DP0 pid=2024739) INFO 01-15 04:50:04 [backends.py:703] Dynamo bytecode transform time: 6.04 s
(EngineCore_DP0 pid=2024739) INFO 01-15 04:50:16 [backends.py:226] Directly load the compiled graph(s) for compile range (1, 8192) from the cache, took 8.711 s
(EngineCore_DP0 pid=2024739) INFO 01-15 04:50:16 [monitor.py:34] torch.compile takes 14.75 s in total
(EngineCore_DP0 pid=2024739) INFO 01-15 04:50:18 [gpu_worker.py:375] Available KV cache memory: 24.25 GiB
(EngineCore_DP0 pid=2024739) INFO 01-15 04:50:18 [kv_cache_utils.py:1291] GPU KV cache size: 158,912 tokens
(EngineCore_DP0 pid=2024739) INFO 01-15 04:50:18 [kv_cache_utils.py:1296] Maximum concurrency for 4,096 tokens per request: 38.80x
(EngineCore_DP0 pid=2024739)
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):   0%|          | 0/51 [00:00<?, ?it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):   2%|▏         | 1/51 [00:00<00:05,  8.42it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):   4%|▍         | 2/51 [00:00<00:05,  8.66it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):   6%|▌         | 3/51 [00:00<00:05,  8.73it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):   8%|▊         | 4/51 [00:00<00:05,  8.79it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  10%|▉         | 5/51 [00:00<00:05,  8.81it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  12%|█▏        | 6/51 [00:00<00:05,  8.83it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  14%|█▎        | 7/51 [00:00<00:04,  8.86it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  16%|█▌        | 8/51 [00:00<00:04,  8.85it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  20%|█▉        | 10/51 [00:01<00:04,  9.42it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  24%|██▎       | 12/51 [00:01<00:04,  9.73it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  27%|██▋       | 14/51 [00:01<00:03,  9.94it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  29%|██▉       | 15/51 [00:01<00:03,  9.92it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  33%|███▎      | 17/51 [00:01<00:03, 10.54it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  37%|███▋      | 19/51 [00:01<00:02, 11.50it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  41%|████      | 21/51 [00:02<00:02, 12.20it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  45%|████▌     | 23/51 [00:02<00:02, 12.73it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  49%|████▉     | 25/51 [00:02<00:02, 12.90it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  53%|█████▎    | 27/51 [00:02<00:01, 13.18it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  57%|█████▋    | 29/51 [00:02<00:01, 13.42it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  61%|██████    | 31/51 [00:02<00:01, 13.61it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  65%|██████▍   | 33/51 [00:02<00:01, 14.02it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  69%|██████▊   | 35/51 [00:03<00:01, 14.61it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  73%|███████▎  | 37/51 [00:03<00:00, 14.90it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  76%|███████▋  | 39/51 [00:03<00:00, 15.37it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  80%|████████  | 41/51 [00:03<00:00, 15.89it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  84%|████████▍ | 43/51 [00:03<00:00, 16.43it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  88%|████████▊ | 45/51 [00:03<00:00, 16.85it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  92%|█████████▏| 47/51 [00:03<00:00, 17.12it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  96%|█████████▌| 49/51 [00:03<00:00, 17.43it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 100%|██████████| 51/51 [00:03<00:00, 17.47it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 100%|██████████| 51/51 [00:03<00:00, 12.88it/s]
(EngineCore_DP0 pid=2024739)
Capturing CUDA graphs (decode, FULL):   0%|          | 0/35 [00:00<?, ?it/s]
Capturing CUDA graphs (decode, FULL):   6%|▌         | 2/35 [00:00<00:02, 12.46it/s]
Capturing CUDA graphs (decode, FULL):  11%|█▏        | 4/35 [00:00<00:02, 13.06it/s]
Capturing CUDA graphs (decode, FULL):  17%|█▋        | 6/35 [00:00<00:02, 13.46it/s]
Capturing CUDA graphs (decode, FULL):  23%|██▎       | 8/35 [00:00<00:01, 13.70it/s]
Capturing CUDA graphs (decode, FULL):  29%|██▊       | 10/35 [00:00<00:01, 13.90it/s]
Capturing CUDA graphs (decode, FULL):  34%|███▍      | 12/35 [00:00<00:01, 14.01it/s]
Capturing CUDA graphs (decode, FULL):  40%|████      | 14/35 [00:01<00:01, 14.10it/s]
Capturing CUDA graphs (decode, FULL):  46%|████▌     | 16/35 [00:01<00:01, 14.15it/s]
Capturing CUDA graphs (decode, FULL):  51%|█████▏    | 18/35 [00:01<00:01, 14.91it/s]
Capturing CUDA graphs (decode, FULL):  57%|█████▋    | 20/35 [00:01<00:00, 15.48it/s]
Capturing CUDA graphs (decode, FULL):  63%|██████▎   | 22/35 [00:01<00:00, 15.98it/s]
Capturing CUDA graphs (decode, FULL):  69%|██████▊   | 24/35 [00:01<00:00, 16.30it/s]
Capturing CUDA graphs (decode, FULL):  74%|███████▍  | 26/35 [00:01<00:00, 16.99it/s]
Capturing CUDA graphs (decode, FULL):  80%|████████  | 28/35 [00:01<00:00, 17.27it/s]
Capturing CUDA graphs (decode, FULL):  86%|████████▌ | 30/35 [00:01<00:00, 17.85it/s]
Capturing CUDA graphs (decode, FULL):  91%|█████████▏| 32/35 [00:02<00:00, 18.22it/s]
Capturing CUDA graphs (decode, FULL):  97%|█████████▋| 34/35 [00:02<00:00, 18.57it/s]
Capturing CUDA graphs (decode, FULL): 100%|██████████| 35/35 [00:02<00:00, 15.89it/s]
(EngineCore_DP0 pid=2024739) INFO 01-15 04:50:26 [gpu_model_runner.py:4587] Graph capturing finished in 7 secs, took 0.71 GiB
(EngineCore_DP0 pid=2024739) INFO 01-15 04:50:26 [core.py:259] init engine (profile, create kv cache, warmup model) took 28.81 seconds
INFO 01-15 04:50:26 [llm.py:360] Supported tasks: ['generate', 'transcription']

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:01<00:00,  1.36s/it]
Adding requests: 100%|██████████| 1/1 [00:01<00:00,  1.36s/it]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  1.14it/s, est. speed input: 712.94 toks/s, output: 27.29 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  1.14it/s, est. speed input: 712.94 toks/s, output: 27.29 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  1.14it/s, est. speed input: 712.94 toks/s, output: 27.29 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 274.50it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:01<00:00,  1.12s/it, est. speed input: 561.30 toks/s, output: 30.44 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:01<00:00,  1.12s/it, est. speed input: 561.30 toks/s, output: 30.44 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:01<00:00,  1.12s/it, est. speed input: 561.30 toks/s, output: 30.44 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 262.93it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:02<00:00,  2.83s/it, est. speed input: 221.63 toks/s, output: 32.17 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:02<00:00,  2.83s/it, est. speed input: 221.63 toks/s, output: 32.17 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:02<00:00,  2.83s/it, est. speed input: 221.63 toks/s, output: 32.17 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 261.28it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:01<00:00,  1.15s/it, est. speed input: 546.95 toks/s, output: 30.53 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:01<00:00,  1.15s/it, est. speed input: 546.95 toks/s, output: 30.53 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:01<00:00,  1.15s/it, est. speed input: 546.95 toks/s, output: 30.53 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 305.80it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  1.27it/s, est. speed input: 799.38 toks/s, output: 29.32 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  1.27it/s, est. speed input: 799.38 toks/s, output: 29.32 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  1.27it/s, est. speed input: 799.38 toks/s, outp…41863 tokens truncated…
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  1.11it/s, est. speed input: 693.92 toks/s, output: 29.88 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 172.61it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  1.04it/s, est. speed input: 652.44 toks/s, output: 30.18 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  1.04it/s, est. speed input: 652.44 toks/s, output: 30.18 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  1.04it/s, est. speed input: 652.44 toks/s, output: 30.18 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 287.16it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:01<00:00,  1.59s/it, est. speed input: 394.33 toks/s, output: 31.45 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:01<00:00,  1.59s/it, est. speed input: 394.33 toks/s, output: 31.45 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:01<00:00,  1.59s/it, est. speed input: 394.33 toks/s, output: 31.45 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 111.81it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  1.19it/s, est. speed input: 744.18 toks/s, output: 29.67 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  1.19it/s, est. speed input: 744.18 toks/s, output: 29.67 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  1.19it/s, est. speed input: 744.18 toks/s, output: 29.67 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 74.89it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  1.11it/s, est. speed input: 694.46 toks/s, output: 29.90 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  1.11it/s, est. speed input: 694.46 toks/s, output: 29.90 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  1.11it/s, est. speed input: 694.46 toks/s, output: 29.90 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 316.69it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  1.01it/s, est. speed input: 631.85 toks/s, output: 30.23 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  1.01it/s, est. speed input: 631.85 toks/s, output: 30.23 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  1.01it/s, est. speed input: 631.85 toks/s, output: 30.23 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 68.73it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  1.39it/s, est. speed input: 868.69 toks/s, output: 29.09 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  1.39it/s, est. speed input: 868.69 toks/s, output: 29.09 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  1.38it/s, est. speed input: 868.69 toks/s, output: 29.09 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 299.76it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  1.44it/s, est. speed input: 904.26 toks/s, output: 28.84 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  1.44it/s, est. speed input: 904.26 toks/s, output: 28.84 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  1.44it/s, est. speed input: 904.26 toks/s, output: 28.84 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 301.19it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  1.14it/s, est. speed input: 717.29 toks/s, output: 29.74 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  1.14it/s, est. speed input: 717.29 toks/s, output: 29.74 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  1.14it/s, est. speed input: 717.29 toks/s, output: 29.74 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 212.19it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:01<00:00,  1.02s/it, est. speed input: 613.79 toks/s, output: 30.35 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:01<00:00,  1.02s/it, est. speed input: 613.79 toks/s, output: 30.35 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:01<00:00,  1.02s/it, est. speed input: 613.79 toks/s, output: 30.35 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 274.42it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  1.23it/s, est. speed input: 770.85 toks/s, output: 29.51 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  1.23it/s, est. speed input: 770.85 toks/s, output: 29.51 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  1.23it/s, est. speed input: 770.85 toks/s, output: 29.51 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 310.57it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  1.28it/s, est. speed input: 801.23 toks/s, output: 29.39 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  1.28it/s, est. speed input: 801.23 toks/s, output: 29.39 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  1.28it/s, est. speed input: 801.23 toks/s, output: 29.39 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 233.91it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  1.45it/s, est. speed input: 906.66 toks/s, output: 28.92 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  1.45it/s, est. speed input: 906.66 toks/s, output: 28.92 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  1.45it/s, est. speed input: 906.66 toks/s, output: 28.92 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 277.46it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  1.28it/s, est. speed input: 800.67 toks/s, output: 29.37 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  1.28it/s, est. speed input: 800.67 toks/s, output: 29.37 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  1.28it/s, est. speed input: 800.67 toks/s, output: 29.37 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 314.49it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  1.01it/s, est. speed input: 632.30 toks/s, output: 30.25 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  1.01it/s, est. speed input: 632.30 toks/s, output: 30.25 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  1.01it/s, est. speed input: 632.30 toks/s, output: 30.25 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 299.89it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  1.19it/s, est. speed input: 744.69 toks/s, output: 29.69 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  1.19it/s, est. speed input: 744.69 toks/s, output: 29.69 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  1.19it/s, est. speed input: 744.69 toks/s, output: 29.69 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 311.33it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  1.19it/s, est. speed input: 744.48 toks/s, output: 29.68 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  1.19it/s, est. speed input: 744.48 toks/s, output: 29.68 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  1.19it/s, est. speed input: 744.48 toks/s, output: 29.68 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 309.25it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:01<00:00,  1.83s/it, est. speed input: 342.63 toks/s, output: 31.69 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:01<00:00,  1.83s/it, est. speed input: 342.63 toks/s, output: 31.69 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:01<00:00,  1.83s/it, est. speed input: 342.63 toks/s, output: 31.69 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 323.78it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  1.19it/s, est. speed input: 744.19 toks/s, output: 29.67 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  1.19it/s, est. speed input: 744.19 toks/s, output: 29.67 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  1.19it/s, est. speed input: 744.19 toks/s, output: 29.67 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 109.99it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  1.14it/s, est. speed input: 715.68 toks/s, output: 29.68 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  1.14it/s, est. speed input: 715.68 toks/s, output: 29.68 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  1.14it/s, est. speed input: 715.68 toks/s, output: 29.68 toks/s]
Transcribing ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:04:30
[rank0]:[W115 04:54:57.092160569 ProcessGroupNCCL.cpp:1524] Warning: WARNING: destroy_process_group() was not called before program exit, which can leak resources. For more info, please see https://pytorch.org/docs/stable/distributed.html#shutdown (function operator())
Wrote: outputs/bench/maya_hf_hin_v2/voxtral_small_vllm_prompt/results.jsonl
Wrote: outputs/bench/maya_hf_hin_v2/voxtral_small_vllm_prompt/summary.json
Wrote: outputs/bench/maya_hf_hin_v2/voxtral_small_vllm_prompt/eval.json
{
  "hypothesis": "final",
  "pairs_evaluated": 200,
  "wer_mean": 0.24517020479520482,
  "cer_mean": 0.17030685871627074,
  "wer_p50": 0.15384615384615385,
  "wer_p90": 0.5,
  "wer_p95": 1.0,
  "cer_p50": 0.05128205128205128,
  "cer_p90": 0.2903225806451613,
  "cer_p95": 1.0,
  "segment_error_rate": 0.78,
  "pct_wer_gt_5pct": 0.78,
  "pct_wer_gt_10pct": 0.6,
  "wer_by_language_mean": {
    "hin": 0.24517020479520482
  },
  "wer_by_decision_mean": {
    "accept": 0.21730925715888133,
    "human": 0.30047626502850366
  },
  "wer_by_duration_bucket_mean": {
    "0-3s": 0.15726010101010102,
    "10-30s": 0.29143772893772896,
    "3-10s": 0.24507180172073792
  },
  "cer_by_language_mean": {
    "hin": 0.17030685871627074
  },
  "cer_by_decision_mean": {
    "accept": 0.14997090401433022,
    "human": 0.21067524640818278
  },
  "cer_by_duration_bucket_mean": {
    "0-3s": 0.062306501547987614,
    "10-30s": 0.11599910265956431,
    "3-10s": 0.17491570699886003
  },
  "oracle": {
    "enabled": true,
    "oracle_best_wer_mean": 0.24517020479520482,
    "selection_gap_wer_mean": 0.0,
    "selection_gap_wer_p90": 0.0
  }
}


Summary: auto

## Skills
A skill is a set of local instructions to follow that is stored in a `SKILL.md` file. Below is the list of skills that can be used. Each entry includes a name, description, and file path so you can open the source for full instructions when using a specific skill.
### Available skills
- skill-creator: Guide for creating effective skills. This skill should be used when users want to create a new skill (or update an existing skill) that extends Codex's capabilities with specialized knowledge, workflows, or tool integrations. (file: /home/ubuntu/.codex/skills/.system/skill-creator/SKILL.md)
- skill-installer: Install Codex skills into $CODEX_HOME/skills from a curated list or a GitHub repo path. Use when a user asks to list installable skills, install a curated skill, or install a skill from another repo (including private repos). (file: /home/ubuntu/.codex/skills/.system/skill-installer/SKILL.md)
### How to use skills
- Discovery: The list above is the skills available in this session (name + description + file path). Skill bodies live on disk at the listed paths.
- Trigger rules: If the user names a skill (with `$SkillName` or plain text) OR the task clearly matches a skill's description shown above, you must use that skill for that turn. Multiple mentions mean use them all. Do not carry skills across turns unless re-mentioned.
- Missing/blocked: If a named skill isn't in the list or the path can't be read, say so briefly and continue with the best fallback.
- How to use a skill (progressive disclosure):
  1) After deciding to use a skill, open its `SKILL.md`. Read only enough to follow the workflow.
  2) If `SKILL.md` points to extra folders such as `references/`, load only the specific files needed for the request; don't bulk-load everything.
  3) If `scripts/` exist, prefer running or patching them instead of retyping large code blocks.
  4) If `assets/` or templates exist, reuse them instead of recreating from scratch.
- Coordination and sequencing:
  - If multiple skills apply, choose the minimal set that covers the request and state the order you'll use them.
  - Announce which skill(s) you're using and why (one short line). If you skip an obvious skill, say why.
- Context hygiene:
  - Keep context small: summarize long sections instead of pasting them; only load extra files when needed.
  - Avoid deep reference-chasing: prefer opening only files directly linked from `SKILL.md` unless you're blocked.
  - When variants exist (frameworks, providers, domains), pick only the relevant reference file(s) and note that choice.
- Safety and fallback: If a skill can't be applied cleanly (missing files, unclear instructions), state the issue, pick the next-best approach, and continue.


**Observing prompt impact on WER**

The prompt caused a significant WER increase to 24.5, so I'll run the Voxtral mini next with the prompt to compare results.


python3 -m maya_transcribe.cli bench run --bench-dir data/bench/maya_hf_hin_v2 --out outputs/bench/maya_hf_hin_v2/voxtral_mini_vllm_prompt --skip-whisper --vllm-asr-model mistralai/Voxtral-Mini-3B-2507 --vllm-gpu-memory-utilization 0.9 --vllm-max-model-len 4096 --vllm-transcribe-prompt-file Prompt.md


Output:
Exit code: 0
Wall time: 164.7 seconds
Total output lines: 898
Output:
/home/ubuntu/.local/lib/python3.10/site-packages/torch/cuda/__init__.py:63: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you.
  import pynvml  # type: ignore[import]
2026-01-15 04:55:10.026645: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
/home/ubuntu/.local/lib/python3.10/site-packages/google/api_core/_python_version_support.py:266: FutureWarning: You are using a Python version (3.10.12) which Google will stop supporting in new releases of google.api_core once it reaches its end of life (2026-10-04). Please upgrade to the latest Python version, or at least Python 3.11, to continue receiving updates for google.api_core past that date.
  warnings.warn(message, FutureWarning)
INFO 01-15 04:55:16 [utils.py:253] non-default args: {'trust_remote_code': True, 'max_model_len': 4096, 'disable_log_stats': True, 'model': 'mistralai/Voxtral-Mini-3B-2507'}
The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
INFO 01-15 04:55:17 [model.py:514] Resolved architecture: VoxtralForConditionalGeneration
INFO 01-15 04:55:17 [model.py:1661] Using max model len 4096
INFO 01-15 04:55:17 [scheduler.py:230] Chunked prefill is enabled with max_num_batched_tokens=8192.
[2026-01-15 04:55:17] INFO tekken.py:195: Non special vocabulary size is 130072 with 1000 special tokens.
[2026-01-15 04:55:17] INFO tekken.py:567: Cutting non special vocabulary to first 130072 tokens.
WARNING 01-15 04:55:19 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: CUDA is initialized
/home/ubuntu/.local/lib/python3.10/site-packages/torch/cuda/__init__.py:63: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you.
  import pynvml  # type: ignore[import]
2026-01-15 04:55:24.299260: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
/home/ubuntu/.local/lib/python3.10/site-packages/google/api_core/_python_version_support.py:266: FutureWarning: You are using a Python version (3.10.12) which Google will stop supporting in new releases of google.api_core once it reaches its end of life (2026-10-04). Please upgrade to the latest Python version, or at least Python 3.11, to continue receiving updates for google.api_core past that date.
  warnings.warn(message, FutureWarning)
(EngineCore_DP0 pid=2040645) INFO 01-15 04:55:28 [core.py:93] Initializing a V1 LLM engine (v0.13.0) with config: model='mistralai/Voxtral-Mini-3B-2507', speculative_config=None, tokenizer='mistralai/Voxtral-Mini-3B-2507', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, tokenizer_revision=None, trust_remote_code=True, dtype=torch.bfloat16, max_seq_len=4096, download_dir=None, load_format=auto, tensor_parallel_size=1, pipeline_parallel_size=1, data_parallel_size=1, disable_custom_all_reduce=False, quantization=None, enforce_eager=False, kv_cache_dtype=auto, device_config=cuda, structured_outputs_config=StructuredOutputsConfig(backend='auto', disable_fallback=False, disable_any_whitespace=False, disable_additional_properties=False, reasoning_parser='', reasoning_parser_plugin='', enable_in_reasoning=False), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None, kv_cache_metrics=False, kv_cache_metrics_sample=0.01, cudagraph_metrics=False, enable_layerwise_nvtx_tracing=False), seed=0, served_model_name=mistralai/Voxtral-Mini-3B-2507, enable_prefix_caching=True, enable_chunked_prefill=True, pooler_config=None, compilation_config={'level': None, 'mode': <CompilationMode.VLLM_COMPILE: 3>, 'debug_dump_path': None, 'cache_dir': '', 'compile_cache_save_format': 'binary', 'backend': 'inductor', 'custom_ops': ['none'], 'splitting_ops': ['vllm::unified_attention', 'vllm::unified_attention_with_output', 'vllm::unified_mla_attention', 'vllm::unified_mla_attention_with_output', 'vllm::mamba_mixer2', 'vllm::mamba_mixer', 'vllm::short_conv', 'vllm::linear_attention', 'vllm::plamo2_mamba_mixer', 'vllm::gdn_attention_core', 'vllm::kda_attention', 'vllm::sparse_attn_indexer'], 'compile_mm_encoder': False, 'compile_sizes': [], 'compile_ranges_split_points': [8192], 'inductor_compile_config': {'enable_auto_functionalized_v2': False, 'combo_kernels': True, 'benchmark_combo_kernel': True}, 'inductor_passes': {}, 'cudagraph_mode': <CUDAGraphMode.FULL_AND_PIECEWISE: (2, 1)>, 'cudagraph_num_of_warmups': 1, 'cudagraph_capture_sizes': [1, 2, 4, 8, 16, 24, 32, 40, 48, 56, 64, 72, 80, 88, 96, 104, 112, 120, 128, 136, 144, 152, 160, 168, 176, 184, 192, 200, 208, 216, 224, 232, 240, 248, 256, 272, 288, 304, 320, 336, 352, 368, 384, 400, 416, 432, 448, 464, 480, 496, 512], 'cudagraph_copy_inputs': False, 'cudagraph_specialize_lora': True, 'use_inductor_graph_partition': False, 'pass_config': {'fuse_norm_quant': False, 'fuse_act_quant': False, 'fuse_attn_quant': False, 'eliminate_noops': True, 'enable_sp': False, 'fuse_gemm_comms': False, 'fuse_allreduce_rms': False}, 'max_cudagraph_capture_size': 512, 'dynamic_shapes_config': {'type': <DynamicShapesType.BACKED: 'backed'>, 'evaluate_guards': False}, 'local_cache_dir': None}
(EngineCore_DP0 pid=2040645) [2026-01-15 04:55:29] INFO tekken.py:195: Non special vocabulary size is 130072 with 1000 special tokens.
(EngineCore_DP0 pid=2040645) [2026-01-15 04:55:29] INFO tekken.py:567: Cutting non special vocabulary to first 130072 tokens.
(EngineCore_DP0 pid=2040645) INFO 01-15 04:55:30 [parallel_state.py:1203] world_size=1 rank=0 local_rank=0 distributed_init_method=tcp://216.81.248.184:49009 backend=nccl
(EngineCore_DP0 pid=2040645) INFO 01-15 04:55:30 [parallel_state.py:1411] rank 0 in world size 1 is assigned as DP rank 0, PP rank 0, PCP rank 0, TP rank 0, EP rank 0
(EngineCore_DP0 pid=2040645) INFO 01-15 04:55:31 [gpu_model_runner.py:3562] Starting to load model mistralai/Voxtral-Mini-3B-2507...
(EngineCore_DP0 pid=2040645) INFO 01-15 04:55:32 [cuda.py:351] Using FLASH_ATTN attention backend out of potential backends: ('FLASH_ATTN', 'FLASHINFER', 'TRITON_ATTN', 'FLEX_ATTENTION')
(EngineCore_DP0 pid=2040645) INFO 01-15 04:55:32 [layer.py:477] Using AttentionBackendEnum.FLASH_ATTN for MultiHeadAttention in multimodal encoder.
(EngineCore_DP0 pid=2040645) INFO 01-15 04:55:32 [weight_utils.py:527] No consolidated.safetensors.index.json found in remote.
(EngineCore_DP0 pid=2040645)
Loading safetensors checkpoint shards:   0% Completed | 0/1 [00:00<?, ?it/s]
(EngineCore_DP0 pid=2040645)
Loading safetensors checkpoint shards: 100% Completed | 1/1 [00:03<00:00,  3.71s/it]
(EngineCore_DP0 pid=2040645)
Loading safetensors checkpoint shards: 100% Completed | 1/1 [00:03<00:00,  3.71s/it]
(EngineCore_DP0 pid=2040645)
(EngineCore_DP0 pid=2040645) INFO 01-15 04:55:36 [default_loader.py:308] Loading weights took 4.04 seconds
(EngineCore_DP0 pid=2040645) INFO 01-15 04:55:37 [gpu_model_runner.py:3659] Model loading took 8.7223 GiB memory and 4.717650 seconds
(EngineCore_DP0 pid=2040645) INFO 01-15 04:55:38 [gpu_model_runner.py:4446] Encoder cache will be initialized with a budget of 8192 tokens, and profiled with 2 audio items of the maximum feature size.
(EngineCore_DP0 pid=2040645) INFO 01-15 04:55:43 [backends.py:643] Using cache directory: /home/ubuntu/.cache/vllm/torch_compile_cache/4b019224b0/rank_0_0/backbone for vLLM's torch.compile
(EngineCore_DP0 pid=2040645) INFO 01-15 04:55:43 [backends.py:703] Dynamo bytecode transform time: 4.77 s
(EngineCore_DP0 pid=2040645) INFO 01-15 04:55:49 [backends.py:226] Directly load the compiled graph(s) for compile range (1, 8192) from the cache, took 3.208 s
(EngineCore_DP0 pid=2040645) INFO 01-15 04:55:49 [monitor.py:34] torch.compile takes 7.98 s in total
(EngineCore_DP0 pid=2040645) INFO 01-15 04:55:50 [gpu_worker.py:375] Available KV cache memory: 61.24 GiB
(EngineCore_DP0 pid=2040645) INFO 01-15 04:55:51 [kv_cache_utils.py:1291] GPU KV cache size: 535,152 tokens
(EngineCore_DP0 pid=2040645) INFO 01-15 04:55:51 [kv_cache_utils.py:1296] Maximum concurrency for 4,096 tokens per request: 130.65x
(EngineCore_DP0 pid=2040645)
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):   0%|          | 0/51 [00:00<?, ?it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):   6%|▌         | 3/51 [00:00<00:01, 25.41it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  12%|█▏        | 6/51 [00:00<00:01, 25.89it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  20%|█▉        | 10/51 [00:00<00:01, 28.14it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  27%|██▋       | 14/51 [00:00<00:01, 29.68it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  35%|███▌      | 18/51 [00:00<00:01, 30.80it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  43%|████▎     | 22/51 [00:00<00:00, 30.82it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  51%|█████     | 26/51 [00:00<00:00, 31.80it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  59%|█████▉    | 30/51 [00:00<00:00, 32.43it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  67%|██████▋   | 34/51 [00:01<00:00, 32.67it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  75%|███████▍  | 38/51 [00:01<00:00, 32.58it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  82%|████████▏ | 42/51 [00:01<00:00, 32.78it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  90%|█████████ | 46/51 [00:01<00:00, 32.95it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  98%|█████████▊| 50/51 [00:01<00:00, 31.64it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 100%|██████████| 51/51 [00:01<00:00, 31.07it/s]
(EngineCore_DP0 pid=2040645)
Capturing CUDA graphs (decode, FULL):   0%|          | 0/35 [00:00<?, ?it/s]
Capturing CUDA graphs (decode, FULL):   9%|▊         | 3/35 [00:00<00:01, 27.01it/s]
Capturing CUDA graphs (decode, FULL):  20%|██        | 7/35 [00:00<00:00, 30.21it/s]
Capturing CUDA graphs (decode, FULL):  31%|███▏      | 11/35 [00:00<00:00, 31.21it/s]
Capturing CUDA graphs (decode, FULL):  43%|████▎     | 15/35 [00:00<00:00, 31.66it/s]
Capturing CUDA graphs (decode, FULL):  54%|█████▍    | 19/35 [00:00<00:00, 31.57it/s]
Capturing CUDA graphs (decode, FULL):  66%|██████▌   | 23/35 [00:00<00:00, 31.38it/s]
Capturing CUDA graphs (decode, FULL):  77%|███████▋  | 27/35 [00:00<00:00, 31.60it/s]
Capturing CUDA graphs (decode, FULL):  89%|████████▊ | 31/35 [00:00<00:00, 31.89it/s]
Capturing CUDA graphs (decode, FULL): 100%|██████████| 35/35 [00:01<00:00, 31.84it/s]
Capturing CUDA graphs (decode, FULL): 100%|██████████| 35/35 [00:01<00:00, 31.45it/s]
(EngineCore_DP0 pid=2040645) INFO 01-15 04:55:54 [gpu_model_runner.py:4587] Graph capturing finished in 4 secs, took 0.54 GiB
(EngineCore_DP0 pid=2040645) INFO 01-15 04:55:54 [core.py:259] init engine (profile, create kv cache, warmup model) took 17.18 seconds
INFO 01-15 04:55:55 [llm.py:360] Supported tasks: ['generate', 'transcription']

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:01<00:00,  1.17s/it]
Adding requests: 100%|██████████| 1/1 [00:01<00:00,  1.17s/it]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  3.93it/s, est. speed input: 2464.81 toks/s, output: 106.12 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  3.93it/s, est. speed input: 2464.81 toks/s, output: 106.12 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  3.92it/s, est. speed input: 2464.81 toks/s, output: 106.12 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 257.19it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.03it/s, est. speed input: 2527.36 toks/s, output: 120.91 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.03it/s, est. speed input: 2527.36 toks/s, output: 120.91 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.03it/s, est. speed input: 2527.36 toks/s, output: 120.91 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 284.36it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  3.63it/s, est. speed input: 2276.06 toks/s, output: 123.41 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  3.63it/s, est. speed input: 2276.06 toks/s, output: 123.41 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  3.63it/s, est. speed input: 2276.06 toks/s, output: 123.41 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 324.16it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  3.70it/s, est. speed input: 2322.70 toks/s, output: 122.24 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  3.70it/s, est. speed input: 2322.70 toks/s, output: 122.24 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  3.70it/s, est. speed input: 2322.70 toks/s, output: 122.24 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 72.88it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.90it/s, est. speed input: 3074.23 toks/s, output: 112.76 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.90it/s, est. speed input: 3074.23 toks/s, output: 112.76 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.90it/s, est. speed input: 3074.23 toks/s, output: 112.76 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 63.40it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.15it/s, est. speed input: 2605.00 toks/s, output: 120.47 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.15it/s, est. speed input: 2605.00 toks/s, output: 120.47 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.15it/s, est. speed input: 2605.00 toks/s, output: 120.47 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 70.02it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  3.68it/s, est. speed input: 2304.82 toks/s, output: 121.29 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  3.68it/s, est. speed input: 2304.82 toks/s, output: 121.29 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  3.67it/s, est. speed input: 2304.82 toks/s, output: 121.29 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 283.76it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00, 15.05it/s, est. speed input: 9444.43 toks/s, output: 60.24 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00, 15.03it/…37176 tokens truncated…_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 210.15it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00, 14.70it/s, est. speed input: 9224.91 toks/s, output: 58.82 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00, 14.64it/s, est. speed input: 9224.91 toks/s, output: 58.82 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 305.73it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00, 14.35it/s, est. speed input: 9004.66 toks/s, output: 57.43 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00, 14.31it/s, est. speed input: 9004.66 toks/s, output: 57.43 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 67.16it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00, 14.22it/s, est. speed input: 8921.81 toks/s, output: 56.91 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00, 14.20it/s, est. speed input: 8921.81 toks/s, output: 56.91 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 62.73it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.32it/s, est. speed input: 2710.28 toks/s, output: 121.02 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.32it/s, est. speed input: 2710.28 toks/s, output: 121.02 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.32it/s, est. speed input: 2710.28 toks/s, output: 121.02 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 67.73it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  3.58it/s, est. speed input: 2242.65 toks/s, output: 125.17 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  3.58it/s, est. speed input: 2242.65 toks/s, output: 125.17 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  3.57it/s, est. speed input: 2242.65 toks/s, output: 125.17 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 323.16it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.66it/s, est. speed input: 3549.24 toks/s, output: 113.20 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.66it/s, est. speed input: 3549.24 toks/s, output: 113.20 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.65it/s, est. speed input: 3549.24 toks/s, output: 113.20 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 291.74it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00, 14.31it/s, est. speed input: 8978.44 toks/s, output: 57.26 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00, 14.27it/s, est. speed input: 8978.44 toks/s, output: 57.26 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 211.73it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.55it/s, est. speed input: 2855.87 toks/s, output: 118.41 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.55it/s, est. speed input: 2855.87 toks/s, output: 118.41 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.55it/s, est. speed input: 2855.87 toks/s, output: 118.41 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 303.56it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.85it/s, est. speed input: 3041.96 toks/s, output: 116.43 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.85it/s, est. speed input: 3041.96 toks/s, output: 116.43 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.84it/s, est. speed input: 3041.96 toks/s, output: 116.43 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 299.81it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00, 14.68it/s, est. speed input: 9212.95 toks/s, output: 58.76 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00, 14.64it/s, est. speed input: 9212.95 toks/s, output: 58.76 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 318.43it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  3.92it/s, est. speed input: 2455.59 toks/s, output: 121.40 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  3.92it/s, est. speed input: 2455.59 toks/s, output: 121.40 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  3.91it/s, est. speed input: 2455.59 toks/s, output: 121.40 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 311.13it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.64it/s, est. speed input: 3534.58 toks/s, output: 112.73 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.64it/s, est. speed input: 3534.58 toks/s, output: 112.73 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.63it/s, est. speed input: 3534.58 toks/s, output: 112.73 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 312.26it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.43it/s, est. speed input: 3403.40 toks/s, output: 113.98 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.43it/s, est. speed input: 3403.40 toks/s, output: 113.98 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.42it/s, est. speed input: 3403.40 toks/s, output: 113.98 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 317.63it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00, 14.63it/s, est. speed input: 9181.21 toks/s, output: 58.55 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00, 14.59it/s, est. speed input: 9181.21 toks/s, output: 58.55 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 313.94it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00, 14.01it/s, est. speed input: 8792.91 toks/s, output: 56.08 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00, 13.98it/s, est. speed input: 8792.91 toks/s, output: 56.08 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 310.57it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00, 14.85it/s, est. speed input: 9315.86 toks/s, output: 59.42 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00, 14.83it/s, est. speed input: 9315.86 toks/s, output: 59.42 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 241.02it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00, 14.80it/s, est. speed input: 9289.23 toks/s, output: 59.24 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00, 14.76it/s, est. speed input: 9289.23 toks/s, output: 59.24 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 253.82it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00, 14.28it/s, est. speed input: 8961.42 toks/s, output: 57.16 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00, 14.25it/s, est. speed input: 8961.42 toks/s, output: 57.16 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 298.85it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  2.28it/s, est. speed input: 1427.59 toks/s, output: 132.05 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  2.28it/s, est. speed input: 1427.59 toks/s, output: 132.05 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  2.28it/s, est. speed input: 1427.59 toks/s, output: 132.05 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 324.94it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00, 14.44it/s, est. speed input: 9061.91 toks/s, output: 57.79 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00, 14.40it/s, est. speed input: 9061.91 toks/s, output: 57.79 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 304.84it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00, 14.44it/s, est. speed input: 9060.84 toks/s, output: 57.79 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00, 14.40it/s, est. speed input: 9060.84 toks/s, output: 57.79 toks/s]
Transcribing ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:01:50
[rank0]:[W115 04:57:45.281780100 ProcessGroupNCCL.cpp:1524] Warning: WARNING: destroy_process_group() was not called before program exit, which can leak resources. For more info, please see https://pytorch.org/docs/stable/distributed.html#shutdown (function operator())
Wrote: outputs/bench/maya_hf_hin_v2/voxtral_mini_vllm_prompt/results.jsonl
Wrote: outputs/bench/maya_hf_hin_v2/voxtral_mini_vllm_prompt/summary.json
Wrote: outputs/bench/maya_hf_hin_v2/voxtral_mini_vllm_prompt/eval.json
{
  "hypothesis": "final",
  "pairs_evaluated": 200,
  "wer_mean": 0.9848948273948275,
  "cer_mean": 0.9987700421526944,
  "wer_p50": 1.0,
  "wer_p90": 1.0,
  "wer_p95": 2.1538461538461537,
  "cer_p50": 1.0,
  "cer_p90": 1.3333333333333333,
  "cer_p95": 2.406779661016949,
  "segment_error_rate": 0.975,
  "pct_wer_gt_5pct": 0.975,
  "pct_wer_gt_10pct": 0.945,
  "wer_by_language_mean": {
    "hin": 0.9848948273948275
  },
  "wer_by_decision_mean": {
    "accept": 1.0673425989215464,
    "human": 0.9520310303527084
  },
  "wer_by_duration_bucket_mean": {
    "0-3s": 1.0,
    "10-30s": 1.0982142857142858,
    "3-10s": 0.9797513361343151
  },
  "cer_by_language_mean": {
    "hin": 0.9987700421526944
  },
  "cer_by_decision_mean": {
    "accept": 1.1124357821321484,
    "human": 0.953462719223821
  },
  "cer_by_duration_bucket_mean": {
    "0-3s": 1.0,
    "10-30s": 0.9977661950856291,
    "3-10s": 0.9987865897332651
  },
  "oracle": {
    "enabled": true,
    "oracle_best_wer_mean": 0.9848948273948275,
    "selection_gap_wer_mean": 0.0,
    "selection_gap_wer_p90": 0.0
  }
}


Summary: auto


**Observing extremely poor transcription on Voxtral mini**

The Voxtral mini shows a word error rate near 98%, likely outputting mostly unknown tokens; I'll switch to testing Gemma with the prompt next.


python3 -m maya_transcribe.cli bench run --bench-dir data/bench/maya_hf_hin_v2 --out outputs/bench/maya_hf_hin_v2/gemma3n_e2b_vllm_prompt --skip-whisper --vllm-asr-model google/gemma-3n-E2B-it --vllm-gpu-memory-utilization 0.9 --vllm-max-model-len 4096 --vllm-transcribe-prompt-file Prompt.md