Output:
Exit code: 124
Wall time: 120.3 seconds
Total output lines: 370
Output:
command timed out after 120309 milliseconds
/home/ubuntu/.local/lib/python3.10/site-packages/torch/cuda/__init__.py:63: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you.
  import pynvml  # type: ignore[import]
2026-01-14 23:19:51.280789: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
/home/ubuntu/.local/lib/python3.10/site-packages/google/api_core/_python_version_support.py:266: FutureWarning: You are using a Python version (3.10.12) which Google will stop supporting in new releases of google.api_core once it reaches its end of life (2026-10-04). Please upgrade to the latest Python version, or at least Python 3.11, to continue receiving updates for google.api_core past that date.
  warnings.warn(message, FutureWarning)
INFO 01-14 23:19:57 [utils.py:253] non-default args: {'trust_remote_code': True, 'max_model_len': 4096, 'gpu_memory_utilization': 0.15, 'disable_log_stats': True, 'model': 'mistralai/Voxtral-Mini-3B-2507'}
The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
INFO 01-14 23:20:10 [model.py:514] Resolved architecture: VoxtralForConditionalGeneration
INFO 01-14 23:20:10 [model.py:1661] Using max model len 4096
INFO 01-14 23:20:10 [scheduler.py:230] Chunked prefill is enabled with max_num_batched_tokens=8192.
[2026-01-14 23:20:10] INFO tekken.py:195: Non special vocabulary size is 130072 with 1000 special tokens.
[2026-01-14 23:20:10] INFO tekken.py:567: Cutting non special vocabulary to first 130072 tokens.
WARNING 01-14 23:20:11 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: CUDA is initialized
/home/ubuntu/.local/lib/python3.10/site-packages/torch/cuda/__init__.py:63: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you.
  import pynvml  # type: ignore[import]
2026-01-14 23:20:16.770346: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
/home/ubuntu/.local/lib/python3.10/site-packages/google/api_core/_python_version_support.py:266: FutureWarning: You are using a Python version (3.10.12) which Google will stop supporting in new releases of google.api_core once it reaches its end of life (2026-10-04). Please upgrade to the latest Python version, or at least Python 3.11, to continue receiving updates for google.api_core past that date.
  warnings.warn(message, FutureWarning)
(EngineCore_DP0 pid=1837513) INFO 01-14 23:20:20 [core.py:93] Initializing a V1 LLM engine (v0.13.0) with config: model='mistralai/Voxtral-Mini-3B-2507', speculative_config=None, tokenizer='mistralai/Voxtral-Mini-3B-2507', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, tokenizer_revision=None, trust_remote_code=True, dtype=torch.bfloat16, max_seq_len=4096, download_dir=None, load_format=auto, tensor_parallel_size=1, pipeline_parallel_size=1, data_parallel_size=1, disable_custom_all_reduce=False, quantization=None, enforce_eager=False, kv_cache_dtype=auto, device_config=cuda, structured_outputs_config=StructuredOutputsConfig(backend='auto', disable_fallback=False, disable_any_whitespace=False, disable_additional_properties=False, reasoning_parser='', reasoning_parser_plugin='', enable_in_reasoning=False), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None, kv_cache_metrics=False, kv_cache_metrics_sample=0.01, cudagraph_metrics=False, enable_layerwise_nvtx_tracing=False), seed=0, served_model_name=mistralai/Voxtral-Mini-3B-2507, enable_prefix_caching=True, enable_chunked_prefill=True, pooler_config=None, compilation_config={'level': None, 'mode': <CompilationMode.VLLM_COMPILE: 3>, 'debug_dump_path': None, 'cache_dir': '', 'compile_cache_save_format': 'binary', 'backend': 'inductor', 'custom_ops': ['none'], 'splitting_ops': ['vllm::unified_attention', 'vllm::unified_attention_with_output', 'vllm::unified_mla_attention', 'vllm::unified_mla_attention_with_output', 'vllm::mamba_mixer2', 'vllm::mamba_mixer', 'vllm::short_conv', 'vllm::linear_attention', 'vllm::plamo2_mamba_mixer', 'vllm::gdn_attention_core', 'vllm::kda_attention', 'vllm::sparse_attn_indexer'], 'compile_mm_encoder': False, 'compile_sizes': [], 'compile_ranges_split_points': [8192], 'inductor_compile_config': {'enable_auto_functionalized_v2': False, 'combo_kernels': True, 'benchmark_combo_kernel': True}, 'inductor_passes': {}, 'cudagraph_mode': <CUDAGraphMode.FULL_AND_PIECEWISE: (2, 1)>, 'cudagraph_num_of_warmups': 1, 'cudagraph_capture_sizes': [1, 2, 4, 8, 16, 24, 32, 40, 48, 56, 64, 72, 80, 88, 96, 104, 112, 120, 128, 136, 144, 152, 160, 168, 176, 184, 192, 200, 208, 216, 224, 232, 240, 248, 256, 272, 288, 304, 320, 336, 352, 368, 384, 400, 416, 432, 448, 464, 480, 496, 512], 'cudagraph_copy_inputs': False, 'cudagraph_specialize_lora': True, 'use_inductor_graph_partition': False, 'pass_config': {'fuse_norm_quant': False, 'fuse_act_quant': False, 'fuse_attn_quant': False, 'eliminate_noops': True, 'enable_sp': False, 'fuse_gemm_comms': False, 'fuse_allreduce_rms': False}, 'max_cudagraph_capture_size': 512, 'dynamic_shapes_config': {'type': <DynamicShapesType.BACKED: 'backed'>, 'evaluate_guards': False}, 'local_cache_dir': None}
(EngineCore_DP0 pid=1837513) [2026-01-14 23:20:21] INFO tekken.py:195: Non special vocabulary size is 130072 with 1000 special tokens.
(EngineCore_DP0 pid=1837513) [2026-01-14 23:20:21] INFO tekken.py:567: Cutting non special vocabulary to first 130072 tokens.
(EngineCore_DP0 pid=1837513) INFO 01-14 23:20:23 [parallel_state.py:1203] world_size=1 rank=0 local_rank=0 distributed_init_method=tcp://216.81.248.184:49831 backend=nccl
(EngineCore_DP0 pid=1837513) INFO 01-14 23:20:23 [parallel_state.py:1411] rank 0 in world size 1 is assigned as DP rank 0, PP rank 0, PCP rank 0, TP rank 0, EP rank 0
(EngineCore_DP0 pid=1837513) INFO 01-14 23:20:24 [gpu_model_runner.py:3562] Starting to load model mistralai/Voxtral-Mini-3B-2507...
(EngineCore_DP0 pid=1837513) INFO 01-14 23:20:25 [cuda.py:351] Using FLASH_ATTN attention backend out of potential backends: ('FLASH_ATTN', 'FLASHINFER', 'TRITON_ATTN', 'FLEX_ATTENTION')
(EngineCore_DP0 pid=1837513) INFO 01-14 23:20:25 [layer.py:477] Using AttentionBackendEnum.FLASH_ATTN for MultiHeadAttention in multimodal encoder.
(EngineCore_DP0 pid=1837513) INFO 01-14 23:20:25 [weight_utils.py:527] No consolidated.safetensors.index.json found in remote.
(EngineCore_DP0 pid=1837513)
Loading safetensors checkpoint shards:   0% Completed | 0/1 [00:00<?, ?it/s]
(EngineCore_DP0 pid=1837513)
Loading safetensors checkpoint shards: 100% Completed | 1/1 [00:02<00:00,  2.00s/it]
(EngineCore_DP0 pid=1837513)
Loading safetensors checkpoint shards: 100% Completed | 1/1 [00:02<00:00,  2.00s/it]
(EngineCore_DP0 pid=1837513)
(EngineCore_DP0 pid=1837513) INFO 01-14 23:20:27 [default_loader.py:308] Loading weights took 2.35 seconds
(EngineCore_DP0 pid=1837513) INFO 01-14 23:20:29 [gpu_model_runner.py:3659] Model loading took 8.7223 GiB memory and 3.108867 seconds
(EngineCore_DP0 pid=1837513) INFO 01-14 23:20:29 [gpu_model_runner.py:4446] Encoder cache will be initialized with a budget of 8192 tokens, and profiled with 2 audio items of the maximum feature size.
(EngineCore_DP0 pid=1837513) INFO 01-14 23:20:35 [backends.py:643] Using cache directory: /home/ubuntu/.cache/vllm/torch_compile_cache/4b019224b0/rank_0_0/backbone for vLLM's torch.compile
(EngineCore_DP0 pid=1837513) INFO 01-14 23:20:35 [backends.py:703] Dynamo bytecode transform time: 5.51 s
(EngineCore_DP0 pid=1837513) INFO 01-14 23:20:42 [backends.py:261] Cache the graph of compile range (1, 8192) for later use
(EngineCore_DP0 pid=1837513) INFO 01-14 23:20:49 [backends.py:278] Compiling a graph for compile range (1, 8192) takes 10.95 s
(EngineCore_DP0 pid=1837513) INFO 01-14 23:20:49 [monitor.py:34] torch.compile takes 16.46 s in total
(EngineCore_DP0 pid=1837513) INFO 01-14 23:20:50 [gpu_worker.py:375] Available KV cache memory: 1.80 GiB
(EngineCore_DP0 pid=1837513) INFO 01-14 23:20:51 [kv_cache_utils.py:1291] GPU KV cache size: 15,760 tokens
(EngineCore_DP0 pid=1837513) INFO 01-14 23:20:51 [kv_cache_utils.py:1296] Maximum concurrency for 4,096 tokens per request: 3.85x
(EngineCore_DP0 pid=1837513)
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):   0%|          | 0/51 [00:00<?, ?it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):   6%|▌         | 3/51 [00:00<00:01, 25.15it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  12%|█▏        | 6/51 [00:00<00:01, 25.93it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  18%|█▊        | 9/51 [00:00<00:01, 26.74it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  25%|██▌       | 13/51 [00:00<00:01, 28.44it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  33%|███▎      | 17/51 [00:00<00:01, 29.64it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  41%|████      | 21/51 [00:00<00:01, 29.95it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  49%|████▉     | 25/51 [00:00<00:00, 30.53it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  57%|█████▋    | 29/51 [00:00<00:00, 31.45it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  65%|██████▍   | 33/51 [00:01<00:00, 31.93it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  73%|███████▎  | 37/51 [00:01<00:00, 32.00it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  80%|████████  | 41/51 [00:01<00:00, 32.02it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  88%|████████▊ | 45/51 [00:01<00:00, 32.28it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  96%|█████████▌| 49/51 [00:01<00:00, 30.39it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 100%|██████████| 51/51 [00:01<00:00, 30.11it/s]
(EngineCore_DP0 pid=1837513)
Capturing CUDA graphs (decode, FULL):   0%|          | 0/35 [00:00<?, ?it/s]
Capturing CUDA graphs (decode, FULL):   9%|▊         | 3/35 [00:00<00:01, 23.16it/s]
Capturing CUDA graphs (decode, FULL):  20%|██        | 7/35 [00:00<00:01, 27.84it/s]
Capturing CUDA graphs (decode, FULL):  31%|███▏      | 11/35 [00:00<00:00, 29.24it/s]
Capturing CUDA graphs (decode, FULL):  43%|████▎     | 15/35 [00:00<00:00, 30.13it/s]
Capturing CUDA graphs (decode, FULL):  54%|█████▍    | 19/35 [00:00<00:00, 30.27it/s]
Capturing CUDA graphs (decode, FULL):  66%|██████▌   | 23/35 [00:00<00:00, 30.21it/s]
Capturing CUDA graphs (decode, FULL):  77%|███████▋  | 27/35 [00:00<00:00, 30.43it/s]
Capturing CUDA graphs (decode, FULL):  89%|████████▊ | 31/35 [00:01<00:00, 30.74it/s]
Capturing CUDA graphs (decode, FULL): 100%|██████████| 35/35 [00:01<00:00, 30.78it/s]
Capturing CUDA graphs (decode, FULL): 100%|██████████| 35/35 [00:01<00:00, 30.04it/s]
(EngineCore_DP0 pid=1837513) INFO 01-14 23:20:55 [gpu_model_runner.py:4587] Graph capturing finished in 4 secs, took 0.54 GiB
(EngineCore_DP0 pid=1837513) INFO 01-14 23:20:55 [core.py:259] init engine (profile, create kv cache, warmup model) took 26.48 seconds
INFO 01-14 23:20:56 [llm.py:360] Supported tasks: ['generate', 'transcription']

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:01<00:00,  1.20s/it]
Adding requests: 100%|██████████| 1/1 [00:01<00:00,  1.20s/it]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.01it/s, est. speed input: 1917.24 toks/s, output: 90.10 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.01it/s, est. speed input: 1917.24 toks/s, output: 90.10 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.00it/s, est. speed input: 1917.24 toks/s, output: 90.10 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 75.63it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.06it/s, est. speed input: 1554.94 toks/s, output: 121.79 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.06it/s, est. speed input: 1554.94 toks/s, output: 121.79 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.06it/s, est. speed input: 1554.94 toks/s, output: 121.79 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 52.10it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  3.45it/s, est. speed input: 1322.36 toks/s, output: 124.29 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  3.45it/s, est. speed input: 1322.36 toks/s, output: 124.29 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  3.45it/s, est. speed input: 1322.36 toks/s, output: 124.29 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 82.96it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  3.89it/s, est. speed input: 1489.65 toks/s, output: 120.56 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  3.89it/s, est. speed input: 1489.65 toks/s, output: 120.56 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  3.88it/s, est. speed input: 1489.65 toks/s, output: 120.56 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 75.48it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  6.24it/s, est. speed input: 2389.21 toks/s, output: 106.04 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  6.24it/s, est. speed input: 2389.21 toks/s, output: 106.04 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  6.23it/s, est. speed input: 2389.21 toks/s, output: 106.04 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 70.39it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.16it/s, est. speed input: 1976.34 toks/s, output: 113.52 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.16it/s, est. speed input: 1976.34 toks/s, output: 113.52 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.15it/s, est. speed input: 1976.34 toks/s, output: 113.52 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 51.05it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.86it/s, est. speed input: 1860.94 toks/s, output: 116.60 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.86it/s, est. speed input: 1860.94 toks/s, output: 116.60 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.85it/s, est. speed input: 1860.94 toks/s, output: 116.60 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 72.54it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/…12211 tokens truncated…ed input: 1810.59 toks/s, output: 118.17 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.73it/s, est. speed input: 1810.59 toks/s, output: 118.17 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.72it/s, est. speed input: 1810.59 toks/s, output: 118.17 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 70.61it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.63it/s, est. speed input: 1772.45 toks/s, output: 120.31 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.63it/s, est. speed input: 1772.45 toks/s, output: 120.31 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.62it/s, est. speed input: 1772.45 toks/s, output: 120.31 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 78.12it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.62it/s, est. speed input: 2153.00 toks/s, output: 112.42 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.62it/s, est. speed input: 2153.00 toks/s, output: 112.42 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.61it/s, est. speed input: 2153.00 toks/s, output: 112.42 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 64.73it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.05it/s, est. speed input: 1933.87 toks/s, output: 116.12 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.05it/s, est. speed input: 1933.87 toks/s, output: 116.12 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.04it/s, est. speed input: 1933.87 toks/s, output: 116.12 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 79.73it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.07it/s, est. speed input: 1943.85 toks/s, output: 116.72 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.07it/s, est. speed input: 1943.85 toks/s, output: 116.72 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.07it/s, est. speed input: 1943.85 toks/s, output: 116.72 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 88.85it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.14it/s, est. speed input: 1969.09 toks/s, output: 113.10 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.14it/s, est. speed input: 1969.09 toks/s, output: 113.10 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.14it/s, est. speed input: 1969.09 toks/s, output: 113.10 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 74.85it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.23it/s, est. speed input: 2003.13 toks/s, output: 115.05 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.23it/s, est. speed input: 2003.13 toks/s, output: 115.05 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.22it/s, est. speed input: 2003.13 toks/s, output: 115.05 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 71.50it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  6.33it/s, est. speed input: 2425.58 toks/s, output: 107.65 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  6.33it/s, est. speed input: 2425.58 toks/s, output: 107.65 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  6.32it/s, est. speed input: 2425.58 toks/s, output: 107.65 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 76.27it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.00it/s, est. speed input: 1913.39 toks/s, output: 114.90 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.00it/s, est. speed input: 1913.39 toks/s, output: 114.90 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.99it/s, est. speed input: 1913.39 toks/s, output: 114.90 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 82.77it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.36it/s, est. speed input: 2054.31 toks/s, output: 112.63 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.36it/s, est. speed input: 2054.31 toks/s, output: 112.63 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.36it/s, est. speed input: 2054.31 toks/s, output: 112.63 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 56.23it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.37it/s, est. speed input: 2055.62 toks/s, output: 112.70 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.37it/s, est. speed input: 2055.62 toks/s, output: 112.70 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.36it/s, est. speed input: 2055.62 toks/s, output: 112.70 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 48.60it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  7.72it/s, est. speed input: 2956.30 toks/s, output: 100.32 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  7.72it/s, est. speed input: 2956.30 toks/s, output: 100.32 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  7.70it/s, est. speed input: 2956.30 toks/s, output: 100.32 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 77.43it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  6.50it/s, est. speed input: 2490.71 toks/s, output: 110.54 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  6.50it/s, est. speed input: 2490.71 toks/s, output: 110.54 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  6.49it/s, est. speed input: 2490.71 toks/s, output: 110.54 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 76.89it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.63it/s, est. speed input: 1772.76 toks/s, output: 120.34 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.63it/s, est. speed input: 1772.76 toks/s, output: 120.34 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.62it/s, est. speed input: 1772.76 toks/s, output: 120.34 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 73.43it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.42it/s, est. speed input: 1693.63 toks/s, output: 119.39 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.42it/s, est. speed input: 1693.63 toks/s, output: 119.39 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.42it/s, est. speed input: 1693.63 toks/s, output: 119.39 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 68.83it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.02it/s, est. speed input: 1921.70 toks/s, output: 115.39 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.02it/s, est. speed input: 1921.70 toks/s, output: 115.39 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.01it/s, est. speed input: 1921.70 toks/s, output: 115.39 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 68.85it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.36it/s, est. speed input: 2053.71 toks/s, output: 112.60 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.36it/s, est. speed input: 2053.71 toks/s, output: 112.60 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.36it/s, est. speed input: 2053.71 toks/s, output: 112.60 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 60.33it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  6.02it/s, est. speed input: 2307.84 toks/s, output: 108.45 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  6.02it/s, est. speed input: 2307.84 toks/s, output: 108.45 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  6.01it/s, est. speed input: 2307.84 toks/s, output: 108.45 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 69.30it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.72it/s, est. speed input: 2190.80 toks/s, output: 114.39 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.72it/s, est. speed input: 2190.80 toks/s, output: 114.39 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.71it/s, est. speed input: 2190.80 toks/s, output: 114.39 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 70.39it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  7.51it/s, est. speed input: 2876.14 toks/s, output: 97.61 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  7.51it/s, est. speed input: 2876.14 toks/s, output: 97.61 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  7.50it/s, est. speed input: 2876.14 toks/s, output: 97.61 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 63.66it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  6.21it/s, est. speed input: 2379.94 toks/s, output: 111.84 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  6.21it/s, est. speed input: 2379.94 toks/s, output: 111.84 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  6.20it/s, est. speed input: 2379.94 toks/s, output: 111.84 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 88.85it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]


Summary: auto

## Skills
A skill is a set of local instructions to follow that is stored in a `SKILL.md` file. Below is the list of skills that can be used. Each entry includes a name, description, and file path so you can open the source for full instructions when using a specific skill.
### Available skills
- skill-creator: Guide for creating effective skills. This skill should be used when users want to create a new skill (or update an existing skill) that extends Codex's capabilities with specialized knowledge, workflows, or tool integrations. (file: /home/ubuntu/.codex/skills/.system/skill-creator/SKILL.md)
- skill-installer: Install Codex skills into $CODEX_HOME/skills from a curated list or a GitHub repo path. Use when a user asks to list installable skills, install a curated skill, or install a skill from another repo (including private repos). (file: /home/ubuntu/.codex/skills/.system/skill-installer/SKILL.md)
### How to use skills
- Discovery: The list above is the skills available in this session (name + description + file path). Skill bodies live on disk at the listed paths.
- Trigger rules: If the user names a skill (with `$SkillName` or plain text) OR the task clearly matches a skill's description shown above, you must use that skill for that turn. Multiple mentions mean use them all. Do not carry skills across turns unless re-mentioned.
- Missing/blocked: If a named skill isn't in the list or the path can't be read, say so briefly and continue with the best fallback.
- How to use a skill (progressive disclosure):
  1) After deciding to use a skill, open its `SKILL.md`. Read only enough to follow the workflow.
  2) If `SKILL.md` points to extra folders such as `references/`, load only the specific files needed for the request; don't bulk-load everything.
  3) If `scripts/` exist, prefer running or patching them instead of retyping large code blocks.
  4) If `assets/` or templates exist, reuse them instead of recreating from scratch.
- Coordination and sequencing:
  - If multiple skills apply, choose the minimal set that covers the request and state the order you'll use them.
  - Announce which skill(s) you're using and why (one short line). If you skip an obvious skill, say why.
- Context hygiene:
  - Keep context small: summarize long sections instead of pasting them; only load extra files when needed.
  - Avoid deep reference-chasing: prefer opening only files directly linked from `SKILL.md` unless you're blocked.
  - When variants exist (frameworks, providers, domains), pick only the relevant reference file(s) and note that choice.
- Safety and fallback: If a skill can't be applied cleanly (missing files, unclear instructions), state the issue, pick the next-best approach, and continue.


**Adjusting timeout for long transcription**

The 120-second timeout is too short for processing 200+ segments, so I'll increase `timeout_ms` to 1 hour (3600000 ms) to safely cover the workload before rerunning.


**Checking for lingering vLLM worker processes**

Planning to verify if any vLLM worker processes remain running and to check GPU memory usage with nvidia-smi.


Output:
Exit code: 0
Wall time: 0.3 seconds
Output:
Wed Jan 14 23:21:58 2026
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 560.35.05              Driver Version: 560.35.05      CUDA Version: 12.6     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA A100-SXM4-80GB          On  |   00000000:01:00.0 Off |                    0 |
| N/A   32C    P0             58W /  400W |   61611MiB /  81920MiB |      0%      Default |
|                                         |                        |             Disabled |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI        PID   Type   Process name                              GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|    0   N/A  N/A    738802      C   /usr/bin/python3.12                         61602MiB |
+-----------------------------------------------------------------------------------------+


Summary: auto


**Adjusting timeout to 30 minutes**

Decided to increase the timeout to 1.8 million ms (30 minutes) to safely cover processing of 200 segments, balancing between expected run time and overhead before rerunning the vLLM process.


python3 -m maya_transcribe.cli bench run --bench-dir data/bench/maya_hf_hin_v2 --out outputs/bench/maya_hf_hin_v2/voxtral_mini_vllm --skip-whisper --vllm-asr-model mistralai/Voxtral-Mini-3B-2507 --vllm-gpu-memory-utilization 0.15 --vllm-max-model-len 4096


Output:
Exit code: 0
Wall time: 178.1 seconds
Total output lines: 898
Output:
/home/ubuntu/.local/lib/python3.10/site-packages/torch/cuda/__init__.py:63: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you.
  import pynvml  # type: ignore[import]
2026-01-14 23:22:11.107072: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
/home/ubuntu/.local/lib/python3.10/site-packages/google/api_core/_python_version_support.py:266: FutureWarning: You are using a Python version (3.10.12) which Google will stop supporting in new releases of google.api_core once it reaches its end of life (2026-10-04). Please upgrade to the latest Python version, or at least Python 3.11, to continue receiving updates for google.api_core past that date.
  warnings.warn(message, FutureWarning)
INFO 01-14 23:22:17 [utils.py:253] non-default args: {'trust_remote_code': True, 'max_model_len': 4096, 'gpu_memory_utilization': 0.15, 'disable_log_stats': True, 'model': 'mistralai/Voxtral-Mini-3B-2507'}
The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
INFO 01-14 23:22:18 [model.py:514] Resolved architecture: VoxtralForConditionalGeneration
INFO 01-14 23:22:18 [model.py:1661] Using max model len 4096
INFO 01-14 23:22:18 [scheduler.py:230] Chunked prefill is enabled with max_num_batched_tokens=8192.
[2026-01-14 23:22:18] INFO tekken.py:195: Non special vocabulary size is 130072 with 1000 special tokens.
[2026-01-14 23:22:18] INFO tekken.py:567: Cutting non special vocabulary to first 130072 tokens.
WARNING 01-14 23:22:20 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: CUDA is initialized
/home/ubuntu/.local/lib/python3.10/site-packages/torch/cuda/__init__.py:63: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you.
  import pynvml  # type: ignore[import]
2026-01-14 23:22:25.003773: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
/home/ubuntu/.local/lib/python3.10/site-packages/google/api_core/_python_version_support.py:266: FutureWarning: You are using a Python version (3.10.12) which Google will stop supporting in new releases of google.api_core once it reaches its end of life (2026-10-04). Please upgrade to the latest Python version, or at least Python 3.11, to continue receiving updates for google.api_core past that date.
  warnings.warn(message, FutureWarning)
(EngineCore_DP0 pid=1844069) INFO 01-14 23:22:28 [core.py:93] Initializing a V1 LLM engine (v0.13.0) with config: model='mistralai/Voxtral-Mini-3B-2507', speculative_config=None, tokenizer='mistralai/Voxtral-Mini-3B-2507', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, tokenizer_revision=None, trust_remote_code=True, dtype=torch.bfloat16, max_seq_len=4096, download_dir=None, load_format=auto, tensor_parallel_size=1, pipeline_parallel_size=1, data_parallel_size=1, disable_custom_all_reduce=False, quantization=None, enforce_eager=False, kv_cache_dtype=auto, device_config=cuda, structured_outputs_config=StructuredOutputsConfig(backend='auto', disable_fallback=False, disable_any_whitespace=False, disable_additional_properties=False, reasoning_parser='', reasoning_parser_plugin='', enable_in_reasoning=False), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None, kv_cache_metrics=False, kv_cache_metrics_sample=0.01, cudagraph_metrics=False, enable_layerwise_nvtx_tracing=False), seed=0, served_model_name=mistralai/Voxtral-Mini-3B-2507, enable_prefix_caching=True, enable_chunked_prefill=True, pooler_config=None, compilation_config={'level': None, 'mode': <CompilationMode.VLLM_COMPILE: 3>, 'debug_dump_path': None, 'cache_dir': '', 'compile_cache_save_format': 'binary', 'backend': 'inductor', 'custom_ops': ['none'], 'splitting_ops': ['vllm::unified_attention', 'vllm::unified_attention_with_output', 'vllm::unified_mla_attention', 'vllm::unified_mla_attention_with_output', 'vllm::mamba_mixer2', 'vllm::mamba_mixer', 'vllm::short_conv', 'vllm::linear_attention', 'vllm::plamo2_mamba_mixer', 'vllm::gdn_attention_core', 'vllm::kda_attention', 'vllm::sparse_attn_indexer'], 'compile_mm_encoder': False, 'compile_sizes': [], 'compile_ranges_split_points': [8192], 'inductor_compile_config': {'enable_auto_functionalized_v2': False, 'combo_kernels': True, 'benchmark_combo_kernel': True}, 'inductor_passes': {}, 'cudagraph_mode': <CUDAGraphMode.FULL_AND_PIECEWISE: (2, 1)>, 'cudagraph_num_of_warmups': 1, 'cudagraph_capture_sizes': [1, 2, 4, 8, 16, 24, 32, 40, 48, 56, 64, 72, 80, 88, 96, 104, 112, 120, 128, 136, 144, 152, 160, 168, 176, 184, 192, 200, 208, 216, 224, 232, 240, 248, 256, 272, 288, 304, 320, 336, 352, 368, 384, 400, 416, 432, 448, 464, 480, 496, 512], 'cudagraph_copy_inputs': False, 'cudagraph_specialize_lora': True, 'use_inductor_graph_partition': False, 'pass_config': {'fuse_norm_quant': False, 'fuse_act_quant': False, 'fuse_attn_quant': False, 'eliminate_noops': True, 'enable_sp': False, 'fuse_gemm_comms': False, 'fuse_allreduce_rms': False}, 'max_cudagraph_capture_size': 512, 'dynamic_shapes_config': {'type': <DynamicShapesType.BACKED: 'backed'>, 'evaluate_guards': False}, 'local_cache_dir': None}
(EngineCore_DP0 pid=1844069) [2026-01-14 23:22:29] INFO tekken.py:195: Non special vocabulary size is 130072 with 1000 special tokens.
(EngineCore_DP0 pid=1844069) [2026-01-14 23:22:29] INFO tekken.py:567: Cutting non special vocabulary to first 130072 tokens.
(EngineCore_DP0 pid=1844069) INFO 01-14 23:22:31 [parallel_state.py:1203] world_size=1 rank=0 local_rank=0 distributed_init_method=tcp://216.81.248.184:59725 backend=nccl
(EngineCore_DP0 pid=1844069) INFO 01-14 23:22:31 [parallel_state.py:1411] rank 0 in world size 1 is assigned as DP rank 0, PP rank 0, PCP rank 0, TP rank 0, EP rank 0
(EngineCore_DP0 pid=1844069) INFO 01-14 23:22:32 [gpu_model_runner.py:3562] Starting to load model mistralai/Voxtral-Mini-3B-2507...
(EngineCore_DP0 pid=1844069) INFO 01-14 23:22:32 [cuda.py:351] Using FLASH_ATTN attention backend out of potential backends: ('FLASH_ATTN', 'FLASHINFER', 'TRITON_ATTN', 'FLEX_ATTENTION')
(EngineCore_DP0 pid=1844069) INFO 01-14 23:22:32 [layer.py:477] Using AttentionBackendEnum.FLASH_ATTN for MultiHeadAttention in multimodal encoder.
(EngineCore_DP0 pid=1844069) INFO 01-14 23:22:33 [weight_utils.py:527] No consolidated.safetensors.index.json found in remote.
(EngineCore_DP0 pid=1844069)
Loading safetensors checkpoint shards:   0% Completed | 0/1 [00:00<?, ?it/s]
(EngineCore_DP0 pid=1844069)
Loading safetensors checkpoint shards: 100% Completed | 1/1 [00:01<00:00,  1.97s/it]
(EngineCore_DP0 pid=1844069)
Loading safetensors checkpoint shards: 100% Completed | 1/1 [00:01<00:00,  1.97s/it]
(EngineCore_DP0 pid=1844069)
(EngineCore_DP0 pid=1844069) INFO 01-14 23:22:35 [default_loader.py:308] Loading weights took 2.27 seconds
(EngineCore_DP0 pid=1844069) INFO 01-14 23:22:36 [gpu_model_runner.py:3659] Model loading took 8.7223 GiB memory and 2.941973 seconds
(EngineCore_DP0 pid=1844069) INFO 01-14 23:22:36 [gpu_model_runner.py:4446] Encoder cache will be initialized with a budget of 8192 tokens, and profiled with 2 audio items of the maximum feature size.
(EngineCore_DP0 pid=1844069) INFO 01-14 23:22:42 [backends.py:643] Using cache directory: /home/ubuntu/.cache/vllm/torch_compile_cache/4b019224b0/rank_0_0/backbone for vLLM's torch.compile
(EngineCore_DP0 pid=1844069) INFO 01-14 23:22:42 [backends.py:703] Dynamo bytecode transform time: 4.75 s
(EngineCore_DP0 pid=1844069) INFO 01-14 23:22:48 [backends.py:226] Directly load the compiled graph(s) for compile range (1, 8192) from the cache, took 3.095 s
(EngineCore_DP0 pid=1844069) INFO 01-14 23:22:48 [monitor.py:34] torch.compile takes 7.85 s in total
(EngineCore_DP0 pid=1844069) INFO 01-14 23:22:49 [gpu_worker.py:375] Available KV cache memory: 1.80 GiB
(EngineCore_DP0 pid=1844069) INFO 01-14 23:22:49 [kv_cache_utils.py:1291] GPU KV cache size: 15,760 tokens
(EngineCore_DP0 pid=1844069) INFO 01-14 23:22:49 [kv_cache_utils.py:1296] Maximum concurrency for 4,096 tokens per request: 3.85x
(EngineCore_DP0 pid=1844069)
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):   0%|          | 0/51 [00:00<?, ?it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):   6%|▌         | 3/51 [00:00<00:01, 25.28it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  12%|█▏        | 6/51 [00:00<00:01, 26.03it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  18%|█▊        | 9/51 [00:00<00:01, 26.96it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  25%|██▌       | 13/51 [00:00<00:01, 28.85it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  33%|███▎      | 17/51 [00:00<00:01, 30.17it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  41%|████      | 21/51 [00:00<00:00, 30.52it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  49%|████▉     | 25/51 [00:00<00:00, 31.39it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  57%|█████▋    | 29/51 [00:00<00:00, 32.20it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  65%|██████▍   | 33/51 [00:01<00:00, 32.56it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  73%|███████▎  | 37/51 [00:01<00:00, 32.61it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  80%|████████  | 41/51 [00:01<00:00, 32.68it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  88%|████████▊ | 45/51 [00:01<00:00, 32.83it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  96%|█████████▌| 49/51 [00:01<00:00, 31.42it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 100%|██████████| 51/51 [00:01<00:00, 30.83it/s]
(EngineCore_DP0 pid=1844069)
Capturing CUDA graphs (decode, FULL):   0%|          | 0/35 [00:00<?, ?it/s]
Capturing CUDA graphs (decode, FULL):   9%|▊         | 3/35 [00:00<00:01, 26.60it/s]
Capturing CUDA graphs (decode, FULL):  20%|██        | 7/35 [00:00<00:00, 29.70it/s]
Capturing CUDA graphs (decode, FULL):  31%|███▏      | 11/35 [00:00<00:00, 30.63it/s]
Capturing CUDA graphs (decode, FULL):  43%|████▎     | 15/35 [00:00<00:00, 31.01it/s]
Capturing CUDA graphs (decode, FULL):  54%|█████▍    | 19/35 [00:00<00:00, 30.85it/s]
Capturing CUDA graphs (decode, FULL):  66%|██████▌   | 23/35 [00:00<00:00, 30.63it/s]
Capturing CUDA graphs (decode, FULL):  77%|███████▋  | 27/35 [00:00<00:00, 30.83it/s]
Capturing CUDA graphs (decode, FULL):  89%|████████▊ | 31/35 [00:01<00:00, 31.06it/s]
Capturing CUDA graphs (decode, FULL): 100%|██████████| 35/35 [00:01<00:00, 31.04it/s]
Capturing CUDA graphs (decode, FULL): 100%|██████████| 35/35 [00:01<00:00, 30.72it/s]
(EngineCore_DP0 pid=1844069) INFO 01-14 23:22:53 [gpu_model_runner.py:4587] Graph capturing finished in 4 secs, took 0.54 GiB
(EngineCore_DP0 pid=1844069) INFO 01-14 23:22:53 [core.py:259] init engine (profile, create kv cache, warmup model) took 17.22 seconds
INFO 01-14 23:22:54 [llm.py:360] Supported tasks: ['generate', 'transcription']

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:01<00:00,  1.36s/it]
Adding requests: 100%|██████████| 1/1 [00:01<00:00,  1.36s/it]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.32it/s, est. speed input: 2035.98 toks/s, output: 95.68 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.32it/s, est. speed input: 2035.98 toks/s, output: 95.68 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.31it/s, est. speed input: 2035.98 toks/s, output: 95.68 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 64.04it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  3.99it/s, est. speed input: 1526.83 toks/s, output: 119.58 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  3.99it/s, est. speed input: 1526.83 toks/s, output: 119.58 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  3.98it/s, est. speed input: 1526.83 toks/s, output: 119.58 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 43.14it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  3.50it/s, est. speed input: 1340.73 toks/s, output: 126.01 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  3.50it/s, est. speed input: 1340.73 toks/s, output: 126.01 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  3.50it/s, est. speed input: 1340.73 toks/s, output: 126.01 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 74.30it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  3.91it/s, est. speed input: 1497.46 toks/s, output: 121.19 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  3.91it/s, est. speed input: 1497.46 toks/s, output: 121.19 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  3.90it/s, est. speed input: 1497.46 toks/s, output: 121.19 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 44.62it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  6.42it/s, est. speed input: 2460.30 toks/s, output: 109.18 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  6.42it/s, est. speed input: 2460.30 toks/s, output: 109.18 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  6.41it/s, est. speed input: 2460.30 toks/s, output: 109.18 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 72.06it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.23it/s, est. speed input: 2003.52 toks/s, output: 115.07 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.23it/s, est. speed input: 2003.52 toks/s, output: 115.07 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.22it/s, est. speed input: 2003.52 toks/s, output: 115.07 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 66.98it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.70it/s, est. speed input: 1801.35 toks/s, output: 112.85 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.70it/s, est. speed input: 1801.35 toks/s, output: 112.85 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.69it/s, est. speed input: 1801.35 toks/s, output: 112.85 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 70.73it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  3.91it/s, est. speed input: 1496.03 toks/s, output: 121.08 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:0…41432 tokens truncated…t. speed input: 2102.25 toks/s, output: 115.25 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 73.23it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.20it/s, est. speed input: 1991.51 toks/s, output: 114.38 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.20it/s, est. speed input: 1991.51 toks/s, output: 114.38 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.19it/s, est. speed input: 1991.51 toks/s, output: 114.38 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 67.77it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  2.96it/s, est. speed input: 1135.47 toks/s, output: 127.48 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  2.96it/s, est. speed input: 1135.47 toks/s, output: 127.48 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  2.96it/s, est. speed input: 1135.47 toks/s, output: 127.48 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 74.44it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.93it/s, est. speed input: 1887.88 toks/s, output: 113.36 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.93it/s, est. speed input: 1887.88 toks/s, output: 113.36 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.92it/s, est. speed input: 1887.88 toks/s, output: 113.36 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 50.50it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.68it/s, est. speed input: 2175.81 toks/s, output: 113.61 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.68it/s, est. speed input: 2175.81 toks/s, output: 113.61 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.67it/s, est. speed input: 2175.81 toks/s, output: 113.61 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 69.54it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  3.93it/s, est. speed input: 1505.14 toks/s, output: 121.82 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  3.93it/s, est. speed input: 1505.14 toks/s, output: 121.82 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  3.93it/s, est. speed input: 1505.14 toks/s, output: 121.82 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 68.87it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  6.24it/s, est. speed input: 2391.65 toks/s, output: 106.15 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  6.24it/s, est. speed input: 2391.65 toks/s, output: 106.15 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  6.23it/s, est. speed input: 2391.65 toks/s, output: 106.15 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 71.45it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  6.50it/s, est. speed input: 2491.65 toks/s, output: 104.08 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  6.50it/s, est. speed input: 2491.65 toks/s, output: 104.08 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  6.50it/s, est. speed input: 2491.65 toks/s, output: 104.08 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 63.42it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.27it/s, est. speed input: 2019.17 toks/s, output: 110.70 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.27it/s, est. speed input: 2019.17 toks/s, output: 110.70 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.26it/s, est. speed input: 2019.17 toks/s, output: 110.70 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 58.39it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.05it/s, est. speed input: 1552.61 toks/s, output: 121.61 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.05it/s, est. speed input: 1552.61 toks/s, output: 121.61 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.05it/s, est. speed input: 1552.61 toks/s, output: 121.61 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 70.58it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.67it/s, est. speed input: 2173.38 toks/s, output: 113.47 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.67it/s, est. speed input: 2173.38 toks/s, output: 113.47 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.66it/s, est. speed input: 2173.38 toks/s, output: 113.47 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 70.26it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.37it/s, est. speed input: 2058.14 toks/s, output: 112.84 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.37it/s, est. speed input: 2058.14 toks/s, output: 112.84 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.37it/s, est. speed input: 2058.14 toks/s, output: 112.84 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 74.54it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.97it/s, est. speed input: 2285.88 toks/s, output: 107.42 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.97it/s, est. speed input: 2285.88 toks/s, output: 107.42 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.96it/s, est. speed input: 2285.88 toks/s, output: 107.42 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 76.83it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.81it/s, est. speed input: 2224.01 toks/s, output: 110.32 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.81it/s, est. speed input: 2224.01 toks/s, output: 110.32 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.80it/s, est. speed input: 2224.01 toks/s, output: 110.32 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 70.72it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.80it/s, est. speed input: 1840.30 toks/s, output: 120.11 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.80it/s, est. speed input: 1840.30 toks/s, output: 120.11 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.80it/s, est. speed input: 1840.30 toks/s, output: 120.11 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 69.58it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.12it/s, est. speed input: 1962.75 toks/s, output: 112.73 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.12it/s, est. speed input: 1962.75 toks/s, output: 112.73 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.12it/s, est. speed input: 1962.75 toks/s, output: 112.73 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 62.49it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.89it/s, est. speed input: 2254.63 toks/s, output: 111.84 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.89it/s, est. speed input: 2254.63 toks/s, output: 111.84 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.88it/s, est. speed input: 2254.63 toks/s, output: 111.84 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 68.59it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.14it/s, est. speed input: 1970.36 toks/s, output: 113.17 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.14it/s, est. speed input: 1970.36 toks/s, output: 113.17 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.14it/s, est. speed input: 1970.36 toks/s, output: 113.17 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 77.35it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.62it/s, est. speed input: 2154.72 toks/s, output: 112.50 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.62it/s, est. speed input: 2154.72 toks/s, output: 112.50 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.62it/s, est. speed input: 2154.72 toks/s, output: 112.50 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 77.32it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.64it/s, est. speed input: 1778.31 toks/s, output: 116.07 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.64it/s, est. speed input: 1778.31 toks/s, output: 116.07 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.64it/s, est. speed input: 1778.31 toks/s, output: 116.07 toks/s]
Transcribing ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:02:06
[rank0]:[W114 23:25:00.262201670 ProcessGroupNCCL.cpp:1524] Warning: WARNING: destroy_process_group() was not called before program exit, which can leak resources. For more info, please see https://pytorch.org/docs/stable/distributed.html#shutdown (function operator())
Wrote: outputs/bench/maya_hf_hin_v2/voxtral_mini_vllm/results.jsonl
Wrote: outputs/bench/maya_hf_hin_v2/voxtral_mini_vllm/summary.json
Wrote: outputs/bench/maya_hf_hin_v2/voxtral_mini_vllm/eval.json
{
  "hypothesis": "final",
  "pairs_evaluated": 200,
  "wer_mean": 0.1932474886224888,
  "cer_mean": 0.0784881154613213,
  "wer_p50": 0.16666666666666666,
  "wer_p90": 0.36363636363636365,
  "wer_p95": 0.5,
  "cer_p50": 0.05714285714285714,
  "cer_p90": 0.1875,
  "cer_p95": 0.2413793103448276,
  "segment_error_rate": 0.84,
  "pct_wer_gt_5pct": 0.84,
  "pct_wer_gt_10pct": 0.685,
  "wer_by_language_mean": {
    "hin": 0.1932474886224888
  },
  "wer_by_decision_mean": {
    "accept": 0.18736988882974281,
    "human": 0.20602893579084042
  },
  "wer_by_duration_bucket_mean": {
    "0-3s": 0.2192550505050505,
    "10-30s": 0.23200549450549451,
    "3-10s": 0.1910448593959234
  },
  "cer_by_language_mean": {
    "hin": 0.0784881154613213
  },
  "cer_by_decision_mean": {
    "accept": 0.07802402849306707,
    "human": 0.07949732045577881
  },
  "cer_by_duration_bucket_mean": {
    "0-3s": 0.08746130030959753,
    "10-30s": 0.07990253582770336,
    "3-10s": 0.07823700853406511
  },
  "oracle": {
    "enabled": true,
    "oracle_best_wer_mean": 0.1932474886224888,
    "selection_gap_wer_mean": 0.0,
    "selection_gap_wer_p90": 0.0
  }
}


**Planning Gemma3n model run**

I'm targeting the Gemma3n E4B model first, setting conservative GPU memory and sequence length limits to fit within 14GB; if it causes OOM, I'll fallback to the smaller E2B model for stability.


**Choosing to run Gemma3n E2B first**

To ensure success and meet user expectations within memory limits, I'll run Gemma3n E2B initially with reduced GPU memory and max model length settings, saving outputs to a new directory.


python3 -m maya_transcribe.cli bench run --bench-dir data/bench/maya_hf_hin_v2 --out outputs/bench/maya_hf_hin_v2/gemma3n_e2b_vllm --skip-whisper --vllm-asr-model google/gemma-3n-E2B-it --vllm-gpu-memory-utilization 0.15 --vllm-max-model-len 4096