Output:
Exit code: 0
Wall time: 2095.8 seconds
Total output lines: 5820
Output:
/home/ubuntu/.local/lib/python3.10/site-packages/torch/cuda/__init__.py:63: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you.
  import pynvml  # type: ignore[import]
2026-01-15 19:54:44.300232: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
/home/ubuntu/.local/lib/python3.10/site-packages/google/api_core/_python_version_support.py:266: FutureWarning: You are using a Python version (3.10.12) which Google will stop supporting in new releases of google.api_core once it reaches its end of life (2026-10-04). Please upgrade to the latest Python version, or at least Python 3.11, to continue receiving updates for google.api_core past that date.
  warnings.warn(message, FutureWarning)
INFO 01-15 19:54:51 [utils.py:253] non-default args: {'trust_remote_code': True, 'max_model_len': 4096, 'disable_log_stats': True, 'model': 'mistralai/Voxtral-Small-24B-2507'}
The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
INFO 01-15 19:54:51 [model.py:514] Resolved architecture: VoxtralForConditionalGeneration
INFO 01-15 19:54:51 [model.py:1661] Using max model len 4096
INFO 01-15 19:54:51 [scheduler.py:230] Chunked prefill is enabled with max_num_batched_tokens=8192.
[2026-01-15 19:54:52] INFO tekken.py:195: Non special vocabulary size is 130072 with 1000 special tokens.
[2026-01-15 19:54:52] INFO tekken.py:567: Cutting non special vocabulary to first 130072 tokens.
WARNING 01-15 19:54:53 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: CUDA is initialized
/home/ubuntu/.local/lib/python3.10/site-packages/torch/cuda/__init__.py:63: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you.
  import pynvml  # type: ignore[import]
2026-01-15 19:54:58.481240: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
/home/ubuntu/.local/lib/python3.10/site-packages/google/api_core/_python_version_support.py:266: FutureWarning: You are using a Python version (3.10.12) which Google will stop supporting in new releases of google.api_core once it reaches its end of life (2026-10-04). Please upgrade to the latest Python version, or at least Python 3.11, to continue receiving updates for google.api_core past that date.
  warnings.warn(message, FutureWarning)
(EngineCore_DP0 pid=2563033) INFO 01-15 19:55:02 [core.py:93] Initializing a V1 LLM engine (v0.13.0) with config: model='mistralai/Voxtral-Small-24B-2507', speculative_config=None, tokenizer='mistralai/Voxtral-Small-24B-2507', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, tokenizer_revision=None, trust_remote_code=True, dtype=torch.bfloat16, max_seq_len=4096, download_dir=None, load_format=auto, tensor_parallel_size=1, pipeline_parallel_size=1, data_parallel_size=1, disable_custom_all_reduce=False, quantization=None, enforce_eager=False, kv_cache_dtype=auto, device_config=cuda, structured_outputs_config=StructuredOutputsConfig(backend='auto', disable_fallback=False, disable_any_whitespace=False, disable_additional_properties=False, reasoning_parser='', reasoning_parser_plugin='', enable_in_reasoning=False), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None, kv_cache_metrics=False, kv_cache_metrics_sample=0.01, cudagraph_metrics=False, enable_layerwise_nvtx_tracing=False), seed=0, served_model_name=mistralai/Voxtral-Small-24B-2507, enable_prefix_caching=True, enable_chunked_prefill=True, pooler_config=None, compilation_config={'level': None, 'mode': <CompilationMode.VLLM_COMPILE: 3>, 'debug_dump_path': None, 'cache_dir': '', 'compile_cache_save_format': 'binary', 'backend': 'inductor', 'custom_ops': ['none'], 'splitting_ops': ['vllm::unified_attention', 'vllm::unified_attention_with_output', 'vllm::unified_mla_attention', 'vllm::unified_mla_attention_with_output', 'vllm::mamba_mixer2', 'vllm::mamba_mixer', 'vllm::short_conv', 'vllm::linear_attention', 'vllm::plamo2_mamba_mixer', 'vllm::gdn_attention_core', 'vllm::kda_attention', 'vllm::sparse_attn_indexer'], 'compile_mm_encoder': False, 'compile_sizes': [], 'compile_ranges_split_points': [8192], 'inductor_compile_config': {'enable_auto_functionalized_v2': False, 'combo_kernels': True, 'benchmark_combo_kernel': True}, 'inductor_passes': {}, 'cudagraph_mode': <CUDAGraphMode.FULL_AND_PIECEWISE: (2, 1)>, 'cudagraph_num_of_warmups': 1, 'cudagraph_capture_sizes': [1, 2, 4, 8, 16, 24, 32, 40, 48, 56, 64, 72, 80, 88, 96, 104, 112, 120, 128, 136, 144, 152, 160, 168, 176, 184, 192, 200, 208, 216, 224, 232, 240, 248, 256, 272, 288, 304, 320, 336, 352, 368, 384, 400, 416, 432, 448, 464, 480, 496, 512], 'cudagraph_copy_inputs': False, 'cudagraph_specialize_lora': True, 'use_inductor_graph_partition': False, 'pass_config': {'fuse_norm_quant': False, 'fuse_act_quant': False, 'fuse_attn_quant': False, 'eliminate_noops': True, 'enable_sp': False, 'fuse_gemm_comms': False, 'fuse_allreduce_rms': False}, 'max_cudagraph_capture_size': 512, 'dynamic_shapes_config': {'type': <DynamicShapesType.BACKED: 'backed'>, 'evaluate_guards': False}, 'local_cache_dir': None}
(EngineCore_DP0 pid=2563033) [2026-01-15 19:55:03] INFO tekken.py:195: Non special vocabulary size is 130072 with 1000 special tokens.
(EngineCore_DP0 pid=2563033) [2026-01-15 19:55:03] INFO tekken.py:567: Cutting non special vocabulary to first 130072 tokens.
(EngineCore_DP0 pid=2563033) INFO 01-15 19:55:04 [parallel_state.py:1203] world_size=1 rank=0 local_rank=0 distributed_init_method=tcp://216.81.248.184:50355 backend=nccl
(EngineCore_DP0 pid=2563033) INFO 01-15 19:55:04 [parallel_state.py:1411] rank 0 in world size 1 is assigned as DP rank 0, PP rank 0, PCP rank 0, TP rank 0, EP rank 0
(EngineCore_DP0 pid=2563033) INFO 01-15 19:55:05 [gpu_model_runner.py:3562] Starting to load model mistralai/Voxtral-Small-24B-2507...
(EngineCore_DP0 pid=2563033) INFO 01-15 19:55:06 [cuda.py:351] Using FLASH_ATTN attention backend out of potential backends: ('FLASH_ATTN', 'FLASHINFER', 'TRITON_ATTN', 'FLEX_ATTENTION')
(EngineCore_DP0 pid=2563033) INFO 01-15 19:55:06 [layer.py:477] Using AttentionBackendEnum.FLASH_ATTN for MultiHeadAttention in multimodal encoder.
(EngineCore_DP0 pid=2563033) INFO 01-15 19:55:06 [weight_utils.py:527] No consolidated.safetensors.index.json found in remote.
(EngineCore_DP0 pid=2563033)
Loading safetensors checkpoint shards:   0% Completed | 0/1 [00:00<?, ?it/s]
(EngineCore_DP0 pid=2563033)
Loading safetensors checkpoint shards: 100% Completed | 1/1 [00:07<00:00,  7.12s/it]
(EngineCore_DP0 pid=2563033)
Loading safetensors checkpoint shards: 100% Completed | 1/1 [00:07<00:00,  7.12s/it]
(EngineCore_DP0 pid=2563033)
(EngineCore_DP0 pid=2563033) INFO 01-15 19:55:15 [default_loader.py:308] Loading weights took 8.66 seconds
(EngineCore_DP0 pid=2563033) INFO 01-15 19:55:16 [gpu_model_runner.py:3659] Model loading took 45.2032 GiB memory and 9.463496 seconds
(EngineCore_DP0 pid=2563033) INFO 01-15 19:55:17 [gpu_model_runner.py:4446] Encoder cache will be initialized with a budget of 8192 tokens, and profiled with 2 audio items of the maximum feature size.
(EngineCore_DP0 pid=2563033) INFO 01-15 19:55:24 [backends.py:643] Using cache directory: /home/ubuntu/.cache/vllm/torch_compile_cache/936cd8900f/rank_0_0/backbone for vLLM's torch.compile
(EngineCore_DP0 pid=2563033) INFO 01-15 19:55:24 [backends.py:703] Dynamo bytecode transform time: 6.11 s
(EngineCore_DP0 pid=2563033) INFO 01-15 19:55:36 [backends.py:226] Directly load the compiled graph(s) for compile range (1, 8192) from the cache, took 8.767 s
(EngineCore_DP0 pid=2563033) INFO 01-15 19:55:36 [monitor.py:34] torch.compile takes 14.87 s in total
(EngineCore_DP0 pid=2563033) INFO 01-15 19:55:37 [gpu_worker.py:375] Available KV cache memory: 24.25 GiB
(EngineCore_DP0 pid=2563033) INFO 01-15 19:55:38 [kv_cache_utils.py:1291] GPU KV cache size: 158,912 tokens
(EngineCore_DP0 pid=2563033) INFO 01-15 19:55:38 [kv_cache_utils.py:1296] Maximum concurrency for 4,096 tokens per request: 38.80x
(EngineCore_DP0 pid=2563033)
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):   0%|          | 0/51 [00:00<?, ?it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):   2%|▏         | 1/51 [00:00<00:06,  8.33it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):   4%|▍         | 2/51 [00:00<00:05,  8.57it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):   6%|▌         | 3/51 [00:00<00:05,  8.65it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):   8%|▊         | 4/51 [00:00<00:05,  8.71it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  10%|▉         | 5/51 [00:00<00:05,  8.61it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  12%|█▏        | 6/51 [00:00<00:05,  8.63it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  14%|█▎        | 7/51 [00:00<00:05,  8.69it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  16%|█▌        | 8/51 [00:00<00:04,  8.64it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  20%|█▉        | 10/51 [00:01<00:04,  9.24it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  24%|██▎       | 12/51 [00:01<00:04,  9.58it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  27%|██▋       | 14/51 [00:01<00:03,  9.82it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  31%|███▏      | 16/51 [00:01<00:03, 10.00it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  35%|███▌      | 18/51 [00:01<00:03, 10.95it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  39%|███▉      | 20/51 [00:02<00:02, 11.74it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  43%|████▎     | 22/51 [00:02<00:02, 12.34it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  47%|████▋     | 24/51 [00:02<00:02, 12.79it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  51%|█████     | 26/51 [00:02<00:01, 13.14it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  55%|█████▍    | 28/51 [00:02<00:01, 13.37it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  59%|█████▉    | 30/51 [00:02<00:01, 13.56it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  63%|██████▎   | 32/51 [00:02<00:01, 13.66it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  67%|██████▋   | 34/51 [00:02<00:01, 14.37it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  71%|███████   | 36/51 [00:03<00:01, 14.84it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  75%|███████▍  | 38/51 [00:03<00:00, 14.99it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  78%|███████▊  | 40/51 [00:03<00:00, 15.44it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  82%|████████▏ | 42/51 [00:03<00:00, 16.02it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  86%|████████▋ | 44/51 [00:03<00:00, 16.49it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  90%|█████████ | 46/51 [00:03<00:00, 16.79it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  94%|█████████▍| 48/51 [00:03<00:00, 17.05it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  98%|█████████▊| 50/51 [00:03<00:00, 17.35it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 100%|██████████| 51/51 [00:03<00:00, 12.79it/s]
(EngineCore_DP0 pid=2563033)
Capturing CUDA graphs (decode, FULL):   0%|          | 0/35 [00:00<?, ?it/s]
Capturing CUDA graphs (decode, FULL):   6%|▌         | 2/35 [00:00<00:02, 12.29it/s]
Capturing CUDA graphs (decode, FULL):  11%|█▏        | 4/35 [00:00<00:02, 13.14it/s]
Capturing CUDA graphs (decode, FULL):  17%|█▋        | 6/35 [00:00<00:02, 13.48it/s]
Capturing CUDA graphs (decode, FULL):  23%|██▎       | 8/35 [00:00<00:01, 13.67it/s]
Capturing CUDA graphs (decode, FULL):  29%|██▊       | 10/35 [00:00<00:01, 13.82it/s]
Capturing CUDA graphs (decode, FULL):  34%|███▍      | 12/35 [00:00<00:01, 13.89it/s]
Capturing CUDA graphs (decode, FULL):  40%|████      | 14/35 [00:01<00:01, 13.95it/s]
Capturing CUDA graphs (decode, FULL):  46%|████▌     | 16/35 [00:01<00:01, 13.94it/s]
Capturing CUDA graphs (decode, FULL):  51%|█████▏    | 18/35 [00:01<00:01, 14.68it/s]
Capturing CUDA graphs (decode, FULL):  57%|█████▋    | 20/35 [00:01<00:00, 15.23it/s]
Capturing CUDA graphs (decode, FULL):  63%|██████▎   | 22/35 [00:01<00:00, 15.68it/s]
Capturing CUDA graphs (decode, FULL):  69%|██████▊   | 24/35 [00:01<00:00, 16.06it/s]
Capturing CUDA graphs (decode, FULL):  74%|███████▍  | 26/35 [00:01<00:00, 16.71it/s]
Capturing CUDA graphs (decode, FULL):  80%|████████  | 28/35 [00:01<00:00, 17.22it/s]
Capturing CUDA graphs (decode, FULL):  86%|████████▌ | 30/35 [00:01<00:00, 17.72it/s]
Capturing CUDA graphs (decode, FULL):  91%|█████████▏| 32/35 [00:02<00:00, 18.03it/s]
Capturing CUDA graphs (decode, FULL):  97%|█████████▋| 34/35 [00:02<00:00, 18.36it/s]
Capturing CUDA graphs (decode, FULL): 100%|██████████| 35/35 [00:02<00:00, 15.72it/s]
(EngineCore_DP0 pid=2563033) INFO 01-15 19:55:45 [gpu_model_runner.py:4587] Graph capturing finished in 7 secs, took 0.71 GiB
(EngineCore_DP0 pid=2563033) INFO 01-15 19:55:45 [core.py:259] init engine (profile, create kv cache, warmup model) took 29.14 seconds
INFO 01-15 19:55:46 [llm.py:360] Supported tasks: ['generate', 'transcription']
WARNING 01-15 19:55:47 [interfaces.py:920] Language 'as' is not natively supported by VoxtralForConditionalGeneration; results may be less accurate. Supported languages: ['ar', 'nl', 'en', 'fr', 'de', 'hi', 'it', 'pt', 'es']

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:01<00:00,  1.34s/it]
Adding requests: 100%|██████████| 1/1 [00:01<00:00,  1.34s/it]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:01<00:00,  1.23s/it, est. speed input: 312.66 toks/s, output: 31.02 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:01<00:00,  1.23s/it, est. speed input: 312.66 toks/s, output: 31.02 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:01<00:00,  1.23s/it, est. speed input: 312.66 toks/s, output: 31.02 toks/s]
WARNING 01-15 19:55:50 [interfaces.py:920] Language 'as' is not natively supported by VoxtralForConditionalGeneration; results may be less accurate. Supported languages: ['ar', 'nl', 'en', 'fr', 'de', 'hi', 'it', 'pt', 'es']

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 359.38it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:01<00:00,  1.43s/it, est. speed input: 268.43 toks/s, output: 31.54 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:01<00:00,  1.43s/it, est. speed input: 268.43 toks/s, output: 31.54 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:01<00:00,  1.43s/it, est. speed input: 268.43 toks/s, output: 31.54 toks/s]
WARNING 01-15 19:55:52 [interfaces.py:920] Language 'as' is not natively supported by VoxtralForConditionalGeneration; results may be less accurate. Supported languages: ['ar', 'nl', 'en', 'fr', 'de', 'hi', 'it', 'pt', 'es']

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 384.55it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:01<00:00,  1.40s/it, est. speed input: 274.34 toks/s, output: 31.52 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:01<00:00,  1.40s/it, est. speed input: 274.34 toks/s, output: 31.52 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:01<00:00,  1.40s/it, est. speed input: 274.34 toks/s, output: 31.52 toks/s]
WARNING 01-15 19:55:54 [interfaces.py:920] Language 'as' is not natively supported by VoxtralForConditionalGeneration; results may be less accurate. Supported languages: ['ar', 'nl', 'en', 'fr', 'de', 'hi', 'it', 'pt', 'es']

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 285.27it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:01<00:00,  1.85s/it, est. speed input: 207.58 toks/s, output: 31.98 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:01<00:00,  1.85s/it, est. speed input: 207.58 toks/s, output: 31.98 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:01<00:00,  1.85s/it, est. speed input: 207.58 toks/s, output: 31.98 toks/s]
WARNING 01-15 19:55:56 [interfaces.py:920] Language 'as' is not natively supported by VoxtralForConditionalGeneration; results may be less accurate. Su…330580 tokens truncated…ay be less accurate. Supported languages: ['ar', 'nl', 'en', 'fr', 'de', 'hi', 'it', 'pt', 'es']

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 379.16it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:01<00:00,  1.43s/it, est. speed input: 268.09 toks/s, output: 31.50 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:01<00:00,  1.43s/it, est. speed input: 268.09 toks/s, output: 31.50 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:01<00:00,  1.43s/it, est. speed input: 268.09 toks/s, output: 31.50 toks/s]
WARNING 01-15 20:29:04 [interfaces.py:920] Language 'te' is not natively supported by VoxtralForConditionalGeneration; results may be less accurate. Supported languages: ['ar', 'nl', 'en', 'fr', 'de', 'hi', 'it', 'pt', 'es']

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 281.86it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:01<00:00,  1.49s/it, est. speed input: 257.18 toks/s, output: 31.56 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:01<00:00,  1.49s/it, est. speed input: 257.18 toks/s, output: 31.56 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:01<00:00,  1.49s/it, est. speed input: 257.18 toks/s, output: 31.56 toks/s]
WARNING 01-15 20:29:06 [interfaces.py:920] Language 'te' is not natively supported by VoxtralForConditionalGeneration; results may be less accurate. Supported languages: ['ar', 'nl', 'en', 'fr', 'de', 'hi', 'it', 'pt', 'es']

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 399.38it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:01<00:00,  1.82s/it, est. speed input: 210.51 toks/s, output: 31.88 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:01<00:00,  1.82s/it, est. speed input: 210.51 toks/s, output: 31.88 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:01<00:00,  1.82s/it, est. speed input: 210.51 toks/s, output: 31.88 toks/s]
WARNING 01-15 20:29:08 [interfaces.py:920] Language 'te' is not natively supported by VoxtralForConditionalGeneration; results may be less accurate. Supported languages: ['ar', 'nl', 'en', 'fr', 'de', 'hi', 'it', 'pt', 'es']

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 297.68it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:01<00:00,  1.90s/it, est. speed input: 201.26 toks/s, output: 32.05 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:01<00:00,  1.90s/it, est. speed input: 201.26 toks/s, output: 32.05 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:01<00:00,  1.90s/it, est. speed input: 201.26 toks/s, output: 32.05 toks/s]
WARNING 01-15 20:29:11 [interfaces.py:920] Language 'te' is not natively supported by VoxtralForConditionalGeneration; results may be less accurate. Supported languages: ['ar', 'nl', 'en', 'fr', 'de', 'hi', 'it', 'pt', 'es']

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 238.53it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:01<00:00,  1.36s/it, est. speed input: 281.10 toks/s, output: 31.56 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:01<00:00,  1.36s/it, est. speed input: 281.10 toks/s, output: 31.56 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:01<00:00,  1.36s/it, est. speed input: 281.10 toks/s, output: 31.56 toks/s]
WARNING 01-15 20:29:12 [interfaces.py:920] Language 'te' is not natively supported by VoxtralForConditionalGeneration; results may be less accurate. Supported languages: ['ar', 'nl', 'en', 'fr', 'de', 'hi', 'it', 'pt', 'es']

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 218.43it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:01<00:00,  1.28s/it, est. speed input: 299.93 toks/s, output: 31.32 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:01<00:00,  1.28s/it, est. speed input: 299.93 toks/s, output: 31.32 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:01<00:00,  1.28s/it, est. speed input: 299.93 toks/s, output: 31.32 toks/s]
WARNING 01-15 20:29:14 [interfaces.py:920] Language 'te' is not natively supported by VoxtralForConditionalGeneration; results may be less accurate. Supported languages: ['ar', 'nl', 'en', 'fr', 'de', 'hi', 'it', 'pt', 'es']

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 261.39it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:01<00:00,  1.33s/it, est. speed input: 286.97 toks/s, output: 31.47 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:01<00:00,  1.33s/it, est. speed input: 286.97 toks/s, output: 31.47 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:01<00:00,  1.34s/it, est. speed input: 286.97 toks/s, output: 31.47 toks/s]
WARNING 01-15 20:29:16 [interfaces.py:920] Language 'te' is not natively supported by VoxtralForConditionalGeneration; results may be less accurate. Supported languages: ['ar', 'nl', 'en', 'fr', 'de', 'hi', 'it', 'pt', 'es']

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 295.96it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:01<00:00,  1.45s/it, est. speed input: 263.26 toks/s, output: 31.62 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:01<00:00,  1.45s/it, est. speed input: 263.26 toks/s, output: 31.62 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:01<00:00,  1.46s/it, est. speed input: 263.26 toks/s, output: 31.62 toks/s]
WARNING 01-15 20:29:18 [interfaces.py:920] Language 'te' is not natively supported by VoxtralForConditionalGeneration; results may be less accurate. Supported languages: ['ar', 'nl', 'en', 'fr', 'de', 'hi', 'it', 'pt', 'es']

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 213.98it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:01<00:00,  1.19s/it, est. speed input: 322.99 toks/s, output: 31.20 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:01<00:00,  1.19s/it, est. speed input: 322.99 toks/s, output: 31.20 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:01<00:00,  1.19s/it, est. speed input: 322.99 toks/s, output: 31.20 toks/s]
WARNING 01-15 20:29:19 [interfaces.py:920] Language 'te' is not natively supported by VoxtralForConditionalGeneration; results may be less accurate. Supported languages: ['ar', 'nl', 'en', 'fr', 'de', 'hi', 'it', 'pt', 'es']

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 123.25it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:01<00:00,  1.22s/it, est. speed input: 314.16 toks/s, output: 31.17 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:01<00:00,  1.22s/it, est. speed input: 314.16 toks/s, output: 31.17 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:01<00:00,  1.22s/it, est. speed input: 314.16 toks/s, output: 31.17 toks/s]
WARNING 01-15 20:29:21 [interfaces.py:920] Language 'te' is not natively supported by VoxtralForConditionalGeneration; results may be less accurate. Supported languages: ['ar', 'nl', 'en', 'fr', 'de', 'hi', 'it', 'pt', 'es']

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 290.26it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:01<00:00,  1.62s/it, est. speed input: 236.80 toks/s, output: 31.53 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:01<00:00,  1.62s/it, est. speed input: 236.80 toks/s, output: 31.53 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:01<00:00,  1.62s/it, est. speed input: 236.80 toks/s, output: 31.53 toks/s]
WARNING 01-15 20:29:23 [interfaces.py:920] Language 'te' is not natively supported by VoxtralForConditionalGeneration; results may be less accurate. Supported languages: ['ar', 'nl', 'en', 'fr', 'de', 'hi', 'it', 'pt', 'es']

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 71.66it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:01<00:00,  1.07s/it, est. speed input: 356.48 toks/s, output: 30.71 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:01<00:00,  1.07s/it, est. speed input: 356.48 toks/s, output: 30.71 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:01<00:00,  1.07s/it, est. speed input: 356.48 toks/s, output: 30.71 toks/s]
WARNING 01-15 20:29:24 [interfaces.py:920] Language 'te' is not natively supported by VoxtralForConditionalGeneration; results may be less accurate. Supported languages: ['ar', 'nl', 'en', 'fr', 'de', 'hi', 'it', 'pt', 'es']

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 70.21it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:01<00:00,  1.28s/it, est. speed input: 299.39 toks/s, output: 31.27 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:01<00:00,  1.28s/it, est. speed input: 299.39 toks/s, output: 31.27 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:01<00:00,  1.28s/it, est. speed input: 299.39 toks/s, output: 31.27 toks/s]
WARNING 01-15 20:29:26 [interfaces.py:920] Language 'te' is not natively supported by VoxtralForConditionalGeneration; results may be less accurate. Supported languages: ['ar', 'nl', 'en', 'fr', 'de', 'hi', 'it', 'pt', 'es']

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 325.42it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:01<00:00,  1.39s/it, est. speed input: 274.76 toks/s, output: 31.56 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:01<00:00,  1.39s/it, est. speed input: 274.76 toks/s, output: 31.56 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:01<00:00,  1.39s/it, est. speed input: 274.76 toks/s, output: 31.56 toks/s]
WARNING 01-15 20:29:28 [interfaces.py:920] Language 'te' is not natively supported by VoxtralForConditionalGeneration; results may be less accurate. Supported languages: ['ar', 'nl', 'en', 'fr', 'de', 'hi', 'it', 'pt', 'es']

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]/home/ubuntu/.local/lib/python3.10/site-packages/mistral_common/tokens/tokenizers/tekken.py:462: FutureWarning: `get_control_token` is deprecated. Use `get_special_token` instead.
  warnings.warn("`get_control_token` is deprecated. Use `get_special_token` instead.", FutureWarning)

Adding requests: 100%|██████████| 1/1 [00:00<00:00, 161.42it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:01<00:00,  1.22s/it, est. speed input: 314.82 toks/s, output: 31.24 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:01<00:00,  1.22s/it, est. speed input: 314.82 toks/s, output: 31.24 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:01<00:00,  1.22s/it, est. speed input: 314.82 toks/s, output: 31.24 toks/s]
Transcribing ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:33:42
[rank0]:[W115 20:29:29.314700736 ProcessGroupNCCL.cpp:1524] Warning: WARNING: destroy_process_group() was not called before program exit, which can leak resources. For more info, please see https://pytorch.org/docs/stable/distributed.html#shutdown (function operator())
Wrote: outputs/bench/maya_hf_v2_100/voxtral_small_vllm/results.jsonl
Wrote: outputs/bench/maya_hf_v2_100/voxtral_small_vllm/summary.json
Wrote: outputs/bench/maya_hf_v2_100/voxtral_small_vllm/eval.json
{
  "hypothesis": "final",
  "pairs_evaluated": 1200,
  "wer_mean": 0.5249478863951226,
  "cer_mean": 0.2788291240348756,
  "wer_p50": 0.45454545454545453,
  "wer_p90": 1.1111111111111112,
  "wer_p95": 1.3,
  "cer_p50": 0.13636363636363635,
  "cer_p90": 1.0,
  "cer_p95": 1.0253164556962024,
  "segment_error_rate": 0.9183333333333333,
  "pct_wer_gt_5pct": 0.905,
  "pct_wer_gt_10pct": 0.8558333333333333,
  "wer_by_language_mean": {
    "asm": 1.078694539177487,
    "ben": 0.37382001332001324,
    "eng": 0.03839471627860255,
    "guj": 0.4977294649794649,
    "hin": 0.14871028971028966,
    "kan": 0.48962099012099003,
    "mal": 0.6257590465090462,
    "mar": 0.39922086247086247,
    "ory": 1.302522311022311,
    "pan": 0.34230536130536116,
    "tam": 0.5272821900321898,
    "tel": 0.4753148518148517
  },
  "wer_by_decision_mean": {
    "accept": 0.5366656009106169,
    "human": 0.36725286092755965
  },
  "wer_by_duration_bucket_mean": {
    "0-3s": 0.3242063492063492,
    "10-30s": 0.6441920665492443,
    "3-10s": 0.50234602811667
  },
  "cer_by_language_mean": {
    "asm": 0.8097244570497816,
    "ben": 0.13486010966620854,
    "eng": 0.01617229744345676,
    "guj": 0.31510373777463957,
    "hin": 0.05589700755155696,
    "kan": 0.1695400083023344,
    "mal": 0.2054416799336396,
    "mar": 0.1404190924292441,
    "ory": 1.0294407368540577,
    "pan": 0.16849425527486336,
    "tam": 0.15723711096696918,
    "tel": 0.14361899517175528
  },
  "cer_by_decision_mean": {
    "accept": 0.28614375388119145,
    "human": 0.1803900693561435
  },
  "cer_by_duration_bucket_mean": {
    "0-3s": 0.2162290550469659,
    "10-30s": 0.3428344731999273,
    "3-10s": 0.2663791045483757
  },
  "oracle": {
    "enabled": true,
    "oracle_best_wer_mean": 0.5249478863951226,
    "selection_gap_wer_mean": 0.0,
    "selection_gap_wer_p90": 0.0
  }
}


Summary: auto

## Skills
A skill is a set of local instructions to follow that is stored in a `SKILL.md` file. Below is the list of skills that can be used. Each entry includes a name, description, and file path so you can open the source for full instructions when using a specific skill.
### Available skills
- skill-creator: Guide for creating effective skills. This skill should be used when users want to create a new skill (or update an existing skill) that extends Codex's capabilities with specialized knowledge, workflows, or tool integrations. (file: /home/ubuntu/.codex/skills/.system/skill-creator/SKILL.md)
- skill-installer: Install Codex skills into $CODEX_HOME/skills from a curated list or a GitHub repo path. Use when a user asks to list installable skills, install a curated skill, or install a skill from another repo (including private repos). (file: /home/ubuntu/.codex/skills/.system/skill-installer/SKILL.md)
### How to use skills
- Discovery: The list above is the skills available in this session (name + description + file path). Skill bodies live on disk at the listed paths.
- Trigger rules: If the user names a skill (with `$SkillName` or plain text) OR the task clearly matches a skill's description shown above, you must use that skill for that turn. Multiple mentions mean use them all. Do not carry skills across turns unless re-mentioned.
- Missing/blocked: If a named skill isn't in the list or the path can't be read, say so briefly and continue with the best fallback.
- How to use a skill (progressive disclosure):
  1) After deciding to use a skill, open its `SKILL.md`. Read only enough to follow the workflow.
  2) If `SKILL.md` points to extra folders such as `references/`, load only the specific files needed for the request; don't bulk-load everything.
  3) If `scripts/` exist, prefer running or patching them instead of retyping large code blocks.
  4) If `assets/` or templates exist, reuse them instead of recreating from scratch.
- Coordination and sequencing:
  - If multiple skills apply, choose the minimal set that covers the request and state the order you'll use them.
  - Announce which skill(s) you're using and why (one short line). If you skip an obvious skill, say why.
- Context hygiene:
  - Keep context small: summarize long sections instead of pasting them; only load extra files when needed.
  - Avoid deep reference-chasing: prefer opening only files directly linked from `SKILL.md` unless you're blocked.
  - When variants exist (frameworks, providers, domains), pick only the relevant reference file(s) and note that choice.
- Safety and fallback: If a skill can't be applied cleanly (missing files, unclear instructions), state the issue, pick the next-best approach, and continue.


**Preparing to run vLLM with Gemma**


python3 -m maya_transcribe.cli bench run --bench-dir data/bench/maya_hf_v2_100 --out outputs/bench/maya_hf_v2_100/gemma3n_e2b_vllm --skip-whisper --vllm-asr-model google/gemma-3n-E2B-it --vllm-gpu-memory-utilization 0.9 --vllm-max-model-len 4096 --thresholds-json configs/bench_v2_100_thresholds.json


Output:
Exit code: 0
Wall time: 1010.5 seconds
Total output lines: 3116
Output:
/home/ubuntu/.local/lib/python3.10/site-packages/torch/cuda/__init__.py:63: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you.
  import pynvml  # type: ignore[import]
2026-01-15 20:29:49.183443: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
/home/ubuntu/.local/lib/python3.10/site-packages/google/api_core/_python_version_support.py:266: FutureWarning: You are using a Python version (3.10.12) which Google will stop supporting in new releases of google.api_core once it reaches its end of life (2026-10-04). Please upgrade to the latest Python version, or at least Python 3.11, to continue receiving updates for google.api_core past that date.
  warnings.warn(message, FutureWarning)
INFO 01-15 20:29:55 [utils.py:253] non-default args: {'trust_remote_code': True, 'max_model_len': 4096, 'disable_log_stats': True, 'limit_mm_per_prompt': {'image': 0, 'audio': 1}, 'model': 'google/gemma-3n-E2B-it'}
The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
INFO 01-15 20:29:56 [model.py:514] Resolved architecture: Gemma3nForConditionalGeneration
INFO 01-15 20:29:56 [model.py:1661] Using max model len 4096
INFO 01-15 20:29:56 [scheduler.py:230] Chunked prefill is enabled with max_num_batched_tokens=8192.
WARNING 01-15 20:29:59 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: CUDA is initialized
/home/ubuntu/.local/lib/python3.10/site-packages/torch/cuda/__init__.py:63: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you.
  import pynvml  # type: ignore[import]
2026-01-15 20:30:04.108011: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
/home/ubuntu/.local/lib/python3.10/site-packages/google/api_core/_python_version_support.py:266: FutureWarning: You are using a Python version (3.10.12) which Google will stop supporting in new releases of google.api_core once it reaches its end of life (2026-10-04). Please upgrade to the latest Python version, or at least Python 3.11, to continue receiving updates for google.api_core past that date.
  warnings.warn(message, FutureWarning)
(EngineCore_DP0 pid=2659694) INFO 01-15 20:30:08 [core.py:93] Initializing a V1 LLM engine (v0.13.0) with config: model='google/gemma-3n-E2B-it', speculative_config=None, tokenizer='google/gemma-3n-E2B-it', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, tokenizer_revision=None, trust_remote_code=True, dtype=torch.bfloat16, max_seq_len=4096, download_dir=None, load_format=auto, tensor_parallel_size=1, pipeline_parallel_size=1, data_parallel_size=1, disable_custom_all_reduce=False, quantization=None, enforce_eager=False, kv_cache_dtype=auto, device_config=cuda, structured_outputs_config=StructuredOutputsConfig(backend='auto', disable_fallback=False, disable_any_whitespace=False, disable_additional_properties=False, reasoning_parser='', reasoning_parser_plugin='', enable_in_reasoning=False), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None, kv_cache_metrics=False, kv_cache_metrics_sample=0.01, cudagraph_metrics=False, enable_layerwise_nvtx_tracing=False), seed=0, served_model_name=google/gemma-3n-E2B-it, enable_prefix_caching=True, enable_chunked_prefill=True, pooler_config=None, compilation_config={'level': None, 'mode': <CompilationMode.VLLM_COMPILE: 3>, 'debug_dump_path': None, 'cache_dir': '', 'compile_cache_save_format': 'binary', 'backend': 'inductor', 'custom_ops': ['none'], 'splitting_ops': ['vllm::unified_attention', 'vllm::unified_attention_with_output', 'vllm::unified_mla_attention', 'vllm::unified_mla_attention_with_output', 'vllm::mamba_mixer2', 'vllm::mamba_mixer', 'vllm::short_conv', 'vllm::linear_attention', 'vllm::plamo2_mamba_mixer', 'vllm::gdn_attention_core', 'vllm::kda_attention', 'vllm::sparse_attn_indexer'], 'compile_mm_encoder': False, 'compile_sizes': [], 'compile_ranges_split_points': [8192], 'inductor_compile_config': {'enable_auto_functionalized_v2': False, 'combo_kernels': True, 'benchmark_combo_kernel': True}, 'inductor_passes': {}, 'cudagraph_mode': <CUDAGraphMode.FULL_AND_PIECEWISE: (2, 1)>, 'cudagraph_num_of_warmups': 1, 'cudagraph_capture_sizes': [1, 2, 4, 8, 16, 24, 32, 40, 48, 56, 64, 72, 80, 88, 96, 104, 112, 120, 128, 136, 144, 152, 160, 168, 176, 184, 192, 200, 208, 216, 224, 232, 240, 248, 256, 272, 288, 304, 320, 336, 352, 368, 384, 400, 416, 432, 448, 464, 480, 496, 512], 'cudagraph_copy_inputs': False, 'cudagraph_specialize_lora': True, 'use_inductor_graph_partition': False, 'pass_config': {'fuse_norm_quant': False, 'fuse_act_quant': False, 'fuse_attn_quant': False, 'eliminate_noops': True, 'enable_sp': False, 'fuse_gemm_comms': False, 'fuse_allreduce_rms': False}, 'max_cudagraph_capture_size': 512, 'dynamic_shapes_config': {'type': <DynamicShapesType.BACKED: 'backed'>, 'evaluate_guards': False}, 'local_cache_dir': None}
(EngineCore_DP0 pid=2659694) INFO 01-15 20:30:12 [parallel_state.py:1203] world_size=1 rank=0 local_rank=0 distributed_init_method=tcp://216.81.248.184:53745 backend=nccl
(EngineCore_DP0 pid=2659694) INFO 01-15 20:30:12 [parallel_state.py:1411] rank 0 in world size 1 is assigned as DP rank 0, PP rank 0, PCP rank 0, TP rank 0, EP rank 0
(EngineCore_DP0 pid=2659694) INFO 01-15 20:30:25 [gpu_model_runner.py:3562] Starting to load model google/gemma-3n-E2B-it...
(EngineCore_DP0 pid=2659694) INFO 01-15 20:30:26 [cuda.py:351] Using FLASH_ATTN attention backend out of potential backends: ('FLASH_ATTN', 'FLASHINFER', 'TRITON_ATTN', 'FLEX_ATTENTION')
(EngineCore_DP0 pid=2659694)
Loading safetensors checkpoint shards:   0% Completed | 0/3 [00:00<?, ?it/s]
(EngineCore_DP0 pid=2659694)
Loading safetensors checkpoint shards:  33% Completed | 1/3 [00:00<00:01,  1.92it/s]
(EngineCore_DP0 pid=2659694)
Loading safetensors checkpoint shards:  67% Completed | 2/3 [00:01<00:00,  1.33it/s]
(EngineCore_DP0 pid=2659694)
Loading safetensors checkpoint shards: 100% Completed | 3/3 [00:02<00:00,  1.35it/s]
(EngineCore_DP0 pid=2659694)
Loading safetensors checkpoint shards: 100% Completed | 3/3 [00:02<00:00,  1.38it/s]
(EngineCore_DP0 pid=2659694)
(EngineCore_DP0 pid=2659694) INFO 01-15 20:30:29 [default_loader.py:308] Loading weights took 2.27 seconds
(EngineCore_DP0 pid=2659694) INFO 01-15 20:30:30 [gpu_model_runner.py:3659] Model loading took 10.2839 GiB memory and 3.136890 seconds
(EngineCore_DP0 pid=2659694) INFO 01-15 20:30:31 [gpu_model_runner.py:4446] Encoder cache will be initialized with a budget of 8192 tokens, and profiled with 43 audio items of the maximum feature size.
(EngineCore_DP0 pid=2659694) INFO 01-15 20:30:45 [backends.py:643] Using cache directory: /home/ubuntu/.cache/vllm/torch_compile_cache/1a387ba34f/rank_0_0/backbone for vLLM's torch.compile
(EngineCore_DP0 pid=2659694) INFO 01-15 20:30:45 [backends.py:703] Dynamo bytecode transform time: 13.22 s
(EngineCore_DP0 pid=2659694) INFO 01-15 20:30:59 [backends.py:226] Directly load the compiled graph(s) for compile range (1, 8192) from the cache, took 7.144 s
(EngineCore_DP0 pid=2659694) INFO 01-15 20:30:59 [monitor.py:34] torch.compile takes 20.36 s in total
(EngineCore_DP0 pid=2659694) INFO 01-15 20:31:02 [gpu_worker.py:375] Available KV cache memory: 58.55 GiB
(EngineCore_DP0 pid=2659694) INFO 01-15 20:31:02 [kv_cache_utils.py:1291] GPU KV cache size: 1,534,896 tokens
(EngineCore_DP0 pid=2659694) INFO 01-15 20:31:02 [kv_cache_utils.py:1296] Maximum concurrency for 4,096 tokens per request: 373.56x
(EngineCore_DP0 pid=2659694)
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):   0%|          | 0/51 [00:00<?, ?it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):   4%|▍         | 2/51 [00:00<00:03, 14.43it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):   8%|▊         | 4/51 [00:00<00:03, 14.62it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  12%|█▏        | 6/51 [00:00<00:03, 14.68it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  16%|█▌        | 8/51 [00:00<00:02, 14.74it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  20%|█▉        | 10/51 [00:00<00:02, 14.73it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  24%|██▎       | 12/51 [00:00<00:02, 14.73it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  27%|██▋       | 14/51 [00:00<00:02, 14.73it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  31%|███▏      | 16/51 [00:01<00:02, 14.65it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  35%|███▌      | 18/51 [00:01<00:02, 14.62it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  39%|███▉      | 20/51 [00:01<00:02, 14.64it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  43%|████▎     | 22/51 [00:01<00:01, 14.66it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  47%|████▋     | 24/51 [00:01<00:01, 14.63it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  51%|█████     | 26/51 [00:01<00:01, 14.64it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  55%|█████▍    | 28/51 [00:01<00:01, 14.56it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  59%|█████▉    | 30/51 [00:02<00:01, 14.54it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  63%|██████▎   | 32/51 [00:02<00:01, 14.55it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  67%|██████▋   | 34/51 [00:02<00:01, 14.31it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  71%|███████   | 36/51 [00:02<00:01, 14.37it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  75%|███████▍  | 38/51 [00:02<00:00, 14.43it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  78%|███████▊  | 40/51 [00:02<00:00, 14.41it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  82%|████████▏ | 42/51 [00:02<00:00, 14.15it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  86%|████████▋ | 44/51 [00:03<00:00, 14.27it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  90%|█████████ | 46/51 [00:03<00:00, 14.47it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  94%|█████████▍| 48/51 [00:03<00:00, 14.30it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  98%|█████████▊| 50/51 [00:03<00:00, 14.20it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 100%|██████████| 51/51 [00:03<00:00, 14.40it/s]
(EngineCore_DP0 pid=2659694)
Capturing CUDA graphs (decode, FULL):   0%|          | 0/35 [00:00<?, ?it/s]
Capturing CUDA graphs (decode, FULL):   6%|▌         | 2/35 [00:00<00:02, 13.00it/s]
Capturing CUDA graphs (decode, FULL):  11%|█▏        | 4/35 [00:00<00:02, 13.92it/s]
Capturing CUDA graphs (decode, FULL):  17%|█▋        | 6/35 [00:00<00:02, 14.30it/s]
Capturing CUDA graphs (decode, FULL):  23%|██▎       | 8/35 [00:00<00:01, 14.32it/s]
Capturing CUDA graphs (decode, FULL):  29%|██▊       | 10/35 [00:00<00:01, 14.44it/s]
Capturing CUDA graphs (decode, FULL):  34%|███▍      | 12/35 [00:00<00:01, 14.50it/s]
Capturing CUDA graphs (decode, FULL):  40%|████      | 14/35 [00:00<00:01, 14.54it/s]
Capturing CUDA graphs (decode, FULL):  46%|████▌     | 16/35 [00:01<00:01, 14.32it/s]
Capturing CUDA graphs (decode, FULL):  51%|█████▏    | 18/35 [00:01<00:01, 14.44it/s]
Capturing CUDA graphs (decode, FULL):  57%|█████▋    | 20/35 [00:01<00:01, 14.54it/s]
Capturing CUDA graphs (decode, FULL):  63%|██████▎   | 22/35 [00:01<00:00, 14.61it/s]
Capturing CUDA graphs (decode, FULL):  69%|██████▊   | 24/35 [00:01<00:00, 14.67it/s]
Capturing CUDA graphs (decode, FULL):  74%|███████▍  | 26/35 [00:01<00:00, 14.72it/s]
Capturing CUDA graphs (decode, FULL):  80%|████████  | 28/35 [00:01<00:00, 14.75it/s]
Capturing CUDA graphs (decode, FULL):  86%|████████▌ | 30/35 [00:02<00:00, 14.79it/s]
Capturing CUDA graphs (decode, FULL):  91%|█████████▏| 32/35 [00:02<00:00, 14.73it/s]
Capturing CUDA graphs (decode, FULL):  97%|█████████▋| 34/35 [00:02<00:00, 14.62it/s]
Capturing CUDA graphs (decode, FULL): 100%|██████████| 35/35 [00:02<00:00, 14.54it/s]
(EngineCore_DP0 pid=2659694) INFO 01-15 20:31:09 [gpu_model_runner.py:4587] Graph capturing finished in 7 secs, took 1.05 GiB
(EngineCore_DP0 pid=2659694) INFO 01-15 20:31:09 [core.py:259] init engine (profile, create kv cache, warmup model) took 38.98 seconds
INFO 01-15 20:31:12 [llm.py:360] Supported tasks: ['generate', 'transcription']
WARNING 01-15 20:31:12 [interfaces.py:920] Language 'as' is not natively supported by Gemma3nForConditionalGeneration; results may be less accurate. Supported languages: ['af', 'ar', 'hy', 'az', 'be', 'bs', 'bg', 'ca', 'zh', 'hr', 'cs', 'da', 'nl', 'en', 'et', 'fi', 'fr', 'gl', 'de', 'el', 'he', 'hi', 'hu', 'is', 'id', 'it', 'ja', 'kn', 'kk', 'ko', 'lv', 'lt', 'mk', 'ms', 'mr', 'mi', 'ne', 'no', 'fa', 'pl', 'pt', 'ro', 'ru', 'sr', 'sk', 'sl', 'es', 'sw', 'sv', 'tl', 'ta', 'th', 'tr', 'uk', 'ur', 'vi', 'cy']

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]
Adding requests: 100%|██████████| 1/1 [00:16<00:00, 16.33s/it]
Adding requests: 100%|██████████| 1/1 [00:16<00:00, 16.33s/it]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  3.54it/s, est. speed input: 733.94 toks/s, output: 77.99 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  3.54it/s, est. speed input: 733.94 toks/s, output: 77.99 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  3.54it/s, est. speed input: 733.94 toks/s, output: 77.99 toks/s]
WARNING 01-15 20:31:29 [interfaces.py:920] Language 'as' is not natively supported by Gemma3nForConditionalGeneration; results may be less accurate. Supported languages: ['af', 'ar', 'hy', 'az', 'be', 'bs', 'bg', 'ca', 'zh', 'hr', 'cs', 'da', 'nl', 'en', 'et', 'fi', 'fr', 'gl', 'de', 'el', 'he', 'hi', 'hu', 'is', 'id', 'it', 'ja', 'kn', 'kk', 'ko', 'lv', 'lt', 'mk', 'ms', 'mr', 'mi', 'ne', 'no', 'fa', 'pl', 'pt', 'ro', 'ru', 'sr', 'sk', 'sl', 'es', 'sw', 'sv', 'tl', 'ta', 'th', 'tr', 'uk', 'ur', 'vi', 'cy']

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]
Adding requests: 100%|██████████| 1/1 [00:00<00:00, 17.35it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  2.73it/s, est. speed input: 564.57 toks/s, output: 100.90 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  2.73it/s, est. speed input: 564.57 toks/s, output: 100.90 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  2.72it/s, est. speed input: 564.57 toks/s, output: 100.90 toks/s]
WARNING 01-15 20:31:30 [interfaces.py:920] Language 'as' is not natively supported by Gemma3nForConditionalGeneration; results may be less accurate. Supported languages: ['af', 'ar', 'hy', 'az', 'be', 'bs', 'bg', 'ca', 'zh', 'hr', 'cs', 'da', 'nl', 'en', 'et', 'fi', 'fr', 'gl', 'de', 'el', 'he', 'hi', 'hu', 'is', 'id', 'it', 'ja', 'kn', 'kk', 'ko', 'lv', 'lt', 'mk', 'ms', 'mr', 'mi', 'ne', 'no', 'fa', 'pl', 'pt', 'ro', 'ru', 'sr', 'sk', 'sl', 'es', 'sw', 'sv', 'tl', 'ta', 'th', 'tr', 'uk', 'ur', 'vi', 'cy']

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]
Adding requests: 100%|██████████| 1/1 [00:00<00:00, 21.39it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  3.31it/s, est. speed input: 685.49 toks/s, output: 102.65 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  3.31it/s, est. speed input: 685.49 toks/s, output: 102.65 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  3.31it/s, est. speed input: 685.49 toks/s, output: 102.65 toks/s]
WARNING 01-15 20:31:31 [interfaces.py:920] Language 'as' is not natively supported by Gemma3nForConditionalGeneration; results may be less accurate. Supported languages: ['af', 'ar', 'hy', 'az', 'be', 'bs', 'bg', 'ca', 'zh', 'hr', 'cs', 'da', 'nl', 'en', 'et', 'fi', 'fr', 'gl', 'de', 'el', 'he', 'hi', 'hu', 'is', 'id', 'it', 'ja', 'kn', 'kk', 'ko', 'lv', 'lt', 'mk', 'ms', 'mr', 'mi', 'ne', 'no', 'fa', 'pl', 'pt', 'ro', 'ru', 'sr', 'sk', 'sl', 'es', 'sw', 'sv', 'tl', 'ta', 'th', 'tr', 'uk', 'ur', 'vi', 'cy']

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]
Adding requests: 100%|██████████| 1/1 [00:00<00:00, 10.99it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  1.81it/s, est. speed input: 375.37 toks/s, output: 106.98 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  1.81it/s, est. speed input: 375.37 toks/s, output: 106.98 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  1.81it/s, est. speed input: 375.37 toks/s, output: 106.98 toks/s]
WARNING 01-15 20:31:32 [interfaces.py:920] Language 'as' is not natively supported by Gemma3nForConditionalGeneration; results may be less accurate. Supported languages: ['af', 'ar', 'hy', 'az', 'be', 'bs', 'bg', 'ca', 'zh', 'hr', 'cs', 'da', 'nl', 'en', 'et', 'fi', 'fr', 'gl', 'de', 'el', 'he', 'hi', 'hu', 'is', 'id', 'it', 'ja', 'kn', 'kk', 'ko', 'lv', 'lt', 'mk', 'ms', 'mr', 'mi', 'ne', 'no', 'fa', 'pl', 'pt', 'ro', 'ru', 'sr', 'sk', 'sl', 'es', 'sw', 'sv', 'tl', 'ta', 'th', 'tr', 'uk', 'ur', 'vi', 'cy']

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]
Adding requests: 100%|██████████| 1/1 [00:00<00:00,  8.44it/s]
Adding requests: 100%|██████████| 1/1 [00:00<00:00,  8.42it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  2.29it/s, est. speed input: 473.16 toks/s, output: 109.71 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  2.29it/s, est. sp…270897 tokens truncated…zh', 'hr', 'cs', 'da', 'nl', 'en', 'et', 'fi', 'fr', 'gl', 'de', 'el', 'he', 'hi', 'hu', 'is', 'id', 'it', 'ja', 'kn', 'kk', 'ko', 'lv', 'lt', 'mk', 'ms', 'mr', 'mi', 'ne', 'no', 'fa', 'pl', 'pt', 'ro', 'ru', 'sr', 'sk', 'sl', 'es', 'sw', 'sv', 'tl', 'ta', 'th', 'tr', 'uk', 'ur', 'vi', 'cy']

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]
Adding requests: 100%|██████████| 1/1 [00:00<00:00, 13.82it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  3.89it/s, est. speed input: 805.58 toks/s, output: 97.29 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  3.89it/s, est. speed input: 805.58 toks/s, output: 97.29 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  3.89it/s, est. speed input: 805.58 toks/s, output: 97.29 toks/s]
WARNING 01-15 20:46:18 [interfaces.py:920] Language 'te' is not natively supported by Gemma3nForConditionalGeneration; results may be less accurate. Supported languages: ['af', 'ar', 'hy', 'az', 'be', 'bs', 'bg', 'ca', 'zh', 'hr', 'cs', 'da', 'nl', 'en', 'et', 'fi', 'fr', 'gl', 'de', 'el', 'he', 'hi', 'hu', 'is', 'id', 'it', 'ja', 'kn', 'kk', 'ko', 'lv', 'lt', 'mk', 'ms', 'mr', 'mi', 'ne', 'no', 'fa', 'pl', 'pt', 'ro', 'ru', 'sr', 'sk', 'sl', 'es', 'sw', 'sv', 'tl', 'ta', 'th', 'tr', 'uk', 'ur', 'vi', 'cy']

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]
Adding requests: 100%|██████████| 1/1 [00:00<00:00, 14.07it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  2.98it/s, est. speed input: 617.38 toks/s, output: 101.40 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  2.98it/s, est. speed input: 617.38 toks/s, output: 101.40 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  2.98it/s, est. speed input: 617.38 toks/s, output: 101.40 toks/s]
WARNING 01-15 20:46:19 [interfaces.py:920] Language 'te' is not natively supported by Gemma3nForConditionalGeneration; results may be less accurate. Supported languages: ['af', 'ar', 'hy', 'az', 'be', 'bs', 'bg', 'ca', 'zh', 'hr', 'cs', 'da', 'nl', 'en', 'et', 'fi', 'fr', 'gl', 'de', 'el', 'he', 'hi', 'hu', 'is', 'id', 'it', 'ja', 'kn', 'kk', 'ko', 'lv', 'lt', 'mk', 'ms', 'mr', 'mi', 'ne', 'no', 'fa', 'pl', 'pt', 'ro', 'ru', 'sr', 'sk', 'sl', 'es', 'sw', 'sv', 'tl', 'ta', 'th', 'tr', 'uk', 'ur', 'vi', 'cy']

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]
Adding requests: 100%|██████████| 1/1 [00:00<00:00, 11.99it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  2.59it/s, est. speed input: 536.18 toks/s, output: 106.19 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  2.59it/s, est. speed input: 536.18 toks/s, output: 106.19 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  2.59it/s, est. speed input: 536.18 toks/s, output: 106.19 toks/s]
WARNING 01-15 20:46:20 [interfaces.py:920] Language 'te' is not natively supported by Gemma3nForConditionalGeneration; results may be less accurate. Supported languages: ['af', 'ar', 'hy', 'az', 'be', 'bs', 'bg', 'ca', 'zh', 'hr', 'cs', 'da', 'nl', 'en', 'et', 'fi', 'fr', 'gl', 'de', 'el', 'he', 'hi', 'hu', 'is', 'id', 'it', 'ja', 'kn', 'kk', 'ko', 'lv', 'lt', 'mk', 'ms', 'mr', 'mi', 'ne', 'no', 'fa', 'pl', 'pt', 'ro', 'ru', 'sr', 'sk', 'sl', 'es', 'sw', 'sv', 'tl', 'ta', 'th', 'tr', 'uk', 'ur', 'vi', 'cy']

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]
Adding requests: 100%|██████████| 1/1 [00:00<00:00, 14.29it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  2.40it/s, est. speed input: 497.08 toks/s, output: 108.06 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  2.40it/s, est. speed input: 497.08 toks/s, output: 108.06 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  2.40it/s, est. speed input: 497.08 toks/s, output: 108.06 toks/s]
WARNING 01-15 20:46:21 [interfaces.py:920] Language 'te' is not natively supported by Gemma3nForConditionalGeneration; results may be less accurate. Supported languages: ['af', 'ar', 'hy', 'az', 'be', 'bs', 'bg', 'ca', 'zh', 'hr', 'cs', 'da', 'nl', 'en', 'et', 'fi', 'fr', 'gl', 'de', 'el', 'he', 'hi', 'hu', 'is', 'id', 'it', 'ja', 'kn', 'kk', 'ko', 'lv', 'lt', 'mk', 'ms', 'mr', 'mi', 'ne', 'no', 'fa', 'pl', 'pt', 'ro', 'ru', 'sr', 'sk', 'sl', 'es', 'sw', 'sv', 'tl', 'ta', 'th', 'tr', 'uk', 'ur', 'vi', 'cy']

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]
Adding requests: 100%|██████████| 1/1 [00:00<00:00, 10.27it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  3.67it/s, est. speed input: 760.32 toks/s, output: 99.16 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  3.67it/s, est. speed input: 760.32 toks/s, output: 99.16 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  3.67it/s, est. speed input: 760.32 toks/s, output: 99.16 toks/s]
WARNING 01-15 20:46:22 [interfaces.py:920] Language 'te' is not natively supported by Gemma3nForConditionalGeneration; results may be less accurate. Supported languages: ['af', 'ar', 'hy', 'az', 'be', 'bs', 'bg', 'ca', 'zh', 'hr', 'cs', 'da', 'nl', 'en', 'et', 'fi', 'fr', 'gl', 'de', 'el', 'he', 'hi', 'hu', 'is', 'id', 'it', 'ja', 'kn', 'kk', 'ko', 'lv', 'lt', 'mk', 'ms', 'mr', 'mi', 'ne', 'no', 'fa', 'pl', 'pt', 'ro', 'ru', 'sr', 'sk', 'sl', 'es', 'sw', 'sv', 'tl', 'ta', 'th', 'tr', 'uk', 'ur', 'vi', 'cy']

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]
Adding requests: 100%|██████████| 1/1 [00:00<00:00, 24.85it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.15it/s, est. speed input: 859.74 toks/s, output: 95.52 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.15it/s, est. speed input: 859.74 toks/s, output: 95.52 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.15it/s, est. speed input: 859.74 toks/s, output: 95.52 toks/s]
WARNING 01-15 20:46:22 [interfaces.py:920] Language 'te' is not natively supported by Gemma3nForConditionalGeneration; results may be less accurate. Supported languages: ['af', 'ar', 'hy', 'az', 'be', 'bs', 'bg', 'ca', 'zh', 'hr', 'cs', 'da', 'nl', 'en', 'et', 'fi', 'fr', 'gl', 'de', 'el', 'he', 'hi', 'hu', 'is', 'id', 'it', 'ja', 'kn', 'kk', 'ko', 'lv', 'lt', 'mk', 'ms', 'mr', 'mi', 'ne', 'no', 'fa', 'pl', 'pt', 'ro', 'ru', 'sr', 'sk', 'sl', 'es', 'sw', 'sv', 'tl', 'ta', 'th', 'tr', 'uk', 'ur', 'vi', 'cy']

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]
Adding requests: 100%|██████████| 1/1 [00:00<00:00, 18.77it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  3.42it/s, est. speed input: 708.13 toks/s, output: 102.62 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  3.42it/s, est. speed input: 708.13 toks/s, output: 102.62 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  3.42it/s, est. speed input: 708.13 toks/s, output: 102.62 toks/s]
WARNING 01-15 20:46:23 [interfaces.py:920] Language 'te' is not natively supported by Gemma3nForConditionalGeneration; results may be less accurate. Supported languages: ['af', 'ar', 'hy', 'az', 'be', 'bs', 'bg', 'ca', 'zh', 'hr', 'cs', 'da', 'nl', 'en', 'et', 'fi', 'fr', 'gl', 'de', 'el', 'he', 'hi', 'hu', 'is', 'id', 'it', 'ja', 'kn', 'kk', 'ko', 'lv', 'lt', 'mk', 'ms', 'mr', 'mi', 'ne', 'no', 'fa', 'pl', 'pt', 'ro', 'ru', 'sr', 'sk', 'sl', 'es', 'sw', 'sv', 'tl', 'ta', 'th', 'tr', 'uk', 'ur', 'vi', 'cy']

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]
Adding requests: 100%|██████████| 1/1 [00:00<00:00, 12.74it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  3.04it/s, est. speed input: 628.72 toks/s, output: 97.19 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  3.04it/s, est. speed input: 628.72 toks/s, output: 97.19 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  3.04it/s, est. speed input: 628.72 toks/s, output: 97.19 toks/s]
WARNING 01-15 20:46:24 [interfaces.py:920] Language 'te' is not natively supported by Gemma3nForConditionalGeneration; results may be less accurate. Supported languages: ['af', 'ar', 'hy', 'az', 'be', 'bs', 'bg', 'ca', 'zh', 'hr', 'cs', 'da', 'nl', 'en', 'et', 'fi', 'fr', 'gl', 'de', 'el', 'he', 'hi', 'hu', 'is', 'id', 'it', 'ja', 'kn', 'kk', 'ko', 'lv', 'lt', 'mk', 'ms', 'mr', 'mi', 'ne', 'no', 'fa', 'pl', 'pt', 'ro', 'ru', 'sr', 'sk', 'sl', 'es', 'sw', 'sv', 'tl', 'ta', 'th', 'tr', 'uk', 'ur', 'vi', 'cy']

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]
Adding requests: 100%|██████████| 1/1 [00:00<00:00, 14.13it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  3.34it/s, est. speed input: 690.88 toks/s, output: 96.78 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  3.34it/s, est. speed input: 690.88 toks/s, output: 96.78 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  3.33it/s, est. speed input: 690.88 toks/s, output: 96.78 toks/s]
WARNING 01-15 20:46:25 [interfaces.py:920] Language 'te' is not natively supported by Gemma3nForConditionalGeneration; results may be less accurate. Supported languages: ['af', 'ar', 'hy', 'az', 'be', 'bs', 'bg', 'ca', 'zh', 'hr', 'cs', 'da', 'nl', 'en', 'et', 'fi', 'fr', 'gl', 'de', 'el', 'he', 'hi', 'hu', 'is', 'id', 'it', 'ja', 'kn', 'kk', 'ko', 'lv', 'lt', 'mk', 'ms', 'mr', 'mi', 'ne', 'no', 'fa', 'pl', 'pt', 'ro', 'ru', 'sr', 'sk', 'sl', 'es', 'sw', 'sv', 'tl', 'ta', 'th', 'tr', 'uk', 'ur', 'vi', 'cy']

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]
Adding requests: 100%|██████████| 1/1 [00:00<00:00, 16.18it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  3.44it/s, est. speed input: 711.64 toks/s, output: 85.94 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  3.44it/s, est. speed input: 711.64 toks/s, output: 85.94 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  3.43it/s, est. speed input: 711.64 toks/s, output: 85.94 toks/s]
WARNING 01-15 20:46:25 [interfaces.py:920] Language 'te' is not natively supported by Gemma3nForConditionalGeneration; results may be less accurate. Supported languages: ['af', 'ar', 'hy', 'az', 'be', 'bs', 'bg', 'ca', 'zh', 'hr', 'cs', 'da', 'nl', 'en', 'et', 'fi', 'fr', 'gl', 'de', 'el', 'he', 'hi', 'hu', 'is', 'id', 'it', 'ja', 'kn', 'kk', 'ko', 'lv', 'lt', 'mk', 'ms', 'mr', 'mi', 'ne', 'no', 'fa', 'pl', 'pt', 'ro', 'ru', 'sr', 'sk', 'sl', 'es', 'sw', 'sv', 'tl', 'ta', 'th', 'tr', 'uk', 'ur', 'vi', 'cy']

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]
Adding requests: 100%|██████████| 1/1 [00:00<00:00, 19.13it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  3.00it/s, est. speed input: 622.02 toks/s, output: 105.16 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  3.00it/s, est. speed input: 622.02 toks/s, output: 105.16 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  3.00it/s, est. speed input: 622.02 toks/s, output: 105.16 toks/s]
WARNING 01-15 20:46:26 [interfaces.py:920] Language 'te' is not natively supported by Gemma3nForConditionalGeneration; results may be less accurate. Supported languages: ['af', 'ar', 'hy', 'az', 'be', 'bs', 'bg', 'ca', 'zh', 'hr', 'cs', 'da', 'nl', 'en', 'et', 'fi', 'fr', 'gl', 'de', 'el', 'he', 'hi', 'hu', 'is', 'id', 'it', 'ja', 'kn', 'kk', 'ko', 'lv', 'lt', 'mk', 'ms', 'mr', 'mi', 'ne', 'no', 'fa', 'pl', 'pt', 'ro', 'ru', 'sr', 'sk', 'sl', 'es', 'sw', 'sv', 'tl', 'ta', 'th', 'tr', 'uk', 'ur', 'vi', 'cy']

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]
Adding requests: 100%|██████████| 1/1 [00:00<00:00, 23.89it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.42it/s, est. speed input: 914.56 toks/s, output: 97.19 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.42it/s, est. speed input: 914.56 toks/s, output: 97.19 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.41it/s, est. speed input: 914.56 toks/s, output: 97.19 toks/s]
WARNING 01-15 20:46:27 [interfaces.py:920] Language 'te' is not natively supported by Gemma3nForConditionalGeneration; results may be less accurate. Supported languages: ['af', 'ar', 'hy', 'az', 'be', 'bs', 'bg', 'ca', 'zh', 'hr', 'cs', 'da', 'nl', 'en', 'et', 'fi', 'fr', 'gl', 'de', 'el', 'he', 'hi', 'hu', 'is', 'id', 'it', 'ja', 'kn', 'kk', 'ko', 'lv', 'lt', 'mk', 'ms', 'mr', 'mi', 'ne', 'no', 'fa', 'pl', 'pt', 'ro', 'ru', 'sr', 'sk', 'sl', 'es', 'sw', 'sv', 'tl', 'ta', 'th', 'tr', 'uk', 'ur', 'vi', 'cy']

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]
Adding requests: 100%|██████████| 1/1 [00:00<00:00, 18.51it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  3.62it/s, est. speed input: 750.43 toks/s, output: 105.13 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  3.62it/s, est. speed input: 750.43 toks/s, output: 105.13 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  3.62it/s, est. speed input: 750.43 toks/s, output: 105.13 toks/s]
WARNING 01-15 20:46:27 [interfaces.py:920] Language 'te' is not natively supported by Gemma3nForConditionalGeneration; results may be less accurate. Supported languages: ['af', 'ar', 'hy', 'az', 'be', 'bs', 'bg', 'ca', 'zh', 'hr', 'cs', 'da', 'nl', 'en', 'et', 'fi', 'fr', 'gl', 'de', 'el', 'he', 'hi', 'hu', 'is', 'id', 'it', 'ja', 'kn', 'kk', 'ko', 'lv', 'lt', 'mk', 'ms', 'mr', 'mi', 'ne', 'no', 'fa', 'pl', 'pt', 'ro', 'ru', 'sr', 'sk', 'sl', 'es', 'sw', 'sv', 'tl', 'ta', 'th', 'tr', 'uk', 'ur', 'vi', 'cy']

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]
Adding requests: 100%|██████████| 1/1 [00:00<00:00, 14.56it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  3.14it/s, est. speed input: 649.51 toks/s, output: 103.54 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  3.14it/s, est. speed input: 649.51 toks/s, output: 103.54 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  3.14it/s, est. speed input: 649.51 toks/s, output: 103.54 toks/s]
WARNING 01-15 20:46:28 [interfaces.py:920] Language 'te' is not natively supported by Gemma3nForConditionalGeneration; results may be less accurate. Supported languages: ['af', 'ar', 'hy', 'az', 'be', 'bs', 'bg', 'ca', 'zh', 'hr', 'cs', 'da', 'nl', 'en', 'et', 'fi', 'fr', 'gl', 'de', 'el', 'he', 'hi', 'hu', 'is', 'id', 'it', 'ja', 'kn', 'kk', 'ko', 'lv', 'lt', 'mk', 'ms', 'mr', 'mi', 'ne', 'no', 'fa', 'pl', 'pt', 'ro', 'ru', 'sr', 'sk', 'sl', 'es', 'sw', 'sv', 'tl', 'ta', 'th', 'tr', 'uk', 'ur', 'vi', 'cy']

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]
Adding requests: 100%|██████████| 1/1 [00:00<00:00, 18.39it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  3.66it/s, est. speed input: 757.29 toks/s, output: 102.43 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  3.66it/s, est. speed input: 757.29 toks/s, output: 102.43 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  3.65it/s, est. speed input: 757.29 toks/s, output: 102.43 toks/s]
Transcribing ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:15:17
[rank0]:[W115 20:46:29.930845909 ProcessGroupNCCL.cpp:1524] Warning: WARNING: destroy_process_group() was not called before program exit, which can leak resources. For more info, please see https://pytorch.org/docs/stable/distributed.html#shutdown (function operator())
Wrote: outputs/bench/maya_hf_v2_100/gemma3n_e2b_vllm/results.jsonl
Wrote: outputs/bench/maya_hf_v2_100/gemma3n_e2b_vllm/summary.json
Wrote: outputs/bench/maya_hf_v2_100/gemma3n_e2b_vllm/eval.json
{
  "hypothesis": "final",
  "pairs_evaluated": 1200,
  "wer_mean": 0.6356810246425638,
  "cer_mean": 0.37527269011786013,
  "wer_p50": 0.4666666666666667,
  "wer_p90": 1.1428571428571428,
  "wer_p95": 1.3571428571428572,
  "cer_p50": 0.14285714285714285,
  "cer_p90": 1.0,
  "cer_p95": 1.0338983050847457,
  "segment_error_rate": 0.925,
  "pct_wer_gt_5pct": 0.9125,
  "pct_wer_gt_10pct": 0.8483333333333334,
  "wer_by_language_mean": {
    "asm": 0.7798161553305052,
    "ben": 0.34788189588189583,
    "eng": 0.09221403248815119,
    "guj": 0.8734913974913977,
    "hin": 0.1880531968031968,
    "kan": 0.4581712176712177,
    "mal": 0.6506648906648906,
    "mar": 0.3879127539127538,
    "ory": 1.2003219835719836,
    "pan": 1.3496818181818184,
    "tam": 0.788172993672994,
    "tel": 0.51178996003996
  },
  "wer_by_decision_mean": {
    "accept": 0.6373567814067788,
    "human": 0.6131289727675269
  },
  "wer_by_duration_bucket_mean": {
    "0-3s": 0.6460317460317461,
    "10-30s": 0.5514456890139965,
    "3-10s": 0.6525738867533283
  },
  "cer_by_language_mean": {
    "asm": 0.359063074294767,
    "ben": 0.14995087198750315,
    "eng": 0.05342006855623093,
    "guj": 0.7460013714413222,
    "hin": 0.07156804152129903,
    "kan": 0.15175300360159957,
    "mal": 0.28169531754233906,
    "mar": 0.20204553457456698,
    "ory": 1.0035575979374078,
    "pan": 1.1281770687781438,
    "tam": 0.15601961641855255,
    "tel": 0.20002071476059552
  },
  "cer_by_decision_mean": {
    "accept": 0.3723156814455201,
    "human": 0.415067614057667
  },
  "cer_by_duration_bucket_mean": {
    "0-3s": 0.6092436974789915,
    "10-30s": 0.2263381189243435,
    "3-10s": 0.4036202401552979
  },
  "oracle": {
    "enabled": true,
    "oracle_best_wer_mean": 0.6356810246425638,
    "selection_gap_wer_mean": 0.0,
    "selection_gap_wer_p90": 0.0
  }
}


Summary: auto


**Planning results.md update**

I'm deciding to add a section in results.md summarizing the JSON output locations for the sample runs, maintaining consistency with earlier notes, while deferring detailed analysis for later.


**Deciding minimal results.md update**

I'll add a brief section listing output JSON paths and noting eval.json presence without interpretation, keeping results.md minimal and aligned with the user's "store first" instruction while avoiding premature analysis.


Input:
*** Begin Patch
*** Update File: results.md
@@
 ## Notes
 - Gemma-3n now uses a per-audio max_tokens cap (12 tokens/sec, max 1024) and a Gemma stop token (`<end_of_turn>`) to prevent runaway decoding.
 - Best WER on this Hindi bench is currently Voxtral-Small (14.87%).
+
+## Multilingual 100-sample Runs (Transcription Dumps)
+Bench subset: `data/bench/maya_hf_v2_100` (12 languages x 100 samples). These runs are for JSON dumps first; eval.json was generated but not analyzed yet.
+
+- IndicConformer 600M multilingual (HF): `outputs/bench/maya_hf_v2_100/indicconformer_600m_hf/results.jsonl`
+  - Note: English segments failed with `KeyError:'joint_post_net_en'` (100 error rows).
+- Whisper large-v3: `outputs/bench/maya_hf_v2_100/whisper_large_v3/results.jsonl`
+- Voxtral-Small-24B-2507: `outputs/bench/maya_hf_v2_100/voxtral_small_vllm/results.jsonl`
+  - Note: Voxtral only natively supports a small language set (hi/en/...) so non-supported languages likely degrade.
+- Gemma-3n-E2B-it: `outputs/bench/maya_hf_v2_100/gemma3n_e2b_vllm/results.jsonl`
*** End Patch


Output: {"output":"Success. Updated the following files:\nM results.md\n","metadata":{"exit_code":0,"duration_seconds":0.0}}


**Updating plan step with summary**


Add HF IndicConformer backend + CLI/pipeline wiring

Create 100-per-language bench subset and thresholds config

Run individual model transcriptions on 12-language bench (IndicConformer 600m, Whisper large-v3, Voxtral, Gemma)

Summarize output locations for JSON results


Output: Plan updated


**Summarizing final changes and outputs**

The HF IndicConformer backend, new CLI options, config file, and benchmark subset were added, with JSON outputs saved at `configs/bench_v2_100_thresholds.json` and `data/bench/maya_hf_v2_100`. Note that English recognition failed for IndicConformer. No tests were executed, only runs. Next steps could include optional evaluation runs or JSON merging, but no rerun prompt is needed now.


**Detailing code changes, outputs, and next steps**

Added HF IndicConformer backend and new CLI options (`--hf-indicconformer-model`, `--hf-indicconformer-decode`) in `maya_transcribe/asr.py`, `pipeline.py`, and `cli.py`. Created a new benchmark subset with 100 samples per language using symlinks at `data/bench/maya_hf_v2_100`, configured via `configs/bench_v2_100_thresholds.json` to control routing and Whisper sampling.

Ran transcription with four models on this subset; results JSON files are located in `outputs/bench/maya_hf_v2_100/*/results.jsonl`, with evaluation files at `*/eval.json`. The `results.md` file was updated with a new section summarizing experiments.

Note that IndicConformer has about 100 English result rows with errors (`error` field, e.g., KeyError in `joint_post_net_en`) at `outputs/bench/maya_hf_v2_100/indicconformer_600m_hf/results.jsonl`. Gemma and Voxtral issued warnings about unsupported languages implying likely poor performance.

Next suggested steps:

1) Review JSON outputs and generate per-language transcript summaries or WER/CER tables to identify best-performing models.
2) Prepare ensemble approaches like ROVER or judge model evaluation from these outputs.
3) Optionally rerun IndicConformer using CTC decode or skip English to address error rows.

Let me know how you'd like to proceed with analysis or reruns.


Added an HF IndicConformer backend + CLI wiring, built a 12‑language 100‑sample bench subset, and ran all four individual model jobs so you have JSON dumps ready to analyze. The English split failed for the HF IndicConformer model, so those 100 rows are error records.

Details and outputs
- New HF backend + wiring: `maya_transcribe/asr.py`, `maya_transcribe/pipeline.py`, `maya_transcribe/cli.py`
- Bench subset (100 per language): `data/bench/maya_hf_v2_100/refs.jsonl` and `data/bench/maya_hf_v2_100/audio`
- Routing/sampling config used: `configs/bench_v2_100_thresholds.json`
- JSON dumps:
  - HF IndicConformer 600M: `outputs/bench/maya_hf_v2_100/indicconformer_600m_hf/results.jsonl`
    - English failures logged as `error` rows: `KeyError:'joint_post_net_en'` (100 items)
  - Whisper large‑v3: `outputs/bench/maya_hf_v2_100/whisper_large_v3/results.jsonl`
  - Voxtral‑Small: `outputs/bench/maya_hf_v2_100/voxtral_small_vllm/results.jsonl`
  - Gemma‑3n‑E2B: `outputs/bench/maya_hf_v2_100/gemma3n_e2b_vllm/results.jsonl`
- Notes recorded in `results.md`

If you want, I can do one of these next:
1) Generate a per‑language WER/CER table from the `eval.json` files and rank the best model per language.
2) Build a first‑pass ensemble (ROVER or judge) using these dumps and compare against best‑single.
3) Rerun IndicConformer with `ctc` decode or skip English to avoid the `joint_post_net_en` failure.


# Context from my IDE setup:

## Active file: results.md

## Active selection of the file:
## Multilingual 100-sample Runs (Transcription Dumps)
Bench subset: `data/bench/maya_hf_v2_100` (12 languages x 100 samples). These runs are for JSON dumps first; eval.json was generated but not analyzed yet.

- IndicConformer 600M multilingual (HF): `outputs/bench/maya_hf_v2_100/indicconformer_600m_hf/results.jsonl`
  - Note: English segments failed with `KeyError:'joint_post_net_en'` (100 error rows).
- Whisper large-v3: `outputs/bench/maya_hf_v2_100/whisper_large_v3/results.jsonl`
- Voxtral-Small-24B-2507: `outputs/bench/maya_hf_v2_100/voxtral_small_vllm/results.jsonl`
  - Note: Voxtral only natively supports a small language set (hi/en/...) so non-supported languages likely degrade.
- Gemma-3n-E2B-it: `outputs/bench/maya_hf_v2_100/gemma3n_e2b_vllm/results.jsonl`

## Open tabs:
- results.md: results.md
- asr.py: maya_transcribe/asr.py
- .env: .env
- Prompt.md: Prompt.md
- eval.json: outputs/bench/maya_hf_hin_v2/gemma3n_e2b_vllm/eval.json

## My request for Codex:
Analyse and add metrics into results.md for this new section as well. create a table for per lang and model wise performance. and we will analyse the samples later. for now add this new section.

## Multilingual 100-sample Runs (Transcription Dumps)
Bench subset: `data/bench/maya_hf_v2_100` (12 languages x 100 samples). These runs are for JSON dumps first; eval.json was generated but not analyzed yet.

- IndicConformer 600M multilingual (HF): `outputs/bench/maya_hf_v2_100/indicconformer_600m_hf/results.jsonl`
  - Note: English segments failed with `KeyError:'joint_post_net_en'` (100 error rows).
- Whisper large-v3: `outputs/bench/maya_hf_v2_100/whisper_large_v3/results.jsonl`
- Voxtral-Small-24B-2507: `outputs/bench/maya_hf_v2_100/voxtral_small_vllm/results.jsonl`
  - Note: Voxtral only natively supports a small language set (hi/en/...) so non-supported languages likely degrade.
- Gemma-3n-E2B-it: `outputs/bench/maya_hf_v2_100/gemma3n_e2b_vllm/results.jsonl`


**Planning parsing and table creation**

I'll parse each model's eval.json to compute per-language WER percentages and create an ASCII table for results.md showing language-wise and model-wise performance.


python3 - <<'PY'
import json
from pathlib import Path

runs = {
    'IndicConformer-600M (HF)': Path('outputs/bench/maya_hf_v2_100/indicconformer_600m_hf/eval.json'),
    'Whisper large-v3': Path('outputs/bench/maya_hf_v2_100/whisper_large_v3/eval.json'),
    'Voxtral-Small': Path('outputs/bench/maya_hf_v2_100/voxtral_small_vllm/eval.json'),
    'Gemma-3n-E2B': Path('outputs/bench/maya_hf_v2_100/gemma3n_e2b_vllm/eval.json'),
}

langs = ['asm','ben','eng','guj','hin','kan','mal','mar','ory','pan','tam','tel']

# Load WERs
wer = {lang: {} for lang in langs}
overall = {}
for name, path in runs.items():
    data = json.loads(path.read_text())
    overall[name] = data.get('wer_mean')
    by_lang = data.get('wer_by_language_mean') or {}
    for lang in langs:
        if lang in by_lang:
            wer[lang][name] = by_lang[lang]
        else:
            wer[lang][name] = None

# Print in tabular
header = ['lang'] + list(runs.keys())
print('\t'.join(header))
for lang in langs:
    row = [lang]
    for name in runs.keys():
        val = wer[lang][name]
        if val is None:
            row.append('n/a')
        else:
            row.append(f"{val*100:.1f}%")
    print('\t'.join(row))

print('overall')
for name, v in overall.items():
    print(name, f"{v*100:.1f}%" if v is not None else 'n/a')
PY


Output:
Exit code: 0
Wall time: 0 seconds
Output:
lang	IndicConformer-600M (HF)	Whisper large-v3	Voxtral-Small	Gemma-3n-E2B
asm	19.9%	95.9%	107.9%	78.0%
ben	11.2%	64.2%	37.4%	34.8%
eng	n/a	4.5%	3.8%	9.2%
guj	11.7%	49.3%	49.8%	87.3%
hin	9.2%	30.9%	14.9%	18.8%
kan	14.3%	71.7%	49.0%	45.8%
mal	29.9%	122.5%	62.6%	65.1%
mar	14.6%	79.7%	39.9%	38.8%
ory	16.3%	115.5%	130.3%	120.0%
pan	12.6%	64.7%	34.2%	135.0%
tam	20.1%	50.9%	52.7%	78.8%
tel	24.4%	79.0%	47.5%	51.2%
overall
IndicConformer-600M (HF) 16.7%
Whisper large-v3 69.1%
Voxtral-Small 52.5%
Gemma-3n-E2B 63.6%


**Planning results.md update**

I'm preparing to add a metrics table with per-language and per-model WER percentages plus overall WER into the results.md section on transcription dumps, noting IndicConformer English errors as "n/a" and allowing WER values over 100%.


**Adding detailed WER notes and table**

I'm updating results.md with a "WER by language (mean)" subsection containing the ASCII table and an overall row, noting IndicConformer’s missing English results due to errors and computing overall WER on 1100 segments; I'll include cautions on unsupported languages and keep the table consistent and clear.


Input:
*** Begin Patch
*** Update File: results.md
@@
 ## Multilingual 100-sample Runs (Transcription Dumps)
 Bench subset: `data/bench/maya_hf_v2_100` (12 languages x 100 samples). These runs are for JSON dumps first; eval.json was generated but not analyzed yet.
@@
 - Gemma-3n-E2B-it: `outputs/bench/maya_hf_v2_100/gemma3n_e2b_vllm/results.jsonl`
+
+### Per-language WER (mean, %)
+Source: `eval.json` from each run. Note: IndicConformer HF failed on English (`n/a` below), so its overall WER is computed on 1100 segments.
+
+| Lang | IndicConformer-600M (HF) | Whisper large-v3 | Voxtral-Small | Gemma-3n-E2B |
+| --- | --- | --- | --- | --- |
+| asm | 19.9 | 95.9 | 107.9 | 78.0 |
+| ben | 11.2 | 64.2 | 37.4 | 34.8 |
+| eng | n/a | 4.5 | 3.8 | 9.2 |
+| guj | 11.7 | 49.3 | 49.8 | 87.3 |
+| hin | 9.2 | 30.9 | 14.9 | 18.8 |
+| kan | 14.3 | 71.7 | 49.0 | 45.8 |
+| mal | 29.9 | 122.5 | 62.6 | 65.1 |
+| mar | 14.6 | 79.7 | 39.9 | 38.8 |
+| ory | 16.3 | 115.5 | 130.3 | 120.0 |
+| pan | 12.6 | 64.7 | 34.2 | 135.0 |
+| tam | 20.1 | 50.9 | 52.7 | 78.8 |
+| tel | 24.4 | 79.0 | 47.5 | 51.2 |
+| overall | 16.7 | 69.1 | 52.5 | 63.6 |
*** End Patch


Output: {"output":"Success. Updated the following files:\nM results.md\n","metadata":{"exit_code":0,"duration_seconds":0.0}}


**Summarizing metric addition and analysis**

Added a per-language WER table to results.md based on eval.json, highlighting IndicConformer’s strong results across most Indic languages but missing English data; overall performance is included as well. Noted new config and benchmark subset files added; all changes and metrics are documented with file paths as requested.


**Suggesting cautious next steps**

Noted `results.md` was updated with a per-language WER table from eval.json, including overall scores and a mention that IndicConformer lacks English data. Suggested gentle next steps without pushing: rerun IndicConformer English with CTC decode, consider per-language best-of ensemble, or analyze the shared 1100-sample subset later.


Added the per-language WER table for the multilingual 100‑sample runs into `results.md`, sourced from each run’s `eval.json` and including an overall row plus the English gap note for HF IndicConformer.

Details
- Updated section in `results.md` with the per‑language WER (%) table for IndicConformer‑600M (HF), Whisper large‑v3, Voxtral‑Small, and Gemma‑3n‑E2B.
- The table explicitly marks English as `n/a` for HF IndicConformer and notes its overall WER is on 1100 segments.

If you want, I can also add a “shared‑subset overall” row (excluding English) so all models are compared on the same 1100 clips.


# Context from my IDE setup:

## Active file: results.md

## Open tabs:
- results.md: results.md
- asr.py: maya_transcribe/asr.py
- .env: .env
- Prompt.md: Prompt.md
- eval.json: outputs/bench/maya_hf_hin_v2/gemma3n_e2b_vllm/eval.json

## My request for Codex:
Can you now, cleanup the repo. keep the data, results, and all setup. just clean the code files where we didnt had to use them at all. Only required files are files that we used to generate transcriptions for the recent run which are needed for the voxtral, gemma, indicconformer, whisper-large v3 only. everything else is not needed. cleanup the files that we no longer need. just keep files and code that we needed to reproduce the recent run.


Summary: auto


**Requesting clarification before cleanup**

I'll ask the user which specific files or components to keep or remove, noting dependencies and risks of deleting code unexpectedly, and suggest creating a backup or branch before any cleanup.


**Confirming dependency and usage before pruning**

Noting that LID code is still required for reproducing recent runs despite prefer-path-lang settings, and that pipeline uses rover, verify, and text normalization components, so these must be kept; next, I'll list repo files to identify candidates unused in recent runs for further review.