Output:
Exit code: 0
Wall time: 237.4 seconds
Total output lines: 494
Output:
/home/ubuntu/.local/lib/python3.10/site-packages/torch/cuda/__init__.py:63: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you.
  import pynvml  # type: ignore[import]
2026-01-15 04:58:00.589486: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
/home/ubuntu/.local/lib/python3.10/site-packages/google/api_core/_python_version_support.py:266: FutureWarning: You are using a Python version (3.10.12) which Google will stop supporting in new releases of google.api_core once it reaches its end of life (2026-10-04). Please upgrade to the latest Python version, or at least Python 3.11, to continue receiving updates for google.api_core past that date.
  warnings.warn(message, FutureWarning)
INFO 01-15 04:58:07 [utils.py:253] non-default args: {'trust_remote_code': True, 'max_model_len': 4096, 'disable_log_stats': True, 'limit_mm_per_prompt': {'image': 0, 'audio': 1}, 'model': 'google/gemma-3n-E2B-it'}
The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
INFO 01-15 04:58:08 [model.py:514] Resolved architecture: Gemma3nForConditionalGeneration
INFO 01-15 04:58:08 [model.py:1661] Using max model len 4096
INFO 01-15 04:58:08 [scheduler.py:230] Chunked prefill is enabled with max_num_batched_tokens=8192.
WARNING 01-15 04:58:10 [system_utils.py:136] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: CUDA is initialized
/home/ubuntu/.local/lib/python3.10/site-packages/torch/cuda/__init__.py:63: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you.
  import pynvml  # type: ignore[import]
2026-01-15 04:58:15.389641: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
/home/ubuntu/.local/lib/python3.10/site-packages/google/api_core/_python_version_support.py:266: FutureWarning: You are using a Python version (3.10.12) which Google will stop supporting in new releases of google.api_core once it reaches its end of life (2026-10-04). Please upgrade to the latest Python version, or at least Python 3.11, to continue receiving updates for google.api_core past that date.
  warnings.warn(message, FutureWarning)
(EngineCore_DP0 pid=2055571) INFO 01-15 04:58:19 [core.py:93] Initializing a V1 LLM engine (v0.13.0) with config: model='google/gemma-3n-E2B-it', speculative_config=None, tokenizer='google/gemma-3n-E2B-it', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, tokenizer_revision=None, trust_remote_code=True, dtype=torch.bfloat16, max_seq_len=4096, download_dir=None, load_format=auto, tensor_parallel_size=1, pipeline_parallel_size=1, data_parallel_size=1, disable_custom_all_reduce=False, quantization=None, enforce_eager=False, kv_cache_dtype=auto, device_config=cuda, structured_outputs_config=StructuredOutputsConfig(backend='auto', disable_fallback=False, disable_any_whitespace=False, disable_additional_properties=False, reasoning_parser='', reasoning_parser_plugin='', enable_in_reasoning=False), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None, kv_cache_metrics=False, kv_cache_metrics_sample=0.01, cudagraph_metrics=False, enable_layerwise_nvtx_tracing=False), seed=0, served_model_name=google/gemma-3n-E2B-it, enable_prefix_caching=True, enable_chunked_prefill=True, pooler_config=None, compilation_config={'level': None, 'mode': <CompilationMode.VLLM_COMPILE: 3>, 'debug_dump_path': None, 'cache_dir': '', 'compile_cache_save_format': 'binary', 'backend': 'inductor', 'custom_ops': ['none'], 'splitting_ops': ['vllm::unified_attention', 'vllm::unified_attention_with_output', 'vllm::unified_mla_attention', 'vllm::unified_mla_attention_with_output', 'vllm::mamba_mixer2', 'vllm::mamba_mixer', 'vllm::short_conv', 'vllm::linear_attention', 'vllm::plamo2_mamba_mixer', 'vllm::gdn_attention_core', 'vllm::kda_attention', 'vllm::sparse_attn_indexer'], 'compile_mm_encoder': False, 'compile_sizes': [], 'compile_ranges_split_points': [8192], 'inductor_compile_config': {'enable_auto_functionalized_v2': False, 'combo_kernels': True, 'benchmark_combo_kernel': True}, 'inductor_passes': {}, 'cudagraph_mode': <CUDAGraphMode.FULL_AND_PIECEWISE: (2, 1)>, 'cudagraph_num_of_warmups': 1, 'cudagraph_capture_sizes': [1, 2, 4, 8, 16, 24, 32, 40, 48, 56, 64, 72, 80, 88, 96, 104, 112, 120, 128, 136, 144, 152, 160, 168, 176, 184, 192, 200, 208, 216, 224, 232, 240, 248, 256, 272, 288, 304, 320, 336, 352, 368, 384, 400, 416, 432, 448, 464, 480, 496, 512], 'cudagraph_copy_inputs': False, 'cudagraph_specialize_lora': True, 'use_inductor_graph_partition': False, 'pass_config': {'fuse_norm_quant': False, 'fuse_act_quant': False, 'fuse_attn_quant': False, 'eliminate_noops': True, 'enable_sp': False, 'fuse_gemm_comms': False, 'fuse_allreduce_rms': False}, 'max_cudagraph_capture_size': 512, 'dynamic_shapes_config': {'type': <DynamicShapesType.BACKED: 'backed'>, 'evaluate_guards': False}, 'local_cache_dir': None}
(EngineCore_DP0 pid=2055571) INFO 01-15 04:58:22 [parallel_state.py:1203] world_size=1 rank=0 local_rank=0 distributed_init_method=tcp://216.81.248.184:52035 backend=nccl
(EngineCore_DP0 pid=2055571) INFO 01-15 04:58:22 [parallel_state.py:1411] rank 0 in world size 1 is assigned as DP rank 0, PP rank 0, PCP rank 0, TP rank 0, EP rank 0
(EngineCore_DP0 pid=2055571) INFO 01-15 04:58:36 [gpu_model_runner.py:3562] Starting to load model google/gemma-3n-E2B-it...
(EngineCore_DP0 pid=2055571) INFO 01-15 04:58:36 [cuda.py:351] Using FLASH_ATTN attention backend out of potential backends: ('FLASH_ATTN', 'FLASHINFER', 'TRITON_ATTN', 'FLEX_ATTENTION')
(EngineCore_DP0 pid=2055571)
Loading safetensors checkpoint shards:   0% Completed | 0/3 [00:00<?, ?it/s]
(EngineCore_DP0 pid=2055571)
Loading safetensors checkpoint shards:  33% Completed | 1/3 [00:00<00:00,  2.13it/s]
(EngineCore_DP0 pid=2055571)
Loading safetensors checkpoint shards:  67% Completed | 2/3 [00:01<00:00,  1.39it/s]
(EngineCore_DP0 pid=2055571)
Loading safetensors checkpoint shards: 100% Completed | 3/3 [00:02<00:00,  1.38it/s]
(EngineCore_DP0 pid=2055571)
Loading safetensors checkpoint shards: 100% Completed | 3/3 [00:02<00:00,  1.43it/s]
(EngineCore_DP0 pid=2055571)
(EngineCore_DP0 pid=2055571) INFO 01-15 04:58:39 [default_loader.py:308] Loading weights took 2.22 seconds
(EngineCore_DP0 pid=2055571) INFO 01-15 04:58:40 [gpu_model_runner.py:3659] Model loading took 10.2839 GiB memory and 3.066085 seconds
(EngineCore_DP0 pid=2055571) INFO 01-15 04:58:40 [gpu_model_runner.py:4446] Encoder cache will be initialized with a budget of 8192 tokens, and profiled with 43 audio items of the maximum feature size.
(EngineCore_DP0 pid=2055571) INFO 01-15 04:58:53 [backends.py:643] Using cache directory: /home/ubuntu/.cache/vllm/torch_compile_cache/1a387ba34f/rank_0_0/backbone for vLLM's torch.compile
(EngineCore_DP0 pid=2055571) INFO 01-15 04:58:53 [backends.py:703] Dynamo bytecode transform time: 12.73 s
(EngineCore_DP0 pid=2055571) INFO 01-15 04:59:08 [backends.py:226] Directly load the compiled graph(s) for compile range (1, 8192) from the cache, took 7.143 s
(EngineCore_DP0 pid=2055571) INFO 01-15 04:59:08 [monitor.py:34] torch.compile takes 19.88 s in total
(EngineCore_DP0 pid=2055571) INFO 01-15 04:59:10 [gpu_worker.py:375] Available KV cache memory: 58.55 GiB
(EngineCore_DP0 pid=2055571) INFO 01-15 04:59:10 [kv_cache_utils.py:1291] GPU KV cache size: 1,534,896 tokens
(EngineCore_DP0 pid=2055571) INFO 01-15 04:59:10 [kv_cache_utils.py:1296] Maximum concurrency for 4,096 tokens per request: 373.56x
(EngineCore_DP0 pid=2055571)
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):   0%|          | 0/51 [00:00<?, ?it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):   4%|▍         | 2/51 [00:00<00:03, 14.65it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):   8%|▊         | 4/51 [00:00<00:03, 14.89it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  12%|█▏        | 6/51 [00:00<00:03, 14.93it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  16%|█▌        | 8/51 [00:00<00:02, 14.99it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  20%|█▉        | 10/51 [00:00<00:02, 14.96it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  24%|██▎       | 12/51 [00:00<00:02, 14.97it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  27%|██▋       | 14/51 [00:00<00:02, 14.96it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  31%|███▏      | 16/51 [00:01<00:02, 14.87it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  35%|███▌      | 18/51 [00:01<00:02, 14.82it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  39%|███▉      | 20/51 [00:01<00:02, 14.77it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  43%|████▎     | 22/51 [00:01<00:01, 14.77it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  47%|████▋     | 24/51 [00:01<00:01, 14.79it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  51%|█████     | 26/51 [00:01<00:01, 14.75it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  55%|█████▍    | 28/51 [00:01<00:01, 14.60it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  59%|█████▉    | 30/51 [00:02<00:01, 14.59it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  63%|██████▎   | 32/51 [00:02<00:01, 14.59it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  67%|██████▋   | 34/51 [00:02<00:01, 14.44it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  71%|███████   | 36/51 [00:02<00:01, 14.52it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  75%|███████▍  | 38/51 [00:02<00:00, 14.58it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  78%|███████▊  | 40/51 [00:02<00:00, 14.56it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  82%|████████▏ | 42/51 [00:02<00:00, 14.60it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  86%|████████▋ | 44/51 [00:02<00:00, 14.61it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  90%|█████████ | 46/51 [00:03<00:00, 14.64it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  94%|█████████▍| 48/51 [00:03<00:00, 14.57it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  98%|█████████▊| 50/51 [00:03<00:00, 14.52it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 100%|██████████| 51/51 [00:03<00:00, 14.61it/s]
(EngineCore_DP0 pid=2055571)
Capturing CUDA graphs (decode, FULL):   0%|          | 0/35 [00:00<?, ?it/s]
Capturing CUDA graphs (decode, FULL):   6%|▌         | 2/35 [00:00<00:02, 13.11it/s]
Capturing CUDA graphs (decode, FULL):  11%|█▏        | 4/35 [00:00<00:02, 14.20it/s]
Capturing CUDA graphs (decode, FULL):  17%|█▋        | 6/35 [00:00<00:01, 14.57it/s]
Capturing CUDA graphs (decode, FULL):  23%|██▎       | 8/35 [00:00<00:01, 14.65it/s]
Capturing CUDA graphs (decode, FULL):  29%|██▊       | 10/35 [00:00<00:01, 14.76it/s]
Capturing CUDA graphs (decode, FULL):  34%|███▍      | 12/35 [00:00<00:01, 14.84it/s]
Capturing CUDA graphs (decode, FULL):  40%|████      | 14/35 [00:00<00:01, 14.87it/s]
Capturing CUDA graphs (decode, FULL):  46%|████▌     | 16/35 [00:01<00:01, 14.87it/s]
Capturing CUDA graphs (decode, FULL):  51%|█████▏    | 18/35 [00:01<00:01, 14.84it/s]
Capturing CUDA graphs (decode, FULL):  57%|█████▋    | 20/35 [00:01<00:01, 14.91it/s]
Capturing CUDA graphs (decode, FULL):  63%|██████▎   | 22/35 [00:01<00:00, 14.99it/s]
Capturing CUDA graphs (decode, FULL):  69%|██████▊   | 24/35 [00:01<00:00, 15.04it/s]
Capturing CUDA graphs (decode, FULL):  74%|███████▍  | 26/35 [00:01<00:00, 15.04it/s]
Capturing CUDA graphs (decode, FULL):  80%|████████  | 28/35 [00:01<00:00, 15.04it/s]
Capturing CUDA graphs (decode, FULL):  86%|████████▌ | 30/35 [00:02<00:00, 15.09it/s]
Capturing CUDA graphs (decode, FULL):  91%|█████████▏| 32/35 [00:02<00:00, 15.03it/s]
Capturing CUDA graphs (decode, FULL):  97%|█████████▋| 34/35 [00:02<00:00, 14.97it/s]
Capturing CUDA graphs (decode, FULL): 100%|██████████| 35/35 [00:02<00:00, 14.88it/s]
(EngineCore_DP0 pid=2055571) INFO 01-15 04:59:18 [gpu_model_runner.py:4587] Graph capturing finished in 7 secs, took 1.05 GiB
(EngineCore_DP0 pid=2055571) INFO 01-15 04:59:18 [core.py:259] init engine (profile, create kv cache, warmup model) took 37.65 seconds
INFO 01-15 04:59:20 [llm.py:360] Supported tasks: ['generate', 'transcription']

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]
Adding requests: 100%|██████████| 1/1 [00:15<00:00, 15.68s/it]
Adding requests: 100%|██████████| 1/1 [00:15<00:00, 15.68s/it]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  2.72it/s, est. speed input: 1247.62 toks/s, output: 87.16 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  2.72it/s, est. speed input: 1247.62 toks/s, output: 87.16 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  2.72it/s, est. speed input: 1247.62 toks/s, output: 87.16 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]
Adding requests: 100%|██████████| 1/1 [00:00<00:00, 14.99it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  6.53it/s, est. speed input: 2993.53 toks/s, output: 78.42 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  6.53it/s, est. speed input: 2993.53 toks/s, output: 78.42 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  6.52it/s, est. speed input: 2993.53 toks/s, output: 78.42 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]
Adding requests: 100%|██████████| 1/1 [00:00<00:00, 14.16it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  3.07it/s, est. speed input: 1407.72 toks/s, output: 98.35 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  3.07it/s, est. speed input: 1407.72 toks/s, output: 98.35 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  3.07it/s, est. speed input: 1407.72 toks/s, output: 98.35 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]
Adding requests: 100%|██████████| 1/1 [00:00<00:00, 16.75it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.67it/s, est. speed input: 2137.33 toks/s, output: 69.99 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.67it/s, est. speed input: 2137.33 toks/s, output: 69.99 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.66it/s, est. speed input: 2137.33 toks/s, output: 69.99 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]
Adding requests: 100%|██████████| 1/1 [00:00<00:00, 19.44it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  6.08it/s, est. speed input: 2785.32 toks/s, output: 72.97 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  6.08it/s, est. speed input: 2785.32 toks/s, output: 72.97 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  6.07it/s, est. speed input: 2785.32 toks/s, output: 72.97 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]
Adding requests: 100%|██████████| 1/1 [00:00<00:00, 13.42it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.40it/s, est. speed input: 2471.58 toks/s, output: 80.94 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.40it/s, est. speed input: 2471.58 toks/s, output: 80.94 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.39it/s, est. speed input: 2471.58 toks/s, output: 80.94 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]
Adding requests: 100%|██████████| 1/1 [00:00<00:00, 16.22it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.09it/s, est. speed input: 2331.16 toks/s, output: 81.43 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.09it/s, est. speed input: 2331.16 toks/s, output: 81.43 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.08it/s, est. speed input: 2331.16 toks/s, output: 81.43 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]
Adding requests: 100%|██████████| 1/1 [00:00<00:00, 16.70it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  3.00it/s, est. speed input: 1375.42 toks/s, output: 93.09 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  3.00it/s, est. speed input: 1375.42 toks/s, output: 93.09 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  3.00it/s, est. speed input: 1375.42 toks/s, output: 93.09 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]
Adding requests: 100%|██████████| 1/1 [00:00<00:00, 13.19it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.79it/s, est. speed input: 2195.69 toks/s, output: 71.90 toks/s]
Processed prompts: 100%|████████…27631 tokens truncated… 71.74 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]
Adding requests: 100%|██████████| 1/1 [00:00<00:00, 15.31it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  2.06it/s, est. speed input: 941.61 toks/s, output: 106.90 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  2.06it/s, est. speed input: 941.61 toks/s, output: 106.90 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  2.05it/s, est. speed input: 941.61 toks/s, output: 106.90 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]
Adding requests: 100%|██████████| 1/1 [00:00<00:00, 14.05it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.95it/s, est. speed input: 2267.59 toks/s, output: 94.06 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.95it/s, est. speed input: 2267.59 toks/s, output: 94.06 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.95it/s, est. speed input: 2267.59 toks/s, output: 94.06 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]
Adding requests: 100%|██████████| 1/1 [00:00<00:00, 17.01it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  2.10it/s, est. speed input: 961.42 toks/s, output: 107.05 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  2.10it/s, est. speed input: 961.42 toks/s, output: 107.05 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  2.10it/s, est. speed input: 961.42 toks/s, output: 107.05 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]
Adding requests: 100%|██████████| 1/1 [00:00<00:00, 12.87it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  6.12it/s, est. speed input: 2805.37 toks/s, output: 85.74 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  6.12it/s, est. speed input: 2805.37 toks/s, output: 85.74 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  6.11it/s, est. speed input: 2805.37 toks/s, output: 85.74 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]
Adding requests: 100%|██████████| 1/1 [00:00<00:00, 23.53it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.80it/s, est. speed input: 2657.17 toks/s, output: 81.21 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.80it/s, est. speed input: 2657.17 toks/s, output: 81.21 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.79it/s, est. speed input: 2657.17 toks/s, output: 81.21 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]
Adding requests: 100%|██████████| 1/1 [00:00<00:00, 21.07it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.90it/s, est. speed input: 2245.13 toks/s, output: 88.23 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.90it/s, est. speed input: 2245.13 toks/s, output: 88.23 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.89it/s, est. speed input: 2245.13 toks/s, output: 88.23 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]
Adding requests: 100%|██████████| 1/1 [00:00<00:00, 15.81it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.73it/s, est. speed input: 2626.12 toks/s, output: 80.27 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.73it/s, est. speed input: 2626.12 toks/s, output: 80.27 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.73it/s, est. speed input: 2626.12 toks/s, output: 80.27 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]
Adding requests: 100%|██████████| 1/1 [00:00<00:00, 16.93it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.36it/s, est. speed input: 2454.78 toks/s, output: 80.39 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.36it/s, est. speed input: 2454.78 toks/s, output: 80.39 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.35it/s, est. speed input: 2454.78 toks/s, output: 80.39 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]
Adding requests: 100%|██████████| 1/1 [00:00<00:00, 14.37it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.42it/s, est. speed input: 2025.21 toks/s, output: 88.42 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.42it/s, est. speed input: 2025.21 toks/s, output: 88.42 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.42it/s, est. speed input: 2025.21 toks/s, output: 88.42 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]
Adding requests: 100%|██████████| 1/1 [00:00<00:00, 14.59it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  3.77it/s, est. speed input: 1727.88 toks/s, output: 90.54 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  3.77it/s, est. speed input: 1727.88 toks/s, output: 90.54 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  3.77it/s, est. speed input: 1727.88 toks/s, output: 90.54 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]
Adding requests: 100%|██████████| 1/1 [00:00<00:00,  9.77it/s]
Adding requests: 100%|██████████| 1/1 [00:00<00:00,  9.75it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  2.22it/s, est. speed input: 1017.36 toks/s, output: 108.84 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  2.22it/s, est. speed input: 1017.36 toks/s, output: 108.84 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  2.22it/s, est. speed input: 1017.36 toks/s, output: 108.84 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]
Adding requests: 100%|██████████| 1/1 [00:00<00:00, 35.51it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  6.11it/s, est. speed input: 2799.31 toks/s, output: 79.45 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  6.11it/s, est. speed input: 2799.31 toks/s, output: 79.45 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  6.10it/s, est. speed input: 2799.31 toks/s, output: 79.45 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]
Adding requests: 100%|██████████| 1/1 [00:00<00:00, 36.07it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.32it/s, est. speed input: 1979.08 toks/s, output: 90.74 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.32it/s, est. speed input: 1979.08 toks/s, output: 90.74 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.31it/s, est. speed input: 1979.08 toks/s, output: 90.74 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]
Adding requests: 100%|██████████| 1/1 [00:00<00:00, 19.92it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.66it/s, est. speed input: 2133.78 toks/s, output: 79.19 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.66it/s, est. speed input: 2133.78 toks/s, output: 79.19 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.65it/s, est. speed input: 2133.78 toks/s, output: 79.19 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]
Adding requests: 100%|██████████| 1/1 [00:00<00:00, 21.74it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  2.90it/s, est. speed input: 1330.01 toks/s, output: 101.63 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  2.90it/s, est. speed input: 1330.01 toks/s, output: 101.63 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  2.90it/s, est. speed input: 1330.01 toks/s, output: 101.63 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]
Adding requests: 100%|██████████| 1/1 [00:00<00:00, 19.14it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.09it/s, est. speed input: 2331.70 toks/s, output: 71.27 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.09it/s, est. speed input: 2331.70 toks/s, output: 71.27 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.09it/s, est. speed input: 2331.70 toks/s, output: 71.27 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]
Adding requests: 100%|██████████| 1/1 [00:00<00:00, 20.28it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.81it/s, est. speed input: 2204.78 toks/s, output: 91.45 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.81it/s, est. speed input: 2204.78 toks/s, output: 91.45 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.81it/s, est. speed input: 2204.78 toks/s, output: 91.45 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]
Adding requests: 100%|██████████| 1/1 [00:00<00:00, 23.64it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.61it/s, est. speed input: 2567.71 toks/s, output: 84.09 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.61it/s, est. speed input: 2567.71 toks/s, output: 84.09 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.60it/s, est. speed input: 2567.71 toks/s, output: 84.09 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]
Adding requests: 100%|██████████| 1/1 [00:00<00:00, 28.94it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  6.08it/s, est. speed input: 2783.77 toks/s, output: 85.09 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  6.08it/s, est. speed input: 2783.77 toks/s, output: 85.09 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  6.07it/s, est. speed input: 2783.77 toks/s, output: 85.09 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]
Adding requests: 100%|██████████| 1/1 [00:00<00:00, 26.39it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  2.77it/s, est. speed input: 1270.83 toks/s, output: 105.43 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  2.77it/s, est. speed input: 1270.83 toks/s, output: 105.43 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  2.77it/s, est. speed input: 1270.83 toks/s, output: 105.43 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]
Adding requests: 100%|██████████| 1/1 [00:00<00:00, 26.81it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  6.12it/s, est. speed input: 2802.15 toks/s, output: 79.53 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  6.12it/s, est. speed input: 2802.15 toks/s, output: 79.53 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  6.11it/s, est. speed input: 2802.15 toks/s, output: 79.53 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]
Adding requests: 100%|██████████| 1/1 [00:00<00:00, 21.32it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.82it/s, est. speed input: 2206.15 toks/s, output: 81.88 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.82it/s, est. speed input: 2206.15 toks/s, output: 81.88 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.81it/s, est. speed input: 2206.15 toks/s, output: 81.88 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]
Adding requests: 100%|██████████| 1/1 [00:00<00:00, 22.75it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.90it/s, est. speed input: 2245.71 toks/s, output: 88.25 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.90it/s, est. speed input: 2245.71 toks/s, output: 88.25 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.90it/s, est. speed input: 2245.71 toks/s, output: 88.25 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]
Adding requests: 100%|██████████| 1/1 [00:00<00:00, 18.60it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.17it/s, est. speed input: 2367.39 toks/s, output: 82.70 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.17it/s, est. speed input: 2367.39 toks/s, output: 82.70 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.16it/s, est. speed input: 2367.39 toks/s, output: 82.70 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]
Adding requests: 100%|██████████| 1/1 [00:00<00:00, 26.67it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.58it/s, est. speed input: 2095.81 toks/s, output: 91.51 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.58it/s, est. speed input: 2095.81 toks/s, output: 91.51 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  4.57it/s, est. speed input: 2095.81 toks/s, output: 91.51 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]
Adding requests: 100%|██████████| 1/1 [00:00<00:00, 15.30it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.71it/s, est. speed input: 2614.74 toks/s, output: 85.63 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.71it/s, est. speed input: 2614.74 toks/s, output: 85.63 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.70it/s, est. speed input: 2614.74 toks/s, output: 85.63 toks/s]

Adding requests:   0%|          | 0/1 [00:00<?, ?it/s]
Adding requests: 100%|██████████| 1/1 [00:00<00:00, 25.39it/s]

Processed prompts:   0%|          | 0/1 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.34it/s, est. speed input: 2446.03 toks/s, output: 85.44 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.34it/s, est. speed input: 2446.03 toks/s, output: 85.44 toks/s]
Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.33it/s, est. speed input: 2446.03 toks/s, output: 85.44 toks/s]
Transcribing ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:02:28
[rank0]:[W115 05:01:48.916162823 ProcessGroupNCCL.cpp:1524] Warning: WARNING: destroy_process_group() was not called before program exit, which can leak resources. For more info, please see https://pytorch.org/docs/stable/distributed.html#shutdown (function operator())
Wrote: outputs/bench/maya_hf_hin_v2/gemma3n_e2b_vllm_prompt/results.jsonl
Wrote: outputs/bench/maya_hf_hin_v2/gemma3n_e2b_vllm_prompt/summary.json
Wrote: outputs/bench/maya_hf_hin_v2/gemma3n_e2b_vllm_prompt/eval.json
{
  "hypothesis": "final",
  "pairs_evaluated": 200,
  "wer_mean": 0.2425516844266846,
  "cer_mean": 0.13898414203048856,
  "wer_p50": 0.13333333333333333,
  "wer_p90": 0.5,
  "wer_p95": 1.0,
  "cer_p50": 0.05128205128205128,
  "cer_p90": 0.225,
  "cer_p95": 1.0,
  "segment_error_rate": 0.805,
  "pct_wer_gt_5pct": 0.805,
  "pct_wer_gt_10pct": 0.605,
  "wer_by_language_mean": {
    "hin": 0.2425516844266846
  },
  "wer_by_decision_mean": {
    "accept": 0.24614720235158194,
    "human": 0.2347328597328597
  },
  "wer_by_duration_bucket_mean": {
    "0-3s": 0.9305555555555556,
    "10-30s": 0.3342948717948718,
    "3-10s": 0.22400933876997714
  },
  "cer_by_language_mean": {
    "hin": 0.13898414203048856
  },
  "cer_by_decision_mean": {
    "accept": 0.14380756568172298,
    "human": 0.1284951096460582
  },
  "cer_by_duration_bucket_mean": {
    "0-3s": 0.7941176470588235,
    "10-30s": 0.06913239817186206,
    "3-10s": 0.12801754591748682
  },
  "oracle": {
    "enabled": true,
    "oracle_best_wer_mean": 0.2425516844266846,
    "selection_gap_wer_mean": 0.0,
    "selection_gap_wer_p90": 0.0
  }
}


# AGENTS.md instructions for /home/ubuntu/maya-transcribe

<INSTRUCTIONS>
## Skills
A skill is a set of local instructions to follow that is stored in a `SKILL.md` file. Below is the list of skills that can be used. Each entry includes a name, description, and file path so you can open the source for full instructions when using a specific skill.
### Available skills
- skill-creator: Guide for creating effective skills. This skill should be used when users want to create a new skill (or update an existing skill) that extends Codex's capabilities with specialized knowledge, workflows, or tool integrations. (file: /home/ubuntu/.codex/skills/.system/skill-creator/SKILL.md)
- skill-installer: Install Codex skills into $CODEX_HOME/skills from a curated list or a GitHub repo path. Use when a user asks to list installable skills, install a curated skill, or install a skill from another repo (including private repos). (file: /home/ubuntu/.codex/skills/.system/skill-installer/SKILL.md)
### How to use skills
- Discovery: The list above is the skills available in this session (name + description + file path). Skill bodies live on disk at the listed paths.
- Trigger rules: If the user names a skill (with `$SkillName` or plain text) OR the task clearly matches a skill's description shown above, you must use that skill for that turn. Multiple mentions mean use them all. Do not carry skills across turns unless re-mentioned.
- Missing/blocked: If a named skill isn't in the list or the path can't be read, say so briefly and continue with the best fallback.
- How to use a skill (progressive disclosure):
  1) After deciding to use a skill, open its `SKILL.md`. Read only enough to follow the workflow.
  2) If `SKILL.md` points to extra folders such as `references/`, load only the specific files needed for the request; don't bulk-load everything.
  3) If `scripts/` exist, prefer running or patching them instead of retyping large code blocks.
  4) If `assets/` or templates exist, reuse them instead of recreating from scratch.
- Coordination and sequencing:
  - If multiple skills apply, choose the minimal set that covers the request and state the order you'll use them.
  - Announce which skill(s) you're using and why (one short line). If you skip an obvious skill, say why.
- Context hygiene:
  - Keep context small: summarize long sections instead of pasting them; only load extra files when needed.
  - Avoid deep reference-chasing: prefer opening only files directly linked from `SKILL.md` unless you're blocked.
  - When variants exist (frameworks, providers, domains), pick only the relevant reference file(s) and note that choice.
- Safety and fallback: If a skill can't be applied cleanly (missing files, unclear instructions), state the issue, pick the next-best approach, and continue.
</INSTRUCTIONS>

<environment_context>
  <cwd>/home/ubuntu/maya-transcribe</cwd>
  <shell>bash</shell>
</environment_context>

plan was [master.md](master.md)

Full Benchmark Analysis Summary

  Run Comparison
  ┌─────────────┬────────────────────┬──────────────────┬───────────────────┐
  │   Metric    │ Baseline (FW only) │ + Vaani (strict) │ + Vaani (relaxed) │
  ├─────────────┼────────────────────┼──────────────────┼───────────────────┤
  │ WER (final) │ 30.6%              │ 23.7%            │ 23.7%             │
  ├─────────────┼────────────────────┼──────────────────┼───────────────────┤
  │ Accept rate │ 0/200              │ 11/200           │ 81/200            │
  ├─────────────┼────────────────────┼──────────────────┼───────────────────┤
  │ Human rate  │ 200/200            │ 189/200          │ 119/200           │
  ├─────────────┼────────────────────┼──────────────────┼───────────────────┤
  │ Oracle best │ 29.7%              │ 9.8%             │ 9.8%              │
  └─────────────┴────────────────────┴──────────────────┴───────────────────┘
  Decision Quality with Relaxed Threshold
  ┌──────────┬───────┬──────────┬───────────────┐
  │ Decision │ Count │ WER Mean │ Disagree Rate │
  ├──────────┼───────┼──────────┼───────────────┤
  │ Accept   │ 81    │ 18.6%    │ 8.5%          │
  ├──────────┼───────┼──────────┼───────────────┤
  │ Human    │ 119   │ 27.1%    │ 13.2%         │
  └──────────┴───────┴──────────┴───────────────┘
  WER by Disagreement Rate Bucket

  - 0-5%: n=35, WER=6.0% (excellent)
  - 5-10%: n=47, WER=19.6%
  - 10-15%: n=68, WER=24.6%
  - 15-25%: n=44, WER=36.9%
  - 25-100%: n=6, WER=50.9%

  Remaining Human Decision Reasons

  - lid_uncertain: 63 (LID confused between Hindi/Urdu/related languages)
  - asr_disagreement: 50 (still above 15% threshold)
  - ctc_ambiguous: 49 (CTC margin too low)

  LID Analysis

  - 82% correctly identified as Hindi
  - 31.5% flagged uncertain due to low margin (Hindi vs Urdu/Gujarati/Bhojpuri)
  - This is expected: MMS-LID struggles with closely related Indic languages

  Final Selection Distribution

  - ROVER: 64.5% (129/200)
  - Vaani Hindi: 34.0% (68/200)
  - FW large-v3: 1.5% (3/200)

  Key Insights

  1. Oracle potential is real: 9.8% WER is achievable with perfect selection
  2. Current selection gap: 13.9pp left on the table due to suboptimal hypothesis selection
  3. Gating works directionally: Accept segments (18.6% WER) are better than Human segments (27.1% WER)
  4. Disagreement correlates with quality: Strong correlation between disagreement rate and WER
  5. LID is the new bottleneck: 31.5% flagged uncertain due to Hindi/Urdu confusion

  Recommendations

  1. Consider LID-Urdu equivalence for Hindi: Treat Hindi and Urdu as equivalent for routing since they share script (when using Devanagari transcription)
  2. Better CTC selection: Current CTC selection isn't capturing the oracle improvement - the selection_gap of 13.9pp suggests the CTC model isn't reliably picking the best hypothesis
  3. Two-stage approach: Accept high-confidence segments (low disagreement), use LLM-based selection for ambiguous cases

✻ Sautéed for 32m 59s

● Background command "Run benchmark with relaxed disagreement threshold" completed (exit code 0).

● I've already analyzed those results above. The benchmark with relaxed threshold (15% disagreement rate) completed successfully:

  Key result: Accept rate improved from 11 → 81 segments (40.5%) while maintaining the same 23.7% WER.

  The analysis shows the pipeline is working - accepted segments have lower WER (18.6%) than segments routed to human review (27.1%). The main remaining bottlenecks are:
  - LID uncertainty (Hindi/Urdu confusion) - 63 segments
  - Disagreement still above threshold - 50 segments
  - CTC ambiguity - 49 segments

  The oracle potential (9.8% WER) is significantly better than what we're achieving (23.7%), indicating room for improved hypothesis selection.

But these WER scores are not what i'm looking for. i need <5% WER scores for all languages consistently. For that to happen, i can run any LLMs like Gemma3n, voxtral.

Can we try these LLMs for audio transcription. can you test and give the performance scores or WER scores when compared to our recent ones above. download these models from huggingface and integrate them and do inference on transcription tasks and get full scores. both models wont fit at once, onload one, then another. you should basically not run transformers based infernece. use vLLM or any other inference engine for this to run the gemma3n models or voxtral models.

i've cleared the other task which is using GPU. you now have 80GB GPU. use more utilization for GPU.

# Context from my IDE setup:

## Open tabs:
- eval.json: outputs/bench/maya_hf_hin_v2/gemma3n_e2b_vllm/eval.json

## My request for Codex:
Can you now check why this gemma 3n model is at such high WER rate ? take it up personally and compare the actual results the model is providing. if after everything the performance is from model's capability, then i agree we can't do much here. but if there is something wrong in our setup or prompt or whatever, we can fix it. dig deeper. Voxtral-small's performance is actually really good. can you note it down in the readme or somewhere for us to note the results. keep this in a file called results.md and explain what the experiemnt and it's results are, so we know which experiement gave best WER results. once you are done tweaking gemma model, let's note those results as well.

# Context from my IDE setup:

## Open tabs:
- eval.json: outputs/bench/maya_hf_hin_v2/gemma3n_e2b_vllm/eval.json

## My request for Codex:
can you summarize our findings on ASR with LLMs sofar.

# Context from my IDE setup:

## Open tabs:
- eval.json: outputs/bench/maya_hf_hin_v2/gemma3n_e2b_vllm/eval.json

## My request for Codex:
How about we do an ensemble of both ?

Run these next (ordered by highest expected WER drop for your exact setup).

## 1) Make sure Voxtral-Small is running in **true transcription mode**

You may already be close, but Voxtral has a dedicated “pure speech transcription mode” + explicit decoding guidance: **temperature=0.0 for transcription**. ([Hugging Face][1])
**Experiment**

* Use the OpenAI-compatible **audio.transcriptions** path (TranscriptionRequest) instead of chat prompting.
* Force `language="hi"` for your Hindi bench (removes one failure mode vs auto-detect). Voxtral supports Hindi explicitly. ([Hugging Face][1])
  **Why**: people often benchmark Voxtral via chat-completions + a prompt, which is measurably worse than its transcription API/mode.

## 2) Add a **2-pass “proofread” loop using the same audio**

This is the biggest “LLM-style” trick that often moves the needle without swapping models.

**Pass 1 (ASR):** normal transcription (temp 0).
**Pass 2 (repair):** feed *audio + pass1 transcript* and ask:

* “Correct transcription errors using the audio. Keep the same language/script. Do not add new content. Output only the corrected transcript.”

This specifically attacks your remaining Voxtral-Small 14.9% WER gap, which is usually dominated by (a) proper nouns, (b) word boundary errors, (c) dropped short words.

## 3) Stop optimizing Gemma-3n before you fix its known constraints

Gemma-3n’s audio encoder was (at launch) implemented for **up to ~30s clips**; longer segments tend to degrade or behave oddly unless you do long-form handling. ([Google Developers Blog][2])
If your Hindi bench has segments near 30–60s, cap/split to ≤20–25s and retry Gemma-3n *only after* that.

## 4) Add 1–2 more audio-LLM candidates worth testing for Indic

### Shuka v1 (Sarvam) — audio-native + Indic focus

Shuka is explicitly built for **audio in Indic languages** and reports testing on multiple Indic languages (including Hindi, Telugu, Tamil, etc.). ([Hugging Face][3])
**Run:** same Hindi bench with a strict “transcribe” prompt (it’s QA-oriented, but still worth a pass because it’s Indic-targeted).

### Qwen2-Audio / Qwen2.5-Omni

Worth testing, but be aware: vLLM users have hit prompt-format gotchas (e.g., requiring an explicit `<|AUDIO|>` tag depending on the serving path). ([GitHub][4])
**Run:** if you do it in vLLM, copy the exact audio prompt/template requirements.

## 5) If the real goal is **<5% WER**, add an Indic-ASR “ringer” (open-weight)

Even if your current focus is “LLM-based ASR,” the fastest way to hit the target is to include at least one strong Hindi-specialized ASR model and use it as:

* a fallback for hard segments, or
* a teacher / tie-breaker in an ensemble.

Two high-ROI options:

* **Whisper-Hindi v2 (Collabora)** claims ~**5% WER on google/fleurs** for Hindi. ([collabora.com][5])
* **AI4Bharat IndicConformer / IndicASR**: explicitly targets all 22 scheduled Indian languages (good for your “other language benches” plan). ([ai4bharat.iitm.ac.in][6])

## 6) Don’t waste cycles on models that won’t help Hindi

* **Nemotron Speech ASR** is optimized for low-latency streaming and (as described) is **English** transcription oriented; not a Hindi WER play. ([Hugging Face][7])
* **NVIDIA Canary** coverage is largely European languages (not Hindi). ([Hugging Face][8])

## 7) One eval tweak that can “free WER” without cheating

Before you chase model changes, confirm your WER pipeline matches Hindi realities:

* normalize punctuation consistently (or strip it)
* normalize digits (०/0), nukta variants, diacritics normalization
* consistent tokenization rules for compound words

This often shaves a couple points on Hindi WER when the model is already close.

---

### Minimal “next run set” (fast)

1. Voxtral-Small in **TranscriptionRequest** mode, `language=hi`, `temperature=0.0`. ([Hugging Face][1])
2. Voxtral-Small **2-pass repair** (audio + pass1 transcript).
3. Shuka v1 on same bench. ([Hugging Face][3])
4. Qwen2-Audio-7B-Instruct (template-correct). ([GitHub][4])

If you run only one thing: do (1)+(2). That’s the most likely to move 14.9% → materially lower without leaving the “audio LLM” lane.

Ignore the suggestions about the ASR based transformers models. they dont work. we've already tested them. see if the other suggestions help reduce further WER.

# Context from my IDE setup:

## Open tabs:
- eval.json: outputs/bench/maya_hf_hin_v2/gemma3n_e2b_vllm/eval.json

## My request for Codex:
Can you explain how did we do do the experiments in this run. how did we choose the ensemble transcription. who chose the final output from the ensemble models. and what was the criteria. Also explain about the prompt we used to transcribe these models. explain this and let me come with a better strategy than the ones we tried sofar. you can actually add this information in the results.md so i have all info  at one place.

# Context from my IDE setup:

## Active file: Prompt.md

## Open tabs:
- Prompt.md: Prompt.md
- eval.json: outputs/bench/maya_hf_hin_v2/gemma3n_e2b_vllm/eval.json

## My request for Codex:
can you do one more run with all previous experiments with voxtral, gemma, ensemble and whatever experiments we did with this new prompt instruct at [Prompt.md](Prompt.md) and update it in results as new prompt experiments in a new section or add new point in same section as previous experiment, or maybe little organized like a tabulated but more verbosic way.


Summary: auto

## Skills
A skill is a set of local instructions to follow that is stored in a `SKILL.md` file. Below is the list of skills that can be used. Each entry includes a name, description, and file path so you can open the source for full instructions when using a specific skill.
### Available skills
- skill-creator: Guide for creating effective skills. This skill should be used when users want to create a new skill (or update an existing skill) that extends Codex's capabilities with specialized knowledge, workflows, or tool integrations. (file: /home/ubuntu/.codex/skills/.system/skill-creator/SKILL.md)
- skill-installer: Install Codex skills into $CODEX_HOME/skills from a curated list or a GitHub repo path. Use when a user asks to list installable skills, install a curated skill, or install a skill from another repo (including private repos). (file: /home/ubuntu/.codex/skills/.system/skill-installer/SKILL.md)
### How to use skills
- Discovery: The list above is the skills available in this session (name + description + file path). Skill bodies live on disk at the listed paths.
- Trigger rules: If the user names a skill (with `$SkillName` or plain text) OR the task clearly matches a skill's description shown above, you must use that skill for that turn. Multiple mentions mean use them all. Do not carry skills across turns unless re-mentioned.
- Missing/blocked: If a named skill isn't in the list or the path can't be read, say so briefly and continue with the best fallback.
- How to use a skill (progressive disclosure):
  1) After deciding to use a skill, open its `SKILL.md`. Read only enough to follow the workflow.
  2) If `SKILL.md` points to extra folders such as `references/`, load only the specific files needed for the request; don't bulk-load everything.
  3) If `scripts/` exist, prefer running or patching them instead of retyping large code blocks.
  4) If `assets/` or templates exist, reuse them instead of recreating from scratch.
- Coordination and sequencing:
  - If multiple skills apply, choose the minimal set that covers the request and state the order you'll use them.
  - Announce which skill(s) you're using and why (one short line). If you skip an obvious skill, say why.
- Context hygiene:
  - Keep context small: summarize long sections instead of pasting them; only load extra files when needed.
  - Avoid deep reference-chasing: prefer opening only files directly linked from `SKILL.md` unless you're blocked.
  - When variants exist (frameworks, providers, domains), pick only the relevant reference file(s) and note that choice.
- Safety and fallback: If a skill can't be applied cleanly (missing files, unclear instructions), state the issue, pick the next-best approach, and continue.


**Planning complex experiment runs**

I'm mapping out multiple experiment runs with different prompts (voxtral, gemma, ensemble, proofread) and updating results in a structured, detailed way; I'll confirm current progress and ensure tasks persist before proceeding.


**Clarifying prompt usage and experiment setup**

Decided to run proofread and ensemble experiments using the new prompt for transcription but keep proofread prompt default; will verify Prompt.md and update results.md accordingly before running GPU-intensive experiments.


Output:
Exit code: 0
Wall time: 0 seconds
Output:
Agents.md
data
docs
master.md
maya_transcribe
maya_transcribe.egg-info
outputs
Prompt.md
pyproject.toml
README.md
results.md
task.md


Summary: auto


**Checking initial lines of results.md**


Output:
Exit code: 0
Wall time: 0 seconds
Output:
# Benchmark Results

Dataset: `data/bench/maya_hf_hin_v2` (Hindi, 200 segments)
Metric: mean WER (lower is better); CER included for context.
Inference: vLLM audio transcription, Whisper skipped.

## How We Ran These Experiments
- Pipeline: preprocess -> MMS-LID -> ASR -> ROVER -> verify. Language routing is forced from the path (`/hin/`), so Voxtral receives `language=hi`.
- Final selection in the pipeline: `final` defaults to `rover.text` unless CTC scoring or explicit router preferences are enabled. We did not enable CTC or router overrides in these runs, so `final` == ROVER output.
- Verification (accept/human) only labels segments; it does not change the transcript.

### ROVER / Ensemble Logic (inside pipeline)
- Implemented in `maya_transcribe/rover.py`:
  - Tokenize each hypothesis.
  - Choose a “centroid” hypothesis (lowest average normalized edit distance to others).
  - Align all hypotheses to the centroid.
  - Vote per token position; ties fall back to the centroid token.
  - Gap insertions are kept only if >=50% support.
- With two hypotheses, ROVER tends to pick the centroid transcript and keeps only gaps supported by both.

### Voxtral + Gemma Offline Ensemble
- We merged `results.jsonl` from Voxtral and Gemma, then ran ROVER on their `final` texts only.
- No verification gating or CTC reranking in that offline ensemble; the output is purely ROVER.

### Prompts / Decoding
- Voxtral transcription mode: vLLM’s Voxtral `TranscriptionRequest` path (audio-only, no extra text prompt). Language is set to `hi` from path routing.
- Gemma-3n transcription mode: vLLM’s built-in Gemma3n prompt template:
  - `"Transcribe this audio into Hindi: <audio_soft_token><end_of_turn>\n<start_of_turn>model\n"`
- Voxtral proofread prompt (audio + pass1 transcript, chat mode):
  - `"Correct transcription errors using the audio. Keep the same language/script. Do not add new content. Output only the corrected transcript.\n\nTranscript:\n{transcript}"`
- Decoding: `temperature=0.0`, `top_p=1.0`. For Gemma-3n we also add stop token `<end_of_turn>` and cap output length to 12 tokens/sec (min 32, max 1024).

## Voxtral
- Voxtral-Small-24B-2507: WER 14.87%, CER 5.49% (best so far).
  - Eval: `outputs/bench/maya_hf_hin_v2/voxtral_small_vllm/eval.json`
  - Command: `python3 -m maya_transcribe.cli bench run --bench-dir data/bench/maya_hf_hin_v2 --out outputs/bench/maya_hf_hin_v2/voxtral_small_vllm --skip-whisper --vllm-asr-model mistralai/Voxtral-Small-24B-2507 --vllm-gpu-memory-utilization 0.9 --vllm-max-model-len 4096`
- Voxtral-Small-24B-2507 (2-pass proofread, audio + pass1 transcript via chat): final WER 16.62%, CER 7.32%. Raw pass stayed competitive, proofread pass degraded.
  - Final eval: `outputs/bench/maya_hf_hin_v2/voxtral_small_vllm_proofread/eval.json`
  - Raw eval: `outputs/bench/maya_hf_hin_v2/voxtral_small_vllm_proofread/eval_raw.json` (WER 15.03%, CER 5.56%)
  - Proofread eval: `outputs/bench/maya_hf_hin_v2/voxtral_small_vllm_proofread/eval_proofread.json` (WER 24.58%, CER 17.59%)
  - Command: `python3 -m maya_transcribe.cli bench run --bench-dir data/bench/maya_hf_hin_v2 --out outputs/bench/maya_hf_hin_v2/voxtral_small_vllm_proofread --skip-whisper --vllm-asr-model mistralai/Voxtral-Small-24B-2507 --vllm-gpu-memory-utilization 0.9 --vllm-max-model-len 4096 --vllm-proofread`
- Voxtral-Mini-3B-2507: WER 19.32%, CER 7.85%.
  - Eval: `outputs/bench/maya_hf_hin_v2/voxtral_mini_vllm/eval.json`
  - Command: `python3 -m maya_transcribe.cli bench run --bench-dir data/bench/maya_hf_hin_v2 --out outputs/bench/maya_hf_hin_v2/voxtral_mini_vllm --skip-whisper --vllm-asr-model mistralai/Voxtral-Mini-3B-2507 --vllm-gpu-memory-utilization 0.15 --vllm-max-model-len 4096`

## Gemma-3n
- Gemma-3n-E2B-it (initial run): WER 137.6%, CER 105.2%. This was dominated by runaway repetitions (hundreds/thousands of tokens) when the model did not emit a stop token.
  - Eval: `outputs/bench/maya_hf_hin_v2/gemma3n_e2b_vllm/eval.json`
- Gemma-3n-E2B-it (tuned stop token + dynamic max_tokens): WER 20.63%, CER 10.05%. The runaway repetition is capped, but accuracy still lags Voxtral.
  - Eval: `outputs/bench/maya_hf_hin_v2/gemma3n_e2b_vllm_tuned/eval.json`
  - Command: `python3 -m maya_transcribe.cli bench run --bench-dir data/bench/maya_hf_hin_v2 --out outputs/bench/maya_hf_hin_v2/gemma3n_e2b_vllm_tuned --skip-whisper --vllm-asr-model google/gemma-3n-E2B-it --vllm-gpu-memory-utilization 0.9 --vllm-max-model-len 4096`

## Ensembles
- Voxtral-Small + Gemma-3n (ROVER on both finals): WER 20.81%, CER 11.04% (worse than Voxtral alone).
  - Eval: `outputs/bench/maya_hf_hin_v2/voxtral_gemma_ensemble/eval.json`

## Notes
- Gemma-3n now uses a per-audio max_tokens cap (12 tokens/sec, max 1024) and a Gemma stop token (`<end_of_turn>`) to prevent runaway decoding.
- Best WER on this Hindi bench is currently Voxtral-Small (14.87%).


**Planning documentation update**


Output:
Exit code: 0
Wall time: 0 seconds
Output:
This is a prompt for any **audio-native** model (Gemini, Voxtral, etc.) used as an extra ASR hypothesis.
It is written to minimize hallucinations and keep outputs easy to validate.

`prompt`

`Instruction: You are a strict, verbatim transcription engine.
Input: Audio file attached
Language hint (primary): <language>

Task:
- Transcribe only what is clearly spoken in the audio.
- Do NOT guess or “fill in” missing words. If a word or phrase is unclear, output <unk> for that span.
- Preserve repetitions, filler words, stutters, and false starts exactly as spoken.
- Do NOT correct grammar, spelling, or pronunciation.
- Do NOT translate or summarize.
- Do NOT add punctuation.

Code-switching:
- Use the native script for the primary language.
- If the language hint is unknown, detect the primary spoken language and use its native script.
- If English words are spoken, keep them in Latin script as spoken (do not transliterate).

OUTPUT FORMAT (exact):
Output 1: Transcript
<single transcript line>

Do not output anything else (no extra headings, notes, or additional outputs).`


**Checking prompt language handling**

I’ll verify that the ASR code replaces <language> with full language names like "Hindi" instead of abbreviations, ensuring the new prompt formatting matches the expected mappings.


rg -n "vllm_transcribe_prompt|_render_prompt|_build_gemma_prompt|Prompt" -n maya_transcribe/asr.py


Output:
Exit code: 0
Wall time: 0 seconds
Output:
483:    def _render_prompt(self, template: str, lang: str | None) -> str:
496:    def _build_gemma_prompt(self, audio: "np.ndarray", sample_rate: int, prompt_text: str) -> dict[str, Any]:
556:            prompt_payload = self._build_voxtral_chat_prompt(wav, sr, self._render_prompt(prompt_text, lang))
558:            prompt_payload = self._build_gemma_prompt(wav, sr, self._render_prompt(prompt_text, lang))


**Inspecting code lines for correctness**


sed -n '430,620p' maya_transcribe/asr.py


Output:
Exit code: 0
Wall time: 0 seconds
Output:
        self._max_model_len = int(max_model_len)
        self._fixed_max_tokens = int(max_tokens) if max_tokens is not None else None
        if self._fixed_max_tokens is not None:
            self._fixed_max_tokens = min(self._fixed_max_tokens, self._max_model_len)

        self._tokens_per_second = 12
        self._max_tokens_cap = min(1024, self._max_model_len)

        self._stop_token_ids: list[int] | None = None
        if "gemma-3n" in model_id.lower():
            try:
                tok = self._llm.get_tokenizer()
                end_id = tok.convert_tokens_to_ids("<end_of_turn>")
                if isinstance(end_id, int) and end_id >= 0:
                    self._stop_token_ids = [end_id]
            except Exception:
                self._stop_token_ids = None

        try:
            self._torch = torch
        except Exception:
            self._torch = None

    @property
    def name(self) -> str:
        return f"vllm:{self.model_id}"

    @property
    def supports_proofread(self) -> bool:
        return self._is_voxtral

    def _build_voxtral_chat_prompt(self, audio: "np.ndarray", sample_rate: int, prompt_text: str) -> dict[str, Any]:
        from mistral_common.protocol.instruct.chunk import AudioChunk, RawAudio, TextChunk
        from mistral_common.protocol.instruct.messages import UserMessage
        from mistral_common.protocol.instruct.request import ChatCompletionRequest
        from mistral_common.tokens.tokenizers.audio import Audio
        from vllm.tokenizers import cached_tokenizer_from_config

        tokenizer = cached_tokenizer_from_config(self._model_config)
        if tokenizer is None:
            raise RuntimeError("Tokenizer unavailable for Voxtral prompt.")

        audio_item = Audio(audio, sample_rate, format="wav")
        audio_chunk = AudioChunk(input_audio=RawAudio.from_audio(audio_item))
        request = ChatCompletionRequest(messages=[UserMessage(content=[TextChunk(text=prompt_text), audio_chunk])])
        tokenized = tokenizer.mistral.encode_chat_completion(request)
        audio_array = tokenized.audios[0].audio_array if tokenized.audios else audio_item.audio_array

        return {
            "prompt_token_ids": tokenized.tokens,
            "multi_modal_data": {"audio": (audio_array, sample_rate)},
        }

    def _render_prompt(self, template: str, lang: str | None) -> str:
        if not template:
            return template
        lang_name = None
        if lang:
            try:
                lang_name = self._model_cls.supported_languages.get(lang)
            except Exception:
                lang_name = None
        if not lang_name:
            lang_name = lang or "unknown"
        return template.replace("<language>", lang_name)

    def _build_gemma_prompt(self, audio: "np.ndarray", sample_rate: int, prompt_text: str) -> dict[str, Any]:
        prompt = "<start_of_turn>user\n"
        prompt += prompt_text.strip()
        prompt += "\n\n<audio_soft_token><end_of_turn>\n<start_of_turn>model\n"
        return {"multi_modal_data": {"audio": (audio, sample_rate)}, "prompt": prompt}

    def _strip_prompt_output(self, text: str) -> str:
        cleaned = str(text or "").strip()
        if not cleaned:
            return ""
        lines = [ln.strip() for ln in cleaned.splitlines() if ln.strip()]
        if not lines:
            return ""
        for i, line in enumerate(lines):
            low = line.lower()
            if low.startswith("output 1"):
                if ":" in line:
                    after = line.split(":", 1)[1].strip()
                    if after and after.lower() != "transcript":
                        return after
                    if after.lower() == "transcript" and i + 1 < len(lines):
                        return lines[i + 1]
                if i + 1 < len(lines):
                    return lines[i + 1]
        return lines[0]

    def transcribe(
        self,
        audio_path: Path,
        language: str | None = None,
        prompt: str | None = None,
        mode: str = "transcribe",
    ) -> ASRResult:
        from maya_transcribe.audio import decode_pcm16

        sr = int(getattr(self._stt_config, "sample_rate", 16000))
        wav, _ = decode_pcm16(audio_path, sample_rate=sr, mono=True)
        if wav.size == 0:
            return ASRResult(
                model=self.name,
                text="",
                language=language,
                language_probability=None,
                segments=[],
                meta={"dtype": self.dtype, "tensor_parallel_size": self.tensor_parallel_size},
            )

        lang = to_whisper_lang(language)
        if lang:
            try:
                lang = self._model_cls.validate_language(lang)
            except Exception:
                lang = None

        prompt_text = prompt or ""
        if mode == "proofread":
            if not self.supports_proofread:
                raise RuntimeError("Proofread mode requires a Voxtral model.")
            prompt_payload = self._build_voxtral_chat_prompt(wav, sr, prompt_text)
        elif prompt_text and self._is_voxtral:
            prompt_payload = self._build_voxtral_chat_prompt(wav, sr, self._render_prompt(prompt_text, lang))
        elif prompt_text and self._is_gemma3n:
            prompt_payload = self._build_gemma_prompt(wav, sr, self._render_prompt(prompt_text, lang))
        else:
            prompt_payload = self._model_cls.get_generation_prompt(
                audio=wav,
                stt_config=self._stt_config,
                model_config=self._model_config,
                language=lang,
                task_type="transcribe",
                request_prompt=prompt_text,
                to_language=None,
            )

        max_tokens = self._fixed_max_tokens
        if max_tokens is None:
            duration_sec = float(wav.shape[0]) / float(sr) if sr else 0.0
            max_tokens = max(32, int(duration_sec * self._tokens_per_second))
            max_tokens = min(max_tokens, self._max_tokens_cap)

        sampling_params = self._sampling_params_cls(
            temperature=0.0,
            top_p=1.0,
            max_tokens=max_tokens,
            stop_token_ids=self._stop_token_ids,
        )

        outputs = self._llm.generate([prompt_payload], sampling_params)
        text = ""
        if outputs and outputs[0].outputs:
            text = outputs[0].outputs[0].text
        if prompt_text and mode != "proofread":
            text = self._strip_prompt_output(text)

        return ASRResult(
            model=self.name,
            text=str(text).strip(),
            language=lang,
            language_probability=None,
            segments=[],
            meta={"dtype": self.dtype, "tensor_parallel_size": self.tensor_parallel_size},
        )

def _to_iso3(lang: str | None) -> str | None:
    """Convert language code to ISO-639-3 format."""
    if not lang:
        return None
    code = lang.strip().lower()
    if not code:
        return None
    code = code.replace("-", "_")
    if "_" in code:
        code = code.split("_", 1)[0]
    if len(code) == 3:
        return code
    if len(code) == 2:
        return _WHISPER_TO_ISO3.get(code, None)
    return code

class MMSASR:
    """
    MMS ASR backend using CTC decoding (facebook/mms-1b-all).
    Provides architectural diversity from Whisper's encoder-decoder approach.


**Verifying gemma language support flag**

I'm checking asr.py initialization to confirm that the _is_gemma3n flag is set appropriately for distinguishing gemma model language mappings.


rg -n "_is_gemma3n|_is_voxtral" -n maya_transcribe/asr.py


Output:
Exit code: 0
Wall time: 0 seconds
Output:
420:        self._is_voxtral = "voxtral" in self._model_cls.__name__.lower() or "voxtral" in model_id.lower()
421:        self._is_gemma3n = "gemma3n" in self._model_cls.__name__.lower() or "gemma-3n" in model_id.lower()
459:        return self._is_voxtral
555:        elif prompt_text and self._is_voxtral:
557:        elif prompt_text and self._is_gemma3n:


**Planning experiment wrap-up and results update**

I'm preparing to run the remaining experiments with the new prompt, then update results.md with detailed explanations of the ensemble transcription process, selection criteria, and the prompt strategy to consolidate all findings and enable refining strategies.


**Planning comprehensive experiment rerun**

I'm creating a detailed plan to rerun all prior experiments with the new prompt instructions, ensure GPU utilization is optimized, compute ensembles, and update results with a clear, organized section covering the new prompt experiments.


Explanation: Run remaining prompt-based experiments, compute ensemble, then update results documentation.

Run remaining Prompt.md experiments (Voxtral proofread and Voxtral+Gemma ensemble) and capture eval outputs

Update results.md with a new Prompt.md experiments section and summary of outcomes

Summarize findings and next-step suggestions


Output: Plan updated


**Running voxtral proofread experiment**

I'm preparing to run the remaining experiment with the voxtral proofread prompt using GPU and specific vllm options, following the established output directory naming convention.


python3 -m maya_transcribe.cli bench run --bench-dir data/bench/maya_hf_hin_v2 --out outputs/bench/maya_hf_hin_v2/voxtral_small_vllm_proofread_prompt --skip-whisper --vllm-asr-model mistralai/Voxtral-Small-24B-2507 --vllm-gpu-memory-utilization 0.9 --vllm-max-model-len 4096 --vllm-proofread --vllm-transcribe-prompt-file Prompt.md