Explore the /home/ubuntu/maya-transcribe codebase thoroughly. I need to understand:
1. The overall directory structure
2. All Python files and their purposes
3. Any configuration files
4. The main pipeline implementation

Focus on finding:
- LID (Language Identification) implementation
- ASR (Automatic Speech Recognition) models and ensemble logic
- ROVER/system combination implementation
- CTC verification/audio-grounded verification
- Normalization logic
- Any model configurations

Provide a comprehensive summary of the codebase structure and key files.


I'll conduct a thorough exploration of the maya-transcribe codebase. Let me start by examining the directory structure and then systematically search for the key components you're interested in.


Bash

find /home/ubuntu/maya-transcribe -type f -name "*.py" | head -20

List Python files in the codebase


Show directory structure at root level


total 112
drwxrwxr-x  8 ubuntu ubuntu  4096 Jan 13 06:26 .
drwxr-x--- 43 ubuntu ubuntu  4096 Jan 14 00:52 ..
-rw-rw-r--  1 ubuntu ubuntu 41725 Jan 13 04:08 Agents.md
drwxrwxr-x  4 ubuntu ubuntu  4096 Jan 13 07:15 data
drwxrwxr-x  2 ubuntu ubuntu  4096 Jan 13 03:12 docs
-rw-rw-r--  1 ubuntu ubuntu  3704 Jan 13 05:07 .env
-rw-rw-r--  1 ubuntu ubuntu   144 Jan 13 04:16 .gitignore
-rw-rw-r--  1 ubuntu ubuntu  9299 Jan 13 04:09 master.md
drwxrwxr-x  3 ubuntu ubuntu  4096 Jan 13 07:11 maya_transcribe
drwxrwxr-x  2 ubuntu ubuntu  4096 Jan 13 04:29 maya_transcribe.egg-info
drwxrwxr-x 12 ubuntu ubuntu  4096 Jan 13 07:31 outputs
-rw-rw-r--  1 ubuntu ubuntu  1090 Jan 13 06:13 Prompt.md
-rw-rw-r--  1 ubuntu ubuntu  1140 Jan 13 07:12 pyproject.toml
-rw-rw-r--  1 ubuntu ubuntu  1280 Jan 13 05:39 README.md
-rw-rw-r--  1 ubuntu ubuntu  2627 Jan 13 02:35 task.md
drwxrwxr-x  2 ubuntu ubuntu  4096 Jan 13 06:26 .vscode


/home/ubuntu/maya-transcribe/maya_transcribe/io.py
/home/ubuntu/maya-transcribe/maya_transcribe/cli.py
/home/ubuntu/maya-transcribe/maya_transcribe/textnorm.py
/home/ubuntu/maya-transcribe/maya_transcribe/ctc_fit.py
/home/ubuntu/maya-transcribe/maya_transcribe/audio.py
/home/ubuntu/maya-transcribe/maya_transcribe/align.py
/home/ubuntu/maya-transcribe/maya_transcribe/language.py
/home/ubuntu/maya-transcribe/maya_transcribe/lid.py
/home/ubuntu/maya-transcribe/maya_transcribe/__init__.py
/home/ubuntu/maya-transcribe/maya_transcribe/eval.py
/home/ubuntu/maya-transcribe/maya_transcribe/tar.py
/home/ubuntu/maya-transcribe/maya_transcribe/pipeline.py
/home/ubuntu/maya-transcribe/maya_transcribe/ids.py
/home/ubuntu/maya-transcribe/maya_transcribe/gemini_asr.py
/home/ubuntu/maya-transcribe/maya_transcribe/config.py
/home/ubuntu/maya-transcribe/maya_transcribe/dataset.py
/home/ubuntu/maya-transcribe/maya_transcribe/db.py
/home/ubuntu/maya-transcribe/maya_transcribe/verify.py
/home/ubuntu/maya-transcribe/maya_transcribe/bench.py
/home/ubuntu/maya-transcribe/maya_transcribe/asr.py


/home/ubuntu/maya-transcribe/maya_transcribe/__init__.py
/home/ubuntu/maya-transcribe/maya_transcribe/config.py
/home/ubuntu/maya-transcribe/maya_transcribe/tar.py
/home/ubuntu/maya-transcribe/maya_transcribe/dataset.py
/home/ubuntu/maya-transcribe/maya_transcribe/audio.py
/home/ubuntu/maya-transcribe/maya_transcribe/io.py
/home/ubuntu/maya-transcribe/maya_transcribe/align.py
/home/ubuntu/maya-transcribe/maya_transcribe/textnorm.py
/home/ubuntu/maya-transcribe/maya_transcribe/r2.py
/home/ubuntu/maya-transcribe/maya_transcribe/lid.py
/home/ubuntu/maya-transcribe/maya_transcribe/db.py
/home/ubuntu/maya-transcribe/maya_transcribe/rover.py
/home/ubuntu/maya-transcribe/maya_transcribe/gemini_asr.py
/home/ubuntu/maya-transcribe/maya_transcribe/verify.py
/home/ubuntu/maya-transcribe/maya_transcribe/ids.py
/home/ubuntu/maya-transcribe/maya_transcribe/ctc_fit.py
/home/ubuntu/maya-transcribe/maya_transcribe/bench.py
/home/ubuntu/maya-transcribe/maya_transcribe/language.py
/home/ubuntu/maya-transcribe/maya_transcribe/asr.py
/home/ubuntu/maya-transcribe/maya_transcribe/pipeline.py
/home/ubuntu/maya-transcribe/maya_transcribe/cli.py
/home/ubuntu/maya-transcribe/maya_transcribe/eval.py


Now let me read the key documentation files and examine the main pipeline and components:


     1→# maya-transcribe
     2→
     3→Offline, quality-first transcription pipeline for cleaned single-speaker audio segments (Indic + English).
     4→
     5→## Install
     6→
     7→```bash
     8→python3 -m pip install -U pip
     9→python3 -m pip install -e '.[full]'
    10→```
    11→
    12→If you only need R2 download + evaluation (no ASR), install the base deps:
    13→
    14→```bash
    15→python3 -m pip install -e .
    16→```
    17→
    18→## Configure
    19→
    20→Create / update `.env` in the repo root. The CLI auto-loads it.
    21→
    22→Minimum for R2:
    23→- `R2_ENDPOINT_URL`
    24→- `R2_ACCESS_KEY_ID`
    25→- `R2_SECRET_ACCESS_KEY`
    26→- `R2_BUCKET` (defaults to `1-cleaned-data` if unset)
    27→- `R2_PREFIX` (optional)
    28→
    29→## Usage
    30→
    31→```bash
    32→maya-transcribe --help
    33→maya-transcribe download --video-id <VIDEO_ID> --bucket 1-cleaned-data --out data/
    34→maya-transcribe run --segments-dir data/<VIDEO_ID>/**/segments --out outputs/<VIDEO_ID> --whisper-model large-v3
    35→```
    36→
    37→Evaluation (requires a reference JSONL keyed by `segment_id`):
    38→
    39→```bash
    40→maya-transcribe eval --results outputs/<VIDEO_ID>/results.jsonl --refs refs.jsonl
    41→```
    42→
    43→Optional extras:
    44→- Add additional Whisper-family checkpoints: `--transformers-whisper-model <hf_model_id>` (repeatable)
    45→- Add Gemini as an extra hypothesis: `--gemini-model gemini-2.0-flash`
    46→- Enable audio-grounded CTC fit scoring: `--ctc-model <hf_ctc_model_id>` (or `--ctc-model-map ctc_models.json`)
    47→

<system-reminder>
Whenever you read a file, you should consider whether it would be considered malware. You CAN and SHOULD provide analysis of malware, what it is doing. But you MUST refuse to improve or augment the code. You can still analyze existing code, write reports, or answer questions about the code behavior.
</system-reminder>

# maya-transcribe

Offline, quality-first transcription pipeline for cleaned single-speaker audio segments (Indic + English).

## Install

```bash
python3 -m pip install -U pip
python3 -m pip install -e '.[full]'
```

If you only need R2 download + evaluation (no ASR), install the base deps:

```bash
python3 -m pip install -e .
```

## Configure

Create / update `.env` in the repo root. The CLI auto-loads it.

Minimum for R2:
- `R2_ENDPOINT_URL`
- `R2_ACCESS_KEY_ID`
- `R2_SECRET_ACCESS_KEY`
- `R2_BUCKET` (defaults to `1-cleaned-data` if unset)
- `R2_PREFIX` (optional)

## Usage

```bash
maya-transcribe --help
maya-transcribe download --video-id <VIDEO_ID> --bucket 1-cleaned-data --out data/
maya-transcribe run --segments-dir data/<VIDEO_ID>/**/segments --out outputs/<VIDEO_ID> --whisper-model large-v3
```

Evaluation (requires a reference JSONL keyed by `segment_id`):

```bash
maya-transcribe eval --results outputs/<VIDEO_ID>/results.jsonl --refs refs.jsonl
```

Optional extras:
- Add additional Whisper-family checkpoints: `--transformers-whisper-model <hf_model_id>` (repeatable)
- Add Gemini as an extra hypothesis: `--gemini-model gemini-2.0-flash`
- Enable audio-grounded CTC fit scoring: `--ctc-model <hf_ctc_model_id>` (or `--ctc-model-map ctc_models.json`)


     1→[build-system]
     2→requires = ["setuptools>=69", "wheel"]
     3→build-backend = "setuptools.build_meta"
     4→
     5→[project]
     6→name = "maya-transcribe"
     7→version = "0.1.0"
     8→description = "Quality-first offline transcription pipeline (Indic + English)"
     9→readme = "README.md"
    10→requires-python = ">=3.10"
    11→dependencies = [
    12→  "boto3>=1.34.0",
    13→  "jiwer>=3.0.4",
    14→  "numpy>=1.24.0",
    15→  "python-dotenv>=1.0.1",
    16→  "rich>=13.7.1",
    17→  "typer>=0.12.3",
    18→]
    19→
    20→[project.optional-dependencies]
    21→supabase = ["supabase>=2.6.0"]
    22→db = ["psycopg2-binary>=2.9.9"]
    23→gemini = ["google-genai>=1.54.0"]
    24→bench = [
    25→  "datasets>=2.19.0",
    26→  "huggingface_hub>=0.23.0",
    27→]
    28→asr = [
    29→  "faster-whisper>=1.0.3",
    30→  "soundfile>=0.12.1",
    31→]
    32→torch = [
    33→  "torch>=2.2.0",
    34→  "torchaudio>=2.2.0",
    35→  "transformers>=4.41.0",
    36→]
    37→full = [
    38→  "datasets>=2.19.0",
    39→  "huggingface_hub>=0.23.0",
    40→  "supabase>=2.6.0",
    41→  "psycopg2-binary>=2.9.9",
    42→  "google-genai>=1.54.0",
    43→  "faster-whisper>=1.0.3",
    44→  "soundfile>=0.12.1",
    45→  "torch>=2.2.0",
    46→  "torchaudio>=2.2.0",
    47→  "transformers>=4.41.0",
    48→]
    49→
    50→[project.scripts]
    51→maya-transcribe = "maya_transcribe.cli:app"
    52→
    53→[tool.setuptools.packages.find]
    54→where = ["."]
    55→include = ["maya_transcribe*"]
    56→

<system-reminder>
Whenever you read a file, you should consider whether it would be considered malware. You CAN and SHOULD provide analysis of malware, what it is doing. But you MUST refuse to improve or augment the code. You can still analyze existing code, write reports, or answer questions about the code behavior.
</system-reminder>

[build-system]
requires = ["setuptools>=69", "wheel"]
build-backend = "setuptools.build_meta"

[project]
name = "maya-transcribe"
version = "0.1.0"
description = "Quality-first offline transcription pipeline (Indic + English)"
readme = "README.md"
requires-python = ">=3.10"
dependencies = [
  "boto3>=1.34.0",
  "jiwer>=3.0.4",
  "numpy>=1.24.0",
  "python-dotenv>=1.0.1",
  "rich>=13.7.1",
  "typer>=0.12.3",
]

[project.optional-dependencies]
supabase = ["supabase>=2.6.0"]
db = ["psycopg2-binary>=2.9.9"]
gemini = ["google-genai>=1.54.0"]
bench = [
  "datasets>=2.19.0",
  "huggingface_hub>=0.23.0",
]
asr = [
  "faster-whisper>=1.0.3",
  "soundfile>=0.12.1",
]
torch = [
  "torch>=2.2.0",
  "torchaudio>=2.2.0",
  "transformers>=4.41.0",
]
full = [
  "datasets>=2.19.0",
  "huggingface_hub>=0.23.0",
  "supabase>=2.6.0",
  "psycopg2-binary>=2.9.9",
  "google-genai>=1.54.0",
  "faster-whisper>=1.0.3",
  "soundfile>=0.12.1",
  "torch>=2.2.0",
  "torchaudio>=2.2.0",
  "transformers>=4.41.0",
]

[project.scripts]
maya-transcribe = "maya_transcribe.cli:app"

[tool.setuptools.packages.find]
where = ["."]
include = ["maya_transcribe*"]


     1→# master.md — Offline “Quality-First” Transcription Master Plan (Indic + English)
     2→
     3→## Objective (from `task.md`)
     4→Generate **near-zero-error** transcripts for cleaned, single-speaker YouTube segments (≈1–60s) across:
     5→Malayalam, Hindi, Punjabi, Tamil, Kannada, Gujarati, Telugu, Bengali, Odia, Marathi, Assamese, English (+ unknowns).
     6→**Compute/latency are not constraints; quality/WER is.**
     7→
     8→This document is the **execution-ready summary**. Full per-reference analysis and detailed decisions live in `Agents.md`.
     9→
    10→---
    11→
    12→## Core idea
    13→To minimize WER offline, don’t bet on one model. Instead:
    14→1) **Route correctly (spoken LID)**
    15→2) **Generate diverse hypotheses (ASR ensemble + N-best)**
    16→3) **Combine (ROVER/confusion network)**
    17→4) **Prove audio-text match (CTC/forced-alignment verification gate)**
    18→5) **Reject uncertainty + human-correct the tail**
    19→
    20→---
    21→
    22→## Pipeline block diagram
    23→```text
    24→┌───────────────────────────────┐
    25→│          Audio segment         │
    26→└───────────────┬───────────────┘
    27→                │
    28→                v
    29→┌───────────────────────────────┐
    30→│        Pre-QC + VAD trim       │
    31→│   (format normalize + sanity)  │
    32→└───────────────┬───────────────┘
    33→                │
    34→                v
    35→┌───────────────────────────────┐
    36→│        Spoken LID ensemble     │
    37→│  MMS-LID (+ optional 2nd vote) │
    38→└───────────────┬───────────────┘
    39→                │
    40→      ┌─────────┴──────────┐
    41→      │                    │
    42→      v                    v
    43→┌───────────────────────┐  ┌────────────────────────┐
    44→│ Confident language     │  │ Uncertain / mixed       │
    45→│ route                  │  │ route                   │
    46→└───────────┬───────────┘  └───────────┬────────────┘
    47→            │                          │
    48→            v                          v
    49→┌───────────────────────┐  ┌────────────────────────┐
    50→│ ASR ensemble (per-L)   │  │ ASR ensemble (multi)    │
    51→│ IndicWhisper           │  │ Whisper v3              │
    52→│ + Whisper large-v3     │  │ + IndicConformer        │
    53→│ + IndicConformer       │  │ + MMS (+ Gemma 3n opt.)  │
    54→│ + Gemma 3n(probably)   │  │                        │
    55→└───────────┬───────────┘  └───────────┬────────────┘
    56→            │                          │
    57→            └───────────┬──────────────┘
    58→                        │
    59→                        v
    60→┌───────────────────────────────┐
    61→│     Normalize all hypotheses   │
    62→│ (Unicode + IndicNLP + numbers) │
    63→└───────────────┬───────────────┘
    64→                │
    65→                v
    66→┌───────────────────────────────┐
    67→│  System combination (ROVER)    │
    68→│   + disagreement map (audit)   │
    69→└───────────────┬───────────────┘
    70→                │
    71→                v
    72→┌───────────────────────────────┐
    73→│   Audio-grounded verification  │
    74→│  CTC “fit” scoring (+ aligner) │
    75→└───────┬───────────────┬───────┘
    76→        │ PASS          │ FAIL
    77→        v               v
    78→┌───────────────────┐   ┌──────────────────────────────┐
    79→│ ACCEPT            │   │ Re-decode / add model / retry │
    80→│ transcript+scores │   │ (then repeat from ASR stage)  │
    81→└───────────────────┘   └───────────────┬──────────────┘
    82→                                        │ still FAIL after retries
    83→                                        v
    84→                              ┌──────────────────────────┐
    85→                              │ HUMAN REVIEW (tail only)  │
    86→                              │ correct → ACCEPT          │
    87→                              │ discard → DROP            │
    88→                              └──────────────────────────┘
    89→```
    90→
    91→---
    92→
    93→## Initial model stack (validate on a gold set; tune per language)
    94→**Spoken LID**
    95→- Primary: **MMS-LID** (spoken LID coverage is the key requirement)
    96→- Optional 2nd opinion: **SpeechBrain VoxLingua107 ECAPA** (and/or Whisper LID as tie-break)
    97→
    98→**ASR (Indic)**
    99→- **IndicWhisper** (Indic fine-tuned Whisper; “specialist”)
   100→- **Whisper large-v3** (robust multilingual “generalist”)
   101→- **IndicConformer-600M multilingual** (different architecture; error diversity)
   102→- Optional diversity: **MMS ASR** (especially when others disagree / lower-resource languages)
   103→
   104→**ASR (English)**
   105→- **Whisper large-v3** + **Voxtral** (treat Voxtral as “measure on your gold set” for your domains)
   106→
   107→**System combination**
   108→- **ROVER / confusion-network voting** (keep the disagreement map)
   109→
   110→**Audio-grounded verification**
   111→- Primary: **CTC “fit” scoring** (score candidates against audio; reject low-fit)
   112→- Optional: forced alignment where you can support it cleanly
   113→
   114→---
   115→
   116→## Why this approach is the best way to minimize WER offline (and avoid hallucinations)
   117→This is the strongest generally applicable strategy under “unlimited compute / offline” because it attacks the dominant error sources:
   118→- **Wrong routing is catastrophic:** Spoken LID prevents wrong-language decodes (script/vocabulary mismatch) that can silently poison training data.
   119→- **Ensembles lower error by diversity:** Different models/architectures make different mistakes; if you can afford it, generating multiple hypotheses increases the odds that at least one is correct.
   120→- **ROVER reduces WER when errors differ:** System combination exploits that diversity instead of “picking a winner” blindly.
   121→- **Audio-grounded verification is the key safety gate:** A transcript can look fluent but still be wrong; independent alignment/CTC scoring forces the text to match the audio and catches hallucination patterns.
   122→- **Precision-first acceptance yields near-zero-error training text:** If a segment can’t be verified confidently, rejecting it (and sending only the tail to humans) minimizes label noise, which matters more than maximizing coverage for training.
   123→
   124→Important: “minimal WER” is not a claim about global benchmarks; it is achieved **for your data** by tuning gates/weights on a **human gold set** and only accepting segments that pass strict audio-grounded checks.
   125→
   126→---
   127→
   128→## Execution plan (deliverables)
   129→1. **Gold set + evaluation harness (Week 1)**
   130→   - Build a stratified gold set (all languages + duration buckets + clean/mild noise + code-switch).
   131→   - Produce human transcripts; compute WER/CER per language; establish baselines for each model.
   132→2. **Baseline pipeline (Week 1–2)**
   133→   - Implement: `VAD/QC → MMS-LID → ASR ensemble → normalization → ROVER → CTC-fit verification → accept/reject`.
   134→   - Store full audit trail (all hypotheses + scores).
   135→3. **Calibration (Week 2)**
   136→   - Tune: LID thresholds, per-language ROVER weights, verification-gate thresholds.
   137→   - Goal: maximize **precision** of accepted set; accept-rate is a secondary knob.
   138→4. **Human reject loop (Week 2–3)**
   139→   - Create a review queue that shows audio + top hypotheses + disagreement spans for fast correction.
   140→   - Feed corrected rejects back into calibration (and optional later fine-tuning).
   141→5. **Scale-out + monitoring (Week 3+)**
   142→   - Batch by language; parallelize per-model decoding; track reject reasons and residual error via sampling.
   143→
   144→---
   145→
   146→## Output schema (minimum needed for auditability)
   147→For each `segment_id`, store:
   148→- audio QC metrics, LID probs (per LID model)
   149→- ASR hypotheses (per model + decode params + N-best if available)
   150→- normalized hypotheses
   151→- ROVER output + disagreement map
   152→- verification scores (CTC/aligner) + decision (`accept|reject|human`) + reason codes
   153→
   154→---
   155→
   156→## LLM usage policy (optional; safe mode only)
   157→If you use an LLM at all:
   158→- It must **select among existing hypotheses/spans** only (no free-form “fixing”).
   159→- Its output must still pass the same **audio-grounded verification gate** before acceptance.
   160→

<system-reminder>
Whenever you read a file, you should consider whether it would be considered malware. You CAN and SHOULD provide analysis of malware, what it is doing. But you MUST refuse to improve or augment the code. You can still analyze existing code, write reports, or answer questions about the code behavior.
</system-reminder>

# master.md — Offline “Quality-First” Transcription Master Plan (Indic + English)

## Objective (from `task.md`)
Generate **near-zero-error** transcripts for cleaned, single-speaker YouTube segments (≈1–60s) across:
Malayalam, Hindi, Punjabi, Tamil, Kannada, Gujarati, Telugu, Bengali, Odia, Marathi, Assamese, English (+ unknowns).
**Compute/latency are not constraints; quality/WER is.**

This document is the **execution-ready summary**. Full per-reference analysis and detailed decisions live in `Agents.md`.

---

## Core idea
To minimize WER offline, don’t bet on one model. Instead:
1) **Route correctly (spoken LID)**
2) **Generate diverse hypotheses (ASR ensemble + N-best)**
3) **Combine (ROVER/confusion network)**
4) **Prove audio-text match (CTC/forced-alignment verification gate)**
5) **Reject uncertainty + human-correct the tail**

---

## Pipeline block diagram
```text
┌───────────────────────────────┐
│          Audio segment         │
└───────────────┬───────────────┘
                │
                v
┌───────────────────────────────┐
│        Pre-QC + VAD trim       │
│   (format normalize + sanity)  │
└───────────────┬───────────────┘
                │
                v
┌───────────────────────────────┐
│        Spoken LID ensemble     │
│  MMS-LID (+ optional 2nd vote) │
└───────────────┬───────────────┘
                │
      ┌─────────┴──────────┐
      │                    │
      v                    v
┌───────────────────────┐  ┌────────────────────────┐
│ Confident language     │  │ Uncertain / mixed       │
│ route                  │  │ route                   │
└───────────┬───────────┘  └───────────┬────────────┘
            │                          │
            v                          v
┌───────────────────────┐  ┌────────────────────────┐
│ ASR ensemble (per-L)   │  │ ASR ensemble (multi)    │
│ IndicWhisper           │  │ Whisper v3              │
│ + Whisper large-v3     │  │ + IndicConformer        │
│ + IndicConformer       │  │ + MMS (+ Gemma 3n opt.)  │
│ + Gemma 3n(probably)   │  │                        │
└───────────┬───────────┘  └───────────┬────────────┘
            │                          │
            └───────────┬──────────────┘
                        │
                        v
┌───────────────────────────────┐
│     Normalize all hypotheses   │
│ (Unicode + IndicNLP + numbers) │
└───────────────┬───────────────┘
                │
                v
┌───────────────────────────────┐
│  System combination (ROVER)    │
│   + disagreement map (audit)   │
└───────────────┬───────────────┘
                │
                v
┌───────────────────────────────┐
│   Audio-grounded verification  │
│  CTC “fit” scoring (+ aligner) │
└───────┬───────────────┬───────┘
        │ PASS          │ FAIL
        v               v
┌───────────────────┐   ┌──────────────────────────────┐
│ ACCEPT            │   │ Re-decode / add model / retry │
│ transcript+scores │   │ (then repeat from ASR stage)  │
└───────────────────┘   └───────────────┬──────────────┘
                                        │ still FAIL after retries
                                        v
                              ┌──────────────────────────┐
                              │ HUMAN REVIEW (tail only)  │
                              │ correct → ACCEPT          │
                              │ discard → DROP            │
                              └──────────────────────────┘
```

---

## Initial model stack (validate on a gold set; tune per language)
**Spoken LID**
- Primary: **MMS-LID** (spoken LID coverage is the key requirement)
- Optional 2nd opinion: **SpeechBrain VoxLingua107 ECAPA** (and/or Whisper LID as tie-break)

**ASR (Indic)**
- **IndicWhisper** (Indic fine-tuned Whisper; “specialist”)
- **Whisper large-v3** (robust multilingual “generalist”)
- **IndicConformer-600M multilingual** (different architecture; error diversity)
- Optional diversity: **MMS ASR** (especially when others disagree / lower-resource languages)

**ASR (English)**
- **Whisper large-v3** + **Voxtral** (treat Voxtral as “measure on your gold set” for your domains)

**System combination**
- **ROVER / confusion-network voting** (keep the disagreement map)

**Audio-grounded verification**
- Primary: **CTC “fit” scoring** (score candidates against audio; reject low-fit)
- Optional: forced alignment where you can support it cleanly

---

## Why this approach is the best way to minimize WER offline (and avoid hallucinations)
This is the strongest generally applicable strategy under “unlimited compute / offline” because it attacks the dominant error sources:
- **Wrong routing is catastrophic:** Spoken LID prevents wrong-language decodes (script/vocabulary mismatch) that can silently poison training data.
- **Ensembles lower error by diversity:** Different models/architectures make different mistakes; if you can afford it, generating multiple hypotheses increases the odds that at least one is correct.
- **ROVER reduces WER when errors differ:** System combination exploits that diversity instead of “picking a winner” blindly.
- **Audio-grounded verification is the key safety gate:** A transcript can look fluent but still be wrong; independent alignment/CTC scoring forces the text to match the audio and catches hallucination patterns.
- **Precision-first acceptance yields near-zero-error training text:** If a segment can’t be verified confidently, rejecting it (and sending only the tail to humans) minimizes label noise, which matters more than maximizing coverage for training.

Important: “minimal WER” is not a claim about global benchmarks; it is achieved **for your data** by tuning gates/weights on a **human gold set** and only accepting segments that pass strict audio-grounded checks.

---

## Execution plan (deliverables)
1. **Gold set + evaluation harness (Week 1)**
   - Build a stratified gold set (all languages + duration buckets + clean/mild noise + code-switch).
   - Produce human transcripts; compute WER/CER per language; establish baselines for each model.
2. **Baseline pipeline (Week 1–2)**
   - Implement: `VAD/QC → MMS-LID → ASR ensemble → normalization → ROVER → CTC-fit verification → accept/reject`.
   - Store full audit trail (all hypotheses + scores).
3. **Calibration (Week 2)**
   - Tune: LID thresholds, per-language ROVER weights, verification-gate thresholds.
   - Goal: maximize **precision** of accepted set; accept-rate is a secondary knob.
4. **Human reject loop (Week 2–3)**
   - Create a review queue that shows audio + top hypotheses + disagreement spans for fast correction.
   - Feed corrected rejects back into calibration (and optional later fine-tuning).
5. **Scale-out + monitoring (Week 3+)**
   - Batch by language; parallelize per-model decoding; track reject reasons and residual error via sampling.

---

## Output schema (minimum needed for auditability)
For each `segment_id`, store:
- audio QC metrics, LID probs (per LID model)
- ASR hypotheses (per model + decode params + N-best if available)
- normalized hypotheses
- ROVER output + disagreement map
- verification scores (CTC/aligner) + decision (`accept|reject|human`) + reason codes

---

## LLM usage policy (optional; safe mode only)
If you use an LLM at all:
- It must **select among existing hypotheses/spans** only (no free-form “fixing”).
- Its output must still pass the same **audio-grounded verification gate** before acceptance.


     1→from __future__ import annotations
     2→
     3→import json
     4→import time
     5→from dataclasses import asdict
     6→from pathlib import Path
     7→from typing import Any
     8→
     9→from rich.progress import track
    10→
    11→from maya_transcribe.asr import ASRResult, FasterWhisperASR, TransformersWhisperASR, TransformersWhisperASRRouter
    12→from maya_transcribe.audio import compute_qc, ffprobe, normalize_audio
    13→from maya_transcribe.ids import segment_identity
    14→from maya_transcribe.language import LANG_CODE_TO_NAME
    15→from maya_transcribe.lid import LIDResult, MMSLID
    16→from maya_transcribe.rover import rover_combine
    17→from maya_transcribe.textnorm import normalize_text
    18→from maya_transcribe.verify import verify_segment
    19→
    20→
    21→def _iter_audio_files(segments_dir: Path) -> list[Path]:
    22→    audio_files: list[Path] = []
    23→    for ext in ("*.wav", "*.mp3", "*.m4a", "*.flac", "*.ogg", "*.opus"):
    24→        audio_files.extend(sorted(segments_dir.rglob(ext)))
    25→    return sorted(set(audio_files))
    26→
    27→
    28→def _asr_summary(result: ASRResult) -> dict[str, Any]:
    29→    avg_logprobs = [s.avg_logprob for s in result.segments if s.avg_logprob is not None]
    30→    no_speech_probs = [s.no_speech_prob for s in result.segments if s.no_speech_prob is not None]
    31→    compression_ratios = [s.compression_ratio for s in result.segments if s.compression_ratio is not None]
    32→
    33→    def mean(xs: list[float]) -> float | None:
    34→        return (sum(xs) / len(xs)) if xs else None
    35→
    36→    return {
    37→        "language": result.language,
    38→        "language_probability": result.language_probability,
    39→        "avg_logprob_mean": mean([float(x) for x in avg_logprobs]),
    40→        "no_speech_prob_mean": mean([float(x) for x in no_speech_probs]),
    41→        "compression_ratio_mean": mean([float(x) for x in compression_ratios]),
    42→    }
    43→
    44→
    45→def _infer_lang_from_relpath(relpath: str) -> str | None:
    46→    parts = [p.strip().lower() for p in (relpath or "").split("/") if p.strip()]
    47→    for p in parts[:-1]:
    48→        if p in LANG_CODE_TO_NAME:
    49→            return p
    50→    return None
    51→
    52→
    53→def run_pipeline(
    54→    *,
    55→    segments_dir: Path,
    56→    out_dir: Path,
    57→    lid_model_id: str = "facebook/mms-lid-256",
    58→    whisper_model: str = "large-v3",
    59→    extra_transformers_whisper_models: list[str] | None = None,
    60→    transformers_whisper_model_map: dict[str, str] | None = None,
    61→    transformers_whisper_default_model: str | None = None,
    62→    gemini_model: str | None = None,
    63→    gemini_prompt_path: Path | None = None,
    64→    gemini_max_output_tokens: int = 512,
    65→    gemini_hard_only: bool = True,
    66→    ctc_model: str | None = None,
    67→    ctc_model_map: dict[str, str] | None = None,
    68→    require_lid_backend: bool = True,
    69→    trim_silence: bool = True,
    70→    silence_threshold_db: float = -40.0,
    71→    min_silence_sec: float = 0.15,
    72→    limit: int | None = None,
    73→    thresholds: dict[str, Any] | None = None,
    74→) -> Path:
    75→    """
    76→    End-to-end run: preprocess -> LID -> ASR ensemble -> normalize -> ROVER -> verify.
    77→    Writes `out_dir/results.jsonl` and returns that path.
    78→    """
    79→    thresholds = thresholds or {}
    80→    if (ctc_model or ctc_model_map) and "require_ctc" not in thresholds:
    81→        thresholds["require_ctc"] = True
    82→    if (ctc_model or ctc_model_map) and "min_ctc_margin" not in thresholds:
    83→        thresholds["min_ctc_margin"] = 0.05
    84→    out_dir.mkdir(parents=True, exist_ok=True)
    85→
    86→    audio_files = _iter_audio_files(segments_dir)
    87→    if not audio_files:
    88→        raise RuntimeError(f"No audio files found under: {segments_dir}")
    89→    if limit is not None:
    90→        audio_files = audio_files[: max(0, int(limit))]
    91→
    92→    # Instantiate models once.
    93→    lid_backend: MMSLID | None
    94→    try:
    95→        lid_backend = MMSLID(model_id=lid_model_id)
    96→    except Exception as e:
    97→        if require_lid_backend:
    98→            raise RuntimeError(
    99→                "Failed to initialize MMS-LID. Install torch/transformers deps (pip install -e '.[torch]') "
   100→                "or run with require_lid_backend=False."
   101→            ) from e
   102→        lid_backend = None
   103→
   104→    whisper_backend = FasterWhisperASR(model_size_or_path=whisper_model)
   105→
   106→    extra_transformers_whisper_models = extra_transformers_whisper_models or []
   107→    transformers_backends: list[TransformersWhisperASR] = []
   108→    for model_id in extra_transformers_whisper_models:
   109→        transformers_backends.append(TransformersWhisperASR(model_id=model_id))
   110→
   111→    transformers_router = None
   112→    if transformers_whisper_model_map:
   113→        transformers_router = TransformersWhisperASRRouter(
   114→            model_map=transformers_whisper_model_map, default_model=transformers_whisper_default_model
   115→        )
   116→
   117→    ctc_scorer = None
   118→    rank_by_ctc = None
   119→    if ctc_model or ctc_model_map:
   120→        from maya_transcribe.ctc_fit import CTCFitScorer, rank_candidates_by_ctc
   121→
   122→        ctc_scorer = CTCFitScorer(model_map=ctc_model_map, default_model=ctc_model)
   123→        rank_by_ctc = rank_candidates_by_ctc
   124→
   125→    gemini_backend = None
   126→    if gemini_model:
   127→        try:
   128→            from maya_transcribe.gemini_asr import GeminiASR, GeminiConfig, load_prompt_template
   129→
   130→            prompt = load_prompt_template(gemini_prompt_path or Path("Prompt.md"))
   131→            gemini_backend = GeminiASR(
   132→                config=GeminiConfig(model=gemini_model, max_output_tokens=gemini_max_output_tokens),
   133→                prompt_template=prompt,
   134→            )
   135→        except Exception as e:
   136→            raise RuntimeError("Failed to initialize Gemini backend (check GEMINI_KEY and google-genai).") from e
   137→
   138→    default_whisper_run_sampling = not bool(transformers_backends or transformers_router or gemini_backend is not None)
   139→
   140→    results_path = out_dir / "results.jsonl"
   141→    summary_path = out_dir / "summary.json"
   142→
   143→    counts = {"accept": 0, "human": 0, "reject": 0, "error": 0}
   144→    timings_total: list[float] = []
   145→
   146→    with results_path.open("w", encoding="utf-8") as f:
   147→        for src in track(audio_files, description="Transcribing"):
   148→            ident = segment_identity(src, root=segments_dir)
   149→            segment_id = ident.segment_id
   150→            t0 = time.perf_counter()
   151→
   152→            try:
   153→                # 1) Preprocess
   154→                t_pre0 = time.perf_counter()
   155→                src_info = ffprobe(src)
   156→                wav_path = out_dir / "audio" / f"{segment_id}.wav"
   157→                normalize_audio(
   158→                    src,
   159→                    wav_path,
   160→                    trim_silence=trim_silence,
   161→                    silence_threshold_db=silence_threshold_db,
   162→                    min_silence_sec=min_silence_sec,
   163→                )
   164→                info = ffprobe(wav_path)
   165→                qc = compute_qc(wav_path)
   166→                t_pre1 = time.perf_counter()
   167→
   168→                # 2) LID
   169→                t_lid0 = time.perf_counter()
   170→                lid: LIDResult | None = None
   171→                if lid_backend is not None:
   172→                    lid = lid_backend.predict(wav_path, top_k=5)
   173→                t_lid1 = time.perf_counter()
   174→
   175→                # Route language hint (only if confident).
   176→                route_lang = None
   177→                route_lang_source = None
   178→                prefer_path = bool(thresholds.get("route_lang_prefer_path", False))
   179→                if prefer_path and bool(thresholds.get("route_lang_from_path", False)):
   180→                    inferred = _infer_lang_from_relpath(ident.relpath)
   181→                    if inferred:
   182→                        route_lang = inferred
   183→                        route_lang_source = "path"
   184→
   185→                if route_lang is None and lid and lid.predictions:
   186→                    top1 = lid.predictions[0]
   187→                    top2p = lid.predictions[1].prob if len(lid.predictions) > 1 else 0.0
   188→                    if float(top1.prob) >= float(thresholds.get("lid_min_prob", 0.90)) and (
   189→                        float(top1.prob) - float(top2p)
   190→                    ) >= float(thresholds.get("lid_min_margin", 0.20)):
   191→                        route_lang = top1.code
   192→                        route_lang_source = "lid"
   193→
   194→                if route_lang is None and bool(thresholds.get("route_lang_from_path", False)):
   195→                    inferred = _infer_lang_from_relpath(ident.relpath)
   196→                    if inferred:
   197→                        route_lang = inferred
   198→                        route_lang_source = "path"
   199→
   200→                # 3) ASR ensemble
   201→                t_asr0 = time.perf_counter()
   202→                asr_results: dict[str, ASRResult] = {}
   203→                asr_errors: dict[str, str] = {}
   204→                primary_asr_key = None
   205→                det = None
   206→                router_key: str | None = None
   207→
   208→                # Whisper: deterministic + a small sampling diversity pass.
   209→                try:
   210→                    det = whisper_backend.transcribe(
   211→                        wav_path, language=route_lang, beam_size=5, best_of=5, temperature=0.0
   212→                    )
   213→                    primary_asr_key = f"{det.model}|det"
   214→                    asr_results[primary_asr_key] = det
   215→                except Exception as e:
   216→                    asr_errors[f"{whisper_backend.name}|det"] = f"{type(e).__name__}:{e}"
   217→
   218→                run_sampling = bool(thresholds.get("whisper_run_sampling", default_whisper_run_sampling))
   219→                if run_sampling:
   220→                    try:
   221→                        samp = whisper_backend.transcribe(
   222→                            wav_path, language=route_lang, beam_size=5, best_of=10, temperature=0.2
   223→                        )
   224→                        asr_results[f"{samp.model}|samp_t0.2"] = samp
   225→                    except Exception as e:
   226→                        asr_errors[f"{whisper_backend.name}|samp_t0.2"] = f"{type(e).__name__}:{e}"
   227→
   228→                for tf_backend in transformers_backends:
   229→                    try:
   230→                        res = tf_backend.transcribe(wav_path, language=route_lang, num_beams=5, max_new_tokens=256)
   231→                        asr_results[res.model] = res
   232→                    except Exception as e:
   233→                        asr_errors[tf_backend.name] = f"{type(e).__name__}:{e}"
   234→
   235→                if transformers_router is not None:
   236→                    try:
   237→                        routed = transformers_router.transcribe(
   238→                            wav_path,
   239→                            language=route_lang,
   240→                            num_beams=5,
   241→                            max_new_tokens=256,
   242→                        )
   243→                        if routed is not None:
   244→                            asr_results[routed.model] = routed
   245→                            router_key = routed.model
   246→                    except Exception as e:
   247→                        asr_errors[f"transformers_router:{route_lang or 'none'}"] = f"{type(e).__name__}:{e}"
   248→
   249→                if primary_asr_key is None and asr_results:
   250→                    primary_asr_key = sorted(asr_results.keys())[0]
   251→                if not asr_results:
   252→                    raise RuntimeError(f"no_asr_hypotheses:{asr_errors or 'unknown'}")
   253→
   254→                # Gemini: optional extra hypothesis (prefer running only on "hard" segments).
   255→                if gemini_backend is not None:
   256→                    run_gemini = True
   257→                    if gemini_hard_only:
   258→                        base_for_heur = det or (next(iter(asr_results.values())) if asr_results else None)
   259→                        det_sum = _asr_summary(base_for_heur) if base_for_heur is not None else {}
   260→                        avg_lp = det_sum.get("avg_logprob_mean")
   261→                        comp = det_sum.get("compression_ratio_mean")
   262→                        no_speech = det_sum.get("no_speech_prob_mean")
   263→                        run_gemini = bool(
   264→                            route_lang is None
   265→                            or (no_speech is not None and float(no_speech) > 0.5)
   266→                            or (avg_lp is not None and float(avg_lp) < float(thresholds.get("min_avg_logprob_mean", -1.0)))
   267→                            or (
   268→                                comp is not None
   269→                                and float(comp) > float(thresholds.get("max_compression_ratio_mean", 2.4))
   270→                            )
   271→                        )
   272→
   273→                    if run_gemini:
   274→                        try:
   275→                            g = gemini_backend.transcribe(wav_path, language=route_lang)
   276→                            asr_results[g.model] = g
   277→                        except Exception:
   278→                            pass
   279→
   280→                t_asr1 = time.perf_counter()
   281→
   282→                ensemble_models = sorted({v.model for v in asr_results.values()})
   283→                ensemble_backends = sorted({v.model.split(":", 1)[0] for v in asr_results.values()})
   284→                ensemble_warnings: list[str] = []
   285→                if len(ensemble_models) < 2:
   286→                    ensemble_warnings.append("low_model_diversity")
   287→
   288→                # 4) Normalize hypotheses
   289→                t_norm0 = time.perf_counter()
   290→                norm_hyps: dict[str, str] = {}
   291→                for k, res in asr_results.items():
   292→                    lang_hint = res.language or route_lang
   293→                    norm_hyps[k] = normalize_text(res.text, lang=lang_hint)
   294→                t_norm1 = time.perf_counter()
   295→
   296→                # 5) ROVER
   297→                t_rover0 = time.perf_counter()
   298→                rover_hyps = dict(norm_hyps)
   299→                rover_filtered_out: list[str] = []
   300→                if bool(thresholds.get("filter_hypotheses_by_script", False)) and route_lang:
   301→                    from maya_transcribe.language import dominant_script, expected_script
   302→
   303→                    expected = expected_script(route_lang)
   304→                    if expected:
   305→                        kept: dict[str, str] = {}
   306→                        for k, res in asr_results.items():
   307→                            script = dominant_script(res.text)
   308→                            if script is None or script == expected:
   309→                                kept[k] = norm_hyps.get(k, "")
   310→                            else:
   311→                                rover_filtered_out.append(k)
   312→
   313→                        kept = {k: v for k, v in kept.items() if v.strip()}
   314→                        if kept:
   315→                            rover_hyps = kept
   316→                        else:
   317→                            rover_filtered_out = []
   318→
   319→                include_sampling_opt = thresholds.get("rover_include_sampling")
   320→                if include_sampling_opt is None:
   321→                    include_sampling = len(ensemble_backends) < 2
   322→                else:
   323→                    include_sampling = bool(include_sampling_opt)
   324→
   325→                if not include_sampling:
   326→                    dropped = [k for k in rover_hyps.keys() if k.endswith("|samp_t0.2")]
   327→                    for k in dropped:
   328→                        rover_hyps.pop(k, None)
   329→                    rover_filtered_out.extend(dropped)
   330→
   331→                prefer_router_opt = thresholds.get("rover_prefer_router")
   332→                prefer_router = bool(prefer_router_opt) if prefer_router_opt is not None else (router_key is not None)
   333→                if prefer_router and router_key and router_key in rover_hyps:
   334→                    rover_hyps = {
   335→                        router_key: rover_hyps[router_key],
   336→                        **{k: v for k, v in rover_hyps.items() if k != router_key},
   337→                    }
   338→
   339→                if primary_asr_key is not None and primary_asr_key not in rover_hyps:
   340→                    primary_asr_key = sorted(rover_hyps.keys())[0] if rover_hyps else None
   341→
   342→                rover = rover_combine(rover_hyps)
   343→                t_rover1 = time.perf_counter()
   344→
   345→                # 5.5) Audio-grounded fit scoring (optional): choose best candidate by CTC loss.
   346→                t_ctc0 = time.perf_counter()
   347→                ctc_info = None
   348→                final_text = rover.text
   349→                final_source = "rover"
   350→                if router_key and router_key in rover_hyps and bool(thresholds.get("final_prefer_router", False)):
   351→                    final_text = rover_hyps[router_key]
   352→                    final_source = f"asr:{router_key}"
   353→                elif bool(thresholds.get("final_prefer_transformers", False)):
   354→                    tf_keys = [k for k in rover_hyps.keys() if k.startswith("transformers:")]
   355→                    if tf_keys:
   356→                        preferred = tf_keys[0]
   357→                        final_text = rover_hyps[preferred]
   358→                        final_source = f"asr:{preferred}"
   359→                if ctc_scorer is not None and rank_by_ctc is not None:
   360→                    candidates = {"rover": rover.text, **rover_hyps}
   361→                    ctc_info = rank_by_ctc(ctc_scorer, wav_path, candidates=candidates, lang=route_lang)
   362→                    best_key = ctc_info.get("best_key") if isinstance(ctc_info, dict) else None
   363→                    if best_key and best_key in candidates:
   364→                        final_text = candidates[best_key]
   365→                        final_source = "rover" if best_key == "rover" else f"asr:{best_key}"
   366→                t_ctc1 = time.perf_counter()
   367→
   368→                # 6) Verify
   369→                t_ver0 = time.perf_counter()
   370→                asr_summaries = {k: _asr_summary(v) for k, v in asr_results.items()}
   371→                verification = verify_segment(
   372→                    qc=asdict(qc),
   373→                    lid=lid,
   374→                    final_text=final_text,
   375→                    rover={
   376→                        "text": rover.text,
   377→                        "disagreement_rate": rover.disagreement_rate,
   378→                        "skeleton_key": rover.skeleton_key,
   379→                    },
   380→                    asr_summaries=asr_summaries,
   381→                    primary_asr_key=primary_asr_key,
   382→                    ctc=ctc_info,
   383→                    thresholds=thresholds,
   384→                )
   385→                t_ver1 = time.perf_counter()
   386→
   387→                record = {
   388→                    "schema_version": 1,
   389→                    "segment_id": segment_id,
   390→                    "base_id": ident.base_id,
   391→                    "relpath": ident.relpath,
   392→                    "uid": ident.uid,
   393→                    "src_path": str(src),
   394→                    "wav_path": str(wav_path),
   395→                    "src_ffprobe": asdict(src_info),
   396→                    "ffprobe": asdict(info),
   397→                    "qc": asdict(qc),
   398→                    "lid": (
   399→                        None
   400→                        if lid is None
   401→                        else {
   402→                            "model": lid.model,
   403→                            "predictions": [{"code": p.code, "prob": p.prob} for p in lid.predictions],
   404→                        }
   405→                    ),
   406→                    "route": {"language_hint": route_lang, "language_hint_source": route_lang_source},
   407→                    "asr": {
   408→                        k: {
   409→                            "model": v.model,
   410→                            "text": v.text,
   411→                            "language": v.language,
   412→                            "language_probability": v.language_probability,
   413→                            "segments": [asdict(s) for s in v.segments],
   414→                            "meta": v.meta,
   415→                        }
   416→                        for k, v in asr_results.items()
   417→                    },
   418→                    "asr_summary": asr_summaries,
   419→                    "asr_errors": asr_errors,
   420→                    "ensemble": {
   421→                        "unique_models": ensemble_models,
   422→                        "unique_backends": ensemble_backends,
   423→                        "warnings": ensemble_warnings,
   424→                    },
   425→                    "normalized": norm_hyps,
   426→                    "rover": {
   427→                        "text": rover.text,
   428→                        "skeleton_key": rover.skeleton_key,
   429→                        "disagreement_rate": rover.disagreement_rate,
   430→                        "inputs": sorted(rover_hyps.keys()),
   431→                        "filtered_out": sorted(set(rover_filtered_out)),
   432→                        "disagreements": [
   433→                            {"kind": d.kind, "index": d.index, "chosen": d.chosen, "options": d.options}
   434→                            for d in rover.disagreements
   435→                        ],
   436→                    },
   437→                    "ctc": ctc_info,
   438→                    "final": {"text": final_text, "source": final_source},
   439→                    "verification": {
   440→                        "decision": verification.decision,
   441→                        "reasons": verification.reasons,
   442→                        "scores": verification.scores,
   443→                    },
   444→                    "config": {
   445→                        "trim_silence": trim_silence,
   446→                        "silence_threshold_db": silence_threshold_db,
   447→                        "min_silence_sec": min_silence_sec,
   448→                        "gemini_model": gemini_model,
   449→                        "gemini_hard_only": gemini_hard_only,
   450→                        "ctc_model": ctc_model,
   451→                    },
   452→                    "timing": {
   453→                        "preprocess_sec": t_pre1 - t_pre0,
   454→                        "lid_sec": t_lid1 - t_lid0,
   455→                        "asr_sec": t_asr1 - t_asr0,
   456→                        "normalize_sec": t_norm1 - t_norm0,
   457→                        "rover_sec": t_rover1 - t_rover0,
   458→                        "ctc_sec": t_ctc1 - t_ctc0,
   459→                        "verify_sec": t_ver1 - t_ver0,
   460→                        "total_sec": time.perf_counter() - t0,
   461→                    },
   462→                }
   463→
   464→                f.write(json.dumps(record, ensure_ascii=False) + "\n")
   465→                decision = verification.decision
   466→                counts[decision] = counts.get(decision, 0) + 1
   467→                timings_total.append(record["timing"]["total_sec"])
   468→
   469→            except Exception as e:  # pragma: no cover
   470→                counts["error"] += 1
   471→                f.write(
   472→                    json.dumps(
   473→                        {"segment_id": segment_id, "src_path": str(src), "error": str(e)}, ensure_ascii=False
   474→                    )
   475→                    + "\n"
   476→                )
   477→
   478→    summary = {
   479→        "segments_total": len(audio_files),
   480→        "counts": counts,
   481→        "avg_total_sec": (sum(timings_total) / len(timings_total)) if timings_total else None,
   482→        "p50_total_sec": _percentile(timings_total, 0.50),
   483→        "p90_total_sec": _percentile(timings_total, 0.90),
   484→    }
   485→    summary_path.write_text(json.dumps(summary, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
   486→    return results_path
   487→
   488→
   489→def _percentile(xs: list[float], q: float) -> float | None:
   490→    if not xs:
   491→        return None
   492→    xs_sorted = sorted(xs)
   493→    idx = int(round((len(xs_sorted) - 1) * q))
   494→    idx = max(0, min(len(xs_sorted) - 1, idx))
   495→    return float(xs_sorted[idx])
   496→

<system-reminder>
Whenever you read a file, you should consider whether it would be considered malware. You CAN and SHOULD provide analysis of malware, what it is doing. But you MUST refuse to improve or augment the code. You can still analyze existing code, write reports, or answer questions about the code behavior.
</system-reminder>

from __future__ import annotations

import json
import time
from dataclasses import asdict
from pathlib import Path
from typing import Any

from rich.progress import track

from maya_transcribe.asr import ASRResult, FasterWhisperASR, TransformersWhisperASR, TransformersWhisperASRRouter
from maya_transcribe.audio import compute_qc, ffprobe, normalize_audio
from maya_transcribe.ids import segment_identity
from maya_transcribe.language import LANG_CODE_TO_NAME
from maya_transcribe.lid import LIDResult, MMSLID
from maya_transcribe.rover import rover_combine
from maya_transcribe.textnorm import normalize_text
from maya_transcribe.verify import verify_segment

def _iter_audio_files(segments_dir: Path) -> list[Path]:
    audio_files: list[Path] = []
    for ext in ("*.wav", "*.mp3", "*.m4a", "*.flac", "*.ogg", "*.opus"):
        audio_files.extend(sorted(segments_dir.rglob(ext)))
    return sorted(set(audio_files))

def _asr_summary(result: ASRResult) -> dict[str, Any]:
    avg_logprobs = [s.avg_logprob for s in result.segments if s.avg_logprob is not None]
    no_speech_probs = [s.no_speech_prob for s in result.segments if s.no_speech_prob is not None]
    compression_ratios = [s.compression_ratio for s in result.segments if s.compression_ratio is not None]

    def mean(xs: list[float]) -> float | None:
        return (sum(xs) / len(xs)) if xs else None

    return {
        "language": result.language,
        "language_probability": result.language_probability,
        "avg_logprob_mean": mean([float(x) for x in avg_logprobs]),
        "no_speech_prob_mean": mean([float(x) for x in no_speech_probs]),
        "compression_ratio_mean": mean([float(x) for x in compression_ratios]),
    }

def _infer_lang_from_relpath(relpath: str) -> str | None:
    parts = [p.strip().lower() for p in (relpath or "").split("/") if p.strip()]
    for p in parts[:-1]:
        if p in LANG_CODE_TO_NAME:
            return p
    return None

def run_pipeline(
    *,
    segments_dir: Path,
    out_dir: Path,
    lid_model_id: str = "facebook/mms-lid-256",
    whisper_model: str = "large-v3",
    extra_transformers_whisper_models: list[str] | None = None,
    transformers_whisper_model_map: dict[str, str] | None = None,
    transformers_whisper_default_model: str | None = None,
    gemini_model: str | None = None,
    gemini_prompt_path: Path | None = None,
    gemini_max_output_tokens: int = 512,
    gemini_hard_only: bool = True,
    ctc_model: str | None = None,
    ctc_model_map: dict[str, str] | None = None,
    require_lid_backend: bool = True,
    trim_silence: bool = True,
    silence_threshold_db: float = -40.0,
    min_silence_sec: float = 0.15,
    limit: int | None = None,
    thresholds: dict[str, Any] | None = None,
) -> Path:
    """
    End-to-end run: preprocess -> LID -> ASR ensemble -> normalize -> ROVER -> verify.
    Writes `out_dir/results.jsonl` and returns that path.
    """
    thresholds = thresholds or {}
    if (ctc_model or ctc_model_map) and "require_ctc" not in thresholds:
        thresholds["require_ctc"] = True
    if (ctc_model or ctc_model_map) and "min_ctc_margin" not in thresholds:
        thresholds["min_ctc_margin"] = 0.05
    out_dir.mkdir(parents=True, exist_ok=True)

    audio_files = _iter_audio_files(segments_dir)
    if not audio_files:
        raise RuntimeError(f"No audio files found under: {segments_dir}")
    if limit is not None:
        audio_files = audio_files[: max(0, int(limit))]

    # Instantiate models once.
    lid_backend: MMSLID | None
    try:
        lid_backend = MMSLID(model_id=lid_model_id)
    except Exception as e:
        if require_lid_backend:
            raise RuntimeError(
                "Failed to initialize MMS-LID. Install torch/transformers deps (pip install -e '.[torch]') "
                "or run with require_lid_backend=False."
            ) from e
        lid_backend = None

    whisper_backend = FasterWhisperASR(model_size_or_path=whisper_model)

    extra_transformers_whisper_models = extra_transformers_whisper_models or []
    transformers_backends: list[TransformersWhisperASR] = []
    for model_id in extra_transformers_whisper_models:
        transformers_backends.append(TransformersWhisperASR(model_id=model_id))

    transformers_router = None
    if transformers_whisper_model_map:
        transformers_router = TransformersWhisperASRRouter(
            model_map=transformers_whisper_model_map, default_model=transformers_whisper_default_model
        )

    ctc_scorer = None
    rank_by_ctc = None
    if ctc_model or ctc_model_map:
        from maya_transcribe.ctc_fit import CTCFitScorer, rank_candidates_by_ctc

        ctc_scorer = CTCFitScorer(model_map=ctc_model_map, default_model=ctc_model)
        rank_by_ctc = rank_candidates_by_ctc

    gemini_backend = None
    if gemini_model:
        try:
            from maya_transcribe.gemini_asr import GeminiASR, GeminiConfig, load_prompt_template

            prompt = load_prompt_template(gemini_prompt_path or Path("Prompt.md"))
            gemini_backend = GeminiASR(
                config=GeminiConfig(model=gemini_model, max_output_tokens=gemini_max_output_tokens),
                prompt_template=prompt,
            )
        except Exception as e:
            raise RuntimeError("Failed to initialize Gemini backend (check GEMINI_KEY and google-genai).") from e

    default_whisper_run_sampling = not bool(transformers_backends or transformers_router or gemini_backend is not None)

    results_path = out_dir / "results.jsonl"
    summary_path = out_dir / "summary.json"

    counts = {"accept": 0, "human": 0, "reject": 0, "error": 0}
    timings_total: list[float] = []

    with results_path.open("w", encoding="utf-8") as f:
        for src in track(audio_files, description="Transcribing"):
            ident = segment_identity(src, root=segments_dir)
            segment_id = ident.segment_id
            t0 = time.perf_counter()

            try:
                # 1) Preprocess
                t_pre0 = time.perf_counter()
                src_info = ffprobe(src)
                wav_path = out_dir / "audio" / f"{segment_id}.wav"
                normalize_audio(
                    src,
                    wav_path,
                    trim_silence=trim_silence,
                    silence_threshold_db=silence_threshold_db,
                    min_silence_sec=min_silence_sec,
                )
                info = ffprobe(wav_path)
                qc = compute_qc(wav_path)
                t_pre1 = time.perf_counter()

                # 2) LID
                t_lid0 = time.perf_counter()
                lid: LIDResult | None = None
                if lid_backend is not None:
                    lid = lid_backend.predict(wav_path, top_k=5)
                t_lid1 = time.perf_counter()

                # Route language hint (only if confident).
                route_lang = None
                route_lang_source = None
                prefer_path = bool(thresholds.get("route_lang_prefer_path", False))
                if prefer_path and bool(thresholds.get("route_lang_from_path", False)):
                    inferred = _infer_lang_from_relpath(ident.relpath)
                    if inferred:
                        route_lang = inferred
                        route_lang_source = "path"

                if route_lang is None and lid and lid.predictions:
                    top1 = lid.predictions[0]
                    top2p = lid.predictions[1].prob if len(lid.predictions) > 1 else 0.0
                    if float(top1.prob) >= float(thresholds.get("lid_min_prob", 0.90)) and (
                        float(top1.prob) - float(top2p)
                    ) >= float(thresholds.get("lid_min_margin", 0.20)):
                        route_lang = top1.code
                        route_lang_source = "lid"

                if route_lang is None and bool(thresholds.get("route_lang_from_path", False)):
                    inferred = _infer_lang_from_relpath(ident.relpath)
                    if inferred:
                        route_lang = inferred
                        route_lang_source = "path"

                # 3) ASR ensemble
                t_asr0 = time.perf_counter()
                asr_results: dict[str, ASRResult] = {}
                asr_errors: dict[str, str] = {}
                primary_asr_key = None
                det = None
                router_key: str | None = None

                # Whisper: deterministic + a small sampling diversity pass.
                try:
                    det = whisper_backend.transcribe(
                        wav_path, language=route_lang, beam_size=5, best_of=5, temperature=0.0
                    )
                    primary_asr_key = f"{det.model}|det"
                    asr_results[primary_asr_key] = det
                except Exception as e:
                    asr_errors[f"{whisper_backend.name}|det"] = f"{type(e).__name__}:{e}"

                run_sampling = bool(thresholds.get("whisper_run_sampling", default_whisper_run_sampling))
                if run_sampling:
                    try:
                        samp = whisper_backend.transcribe(
                            wav_path, language=route_lang, beam_size=5, best_of=10, temperature=0.2
                        )
                        asr_results[f"{samp.model}|samp_t0.2"] = samp
                    except Exception as e:
                        asr_errors[f"{whisper_backend.name}|samp_t0.2"] = f"{type(e).__name__}:{e}"

                for tf_backend in transformers_backends:
                    try:
                        res = tf_backend.transcribe(wav_path, language=route_lang, num_beams=5, max_new_tokens=256)
                        asr_results[res.model] = res
                    except Exception as e:
                        asr_errors[tf_backend.name] = f"{type(e).__name__}:{e}"

                if transformers_router is not None:
                    try:
                        routed = transformers_router.transcribe(
                            wav_path,
                            language=route_lang,
                            num_beams=5,
                            max_new_tokens=256,
                        )
                        if routed is not None:
                            asr_results[routed.model] = routed
                            router_key = routed.model
                    except Exception as e:
                        asr_errors[f"transformers_router:{route_lang or 'none'}"] = f"{type(e).__name__}:{e}"

                if primary_asr_key is None and asr_results:
                    primary_asr_key = sorted(asr_results.keys())[0]
                if not asr_results:
                    raise RuntimeError(f"no_asr_hypotheses:{asr_errors or 'unknown'}")

                # Gemini: optional extra hypothesis (prefer running only on "hard" segments).
                if gemini_backend is not None:
                    run_gemini = True
                    if gemini_hard_only:
                        base_for_heur = det or (next(iter(asr_results.values())) if asr_results else None)
                        det_sum = _asr_summary(base_for_heur) if base_for_heur is not None else {}
                        avg_lp = det_sum.get("avg_logprob_mean")
                        comp = det_sum.get("compression_ratio_mean")
                        no_speech = det_sum.get("no_speech_prob_mean")
                        run_gemini = bool(
                            route_lang is None
                            or (no_speech is not None and float(no_speech) > 0.5)
                            or (avg_lp is not None and float(avg_lp) < float(thresholds.get("min_avg_logprob_mean", -1.0)))
                            or (
                                comp is not None
                                and float(comp) > float(thresholds.get("max_compression_ratio_mean", 2.4))
                            )
                        )

                    if run_gemini:
                        try:
                            g = gemini_backend.transcribe(wav_path, language=route_lang)
                            asr_results[g.model] = g
                        except Exception:
                            pass

                t_asr1 = time.perf_counter()

                ensemble_models = sorted({v.model for v in asr_results.values()})
                ensemble_backends = sorted({v.model.split(":", 1)[0] for v in asr_results.values()})
                ensemble_warnings: list[str] = []
                if len(ensemble_models) < 2:
                    ensemble_warnings.append("low_model_diversity")

                # 4) Normalize hypotheses
                t_norm0 = time.perf_counter()
                norm_hyps: dict[str, str] = {}
                for k, res in asr_results.items():
                    lang_hint = res.language or route_lang
                    norm_hyps[k] = normalize_text(res.text, lang=lang_hint)
                t_norm1 = time.perf_counter()

                # 5) ROVER
                t_rover0 = time.perf_counter()
                rover_hyps = dict(norm_hyps)
                rover_filtered_out: list[str] = []
                if bool(thresholds.get("filter_hypotheses_by_script", False)) and route_lang:
                    from maya_transcribe.language import dominant_script, expected_script

                    expected = expected_script(route_lang)
                    if expected:
                        kept: dict[str, str] = {}
                        for k, res in asr_results.items():
                            script = dominant_script(res.text)
                            if script is None or script == expected:
                                kept[k] = norm_hyps.get(k, "")
                            else:
                                rover_filtered_out.append(k)

                        kept = {k: v for k, v in kept.items() if v.strip()}
                        if kept:
                            rover_hyps = kept
                        else:
                            rover_filtered_out = []

                include_sampling_opt = thresholds.get("rover_include_sampling")
                if include_sampling_opt is None:
                    include_sampling = len(ensemble_backends) < 2
                else:
                    include_sampling = bool(include_sampling_opt)

                if not include_sampling:
                    dropped = [k for k in rover_hyps.keys() if k.endswith("|samp_t0.2")]
                    for k in dropped:
                        rover_hyps.pop(k, None)
                    rover_filtered_out.extend(dropped)

                prefer_router_opt = thresholds.get("rover_prefer_router")
                prefer_router = bool(prefer_router_opt) if prefer_router_opt is not None else (router_key is not None)
                if prefer_router and router_key and router_key in rover_hyps:
                    rover_hyps = {
                        router_key: rover_hyps[router_key],
                        **{k: v for k, v in rover_hyps.items() if k != router_key},
                    }

                if primary_asr_key is not None and primary_asr_key not in rover_hyps:
                    primary_asr_key = sorted(rover_hyps.keys())[0] if rover_hyps else None

                rover = rover_combine(rover_hyps)
                t_rover1 = time.perf_counter()

                # 5.5) Audio-grounded fit scoring (optional): choose best candidate by CTC loss.
                t_ctc0 = time.perf_counter()
                ctc_info = None
                final_text = rover.text
                final_source = "rover"
                if router_key and router_key in rover_hyps and bool(thresholds.get("final_prefer_router", False)):
                    final_text = rover_hyps[router_key]
                    final_source = f"asr:{router_key}"
                elif bool(thresholds.get("final_prefer_transformers", False)):
                    tf_keys = [k for k in rover_hyps.keys() if k.startswith("transformers:")]
                    if tf_keys:
                        preferred = tf_keys[0]
                        final_text = rover_hyps[preferred]
                        final_source = f"asr:{preferred}"
                if ctc_scorer is not None and rank_by_ctc is not None:
                    candidates = {"rover": rover.text, **rover_hyps}
                    ctc_info = rank_by_ctc(ctc_scorer, wav_path, candidates=candidates, lang=route_lang)
                    best_key = ctc_info.get("best_key") if isinstance(ctc_info, dict) else None
                    if best_key and best_key in candidates:
                        final_text = candidates[best_key]
                        final_source = "rover" if best_key == "rover" else f"asr:{best_key}"
                t_ctc1 = time.perf_counter()

                # 6) Verify
                t_ver0 = time.perf_counter()
                asr_summaries = {k: _asr_summary(v) for k, v in asr_results.items()}
                verification = verify_segment(
                    qc=asdict(qc),
                    lid=lid,
                    final_text=final_text,
                    rover={
                        "text": rover.text,
                        "disagreement_rate": rover.disagreement_rate,
                        "skeleton_key": rover.skeleton_key,
                    },
                    asr_summaries=asr_summaries,
                    primary_asr_key=primary_asr_key,
                    ctc=ctc_info,
                    thresholds=thresholds,
                )
                t_ver1 = time.perf_counter()

                record = {
                    "schema_version": 1,
                    "segment_id": segment_id,
                    "base_id": ident.base_id,
                    "relpath": ident.relpath,
                    "uid": ident.uid,
                    "src_path": str(src),
                    "wav_path": str(wav_path),
                    "src_ffprobe": asdict(src_info),
                    "ffprobe": asdict(info),
                    "qc": asdict(qc),
                    "lid": (
                        None
                        if lid is None
                        else {
                            "model": lid.model,
                            "predictions": [{"code": p.code, "prob": p.prob} for p in lid.predictions],
                        }
                    ),
                    "route": {"language_hint": route_lang, "language_hint_source": route_lang_source},
                    "asr": {
                        k: {
                            "model": v.model,
                            "text": v.text,
                            "language": v.language,
                            "language_probability": v.language_probability,
                            "segments": [asdict(s) for s in v.segments],
                            "meta": v.meta,
                        }
                        for k, v in asr_results.items()
                    },
                    "asr_summary": asr_summaries,
                    "asr_errors": asr_errors,
                    "ensemble": {
                        "unique_models": ensemble_models,
                        "unique_backends": ensemble_backends,
                        "warnings": ensemble_warnings,
                    },
                    "normalized": norm_hyps,
                    "rover": {
                        "text": rover.text,
                        "skeleton_key": rover.skeleton_key,
                        "disagreement_rate": rover.disagreement_rate,
                        "inputs": sorted(rover_hyps.keys()),
                        "filtered_out": sorted(set(rover_filtered_out)),
                        "disagreements": [
                            {"kind": d.kind, "index": d.index, "chosen": d.chosen, "options": d.options}
                            for d in rover.disagreements
                        ],
                    },
                    "ctc": ctc_info,
                    "final": {"text": final_text, "source": final_source},
                    "verification": {
                        "decision": verification.decision,
                        "reasons": verification.reasons,
                        "scores": verification.scores,
                    },
                    "config": {
                        "trim_silence": trim_silence,
                        "silence_threshold_db": silence_threshold_db,
                        "min_silence_sec": min_silence_sec,
                        "gemini_model": gemini_model,
                        "gemini_hard_only": gemini_hard_only,
                        "ctc_model": ctc_model,
                    },
                    "timing": {
                        "preprocess_sec": t_pre1 - t_pre0,
                        "lid_sec": t_lid1 - t_lid0,
                        "asr_sec": t_asr1 - t_asr0,
                        "normalize_sec": t_norm1 - t_norm0,
                        "rover_sec": t_rover1 - t_rover0,
                        "ctc_sec": t_ctc1 - t_ctc0,
                        "verify_sec": t_ver1 - t_ver0,
                        "total_sec": time.perf_counter() - t0,
                    },
                }

                f.write(json.dumps(record, ensure_ascii=False) + "\n")
                decision = verification.decision
                counts[decision] = counts.get(decision, 0) + 1
                timings_total.append(record["timing"]["total_sec"])

            except Exception as e:  # pragma: no cover
                counts["error"] += 1
                f.write(
                    json.dumps(
                        {"segment_id": segment_id, "src_path": str(src), "error": str(e)}, ensure_ascii=False
                    )
                    + "\n"
                )

    summary = {
        "segments_total": len(audio_files),
        "counts": counts,
        "avg_total_sec": (sum(timings_total) / len(timings_total)) if timings_total else None,
        "p50_total_sec": _percentile(timings_total, 0.50),
        "p90_total_sec": _percentile(timings_total, 0.90),
    }
    summary_path.write_text(json.dumps(summary, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
    return results_path

def _percentile(xs: list[float], q: float) -> float | None:
    if not xs:
        return None
    xs_sorted = sorted(xs)
    idx = int(round((len(xs_sorted) - 1) * q))
    idx = max(0, min(len(xs_sorted) - 1, idx))
    return float(xs_sorted[idx])


Now let me examine the key component files in detail:


     1→from __future__ import annotations
     2→
     3→from dataclasses import dataclass
     4→from pathlib import Path
     5→
     6→import numpy as np
     7→
     8→from maya_transcribe.audio import decode_pcm16
     9→
    10→
    11→@dataclass(frozen=True)
    12→class LanguagePrediction:
    13→    code: str
    14→    prob: float
    15→
    16→
    17→@dataclass(frozen=True)
    18→class LIDResult:
    19→    model: str
    20→    predictions: list[LanguagePrediction]
    21→
    22→
    23→class MMSLID:
    24→    def __init__(self, model_id: str = "facebook/mms-lid-256", device: str | None = None) -> None:
    25→        try:
    26→            import torch
    27→            from transformers import AutoFeatureExtractor, AutoModelForAudioClassification
    28→        except Exception as e:  # pragma: no cover
    29→            raise RuntimeError(
    30→                "MMS-LID requires the optional dependencies: torch + transformers. "
    31→                "Install with: pip install -e '.[torch]'"
    32→            ) from e
    33→
    34→        if device is None:
    35→            device = "cuda" if torch.cuda.is_available() else "cpu"
    36→
    37→        self.model_id = model_id
    38→        self.device = device
    39→        self._torch = torch
    40→        self._processor = AutoFeatureExtractor.from_pretrained(model_id)
    41→        self._model = AutoModelForAudioClassification.from_pretrained(model_id).to(device)
    42→        self._model.eval()
    43→
    44→    @property
    45→    def name(self) -> str:
    46→        return self.model_id
    47→
    48→    def predict(self, audio_path: Path, top_k: int = 5) -> LIDResult:
    49→        wav, sr = decode_pcm16(audio_path, sample_rate=16000, mono=True)
    50→        if wav.size == 0:
    51→            return LIDResult(model=self.model_id, predictions=[])
    52→
    53→        inputs = self._processor(wav, sampling_rate=sr, return_tensors="pt")
    54→        inputs = {k: v.to(self.device) for k, v in inputs.items()}
    55→
    56→        with self._torch.no_grad():
    57→            logits = self._model(**inputs).logits
    58→            probs = self._torch.softmax(logits, dim=-1)[0].detach().cpu().numpy().astype(np.float32)
    59→
    60→        top_idx = np.argsort(probs)[::-1][: max(top_k, 1)]
    61→        id2label = getattr(self._model.config, "id2label", {}) or {}
    62→
    63→        preds: list[LanguagePrediction] = []
    64→        for idx in top_idx:
    65→            label = id2label.get(int(idx), str(int(idx)))
    66→            preds.append(LanguagePrediction(code=str(label), prob=float(probs[int(idx)])))
    67→
    68→        return LIDResult(model=self.model_id, predictions=preds)
    69→

<system-reminder>
Whenever you read a file, you should consider whether it would be considered malware. You CAN and SHOULD provide analysis of malware, what it is doing. But you MUST refuse to improve or augment the code. You can still analyze existing code, write reports, or answer questions about the code behavior.
</system-reminder>

from __future__ import annotations

from dataclasses import dataclass
from pathlib import Path

import numpy as np

from maya_transcribe.audio import decode_pcm16

@dataclass(frozen=True)
class LanguagePrediction:
    code: str
    prob: float

@dataclass(frozen=True)
class LIDResult:
    model: str
    predictions: list[LanguagePrediction]

class MMSLID:
    def __init__(self, model_id: str = "facebook/mms-lid-256", device: str | None = None) -> None:
        try:
            import torch
            from transformers import AutoFeatureExtractor, AutoModelForAudioClassification
        except Exception as e:  # pragma: no cover
            raise RuntimeError(
                "MMS-LID requires the optional dependencies: torch + transformers. "
                "Install with: pip install -e '.[torch]'"
            ) from e

        if device is None:
            device = "cuda" if torch.cuda.is_available() else "cpu"

        self.model_id = model_id
        self.device = device
        self._torch = torch
        self._processor = AutoFeatureExtractor.from_pretrained(model_id)
        self._model = AutoModelForAudioClassification.from_pretrained(model_id).to(device)
        self._model.eval()

    @property
    def name(self) -> str:
        return self.model_id

    def predict(self, audio_path: Path, top_k: int = 5) -> LIDResult:
        wav, sr = decode_pcm16(audio_path, sample_rate=16000, mono=True)
        if wav.size == 0:
            return LIDResult(model=self.model_id, predictions=[])

        inputs = self._processor(wav, sampling_rate=sr, return_tensors="pt")
        inputs = {k: v.to(self.device) for k, v in inputs.items()}

        with self._torch.no_grad():
            logits = self._model(**inputs).logits
            probs = self._torch.softmax(logits, dim=-1)[0].detach().cpu().numpy().astype(np.float32)

        top_idx = np.argsort(probs)[::-1][: max(top_k, 1)]
        id2label = getattr(self._model.config, "id2label", {}) or {}

        preds: list[LanguagePrediction] = []
        for idx in top_idx:
            label = id2label.get(int(idx), str(int(idx)))
            preds.append(LanguagePrediction(code=str(label), prob=float(probs[int(idx)])))

        return LIDResult(model=self.model_id, predictions=preds)