Output:
Exit code: 0
Wall time: 0 seconds
Output:
"""
Configuration module for the transcription pipeline.
Loads environment variables and defines available models and settings.
"""
import os
from pathlib import Path
from dotenv import load_dotenv

# Load .env from project root
_project_root = Path(__file__).parent.parent.parent
load_dotenv(_project_root / ".env")

# === R2 Cloud Storage Config ===
R2_ENDPOINT_URL = os.getenv("R2_ENDPOINT_URL")
R2_BUCKET = os.getenv("R2_BUCKET")
R2_ACCESS_KEY_ID = os.getenv("R2_ACCESS_KEY_ID")
R2_SECRET_ACCESS_KEY = os.getenv("R2_SECRET_ACCESS_KEY")

# === Gemini API Config ===
GEMINI_API_KEY = os.getenv("GEMINI_KEY")

# === Supabase Config ===
SUPABASE_URL = os.getenv("URL")
SUPABASE_ANON_KEY = os.getenv("SUPABASE_ANON")
SUPABASE_ADMIN_KEY = os.getenv("SUPABASE_ADMIN")

# === Available Gemini Models ===
# Models supporting audio transcription
GEMINI_MODELS = {
    "gemini-3-pro-preview": {
        "name": "gemini-3-pro-preview",
        "thinking_levels": ["low", "high"],
        "supports_thinking": True,
        "cost_tier": "premium"
    },
    "gemini-3-flash-preview": {
        "name": "gemini-3-flash-preview",
        "thinking_levels": ["minimal", "low", "medium", "high"],
        "supports_thinking": True,
        "cost_tier": "standard"
    },
    "gemini-2.5-pro": {
        "name": "gemini-2.5-pro",
        "thinking_levels": None,
        "supports_thinking": True,
        "cost_tier": "premium"
    },
    "gemini-2.5-flash": {
        "name": "gemini-2.5-flash",
        "thinking_levels": None,
        "supports_thinking": True,
        "cost_tier": "standard"
    },
    "gemini-2.5-flash-lite": {
        "name": "gemini-2.5-flash-lite",
        "thinking_levels": None,
        "supports_thinking": True,
        "cost_tier": "lite"
    },
    "gemini-2.0-flash": {
        "name": "gemini-2.0-flash",
        "thinking_levels": None,
        "supports_thinking": False,
        "cost_tier": "standard"
    }
}

# Alias mapping for easier usage
MODEL_ALIASES = {
    "gemini-flash-latest": "gemini-3-flash-preview",
    "gemini-flash-lite-latest": "gemini-2.5-flash-lite",
}

# === Default Pipeline Settings ===
DEFAULT_SETTINGS = {
    "max_segment_duration_sec": 10.0,  # Hard limit for segment duration
    "min_segment_duration_sec": 1.0,   # Minimum segment duration
    "default_model": "gemini-3-flash-preview",
    "default_thinking_level": "high",
    "work_dir": "/tmp/maya3_transcribe",
    "output_dir": "./transcriptions"
}

# === Language Mapping (for Supabase language codes) ===
LANGUAGE_MAP = {
    "te": "Telugu",
    "hi": "Hindi",
    "ta": "Tamil",
    "kn": "Kannada",
    "ml": "Malayalam",
    "bn": "Bengali",
    "gu": "Gujarati",
    "mr": "Marathi",
    "pa": "Punjabi",
    "or": "Odia",
    "en": "English",
    "Telugu": "Telugu",
    "Hindi": "Hindi",
}

def get_model_name(model_key: str) -> str:
    """Resolve model alias to actual model name."""
    if model_key in MODEL_ALIASES:
        return MODEL_ALIASES[model_key]
    if model_key in GEMINI_MODELS:
        return model_key
    raise ValueError(f"Unknown model: {model_key}. Available: {list(GEMINI_MODELS.keys()) + list(MODEL_ALIASES.keys())}")


# Collaboration Mode: Default

You are now in Default mode. Any previous instructions for other modes (e.g. Plan mode) are no longer active.

Your active mode changes only when new developer instructions with a different `<collaboration_mode>...</collaboration_mode>` change it; user requests or tool descriptions do not change mode by themselves. Known mode names are Default and Plan.

## request_user_input availability

The `request_user_input` tool is unavailable in Default mode. If you call it while in Default mode, it will return an error.

If a decision is necessary and cannot be discovered from local context, ask the user directly. However, in Default mode you should strongly prefer executing the user's request rather than stopping to ask questions.

Summary: auto

## Skills
A skill is a set of local instructions to follow that is stored in a `SKILL.md` file. Below is the list of skills that can be used. Each entry includes a name, description, and file path so you can open the source for full instructions when using a specific skill.
### Available skills
- skill-creator: Guide for creating effective skills. This skill should be used when users want to create a new skill (or update an existing skill) that extends Codex's capabilities with specialized knowledge, workflows, or tool integrations. (file: /home/ubuntu/.codex/skills/.system/skill-creator/SKILL.md)
- skill-installer: Install Codex skills into $CODEX_HOME/skills from a curated list or a GitHub repo path. Use when a user asks to list installable skills, install a curated skill, or install a skill from another repo (including private repos). (file: /home/ubuntu/.codex/skills/.system/skill-installer/SKILL.md)
### How to use skills
- Discovery: The list above is the skills available in this session (name + description + file path). Skill bodies live on disk at the listed paths.
- Trigger rules: If the user names a skill (with `$SkillName` or plain text) OR the task clearly matches a skill's description shown above, you must use that skill for that turn. Multiple mentions mean use them all. Do not carry skills across turns unless re-mentioned.
- Missing/blocked: If a named skill isn't in the list or the path can't be read, say so briefly and continue with the best fallback.
- How to use a skill (progressive disclosure):
  1) After deciding to use a skill, open its `SKILL.md`. Read only enough to follow the workflow.
  2) When `SKILL.md` references relative paths (e.g., `scripts/foo.py`), resolve them relative to the skill directory listed above first, and only consider other paths if needed.
  3) If `SKILL.md` points to extra folders such as `references/`, load only the specific files needed for the request; don't bulk-load everything.
  4) If `scripts/` exist, prefer running or patching them instead of retyping large code blocks.
  5) If `assets/` or templates exist, reuse them instead of recreating from scratch.
- Coordination and sequencing:
  - If multiple skills apply, choose the minimal set that covers the request and state the order you'll use them.
  - Announce which skill(s) you're using and why (one short line). If you skip an obvious skill, say why.
- Context hygiene:
  - Keep context small: summarize long sections instead of pasting them; only load extra files when needed.
  - Avoid deep reference-chasing: prefer opening only files directly linked from `SKILL.md` unless you're blocked.
  - When variants exist (frameworks, providers, domains), pick only the relevant reference file(s) and note that choice.
- Safety and fallback: If a skill can't be applied cleanly (missing files, unclear instructions), state the issue, pick the next-best approach, and continue.


**Reviewing README for validation details**


Output:
Exit code: 0
Wall time: 0 seconds
Output:
# Maya3 Transcription Pipeline

Audio transcription pipeline for Indian languages using Google Gemini AI.

## Production Configuration

```
Model: gemini-3-flash-preview
Temperature: 1.0 (Gemini 3 default - below 1.0 causes looping)
Thinking: low (fast, prevents loops)
Validation: Native script + CTC alignment
```

## Quick Start

```bash
# Setup
python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt

# Run pipeline
python pipeline.py VIDEO_ID --language Telugu --max-segments 10
```

## Pipeline Usage

```bash
# Basic usage
python pipeline.py pF_BQpHaIdU

# With options
python pipeline.py pF_BQpHaIdU \
    --language Telugu \
    --model gemini-3-flash-preview \
    --thinking low \
    --temperature 0.0 \
    --max-segments 20 \
    --output ./transcriptions

# Skip validation (faster but no quality check)
python pipeline.py pF_BQpHaIdU --no-validate
```

## Python API

```python
from pipeline import run_pipeline

# Run full pipeline
result = run_pipeline(
    video_id="pF_BQpHaIdU",
    language="Telugu",
    model="gemini-3-flash-preview",
    thinking_level="low",
    temperature=0.0,
    max_segments=20,
    validate=True
)

print(f"Processed: {result.segments_processed} segments")
print(f"Output: {result.output_file}")
```

## Validation

Transcriptions are validated for:

1. **Character Check** (instant)
   - Only valid native script characters
   - Numbers and punctuation allowed
   - No English/foreign characters

2. **Audio Match** (0.1s per segment)
   - CTC-based alignment score
   - Detects if text doesn't match audio

```python
from src.validators import validate_transcription, quick_validate

# Quick check (character only)
result = quick_validate("నాకు కొన్ని గుర్తుంటాయి", language="te")
print(result["valid"])  # True

# Full validation (with audio)
result = validate_transcription("audio.flac", "నాకు కొన్ని", language="te")
print(result.status)  # accept / review / reject
```

## Output Format

Each transcription includes 3 text formats + 1 tagged format + speaker metadata:

```json
{
  "segment_id": "SPEAKER_00_0000_0.03-2.61.flac",
  "transcription": {
    "transcription": "నాకు కొన్ని యాడ్స్ గుర్తుంటాయి.",
    "code_switch": "నాకు కొన్ని ads గుర్తుంటాయి.",
    "romanized": "naku konni ads gurtuntayi.",
    "tagged": "నాకు కొన్ని ads గుర్తుంటాయి [laugh] like example కచ్చా mango bite.",
    "speaker": {
      "emotion": "happy",
      "emotion_intensity": "mild",
      "speaking_style": "conversational",
      "pace": "normal",
      "energy": "medium",
      "accent": "Telangana",
      "audio_quality": "clean",
      "background": "silence"
    },
    "confidence": 0.95
  },
  "validation_status": "accept",
  "validation_score": 0.798
}
```

## Project Structure

```
maya3_transcribe/
├── pipeline.py              # Main pipeline orchestrator
├── src/
│   ├── backend/             # Core modules
│   │   ├── config.py
│   │   ├── r2_storage.py
│   │   ├── supabase_client.py
│   │   ├── audio_processor.py
│   │   ├── audio_polisher.py     # NEW: VAD boundary cleanup
│   │   ├── gemini_transcriber.py
│   │   └── transcription_schema.py
│   └── validators/          # Validation module
│       ├── __init__.py
│       ├── simple_validator.py   # Main validator
│       └── ctc_forced_aligner.py # CTC alignment
├── models/                  # Downloaded models (gitignored)
├── analysis_results/        # Analysis output (gitignored)
├── transcriptions/          # Pipeline output (gitignored)
└── docs/                    # Documentation
```

## Supported Languages

| Language | Code | Script |
|----------|------|--------|
| Telugu | te | Telugu |
| Hindi | hi | Devanagari |
| Tamil | ta | Tamil |
| Kannada | kn | Kannada |
| Malayalam | ml | Malayalam |
| Bengali | bn | Bengali |
| Marathi | mr | Devanagari |
| Gujarati | gu | Gujarati |
| Punjabi | pa | Gurmukhi |
| Odia | or | Odia |

## Audio Polisher (Pre-processing)

Lightweight signal-processing pass that cleans VAD-generated segment boundaries
before transcription. No ML models, runs in ~30ms/segment.

**What it does:**
- Detects and trims leading cut artifacts (partial phonemes from imprecise VAD)
- Detects trailing speech bursts (leaked from next segment)
- Preserves silence at boundaries (silence = good padding, never trimmed)
- Measures SNR, conditionally normalizes volume for quiet segments
- Integrated into pipeline as Step 3.5 (automatic, `polish_audio=True` by default)

**Conservative by design** - only trims cut speech artifacts, never silence.
Already-clean segments pass through untouched (~76% skip rate on test data).

```python
from src.backend.audio_polisher import AudioPolisher

polisher = AudioPolisher()
result = polisher.polish("segment.flac", output_dir="./polished/")
print(result.summary())
# POLISHED | 2582ms -> 2557ms | start -25ms (artifact_trimmed)
```

## Configuration

Environment variables (`.env`):

```
GEMINI_API_KEY=your_key
R2_ACCESS_KEY_ID=your_key
R2_SECRET_ACCESS_KEY=your_secret
R2_ENDPOINT_URL=your_endpoint
R2_BUCKET_NAME=your_bucket
SUPABASE_URL=your_url
SUPABASE_KEY=your_key
```


Summary: auto


**Inspecting scoring and alignment scripts**


Output:
Exit code: 0
Wall time: 0 seconds
Output:
total 132
drwxrwxr-x  2 ubuntu ubuntu  4096 Feb  7 03:46 .
drwxrwxr-x 12 ubuntu ubuntu  4096 Feb  7 04:03 ..
-rw-rw-r--  1 ubuntu ubuntu 58301 Feb  7 03:26 comparison.json
-rw-rw-r--  1 ubuntu ubuntu  3161 Feb  7 03:48 roman_alignment.json
-rw-rw-r--  1 ubuntu ubuntu 17942 Feb  7 03:24 run_1.json
-rw-rw-r--  1 ubuntu ubuntu 17941 Feb  7 03:25 run_2.json
-rw-rw-r--  1 ubuntu ubuntu 17940 Feb  7 03:26 run_3.json


**Planning JSON stats extraction**


I’m now quantifying this with actual stats (status distribution and score behavior), then I’ll give a hard recommendation for gating logic (`and/or/avg/weighted`) and whether this is enough for large-scale TTS/ASR.


jq '[to_entries[] | .value.validation.status] | group_by(.) | map({status: .[0], count: length})' consistency_test/run_1.json


jq '[to_entries[] | .value.validation.alignment_score] | {mean:(add/length), min:min, max:max, p50:(sort|.[length/2]), below_0_7: map(select(.<0.7))|length, below_0_5: map(select(.<0.5))|length, total:length}' consistency_test/run_1.json


jq '{mean_native_ctc: ([to_entries[]|.value.native_ctc_score]|add/length), mean_mms: ([to_entries[]|.value.mms_score]|add/length), min_native: ([to_entries[]|.value.native_ctc_score]|min), min_mms: ([to_entries[]|.value.mms_score]|min), disagreed_70: ([to_entries[] | select((.value.native_ctc_score>=0.7 and .value.mms_score<0.7) or (.value.native_ctc_score<0.7 and .value.mms_score>=0.7))]|length), both_below_70: ([to_entries[] | select(.value.native_ctc_score<0.7 and .value.mms_score<0.7)]|length), either_below_70: ([to_entries[] | select(.value.native_ctc_score<0.7 or .value.mms_score<0.7)]|length), total:(length)}' consistency_test/roman_alignment.json


Output:
Exit code: 0
Wall time: 0 seconds
Output:
total 72
-rw-rw-r-- 1 ubuntu ubuntu 32359 Feb  6 04:47 test_new_schema_20seg.json
-rw-rw-r-- 1 ubuntu ubuntu  7887 Feb  6 04:42 test_new_schema_5segments.json
-rw-rw-r-- 1 ubuntu ubuntu  3676 Feb  3 23:02 pF_BQpHaIdU_gemini_3_flash_preview_20260203_230246.json
-rw-rw-r-- 1 ubuntu ubuntu  3676 Feb  3 23:02 pF_BQpHaIdU_gemini_3_flash_preview_20260203_230246_partial.json
-rw-rw-r-- 1 ubuntu ubuntu  3582 Feb  3 22:53 pF_BQpHaIdU_gemini_2.5_flash_20260203_225339.json
-rw-rw-r-- 1 ubuntu ubuntu  3582 Feb  3 22:53 pF_BQpHaIdU_gemini_2.5_flash_20260203_225339_partial.json
-rw-rw-r-- 1 ubuntu ubuntu  6969 Feb  3 22:52 pF_BQpHaIdU_gemini_3_flash_preview_20260203_225233.json
-rw-rw-r-- 1 ubuntu ubuntu  6969 Feb  3 22:52 pF_BQpHaIdU_gemini_3_flash_preview_20260203_225233_partial.json


Output:
Exit code: 0
Wall time: 0 seconds
Output:
{
  "total": 10,
  "identical": 10,
  "minor_diff": 0,
  "major_diff": 0
}


Output:
Exit code: 0
Wall time: 0 seconds
Output:
[
  {
    "status": "accept",
    "count": 5
  },
  {
    "status": "reject",
    "count": 1
  },
  {
    "status": "review",
    "count": 4
  }
]


Output:
Exit code: 0
Wall time: 0 seconds
Output:
{
  "mean": 0.7170599999999999,
  "min": 0.4755,
  "max": 0.8389,
  "p50": 0.7218,
  "below_0_7": 2,
  "below_0_5": 1,
  "total": 10
}


Output:
Exit code: 0
Wall time: 0 seconds
Output:
{
  "mean_native_ctc": 0.7170599999999999,
  "mean_mms": 0.7979999999999999,
  "min_native": 0.4755,
  "min_mms": 0.6177,
  "disagreed_70": 3,
  "both_below_70": 0,
  "either_below_70": 3,
  "total": 10
}


**Analyzing larger test schema sample**


jq '.results_count, .total_duration_sec, (.results | length)' transcriptions/test_new_schema_20seg.json


jq '[.results[] | .validation_status] | group_by(.) | map({status: .[0], count:length})' transcriptions/test_new_schema_20seg.json


jq '[.results[] | .validation_score] | {mean:(add/length), min:min, max:max, p50:(sort|.[length/2]), below_0_7:(map(select(.<0.7))|length), below_0_5:(map(select(.<0.5))|length), total:length}' transcriptions/test_new_schema_20seg.json


jq '.results[] | select(.validation_status=="reject" or .validation_status=="review") | {segment_id, duration_sec, validation_status, validation_score, text:(.transcription.transcription)}' transcriptions/test_new_schema_20seg.json


Output:
Exit code: 0
Wall time: 0 seconds
Output:
22
null
22


Output:
Exit code: 0
Wall time: 0 seconds
Output:
[
  {
    "status": null,
    "count": 22
  }
]


Output:
Exit code: 5
Wall time: 0 seconds
Output:
jq: error (at transcriptions/test_new_schema_20seg.json:535): null (null) and number (22) cannot be divided


Output:
Exit code: 0
Wall time: 0 seconds
Output:


**Assessing data presence**


sed -n '1,200p' transcriptions/test_new_schema_20seg.json


Output:
Exit code: 0
Wall time: 0 seconds
Output:
{
  "video_id": "pF_BQpHaIdU",
  "model": "gemini-3-flash-preview",
  "language": "Telugu",
  "results_count": 22,
  "results": [
    {
      "segment_id": "SPEAKER_00_0000_0.03-2.61.flac",
      "chunk_index": 0,
      "total_chunks": 1,
      "duration_sec": 2.582,
      "language": "Telugu",
      "transcription": {
        "transcription": "సున్నా సున్నా. నాకు కొన్ని యాడ్స్ గుర్తుండేవి లైక్ ఎగ్జామ్పుల్ కచ్చా మ్యాంగో బైట్.",
        "code_switch": "సున్నా సున్నా. నాకు కొన్ని ads గుర్తుండేవి like example కచ్చా మ్యాంగో బైట్.",
        "romanized": "sunna sunna. naaku konni ads gurtundevi like example kaccha mango bite.",
        "tagged": "సున్నా సున్నా. [noise] నాకు కొన్ని ads గుర్తుండేవి like example కచ్చా మ్యాంగో బైట్.",
        "speaker": {
          "emotion": "neutral",
          "speaking_style": "conversational",
          "pace": "normal",
          "accent": "Standard Telugu"
        }
      },
      "model_used": "gemini-3-flash-preview",
      "thinking_level": "low",
      "processing_time_sec": 2.8057973384857178,
      "validation_status": null,
      "validation_score": null
    },
    {
      "segment_id": "SPEAKER_00_0001_3.05-13.40.flac",
      "chunk_index": 0,
      "total_chunks": 2,
      "duration_sec": 10.0,
      "language": "Telugu",
      "transcription": {
        "transcription": "మా అచ్చమ్మ మ్యాంగో వెరైటీ అడిగితే మా వాడు ఒకడు ఉన్నాడు ఐ విల్ నాట్ నేమ్ హిమ్ బట్ వాడు ఏం చేశాడంటే మ్యాంగోస్ తీసుకొని జిరాక్స్ షాప్ కి వెళ్ళిపోయాడన్నమాట. ఇన్ ఫ్యాక్ట్ నాకు ఫోటోకాపీ అంటే వేరు జిరాక్స్ అంటే వేరు అని తెలిసింది అక్కడే.",
        "code_switch": "మా అచ్చమ్మ mango variety అడిగితే మా వాడు ఒకడు ఉన్నాడు I will not name him but వాడు ఏం చేశాడంటే mangoes తీసుకొని xerox shop కి వెళ్ళిపోయాడన్నమాట. In fact నాకు photocopy అంటే వేరు xerox అంటే వేరు అని తెలిసింది అక్కడే.",
        "romanized": "maa achamma mango variety adigite maa vaadu okadu unnadu I will not name him but vaadu em chesaadante mangoes teesukoni xerox shop ki vellipoyaadanmaata. In fact naaku photocopy ante veru xerox ante veru ani telisindi akkade.",
        "tagged": "మా అచ్చమ్మ mango variety అడిగితే మా వాడు ఒకడు ఉన్నాడు I will not name him but [laugh] వాడు ఏం చేశాడంటే mangoes తీసుకొని xerox shop కి వెళ్ళిపోయాడన్నమాట. In fact నాకు photocopy అంటే వేరు xerox అంటే వేరు అని తెలిసింది అక్కడే.",
        "speaker": {
          "emotion": "happy",
          "speaking_style": "narrative",
          "pace": "normal",
          "accent": "Coastal Andhra"
        }
      },
      "model_used": "gemini-3-flash-preview",
      "thinking_level": "low",
      "processing_time_sec": 3.683773994445801,
      "validation_status": null,
      "validation_score": null
    },
    {
      "segment_id": "SPEAKER_00_0002_16.08-28.31.flac",
      "chunk_index": 0,
      "total_chunks": 2,
      "duration_sec": 10.0,
      "language": "Telugu",
      "transcription": {
        "transcription": "అంటే ద ప్రాసెస్ అని చెప్పి వాడు రెండు మ్యాంగోస్ తీసుకెళ్ళిపోయి అంటే క్యూరియాసిటీ అవును అవును తెలియకపోవడం కాదు అక్కడికెళ్ళి అరే ఇది ఇస్తే నిజంగా ఏమంటాడో చూద్దాము అంతా కనెక్ట్ అయిపోయారంటే ఇట్స్ లైక్",
        "code_switch": "అంటే the process అని చెప్పి వాడు రెండు mangoes తీసుకెళ్ళిపోయి అంటే curiosity అవును అవును తెలియకపోవడం కాదు అక్కడికెళ్ళి అరే ఇది ఇస్తే నిజంగా ఏమంటాడో చూద్దాము అంతా కనెక్ట్ అయిపోయారంటే it's like",
        "romanized": "ante the process ani cheppi vaadu rendu mangoes theeskellipoyi ante curiosity avunu avunu theliyakapovadam kaadu akkadikelli arey idhi isthe nijamga emantado chuddamu antha connect ayipoyarante it's like",
        "tagged": "అంటే the process అని చెప్పి వాడు రెండు mangoes తీసుకెళ్ళిపోయి అంటే curiosity అవును అవును తెలియకపోవడం కాదు అక్కడికెళ్ళి అరే ఇది ఇస్తే నిజంగా ఏమంటాడో చూద్దాము అంతా కనెక్ట్ అయిపోయారంటే it's like",
        "speaker": {
          "emotion": "excited",
          "speaking_style": "conversational",
          "pace": "normal",
          "accent": ""
        }
      },
      "model_used": "gemini-3-flash-preview",
      "thinking_level": "low",
      "processing_time_sec": 3.096914052963257,
      "validation_status": null,
      "validation_score": null
    },
    {
      "segment_id": "SPEAKER_00_0002_16.08-28.31.flac",
      "chunk_index": 1,
      "total_chunks": 2,
      "duration_sec": 2.237,
      "language": "Telugu",
      "transcription": {
        "transcription": "అప్పా, ఎవరైతే చూసుంటారో వాళ్ళకి అర్థమై ఉంటుంది.",
        "code_switch": "అప్పా, ఎవరైతే చూసుంటారో వాళ్ళకి అర్థమై ఉంటుంది.",
        "romanized": "appa, evaraite choosuntaro vallaki arthamai untundi.",
        "tagged": "అప్పా, ఎవరైతే చూసుంటారో వాళ్ళకి అర్థమై ఉంటుంది.",
        "speaker": {
          "emotion": "neutral",
          "speaking_style": "conversational",
          "pace": "normal",
          "accent": "Standard Telugu"
        }
      },
      "model_used": "gemini-3-flash-preview",
      "thinking_level": "low",
      "processing_time_sec": 2.7945642471313477,
      "validation_status": null,
      "validation_score": null
    },
    {
      "segment_id": "SPEAKER_00_0003_29.78-34.93.flac",
      "chunk_index": 0,
      "total_chunks": 1,
      "duration_sec": 5.149,
      "language": "Telugu",
      "transcription": {
        "transcription": "అసలు ఆ ఇన్ ఫాక్ట్ లైక్ ఐ గాట్ టు నో దట్ యు వర్ బిహైండ్ ది కచ్చా మ్యాంగోస్.",
        "code_switch": "అసలు ఆ in fact like I got to know that you were behind the Kachcha Mangoes.",
        "romanized": "asalu aa in fact like I got to know that you were behind the Kachcha Mangoes.",
        "tagged": "అసలు ఆ in fact like I got to know that you were behind the Kachcha Mangoes.",
        "speaker": {
          "emotion": "excited",
          "speaking_style": "conversational",
          "pace": "normal",
          "accent": ""
        }
      },
      "model_used": "gemini-3-flash-preview",
      "thinking_level": "low",
      "processing_time_sec": 2.646737813949585,
      "validation_status": null,
      "validation_score": null
    },
    {
      "segment_id": "SPEAKER_00_0004_36.29-37.41.flac",
      "chunk_index": 0,
      "total_chunks": 1,
      "duration_sec": 1.115,
      "language": "Telugu",
      "transcription": {
        "transcription": "టాక్ అబౌట్ దట్ స్టోరీ యాస్ వెల్ యా యా.",
        "code_switch": "talk about that story as well yeah yeah.",
        "romanized": "talk about that story as well yeah yeah.",
        "tagged": "talk about that story as well yeah yeah.",
        "speaker": {
          "emotion": "neutral",
          "speaking_style": "conversational",
          "pace": "normal",
          "accent": "Telugu accented English"
        }
      },
      "model_used": "gemini-3-flash-preview",
      "thinking_level": "low",
      "processing_time_sec": 2.635169744491577,
      "validation_status": null,
      "validation_score": null
    },
    {
      "segment_id": "SPEAKER_00_0026_383.26-390.66.flac",
      "chunk_index": 0,
      "total_chunks": 1,
      "duration_sec": 7.409,
      "language": "Telugu",
      "transcription": {
        "transcription": "కనెక్షన్ ఉంటుందన్నమాట. ఆ అంటే దీని గురించి చాలా కనెక్టెడ్ క్వశ్చన్స్ ఉన్నాయి నాకు, దీనికంటే ముందు ఇందాక పార్లేజీ గురించి తీసుకొచ్చారు కాబట్టి అవును. ఇప్పటి వరకు పాడ్ కాస్ట్ చూశారు కదా!",
        "code_switch": "connection ఉంటుందన్నమాట. ఆ అంటే దీని గురించి చాలా connected questions ఉన్నాయి నాకు, దీనికంటే ముందు ఇందాక Parle-G గురించి తీసుకొచ్చారు కాబట్టి అవును. ఇప్పటి వరకు podcast చూశారు కదా!",
        "romanized": "connection untundannamaata. aa ante deeni gurinchi chaala connected questions unnaayi naaku, deenikante mundu indaaka Parle-G gurinchi teeskochaaru kaabatti avunu. ippati varaku podcast chooshaaru kadaa!",
        "tagged": "connection ఉంటుందన్నమాట. [laugh] ఆ అంటే దీని గురించి చాలా connected questions ఉన్నాయి నాకు, దీనికంటే ముందు ఇందాక Parle-G గురించి తీసుకొచ్చారు కాబట్టి అవును. ఇప్పటి వరకు podcast చూశారు కదా!",
        "speaker": {
          "emotion": "happy",
          "speaking_style": "conversational",
          "pace": "normal",
          "accent": "Standard Telugu"
        }
      },
      "model_used": "gemini-3-flash-preview",
      "thinking_level": "low",
      "processing_time_sec": 4.024925947189331,
      "validation_status": null,
      "validation_score": null
    },
    {
      "segment_id": "SPEAKER_00_0027_394.58-397.79.flac",
      "chunk_index": 0,
      "total_chunks": 1,
      "duration_sec": 3.206,
      "language": "Telugu",
      "transcription": {
        "transcription": "పాడ్కాస్ట్ చూస్తున్న పది మందిలో ఐదుగురికి మార్కెటర్ అవ్వాలన్న థాట్ వస్తది.",
        "code_switch": "Podcast చూస్తున్న పది మందిలో ఐదుగురికి marketer అవ్వాలన్న thought వస్తది.",
        "romanized": "Podcast chustunna padhi mandhilo aidhuguriki marketer avvalanna thought vasthadhi.",
        "tagged": "Podcast చూస్తున్న పది మందిలో ఐదుగురికి marketer అవ్వాలన్న thought వస్తది.",
        "speaker": {
          "emotion": "neutral",
          "speaking_style": "conversational",
          "pace": "normal",
          "accent": "Standard Telugu"
        }
      },
      "model_used": "gemini-3-flash-preview",
      "thinking_level": "low",
      "processing_time_sec": 3.1701323986053467,
      "validation_status": null,
      "validation_score": null
    },
    {
      "segment_id": "SPEAKER_00_0028_420.53-426.76.flac",


**Analyzing differing alignment score distributions**


jq '.metadata, .summary' analysis_results/scoring_results.json


jq '[.results[].scoring.alignment_score] | {mean:(add/length), min:min, max:max, p50:(sort|.[length/2]), below_0_7:(map(select(.<0.7))|length), below_0_5:(map(select(.<0.5))|length), total:length}' analysis_results/scoring_results.json


jq '[.results[] | {id:.segment_id,score:.scoring.alignment_score,text:.transcription}] | sort_by(.score) | .[:8]' analysis_results/scoring_results.json


Output:
Exit code: 0
Wall time: 0 seconds
Output:
{
  "model": "gemini-3-flash-preview",
  "config": "temp0_low",
  "language": "Telugu",
  "segments_scored": 20,
  "timestamp": "2026-02-05T01:05:40.295126"
}
{
  "mean_alignment_score": 0.91371,
  "mean_confidence": 0.91679
}


Output:
Exit code: 0
Wall time: 0 seconds
Output:
{
  "mean": 0.91371,
  "min": 0.8449,
  "max": 0.9431,
  "p50": 0.9211,
  "below_0_7": 0,
  "below_0_5": 0,
  "total": 20
}


Output:
Exit code: 0
Wall time: 0 seconds
Output:
[
  {
    "id": "SPEAKER_00_0000_0.03-2.61.flac",
    "score": 0.8449,
    "text": "నాకు కొన్ని యాడ్స్ గుర్తుండేవి లైక్ ఎగ్జాంపుల్ కచ్చా మ్యాంగో బైట్"
  },
  {
    "id": "SPEAKER_01_0389_5084.82-5089.52.flac",
    "score": 0.8863,
    "text": "ఒక ఎగ్జిస్టింగ్ బ్రాండ్ ఇప్పుడు వంద కోట్ల బ్రాండ్ ఉందండి దానికి నాకు ప్రాఫిట్ ముప్పై కోట్లు ఉంది అనుకుందాం"
  },
  {
    "id": "SPEAKER_00_0350_4507.00-4520.45.flac",
    "score": 0.894,
    "text": "అబ్రాడ్ కంట్రీస్ నుంచి చూస్తే పాపులేషన్ చూసినా గాని లేకపోతే రిలీజియన్ గాని లేకపోతే వాళ్ళ వాళ్ళ ఓన్ కల్చర్స్ గాని ఇవన్నీ చూస్తే ట్వంటీ నైన్ కంట్రీస్ మధ్యలో ఉన్నాం మన ఇండియా అనే ఒక కంట్రీలో"
  },
  {
    "id": "SPEAKER_00_0107_1343.96-1350.94.flac",
    "score": 0.8949,
    "text": "ఇప్పుడు మీరు పాపిన్స్ గురించి ఇంకా కిస్మి గురించి చెప్తున్నప్పుడు రెండిటి గురించి రకరకాల స్టోరీస్ ఉన్నాయి అవును నాకు స్కూల్లో ఉన్నప్పుడు ఐ గాట్ ఏ డేర్ ట్రూత్ ఆర్ డేర్ లో"
  },
  {
    "id": "SPEAKER_01_0422_5442.68-5452.49.flac",
    "score": 0.8984,
    "text": "లోపలిచ్చాక వాళ్ళు మళ్ళీ అంటే ఫోర్ రూపీస్ ఇచ్చి ఉంచేస్తున్నారు ఫోర్ ఫిఫ్టీ మీరు ప్రైస్ పెట్టేసారు పార్లేజీ సో చాలా బ్యాక్ లాష్ వచ్చింది వెంటనే రోల్ బ్యాక్ చేసుకున్నారు"
  },
  {
    "id": "SPEAKER_01_0034_481.64-493.94.flac",
    "score": 0.9046,
    "text": "ఎందుకంటే చిన్నప్పుడు అమ్మ అంటే మేము వేరే వేరే నాన్నగారు రైల్వేస్ లో ఉండేవారు సో వేరే వేరే చోట్ల ఉంటే కాలనీస్ లో అంటే ఇంకో ఫుడ్ కార్పొరేషన్ ఆఫ్ ఇండియా అక్కడ కూడా"
  },
  {
    "id": "SPEAKER_00_0176_2259.17-2263.23.flac",
    "score": 0.9059,
    "text": "ఆప్షన్స్ ఉన్నప్పుడు ఇంత కట్ త్రోట్ సేమ్ ఫార్ములా సేమ్ టేస్ట్ ఎట్లా తీసుకొస్తున్నారు"
  },
  {
    "id": "SPEAKER_01_0248_3220.40-3227.84.flac",
    "score": 0.9121,
    "text": "ఈవెన్ ఆంధ్ర తమిళనాడు సో అందుకే మారి ఏంటంటే ఒక మారి కన్నా ముందు యారో రూట్ ఒక బిస్కెట్ ఒక రకమైన"
  }
]


Summary: auto


**Assessing alignment scoring discrepancies**


rg -n "mms_score|roman_alignment|MMS|roman|MFA|indicmfa|retry|validation_action" src pipeline.py -g '!*.pyc'


Output:
Exit code: 0
Wall time: 0 seconds
Output:
pipeline.py:59:    validation_action: str = "flag"         # "flag" (mark status) or "retry" (re-transcribe)
src/validators/ctc_forced_aligner.py:6:This is equivalent to MFA but using neural CTC instead of GMM-HMM.
src/validators/ctc_forced_aligner.py:101:    This provides MFA-equivalent functionality using neural CTC models.
src/backend/gemini_transcriber.py:196:                romanized=f"[ERROR: {error_msg}]",
src/backend/gemini_transcriber.py:207:                romanized=raw_result.get('romanized', ''),
src/backend/transcription_schema.py:7:  3. romanized       - Full Latin transliteration (derived from 1)
src/backend/transcription_schema.py:129:    romanized: str = Field(
src/backend/transcription_schema.py:244:"transcription" is the PRIMARY output. code_switch and romanized must contain the same words in the same order, differing only in script. "tagged" is code_switch with audio event markers inserted at their positions. Do not re-interpret the audio independently for each field.
src/backend/transcription_schema.py:255:3. romanized (derived from transcription - Latin script)
src/backend/transcription_schema.py:295:        "romanized": {
src/backend/transcription_schema.py:334:    "required": ["transcription", "code_switch", "romanized", "tagged", "speaker"],


**Checking validation action usage**


rg -n "validation_action" -n pipeline.py src -g '!*.pyc'


Output:
Exit code: 0
Wall time: 0 seconds
Output:
pipeline.py:59:    validation_action: str = "flag"         # "flag" (mark status) or "retry" (re-transcribe)


**Analyzing transcription validation gaps**


I’ve got enough evidence to answer directly. I’ll give you a blunt rating, whether it’s enough for large TTS/ASR, the missing piece, and a concrete scoring rule recommendation for `native CTC + roman MFA + retry`.


for f in transcriptions/*.json; do echo "--- $f"; jq '{results_count, non_null_validation: ([.results[]?.validation_score | select(.!=null)]|length), statuses: ([.results[]?.validation_status]|group_by(.)|map({s:.[0],c:length}))}' "$f" 2>/dev/null; done


nl -ba src/validators/simple_validator.py | sed -n '190,290p'


nl -ba consistency_test/roman_alignment.json | sed -n '1,140p'


Output:
Exit code: 0
Wall time: 0.2 seconds
Output:
--- transcriptions/pF_BQpHaIdU_gemini_2.5_flash_20260203_225339.json
{
  "results_count": 2,
  "non_null_validation": 0,
  "statuses": [
    {
      "s": null,
      "c": 2
    }
  ]
}
--- transcriptions/pF_BQpHaIdU_gemini_2.5_flash_20260203_225339_partial.json
{
  "results_count": 2,
  "non_null_validation": 0,
  "statuses": [
    {
      "s": null,
      "c": 2
    }
  ]
}
--- transcriptions/pF_BQpHaIdU_gemini_3_flash_preview_20260203_225233.json
{
  "results_count": 4,
  "non_null_validation": 0,
  "statuses": [
    {
      "s": null,
      "c": 4
    }
  ]
}
--- transcriptions/pF_BQpHaIdU_gemini_3_flash_preview_20260203_225233_partial.json
{
  "results_count": 4,
  "non_null_validation": 0,
  "statuses": [
    {
      "s": null,
      "c": 4
    }
  ]
}
--- transcriptions/pF_BQpHaIdU_gemini_3_flash_preview_20260203_230246.json
{
  "results_count": 2,
  "non_null_validation": 0,
  "statuses": [
    {
      "s": null,
      "c": 2
    }
  ]
}
--- transcriptions/pF_BQpHaIdU_gemini_3_flash_preview_20260203_230246_partial.json
{
  "results_count": 2,
  "non_null_validation": 0,
  "statuses": [
    {
      "s": null,
      "c": 2
    }
  ]
}
--- transcriptions/test_new_schema_20seg.json
{
  "results_count": 22,
  "non_null_validation": 0,
  "statuses": [
    {
      "s": null,
      "c": 22
    }
  ]
}
--- transcriptions/test_new_schema_5segments.json
{
  "results_count": 5,
  "non_null_validation": 0,
  "statuses": [
    {
      "s": null,
      "c": 5
    }
  ]
}


Output:
Exit code: 0
Wall time: 0 seconds
Output:
    50	    skip_short_segments: bool = True        # Skip segments < min_duration
    51
    52	    # === Transcription Settings ===
    53	    model: str = "gemini-3-flash-preview"   # Gemini model to use
    54	    thinking_level: Optional[str] = "low"   # minimal, low, medium, high (Gemini 3)
    55	    temperature: float = 0.0                # 0 for deterministic (tested safe with strict prompt)
    56
    57	    # === Validation Settings ===
    58	    validate_transcriptions: bool = True    # Validate native transcriptions
    59	    validation_action: str = "flag"         # "flag" (mark status) or "retry" (re-transcribe)
    60
    61	    # === Output Settings ===
    62	    output_dir: str = "./transcriptions"
    63	    save_intermediate: bool = True          # Save results after each batch
    64	    batch_size: int = 10                    # How many to process before saving
    65
    66	    # === Audio Polishing Settings ===
    67	    polish_audio: bool = True               # Polish segment boundaries before transcription
    68
    69	    # === Storage Settings ===
    70	    work_dir: str = field(default_factory=lambda: DEFAULT_SETTINGS["work_dir"])
    71	    cleanup_after: bool = False             # Remove downloaded files after processing
    72	    use_regex_fallback: bool = True         # Try regex if exact R2 match fails
    73
    74
    75	@dataclass
    76	class PipelineResult:
    77	    """Results from a pipeline run."""
    78	    video_id: str
    79	    language: str
    80	    model: str
    81	    thinking_level: Optional[str]
    82
    83	    total_segments_found: int
    84	    segments_processed: int
    85	    chunks_created: int
    86	    transcriptions_completed: int
    87
    88	    total_audio_duration_sec: float
    89	    total_processing_time_sec: float
    90
    91	    output_file: str
    92	    errors: List[str]
    93
    94	    started_at: str
    95	    completed_at: str
    96
    97
    98	class TranscriptionPipeline:
    99	    """
   100	    Main pipeline orchestrating the full transcription workflow.
   101	    """
   102
   103	    def __init__(self, config: PipelineConfig):
   104	        """Initialize pipeline with configuration."""
   105	        self.config = config
   106	        self.results: List[TranscriptionResult] = []
   107	        self.errors: List[str] = []
   108	        self.start_time: Optional[float] = None
   109
   110	    def _log(self, message: str, level: str = "INFO"):
   111	        """Log a message with timestamp."""
   112	        timestamp = datetime.now().strftime("%H:%M:%S")
   113	        print(f"[{timestamp}] [{level}] {message}")
   114
   115	    def _get_language_code(self, language: str) -> str:
   116	        """Convert language name to code for validation."""
   117	        lang_map = {
   118	            "Telugu": "te", "Hindi": "hi", "Tamil": "ta",
   119	            "Kannada": "kn", "Malayalam": "ml", "Bengali": "bn",
   120	            "Marathi": "mr", "Gujarati": "gu", "Punjabi": "pa",
   121	            "Odia": "or",
   122	        }
   123	        return lang_map.get(language, "te")
   124
   125	    def run(self) -> PipelineResult:
   126	        """
   127	        Execute the full pipeline.
   128
   129	        Returns:
   130	            PipelineResult with summary and statistics
   131	        """
   132	        self.start_time = time.time()
   133	        started_at = datetime.now().isoformat()
   134
   135	        self._log(f"Starting pipeline for video: {self.config.video_id}")
   136	        self._log(f"Model: {self.config.model}, Thinking: {self.config.thinking_level}")
   137
   138	        # === Step 1: Download from R2 ===
   139	        self._log("Step 1: Downloading audio segments from R2...")
   140	        try:


Output:
Exit code: 0
Wall time: 0 seconds
Output:
   210	            thinking_level=self.config.thinking_level,
   211	            temperature=self.config.temperature,
   212	            language=language
   213	        )
   214
   215	        # Process in batches
   216	        for batch_start in range(0, len(chunks), self.config.batch_size):
   217	            batch_end = min(batch_start + self.config.batch_size, len(chunks))
   218	            batch = chunks[batch_start:batch_end]
   219
   220	            self._log(f"Processing batch {batch_start + 1}-{batch_end} of {len(chunks)}")
   221
   222	            batch_results = transcriber.transcribe_batch(
   223	                batch,
   224	                transcription_config,
   225	                progress_callback=lambda c, t: self._log(
   226	                    f"  Chunk {batch_start + c}/{len(chunks)}: "
   227	                    f"{batch[c-1].original_segment}"
   228	                )
   229	            )
   230
   231	            # === Validate transcriptions ===
   232	            if self.config.validate_transcriptions:
   233	                lang_code = self._get_language_code(language)
   234	                for i, result in enumerate(batch_results):
   235	                    if result.native:
   236	                        validation = validate_transcription(
   237	                            batch[i].file_path,
   238	                            result.native,
   239	                            language=lang_code
   240	                        )
   241	                        result.validation_status = validation.status
   242	                        result.validation_score = validation.alignment_score
   243	                        if validation.status == "reject":
   244	                            self._log(f"  ⚠ Validation failed for {result.segment_id}: "
   245	                                     f"{validation.reasons}", "WARN")
   246
   247	            self.results.extend(batch_results)
   248
   249	            # Save intermediate results
   250	            if self.config.save_intermediate:
   251	                self._save_results(partial=True)
   252
   253	        # === Step 5: Save Final Results ===
   254	        self._log("Step 5: Saving results...")
   255	        output_file = self._save_results(partial=False)
   256
   257	        # === Cleanup ===
   258	        if self.config.validate_transcriptions:
   259	            cleanup_validator()  # Release CTC model resources
   260
   261	        if self.config.cleanup_after:
   262	            self._log("Cleaning up downloaded files...")
   263	            import shutil
   264	            video_work_dir = os.path.join(self.config.work_dir, self.config.video_id)
   265	            if os.path.exists(video_work_dir):
   266	                shutil.rmtree(video_work_dir)
   267
   268	        # === Build Result ===
   269	        total_time = time.time() - self.start_time
   270	        completed_at = datetime.now().isoformat()
   271
   272	        total_audio = sum(r.duration_sec for r in self.results)
   273
   274	        result = PipelineResult(
   275	            video_id=self.config.video_id,
   276	            language=language,
   277	            model=self.config.model,
   278	            thinking_level=self.config.thinking_level,
   279	            total_segments_found=stats.get('total_files', 0),
   280	            segments_processed=len(set(r.segment_id for r in self.results)),
   281	            chunks_created=len(chunks),
   282	            transcriptions_completed=len(self.results),
   283	            total_audio_duration_sec=total_audio,
   284	            total_processing_time_sec=total_time,
   285	            output_file=output_file,
   286	            errors=self.errors,
   287	            started_at=started_at,
   288	            completed_at=completed_at
   289	        )
   290


Output:
Exit code: 0
Wall time: 0 seconds
Output:
   190	    return re.sub(r'[।,.!?;:\'"()\-–—]', '', text).strip()
   191
   192
   193	def validate_transcription(
   194	    audio_path: str,
   195	    transcription: str,
   196	    language: str = "te",
   197	    check_audio: bool = True,
   198	    low_conf_threshold: float = 0.3,
   199	    low_conf_ratio_threshold: float = 0.8,  # Raised from 0.5 - code-mixed is normal
   200	    min_alignment_score: float = 0.5  # Below this = likely wrong transcription
   201	) -> ValidationResult:
   202	    """
   203	    Validate a transcription (native script with punctuation).
   204
   205	    Punctuation is stripped before character check and CTC alignment,
   206	    so this works with the new 'transcription' field that includes punctuation.
   207
   208	    Args:
   209	        audio_path: Path to audio file
   210	        transcription: Native script text (may include punctuation)
   211	        language: Language code (te, hi, ta, etc.)
   212	        check_audio: Whether to check audio-text match (slower)
   213	        low_conf_threshold: Per-word confidence threshold
   214	        low_conf_ratio_threshold: Max ratio of low-conf words before review
   215
   216	    Returns:
   217	        ValidationResult with status and details
   218	    """
   219	    reasons = []
   220
   221	    # Strip punctuation for validation (transcription field now includes punctuation)
   222	    clean_text = _strip_punctuation(transcription)
   223
   224	    # === STEP 1: Character validation (instant) ===
   225	    # Native transcription = no English allowed
   226	    char_check = check_characters(clean_text, language, allow_english=False)
   227
   228	    if not char_check["valid"]:
   229	        return ValidationResult(
   230	            status="reject",
   231	            char_valid=False,
   232	            invalid_chars=char_check["invalid_chars"],
   233	            script_ratio=char_check["script_ratio"],
   234	            reasons=["Invalid/alien characters found"]
   235	        )
   236
   237	    # Should be mostly native script
   238	    if char_check["script_ratio"] < 0.5:
   239	        reasons.append(f"Too few native chars ({char_check['script_ratio']:.0%})")
   240
   241	    # === STEP 2: Audio-text match (0.1s) ===
   242	    alignment_score = 0.0
   243	    low_conf_words = []
   244	    low_conf_ratio = 0.0
   245
   246	    if check_audio:
   247	        try:
   248	            aligner = _get_ctc_aligner(language)
   249	            # Use punctuation-stripped text for CTC alignment
   250	            result = aligner.align(audio_path, clean_text)
   251
   252	            alignment_score = result.alignment_score
   253	            low_conf_words = result.low_confidence_words
   254	            low_conf_ratio = result.low_confidence_ratio
   255
   256	            # REJECT if alignment is very poor (likely wrong transcription)
   257	            if alignment_score < min_alignment_score:
   258	                reasons.append(f"Poor alignment ({alignment_score:.2f} < {min_alignment_score})")
   259
   260	            # REVIEW if many low-confidence words (but could be code-mixed)
   261	            elif low_conf_ratio > low_conf_ratio_threshold:
   262	                reasons.append(f"Many low-confidence words ({low_conf_ratio:.0%})")
   263
   264	        except Exception as e:
   265	            reasons.append(f"Audio check failed: {str(e)[:50]}")
   266
   267	    # === Determine status ===
   268	    # Reject: invalid chars OR very poor alignment
   269	    # Review: high low-conf ratio
   270	    # Accept: everything else
   271	    if not char_check["valid"]:
   272	        status = "reject"
   273	    elif any("Poor alignment" in r for r in reasons):
   274	        status = "reject"
   275	    elif len(reasons) > 0:
   276	        status = "review"
   277	    else:
   278	        status = "accept"
   279
   280	    return ValidationResult(
   281	        status=status,
   282	        char_valid=char_check["valid"],
   283	        invalid_chars=char_check.get("invalid_chars"),
   284	        script_ratio=char_check["script_ratio"],
   285	        alignment_score=alignment_score,
   286	        low_confidence_words=low_conf_words,
   287	        low_confidence_ratio=low_conf_ratio,
   288	        reasons=reasons if reasons else None
   289	    )
   290


Output:
Exit code: 0
Wall time: 0 seconds
Output:
     1	{
     2	  "SPEAKER_00_0000_0.03-2.61.flac": {
     3	    "romanized_text": "vastundannamaata. naaku konni ads gurtuntaayi like example kachcha mango bite.",
     4	    "mms_score": 0.6177,
     5	    "mms_log_prob": -1.9116,
     6	    "native_ctc_score": 0.7173,
     7	    "native_status": "review"
     8	  },
     9	  "SPEAKER_00_0001_3.05-13.40.flac": {
    10	    "romanized_text": "aa maa chemmangilo oka retired taatamma okadunnaadu I will not name him but mari entante mangoes teesukuni xerox shop ki vellaadannamaata in fact naaku photocopy ante veru xerox ante veru ani telisindi akkade avunu",
    11	    "mms_score": 0.7987,
    12	    "mms_log_prob": -1.0066,
    13	    "native_ctc_score": 0.7026,
    14	    "native_status": "review"
    15	  },
    16	  "SPEAKER_00_0003_29.78-34.93.flac": {
    17	    "romanized_text": "asalu aa in fact like I got to know that you were behind the Kaccha Mangoes",
    18	    "mms_score": 0.9046,
    19	    "mms_log_prob": -0.477,
    20	    "native_ctc_score": 0.8158,
    21	    "native_status": "accept"
    22	  },
    23	  "SPEAKER_00_0004_36.29-37.41.flac": {
    24	    "romanized_text": "talk about that story as well.",
    25	    "mms_score": 0.7689,
    26	    "mms_log_prob": -1.1556,
    27	    "native_ctc_score": 0.4755,
    28	    "native_status": "reject"
    29	  },
    30	  "SPEAKER_00_0026_383.26-390.66.flac": {
    31	    "romanized_text": "connection untundannamaata. aa ante deeni gurinchi chaala connected questions unnaayi naaku. deenikante mundu indaaka Parle-G gurinchi teesukochhaaru kaabatti avunu. ippativaraku podcast chooshaaru kada.",
    32	    "mms_score": 0.747,
    33	    "mms_log_prob": -1.2649,
    34	    "native_ctc_score": 0.7138,
    35	    "native_status": "review"
    36	  },
    37	  "SPEAKER_00_0027_394.58-397.79.flac": {
    38	    "romanized_text": "Podcast chustunna padi mandilo aiduguriki Marketer avvalanna Thought vastadi",
    39	    "mms_score": 0.8147,
    40	    "mms_log_prob": -0.9266,
    41	    "native_ctc_score": 0.7259,
    42	    "native_status": "accept"
    43	  },
    44	  "SPEAKER_00_0031_450.01-460.11.flac": {
    45	    "romanized_text": "Sir almost fifty one fifty one podcast fifty podcasts ayipoyinayi. Manadi fifty second approximately. Congratulations. Thank you thank you. Mee mee Telugu chaala classy ga undi sir.",
    46	    "mms_score": 0.8361,
    47	    "mms_log_prob": -0.8194,
    48	    "native_ctc_score": 0.7218,
    49	    "native_status": "accept"
    50	  },
    51	  "SPEAKER_00_0037_521.61-528.64.flac": {
    52	    "romanized_text": "smooth undi kakapothe konchem classy words vasthunnayi anyways. so inko rendu moodu products gurinchi kooda questions unnayi anuko.",
    53	    "mms_score": 0.7957,
    54	    "mms_log_prob": -1.0216,
    55	    "native_ctc_score": 0.6768,
    56	    "native_status": "review"
    57	  },
    58	  "SPEAKER_00_0050_627.86-632.46.flac": {
    59	    "romanized_text": "so ee glucose biscuit nunchi gluco endi gluco nunchi G endi correct Parle-G correct correct",
    60	    "mms_score": 0.8312,
    61	    "mms_log_prob": -0.844,
    62	    "native_ctc_score": 0.7822,
    63	    "native_status": "accept"
    64	  },
    65	  "SPEAKER_00_0058_728.26-741.21.flac": {
    66	    "romanized_text": "four fifty five hundred varaku unnayandi. okay andulo oka SKU gurinchi ha I heard it correct me if I am wrong Rola Cola ane oka product danni descale cheddamani apesi tarvata evaro Twitter lo tweet cheste",
    67	    "mms_score": 0.8654,
    68	    "mms_log_prob": -0.6731,
    69	    "native_ctc_score": 0.8389,
    70	    "native_status": "accept"
    71	  }
    72	}


**Planning dynamic gate evaluation**


jq 'to_entries | map({id:.key,n:.value.native_ctc_score,r:.value.mms_score,and:(.value.native_ctc_score>=0.7 and .value.mms_score>=0.7),or:(.value.native_ctc_score>=0.7 or .value.mms_score>=0.7),avg:((.value.native_ctc_score+.value.mms_score)/2),weighted:(0.65*.value.native_ctc_score + 0.35*.value.mms_score),gap:((.value.native_ctc_score-.value.mms_score)|if .<0 then -.*1 else . end)}) as $rows | {and_pass:([$rows[]|select(.and)]|length), or_pass:([$rows[]|select(.or)]|length), avg_pass:([$rows[]|select(.avg>=0.7)]|length), weighted_pass:([$rows[]|select(.weighted>=0.7)]|length), weighted_and_floor_pass:([$rows[]|select(.weighted>=0.7 and (.n>=0.55 and .r>=0.55) and .gap<=0.25)]|length), total:($rows|length)}' consistency_test/roman_alignment.json


Output:
Exit code: 0
Wall time: 0 seconds
Output:
{
  "and_pass": 7,
  "or_pass": 10,
  "avg_pass": 8,
  "weighted_pass": 8,
  "weighted_and_floor_pass": 8,
  "total": 10
}


**Planning detailed failure analysis**


jq 'to_entries | map({id:.key,n:.value.native_ctc_score,r:.value.mms_score,avg:((.value.native_ctc_score+.value.mms_score)/2),weighted:(0.65*.value.native_ctc_score + 0.35*.value.mms_score)}) | map(select(.weighted<0.7 or .avg<0.7 or .n<0.7 or .r<0.7))' consistency_test/roman_alignment.json


Output:
Exit code: 0
Wall time: 0 seconds
Output:
[
  {
    "id": "SPEAKER_00_0000_0.03-2.61.flac",
    "n": 0.7173,
    "r": 0.6177,
    "avg": 0.6675,
    "weighted": 0.68244
  },
  {
    "id": "SPEAKER_00_0004_36.29-37.41.flac",
    "n": 0.4755,
    "r": 0.7689,
    "avg": 0.6222,
    "weighted": 0.57819
  },
  {
    "id": "SPEAKER_00_0037_521.61-528.64.flac",
    "n": 0.6768,
    "r": 0.7957,
    "avg": 0.73625,
    "weighted": 0.718415
  }
]


**Identifying transcription and validation inconsistencies**


nl -ba src/validators/simple_validator.py | sed -n '1,120p'


nl -ba src/backend/transcription_schema.py | sed -n '20,95p'


nl -ba src/backend/audio_processor.py | sed -n '20,90p'


Output:
Exit code: 0
Wall time: 0 seconds
Output:
     1	"""
     2	Simple Transcription Validator
     3	==============================
     4
     5	Two checks:
     6	1. Character validation - catch garbage/alien characters
     7	2. Audio-text match - verify words match what's spoken
     8
     9	Usage:
    10	    from src.validators.simple_validator import validate_transcription
    11
    12	    result = validate_transcription("audio.flac", "transcription", language="te")
    13	    print(result['status'])  # accept / review / reject
    14	"""
    15	import re
    16	from dataclasses import dataclass
    17	from typing import List, Dict, Optional, Set
    18
    19
    20	# Unicode ranges for Indic scripts
    21	SCRIPT_RANGES = {
    22	    "te": {  # Telugu
    23	        "name": "Telugu",
    24	        "ranges": [(0x0C00, 0x0C7F)],  # Telugu block
    25	        "allow_ascii_punct": True,
    26	        "allow_digits": True,
    27	    },
    28	    "hi": {  # Hindi (Devanagari)
    29	        "name": "Hindi",
    30	        "ranges": [(0x0900, 0x097F)],  # Devanagari block
    31	        "allow_ascii_punct": True,
    32	        "allow_digits": True,
    33	    },
    34	    "ta": {  # Tamil
    35	        "name": "Tamil",
    36	        "ranges": [(0x0B80, 0x0BFF)],
    37	        "allow_ascii_punct": True,
    38	        "allow_digits": True,
    39	    },
    40	    "kn": {  # Kannada
    41	        "name": "Kannada",
    42	        "ranges": [(0x0C80, 0x0CFF)],
    43	        "allow_ascii_punct": True,
    44	        "allow_digits": True,
    45	    },
    46	    "ml": {  # Malayalam
    47	        "name": "Malayalam",
    48	        "ranges": [(0x0D00, 0x0D7F)],
    49	        "allow_ascii_punct": True,
    50	        "allow_digits": True,
    51	    },
    52	    "bn": {  # Bengali
    53	        "name": "Bengali",
    54	        "ranges": [(0x0980, 0x09FF)],
    55	        "allow_ascii_punct": True,
    56	        "allow_digits": True,
    57	    },
    58	}
    59
    60	# Common allowed characters across all languages
    61	ALLOWED_COMMON = set(" \t\n.,!?;:'\"()-–—0123456789")
    62
    63	# English characters (for code-mixed text)
    64	ENGLISH_CHARS = set("abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ")
    65
    66
    67	def get_valid_chars(language: str, allow_english: bool = True) -> Set[int]:
    68	    """Get set of valid Unicode codepoints for a language."""
    69	    valid = set()
    70
    71	    # Add common chars
    72	    for c in ALLOWED_COMMON:
    73	        valid.add(ord(c))
    74
    75	    # Add script-specific chars
    76	    lang_config = SCRIPT_RANGES.get(language, SCRIPT_RANGES["te"])
    77	    for start, end in lang_config["ranges"]:
    78	        for cp in range(start, end + 1):
    79	            valid.add(cp)
    80
    81	    # Allow English for code-mixed
    82	    if allow_english:
    83	        for c in ENGLISH_CHARS:
    84	            valid.add(ord(c))
    85
    86	    return valid
    87
    88
    89	def check_characters(
    90	    text: str,
    91	    language: str = "te",
    92	    allow_english: bool = False  # Changed: native = no English
    93	) -> Dict:
    94	    """
    95	    Check if text contains only valid characters for the language.
    96
    97	    For native transcription: allow_english=False (default)
    98
    99	    Returns:
   100	        {
   101	            "valid": True/False,
   102	            "invalid_chars": [...],
   103	            "script_ratio": 0.8,  # ratio of native script chars
   104	        }
   105	    """
   106	    if not text or not text.strip():
   107	        return {"valid": False, "invalid_chars": [], "script_ratio": 0, "reason": "empty"}
   108
   109	    valid_chars = get_valid_chars(language, allow_english)
   110	    lang_config = SCRIPT_RANGES.get(language, SCRIPT_RANGES["te"])
   111
   112	    invalid_chars = []
   113	    script_count = 0
   114	    total_alpha = 0
   115
   116	    for i, char in enumerate(text):
   117	        cp = ord(char)
   118
   119	        # Check if in native script
   120	        is_native = any(start <= cp <= end for start, end in lang_config["ranges"])


Output:
Exit code: 0
Wall time: 0 seconds
Output:
   110	    def _log(self, message: str, level: str = "INFO"):
   111	        """Log a message with timestamp."""
   112	        timestamp = datetime.now().strftime("%H:%M:%S")
   113	        print(f"[{timestamp}] [{level}] {message}")
   114
   115	    def _get_language_code(self, language: str) -> str:
   116	        """Convert language name to code for validation."""
   117	        lang_map = {
   118	            "Telugu": "te", "Hindi": "hi", "Tamil": "ta",
   119	            "Kannada": "kn", "Malayalam": "ml", "Bengali": "bn",
   120	            "Marathi": "mr", "Gujarati": "gu", "Punjabi": "pa",
   121	            "Odia": "or",
   122	        }
   123	        return lang_map.get(language, "te")
   124
   125	    def run(self) -> PipelineResult:
   126	        """
   127	        Execute the full pipeline.
   128
   129	        Returns:
   130	            PipelineResult with summary and statistics


Output:
Exit code: 0
Wall time: 0 seconds
Output:
    20	"""
    21	from typing import Optional, Dict
    22	from pydantic import BaseModel, Field
    23
    24
    25	# === Language Pack Configs ===
    26	# Per-language metadata + script-specific rules injected into the master prompt.
    27	# Only rules that prevent ACTUAL transcription errors, not generic grammar.
    28	# v1: 12 languages (11 Indic + English). Add more as needed.
    29
    30	LANGUAGE_CONFIGS: Dict[str, Dict[str, str]] = {
    31	    # 1-2 practical script tips per language. Not grammar rules.
    32	    "Hindi": {
    33	        "bcp47": "hi-IN",
    34	        "script": "Devanagari",
    35	        "script_rules": "Preserve Nukta (ज़ vs ज, फ़ vs फ) when clearly pronounced.",
    36	    },
    37	    "Marathi": {
    38	        "bcp47": "mr-IN",
    39	        "script": "Devanagari",
    40	        "script_rules": "Preserve ळ (retroflex lateral) accurately.",
    41	    },
    42	    "Telugu": {
    43	        "bcp47": "te-IN",
    44	        "script": "Telugu",
    45	        "script_rules": "Don't over-split words. Preserve Sandhi/combined forms as spoken.",
    46	    },
    47	    "Tamil": {
    48	        "bcp47": "ta-IN",
    49	        "script": "Tamil",
    50	        "script_rules": "Distinguish short and long vowels accurately.",
    51	    },
    52	    "Kannada": {
    53	        "bcp47": "kn-IN",
    54	        "script": "Kannada",
    55	        "script_rules": "Preserve agglutinated/combined forms as spoken.",
    56	    },
    57	    "Malayalam": {
    58	        "bcp47": "ml-IN",
    59	        "script": "Malayalam",
    60	        "script_rules": "Don't split agglutinated words. Preserve chillu letters.",
    61	    },
    62	    "Gujarati": {
    63	        "bcp47": "gu-IN",
    64	        "script": "Gujarati",
    65	        "script_rules": "",
    66	    },
    67	    "Punjabi": {
    68	        "bcp47": "pa-IN",
    69	        "script": "Gurmukhi",
    70	        "script_rules": "",
    71	    },
    72	    "Bengali": {
    73	        "bcp47": "bn-IN",
    74	        "script": "Bengali",
    75	        "script_rules": "Preserve Chandrabindu for nasalization where spoken.",
    76	    },
    77	    "Assamese": {
    78	        "bcp47": "as-IN",
    79	        "script": "Assamese",
    80	        # Critical: these chars look like Bengali but are Assamese-specific.
    81	        "script_rules": "Use Assamese-specific ৰ (ra) and ৱ (wa), NOT Bengali equivalents.",
    82	    },
    83	    "Odia": {
    84	        "bcp47": "or-IN",
    85	        "script": "Odia",
    86	        "script_rules": "",
    87	    },
    88	    "English": {
    89	        "bcp47": "en-IN",
    90	        "script": "Latin",
    91	        "script_rules": "Standard English spelling. Don't phonetically approximate accents.",
    92	    },
    93	}
    94
    95


Output:
Exit code: 0
Wall time: 0 seconds
Output:
    20	    start_ms: int          # Start time in milliseconds within original
    21	    end_ms: int            # End time in milliseconds within original
    22	    duration_sec: float    # Duration in seconds
    23
    24	    @property
    25	    def is_split(self) -> bool:
    26	        return self.total_chunks > 1
    27
    28
    29	class AudioProcessor:
    30	    """Handles audio segment processing with duration controls."""
    31
    32	    def __init__(
    33	        self,
    34	        max_duration_sec: float = 10.0,
    35	        min_duration_sec: float = 1.0,
    36	        overlap_sec: float = 0.0,
    37	        output_format: str = "flac"
    38	    ):
    39	        """
    40	        Initialize the audio processor.
    41
    42	        Args:
    43	            max_duration_sec: Maximum allowed segment duration (hard limit)
    44	            min_duration_sec: Minimum segment duration to process
    45	            overlap_sec: Overlap between chunks when splitting (helps with word boundaries)
    46	            output_format: Output format for split chunks (flac, wav, mp3)
    47	        """
    48	        self.max_duration_sec = max_duration_sec
    49	        self.min_duration_sec = min_duration_sec
    50	        self.overlap_sec = overlap_sec
    51	        self.output_format = output_format
    52
    53	    def get_audio_duration(self, file_path: str) -> float:
    54	        """Get duration of an audio file in seconds."""
    55	        audio = AudioSegment.from_file(file_path)
    56	        return len(audio) / 1000.0
    57
    58	    def needs_splitting(self, file_path: str) -> bool:
    59	        """Check if a file needs to be split based on duration."""
    60	        duration = self.get_audio_duration(file_path)
    61	        return duration > self.max_duration_sec
    62
    63	    def split_audio(
    64	        self,
    65	        file_path: str,
    66	        output_dir: str,
    67	        chunk_duration_sec: Optional[float] = None
    68	    ) -> List[AudioChunk]:
    69	        """
    70	        Split an audio file into chunks of specified duration.
    71
    72	        Args:
    73	            file_path: Path to the audio file
    74	            output_dir: Directory to save chunks
    75	            chunk_duration_sec: Target chunk duration (defaults to max_duration_sec)
    76
    77	        Returns:
    78	            List of AudioChunk objects
    79	        """
    80	        if chunk_duration_sec is None:
    81	            chunk_duration_sec = self.max_duration_sec
    82
    83	        audio = AudioSegment.from_file(file_path)
    84	        total_duration_ms = len(audio)
    85	        total_duration_sec = total_duration_ms / 1000.0
    86
    87	        os.makedirs(output_dir, exist_ok=True)
    88	        original_name = Path(file_path).stem
    89
    90	        # If already short enough, return as single chunk


Summary: auto


**Identifying OOV token skips**


nl -ba src/validators/ctc_forced_aligner.py | sed -n '170,225p'


Output:
Exit code: 0
Wall time: 0 seconds
Output:
   170
   171	        if len(data.shape) > 1:
   172	            data = data.mean(axis=1)
   173
   174	        if sr != 16000:
   175	            waveform = torch.from_numpy(data).float().unsqueeze(0)
   176	            data = torchaudio.functional.resample(waveform, sr, 16000).squeeze().numpy()
   177	            sr = 16000
   178
   179	        return data, sr
   180
   181	    def _get_emissions(self, audio: np.ndarray) -> torch.Tensor:
   182	        """Get CTC log-probability emissions."""
   183	        inputs = self.processor(
   184	            audio, sampling_rate=16000, return_tensors="pt", padding=True
   185	        )
   186
   187	        with torch.no_grad():
   188	            logits = self.model(inputs.input_values.to(self.device)).logits
   189
   190	        emissions = torch.log_softmax(logits.cpu(), dim=-1)
   191	        return emissions
   192
   193	    def _text_to_tokens(self, text: str) -> torch.Tensor:
   194	        """Convert text to token IDs, using | as word separator."""
   195	        tokens = []
   196	        for char in text:
   197	            if char == " ":
   198	                if "|" in self.vocab:
   199	                    tokens.append(self.vocab["|"])
   200	            elif char in self.vocab:
   201	                tokens.append(self.vocab[char])
   202	            # Skip OOV characters
   203
   204	        return torch.tensor([tokens], dtype=torch.int32)
   205
   206	    def _compute_word_alignments(
   207	        self,
   208	        text: str,
   209	        alignment: torch.Tensor,
   210	        scores: torch.Tensor,
   211	        frame_duration: float
   212	    ) -> List[WordAlignment]:
   213	        """Extract word-level alignments from frame-level results."""
   214	        words = text.split()
   215
   216	        # Build character-to-word mapping
   217	        char_to_word = []
   218	        for word_idx, word in enumerate(words):
   219	            for _ in word:
   220	                char_to_word.append(word_idx)
   221	            char_to_word.append(-1)  # Space/separator
   222
   223	        # Remove trailing separator
   224	        if char_to_word and char_to_word[-1] == -1:
   225	            char_to_word = char_to_word[:-1]


**Flagging model mismatch for Bengali**


nl -ba src/validators/ctc_forced_aligner.py | sed -n '96,130p'


Output:
Exit code: 0
Wall time: 0 seconds
Output:
    96
    97	class CTCForcedAligner:
    98	    """
    99	    CTC-based forced alignment using torchaudio.
   100
   101	    This provides MFA-equivalent functionality using neural CTC models.
   102	    """
   103
   104	    # Language-specific Wav2Vec2 models
   105	    MODELS = {
   106	        "te": "anuragshas/wav2vec2-large-xlsr-53-telugu",
   107	        "hi": "theainerd/Wav2Vec2-large-xlsr-hindi",
   108	        "ta": "Harveenchadha/wav2vec2-large-xlsr-53-tamil",
   109	        "kn": "Harveenchadha/wav2vec2-large-xlsr-53-kannada",
   110	        "ml": "gvs/wav2vec2-large-xlsr-malayalam",
   111	        "bn": "ai4bharat/indicwav2vec-hindi",
   112	        "default": "facebook/wav2vec2-large-xlsr-53"
   113	    }
   114
   115	    def __init__(
   116	        self,
   117	        language: str = "te",
   118	        device: str = "auto",
   119	        cache_dir: str = "./models/ctc_aligner"
   120	    ):
   121	        self.language = language.lower()[:2]
   122	        self.cache_dir = Path(cache_dir)
   123	        self.cache_dir.mkdir(parents=True, exist_ok=True)
   124
   125	        if device == "auto":
   126	            self.device = "cuda" if torch.cuda.is_available() else "cpu"
   127	        else:
   128	            self.device = device
   129
   130	        self.model = None


**Summarizing system evaluation and risks**


**Realistic Score**
- `6/10` for Telugu-heavy, code-mixed conversational data today.
- `4/10` if you mean pan-Indic production quality across languages.
- So: decent bootstrap pipeline, not yet “large TTS/ASR training-grade” on its own.

**Is it sufficient for large TTS/ASR?**
- Not yet as the sole data gate.
- It is sufficient as a first-pass data factory if you add stricter calibration + retry + audit loops.

**Why I’m saying that**
- Determinism looks strong (`10/10` identical across 3 runs on 10 segments in `consistency_test/comparison.json`).
- Current pipeline only applies native CTC validation in code (`pipeline.py:231`, `src/validators/simple_validator.py:193`), with reject threshold at `0.5`, not your planned `0.7` (`src/validators/simple_validator.py:200`).
- `validation_action` exists but retry logic is not implemented (`pipeline.py:59` only).
- Roman/MFA scoring is currently experimental data file, not integrated gate (`consistency_test/roman_alignment.json`).
- Language coverage is inconsistent:
  - pipeline maps `mr/gu/pa/or` (`pipeline.py:117`)
  - script validator only defines `te/hi/ta/kn/ml/bn` (`src/validators/simple_validator.py:21`)
  - CTC model map is limited and even maps `bn` to a Hindi model (`src/validators/ctc_forced_aligner.py:105`).
- CTC tokenization silently skips OOV chars, which can hide some errors (`src/validators/ctc_forced_aligner.py:193`).

**Scoring design (`native CTC + roman MFA`)**
- Do **not** use `OR` (too lenient).
- Do **not** use strict `AND` (too many false retries on code-mix).
- Use weighted + floor + disagreement penalty:
  - `S = 0.65*N + 0.35*R - 0.15*abs(N-R)`
  - Accept if `S >= 0.70` and `min(N,R) >= 0.55`
  - Retry (Gemini-3-Pro) otherwise
  - After retry, if still `<0.70`, mark review/ignore (your plan)
  - Force review when `abs(N-R) > 0.20` even if `S >= 0.70`
- On your 10-sample roman file, `OR` would pass all (unsafe), strict `AND` passes 7/10, weighted rule passes 8/10 with better balance.

**Missing piece (the big one)**
- A calibrated, human-audited benchmark slice per language/domain to tune thresholds and detect drift.
- Without that, a deterministic single-source system can deterministically produce high-confidence mistakes at scale.

**Corner cases you should explicitly guard**
- Heavy code-switch + brand/proper nouns.
- Numbers, dates, currency, abbreviations, URLs.
- Dialect/accent variation within same language.
- VAD boundary cuts (you split at 10s with `overlap=0` by default).
- Overlapped speakers, music bed, laughter/crosstalk.
- Script-specific confusions (nukta/chandrabindu/chillu/Assamese variants).
- Romanization spelling variance for same pronunciation.
- Phonetically plausible but semantically wrong words (CTC may still score okay).