1. **[High] Retry path can still hard-fail due invalid default model ID**
- `pipeline.py:60` sets `retry_model = "gemini-2.5-pro-preview-05-06"`.
- That key is not allowed by `get_model_name` in `src/backend/config.py:102` (available list ends at `gemini-2.5-pro`, `gemini-3-pro-preview`, etc. in `src/backend/config.py:29`).
- `get_model_name` is called before error handling in `src/backend/gemini_transcriber.py:116`, so first retry attempt can crash the pipeline.

2. **[Medium] Native CTC score gets OOV penalty twice**
- In `src/validators/ctc_forced_aligner.py:369` and again at `src/validators/ctc_forced_aligner.py:372`, the same OOV penalty block is applied twice.
- This systematically depresses `N` on OOV-heavy/code-mixed segments and can change accept/retry/reject boundaries.

3. **[Medium] Current retry behavior does not match “retry anything below 70%”**
- Scoring labels `<0.55` as `reject` in `src/validators/simple_validator.py:275`.
- Pipeline retries only `status == "retry"` in `pipeline.py:261`.
- So low-score rejects are never retried with pro model, which diverges from your stated `<70 -> retry` policy.

4. **[Low] Fallback reason text can be incorrect in one-sided failure cases**
- In `src/validators/simple_validator.py:391`, message `"Both validators returned 0 — cannot verify"` can be added from the fallback block (`src/validators/simple_validator.py:385`) even when only one validator is zero.
- Not a gating bug, but it can mislead debugging.

**Confirmed fixes (good):**
- CTC tuple-return crash is fixed (`_text_to_tokens` now returns 2 values).
- No more silent auto-accept on total validation failure; forced failure now returns `review` with reason.

**Re-check results on your 22-segment sample:**
- `accept=18`, `retry=2`, `reject=2`
- `mean S=0.7234`, `mean N=0.6761`, `mean R=0.7819`
- CTC is running again (non-zero), and failure path no longer auto-accepts.

**On your disputed policy points:**
- MMS-higher weighting is defensible with your current sample (`R` > `N` on average).
- `0.40` vs `0.60` floor is a precision/recall tradeoff; no blocker from this review.
- `0.25` disagreement threshold is reasonable.
- Using 2.5-pro is fine; the issue is only the configured model string not matching allowed IDs.


All four verified:

1. **Retry model ID** — Changed `"gemini-2.5-pro-preview-05-06"` to `"gemini-2.5-pro"` which exists in `GEMINI_MODELS`. No more crash on first retry.

2. **Double OOV penalty** — Removed duplicate block at lines 371-373. Penalty now applied exactly once.

3. **Retry scope** — Changed from `status == "retry"` to `status in ("retry", "reject")`. All segments below the accept threshold now get a retry with the pro model, matching the plan.

4. **One-sided validator failure** — Added `elif check_audio and roman_mms_score > 0:` branch. Now when CTC fails but MMS returns 0.80, the score is correctly 0.80 with "accept" status instead of being discarded as "both returned 0." Each validator failure path is explicit with clear reasons.


# Collaboration Mode: Default

You are now in Default mode. Any previous instructions for other modes (e.g. Plan mode) are no longer active.

Your active mode changes only when new developer instructions with a different `<collaboration_mode>...</collaboration_mode>` change it; user requests or tool descriptions do not change mode by themselves. Known mode names are Default and Plan.

## request_user_input availability

The `request_user_input` tool is unavailable in Default mode. If you call it while in Default mode, it will return an error.

If a decision is necessary and cannot be discovered from local context, ask the user directly. However, in Default mode you should strongly prefer executing the user's request rather than stopping to ask questions.

Summary: auto

## Skills
A skill is a set of local instructions to follow that is stored in a `SKILL.md` file. Below is the list of skills that can be used. Each entry includes a name, description, and file path so you can open the source for full instructions when using a specific skill.
### Available skills
- skill-creator: Guide for creating effective skills. This skill should be used when users want to create a new skill (or update an existing skill) that extends Codex's capabilities with specialized knowledge, workflows, or tool integrations. (file: /home/ubuntu/.codex/skills/.system/skill-creator/SKILL.md)
- skill-installer: Install Codex skills into $CODEX_HOME/skills from a curated list or a GitHub repo path. Use when a user asks to list installable skills, install a curated skill, or install a skill from another repo (including private repos). (file: /home/ubuntu/.codex/skills/.system/skill-installer/SKILL.md)
### How to use skills
- Discovery: The list above is the skills available in this session (name + description + file path). Skill bodies live on disk at the listed paths.
- Trigger rules: If the user names a skill (with `$SkillName` or plain text) OR the task clearly matches a skill's description shown above, you must use that skill for that turn. Multiple mentions mean use them all. Do not carry skills across turns unless re-mentioned.
- Missing/blocked: If a named skill isn't in the list or the path can't be read, say so briefly and continue with the best fallback.
- How to use a skill (progressive disclosure):
  1) After deciding to use a skill, open its `SKILL.md`. Read only enough to follow the workflow.
  2) When `SKILL.md` references relative paths (e.g., `scripts/foo.py`), resolve them relative to the skill directory listed above first, and only consider other paths if needed.
  3) If `SKILL.md` points to extra folders such as `references/`, load only the specific files needed for the request; don't bulk-load everything.
  4) If `scripts/` exist, prefer running or patching them instead of retyping large code blocks.
  5) If `assets/` or templates exist, reuse them instead of recreating from scratch.
- Coordination and sequencing:
  - If multiple skills apply, choose the minimal set that covers the request and state the order you'll use them.
  - Announce which skill(s) you're using and why (one short line). If you skip an obvious skill, say why.
- Context hygiene:
  - Keep context small: summarize long sections instead of pasting them; only load extra files when needed.
  - Avoid deep reference-chasing: prefer opening only files directly linked from `SKILL.md` unless you're blocked.
  - When variants exist (frameworks, providers, domains), pick only the relevant reference file(s) and note that choice.
- Safety and fallback: If a skill can't be applied cleanly (missing files, unclear instructions), state the issue, pick the next-best approach, and continue.


**Preparing comprehensive code review**


I’ll verify each claimed fix directly in code and run targeted checks for retry behavior and validator fallback paths. After that I’ll report only residual issues, if any.


git diff -- src/validators/simple_validator.py src/validators/ctc_forced_aligner.py pipeline.py src/backend/config.py | sed -n '1,320p'


nl -ba src/validators/ctc_forced_aligner.py | sed -n '356,386p'


Output:
Exit code: 0
Wall time: 0 seconds
Output:
diff --git a/pipeline.py b/pipeline.py
index 18f6fb5..3c3b93c 100644
--- a/pipeline.py
+++ b/pipeline.py
@@ -56,7 +56,10 @@ class PipelineConfig:

     # === Validation Settings ===
     validate_transcriptions: bool = True    # Validate native transcriptions
-    validation_action: str = "flag"         # "flag" (mark status) or "retry" (re-transcribe)
+    validation_action: str = "retry"        # "flag" (mark status) or "retry" (re-transcribe with pro)
+    retry_model: str = "gemini-2.5-pro"                 # Model for retry attempts
+    retry_thinking_level: str = "medium"    # Thinking level for retry
+    min_snr_db: float = 5.0                 # SNR gate: skip segments below this (saves API credits)

     # === Output Settings ===
     output_dir: str = "./transcriptions"
@@ -118,7 +121,7 @@ class TranscriptionPipeline:
             "Telugu": "te", "Hindi": "hi", "Tamil": "ta",
             "Kannada": "kn", "Malayalam": "ml", "Bengali": "bn",
             "Marathi": "mr", "Gujarati": "gu", "Punjabi": "pa",
-            "Odia": "or",
+            "Odia": "or", "Assamese": "as", "English": "en",
         }
         return lang_map.get(language, "te")

@@ -187,19 +190,30 @@ class TranscriptionPipeline:
             self._log(f"Starting from index {self.config.segment_start_index}, "
                       f"{len(chunks)} chunks remaining")

-        # === Step 3.5: Polish Audio Boundaries ===
+        # === Step 3.5: Polish Audio + SNR Gate ===
         if self.config.polish_audio:
             self._log("Step 3.5: Polishing audio boundaries...")
             polisher = AudioPolisher()
             polished_dir = os.path.join(segments_dir, "polished")
             polished_count = 0
+            snr_skipped = 0
+            surviving_chunks = []
             for chunk in chunks:
                 result = polisher.polish(chunk.file_path, output_dir=polished_dir)
                 if result.was_modified:
                     chunk.file_path = result.output_path
                     polished_count += 1
+                # SNR gate: skip segments with very low SNR (likely noise/music)
+                if result.snr_db < self.config.min_snr_db:
+                    snr_skipped += 1
+                    continue
+                surviving_chunks.append(chunk)
             self._log(f"Polished {polished_count}/{len(chunks)} chunks "
                       f"({len(chunks) - polished_count} already clean)")
+            if snr_skipped > 0:
+                self._log(f"SNR gate: skipped {snr_skipped} chunks "
+                          f"(SNR < {self.config.min_snr_db}dB)")
+            chunks = surviving_chunks

         # === Step 4: Transcribe ===
         self._log(f"Step 4: Transcribing {len(chunks)} chunks...")
@@ -228,22 +242,91 @@ class TranscriptionPipeline:
                 )
             )

-            # === Validate transcriptions ===
+            # === Validate transcriptions (dual scoring: native CTC + romanized MMS) ===
             if self.config.validate_transcriptions:
                 lang_code = self._get_language_code(language)
                 for i, result in enumerate(batch_results):
                     if result.native:
+                        romanized = result.transcription.romanized or ""
                         validation = validate_transcription(
                             batch[i].file_path,
                             result.native,
-                            language=lang_code
+                            romanized_text=romanized,
+                            language=lang_code,
                         )
                         result.validation_status = validation.status
-                        result.validation_score = validation.alignment_score
-                        if validation.status == "reject":
-                            self._log(f"  ⚠ Validation failed for {result.segment_id}: "
-                                     f"{validation.reasons}", "WARN")
-
+                        result.validation_score = validation.combined_score
+
+                        # Retry with pro model if below accept threshold
+                        if (validation.status in ("retry", "reject")
+                                and self.config.validation_action == "retry"):
+                            self._log(
+                                f"  Retrying {result.segment_id} with "
+                                f"{self.config.retry_model} "
+                                f"(S={validation.combined_score:.2f})", "WARN"
+                            )
+                            retry_config = TranscriptionConfig(
+                                model=self.config.retry_model,
+                                thinking_level=self.config.retry_thinking_level,
+                                temperature=self.config.temperature,
+                                language=language,
+                            )
+                            retry_raw = transcriber.transcribe_audio(
+                                batch[i].file_path, retry_config
+                            )
+                            if not retry_raw.get("error"):
+                                # Re-validate the retry result
+                                from src.backend.transcription_schema import (
+                                    TranscriptionOutput, SpeakerMeta
+                                )
+                                speaker_data = retry_raw.get('speaker')
+                                speaker = (SpeakerMeta(**speaker_data)
+                                           if isinstance(speaker_data, dict)
+                                           else None)
+                                retry_output = TranscriptionOutput(
+                                    transcription=retry_raw.get('transcription', ''),
+                                    code_switch=retry_raw.get('code_switch', ''),
+                                    romanized=retry_raw.get('romanized', ''),
+                                    tagged=retry_raw.get('tagged', ''),
+                                    speaker=speaker,
+                                )
+                                retry_roman = retry_output.romanized or ""
+                                retry_val = validate_transcription(
+                                    batch[i].file_path,
+                                    retry_output.transcription,
+                                    romanized_text=retry_roman,
+                                    language=lang_code,
+                                )
+                                # Keep the better result
+                                if retry_val.combined_score > validation.combined_score:
+                                    result.transcription = retry_output
+                                    result.model_used = retry_config.model
+                                    result.thinking_level = retry_config.thinking_level
+                                    validation = retry_val
+                                    self._log(
+                                        f"  Retry improved: "
+                                        f"S={retry_val.combined_score:.2f} "
+                                        f"({retry_val.status})"
+                                    )
+                                else:
+                                    self._log(
+                                        f"  Retry did not improve: "
+                                        f"S={retry_val.combined_score:.2f}"
+                                    )
+                            # Update final validation state
+                            result.validation_status = validation.status
+                            result.validation_score = validation.combined_score
+
+                        if validation.status in ("reject", "retry"):
+                            self._log(
+                                f"  Validation: {result.segment_id} -> "
+                                f"{validation.status} "
+                                f"(S={validation.combined_score:.2f}, "
+                                f"CTC={validation.native_ctc_score:.2f}, "
+                                f"MMS={validation.roman_mms_score:.2f})",
+                                "WARN"
+                            )
+
             self.results.extend(batch_results)

             # Save intermediate results
diff --git a/src/validators/ctc_forced_aligner.py b/src/validators/ctc_forced_aligner.py
index 786cfbe..ed56b24 100644
--- a/src/validators/ctc_forced_aligner.py
+++ b/src/validators/ctc_forced_aligner.py
@@ -101,15 +101,23 @@ class CTCForcedAligner:
     This provides MFA-equivalent functionality using neural CTC models.
     """

-    # Language-specific Wav2Vec2 models
+    # Language-specific Wav2Vec2 models for CTC forced alignment
+    # Each model trained on the target language for best phoneme coverage.
+    # "default" (XLSR-53) is multilingual fallback for languages without dedicated models.
     MODELS = {
         "te": "anuragshas/wav2vec2-large-xlsr-53-telugu",
         "hi": "theainerd/Wav2Vec2-large-xlsr-hindi",
+        "mr": "theainerd/Wav2Vec2-large-xlsr-hindi",       # Marathi: Devanagari, use Hindi model
         "ta": "Harveenchadha/wav2vec2-large-xlsr-53-tamil",
         "kn": "Harveenchadha/wav2vec2-large-xlsr-53-kannada",
         "ml": "gvs/wav2vec2-large-xlsr-malayalam",
-        "bn": "ai4bharat/indicwav2vec-hindi",
-        "default": "facebook/wav2vec2-large-xlsr-53"
+        "bn": "arijitx/wav2vec2-large-xlsr-bengali",       # Fixed: was pointing to Hindi
+        "as": "arijitx/wav2vec2-large-xlsr-bengali",       # Assamese: closest to Bengali
+        "gu": "facebook/wav2vec2-large-xlsr-53",            # Gujarati: use multilingual XLSR
+        "pa": "facebook/wav2vec2-large-xlsr-53",            # Punjabi: use multilingual XLSR
+        "or": "facebook/wav2vec2-large-xlsr-53",            # Odia: use multilingual XLSR
+        "en": "facebook/wav2vec2-large-960h-lv60-self",     # English: dedicated English model
+        "default": "facebook/wav2vec2-large-xlsr-53",
     }

     def __init__(
@@ -190,19 +198,28 @@ class CTCForcedAligner:
         emissions = torch.log_softmax(logits.cpu(), dim=-1)
         return emissions

-    def _text_to_tokens(self, text: str) -> torch.Tensor:
-        """Convert text to token IDs, using | as word separator."""
+    def _text_to_tokens(self, text: str) -> tuple:
+        """Convert text to token IDs. Returns (tensor, oov_ratio).
+        OOV chars are dropped but tracked. High OOV = unreliable score."""
         tokens = []
+        total_chars = 0
+        oov_chars = 0
         for char in text:
             if char == " ":
                 if "|" in self.vocab:
                     tokens.append(self.vocab["|"])
             elif char in self.vocab:
                 tokens.append(self.vocab[char])
-            # Skip OOV characters
-
-        return torch.tensor([tokens], dtype=torch.int32)
-
+                total_chars += 1
+            else:
+                oov_chars += 1
+                total_chars += 1
+        oov_ratio = oov_chars / max(total_chars, 1)
+        if oov_ratio > 0.1:
+            print(f"[CTCAligner] WARNING: {oov_ratio:.0%} OOV chars "
+                  f"({oov_chars}/{total_chars})")
+        return torch.tensor([tokens], dtype=torch.int32), oov_ratio
+
     def _compute_word_alignments(
         self,
         text: str,
@@ -311,8 +328,8 @@ class CTCForcedAligner:
             num_frames = emissions.shape[1]
             frame_duration = audio_duration / num_frames

-            # Convert text to tokens
-            targets = self._text_to_tokens(transcription)
+            # Convert text to tokens (returns tuple with OOV ratio)
+            targets, oov_ratio = self._text_to_tokens(transcription)

             if targets.shape[1] == 0:
                 return AlignmentResult(
@@ -348,6 +365,9 @@ class CTCForcedAligner:
             # Typical range: -0.5 (excellent) to -5.0 (poor)
             # Map to 0-1: score = 1 + (log_prob / 5)
             alignment_score = max(0.0, min(1.0, 1.0 + mean_log_prob / 5.0))
+            # Penalize when many chars were OOV (alignment unreliable)
+            if oov_ratio > 0.1:
+                alignment_score *= (1.0 - oov_ratio * 0.5)

             # Find low confidence words
             low_conf_words = [wa.word for wa in word_alignments if wa.is_low_confidence]
diff --git a/src/validators/simple_validator.py b/src/validators/simple_validator.py
index 4e12abf..9d886b5 100644
--- a/src/validators/simple_validator.py
+++ b/src/validators/simple_validator.py
@@ -2,37 +2,53 @@
 Simple Transcription Validator
 ==============================

-Two checks:
+Three checks:
 1. Character validation - catch garbage/alien characters
-2. Audio-text match - verify words match what's spoken
+2. Native CTC alignment - verify native script matches audio
+3. Romanized MMS alignment - verify romanized text matches audio
+
+Combined scoring: S = 0.45*N + 0.55*R - 0.10*abs(N-R)
+  N = native CTC score, R = romanized MMS score
+  Accept if S >= 0.70 and min(N,R) >= 0.40
+  Forced review if abs(N-R) > 0.25 (high validator disagreement)

 Usage:
     from src.validators.simple_validator import validate_transcription

-    result = validate_transcription("audio.flac", "transcription", language="te")
-    print(result['status'])  # accept / review / reject
+    result = validate_transcription(
+        "audio.flac", "నాకు కొన్ని యాడ్స్",
+        romanized_text="naaku konni ads", language="te"
+    )
+    print(result.status)          # accept / review / retry / reject
+    print(result.combined_score)  # 0-1
 """
 import re
 from dataclasses import dataclass
 from typing import List, Dict, Optional, Set

-# Unicode ranges for Indic scripts
+# Unicode ranges for Indic scripts (all 12 supported languages)
 SCRIPT_RANGES = {
     "te": {  # Telugu
         "name": "Telugu",
-        "ranges": [(0x0C00, 0x0C7F)],  # Telugu block
+        "ranges": [(0x0C00, 0x0C7F)],
         "allow_ascii_punct": True,
         "allow_digits": True,
     },
     "hi": {  # Hindi (Devanagari)
         "name": "Hindi",
-        "ranges": [(0x0900, 0x097F)],  # Devanagari block
+        "ranges": [(0x0900, 0x097F)],
+        "allow_ascii_punct": True,
+        "allow_digits": True,
+    },
+    "mr": {  # Marathi (Devanagari, same block as Hindi)
+        "name": "Marathi",
+        "ranges": [(0x0900, 0x097F)],
         "allow_ascii_punct": True,
         "allow_digits": True,
     },
     "ta": {  # Tamil
-        "name": "Tamil",
+        "name": "Tamil",
         "ranges": [(0x0B80, 0x0BFF)],
         "allow_ascii_punct": True,
         "allow_digits": True,
@@ -55,6 +71,36 @@ SCRIPT_RANGES = {
         "allow_ascii_punct": True,
         "allow_digits": True,
     },
+    "as": {  # Assamese (same Unicode block as Bengali, slight char differences)
+        "name": "Assamese",
+        "ranges": [(0x0980, 0x09FF)],
+        "allow_ascii_punct": True,
+        "allow_digits": True,
+    },
+    "gu": {  # Gujarati


Output:
Exit code: 0
Wall time: 0 seconds
Output:
    52	    # === Transcription Settings ===
    53	    model: str = "gemini-3-flash-preview"   # Gemini model to use
    54	    thinking_level: Optional[str] = "low"   # minimal, low, medium, high (Gemini 3)
    55	    temperature: float = 0.0                # 0 for deterministic (tested safe with strict prompt)
    56
    57	    # === Validation Settings ===
    58	    validate_transcriptions: bool = True    # Validate native transcriptions
    59	    validation_action: str = "retry"        # "flag" (mark status) or "retry" (re-transcribe with pro)
    60	    retry_model: str = "gemini-2.5-pro"                 # Model for retry attempts
    61	    retry_thinking_level: str = "medium"    # Thinking level for retry
    62	    min_snr_db: float = 5.0                 # SNR gate: skip segments below this (saves API credits)
    63
    64	    # === Output Settings ===
    65	    output_dir: str = "./transcriptions"
    66	    save_intermediate: bool = True          # Save results after each batch
    67	    batch_size: int = 10                    # How many to process before saving
    68
    69	    # === Audio Polishing Settings ===
    70	    polish_audio: bool = True               # Polish segment boundaries before transcription
    71
    72	    # === Storage Settings ===
    73	    work_dir: str = field(default_factory=lambda: DEFAULT_SETTINGS["work_dir"])
    74	    cleanup_after: bool = False             # Remove downloaded files after processing
    75	    use_regex_fallback: bool = True         # Try regex if exact R2 match fails
    76
    77
    78	@dataclass
    79	class PipelineResult:
    80	    """Results from a pipeline run."""
    81	    video_id: str
    82	    language: str
    83	    model: str
    84	    thinking_level: Optional[str]
    85
    86	    total_segments_found: int
    87	    segments_processed: int
    88	    chunks_created: int
    89	    transcriptions_completed: int
    90


Output:
Exit code: 0
Wall time: 0 seconds
Output:
   356	            # Compute word alignments
   357	            word_alignments = self._compute_word_alignments(
   358	                transcription, alignment, scores, frame_duration
   359	            )
   360
   361	            # Overall scores
   362	            mean_log_prob = scores.mean().item()
   363
   364	            # Convert to 0-1 score
   365	            # Typical range: -0.5 (excellent) to -5.0 (poor)
   366	            # Map to 0-1: score = 1 + (log_prob / 5)
   367	            alignment_score = max(0.0, min(1.0, 1.0 + mean_log_prob / 5.0))
   368	            # Penalize when many chars were OOV (alignment unreliable)
   369	            if oov_ratio > 0.1:
   370	                alignment_score *= (1.0 - oov_ratio * 0.5)
   371
   372	            # Find low confidence words
   373	            low_conf_words = [wa.word for wa in word_alignments if wa.is_low_confidence]
   374	            low_conf_ratio = len(low_conf_words) / max(len(word_alignments), 1)
   375
   376	            processing_time = time.time() - start_time
   377
   378	            return AlignmentResult(
   379	                audio_path=audio_path,
   380	                transcription=transcription,
   381	                alignment_score=alignment_score,
   382	                mean_log_prob=mean_log_prob,
   383	                word_alignments=word_alignments,
   384	                low_confidence_words=low_conf_words,
   385	                low_confidence_ratio=low_conf_ratio,
   386	                audio_duration_sec=audio_duration,


Summary: auto


**Exploring validator retry condition**


nl -ba src/validators/simple_validator.py | sed -n '340,430p'


nl -ba src/backend/config.py | sed -n '26,52p'


Output:
Exit code: 0
Wall time: 0 seconds
Output:
   340	    native_ctc_score = 0.0
   341	    low_conf_words = []
   342	    low_conf_ratio = 0.0
   343
   344	    if check_audio:
   345	        try:
   346	            aligner = _get_ctc_aligner(language)
   347	            ctc_result = aligner.align(audio_path, clean_text)
   348	            native_ctc_score = ctc_result.alignment_score
   349	            low_conf_words = ctc_result.low_confidence_words
   350	            low_conf_ratio = ctc_result.low_confidence_ratio
   351	        except Exception as e:
   352	            reasons.append(f"CTC alignment failed: {str(e)[:50]}")
   353
   354	    # === STEP 3: Romanized MMS alignment ===
   355	    roman_mms_score = 0.0
   356
   357	    if check_audio and clean_roman:
   358	        try:
   359	            mms = _get_mms_aligner()
   360	            mms_result = mms.align(audio_path, clean_roman)
   361	            roman_mms_score = mms_result.alignment_score
   362	        except Exception as e:
   363	            reasons.append(f"MMS alignment failed: {str(e)[:50]}")
   364
   365	    # === STEP 4: Combined scoring ===
   366	    if check_audio and native_ctc_score > 0 and roman_mms_score > 0:
   367	        # Both validators returned scores - use weighted formula
   368	        combined_score, verdict = compute_combined_score(
   369	            native_ctc_score, roman_mms_score
   370	        )
   371	        if verdict == "review" and abs(native_ctc_score - roman_mms_score) > 0.25:
   372	            reasons.append(
   373	                f"Validator disagreement: CTC={native_ctc_score:.2f} "
   374	                f"MMS={roman_mms_score:.2f}"
   375	            )
   376	    elif check_audio and native_ctc_score > 0:
   377	        # CTC only (MMS failed or no romanized text)
   378	        combined_score = native_ctc_score
   379	        if native_ctc_score >= 0.70:
   380	            verdict = "accept"
   381	        elif native_ctc_score >= 0.50:
   382	            verdict = "retry"
   383	        else:
   384	            verdict = "reject"
   385	        if roman_mms_score == 0 and clean_roman:
   386	            reasons.append("MMS failed, using CTC only")
   387	    elif check_audio and roman_mms_score > 0:
   388	        # MMS only (CTC failed)
   389	        combined_score = roman_mms_score
   390	        if roman_mms_score >= 0.70:
   391	            verdict = "accept"
   392	        elif roman_mms_score >= 0.50:
   393	            verdict = "retry"
   394	        else:
   395	            verdict = "reject"
   396	        reasons.append("CTC failed, using MMS only")
   397	    elif check_audio:
   398	        # Both validators returned 0
   399	        combined_score = 0.0
   400	        verdict = "review"
   401	        reasons.append("Both validators returned 0")
   402	    else:
   403	        # No audio check requested
   404	        combined_score = 0.0
   405	        verdict = "review" if reasons else "accept"
   406
   407	    # Character validation failures override
   408	    if not char_check["valid"]:
   409	        verdict = "reject"
   410
   411	    # Append score-based reasons
   412	    if verdict == "retry":
   413	        reasons.append(
   414	            f"Below threshold: combined={combined_score:.2f} "
   415	            f"(CTC={native_ctc_score:.2f}, MMS={roman_mms_score:.2f})"
   416	        )
   417	    elif verdict == "reject" and combined_score > 0:
   418	        reasons.append(
   419	            f"Poor alignment: combined={combined_score:.2f} "
   420	            f"(CTC={native_ctc_score:.2f}, MMS={roman_mms_score:.2f})"
   421	        )
   422
   423	    return ValidationResult(
   424	        status=verdict,
   425	        char_valid=char_check["valid"],
   426	        invalid_chars=char_check.get("invalid_chars"),
   427	        script_ratio=char_check["script_ratio"],
   428	        alignment_score=native_ctc_score,  # backward compat
   429	        native_ctc_score=native_ctc_score,
   430	        roman_mms_score=roman_mms_score,


Output:
Exit code: 0
Wall time: 0 seconds
Output:
   248	                for i, result in enumerate(batch_results):
   249	                    if result.native:
   250	                        romanized = result.transcription.romanized or ""
   251	                        validation = validate_transcription(
   252	                            batch[i].file_path,
   253	                            result.native,
   254	                            romanized_text=romanized,
   255	                            language=lang_code,
   256	                        )
   257	                        result.validation_status = validation.status
   258	                        result.validation_score = validation.combined_score
   259
   260	                        # Retry with pro model if below accept threshold
   261	                        if (validation.status in ("retry", "reject")
   262	                                and self.config.validation_action == "retry"):
   263	                            self._log(
   264	                                f"  Retrying {result.segment_id} with "
   265	                                f"{self.config.retry_model} "
   266	                                f"(S={validation.combined_score:.2f})", "WARN"
   267	                            )
   268	                            retry_config = TranscriptionConfig(
   269	                                model=self.config.retry_model,
   270	                                thinking_level=self.config.retry_thinking_level,
   271	                                temperature=self.config.temperature,
   272	                                language=language,
   273	                            )
   274	                            retry_raw = transcriber.transcribe_audio(
   275	                                batch[i].file_path, retry_config
   276	                            )
   277	                            if not retry_raw.get("error"):
   278	                                # Re-validate the retry result
   279	                                from src.backend.transcription_schema import (
   280	                                    TranscriptionOutput, SpeakerMeta
   281	                                )
   282	                                speaker_data = retry_raw.get('speaker')
   283	                                speaker = (SpeakerMeta(**speaker_data)
   284	                                           if isinstance(speaker_data, dict)
   285	                                           else None)
   286	                                retry_output = TranscriptionOutput(
   287	                                    transcription=retry_raw.get('transcription', ''),
   288	                                    code_switch=retry_raw.get('code_switch', ''),
   289	                                    romanized=retry_raw.get('romanized', ''),
   290	                                    tagged=retry_raw.get('tagged', ''),
   291	                                    speaker=speaker,
   292	                                )
   293	                                retry_roman = retry_output.romanized or ""
   294	                                retry_val = validate_transcription(
   295	                                    batch[i].file_path,
   296	                                    retry_output.transcription,
   297	                                    romanized_text=retry_roman,
   298	                                    language=lang_code,
   299	                                )
   300	                                # Keep the better result
   301	                                if retry_val.combined_score > validation.combined_score:
   302	                                    result.transcription = retry_output
   303	                                    result.model_used = retry_config.model
   304	                                    result.thinking_level = retry_config.thinking_level
   305	                                    validation = retry_val
   306	                                    self._log(
   307	                                        f"  Retry improved: "
   308	                                        f"S={retry_val.combined_score:.2f} "
   309	                                        f"({retry_val.status})"
   310	                                    )
   311	                                else:
   312	                                    self._log(
   313	                                        f"  Retry did not improve: "
   314	                                        f"S={retry_val.combined_score:.2f}"
   315	                                    )
   316	                            # Update final validation state
   317	                            result.validation_status = validation.status
   318	                            result.validation_score = validation.combined_score
   319
   320	                        if validation.status in ("reject", "retry"):
   321	                            self._log(
   322	                                f"  Validation: {result.segment_id} -> "
   323	                                f"{validation.status} "
   324	                                f"(S={validation.combined_score:.2f}, "
   325	                                f"CTC={validation.native_ctc_score:.2f}, "
   326	                                f"MMS={validation.roman_mms_score:.2f})",
   327	                                "WARN"
   328	                            )
   329
   330	            self.results.extend(batch_results)
   331
   332	            # Save intermediate results


Output:
Exit code: 0
Wall time: 0 seconds
Output:
    26
    27	# === Available Gemini Models ===
    28	# Models supporting audio transcription
    29	GEMINI_MODELS = {
    30	    "gemini-3-pro-preview": {
    31	        "name": "gemini-3-pro-preview",
    32	        "thinking_levels": ["low", "high"],
    33	        "supports_thinking": True,
    34	        "cost_tier": "premium"
    35	    },
    36	    "gemini-3-flash-preview": {
    37	        "name": "gemini-3-flash-preview",
    38	        "thinking_levels": ["minimal", "low", "medium", "high"],
    39	        "supports_thinking": True,
    40	        "cost_tier": "standard"
    41	    },
    42	    "gemini-2.5-pro": {
    43	        "name": "gemini-2.5-pro",
    44	        "thinking_levels": None,
    45	        "supports_thinking": True,
    46	        "cost_tier": "premium"
    47	    },
    48	    "gemini-2.5-flash": {
    49	        "name": "gemini-2.5-flash",
    50	        "thinking_levels": None,
    51	        "supports_thinking": True,
    52	        "cost_tier": "standard"


**Planning mock tests for fallback**


Code-level checks match your claims so far. I’m now running behavioral verification: sample-batch validation plus mocked one-sided-failure cases to confirm the new fallback branches and retry gating behave as intended.


venv/bin/python - <<'PY'
import json, os
from collections import Counter
from statistics import mean
from src.validators import validate_transcription, cleanup

input_file='transcriptions/test_new_schema_20seg.json'
base='/tmp/maya3_transcribe/pF_BQpHaIdU/extracted/pF_BQpHaIdU/segments'
with open(input_file, encoding='utf-8') as f:
    d=json.load(f)
rows=[]
for r in d['results']:
    ap=os.path.join(base,r['segment_id'])
    if not os.path.exists(ap):
        continue
    v=validate_transcription(ap, r['transcription']['transcription'], romanized_text=r['transcription'].get('romanized',''), language='te')
    rows.append((r['segment_id'], v.status, v.combined_score, v.native_ctc_score, v.roman_mms_score, v.reasons or []))
cleanup()
print('count', len(rows))
print('status', dict(Counter(x[1] for x in rows)))
print('meanS', round(mean(x[2] for x in rows),4))
print('meanN', round(mean(x[3] for x in rows),4))
print('meanR', round(mean(x[4] for x in rows),4))
print('retry_or_reject', [(x[0],x[1],round(x[2],3)) for x in rows if x[1] in ('retry','reject')])
PY


venv/bin/python - <<'PY'
import src.validators.simple_validator as sv

# Save originals
orig_ctc = sv._get_ctc_aligner
orig_mms = sv._get_mms_aligner

class CTCFail:
    def align(self, *a, **k):
        raise RuntimeError('ctc boom')

class MMSSucc:
    class R:
        alignment_score=0.8
    def align(self, *a, **k):
        return self.R()

class CTCSucc:
    class R:
        alignment_score=0.81
        low_confidence_words=[]
        low_confidence_ratio=0.0
    def align(self, *a, **k):
        return self.R()

class MMSFail:
    def align(self, *a, **k):
        raise RuntimeError('mms boom')

try:
    # CTC fails, MMS succeeds
    sv._get_ctc_aligner = lambda language: CTCFail()
    sv._get_mms_aligner = lambda : MMSSucc()
    v1 = sv.validate_transcription('/tmp/x.flac', 'నాకు కొన్ని యాడ్స్', language='te', romanized_text='naaku konni ads', check_audio=True)
    print('case1', v1.status, v1.combined_score, v1.native_ctc_score, v1.roman_mms_score, v1.reasons)

    # CTC succeeds, MMS fails
    sv._get_ctc_aligner = lambda language: CTCSucc()
    sv._get_mms_aligner = lambda : MMSFail()
    v2 = sv.validate_transcription('/tmp/x.flac', 'నాకు కొన్ని యాడ్స్', language='te', romanized_text='naaku konni ads', check_audio=True)
    print('case2', v2.status, v2.combined_score, v2.native_ctc_score, v2.roman_mms_score, v2.reasons)

    # Both fail
    sv._get_ctc_aligner = lambda language: CTCFail()
    sv._get_mms_aligner = lambda : MMSFail()
    v3 = sv.validate_transcription('/tmp/x.flac', 'నాకు కొన్ని యాడ్స్', language='te', romanized_text='naaku konni ads', check_audio=True)
    print('case3', v3.status, v3.combined_score, v3.native_ctc_score, v3.roman_mms_score, v3.reasons)
finally:
    sv._get_ctc_aligner = orig_ctc
    sv._get_mms_aligner = orig_mms
PY


venv/bin/python - <<'PY'
from src.backend.config import get_model_name
print(get_model_name('gemini-2.5-pro'))
PY


Output:
Exit code: 0
Wall time: 14.5 seconds
Output:
[CTCAligner] Loading anuragshas/wav2vec2-large-xlsr-53-telugu...
[CTCAligner] Ready on cuda
[MMSAligner] Ready on cuda (29 labels, SR=16000)
count 22
status {'accept': 18, 'reject': 2, 'retry': 2}
meanS 0.7234
meanN 0.6761
meanR 0.7819
retry_or_reject [('SPEAKER_00_0004_36.29-37.41.flac', 'reject', 0.475), ('SPEAKER_00_0037_521.61-528.64.flac', 'retry', 0.699), ('SPEAKER_00_0057_715.64-717.95.flac', 'retry', 0.641), ('SPEAKER_00_0077_1026.28-1027.43.flac', 'reject', 0.0)]
/home/ubuntu/maya3_transcribe/venv/lib/python3.10/site-packages/transformers/configuration_utils.py:309: UserWarning: Passing `gradient_checkpointing` to a config initialization is deprecated and will be removed in v5 Transformers. Using `model.gradient_checkpointing_enable()` instead, or if you are using the `Trainer` API, pass `gradient_checkpointing=True` in your `TrainingArguments`.
  warnings.warn(


Output:
Exit code: 0
Wall time: 0 seconds
Output:
case1 accept 0.8 0.0 0.8 ['CTC alignment failed: ctc boom', 'CTC failed, using MMS only']
case2 accept 0.81 0.81 0.0 ['MMS alignment failed: mms boom', 'MMS failed, using CTC only']
case3 review 0.0 0.0 0.0 ['CTC alignment failed: ctc boom', 'MMS alignment failed: mms boom', 'Both validators returned 0']


Output:
Exit code: 0
Wall time: 1.6 seconds
Output:
gemini-2.5-pro


**Reviewing residual validation risks**


rg -n "retry_model|2\.5-pro-preview-05-06|validation_action == \"retry\"|status in \(\"retry\", \"reject\"\)" -S pipeline.py src | sed -n '1,200p'


Output:
Exit code: 0
Wall time: 0 seconds
Output:
pipeline.py:60:    retry_model: str = "gemini-2.5-pro"                 # Model for retry attempts
pipeline.py:261:                        if (validation.status in ("retry", "reject")
pipeline.py:262:                                and self.config.validation_action == "retry"):
pipeline.py:265:                                f"{self.config.retry_model} "
pipeline.py:269:                                model=self.config.retry_model,


**Summarizing key review findings**


**Findings**
- **Medium:** Dual-validation guarantee is weakened when one aligner fails. `src/validators/simple_validator.py:387` to `src/validators/simple_validator.py:396` now allows `accept` on MMS-only or CTC-only paths. In my mocked check, CTC failure + MMS `0.8` returned `accept`. If one validator is silently degraded system-wide, bad data can pass.
- **Medium (cost/routing):** Retry now includes all rejects (`pipeline.py:261`), including deterministic character rejects from `src/validators/simple_validator.py:327`. This can spend pro-model retries on segments unlikely to recover (for example strict script invalid cases), unless that spend is intentional.
- **Low (testing gap):** I don’t see automated tests covering these critical branches (CTC tuple return, both-zero fallback, one-sided fallback, retry-on-reject path). Current validation relies on manual/runtime checks.

**Verified**
- Confirmed fixed: retry model ID now valid (`pipeline.py:60` -> `gemini-2.5-pro`).
- Confirmed fixed: CTC token function returns 2 values and no unpack crash.
- Confirmed fixed: duplicate OOV penalty removed (single application at `src/validators/ctc_forced_aligner.py:369`).
- Confirmed fixed: retry condition is now `status in ("retry", "reject")` (`pipeline.py:261`).
- Confirmed added: explicit MMS-only fallback branch (`src/validators/simple_validator.py:387`).

No blocker regressions beyond the two design risks above.


We're good. Let me explain why each concern is acceptable as-is.

**Single-validator accept (MMS-only when CTC fails):** The concern is valid at system level — if CTC silently degrades across ALL segments, bad data passes. But the fix isn't to block single-validator accepts (that would reject every segment for languages where CTC hasn't been downloaded yet, or every English-heavy segment where CTC legitimately fails). The right answer is **batch-level health monitoring**: if CTC fails on >20% of a batch, something systemic is wrong and the pipeline should warn. At the individual segment level, MMS=0.80 with CTC failure IS a valid accept — the audio matches the romanized text. I wouldn't change this.

**Retry includes character rejects:** Character rejects mean Gemini produced wrong-script characters. A stronger model (pro) genuinely might produce cleaner output — character rejects are a transcription quality issue, not an audio quality issue. The waste case (wrong-language audio) is rare and costs one extra API call. Not worth adding a special-case exclusion for.

**No automated tests:** Fair observation, good practice, but not in the current plan scope. The critical paths are verified via the manual test runs. If you want a test suite later, we can build one — but it's a separate task.

None of these are blockers or regressions. The system is functionally solid with the four fixes we just applied. The next real quality lever is populating the golden test set to calibrate those scoring thresholds against ground truth.


**Agreeing with health monitoring plan**


Your reasoning is sound. I agree these are acceptable tradeoffs and not blockers.

What matters now:
1. Add **batch health monitoring** (CTC success rate, MMS success rate, one-sided-accept rate, reject/retry drift) with alert thresholds, so silent validator degradation is caught quickly.
2. Populate the **golden set** and calibrate thresholds against ground truth; that is the highest-impact next quality lever.

If you want, I can add the batch-level health checks directly into `pipeline.py` next.


do you agree to these justifications for scoring table validation ? think heavily and realistically and decide if you agree/disagree ? provide reasons if so.

Scoring Table Justification
S = 0.45 * N + 0.55 * R - 0.10 * |N - R|
Edge Case Analysis
Case 1: Clean native-heavy segment (Telugu, minimal English)
Example: N=0.82, R=0.85 -> S = 0.369 + 0.468 - 0.003 = 0.834 -> accept
Both validators agree. Penalty negligible. Correct behavior.
Case 2: Code-mixed segment (50% English words)
Example: N=0.55, R=0.82 -> S = 0.248 + 0.451 - 0.027 = 0.672 -> retry
CTC struggles with transliterated English, MMS handles it fine. The weighted formula doesn't auto-accept because CTC is low. Retry might help (or might not, since this is a CTC model limitation, not a transcription quality issue). After retry, if same scores, the segment stays "retry" and gets flagged.
Is this a false negative? Possibly. The transcription could be correct but CTC can't validate it. This is the known limitation of CTC on code-mixed audio. The alternative (auto-accept on high MMS only) risks accepting bad native script. The retry path gives it a second chance.
Case 3: Pure English segment in Telugu pipeline
Example: N=0.48, R=0.77 -> S = 0.216 + 0.424 - 0.029 = 0.611 -> retry
min(N,R) = 0.48 >= 0.40 floor. But S < 0.70. Goes to retry.
After retry with pro model: if pro produces same (correct) transcription, same scores, stays "retry." Flagged for review.
This is correct. A pure English segment in a Telugu pipeline IS suspicious and worth flagging. The detected_language field (now added) will show "English", which tells the pipeline operator this segment was misclassified.
Case 4: Hallucinated transcription (Gemini made up words)
Example: N=0.30, R=0.35 -> S = 0.135 + 0.193 - 0.005 = 0.323 -> reject
Both validators agree it doesn't match audio. Correct rejection.
Case 5: CTC model failure (GPU OOM, model not downloaded)
N=0, R=0.80 -> Falls to MMS-only path: combined=0.80 -> accept
Batch health monitor fires "CTC failing" alert if this happens on >20% of batch.
Individual segment: accepting on MMS only is correct (audio matches romanized text). Batch level: the alert catches systemic CTC failure.
Case 6: MMS failure (model didn't load)
N=0.72, R=0 -> Falls to CTC-only path: combined=0.72 -> accept
Batch health fires "MMS failing" alert.
Case 7: Both validators fail
N=0, R=0 -> review ("Both validators returned 0")
Never auto-accepts. Correct.
Case 8: High disagreement, both scores decent
N=0.85, R=0.55 -> S = 0.383 + 0.303 - 0.030 = 0.656 -> retry
Even though CTC says "great," MMS says "mediocre." Something unusual — maybe bad romanization, or CTC is being fooled by phonetically similar wrong words. Retry is correct.
With |N-R| = 0.30 > 0.25: if S were above 0.70, it would be "review" (forced flag even on passing score).
Case 9: Near-threshold segment
N=0.65, R=0.75 -> S = 0.293 + 0.413 - 0.010 = 0.696 -> retry (just barely below 0.70)
This is borderline. Retry might push it over with a better transcription from the pro model. If not, it stays flagged. This is the intended behavior — the 0.70 threshold is deliberately conservative for training data quality.
Case 10: Boundary artifact (garbled start from VAD cut)
N=0.72, R=0.62 -> S = 0.324 + 0.341 - 0.010 = 0.655 -> retry
Both scores are decent but below threshold because the artifact confuses alignment. The polisher should have caught this, but if it didn't, retry gives a second chance.
Can false positives sneak through? Only if BOTH validators independently agree a wrong transcription matches the audio. For native CTC: the wrong Telugu characters would need to phonetically match the audio. For romanized MMS: the wrong Latin transliteration would need to phonetically match. Both being wrong in the same way requires Gemini to hallucinate text that happens to be phonetically identical to the actual audio — possible for phonetically similar words (corner case #9 from the plan: "phonetically plausible but semantically wrong"), but extremely unlikely for entire sentences. The floor check (min >= 0.40) and disagreement penalty add extra safety.
Is 0.70 the right threshold? We don't know yet — it's uncalibrated. That's what the golden set is for. But 0.70 is conservative: our 10-segment test showed correct transcriptions scoring 0.67-0.86. Setting the bar at 0.70 means ~20% of correct segments get retried (borderline ones), which costs API calls but doesn't lose data.


Summary: auto


**Evaluating scoring justifications and fallback risks**


I’ll quickly verify whether the batch health monitor and `detected_language` handling are actually implemented in the code right now, because your justification depends on those safeguards being real.


rg -n "health|ctc fail|mms fail|failure rate|>20%|alert|detected_language|language_detect|language.*field" -S pipeline.py src transcriptions golden_set


nl -ba src/backend/transcription_schema.py | sed -n '1,240p'


Output:
Exit code: 0
Wall time: 0 seconds
Output:
pipeline.py:102:class _BatchHealth:
pipeline.py:103:    """Tracks validator health per batch. Catches silent degradation."""
pipeline.py:116:    # Alert thresholds
pipeline.py:117:    CTC_FAIL_THRESHOLD = 0.20    # Warn if CTC fails on >20% of batch
pipeline.py:118:    MMS_FAIL_THRESHOLD = 0.20    # Warn if MMS fails on >20% of batch
pipeline.py:141:        """Log batch health summary. Warn on threshold breaches."""
pipeline.py:150:        log_fn(f"  Health: {self.validated} validated | "
pipeline.py:160:        # Alert on threshold breaches
pipeline.py:161:        alerts = []
pipeline.py:163:            alerts.append(f"CTC failing on {1-ctc_rate:.0%} of segments")
pipeline.py:165:            alerts.append(f"MMS failing on {1-mms_rate:.0%} of segments")
pipeline.py:167:            alerts.append(f"Reject rate {rej_rate:.0%} exceeds {self.REJECT_THRESHOLD:.0%}")
pipeline.py:169:            alerts.append(
pipeline.py:173:        for alert in alerts:
pipeline.py:174:            log_fn(f"  ALERT: {alert}", "WARN")
pipeline.py:175:        return alerts
pipeline.py:189:        self._health_alerts: List[str] = []  # Accumulated health alerts
pipeline.py:330:                batch_health = _BatchHealth()
pipeline.py:394:                                    batch_health.retry_improved += 1
pipeline.py:414:                        # Record for batch health monitoring
pipeline.py:415:                        batch_health.record(validation)
pipeline.py:417:                # Batch health report + alerts
pipeline.py:418:                alerts = batch_health.report(self._log)
pipeline.py:419:                self._health_alerts.extend(alerts)
src/backend/gemini_transcriber.py:210:                detected_language=raw_result.get('detected_language', ''),
src/backend/transcription_schema.py:144:    detected_language: str = Field(
src/backend/transcription_schema.py:158:    language: str = Field(description="Primary language of the audio")
src/backend/transcription_schema.py:244:7. LANGUAGE MISMATCH: The target language is {language}, but trust what you hear. If the audio is clearly in a different language, transcribe in that language's script and set detected_language accordingly. If code-mixed, use the dominant language for the native script fields.
src/backend/transcription_schema.py:277:6. detected_language
src/backend/transcription_schema.py:343:        "detected_language": {
src/backend/transcription_schema.py:348:    "required": ["transcription", "code_switch", "romanized", "tagged", "speaker", "detected_language"],
src/validators/simple_validator.py:377:        # CTC only (MMS failed or no romanized text)
src/validators/simple_validator.py:386:            reasons.append("MMS failed, using CTC only")
src/validators/simple_validator.py:388:        # MMS only (CTC failed)
src/validators/simple_validator.py:396:        reasons.append("CTC failed, using MMS only")


Output:
Exit code: 0
Wall time: 0 seconds
Output:
     1	"""
     2	Pydantic schemas for structured transcription output.
     3
     4	Output fields (what Gemini returns per segment):
     5	  1. transcription   - Native script with punctuation (PRIMARY, authoritative)
     6	  2. code_switch     - Mixed script: native + English in Latin (derived from 1)
     7	  3. romanized       - Full Latin transliteration (derived from 1)
     8	  4. tagged          - Code-switch + audio event tags [laugh] etc. (derived from 2)
     9	  5. speaker         - Metadata: emotion, style, pace, accent
    10
    11	Prompt v3 design (evolved from v1 original -> v2 strict -> v3 current):
    12	  - Single master template, dynamic language pack injection for 12 languages
    13	  - "Verbatim transcription system" persona (explicit NOT assistant negation)
    14	  - Acoustic punctuation (prosody-based pauses, not grammar rules)
    15	  - Anti-hallucination: [UNK] for uncertain words, no sentence completion
    16	  - Field derivation ordering: transcription is authoritative, others derive from it
    17	  - Transliterate != Translate distinction (fixed v2 contradiction that caused
    18	    English speech to be translated into Telugu on segment 3 test)
    19	  - Per-language script tips (1-2 practical tips, not grammar rules)
    20	"""
    21	from typing import Optional, Dict
    22	from pydantic import BaseModel, Field
    23
    24
    25	# === Language Pack Configs ===
    26	# Per-language metadata + script-specific rules injected into the master prompt.
    27	# Only rules that prevent ACTUAL transcription errors, not generic grammar.
    28	# v1: 12 languages (11 Indic + English). Add more as needed.
    29
    30	LANGUAGE_CONFIGS: Dict[str, Dict[str, str]] = {
    31	    # 1-2 practical script tips per language. Not grammar rules.
    32	    "Hindi": {
    33	        "bcp47": "hi-IN",
    34	        "script": "Devanagari",
    35	        "script_rules": "Preserve Nukta (ज़ vs ज, फ़ vs फ) when clearly pronounced.",
    36	    },
    37	    "Marathi": {
    38	        "bcp47": "mr-IN",
    39	        "script": "Devanagari",
    40	        "script_rules": "Preserve ळ (retroflex lateral) accurately.",
    41	    },
    42	    "Telugu": {
    43	        "bcp47": "te-IN",
    44	        "script": "Telugu",
    45	        "script_rules": "Don't over-split words. Preserve Sandhi/combined forms as spoken.",
    46	    },
    47	    "Tamil": {
    48	        "bcp47": "ta-IN",
    49	        "script": "Tamil",
    50	        "script_rules": "Distinguish short and long vowels accurately.",
    51	    },
    52	    "Kannada": {
    53	        "bcp47": "kn-IN",
    54	        "script": "Kannada",
    55	        "script_rules": "Preserve agglutinated/combined forms as spoken.",
    56	    },
    57	    "Malayalam": {
    58	        "bcp47": "ml-IN",
    59	        "script": "Malayalam",
    60	        "script_rules": "Don't split agglutinated words. Preserve chillu letters.",
    61	    },
    62	    "Gujarati": {
    63	        "bcp47": "gu-IN",
    64	        "script": "Gujarati",
    65	        "script_rules": "",
    66	    },
    67	    "Punjabi": {
    68	        "bcp47": "pa-IN",
    69	        "script": "Gurmukhi",
    70	        "script_rules": "",
    71	    },
    72	    "Bengali": {
    73	        "bcp47": "bn-IN",
    74	        "script": "Bengali",
    75	        "script_rules": "Preserve Chandrabindu for nasalization where spoken.",
    76	    },
    77	    "Assamese": {
    78	        "bcp47": "as-IN",
    79	        "script": "Assamese",
    80	        # Critical: these chars look like Bengali but are Assamese-specific.
    81	        "script_rules": "Use Assamese-specific ৰ (ra) and ৱ (wa), NOT Bengali equivalents.",
    82	    },
    83	    "Odia": {
    84	        "bcp47": "or-IN",
    85	        "script": "Odia",
    86	        "script_rules": "",
    87	    },
    88	    "English": {
    89	        "bcp47": "en-IN",
    90	        "script": "Latin",
    91	        "script_rules": "Standard English spelling. Don't phonetically approximate accents.",
    92	    },
    93	}
    94
    95
    96	class SpeakerMeta(BaseModel):
    97	    """Speaker metadata for TTS training. Only what matters."""
    98	    emotion: str = Field(
    99	        default="neutral",
   100	        description="neutral, happy, sad, angry, excited, surprised"
   101	    )
   102	    speaking_style: str = Field(
   103	        default="conversational",
   104	        description="conversational, narrative, excited, calm, emphatic, sarcastic, formal"
   105	    )
   106	    pace: str = Field(
   107	        default="normal",
   108	        description="slow, normal, fast"
   109	    )
   110	    accent: str = Field(
   111	        default="",
   112	        description="Regional accent/dialect if detectable, empty string if unknown"
   113	    )
   114
   115
   116	class TranscriptionOutput(BaseModel):
   117	    """Structured output: 4 transcription formats + speaker metadata."""
   118
   119	    transcription: str = Field(
   120	        description="Native script transcription with minimal punctuation. "
   121	                    "English words transliterated phonetically in native script."
   122	    )
   123
   124	    code_switch: str = Field(
   125	        description="Mixed script: native language in native script, "
   126	                    "English words in Latin script. Minimal punctuation."
   127	    )
   128
   129	    romanized: str = Field(
   130	        description="Full Roman/Latin script transliteration as pronounced."
   131	    )
   132
   133	    tagged: str = Field(
   134	        description="Code-switch transcription with audio event tags "
   135	                    "([laugh], [cough], [sigh], [breath], [singing], "
   136	                    "[noise], [music], [applause]) at positions where they occur."
   137	    )
   138
   139	    speaker: Optional[SpeakerMeta] = Field(
   140	        default=None,
   141	        description="Speaker metadata: emotion, style, pace, accent."
   142	    )
   143
   144	    detected_language: str = Field(
   145	        default="",
   146	        description="The language actually spoken in the audio as detected by the model. "
   147	                    "Helps catch language mismatches vs pipeline-provided language."
   148	    )
   149
   150
   151	class TranscriptionResult(BaseModel):
   152	    """Complete result for a transcribed segment including metadata."""
   153
   154	    segment_id: str = Field(description="Identifier for the audio segment")
   155	    chunk_index: int = Field(default=0, description="Chunk index if segment was split")
   156	    total_chunks: int = Field(default=1, description="Total chunks for this segment")
   157	    duration_sec: float = Field(description="Duration of audio in seconds")
   158	    language: str = Field(description="Primary language of the audio")
   159
   160	    transcription: TranscriptionOutput = Field(description="The transcription outputs")
   161
   162	    model_used: str = Field(description="Gemini model used for transcription")
   163	    thinking_level: Optional[str] = Field(default=None, description="Thinking level used")
   164	    processing_time_sec: Optional[float] = Field(default=None, description="API call time")
   165
   166	    # Validation fields (filled by CTC validator, not Gemini)
   167	    validation_status: Optional[str] = Field(default=None, description="accept/review/reject")
   168	    validation_score: Optional[float] = Field(default=None, description="Alignment score 0-1")
   169
   170	    @property
   171	    def native(self) -> str:
   172	        """Shortcut to primary native transcription."""
   173	        return self.transcription.transcription
   174
   175
   176	# === Prompt v3 ===
   177	# Master template: one prompt for all 12 languages, with dynamic language pack.
   178	# Evolution: v1 (original) -> v2 (strict, fixed segment 3 translation) -> v3 (below)
   179	# v3 changes: translate/transliterate fix, field derivation ordering, [UNK] tokens,
   180	#   prosody-based punctuation, simplified language script tips, no-preamble constraint,
   181	#   "not assistant" negation, additionalProperties in schema.
   182
   183	def get_transcription_prompt(language: str) -> str:
   184	    """
   185	    System instruction for Gemini transcription.
   186
   187	    Uses master template with dynamic language pack injection.
   188	    Key design: "transcription" field is authoritative, all others derive from it.
   189	    """
   190	    # Language pack (fallback to generic if unknown language)
   191	    lang_config = LANGUAGE_CONFIGS.get(language, {})
   192	    script_name = lang_config.get("script", f"{language} native")
   193	    bcp47 = lang_config.get("bcp47", "")
   194	    script_rules = lang_config.get("script_rules", "")
   195
   196	    # Build optional script rules section
   197	    script_section = ""
   198	    if script_rules:
   199	        script_section = (
   200	            f"\nSCRIPT RULES FOR {language.upper()}:\n{script_rules}\n"
   201	        )
   202
   203	    # English has different field logic (no transliteration needed)
   204	    if language.lower() == "english":
   205	        native_field_rule = (
   206	            "Write in standard English spelling."
   207	        )
   208	        code_switch_rule = (
   209	            "Same as transcription. If the speaker uses words from another "
   210	            "language, write them in Latin script as heard."
   211	        )
   212	    else:
   213	        # The transliterate vs translate distinction is critical here.
   214	        # v2 prompt had a contradiction: "don't convert English to {language}"
   215	        # + "write English phonetically in {language} script". The word "convert"
   216	        # was ambiguous and caused segment 3 to be translated instead of transcribed.
   217	        native_field_rule = (
   218	            f"Write ONLY in {script_name} script. No Latin characters in this field.\n"
   219	            f"   When the speaker uses English words, TRANSLITERATE them phonetically "
   220	            f"into {script_name} (sound mapping, NOT meaning translation).\n"
   221	            f"   Example: spoken 'computer' → phonetic form in {script_name}, "
   222	            f"NOT the {language} word for computer."
   223	        )
   224	        code_switch_rule = (
   225	            f"Preserve language switching exactly as spoken.\n"
   226	            f"   {language} words stay in {script_name} script. "
   227	            f"English words stay in Latin script.\n"
   228	            f"   Do NOT translate in either direction."
   229	        )
   230
   231	    lang_label = f"{language} ({bcp47})" if bcp47 else language
   232
   233	    return f"""You are a verbatim speech-to-text transcription system. You are NOT a conversational assistant. Your output must precisely match the audio content.
   234
   235	TARGET: {lang_label}
   236
   237	CRITICAL RULES (violations cause rejection):
   238	1. NEVER TRANSLATE. This is transcription, not translation. If the speaker says English words, those are English. If the speaker says {language} words, those are {language}.
   239	2. VERBATIM FIDELITY: Every repetition, filler, stammer, false start, hesitation - exactly as spoken.
   240	3. NO CORRECTION: Do not fix grammar, pronunciation, dialect, or word choice.


**Preparing to evaluate batch health monitor**


I confirmed both `detected_language` and a batch health monitor exist. I’ll inspect the monitor logic itself before giving a final agree/disagree verdict on your scoring justification.


Output:
Exit code: 0
Wall time: 0 seconds
Output:
    88	    chunks_created: int
    89	    transcriptions_completed: int
    90
    91	    total_audio_duration_sec: float
    92	    total_processing_time_sec: float
    93
    94	    output_file: str
    95	    errors: List[str]
    96
    97	    started_at: str
    98	    completed_at: str
    99
   100
   101	@dataclass
   102	class _BatchHealth:
   103	    """Tracks validator health per batch. Catches silent degradation."""
   104	    validated: int = 0
   105	    ctc_success: int = 0       # CTC returned > 0
   106	    mms_success: int = 0       # MMS returned > 0
   107	    dual_success: int = 0      # Both returned > 0
   108	    one_sided_accept: int = 0  # Accepted with only one validator
   109	    accept: int = 0
   110	    review: int = 0
   111	    retry: int = 0
   112	    reject: int = 0
   113	    retry_improved: int = 0
   114	    total_combined: float = 0.0
   115
   116	    # Alert thresholds
   117	    CTC_FAIL_THRESHOLD = 0.20    # Warn if CTC fails on >20% of batch
   118	    MMS_FAIL_THRESHOLD = 0.20    # Warn if MMS fails on >20% of batch
   119	    REJECT_THRESHOLD = 0.30      # Warn if >30% of batch rejected
   120	    ONE_SIDED_THRESHOLD = 0.40   # Warn if >40% accepted on single validator
   121
   122	    def record(self, validation):
   123	        """Record one validation result."""
   124	        self.validated += 1
   125	        ctc_ok = validation.native_ctc_score > 0
   126	        mms_ok = validation.roman_mms_score > 0
   127	        if ctc_ok:
   128	            self.ctc_success += 1
   129	        if mms_ok:
   130	            self.mms_success += 1
   131	        if ctc_ok and mms_ok:
   132	            self.dual_success += 1
   133	        if validation.status == "accept" and not (ctc_ok and mms_ok):
   134	            self.one_sided_accept += 1
   135	        status_attr = validation.status  # accept/review/retry/reject
   136	        if hasattr(self, status_attr):
   137	            setattr(self, status_attr, getattr(self, status_attr) + 1)
   138	        self.total_combined += validation.combined_score
   139
   140	    def report(self, log_fn):
   141	        """Log batch health summary. Warn on threshold breaches."""
   142	        n = max(self.validated, 1)
   143	        avg_s = self.total_combined / n
   144	        ctc_rate = self.ctc_success / n
   145	        mms_rate = self.mms_success / n
   146	        dual_rate = self.dual_success / n
   147	        rej_rate = self.reject / n
   148	        one_sided_rate = self.one_sided_accept / max(self.accept, 1)
   149
   150	        log_fn(f"  Health: {self.validated} validated | "
   151	               f"accept={self.accept} review={self.review} "
   152	               f"retry={self.retry} reject={self.reject} | "
   153	               f"avg_S={avg_s:.3f}")
   154	        log_fn(f"  Validators: CTC={ctc_rate:.0%} MMS={mms_rate:.0%} "
   155	               f"dual={dual_rate:.0%} | "
   156	               f"one-sided accepts={self.one_sided_accept}")
   157	        if self.retry_improved > 0:
   158	            log_fn(f"  Retries improved: {self.retry_improved}")
   159
   160	        # Alert on threshold breaches
   161	        alerts = []
   162	        if ctc_rate < (1 - self.CTC_FAIL_THRESHOLD):
   163	            alerts.append(f"CTC failing on {1-ctc_rate:.0%} of segments")
   164	        if mms_rate < (1 - self.MMS_FAIL_THRESHOLD):
   165	            alerts.append(f"MMS failing on {1-mms_rate:.0%} of segments")
   166	        if rej_rate > self.REJECT_THRESHOLD:
   167	            alerts.append(f"Reject rate {rej_rate:.0%} exceeds {self.REJECT_THRESHOLD:.0%}")
   168	        if self.accept > 0 and one_sided_rate > self.ONE_SIDED_THRESHOLD:
   169	            alerts.append(
   170	                f"One-sided accept rate {one_sided_rate:.0%} "
   171	                f"exceeds {self.ONE_SIDED_THRESHOLD:.0%}"
   172	            )
   173	        for alert in alerts:
   174	            log_fn(f"  ALERT: {alert}", "WARN")
   175	        return alerts
   176
   177
   178	class TranscriptionPipeline:
   179	    """
   180	    Main pipeline orchestrating the full transcription workflow.
   181	    """
   182
   183	    def __init__(self, config: PipelineConfig):
   184	        """Initialize pipeline with configuration."""
   185	        self.config = config
   186	        self.results: List[TranscriptionResult] = []
   187	        self.errors: List[str] = []
   188	        self.start_time: Optional[float] = None
   189	        self._health_alerts: List[str] = []  # Accumulated health alerts
   190


**Inspecting health recording logic**


Output:
Exit code: 0
Wall time: 0 seconds
Output:
   320	                transcription_config,
   321	                progress_callback=lambda c, t: self._log(
   322	                    f"  Chunk {batch_start + c}/{len(chunks)}: "
   323	                    f"{batch[c-1].original_segment}"
   324	                )
   325	            )
   326
   327	            # === Validate transcriptions (dual scoring: native CTC + romanized MMS) ===
   328	            if self.config.validate_transcriptions:
   329	                lang_code = self._get_language_code(language)
   330	                batch_health = _BatchHealth()
   331	                for i, result in enumerate(batch_results):
   332	                    if result.native:
   333	                        romanized = result.transcription.romanized or ""
   334	                        validation = validate_transcription(
   335	                            batch[i].file_path,
   336	                            result.native,
   337	                            romanized_text=romanized,
   338	                            language=lang_code,
   339	                        )
   340	                        result.validation_status = validation.status
   341	                        result.validation_score = validation.combined_score
   342
   343	                        # Retry with pro model if below accept threshold
   344	                        if (validation.status in ("retry", "reject")
   345	                                and self.config.validation_action == "retry"):
   346	                            self._log(
   347	                                f"  Retrying {result.segment_id} with "
   348	                                f"{self.config.retry_model} "
   349	                                f"(S={validation.combined_score:.2f})", "WARN"
   350	                            )
   351	                            retry_config = TranscriptionConfig(
   352	                                model=self.config.retry_model,
   353	                                thinking_level=self.config.retry_thinking_level,
   354	                                temperature=self.config.temperature,
   355	                                language=language,
   356	                            )
   357	                            retry_raw = transcriber.transcribe_audio(
   358	                                batch[i].file_path, retry_config
   359	                            )
   360	                            if not retry_raw.get("error"):
   361	                                # Re-validate the retry result
   362	                                from src.backend.transcription_schema import (
   363	                                    TranscriptionOutput, SpeakerMeta
   364	                                )
   365	                                speaker_data = retry_raw.get('speaker')
   366	                                speaker = (SpeakerMeta(**speaker_data)
   367	                                           if isinstance(speaker_data, dict)
   368	                                           else None)
   369	                                retry_output = TranscriptionOutput(
   370	                                    transcription=retry_raw.get('transcription', ''),
   371	                                    code_switch=retry_raw.get('code_switch', ''),
   372	                                    romanized=retry_raw.get('romanized', ''),
   373	                                    tagged=retry_raw.get('tagged', ''),
   374	                                    speaker=speaker,
   375	                                )
   376	                                retry_roman = retry_output.romanized or ""
   377	                                retry_val = validate_transcription(
   378	                                    batch[i].file_path,
   379	                                    retry_output.transcription,
   380	                                    romanized_text=retry_roman,
   381	                                    language=lang_code,
   382	                                )
   383	                                # Keep the better result
   384	                                if retry_val.combined_score > validation.combined_score:
   385	                                    result.transcription = retry_output
   386	                                    result.model_used = retry_config.model
   387	                                    result.thinking_level = retry_config.thinking_level
   388	                                    validation = retry_val
   389	                                    self._log(
   390	                                        f"  Retry improved: "
   391	                                        f"S={retry_val.combined_score:.2f} "
   392	                                        f"({retry_val.status})"
   393	                                    )
   394	                                    batch_health.retry_improved += 1
   395	                                else:
   396	                                    self._log(
   397	                                        f"  Retry did not improve: "
   398	                                        f"S={retry_val.combined_score:.2f}"
   399	                                    )
   400	                            # Update final validation state
   401	                            result.validation_status = validation.status
   402	                            result.validation_score = validation.combined_score
   403
   404	                        if validation.status in ("reject", "retry"):
   405	                            self._log(
   406	                                f"  Validation: {result.segment_id} -> "
   407	                                f"{validation.status} "
   408	                                f"(S={validation.combined_score:.2f}, "
   409	                                f"CTC={validation.native_ctc_score:.2f}, "
   410	                                f"MMS={validation.roman_mms_score:.2f})",
   411	                                "WARN"
   412	                            )
   413
   414	                        # Record for batch health monitoring
   415	                        batch_health.record(validation)
   416
   417	                # Batch health report + alerts
   418	                alerts = batch_health.report(self._log)
   419	                self._health_alerts.extend(alerts)
   420
   421	            self.results.extend(batch_results)
   422
   423	            # Save intermediate results
   424	            if self.config.save_intermediate:
   425	                self._save_results(partial=True)
   426
   427	        # === Step 5: Save Final Results ===
   428	        self._log("Step 5: Saving results...")
   429	        output_file = self._save_results(partial=False)
   430


**Summarizing nuanced agreement with caveats**


Short answer: **mostly agree** with your justification, with **2 important caveats**.

**What I agree with**
1. The formula is directionally right for your setting:
`S = 0.45N + 0.55R - 0.10|N-R|`
It gives code-mixed robustness via `R` while still requiring native signal through `N`.
2. Your case math and verdicts are consistent for Cases 1, 2, 4, 8, 9, 10.
3. `0.70` as a temporary threshold is a reasonable conservative starting point before golden-set calibration.
4. Your updated behavior for `N=0,R>0` and `R=0,N>0` is practical **if** health monitoring is active (which it now is in code).

**Where I partially disagree**
1. The statement “false positives only if both validators agree on wrong text” is no longer fully true.
Because you now allow **one-sided accepts** (MMS-only / CTC-only), false positives can also pass when one validator fails and the surviving validator is fooled on that segment.
2. Case 3 (“pure English in Telugu pipeline is suspicious”) is policy-dependent, not universally true.
If your objective is strict Telugu-only training, yes, suspicious.
If your objective is realistic Indic code-mixed ASR/TTS, many such segments are valid and should be routed, not inherently penalized.

**Critical condition for me to agree overall**
I agree with your scoring design **only if batch health alerts are treated operationally**, not just logged:
- CTC fail rate >20%
- MMS fail rate >20%
- high one-sided accept rate

If alerts are ignored in production, one-sided accept becomes a real risk.

**Final verdict**
- **Agree (with caveats).**
- Your scoring table is pragmatic and internally coherent for now.
- The next real quality jump is exactly what you said: **populate golden set and calibrate thresholds per language/code-mix profile**.