Input:
*** Begin Patch
*** Update File: pipeline.py
@@
                 batch_health = _BatchHealth()
                 for i, result in enumerate(batch_results):
                     if result.native:
-                        romanized = result.transcription.romanized or ""
+                        romanized_result = derive_romanized_text(
+                            native_text=result.native,
+                            language=lang_code,
+                            fallback_romanized=result.transcription.romanized or "",
+                        )
+                        # Make romanized deterministic across runs.
+                        result.transcription.romanized = romanized_result.text
                         validation = validate_transcription(
                             batch[i].file_path,
                             result.native,
-                            romanized_text=romanized,
+                            romanized_text=romanized_result.text,
                             language=lang_code,
+                            min_duration_sec=self.config.min_segment_duration_sec,
                         )
                         result.validation_status = validation.status
                         result.validation_score = validation.combined_score
@@
                                 retry_output = TranscriptionOutput(
                                     transcription=retry_raw.get('transcription', ''),
                                     code_switch=retry_raw.get('code_switch', ''),
                                     romanized=retry_raw.get('romanized', ''),
                                     tagged=retry_raw.get('tagged', ''),
                                     speaker=speaker,
+                                    detected_language=retry_raw.get('detected_language', ''),
                                 )
-                                retry_roman = retry_output.romanized or ""
+                                retry_romanized = derive_romanized_text(
+                                    native_text=retry_output.transcription,
+                                    language=lang_code,
+                                    fallback_romanized=retry_output.romanized or "",
+                                )
+                                retry_output.romanized = retry_romanized.text
                                 retry_val = validate_transcription(
                                     batch[i].file_path,
                                     retry_output.transcription,
-                                    romanized_text=retry_roman,
+                                    romanized_text=retry_romanized.text,
                                     language=lang_code,
+                                    min_duration_sec=self.config.min_segment_duration_sec,
                                 )
*** End Patch


Output: {"output":"Success. Updated the following files:\nM pipeline.py\n","metadata":{"exit_code":0,"duration_seconds":0.0}}


# Collaboration Mode: Default

You are now in Default mode. Any previous instructions for other modes (e.g. Plan mode) are no longer active.

Your active mode changes only when new developer instructions with a different `<collaboration_mode>...</collaboration_mode>` change it; user requests or tool descriptions do not change mode by themselves. Known mode names are Default and Plan.

## request_user_input availability

The `request_user_input` tool is unavailable in Default mode. If you call it while in Default mode, it will return an error.

If a decision is necessary and cannot be discovered from local context, ask the user directly. However, in Default mode you should strongly prefer executing the user's request rather than stopping to ask questions.

Summary: auto

## Skills
A skill is a set of local instructions to follow that is stored in a `SKILL.md` file. Below is the list of skills that can be used. Each entry includes a name, description, and file path so you can open the source for full instructions when using a specific skill.
### Available skills
- skill-creator: Guide for creating effective skills. This skill should be used when users want to create a new skill (or update an existing skill) that extends Codex's capabilities with specialized knowledge, workflows, or tool integrations. (file: /home/ubuntu/.codex/skills/.system/skill-creator/SKILL.md)
- skill-installer: Install Codex skills into $CODEX_HOME/skills from a curated list or a GitHub repo path. Use when a user asks to list installable skills, install a curated skill, or install a skill from another repo (including private repos). (file: /home/ubuntu/.codex/skills/.system/skill-installer/SKILL.md)
### How to use skills
- Discovery: The list above is the skills available in this session (name + description + file path). Skill bodies live on disk at the listed paths.
- Trigger rules: If the user names a skill (with `$SkillName` or plain text) OR the task clearly matches a skill's description shown above, you must use that skill for that turn. Multiple mentions mean use them all. Do not carry skills across turns unless re-mentioned.
- Missing/blocked: If a named skill isn't in the list or the path can't be read, say so briefly and continue with the best fallback.
- How to use a skill (progressive disclosure):
  1) After deciding to use a skill, open its `SKILL.md`. Read only enough to follow the workflow.
  2) When `SKILL.md` references relative paths (e.g., `scripts/foo.py`), resolve them relative to the skill directory listed above first, and only consider other paths if needed.
  3) If `SKILL.md` points to extra folders such as `references/`, load only the specific files needed for the request; don't bulk-load everything.
  4) If `scripts/` exist, prefer running or patching them instead of retyping large code blocks.
  5) If `assets/` or templates exist, reuse them instead of recreating from scratch.
- Coordination and sequencing:
  - If multiple skills apply, choose the minimal set that covers the request and state the order you'll use them.
  - Announce which skill(s) you're using and why (one short line). If you skip an obvious skill, say why.
- Context hygiene:
  - Keep context small: summarize long sections instead of pasting them; only load extra files when needed.
  - Avoid deep reference-chasing: prefer opening only files directly linked from `SKILL.md` unless you're blocked.
  - When variants exist (frameworks, providers, domains), pick only the relevant reference file(s) and note that choice.
- Safety and fallback: If a skill can't be applied cleanly (missing files, unclear instructions), state the issue, pick the next-best approach, and continue.


**Extending config with new fields**


Input:
*** Begin Patch
*** Update File: pipeline.py
@@
             "config": {
                 "max_segment_duration_sec": self.config.max_segment_duration_sec,
                 "min_segment_duration_sec": self.config.min_segment_duration_sec,
+                "split_boundary_padding_ms": self.config.split_boundary_padding_ms,
                 "max_segments": self.config.max_segments,
             },
*** End Patch


Output: {"output":"Success. Updated the following files:\nM pipeline.py\n","metadata":{"exit_code":0,"duration_seconds":0.0}}


Summary: auto


**Inspecting pipeline signature details**


I’ve wired deterministic romanization into both first-pass and retry validation, and now I’m updating API/CLI parameters so min duration and split padding are first-class knobs.


Output:
Chunk ID: ef274a
Wall time: 0.0517 seconds
Process exited with code 0
Original token count: 1471
Output:
            "model": self.config.model,
            "thinking_level": self.config.thinking_level,
            "language": self.results[0].language if self.results else self.config.default_language,
            "config": {
                "max_segment_duration_sec": self.config.max_segment_duration_sec,
                "min_segment_duration_sec": self.config.min_segment_duration_sec,
                "split_boundary_padding_ms": self.config.split_boundary_padding_ms,
                "max_segments": self.config.max_segments,
            },
            "results_count": len(self.results),
            "total_duration_sec": sum(r.duration_sec for r in self.results),
            "results": [r.model_dump() for r in self.results]
        }

        with open(output_path, 'w', encoding='utf-8') as f:
            json.dump(output_data, f, ensure_ascii=False, indent=2)

        return output_path

def run_pipeline(
    video_id: str,
    language: Optional[str] = None,
    model: str = "gemini-3-flash-preview",
    thinking_level: str = "low",
    temperature: float = 0.0,
    max_segments: Optional[int] = None,
    max_duration_sec: float = 10.0,
    output_dir: str = "./transcriptions",
    validate: bool = True
) -> PipelineResult:
    """
    Convenience function to run the pipeline with common settings.

    Args:
        video_id: YouTube video ID
        language: Primary language (if None, fetched from Supabase)
        model: Gemini model to use
        thinking_level: Thinking level for Gemini 3 (default: low)
        temperature: Generation temperature (default: 0.0 deterministic)
        max_segments: Limit segments for testing
        max_duration_sec: Max segment duration (hard limit)
        output_dir: Directory for output files
        validate: Whether to validate transcriptions

    Returns:
        PipelineResult with summary
    """
    config = PipelineConfig(
        video_id=video_id,
        language=language,
        model=model,
        thinking_level=thinking_level,
        temperature=temperature,
        max_segments=max_segments,
        max_segment_duration_sec=max_duration_sec,
        output_dir=output_dir,
        validate_transcriptions=validate
    )

    pipeline = TranscriptionPipeline(config)
    return pipeline.run()

def test_models(
    video_id: str,
    language: str,
    models: List[str],
    thinking_levels: List[str] = ["high"],
    max_segments: int = 3
) -> Dict[str, PipelineResult]:
    """
    Test multiple models on the same video for comparison.

    Args:
        video_id: Video ID to test
        language: Primary language
        models: List of model names to test
        thinking_levels: Thinking levels to try
        max_segments: Segments per test

    Returns:
        Dict mapping model_thinking to PipelineResult
    """
    results = {}

    for model in models:
        for thinking in thinking_levels:
            key = f"{model}_{thinking}"
            print(f"\n{'='*60}")
            print(f"Testing: {key}")
            print(f"{'='*60}")

            try:
                result = run_pipeline(
                    video_id=video_id,
                    language=language,
                    model=model,
                    thinking_level=thinking,
                    max_segments=max_segments,
                    output_dir=f"./transcriptions/test_{video_id}"
                )
                results[key] = result
            except Exception as e:
                print(f"ERROR testing {key}: {e}")

    return results

if __name__ == "__main__":
    import argparse

    parser = argparse.ArgumentParser(description="Transcription Pipeline")
    parser.add_argument("video_id", help="YouTube video ID")
    parser.add_argument("--language", "-l", help="Primary language (default: auto-detect)")
    parser.add_argument("--model", "-m", default="gemini-3-flash-preview",
                        help="Gemini model to use")
    parser.add_argument("--thinking", "-t", default="low",
                        choices=["minimal", "low", "medium", "high"],
                        help="Thinking level for Gemini 3 (default: low)")
    parser.add_argument("--temperature", type=float, default=0.0,
                        help="Generation temperature (default: 0.0 deterministic)")
    parser.add_argument("--no-validate", action="store_true",
                        help="Skip transcription validation")
    parser.add_argument("--max-segments", "-n", type=int,
                        help="Maximum segments to process")
    parser.add_argument("--max-duration", "-d", type=float, default=10.0,
                        help="Maximum segment duration in seconds")
    parser.add_argument("--output", "-o", default="./transcriptions",
                        help="Output directory")

    args = parser.parse_args()

    result = run_pipeline(
        video_id=args.video_id,
        language=args.language,
        model=args.model,
        thinking_level=args.thinking,
        temperature=args.temperature,
        max_segments=args.max_segments,
        max_duration_sec=args.max_duration,
        output_dir=args.output,
        validate=not args.no_validate
    )

    print(f"\n{'='*60}")
    print("Pipeline Summary")
    print(f"{'='*60}")
    print(f"Video ID: {result.video_id}")
    print(f"Language: {result.language}")
    print(f"Model: {result.model} (thinking: {result.thinking_level})")
    print(f"Segments processed: {result.segments_processed}")
    print(f"Chunks transcribed: {result.transcriptions_completed}")
    print(f"Total audio: {result.total_audio_duration_sec:.1f}s")
    print(f"Total time: {result.total_processing_time_sec:.1f}s")
    print(f"Output: {result.output_file}")
    if result.errors:
        print(f"Errors: {result.errors}")


Output:
Chunk ID: 0b98e1
Wall time: 0.0506 seconds
Process exited with code 0
Original token count: 1381
Output:
"""
Main transcription pipeline orchestrating the full workflow:
1. Download audio segments from R2
2. Fetch language info from Supabase
3. Process/split audio segments
4. Transcribe using Gemini
5. Output results

Full control over all parameters for testing and production use.
"""
import os
import json
import time
from typing import Optional, List, Dict
from dataclasses import dataclass, field
from datetime import datetime

# Import from src.backend package
from src.backend.config import DEFAULT_SETTINGS, GEMINI_MODELS, get_model_name
from src.backend.r2_storage import download_video_segments
from src.backend.supabase_client import get_video_language
from src.backend.audio_processor import AudioProcessor, AudioChunk, get_segment_stats
from src.backend.audio_polisher import AudioPolisher
from src.backend.gemini_transcriber import GeminiTranscriber, TranscriptionConfig
from src.backend.transcription_schema import TranscriptionResult
from src.romanization import derive_romanized_text

# Validation
from src.validators import validate_transcription, quick_validate, cleanup as cleanup_validator

@dataclass
class PipelineConfig:
    """
    Complete configuration for the transcription pipeline.
    All parameters are exposed for full control.
    """
    # === Video Settings ===
    video_id: str
    language: Optional[str] = None  # If None, fetched from Supabase
    default_language: str = "Telugu"  # Fallback if Supabase lookup fails

    # === Audio Processing Settings ===
    max_segment_duration_sec: float = 10.0  # Hard limit - segments longer than this are split
    min_segment_duration_sec: float = 2.0   # Skip segments shorter than this
    chunk_overlap_sec: float = 0.0          # Overlap when splitting (for word boundaries)
    split_boundary_padding_ms: float = 80.0 # Add a bit of silence context around split cuts

    # === Segment Selection Settings ===
    max_segments: Optional[int] = None      # Limit total segments processed (for testing)
    segment_start_index: int = 0            # Start from this segment index
    skip_short_segments: bool = True        # Skip segments < min_duration

    # === Transcription Settings ===
    model: str = "gemini-3-flash-preview"   # Gemini model to use
    thinking_level: Optional[str] = "low"   # minimal, low, medium, high (Gemini 3)
    temperature: float = 0.0                # 0 for deterministic (tested safe with strict prompt)

    # === Validation Settings ===
    validate_transcriptions: bool = True    # Validate native transcriptions
    validation_action: str = "retry"        # "flag" (mark status) or "retry" (re-transcribe with pro)
    retry_model: str = "gemini-3-pro-preview"  # Model for retry attempts
    retry_thinking_level: str = "low"        # Thinking level for retry
    min_snr_db: float = 5.0                 # SNR gate: skip segments below this (saves API credits)

    # === Output Settings ===
    output_dir: str = "./transcriptions"
    save_intermediate: bool = True          # Save results after each batch
    batch_size: int = 10                    # How many to process before saving

    # === Audio Polishing Settings ===
    polish_audio: bool = True               # Polish segment boundaries before transcription

    # === Storage Settings ===
    work_dir: str = field(default_factory=lambda: DEFAULT_SETTINGS["work_dir"])
    cleanup_after: bool = False             # Remove downloaded files after processing
    use_regex_fallback: bool = True         # Try regex if exact R2 match fails

@dataclass
class PipelineResult:
    """Results from a pipeline run."""
    video_id: str
    language: str
    model: str
    thinking_level: Optional[str]

    total_segments_found: int
    segments_processed: int
    chunks_created: int
    transcriptions_completed: int

    total_audio_duration_sec: float
    total_processing_time_sec: float

    output_file: str
    errors: List[str]

    started_at: str
    completed_at: str

@dataclass
class _BatchHealth:
    """Tracks validator health per batch. Catches silent degradation."""
    validated: int = 0
    ctc_success: int = 0       # CTC returned > 0
    mms_success: int = 0       # MMS returned > 0
    dual_success: int = 0      # Both returned > 0
    one_sided_accept: int = 0  # Accepted with only one validator
    accept: int = 0
    review: int = 0
    retry: int = 0
    reject: int = 0
    retry_improved: int = 0
    total_combined: float = 0.0

    # Alert thresholds
    CTC_FAIL_THRESHOLD = 0.20    # Warn if CTC fails on >20% of batch
    MMS_FAIL_THRESHOLD = 0.20    # Warn if MMS fails on >20% of batch
    REJECT_THRESHOLD = 0.30      # Warn if >30% of batch rejected
    ONE_SIDED_THRESHOLD = 0.40   # Warn if >40% accepted on single validator

    def record(self, validation):
        """Record one validation result."""
        self.validated += 1
        ctc_ok = validation.native_ctc_score > 0
        mms_ok = validation.roman_mms_score > 0
        if ctc_ok:
            self.ctc_success += 1
        if mms_ok:
            self.mms_success += 1
        if ctc_ok and mms_ok:
            self.dual_success += 1
        if validation.status == "accept" and not (ctc_ok and mms_ok):
            self.one_sided_accept += 1
        status_attr = validation.status  # accept/review/retry/reject
        if hasattr(self, status_attr):
            setattr(self, status_attr, getattr(self, status_attr) + 1)
        self.total_combined += validation.combined_score


**Applying patch to pipeline signature**


Input:
*** Begin Patch
*** Update File: pipeline.py
@@
 def run_pipeline(
     video_id: str,
     language: Optional[str] = None,
     model: str = "gemini-3-flash-preview",
     thinking_level: str = "low",
     temperature: float = 0.0,
     max_segments: Optional[int] = None,
     max_duration_sec: float = 10.0,
+    min_duration_sec: float = 2.0,
+    split_pad_ms: float = 80.0,
     output_dir: str = "./transcriptions",
     validate: bool = True
 ) -> PipelineResult:
@@
         temperature: Generation temperature (default: 0.0 deterministic)
         max_segments: Limit segments for testing
         max_duration_sec: Max segment duration (hard limit)
+        min_duration_sec: Skip segments shorter than this duration
+        split_pad_ms: Padding context to add around split boundaries
         output_dir: Directory for output files
         validate: Whether to validate transcriptions
@@
         max_segments=max_segments,
         max_segment_duration_sec=max_duration_sec,
+        min_segment_duration_sec=min_duration_sec,
+        split_boundary_padding_ms=split_pad_ms,
         output_dir=output_dir,
         validate_transcriptions=validate
     )
@@
     parser.add_argument("--max-duration", "-d", type=float, default=10.0,
                         help="Maximum segment duration in seconds")
+    parser.add_argument("--min-duration", type=float, default=2.0,
+                        help="Minimum segment duration in seconds (default: 2.0)")
+    parser.add_argument("--split-pad-ms", type=float, default=80.0,
+                        help="Padding (ms) added around split boundaries (default: 80)")
     parser.add_argument("--output", "-o", default="./transcriptions",
                         help="Output directory")
@@
         temperature=args.temperature,
         max_segments=args.max_segments,
         max_duration_sec=args.max_duration,
+        min_duration_sec=args.min_duration,
+        split_pad_ms=args.split_pad_ms,
         output_dir=args.output,
         validate=not args.no_validate
     )
*** End Patch


Output: {"output":"Success. Updated the following files:\nM pipeline.py\n","metadata":{"exit_code":0,"duration_seconds":0.0}}


**Planning audio processor patch**


sed -n '1,260p' src/backend/audio_processor.py


Output:
Chunk ID: 9072a3
Wall time: 0.0512 seconds
Process exited with code 0
Original token count: 2429
Output:
"""
Audio processing module for handling segment duration and chunking.

Splitting strategy: VAD-aware energy-based cutting.
When segments >max_duration_sec, finds the lowest-energy point in a window
around the target split time (8-13s) to avoid cutting mid-speech.
Sequential chunks track their original segment + cut points for later merging.
"""
import os
import math
import numpy as np
from pathlib import Path
from typing import List, Dict, Optional
from dataclasses import dataclass
from pydub import AudioSegment

@dataclass
class AudioChunk:
    """Represents an audio chunk ready for transcription."""
    file_path: str
    original_segment: str  # Original segment filename
    chunk_index: int       # Index within the segment (0 if not split)
    total_chunks: int      # Total chunks for this segment
    start_ms: int          # Start time in milliseconds within original
    end_ms: int            # End time in milliseconds within original
    duration_sec: float    # Duration in seconds

    @property
    def is_split(self) -> bool:
        return self.total_chunks > 1

class AudioProcessor:
    """Handles audio segment processing with duration controls."""

    def __init__(
        self,
        max_duration_sec: float = 10.0,
        min_duration_sec: float = 1.0,
        overlap_sec: float = 0.0,
        output_format: str = "flac"
    ):
        """
        Initialize the audio processor.

        Args:
            max_duration_sec: Maximum allowed segment duration (hard limit)
            min_duration_sec: Minimum segment duration to process
            overlap_sec: Overlap between chunks when splitting (helps with word boundaries)
            output_format: Output format for split chunks (flac, wav, mp3)
        """
        self.max_duration_sec = max_duration_sec
        self.min_duration_sec = min_duration_sec
        self.overlap_sec = overlap_sec
        self.output_format = output_format

    def get_audio_duration(self, file_path: str) -> float:
        """Get duration of an audio file in seconds."""
        audio = AudioSegment.from_file(file_path)
        return len(audio) / 1000.0

    def needs_splitting(self, file_path: str) -> bool:
        """Check if a file needs to be split based on duration."""
        duration = self.get_audio_duration(file_path)
        return duration > self.max_duration_sec

    def _find_energy_valley(self, samples: np.ndarray, sr: int,
                            target_ms: int, search_window_ms: int = 3000
                            ) -> int:
        """
        Find the lowest-energy point near target_ms for VAD-aware splitting.
        Searches +/- search_window_ms around target for a silence/low-energy valley.
        Returns the best cut point in milliseconds.
        """
        # RMS energy in 20ms windows
        frame_ms = 20
        frame_samples = int(sr * frame_ms / 1000)
        total_ms = int(len(samples) / sr * 1000)

        search_start = max(0, target_ms - search_window_ms)
        search_end = min(total_ms, target_ms + search_window_ms)

        best_energy = float('inf')
        best_ms = target_ms  # fallback to target if no valley found

        for t_ms in range(search_start, search_end, frame_ms):
            idx = int(t_ms * sr / 1000)
            frame = samples[idx:idx + frame_samples]
            if len(frame) < frame_samples // 2:
                continue
            rms = np.sqrt(np.mean(frame ** 2)) + 1e-10
            energy_db = 20 * np.log10(rms)
            if energy_db < best_energy:
                best_energy = energy_db
                best_ms = t_ms

        return best_ms

    def split_audio(
        self,
        file_path: str,
        output_dir: str,
        chunk_duration_sec: Optional[float] = None
    ) -> List[AudioChunk]:
        """
        Split audio using VAD-aware energy-based cutting.

        Instead of chopping at fixed intervals, finds the lowest-energy point
        in a window around the target split time to avoid cutting mid-speech.
        Target: 8-13s chunks. Searches +/- 3s around target for silence valleys.

        Sequential chunks track their original segment + cut points so they
        can be merged back later (transcripts appended, original audio retained).

        Args:
            file_path: Path to the audio file
            output_dir: Directory to save chunks
            chunk_duration_sec: Target chunk duration (defaults to max_duration_sec)

        Returns:
            List of AudioChunk objects with sequential chunk indices
        """
        if chunk_duration_sec is None:
            chunk_duration_sec = self.max_duration_sec

        audio = AudioSegment.from_file(file_path)
        total_duration_ms = len(audio)
        total_duration_sec = total_duration_ms / 1000.0

        os.makedirs(output_dir, exist_ok=True)
        original_name = Path(file_path).stem

        # If already short enough, return as single chunk
        if total_duration_sec <= self.max_duration_sec:
            return [AudioChunk(
                file_path=file_path,
                original_segment=Path(file_path).name,
                chunk_index=0,
                total_chunks=1,
                start_ms=0,
                end_ms=total_duration_ms,
                duration_sec=total_duration_sec
            )]

        # Convert to numpy for energy analysis
        samples = np.array(audio.get_array_of_samples(), dtype=np.float32)
        if audio.channels > 1:
            samples = samples.reshape(-1, audio.channels).mean(axis=1)
        samples = samples / (2**15)  # normalize int16 to float
        sr = audio.frame_rate

        # Find VAD-aware split points using energy valleys
        target_ms = int(chunk_duration_sec * 1000)
        split_points = [0]  # start of first chunk
        cursor = 0

        while cursor + target_ms < total_duration_ms:
            # Look for lowest energy near the target split point
            candidate = cursor + target_ms
            cut_ms = self._find_energy_valley(samples, sr, candidate)
            # Don't cut too short (min 5s) or too long (max 15s)
            chunk_len = cut_ms - cursor
            if chunk_len < 5000:
                cut_ms = cursor + 5000
            elif chunk_len > 15000:
                cut_ms = cursor + int(chunk_duration_sec * 1000)
            split_points.append(cut_ms)
            cursor = cut_ms

        split_points.append(total_duration_ms)  # end of last chunk

        # Build chunks from split points
        chunks = []
        num_chunks = len(split_points) - 1

        for i in range(num_chunks):
            start_ms = split_points[i]
            end_ms = split_points[i + 1]
            chunk_audio = audio[start_ms:end_ms]
            chunk_duration = len(chunk_audio) / 1000.0

            # Skip tiny trailing chunks
            if chunk_duration < self.min_duration_sec and i > 0:
                continue

            chunk_filename = f"{original_name}_chunk{i:03d}.{self.output_format}"
            chunk_path = os.path.join(output_dir, chunk_filename)
            chunk_audio.export(chunk_path, format=self.output_format)

            chunks.append(AudioChunk(
                file_path=chunk_path,
                original_segment=Path(file_path).name,
                chunk_index=i,
                total_chunks=num_chunks,
                start_ms=start_ms,
                end_ms=end_ms,
                duration_sec=chunk_duration
            ))

        return chunks

    def process_segments_directory(
        self,
        segments_dir: str,
        output_dir: Optional[str] = None,
        max_segments: Optional[int] = None,
        skip_short: bool = True
    ) -> List[AudioChunk]:
        """
        Process all segments in a directory, splitting as needed.

        Args:
            segments_dir: Directory containing audio segments
            output_dir: Directory for split chunks (default: segments_dir/chunks)
            max_segments: Maximum number of segments to process (for testing)
            skip_short: Skip segments shorter than min_duration_sec

        Returns:
            List of AudioChunk objects ready for transcription
        """
        if output_dir is None:
            output_dir = os.path.join(segments_dir, "chunks")

        # Find all audio files
        audio_extensions = {'.flac', '.wav', '.mp3', '.ogg', '.m4a'}
        segment_files = []

        for ext in audio_extensions:
            segment_files.extend(Path(segments_dir).glob(f"*{ext}"))

        # Sort by filename for consistent ordering
        segment_files = sorted(segment_files, key=lambda x: x.name)

        if max_segments:
            segment_files = segment_files[:max_segments]

        print(f"[Audio] Processing {len(segment_files)} segments...")

        all_chunks = []
        split_count = 0
        skipped_count = 0

        for segment_path in segment_files:
            duration = self.get_audio_duration(str(segment_path))

            # Skip very short segments
            if skip_short and duration < self.min_duration_sec:
                skipped_count += 1
                continue

            # Split if needed
            if duration > self.max_duration_sec:
                chunks = self.split_audio(str(segment_path), output_dir)
                split_count += 1
                print(f"[Audio] Split {segment_path.name}: {duration:.1f}s -> {len(chunks)} chunks")
            else:
                # Use original file
                chunks = [AudioChunk(
                    file_path=str(segment_path),


**Refining chunk splitting logic**


Input:
*** Begin Patch
*** Update File: src/backend/audio_processor.py
@@
 import os
-import math
 import numpy as np
 from pathlib import Path
 from typing import List, Dict, Optional
 from dataclasses import dataclass
 from pydub import AudioSegment
@@
     def __init__(
         self,
         max_duration_sec: float = 10.0,
-        min_duration_sec: float = 1.0,
+        min_duration_sec: float = 2.0,
         overlap_sec: float = 0.0,
+        boundary_padding_ms: float = 80.0,
         output_format: str = "flac"
     ):
@@
             max_duration_sec: Maximum allowed segment duration (hard limit)
             min_duration_sec: Minimum segment duration to process
             overlap_sec: Overlap between chunks when splitting (helps with word boundaries)
+            boundary_padding_ms: Context padding around split boundaries
             output_format: Output format for split chunks (flac, wav, mp3)
         """
         self.max_duration_sec = max_duration_sec
         self.min_duration_sec = min_duration_sec
         self.overlap_sec = overlap_sec
+        self.boundary_padding_ms = boundary_padding_ms
         self.output_format = output_format
@@
         while cursor + target_ms < total_duration_ms:
@@
             split_points.append(cut_ms)
             cursor = cut_ms

         split_points.append(total_duration_ms)  # end of last chunk
+
+        # Avoid tiny final chunks by merging tail into previous chunk.
+        min_chunk_ms = int(self.min_duration_sec * 1000)
+        if (
+            len(split_points) > 2
+            and (split_points[-1] - split_points[-2]) < min_chunk_ms
+        ):
+            split_points.pop(-2)

         # Build chunks from split points
         chunks = []
         num_chunks = len(split_points) - 1
+        overlap_ms = int(max(0.0, self.overlap_sec) * 1000)
+        pad_ms = int(max(0.0, self.boundary_padding_ms))

         for i in range(num_chunks):
-            start_ms = split_points[i]
-            end_ms = split_points[i + 1]
-            chunk_audio = audio[start_ms:end_ms]
+            raw_start_ms = split_points[i]
+            raw_end_ms = split_points[i + 1]
+            start_ms = max(0, raw_start_ms - overlap_ms - pad_ms)
+            end_ms = min(total_duration_ms, raw_end_ms + overlap_ms + pad_ms)
+            chunk_audio = audio[start_ms:end_ms]
             chunk_duration = len(chunk_audio) / 1000.0

             # Skip tiny trailing chunks
             if chunk_duration < self.min_duration_sec and i > 0:
                 continue
*** End Patch


Output: {"output":"Success. Updated the following files:\nM src/backend/audio_processor.py\n","metadata":{"exit_code":0,"duration_seconds":0.0}}


**Updating segment stats patch**


Input:
*** Begin Patch
*** Update File: src/backend/audio_processor.py
@@
     return {
         "total_files": len(durations),
         "total_duration_sec": sum(durations),
         "avg_duration_sec": sum(durations) / len(durations),
         "min_duration_sec": min(durations),
         "max_duration_sec": max(durations),
         "over_10s_count": sum(1 for d in durations if d > 10),
-        "under_1s_count": sum(1 for d in durations if d < 1)
+        "under_2s_count": sum(1 for d in durations if d < 2),
+        "under_1s_count": sum(1 for d in durations if d < 1),
     }
*** End Patch


Output: {"output":"Success. Updated the following files:\nM src/backend/audio_processor.py\n","metadata":{"exit_code":0,"duration_seconds":0.0}}


**Adjusting config defaults**


Input:
*** Begin Patch
*** Update File: src/backend/config.py
@@
 DEFAULT_SETTINGS = {
     "max_segment_duration_sec": 10.0,  # Hard limit for segment duration
-    "min_segment_duration_sec": 1.0,   # Minimum segment duration
+    "min_segment_duration_sec": 2.0,   # Minimum segment duration
+    "split_boundary_padding_ms": 80.0,
     "default_model": "gemini-3-flash-preview",
     "default_thinking_level": "high",
     "work_dir": "/tmp/maya3_transcribe",
*** End Patch


Output: {"output":"Success. Updated the following files:\nM src/backend/config.py\n","metadata":{"exit_code":0,"duration_seconds":0.0}}


**Adding under-threshold logging**


Input:
*** Begin Patch
*** Update File: pipeline.py
@@
         self._log(f"Found {stats.get('total_files', 0)} audio files, "
                   f"total duration: {stats.get('total_duration_sec', 0):.1f}s")
         if stats.get('over_10s_count', 0) > 0:
             self._log(f"  -> {stats['over_10s_count']} segments over 10s will be split")
+        if stats.get('under_2s_count', 0) > 0:
+            self._log(
+                f"  -> {stats['under_2s_count']} segments under 2s may be skipped",
+                "WARN",
+            )
*** End Patch


Output: {"output":"Success. Updated the following files:\nM pipeline.py\n","metadata":{"exit_code":0,"duration_seconds":0.0}}


**Refining pipeline and cleanup imports**


Input:
*** Begin Patch
*** Update File: src/validators/simple_validator.py
@@
-import re
 from dataclasses import dataclass
 from typing import List, Dict, Optional, Set
*** End Patch


Output: {"output":"Success. Updated the following files:\nM src/validators/simple_validator.py\n","metadata":{"exit_code":0,"duration_seconds":0.0}}


**Updating imports and requirements**


Input:
*** Begin Patch
*** Update File: requirements.txt
@@
 python-dotenv>=1.2.0
+Unidecode>=1.3.8
+indic-transliteration>=2.3.65
*** End Patch


Output: {"output":"Success. Updated the following files:\nM requirements.txt\n","metadata":{"exit_code":0,"duration_seconds":0.0}}


**Preparing minimal README and code cleanup**


Input:
*** Begin Patch
*** Update File: src/romanization.py
@@
 from __future__ import annotations

 import re
 import unicodedata
 from dataclasses import dataclass
-from typing import Optional, Tuple
+from typing import Optional
*** End Patch


Output: {"output":"Success. Updated the following files:\nM src/romanization.py\n","metadata":{"exit_code":0,"duration_seconds":0.0}}


**Updating validation function docs**


Input:
*** Begin Patch
*** Update File: src/validators/simple_validator.py
@@
     Validate a transcription using dual scoring (native CTC + romanized MMS).
@@
-    3. Romanized MMS forced alignment (language-agnostic, if romanized_text provided)
+    3. Romanized MMS forced alignment (language-agnostic, deterministic romanized text)
@@
-        romanized_text: Latin-script romanization (for MMS alignment)
+        romanized_text: Optional model romanization fallback
         check_audio: Whether to run audio alignment checks (slower)
+        min_duration_sec: Segments shorter than this cannot auto-accept
*** End Patch


Output: {"output":"Success. Updated the following files:\nM src/validators/simple_validator.py\n","metadata":{"exit_code":0,"duration_seconds":0.0}}


**Planning validation and logging patches**


Core edits are in place. I’m running a compile/sanity pass now to catch syntax or import issues, then I’ll fix any breakages and summarize exactly what changed.


**Running bytecode compilation**


Output:
Chunk ID: d60a4e
Wall time: 0.0512 seconds
Process exited with code 0
Original token count: 126
Output:
Compiling 'pipeline.py'...
Listing 'src'...
Listing 'src/backend'...
Compiling 'src/backend/audio_processor.py'...
Compiling 'src/backend/config.py'...
Compiling 'src/backend/gemini_transcriber.py'...
Compiling 'src/backend/transcription_schema.py'...
Compiling 'src/romanization.py'...
Listing 'src/validators'...
Compiling 'src/validators/__init__.py'...
Compiling 'src/validators/ctc_forced_aligner.py'...
Compiling 'src/validators/mms_aligner.py'...
Compiling 'src/validators/simple_validator.py'...


Summary: auto


**Running targeted tests**


python3 - <<'PY'
from src.romanization import derive_romanized_text, normalize_roman_for_mms

samples=[('నాకు కొన్ని యాడ్స్ గుర్తుంటాయి','te','naaku konni ads gurtuntaayi'),('ఇంకా','te','and'),('Hello and welcome','en','')]
for native,lang,fallback in samples:
    r=derive_romanized_text(native,lang,fallback)
    print(lang,native,'=>',r.text,'|',r.source)
print(normalize_roman_for_mms("yaa' / test -- 123"))
PY


Output:
Chunk ID: f2ca69
Wall time: 0.0514 seconds
Process exited with code 0
Original token count: 60
Output:
te నాకు కొన్ని యాడ్స్ గుర్తుంటాయి => naaku konni yaadds gurtunttaayi | unidecode
te ఇంకా => inkaa | unidecode
en Hello and welcome => hello and welcome | passthrough
yaa' test


**Validating transliteration fallback**


python3 - <<'PY'
from src.validators.simple_validator import validate_transcription

r=validate_transcription(audio_path='x.wav', transcription='నాకు కొన్ని యాడ్స్', language='te', check_audio=False)
print(r.status, r.romanization_source, r.combined_score)
print(r.to_dict().keys())
PY


Output:
Chunk ID: 10ff68
Wall time: 0.0514 seconds
Process exited with code 0
Original token count: 97
Output:
accept unidecode 0.0
dict_keys(['status', 'char_valid', 'invalid_chars', 'script_ratio', 'native_ctc_score', 'roman_mms_score', 'romanization_source', 'combined_score', 'alignment_score', 'low_confidence_words', 'low_confidence_ratio', 'audio_duration_sec', 'boundary_start_confidence', 'boundary_end_confidence', 'min_internal_word_confidence', 'internal_low_conf_ratio', 'reasons'])


**Reviewing pipeline validation and defaults**


git diff -- pipeline.py src/backend/audio_processor.py src/validators/simple_validator.py src/validators/mms_aligner.py src/backend/config.py src/romanization.py requirements.txt


Output:
Chunk ID: c592d5
Wall time: 0.0505 seconds
Process exited with code 0
Original token count: 335
Output:
 M pipeline.py
 M requirements.txt
 M src/backend/__pycache__/audio_processor.cpython-310.pyc
 M src/backend/__pycache__/config.cpython-310.pyc
 M src/backend/__pycache__/gemini_transcriber.cpython-310.pyc
 M src/backend/__pycache__/transcription_schema.cpython-310.pyc
 M src/backend/audio_processor.py
 M src/backend/config.py
 M src/backend/gemini_transcriber.py
 M src/backend/transcription_schema.py
 M src/validators/__init__.py
 M src/validators/__pycache__/__init__.cpython-310.pyc
 M src/validators/__pycache__/ctc_forced_aligner.cpython-310.pyc
 M src/validators/__pycache__/simple_validator.cpython-310.pyc
 M src/validators/ctc_forced_aligner.py
 M src/validators/simple_validator.py
?? SPEAKER_00_0000_0.03-2.61.flac
?? SPEAKER_00_0004_36.29-37.41.wav
?? SPEAKER_00_0038_529.16-531.25.wav
?? SPEAKER_00_0059_741.63-748.92.wav
?? SPEAKER_00_0076_1013.51-1020.24.wav
?? SPEAKER_00_0077_1026.28-1027.43.wav
?? SPEAKER_00_0082_1061.11-1065.06.wav
?? SPEAKER_00_0107_1343.96-1350.94.wav
?? SPEAKER_01_0422_5442.68-5452.49.flac
?? __pycache__/
?? analysis_results/
?? bin/
?? consistency_test/
?? golden_set/
?? models/
?? src/__pycache__/romanization.cpython-310.pyc
?? src/romanization.py
?? src/validators/__pycache__/mms_aligner.cpython-310.pyc
?? src/validators/mms_aligner.py
?? transcriptions/
?? validation_results/
?? venv/


Output:
Chunk ID: 268e14
Wall time: 0.0515 seconds
Process exited with code 0
Original token count: 12782
Output:
Total output lines: 1247

diff --git a/pipeline.py b/pipeline.py
index 18f6fb5..9a87742 100644
--- a/pipeline.py
+++ b/pipeline.py
@@ -23,6 +23,7 @@ from src.backend.audio_processor import AudioProcessor, AudioChunk, get_segment_
 from src.backend.audio_polisher import AudioPolisher
 from src.backend.gemini_transcriber import GeminiTranscriber, TranscriptionConfig
 from src.backend.transcription_schema import TranscriptionResult
+from src.romanization import derive_romanized_text

 # Validation
 from src.validators import validate_transcription, quick_validate, cleanup as cleanup_validator
@@ -41,8 +42,9 @@ class PipelineConfig:

     # === Audio Processing Settings ===
     max_segment_duration_sec: float = 10.0  # Hard limit - segments longer than this are split
-    min_segment_duration_sec: float = 1.0   # Skip segments shorter than this
+    min_segment_duration_sec: float = 2.0   # Skip segments shorter than this
     chunk_overlap_sec: float = 0.0          # Overlap when splitting (for word boundaries)
+    split_boundary_padding_ms: float = 80.0 # Add a bit of silence context around split cuts

     # === Segment Selection Settings ===
     max_segments: Optional[int] = None      # Limit total segments processed (for testing)
@@ -56,7 +58,10 @@ class PipelineConfig:

     # === Validation Settings ===
     validate_transcriptions: bool = True    # Validate native transcriptions
-    validation_action: str = "flag"         # "flag" (mark status) or "retry" (re-transcribe)
+    validation_action: str = "retry"        # "flag" (mark status) or "retry" (re-transcribe with pro)
+    retry_model: str = "gemini-3-pro-preview"  # Model for retry attempts
+    retry_thinking_level: str = "low"        # Thinking level for retry
+    min_snr_db: float = 5.0                 # SNR gate: skip segments below this (saves API credits)

     # === Output Settings ===
     output_dir: str = "./transcriptions"
@@ -95,6 +100,83 @@ class PipelineResult:
     completed_at: str

+@dataclass
+class _BatchHealth:
+    """Tracks validator health per batch. Catches silent degradation."""
+    validated: int = 0
+    ctc_success: int = 0       # CTC returned > 0
+    mms_success: int = 0       # MMS returned > 0
+    dual_success: int = 0      # Both returned > 0
+    one_sided_accept: int = 0  # Accepted with only one validator
+    accept: int = 0
+    review: int = 0
+    retry: int = 0
+    reject: int = 0
+    retry_improved: int = 0
+    total_combined: float = 0.0
+
+    # Alert thresholds
+    CTC_FAIL_THRESHOLD = 0.20    # Warn if CTC fails on >20% of batch
+    MMS_FAIL_THRESHOLD = 0.20    # Warn if MMS fails on >20% of batch
+    REJECT_THRESHOLD = 0.30      # Warn if >30% of batch rejected
+    ONE_SIDED_THRESHOLD = 0.40   # Warn if >40% accepted on single validator
+
+    def record(self, validation):
+        """Record one validation result."""
+        self.validated += 1
+        ctc_ok = validation.native_ctc_score > 0
+        mms_ok = validation.roman_mms_score > 0
+        if ctc_ok:
+            self.ctc_success += 1
+        if mms_ok:
+            self.mms_success += 1
+        if ctc_ok and mms_ok:
+            self.dual_success += 1
+        if validation.status == "accept" and not (ctc_ok and mms_ok):
+            self.one_sided_accept += 1
+        status_attr = validation.status  # accept/review/retry/reject
+        if hasattr(self, status_attr):
+            setattr(self, status_attr, getattr(self, status_attr) + 1)
+        self.total_combined += validation.combined_score
+
+    def report(self, log_fn):
+        """Log batch health summary. Warn on threshold breaches."""
+        n = max(self.validated, 1)
+        avg_s = self.total_combined / n
+        ctc_rate = self.ctc_success / n
+        mms_rate = self.mms_success / n
+        dual_rate = self.dual_success / n
+        rej_rate = self.reject / n
+        one_sided_rate = self.one_sided_accept / max(self.accept, 1)
+
+        log_fn(f"  Health: {self.validated} validated | "
+               f"accept={self.accept} review={self.review} "
+               f"retry={self.retry} reject={self.reject} | "
+               f"avg_S={avg_s:.3f}")
+        log_fn(f"  Validators: CTC={ctc_rate:.0%} MMS={mms_rate:.0%} "
+               f"dual={dual_rate:.0%} | "
+               f"one-sided accepts={self.one_sided_accept}")
+        if self.retry_improved > 0:
+            log_fn(f"  Retries improved: {self.retry_improved}")
+
+        # Alert on threshold breaches
+        alerts = []
+        if ctc_rate < (1 - self.CTC_FAIL_THRESHOLD):
+            alerts.append(f"CTC failing on {1-ctc_rate:.0%} of segments")
+        if mms_rate < (1 - self.MMS_FAIL_THRESHOLD):
+            alerts.append(f"MMS failing on {1-mms_rate:.0%} of segments")
+        if rej_rate > self.REJECT_THRESHOLD:
+            alerts.append(f"Reject rate {rej_rate:.0%} exceeds {self.REJECT_THRESHOLD:.0%}")
+        if self.accept > 0 and one_sided_rate > self.ONE_SIDED_THRESHOLD:
+            alerts.append(
+                f"One-sided accept rate {one_sided_rate:.0%} "
+                f"exceeds {self.ONE_SIDED_THRESHOLD:.0%}"
+            )
+        for alert in alerts:
+            log_fn(f"  ALERT: {alert}", "WARN")
+        return alerts
+
+
 class TranscriptionPipeline:
     """
     Main pipeline orchestrating the full transcription workflow.
@@ -106,6 +188,7 @@ class TranscriptionPipeline:
         self.results: List[TranscriptionResult] = []
         self.errors: List[str] = []
         self.start_time: Optional[float] = None
+        self._health_alerts: List[str] = []  # Accumulated health alerts

     def _log(self, message: str, level: str = "INFO"):
         """Log a message with timestamp."""
@@ -118,7 +201,7 @@ class TranscriptionPipeline:
             "Telugu": "te", "Hindi": "hi", "Tamil": "ta",
             "Kannada": "kn", "Malayalam": "ml", "Bengali": "bn",
             "Marathi": "mr", "Gujarati": "gu", "Punjabi": "pa",
-            "Odia": "or",
+            "Odia": "or", "Assamese": "as", "English": "en",
         }
         return lang_map.get(language, "te")

@@ -154,6 +237,11 @@ class TranscriptionPipeline:
                   f"total duration: {stats.get('total_duration_sec', 0):.1f}s")
         if stats.get('over_10s_count', 0) > 0:
             self._log(f"  -> {stats['over_10s_count']} segments over 10s will be split")
+        if stats.get('under_2s_count', 0) > 0:
+            self._log(
+                f"  -> {stats['under_2s_count']} segments under 2s may be skipped",
+                "WARN",
+            )

         # === Step 2: Get Language ===
         self._log("Step 2: Fetching language info...")
@@ -172,7 +260,8 @@ class TranscriptionPipeline:
         processor = AudioProcessor(
             max_duration_sec=self.config.max_segment_duration_sec,
             min_duration_sec=self.config.min_segment_duration_sec,
-            overlap_sec=self.config.chunk_overlap_sec
+            overlap_sec=self.config.chunk_overlap_sec,
+            boundary_padding_ms=self.config.split_boundary_padding_ms,
         )

         chunks = processor.process_segments_directory(
@@ -187,19 +276,34 @@ class TranscriptionPipeline:
             self._log(f"Starting from index {self.config.segment_start_index}, "
                       f"{len(chunks)} chunks remaining")

-        # === Step 3.5: Polish Audio Boundaries ===
+        # === Step 3.5: Polish Audio + SNR Gate ===
         if self.config.polish_audio:
             self._log("Step 3.5: Polishing audio boundaries...")
             polisher = AudioPolisher()
             polished_dir = os.path.join(segments_dir, "polished")
             polished_count = 0
+            snr_skipped = 0
+            surviving_chunks = []
             for chunk in chunks:
                 result = polisher.polish(chunk.file_path, output_dir=polished_dir)
                 if result.was_modified:
                     chunk.file_path = result.output_path
                     polished_count += 1
+                # SNR gate: skip segments with very low SNR (likely noise/music)
+                if result.snr_db < self.config.min_snr_db:
+                    snr_skipped += 1
+                    continue
+                # Duration gate: polishing might shrink segments below minimum
+                if result.polished_duration_ms / 1000.0 < self.config.min_segment_duration_sec:
+                    snr_skipped += 1  # reuse counter, both are pre-tx filters
+                    continue
+                surviving_chunks.append(chunk)
             self._log(f"Polished {polished_count}/{len(chunks)} chunks "
                       f"({len(chunks) - polished_count} already clean)")
+            if snr_skipped > 0:
+                self._log(f"SNR gate: skipped {snr_skipped} chunks "
+                          f"(SNR < {self.config.min_snr_db}dB)")
+            chunks = surviving_chunks

         # === Step 4: Transcribe ===
         self._log(f"Step 4: Transcribing {len(chunks)} chunks...")
@@ -228,22 +332,114 @@ class TranscriptionPipeline:
                 )
             )

-            # === Validate transcriptions ===
+            # === Validate transcriptions (dual scoring: native CTC + romanized MMS) ===
             if self.config.validate_transcriptions:
                 lang_code = self._get_language_code(language)
+                batch_health = _BatchHealth()
                 for i, result in enumerate(batch_results):
                     if result.native:
+                        romanized_result = derive_romanized_text(
+                            native_text=result.native,
+                            language=lang_code,
+                            fallback_romanized=result.transcription.romanized or "",
+                        )
+                        # Make romanized deterministic across runs.
+                        result.transcription.romanized = romanized_result.text
                         validation = validate_transcription(
                             batch[i].file_path,
                             result.native,
-                            language=lang_code
+                            romanized_text=romanized_result.text,
+                            language=lang_code,
+                            min_duration_sec=self.config.min_segment_duration_sec,
                         )
                         result.validation_status = validation.status
-                        result.validation_score = validation.alignment_score
-                        if validation.status == "reject":
-                            self._log(f"  ⚠ Validation failed for {result.segment_id}: "
-                                     f"{validation.reasons}", "WARN")
-
+                        result.validation_score = validation.combined_score
+
+                        # Retry with pro model if below accept threshold
+                        if (validation.status in ("retry", "reject")
+                                and self.config.validation_action == "retry"):
+                            self._log(
+                                f"  Retrying {result.segment_id} with "
+                                f"{self.config.retry_model} "
+                                f"(S={validation.combined_score:.2f})", "WARN"
+                            )
+                            retry_config = TranscriptionConfig(
+                                model=self.config.retry_model,
+                                thinking_level=self.config.retry_thinking_level,
+                                temperature=self.config.temperature,
+                                language=language,
+                            )
+                            retry_raw = transcriber.transcribe_audio(
+                                batch[i].file_path, retry_config
+                            )
+                            if not retry_raw.get("error"):
+                                # Re-validate the retry result
+                                from src.backend.transcription_schema import (
+                                    TranscriptionOutput, SpeakerMeta
+                                )
+                                speaker_data = retry_raw.get('speaker')
+                                speaker = (SpeakerMeta(**speaker_data)
+                                           if isinstance(speaker_data, dict)
+                                           else None)
+                                retry_output = TranscriptionOutput(
+                                    transcription=retry_raw.get('transcription', ''),
+                                    code_switch=retry_raw.get('code_switch', ''),
+                                    romanized=retry_raw.get('romanized', ''),
+                                    tagged=retry_raw.get('tagged', ''),
+                                    speaker=speaker,
+                                    detected_language=retry_raw.get('detected_language', ''),
+                                )
+                                retry_romanized = derive_romanized_text(
+                                    native_text=retry_output.transcription,
+                                    language=lang_code,
+                                    fallback_romanized=retry_output.romanized or "",
+                                )
+                                retry_output.romanized = retry_romanized.text
+                                retry_val = validate_transcription(
+                                    batch[i].file_path,
+                                    retry_output.transcription,
+                                    romanized_text=retry_romanized.text,
+                                    language=lang_code,
+                                    min_duration_sec=self.config.min_segment_duration_sec,
+                                )
+                                # Keep the better result
+                                if retry_val.combined_score > validation.combined_score:
+                                    result.transcription = retry_output
+                                    result.model_used = retry_config.model
+                                    result.thinking_level = retry_config.thinking_level
+                                    validation = retry_val
+                                    self._log(
+                                        f"  Retry improved: "
+                                        f"S={retry_val.combined_score:.2f} "
+                                        f"({retry_val.status})"
+                                    )
+                                    batch_health.retry_improved += 1
+                                else:
+                                    self._log(
+                                        f"  Retry did not improve: "
+                                        f"S={retry_val.combined_score:.2f}"
+                                    )
+                            # Update final validation state
+                            result.validation_status = validation.status
+                            result.validation_score = validation.combined_score
+
+                        if validation.status in ("reject", "retry"):
+                            self._log(
+                                f"  Validation: {result.segment_id} -> "
+                                f"{validation.status} "
+                                f"(S={validation.combined_score:.2f}, "
+                                f"CTC={validation.native_ctc_score:.2f}, "
+                                f"MMS={validation.roman_mms_score:.2f})",
+                                "WARN"
+                            )
+
+                        # Record for batch health monitoring
+                        batch_health.record(validation)
+
+                # Batch health report + alerts
+                alerts = batch_health.report(self._log)
+                self._health_alerts.extend(alerts)
+
             self.results.extend(batch_results)

             # Save intermediate results
@@ -312,6 +508,7 @@ class TranscriptionPipeline:
             "config": {
                 "max_segment_duration_sec": self.config.max_segment_duration_sec,
                 "min_segment_duration_sec": self.config.min_segment_duration_sec,
+                "split_boundary_padding_ms": self.config.split_boundary_padding_ms,
                 "max_segments": self.config.max_segments,
             },
             "results_count": len(self.results),
@@ -333,6 +530,8 @@ def run_pipeline(
     temperature: float = 0.0,
     max_segments: Optional[int] = None,
     max_duration_sec: float = 10.0,
+    min_duration_sec: float = 2.0,
+    split_pad_ms: float = 80.0,
     output_dir: str = "./transcriptions",
     validate: bool = True
 ) -> PipelineResult:
@@ -347,6 +546,8 @@ def run_pipeline(
         temperature: Generation temperature (default: 0.0 deterministic)
         max_segments: Limit segments for testing
         max_duration_sec: Max segment duration (hard limit)
+        min_duration_sec: Skip segments shorter than this duration
+        split_pad_ms: Padding context to add around split boundaries
         output_dir: Directory for output files
         validate: Whether to validate transcriptions

@@ -361,6 +562,8 @@ def run_pipeline(
         temperature=temperature,
         max_segments=max_segments,
         max_segment_duration_sec=max_duration_sec,
+        min_segment_duration_sec=min_duration_sec,
+        split_boundary_padding_ms=split_pad_ms,
         output_dir=output_dir,
         validate_transcriptions=validate
     )
@@ -433,6 +636,10 @@ if __name__ == "__main__":
                         help="Maximum segments to process")
     parser.add_argument("--max-duration", "-d", type=float, default=10.0,
                         help="Maximum segment duration in seconds")
+    parser.add_argument("--min-duration", type=float, default=2.0,
+                        help="Minimum segment duration in seconds (default: 2.0)")
+    parser.add_argument("--split-pad-ms", type=float, default=80.0,
+                        help="Padding (ms) added around split boundaries (default: 80)")
     parser.add_argument("--output", "-o", default="./transcriptions",
                         help="Output directory")

@@ -446,6 +653,8 @@ if __name__ == "__main__":
         temperature=args.temperature,
         max_segments=args.max_segments,
         max_duration_sec=args.max_duration,
+        min_duration_sec=args.min_duration,
+        split_pad_ms=args.split_pad_ms,
         output_dir=args.output,
         validate=not args.no_validate
     )
diff --git a/requirements.txt b/requirements.txt
index 45324f3..b1efb05 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -4,3 +4,5 @@ pydantic>=2.12.0
 supabase>=2.27.0
 pydub>=0.25.0
 python-dotenv>=1.2.0
+Unidecode>=1.3.8
+indic-transliteration>=2.3.65
diff --git a/src/backend/audio_processor.py b/src/backend/audio_processor.py
index 744c79d..e5b14a2 100644
--- a/src/backend/audio_processor.py
+++ b/src/backend/audio_processor.py
@@ -1,9 +1,13 @@
 """
 Audio processing module for handling segment duration and chunking.
-Provides controls for splitting long segments to meet the 10s hard limit.
+
+Splitting strategy: VAD-aware energy-based cutting.
+When segments >max_duration_sec, finds the lowest-energy point in a window
+around the target split time (8-13s) to avoid cutting mid-speech.
+Sequential chunks track their original segment + cut points for later merging.
 """
 import os
-import math
+import numpy as np
 from pathlib import Path
 from typing import List, Dict, Optional
 from dataclasses import dataclass
@@ -32,8 +36,9 @@ class AudioProcessor:
     def __init__(
         self,
         max_duration_sec: float = 10.0,
-        min_duration_sec: float = 1.0,
+        min_duration_sec: float = 2.0,
         overlap_sec: float = 0.0,
+        boundary_padding_ms: float = 80.0,
         output_format: str = "flac"
     ):
         """
@@ -43,11 +48,13 @@ class AudioProcessor:
             max_duration_sec: Maximum allowed segment duration (hard limit)
             min_duration_sec: Minimum segment duration to process
             overlap_sec: Overlap between chunks when splitting (helps with word boundaries)
+            boundary_padding_ms: Context padding around split boundaries
             output_format: Output format for split chunks (flac, wav, mp3)
         """
         self.max_duration_sec = max_duration_sec
         self.min_duration_sec = min_duration_sec
         self.overlap_sec = overlap_sec
+        self.boundary_padding_ms = boundary_padding_ms
         self.output_format = output_format

     def get_audio_duration(self, file_path: str) -> float:
@@ -60,6 +67,38 @@ class AudioProcessor:
         duration = self.get_audio_duration(file_path)
         return duration > self.max_duration_sec

+    def _find_energy_valley(self, samples: np.ndarray, sr: int,
+                            target_ms: int, search_window_ms: int = 3000
+                            ) -> int:
+        """
+        Find the lowest-energy point near target_ms for VAD-aware splitting.
+        Searches +/- search_window_ms around target for a silence/low-energy valley.
+        Returns the best cut point in milliseconds.
+        """
+        # RMS energy in 20ms windows
+        frame_ms = 20
+        frame_samples = int(sr * frame_ms / 1000)
+        total_ms = int(len(samples) / sr * 1000)
+
+        search_start = max(0, target_ms - search_window_ms)
+        search_end = min(total_ms, target_ms + search_window_ms)
+
+        best_energy = float('inf')
+        best_ms = target_ms  # fallback to target if no valley found
+
+        for t_ms in range(search_start, search_end, frame_ms):
+            idx = int(t_ms * sr / 1000)
+            frame = samples[idx:idx + frame_samples]
+            if len(frame) < frame_samples // 2:
+                continue
+            rms = np.sqrt(np.mean(frame ** 2)) + 1e-10
+            energy_db = 20 * np.log10(rms)
+            if energy_db < best_energy:
+                best_energy = energy_db
+                best_ms = t_ms
+
+        return best_ms
+
     def split_audio(
         self,
         file_path: str,
@@ -67,26 +106,33 @@ class AudioProcessor:
         chunk_duration_sec: Optional[float] = None
     ) -> List[AudioChunk]:
         """
-        Split an audio file into chunks of specified duration.
-
+        Split audio using VAD-aware energy-based cutting.
+
+        Instead of chopping at fixed intervals, finds the lowest-energy point
+        in a window around the target split time to avoid cutting mid-speech.
+        Target: 8-13s chunks. Searches +/- 3s around target for silence valleys.
+
+        Sequential chunks track their original segment + cut points so they
+        can be merged back later (transcripts appended, original audio retained).
+
         Args:
             file_path: Path to the audio file
             output_dir: Directory to save chunks
             chunk_duration_sec: Target chunk duration (defaults to max_duration_sec)
-
+
         Returns:
-            List of AudioChunk objects
+            List of AudioChunk objects with sequential chunk indices
         """
         if chunk_duration_sec is None:
             chunk_duration_sec = self.max_duration_sec
-
+
         audio = AudioSegment.from_file(file_path)
         total_duration_ms = len(audio)
         total_duration_sec = total_duration_ms / 1000.0
-
+
         os.makedirs(output_dir, exist_ok=True)
         original_name = Path(file_path).stem
-
+
         # If already short enough, return as single chunk
         if total_duration_sec <= self.max_duration_sec:
             return [AudioChunk(
@@ -98,33 +144,64…782 tokens truncated…th = os.path.join(output_dir, chunk_filename)
             chunk_audio.export(chunk_path, format=self.output_format)
-
+
             chunks.append(AudioChunk(
                 file_path=chunk_path,
                 original_segment=Path(file_path).name,
@@ -134,7 +211,7 @@ class AudioProcessor:
                 end_ms=end_ms,
                 duration_sec=chunk_duration
             ))
-
+
         return chunks

     def process_segments_directory(
@@ -243,7 +320,8 @@ def get_segment_stats(segments_dir: str) -> Dict:
         "min_duration_sec": min(durations),
         "max_duration_sec": max(durations),
         "over_10s_count": sum(1 for d in durations if d > 10),
-        "under_1s_count": sum(1 for d in durations if d < 1)
+        "under_2s_count": sum(1 for d in durations if d < 2),
+        "under_1s_count": sum(1 for d in durations if d < 1),
     }

diff --git a/src/backend/config.py b/src/backend/config.py
index 2115dae..0c246a3 100644
--- a/src/backend/config.py
+++ b/src/backend/config.py
@@ -74,7 +74,8 @@ MODEL_ALIASES = {
 # === Default Pipeline Settings ===
 DEFAULT_SETTINGS = {
     "max_segment_duration_sec": 10.0,  # Hard limit for segment duration
-    "min_segment_duration_sec": 1.0,   # Minimum segment duration
+    "min_segment_duration_sec": 2.0,   # Minimum segment duration
+    "split_boundary_padding_ms": 80.0,
     "default_model": "gemini-3-flash-preview",
     "default_thinking_level": "high",
     "work_dir": "/tmp/maya3_transcribe",
diff --git a/src/validators/simple_validator.py b/src/validators/simple_validator.py
index 4e12abf..721fef4 100644
--- a/src/validators/simple_validator.py
+++ b/src/validators/simple_validator.py
@@ -2,37 +2,58 @@
 Simple Transcription Validator
 ==============================

-Two checks:
+Three checks:
 1. Character validation - catch garbage/alien characters
-2. Audio-text match - verify words match what's spoken
+2. Native CTC alignment - verify native script matches audio
+3. Romanized MMS alignment - verify deterministic romanized text matches audio
+
+Combined scoring: S = 0.45*N + 0.55*R - 0.10*abs(N-R)
+  N = native CTC score, R = romanized MMS score
+  Accept if S >= 0.78 and min(N,R) >= 0.55
+  Forced review if abs(N-R) > 0.20 (high validator disagreement)

 Usage:
     from src.validators.simple_validator import validate_transcription

-    result = validate_transcription("audio.flac", "transcription", language="te")
-    print(result['status'])  # accept / review / reject
+    result = validate_transcription(
+        "audio.flac", "నాకు కొన్ని యాడ్స్",
+        romanized_text="naaku konni ads", language="te"
+    )
+    print(result.status)          # accept / review / retry / reject
+    print(result.combined_score)  # 0-1
 """
-import re
 from dataclasses import dataclass
 from typing import List, Dict, Optional, Set

+from src.romanization import (
+    derive_romanized_text,
+    normalize_roman_for_mms,
+    strip_placeholders,
+)

-# Unicode ranges for Indic scripts
+
+# Unicode ranges for Indic scripts (all 12 supported languages)
 SCRIPT_RANGES = {
     "te": {  # Telugu
         "name": "Telugu",
-        "ranges": [(0x0C00, 0x0C7F)],  # Telugu block
+        "ranges": [(0x0C00, 0x0C7F)],
         "allow_ascii_punct": True,
         "allow_digits": True,
     },
     "hi": {  # Hindi (Devanagari)
         "name": "Hindi",
-        "ranges": [(0x0900, 0x097F)],  # Devanagari block
+        "ranges": [(0x0900, 0x097F)],
+        "allow_ascii_punct": True,
+        "allow_digits": True,
+    },
+    "mr": {  # Marathi (Devanagari, same block as Hindi)
+        "name": "Marathi",
+        "ranges": [(0x0900, 0x097F)],
         "allow_ascii_punct": True,
         "allow_digits": True,
     },
     "ta": {  # Tamil
-        "name": "Tamil",
+        "name": "Tamil",
         "ranges": [(0x0B80, 0x0BFF)],
         "allow_ascii_punct": True,
         "allow_digits": True,
@@ -55,15 +76,58 @@ SCRIPT_RANGES = {
         "allow_ascii_punct": True,
         "allow_digits": True,
     },
+    "as": {  # Assamese (same Unicode block as Bengali, slight char differences)
+        "name": "Assamese",
+        "ranges": [(0x0980, 0x09FF)],
+        "allow_ascii_punct": True,
+        "allow_digits": True,
+    },
+    "gu": {  # Gujarati
+        "name": "Gujarati",
+        "ranges": [(0x0A80, 0x0AFF)],
+        "allow_ascii_punct": True,
+        "allow_digits": True,
+    },
+    "pa": {  # Punjabi (Gurmukhi)
+        "name": "Punjabi",
+        "ranges": [(0x0A00, 0x0A7F)],
+        "allow_ascii_punct": True,
+        "allow_digits": True,
+    },
+    "or": {  # Odia
+        "name": "Odia",
+        "ranges": [(0x0B00, 0x0B7F)],
+        "allow_ascii_punct": True,
+        "allow_digits": True,
+    },
+    "en": {  # English (Latin)
+        "name": "English",
+        "ranges": [(0x0041, 0x005A), (0x0061, 0x007A)],  # A-Z, a-z
+        "allow_ascii_punct": True,
+        "allow_digits": True,
+    },
 }

 # Common allowed characters across all languages
-ALLOWED_COMMON = set(" \t\n.,!?;:'\"()-–—0123456789")
+ALLOWED_COMMON = set(" \t\n.,!?;:'\"()-–—[]0123456789")

 # English characters (for code-mixed text)
 ENGLISH_CHARS = set("abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ")

+# Validation thresholds (strict defaults)
+ACCEPT_THRESHOLD = 0.78
+REVIEW_THRESHOLD = 0.68
+RETRY_THRESHOLD = 0.55
+MIN_ACCEPT_VALIDATOR_SCORE = 0.55
+MAX_ACCEPT_DISAGREEMENT = 0.20
+
+# Word-level quality gates from native CTC alignments
+MIN_INTERNAL_WORD_CONFIDENCE = 0.55
+MAX_INTERNAL_LOW_CONF_RATIO = 0.20
+BOUNDARY_LOW_CONFIDENCE = 0.35
+
+
 def get_valid_chars(language: str, allow_english: bool = True) -> Set[int]:
     """Get set of valid Unicode codepoints for a language."""
     valid = set()
@@ -103,6 +167,7 @@ def check_characters(
             "script_ratio": 0.8,  # ratio of native script chars
         }
     """
+    text = strip_placeholders(text or "")
     if not text or not text.strip():
         return {"valid": False, "invalid_chars": [], "script_ratio": 0, "reason": "empty"}

@@ -115,13 +180,14 @@ def check_characters(

     for i, char in enumerate(text):
         cp = ord(char)
-
+        if char.isalpha():
+            total_alpha += 1
+
         # Check if in native script
         is_native = any(start <= cp <= end for start, end in lang_config["ranges"])
         if is_native:
             script_count += 1
-            total_alpha += 1
-
+
         # Check if invalid (includes English if not allowed)
         elif cp not in valid_chars:
             invalid_chars.append({
@@ -129,8 +195,6 @@ def check_characters(
                 "codepoint": cp,
                 "position": i
             })
-            if char in ENGLISH_CHARS:
-                total_alpha += 1

     return {
         "valid": len(invalid_chars) == 0,
@@ -142,46 +206,113 @@ def check_characters(

 @dataclass
 class ValidationResult:
-    """Result of transcription validation."""
-    status: str  # "accept", "review", "reject"
-
+    """Result of transcription validation with dual scoring."""
+    status: str  # "accept", "review", "retry", "reject"
+
     # Character check
     char_valid: bool = True
     invalid_chars: List[Dict] = None
     script_ratio: float = 0.0
-
-    # Audio match check
-    alignment_score: float = 0.0
+
+    # Native CTC alignment (language-specific wav2vec2)
+    alignment_score: float = 0.0       # kept for backward compat
+    native_ctc_score: float = 0.0
     low_confidence_words: List[str] = None
     low_confidence_ratio: float = 0.0
-
+
+    # Romanized MMS alignment (language-agnostic torchaudio MMS_FA)
+    roman_mms_score: float = 0.0
+    romanization_source: str = ""
+
+    # Combined weighted score: S = 0.45*N + 0.55*R - 0.10*abs(N-R)
+    combined_score: float = 0.0
+    audio_duration_sec: float = 0.0
+
+    # Word-level quality gates
+    boundary_start_confidence: float = 0.0
+    boundary_end_confidence: float = 0.0
+    min_internal_word_confidence: float = 0.0
+    internal_low_conf_ratio: float = 0.0
+
     # Reasons
     reasons: List[str] = None
-
+
     def to_dict(self) -> Dict:
         return {
             "status": self.status,
             "char_valid": self.char_valid,
             "invalid_chars": self.invalid_chars or [],
             "script_ratio": round(self.script_ratio, 3),
+            "native_ctc_score": round(self.native_ctc_score, 4),
+            "roman_mms_score": round(self.roman_mms_score, 4),
+            "romanization_source": self.romanization_source,
+            "combined_score": round(self.combined_score, 4),
             "alignment_score": round(self.alignment_score, 4),
             "low_confidence_words": self.low_confidence_words or [],
             "low_confidence_ratio": round(self.low_confidence_ratio, 3),
+            "audio_duration_sec": round(self.audio_duration_sec, 3),
+            "boundary_start_confidence": round(self.boundary_start_confidence, 4),
+            "boundary_end_confidence": round(self.boundary_end_confidence, 4),
+            "min_internal_word_confidence": round(self.min_internal_word_confidence, 4),
+            "internal_low_conf_ratio": round(self.internal_low_conf_ratio, 4),
             "reasons": self.reasons or []
         }

-# Global CTC aligner (lazy loaded)
+# Global aligners (lazy loaded)
 _ctc_aligner = None
+_mms_aligner = None

 def _get_ctc_aligner(language: str):
-    """Lazy load CTC aligner."""
+    """Lazy load native CTC aligner."""
     global _ctc_aligner
     if _ctc_aligner is None:
         from .ctc_forced_aligner import CTCForcedAligner
         _ctc_aligner = CTCForcedAligner(language=language)
     return _ctc_aligner

+def _get_mms_aligner():
+    """Lazy load romanized MMS aligner."""
+    global _mms_aligner
+    if _mms_aligner is None:
+        from .mms_aligner import MMSAligner
+        _mms_aligner = MMSAligner()
+    return _mms_aligner
+
+
+def compute_combined_score(
+    native_ctc: float,
+    roman_mms: float,
+) -> tuple:
+    """
+    Compute weighted validation score with disagreement penalty.
+
+    Formula: S = 0.45*N + 0.55*R - 0.10*abs(N-R)
+    MMS weighted higher (0.55): more stable on code-mixed Indic audio.
+    Disagreement penalty: flags when validators diverge (suspicious).
+
+    Returns:
+        (combined_score, verdict)
+        verdict: "accept" / "review" / "retry" / "reject"
+    """
+    N, R = native_ctc, roman_mms
+    S = 0.45 * N + 0.55 * R - 0.10 * abs(N - R)
+
+    if S >= ACCEPT_THRESHOLD and min(N, R) >= MIN_ACCEPT_VALIDATOR_SCORE:
+        if abs(N - R) > MAX_ACCEPT_DISAGREEMENT:
+            # High disagreement — flag even though score passes
+            verdict = "review"
+        else:
+            verdict = "accept"
+    elif S >= REVIEW_THRESHOLD:
+        verdict = "review"
+    elif S >= RETRY_THRESHOLD:
+        verdict = "retry"
+    else:
+        verdict = "reject"
+
+    return round(S, 4), verdict
+

 def _strip_punctuation(text: str) -> str:
     """Strip punctuation for CTC alignment. Keeps native script + spaces."""
@@ -190,41 +321,95 @@ def _strip_punctuation(text: str) -> str:
     return re.sub(r'[।,.!?;:\'"()\-–—]', '', text).strip()

+def _compute_word_quality_metrics(word_alignments: List) -> Dict[str, float]:
+    """
+    Compute boundary/internal confidence metrics from word-level CTC alignment.
+    """
+    confidences = [
+        float(getattr(wa, "confidence", 0.0))
+        for wa in (word_alignments or [])
+        if getattr(wa, "word", "").strip()
+    ]
+    if not confidences:
+        return {
+            "boundary_start": 0.0,
+            "boundary_end": 0.0,
+            "min_internal": 0.0,
+            "internal_low_ratio": 1.0,
+            "has_metrics": False,
+        }
+
+    if len(confidences) >= 3:
+        internal = confidences[1:-1]
+        boundary_start = confidences[0]
+        boundary_end = confidences[-1]
+    else:
+        internal = confidences
+        boundary_start = confidences[0]
+        boundary_end = confidences[-1]
+
+    if internal:
+        min_internal = min(internal)
+        low_internal = sum(1 for c in internal if c < 0.60)
+        internal_low_ratio = low_internal / len(internal)
+    else:
+        min_internal = 0.0
+        internal_low_ratio = 1.0
+
+    return {
+        "boundary_start": boundary_start,
+        "boundary_end": boundary_end,
+        "min_internal": min_internal,
+        "internal_low_ratio": internal_low_ratio,
+        "has_metrics": True,
+    }
+
+
 def validate_transcription(
     audio_path: str,
     transcription: str,
     language: str = "te",
+    romanized_text: str = "",
     check_audio: bool = True,
-    low_conf_threshold: float = 0.3,
-    low_conf_ratio_threshold: float = 0.8,  # Raised from 0.5 - code-mixed is normal
-    min_alignment_score: float = 0.5  # Below this = likely wrong transcription
+    min_duration_sec: float = 2.0,
 ) -> ValidationResult:
     """
-    Validate a transcription (native script with punctuation).
+    Validate a transcription using dual scoring (native CTC + romanized MMS).
+
+    Runs:
+    1. Character validation (instant) on native script
+    2. Native CTC forced alignment (language-specific wav2vec2)
+    3. Romanized MMS forced alignment (language-agnostic, deterministic romanized text)
+    4. Weighted combined score: S = 0.45*N + 0.55*R - 0.10*abs(N-R)

-    Punctuation is stripped before character check and CTC alignment,
-    so this works with the new 'transcription' field that includes punctuation.
-
     Args:
         audio_path: Path to audio file
         transcription: Native script text (may include punctuation)
         language: Language code (te, hi, ta, etc.)
-        check_audio: Whether to check audio-text match (slower)
-        low_conf_threshold: Per-word confidence threshold
-        low_conf_ratio_threshold: Max ratio of low-conf words before review
-
+        romanized_text: Optional model romanization fallback
+        check_audio: Whether to run audio alignment checks (slower)
+        min_duration_sec: Segments shorter than this cannot auto-accept
+
     Returns:
-        ValidationResult with status and details
+        ValidationResult with dual scores and combined verdict
     """
     reasons = []

-    # Strip punctuation for validation (transcription field now includes punctuation)
-    clean_text = _strip_punctuation(transcription)
-
+    # Normalize text surfaces before validation.
+    native_no_placeholders = strip_placeholders(transcription or "")
+    clean_text = _strip_punctuation(native_no_placeholders)
+    romanized = derive_romanized_text(
+        native_text=clean_text,
+        language=language,
+        fallback_romanized=romanized_text,
+    )
+    clean_roman = normalize_roman_for_mms(romanized.text)
+
     # === STEP 1: Character validation (instant) ===
-    # Native transcription = no English allowed
-    char_check = check_characters(clean_text, language, allow_english=False)
-
+    # For English language, allow English chars in native check
+    allow_eng = language == "en"
+    char_check = check_characters(clean_text, language, allow_english=allow_eng)
+
     if not char_check["valid"]:
         return ValidationResult(
             status="reject",
@@ -233,68 +418,178 @@ def validate_transcription(
             script_ratio=char_check["script_ratio"],
             reasons=["Invalid/alien characters found"]
         )
-
-    # Should be mostly native script
+
     if char_check["script_ratio"] < 0.5:
         reasons.append(f"Too few native chars ({char_check['script_ratio']:.0%})")
-
-    # === STEP 2: Audio-text match (0.1s) ===
-    alignment_score = 0.0
+
+    # === STEP 2: Native CTC alignment ===
+    native_ctc_score = 0.0
     low_conf_words = []
     low_conf_ratio = 0.0
-
-    if check_audio:
+    audio_duration_sec = 0.0
+    boundary_start_conf = 0.0
+    boundary_end_conf = 0.0
+    min_internal_word_conf = 0.0
+    internal_low_conf_ratio = 0.0
+
+    if check_audio and clean_text:
         try:
             aligner = _get_ctc_aligner(language)
-            # Use punctuation-stripped text for CTC alignment
-            result = aligner.align(audio_path, clean_text)
-
-            alignment_score = result.alignment_score
-            low_conf_words = result.low_confidence_words
-            low_conf_ratio = result.low_confidence_ratio
-
-            # REJECT if alignment is very poor (likely wrong transcription)
-            if alignment_score < min_alignment_score:
-                reasons.append(f"Poor alignment ({alignment_score:.2f} < {min_alignment_score})")
-
-            # REVIEW if many low-confidence words (but could be code-mixed)
-            elif low_conf_ratio > low_conf_ratio_threshold:
-                reasons.append(f"Many low-confidence words ({low_conf_ratio:.0%})")
-
+            ctc_result = aligner.align(audio_path, clean_text)
+            native_ctc_score = ctc_result.alignment_score
+            low_conf_words = ctc_result.low_confidence_words
+            low_conf_ratio = ctc_result.low_confidence_ratio
+            audio_duration_sec = ctc_result.audio_duration_sec
+
+            metrics = _compute_word_quality_metrics(ctc_result.word_alignments)
+            if metrics["has_metrics"]:
+                boundary_start_conf = metrics["boundary_start"]
+                boundary_end_conf = metrics["boundary_end"]
+                min_internal_word_conf = metrics["min_internal"]
+                internal_low_conf_ratio = metrics["internal_low_ratio"]
         except Exception as e:
-            reasons.append(f"Audio check failed: {str(e)[:50]}")
-
-    # === Determine status ===
-    # Reject: invalid chars OR very poor alignment
-    # Review: high low-conf ratio
-    # Accept: everything else
-    if not char_check["valid"]:
-        status = "reject"
-    elif any("Poor alignment" in r for r in reasons):
-        status = "reject"
-    elif len(reasons) > 0:
-        status = "review"
+            reasons.append(f"CTC alignment failed: {str(e)[:50]}")
+    elif check_audio:
+        reasons.append("Empty native text after normalization")
+
+    # === STEP 3: Romanized MMS alignment ===
+    roman_mms_score = 0.0
+
+    if check_audio and clean_roman:
+        try:
+            mms = _get_mms_aligner()
+            mms_result = mms.align(audio_path, clean_roman)
+            roman_mms_score = mms_result.alignment_score
+        except Exception as e:
+            reasons.append(f"MMS alignment failed: {str(e)[:50]}")
+
+    # === STEP 4: Combined scoring ===
+    if check_audio and native_ctc_score > 0 and roman_mms_score > 0:
+        # Both validators returned scores - use weighted formula
+        combined_score, verdict = compute_combined_score(
+            native_ctc_score, roman_mms_score
+        )
+        if abs(native_ctc_score - roman_mms_score) > MAX_ACCEPT_DISAGREEMENT:
+            reasons.append(
+                f"Validator disagreement: CTC={native_ctc_score:.2f} "
+                f"MMS={roman_mms_score:.2f}"
+            )
+    elif check_audio and native_ctc_score > 0:
+        # CTC only (MMS failed or no romanized text)
+        combined_score = native_ctc_score
+        if native_ctc_score >= ACCEPT_THRESHOLD:
+            verdict = "accept"
+        elif native_ctc_score >= REVIEW_THRESHOLD:
+            verdict = "review"
+        elif native_ctc_score >= RETRY_THRESHOLD:
+            verdict = "retry"
+        else:
+            verdict = "reject"
+        if roman_mms_score == 0 and clean_roman:
+            reasons.append("MMS unavailable, using CTC only")
+    elif check_audio and roman_mms_score > 0:
+        # MMS only (CTC failed)
+        combined_score = roman_mms_score
+        if roman_mms_score >= ACCEPT_THRESHOLD:
+            verdict = "accept"
+        elif roman_mms_score >= REVIEW_THRESHOLD:
+            verdict = "review"
+        elif roman_mms_score >= RETRY_THRESHOLD:
+            verdict = "retry"
+        else:
+            verdict = "reject"
+        reasons.append("CTC failed, using MMS only")
+    elif check_audio:
+        # Both validators returned 0
+        combined_score = 0.0
+        verdict = "review"
+        reasons.append("Both validators returned 0")
     else:
-        status = "accept"
-
+        # No audio check requested
+        combined_score = 0.0
+        verdict = "review" if reasons else "accept"
+
+    # Character validation failures override
+    if not char_check["valid"]:
+        verdict = "reject"
+
+    # Duration guard: very short clips are unstable for reliable auto-accept.
+    if check_audio and 0 < audio_duration_sec < min_duration_sec:
+        reasons.append(
+            f"Short clip ({audio_duration_sec:.2f}s < {min_duration_sec:.2f}s)"
+        )
+        if verdict == "accept":
+            verdict = "review"
+
+    # Boundary-aware gates: protect against VAD mis-cuts.
+    boundary_low = (
+        boundary_start_conf > 0 and boundary_start_conf < BOUNDARY_LOW_CONFIDENCE
+    ) or (
+        boundary_end_conf > 0 and boundary_end_conf < BOUNDARY_LOW_CONFIDENCE
+    )
+    internal_bad = (
+        min_internal_word_conf > 0 and (
+            min_internal_word_conf < MIN_INTERNAL_WORD_CONFIDENCE
+            or internal_low_conf_ratio > MAX_INTERNAL_LOW_CONF_RATIO
+        )
+    )
+
+    if verdict == "accept" and internal_bad:
+        verdict = "review"
+        reasons.append(
+            "Internal word confidence too low "
+            f"(min={min_internal_word_conf:.2f}, "
+            f"low_ratio={internal_low_conf_ratio:.0%})"
+        )
+    elif verdict in ("accept", "retry") and boundary_low and not internal_bad:
+        verdict = "review"
+        reasons.append(
+            "Low boundary confidence with strong internal words "
+            "(likely boundary cut artifact)"
+        )
+
+    # Append score-based reasons
+    if verdict == "retry":
+        reasons.append(
+            f"Below threshold: combined={combined_score:.2f} "
+            f"(CTC={native_ctc_score:.2f}, MMS={roman_mms_score:.2f})"
+        )
+    elif verdict == "reject" and combined_score > 0:
+        reasons.append(
+            f"Poor alignment: combined={combined_score:.2f} "
+            f"(CTC={native_ctc_score:.2f}, MMS={roman_mms_score:.2f})"
+        )
+
     return ValidationResult(
-        status=status,
+        status=verdict,
         char_valid=char_check["valid"],
         invalid_chars=char_check.get("invalid_chars"),
         script_ratio=char_check["script_ratio"],
-        alignment_score=alignment_score,
+        alignment_score=native_ctc_score,  # backward compat
+        native_ctc_score=native_ctc_score,
+        roman_mms_score=roman_mms_score,
+        romanization_source=romanized.source,
+        combined_score=combined_score,
         low_confidence_words=low_conf_words,
         low_confidence_ratio=low_conf_ratio,
-        reasons=reasons if reasons else None
+        audio_duration_sec=audio_duration_sec,
+        boundary_start_confidence=boundary_start_conf,
+        boundary_end_confidence=boundary_end_conf,
+        min_internal_word_confidence=min_internal_word_conf,
+        internal_low_conf_ratio=internal_low_conf_ratio,
+        reasons=reasons if reasons else None,
     )

 def cleanup():
-    """Release CTC aligner resources."""
-    global _ctc_aligner
+    """Release all aligner resources (CTC + MMS)."""
+    global _ctc_aligner, _mms_aligner
     if _ctc_aligner is not None:
         _ctc_aligner.cleanup()
         _ctc_aligner = None
+    if _mms_aligner is not None:
+        _mms_aligner.cleanup()
+        _mms_aligner = None

 # === Quick validation (no audio check) ===


**Identifying preexisting and inadvertent repo changes**


rg -n "min_segment_duration_sec|min_duration_sec: float = 1\.0|under_1s_count|split_boundary_padding_ms|--min-duration|--split-pad-ms" pipeline.py src/backend src/validators -S


Output:
Chunk ID: 3068cb
Wall time: 0.0513 seconds
Process exited with code 0
Original token count: 360
Output:
pipeline.py:45:    min_segment_duration_sec: float = 2.0   # Skip segments shorter than this
pipeline.py:47:    split_boundary_padding_ms: float = 80.0 # Add a bit of silence context around split cuts
pipeline.py:262:            min_duration_sec=self.config.min_segment_duration_sec,
pipeline.py:264:            boundary_padding_ms=self.config.split_boundary_padding_ms,
pipeline.py:297:                if result.polished_duration_ms / 1000.0 < self.config.min_segment_duration_sec:
pipeline.py:353:                            min_duration_sec=self.config.min_segment_duration_sec,
pipeline.py:403:                                    min_duration_sec=self.config.min_segment_duration_sec,
pipeline.py:510:                "min_segment_duration_sec": self.config.min_segment_duration_sec,
pipeline.py:511:                "split_boundary_padding_ms": self.config.split_boundary_padding_ms,
pipeline.py:565:        min_segment_duration_sec=min_duration_sec,
pipeline.py:566:        split_boundary_padding_ms=split_pad_ms,
pipeline.py:639:    parser.add_argument("--min-duration", type=float, default=2.0,
pipeline.py:641:    parser.add_argument("--split-pad-ms", type=float, default=80.0,
src/backend/config.py:77:    "min_segment_duration_sec": 2.0,   # Minimum segment duration
src/backend/config.py:78:    "split_boundary_padding_ms": 80.0,
src/backend/audio_processor.py:324:        "under_1s_count": sum(1 for d in durations if d < 1),


**Confirming file changes and pycache handling**


nl -ba src/romanization.py | sed -n '1,260p'


nl -ba src/validators/simple_validator.py | sed -n '1,360p'


nl -ba src/validators/simple_validator.py | sed -n '360,620p'


nl -ba src/backend/audio_processor.py | sed -n '1,260p'


Output:
Chunk ID: 7cad39
Wall time: 0.0510 seconds
Process exited with code 0
Original token count: 1513
Output:
     1	"""
     2	Deterministic romanization utilities for validation/alignment.
     3
     4	Primary goal: convert authoritative native transcription into stable
     5	Latin text suitable for MMS forced alignment.
     6	"""
     7	from __future__ import annotations
     8
     9	import re
    10	import unicodedata
    11	from dataclasses import dataclass
    12	from typing import Optional
    13
    14
    15	_PLACEHOLDER_RE = re.compile(r"\[(UNK|INAUDIBLE|NO_SPEECH)\]", re.IGNORECASE)
    16	_NON_MMS_CHARS_RE = re.compile(r"[^a-z' ]+")
    17	_MULTISPACE_RE = re.compile(r"\s+")
    18
    19	# Normalize common quote/dash variants before filtering.
    20	_APOSTROPHE_TRANSLATION = str.maketrans({
    21	    "’": "'",
    22	    "‘": "'",
    23	    "ʼ": "'",
    24	    "`": "'",
    25	    "´": "'",
    26	})
    27	_DASH_TRANSLATION = str.maketrans({
    28	    "-": " ",
    29	    "_": " ",
    30	    "/": " ",
    31	    "|": " ",
    32	})
    33
    34
    35	@dataclass
    36	class RomanizationResult:
    37	    """Deterministic romanization output."""
    38	    text: str
    39	    source: str  # indic_transliteration | unidecode | fallback_model | passthrough
    40
    41
    42	def _strip_placeholders(text: str) -> str:
    43	    """Remove bracket placeholders that should not be force-aligned."""
    44	    return _PLACEHOLDER_RE.sub(" ", text or "")
    45
    46
    47	def normalize_roman_for_mms(text: str) -> str:
    48	    """
    49	    Normalize Latin text to MMS-friendly alphabet.
    50
    51	    Keeps only: a-z, apostrophe, space.
    52	    """
    53	    cleaned = _strip_placeholders(text)
    54	    cleaned = unicodedata.normalize("NFKC", cleaned)
    55	    cleaned = cleaned.translate(_APOSTROPHE_TRANSLATION)
    56	    cleaned = cleaned.translate(_DASH_TRANSLATION)
    57	    cleaned = cleaned.lower()
    58	    cleaned = _NON_MMS_CHARS_RE.sub(" ", cleaned)
    59	    cleaned = _MULTISPACE_RE.sub(" ", cleaned).strip()
    60	    return cleaned
    61
    62
    63	def _romanize_with_indic_transliteration(
    64	    native_text: str,
    65	    language: str,
    66	) -> Optional[str]:
    67	    """
    68	    Romanize with indic-transliteration if available.
    69
    70	    Returns None when dependency is unavailable or language/script is unsupported.
    71	    """
    72	    try:
    73	        from indic_transliteration.sanscript import (
    74	            ASSAMESE,
    75	            BENGALI,
    76	            DEVANAGARI,
    77	            GUJARATI,
    78	            GURMUKHI,
    79	            KANNADA,
    80	            MALAYALAM,
    81	            ORIYA,
    82	            TAMIL,
    83	            TELUGU,
    84	            transliterate,
    85	        )
    86	    except Exception:
    87	        return None
    88
    89	    lang = (language or "").lower()[:2]
    90	    script_map = {
    91	        "te": TELUGU,
    92	        "hi": DEVANAGARI,
    93	        "mr": DEVANAGARI,
    94	        "ta": TAMIL,
    95	        "kn": KANNADA,
    96	        "ml": MALAYALAM,
    97	        "bn": BENGALI,
    98	        "as": ASSAMESE,
    99	        "gu": GUJARATI,
   100	        "pa": GURMUKHI,
   101	        "or": ORIYA,
   102	    }
   103	    source_scheme = script_map.get(lang)
   104	    if source_scheme is None:
   105	        return None
   106
   107	    # HK keeps simple ASCII letters and avoids diacritics.
   108	    target_scheme = "hk"
   109	    try:
   110	        return transliterate(native_text, source_scheme, target_scheme)
   111	    except Exception:
   112	        return None
   113
   114
   115	def _romanize_with_unidecode(native_text: str) -> str:
   116	    """
   117	    Romanize using Unidecode fallback.
   118
   119	    Unidecode is deterministic and lightweight; quality is lower than language
   120	    specific transliteration, but still robust enough for alignment text.
   121	    """
   122	    try:
   123	        from unidecode import unidecode
   124	    except Exception:
   125	        return native_text
   126	    return unidecode(native_text)
   127
   128
   129	def derive_romanized_text(
   130	    native_text: str,
   131	    language: str,
   132	    fallback_romanized: str = "",
   133	) -> RomanizationResult:
   134	    """
   135	    Derive romanization deterministically from authoritative native text.
   136
   137	    Fallback priority:
   138	    1) indic-transliteration (if installed)
   139	    2) unidecode
   140	    3) model-provided romanized fallback (normalized)
   141	    """
   142	    native = _strip_placeholders(native_text or "").strip()
   143	    fallback_norm = normalize_roman_for_mms(fallback_romanized)
   144
   145	    if not native:
   146	        if fallback_norm:
   147	            return RomanizationResult(text=fallback_norm, source="fallback_model")
   148	        return RomanizationResult(text="", source="passthrough")
   149
   150	    if (language or "").lower()[:2] == "en":
   151	        return RomanizationResult(
   152	            text=normalize_roman_for_mms(native),
   153	            source="passthrough",
   154	        )
   155
   156	    via_indic = _romanize_with_indic_transliteration(native, language)
   157	    if via_indic:
   158	        normalized = normalize_roman_for_mms(via_indic)
   159	        if normalized:
   160	            return RomanizationResult(text=normalized, source="indic_transliteration")
   161
   162	    via_unidecode = normalize_roman_for_mms(_romanize_with_unidecode(native))
   163	    if via_unidecode:
   164	        return RomanizationResult(text=via_unidecode, source="unidecode")
   165
   166	    if fallback_norm:
   167	        return RomanizationResult(text=fallback_norm, source="fallback_model")
   168
   169	    return RomanizationResult(text="", source="passthrough")
   170
   171
   172	def strip_placeholders(text: str) -> str:
   173	    """Public helper used by validators for script checks."""
   174	    return _strip_placeholders(text)