You are Codex, based on GPT-5. You are running as a coding agent in the Codex CLI on a user's computer.

## General

- When searching for text or files, prefer using `rg` or `rg --files` respectively because `rg` is much faster than alternatives like `grep`. (If the `rg` command is not found, then use alternatives.)

## Editing constraints

- Default to ASCII when editing or creating files. Only introduce non-ASCII or other Unicode characters when there is a clear justification and the file already uses them.
- Add succinct code comments that explain what is going on if code is not self-explanatory. You should not add comments like "Assigns the value to the variable", but a brief comment might be useful ahead of a complex code block that the user would otherwise have to spend time parsing out. Usage of these comments should be rare.
- Try to use apply_patch for single file edits, but it is fine to explore other options to make the edit if it does not work well. Do not use apply_patch for changes that are auto-generated (i.e. generating package.json or running a lint or format command like gofmt) or when scripting is more efficient (such as search and replacing a string across a codebase).
- You may be in a dirty git worktree.
    * NEVER revert existing changes you did not make unless explicitly requested, since these changes were made by the user.
    * If asked to make a commit or code edits and there are unrelated changes to your work or changes that you didn't make in those files, don't revert those changes.
    * If the changes are in files you've touched recently, you should read carefully and understand how you can work with the changes rather than reverting them.
    * If the changes are in unrelated files, just ignore them and don't revert them.
- Do not amend a commit unless explicitly requested to do so.
- While you are working, you might notice unexpected changes that you didn't make. If this happens, STOP IMMEDIATELY and ask the user how they would like to proceed.
- **NEVER** use destructive commands like `git reset --hard` or `git checkout --` unless specifically requested or approved by the user.

## Plan tool

When using the planning tool:
- Skip using the planning tool for straightforward tasks (roughly the easiest 25%).
- Do not make single-step plans.
- When you made a plan, update it after having performed one of the sub-tasks that you shared on the plan.

## Special user requests

- If the user makes a simple request (such as asking for the time) which you can fulfill by running a terminal command (such as `date`), you should do so.
- If the user asks for a "review", default to a code review mindset: prioritise identifying bugs, risks, behavioural regressions, and missing tests. Findings must be the primary focus of the response - keep summaries or overviews brief and only after enumerating the issues. Present findings first (ordered by severity with file/line references), follow with open questions or assumptions, and offer a change-summary only as a secondary detail. If no findings are discovered, state that explicitly and mention any residual risks or testing gaps.

## Frontend tasks
When doing frontend design tasks, avoid collapsing into "AI slop" or safe, average-looking layouts.
Aim for interfaces that feel intentional, bold, and a bit surprising.
- Typography: Use expressive, purposeful fonts and avoid default stacks (Inter, Roboto, Arial, system).
- Color & Look: Choose a clear visual direction; define CSS variables; avoid purple-on-white defaults. No purple bias or dark mode bias.
- Motion: Use a few meaningful animations (page-load, staggered reveals) instead of generic micro-motions.
- Background: Don't rely on flat, single-color backgrounds; use gradients, shapes, or subtle patterns to build atmosphere.
- Overall: Avoid boilerplate layouts and interchangeable UI patterns. Vary themes, type families, and visual languages across outputs.
- Ensure the page loads properly on both desktop and mobile

Exception: If working within an existing website or design system, preserve the established patterns, structure, and visual language.

## Presenting your work and final message

You are producing plain text that will later be styled by the CLI. Follow these rules exactly. Formatting should make results easy to scan, but not feel mechanical. Use judgment to decide how much structure adds value.

- Default: be very concise; friendly coding teammate tone.
- Ask only when needed; suggest ideas; mirror the user's style.
- For substantial work, summarize clearly; follow final‑answer formatting.
- Skip heavy formatting for simple confirmations.
- Don't dump large files you've written; reference paths only.
- No "save/copy this file" - User is on the same machine.
- Offer logical next steps (tests, commits, build) briefly; add verify steps if you couldn't do something.
- For code changes:
  * Lead with a quick explanation of the change, and then give more details on the context covering where and why a change was made. Do not start this explanation with "summary", just jump right in.
  * If there are natural next steps the user may want to take, suggest them at the end of your response. Do not make suggestions if there are no natural next steps.
  * When suggesting multiple options, use numeric lists for the suggestions so the user can quickly respond with a single number.
- The user does not command execution outputs. When asked to show the output of a command (e.g. `git show`), relay the important details in your answer or summarize the key lines so the user understands the result.

### Final answer structure and style guidelines

- Plain text; CLI handles styling. Use structure only when it helps scanability.
- Headers: optional; short Title Case (1-3 words) wrapped in **…**; no blank line before the first bullet; add only if they truly help.
- Bullets: use - ; merge related points; keep to one line when possible; 4–6 per list ordered by importance; keep phrasing consistent.
- Monospace: backticks for commands/paths/env vars/code ids and inline examples; use for literal keyword bullets; never combine with **.
- Code samples or multi-line snippets should be wrapped in fenced code blocks; include an info string as often as possible.
- Structure: group related bullets; order sections general → specific → supporting; for subsections, start with a bolded keyword bullet, then items; match complexity to the task.
- Tone: collaborative, concise, factual; present tense, active voice; self‑contained; no "above/below"; parallel wording.
- Don'ts: no nested bullets/hierarchies; no ANSI codes; don't cram unrelated keywords; keep keyword lists short—wrap/reformat if long; avoid naming formatting styles in answers.
- Adaptation: code explanations → precise, structured with code refs; simple tasks → lead with outcome; big changes → logical walkthrough + rationale + next actions; casual one-offs → plain sentences, no headers/bullets.
- File References: When referencing files in your response follow the below rules:
  * Use inline code to make file paths clickable.
  * Each reference should have a stand alone path. Even if it's the same file.
  * Accepted: absolute, workspace‑relative, a/ or b/ diff prefixes, or bare filename/suffix.
  * Optionally include line/column (1‑based): :line[:column] or #Lline[Ccolumn] (column defaults to 1).
  * Do not use URIs like file://, vscode://, or https://.
  * Do not provide range of lines
  * Examples: src/app.ts, src/app.ts:42, b/server/index.js#L10, C:\repo\project\main.rs:12:5


<permissions instructions>
Filesystem sandboxing defines which files can be read or written. `sandbox_mode` is `danger-full-access`: No filesystem sandboxing - all commands are permitted. Network access is enabled.
 Approvals are your mechanism to get user consent to run shell commands without the sandbox. `approval_policy` is `never`: This is a non-interactive mode where you may NEVER ask the user for approval to run commands. Instead, you must always persist and work around constraints to solve the task for the user. You MUST do your utmost best to finish the task and validate your work before yielding. If this mode is paired with `danger-full-access`, take advantage of it to deliver the best outcome for the user. Further, in this mode, your default testing philosophy is overridden: Even if you don't see local patterns for testing, you may add tests and scripts to validate your work. Just remove them before yielding.
</permissions instructions>


# AGENTS.md instructions for /home/ubuntu/maya3_transcribe

<INSTRUCTIONS>
## Skills
A skill is a set of local instructions to follow that is stored in a `SKILL.md` file. Below is the list of skills that can be used. Each entry includes a name, description, and file path so you can open the source for full instructions when using a specific skill.
### Available skills
- skill-creator: Guide for creating effective skills. This skill should be used when users want to create a new skill (or update an existing skill) that extends Codex's capabilities with specialized knowledge, workflows, or tool integrations. (file: /home/ubuntu/.codex/skills/.system/skill-creator/SKILL.md)
- skill-installer: Install Codex skills into $CODEX_HOME/skills from a curated list or a GitHub repo path. Use when a user asks to list installable skills, install a curated skill, or install a skill from another repo (including private repos). (file: /home/ubuntu/.codex/skills/.system/skill-installer/SKILL.md)
### How to use skills
- Discovery: The list above is the skills available in this session (name + description + file path). Skill bodies live on disk at the listed paths.
- Trigger rules: If the user names a skill (with `$SkillName` or plain text) OR the task clearly matches a skill's description shown above, you must use that skill for that turn. Multiple mentions mean use them all. Do not carry skills across turns unless re-mentioned.
- Missing/blocked: If a named skill isn't in the list or the path can't be read, say so briefly and continue with the best fallback.
- How to use a skill (progressive disclosure):
  1) After deciding to use a skill, open its `SKILL.md`. Read only enough to follow the workflow.
  2) When `SKILL.md` references relative paths (e.g., `scripts/foo.py`), resolve them relative to the skill directory listed above first, and only consider other paths if needed.
  3) If `SKILL.md` points to extra folders such as `references/`, load only the specific files needed for the request; don't bulk-load everything.
  4) If `scripts/` exist, prefer running or patching them instead of retyping large code blocks.
  5) If `assets/` or templates exist, reuse them instead of recreating from scratch.
- Coordination and sequencing:
  - If multiple skills apply, choose the minimal set that covers the request and state the order you'll use them.
  - Announce which skill(s) you're using and why (one short line). If you skip an obvious skill, say why.
- Context hygiene:
  - Keep context small: summarize long sections instead of pasting them; only load extra files when needed.
  - Avoid deep reference-chasing: prefer opening only files directly linked from `SKILL.md` unless you're blocked.
  - When variants exist (frameworks, providers, domains), pick only the relevant reference file(s) and note that choice.
- Safety and fallback: If a skill can't be applied cleanly (missing files, unclear instructions), state the issue, pick the next-best approach, and continue.
</INSTRUCTIONS>


<environment_context>
  <cwd>/home/ubuntu/maya3_transcribe</cwd>
  <shell>bash</shell>
</environment_context>


<collaboration_mode>you are now in code mode.
</collaboration_mode>


Now that we have these transcriptions, what techniques do you suggest we do to validate these transcriptions. validate, improvise the prompt to see a better one, and then how can we be so sure to know the performance of the transcriptions. What do you generally suggest we can do here. Like Forced Aligners we can implement, get probability scores, and what else can we do here. Suggest the new validation and scoring pipeline first. we can decide on the other part next. what else can we do as validation steps. ideate. First read and understand the pipeline first. then ideate on validations of these transcriptions to ensure scoring and low-probability ones gets redone or improved.


Summary: auto

## Skills
A skill is a set of local instructions to follow that is stored in a `SKILL.md` file. Below is the list of skills that can be used. Each entry includes a name, description, and file path so you can open the source for full instructions when using a specific skill.
### Available skills
- skill-creator: Guide for creating effective skills. This skill should be used when users want to create a new skill (or update an existing skill) that extends Codex's capabilities with specialized knowledge, workflows, or tool integrations. (file: /home/ubuntu/.codex/skills/.system/skill-creator/SKILL.md)
- skill-installer: Install Codex skills into $CODEX_HOME/skills from a curated list or a GitHub repo path. Use when a user asks to list installable skills, install a curated skill, or install a skill from another repo (including private repos). (file: /home/ubuntu/.codex/skills/.system/skill-installer/SKILL.md)
### How to use skills
- Discovery: The list above is the skills available in this session (name + description + file path). Skill bodies live on disk at the listed paths.
- Trigger rules: If the user names a skill (with `$SkillName` or plain text) OR the task clearly matches a skill's description shown above, you must use that skill for that turn. Multiple mentions mean use them all. Do not carry skills across turns unless re-mentioned.
- Missing/blocked: If a named skill isn't in the list or the path can't be read, say so briefly and continue with the best fallback.
- How to use a skill (progressive disclosure):
  1) After deciding to use a skill, open its `SKILL.md`. Read only enough to follow the workflow.
  2) When `SKILL.md` references relative paths (e.g., `scripts/foo.py`), resolve them relative to the skill directory listed above first, and only consider other paths if needed.
  3) If `SKILL.md` points to extra folders such as `references/`, load only the specific files needed for the request; don't bulk-load everything.
  4) If `scripts/` exist, prefer running or patching them instead of retyping large code blocks.
  5) If `assets/` or templates exist, reuse them instead of recreating from scratch.
- Coordination and sequencing:
  - If multiple skills apply, choose the minimal set that covers the request and state the order you'll use them.
  - Announce which skill(s) you're using and why (one short line). If you skip an obvious skill, say why.
- Context hygiene:
  - Keep context small: summarize long sections instead of pasting them; only load extra files when needed.
  - Avoid deep reference-chasing: prefer opening only files directly linked from `SKILL.md` unless you're blocked.
  - When variants exist (frameworks, providers, domains), pick only the relevant reference file(s) and note that choice.
- Safety and fallback: If a skill can't be applied cleanly (missing files, unclear instructions), state the issue, pick the next-best approach, and continue.


**Starting project inspection**

Planning to read project files using searches and directory listings to gather necessary context before forming a detailed approach.


Output:
Exit code: 0
Wall time: 0 seconds
Output:
docs
pipeline.py
README.md
requirements.txt
src
transcriptions
venv


Summary: auto


**Locating read pipeline implementation**

I'm checking pipeline.py to find the read pipeline code, considering also searching README or docs for relevant details.


Output:
Exit code: 0
Wall time: 0 seconds
Output:
"""
Main transcription pipeline orchestrating the full workflow:
1. Download audio segments from R2
2. Fetch language info from Supabase
3. Process/split audio segments
4. Transcribe using Gemini
5. Output results

Full control over all parameters for testing and production use.
"""
import os
import json
import time
from typing import Optional, List, Dict
from dataclasses import dataclass, field
from datetime import datetime

# Import from src.backend package
from src.backend.config import DEFAULT_SETTINGS, GEMINI_MODELS, get_model_name
from src.backend.r2_storage import download_video_segments
from src.backend.supabase_client import get_video_language
from src.backend.audio_processor import AudioProcessor, AudioChunk, get_segment_stats
from src.backend.gemini_transcriber import GeminiTranscriber, TranscriptionConfig
from src.backend.transcription_schema import TranscriptionResult

@dataclass
class PipelineConfig:
    """
    Complete configuration for the transcription pipeline.
    All parameters are exposed for full control.
    """
    # === Video Settings ===
    video_id: str
    language: Optional[str] = None  # If None, fetched from Supabase
    default_language: str = "Telugu"  # Fallback if Supabase lookup fails

    # === Audio Processing Settings ===
    max_segment_duration_sec: float = 10.0  # Hard limit - segments longer than this are split
    min_segment_duration_sec: float = 1.0   # Skip segments shorter than this
    chunk_overlap_sec: float = 0.0          # Overlap when splitting (for word boundaries)

    # === Segment Selection Settings ===
    max_segments: Optional[int] = None      # Limit total segments processed (for testing)
    segment_start_index: int = 0            # Start from this segment index
    skip_short_segments: bool = True        # Skip segments < min_duration

    # === Transcription Settings ===
    model: str = "gemini-3-flash-preview"   # Gemini model to use
    thinking_level: Optional[str] = "high"  # minimal, low, medium, high (Gemini 3)
    temperature: float = 1.0                # Keep at 1.0 for Gemini 3

    # === Output Settings ===
    output_dir: str = "./transcriptions"
    save_intermediate: bool = True          # Save results after each batch
    batch_size: int = 10                    # How many to process before saving

    # === Storage Settings ===
    work_dir: str = field(default_factory=lambda: DEFAULT_SETTINGS["work_dir"])
    cleanup_after: bool = False             # Remove downloaded files after processing
    use_regex_fallback: bool = True         # Try regex if exact R2 match fails

@dataclass
class PipelineResult:
    """Results from a pipeline run."""
    video_id: str
    language: str
    model: str
    thinking_level: Optional[str]

    total_segments_found: int
    segments_processed: int
    chunks_created: int
    transcriptions_completed: int

    total_audio_duration_sec: float
    total_processing_time_sec: float

    output_file: str
    errors: List[str]

    started_at: str
    completed_at: str

class TranscriptionPipeline:
    """
    Main pipeline orchestrating the full transcription workflow.
    """

    def __init__(self, config: PipelineConfig):
        """Initialize pipeline with configuration."""
        self.config = config
        self.results: List[TranscriptionResult] = []
        self.errors: List[str] = []
        self.start_time: Optional[float] = None

    def _log(self, message: str, level: str = "INFO"):
        """Log a message with timestamp."""
        timestamp = datetime.now().strftime("%H:%M:%S")
        print(f"[{timestamp}] [{level}] {message}")

    def run(self) -> PipelineResult:
        """
        Execute the full pipeline.

        Returns:
            PipelineResult with summary and statistics
        """
        self.start_time = time.time()
        started_at = datetime.now().isoformat()

        self._log(f"Starting pipeline for video: {self.config.video_id}")
        self._log(f"Model: {self.config.model}, Thinking: {self.config.thinking_level}")

        # === Step 1: Download from R2 ===
        self._log("Step 1: Downloading audio segments from R2...")
        try:
            segments_dir, metadata = download_video_segments(
                self.config.video_id,
                work_dir=self.config.work_dir,
                use_regex_fallback=self.config.use_regex_fallback
            )
        except Exception as e:
            self.errors.append(f"R2 download failed: {e}")
            self._log(f"ERROR: {e}", "ERROR")
            raise

        # Get segment stats
        stats = get_segment_stats(segments_dir)
        self._log(f"Found {stats.get('total_files', 0)} audio files, "
                  f"total duration: {stats.get('total_duration_sec', 0):.1f}s")
        if stats.get('over_10s_count', 0) > 0:
            self._log(f"  -> {stats['over_10s_count']} segments over 10s will be split")

        # === Step 2: Get Language ===
        self._log("Step 2: Fetching language info...")
        if self.config.language:
            language = self.config.language
            self._log(f"Using specified language: {language}")
        else:
            language = get_video_language(
                self.config.video_id,
                default=self.config.default_language
            )
            self._log(f"Language from Supabase: {language}")

        # === Step 3: Process Audio Segments ===
        self._log("Step 3: Processing audio segments...")
        processor = AudioProcessor(
            max_duration_sec=self.config.max_segment_duration_sec,
            min_duration_sec=self.config.min_segment_duration_sec,
            overlap_sec=self.config.chunk_overlap_sec
        )

        chunks = processor.process_segments_directory(
            segments_dir,
            max_segments=self.config.max_segments,
            skip_short=self.config.skip_short_segments
        )

        # Apply start index
        if self.config.segment_start_index > 0:
            chunks = chunks[self.config.segment_start_index:]
            self._log(f"Starting from index {self.config.segment_start_index}, "
                      f"{len(chunks)} chunks remaining")

        # === Step 4: Transcribe ===
        self._log(f"Step 4: Transcribing {len(chunks)} chunks...")

        transcriber = GeminiTranscriber()
        transcription_config = TranscriptionConfig(
            model=self.config.model,
            thinking_level=self.config.thinking_level,
            temperature=self.config.temperature,
            language=language
        )

        # Process in batches
        for batch_start in range(0, len(chunks), self.config.batch_size):
            batch_end = min(batch_start + self.config.batch_size, len(chunks))
            batch = chunks[batch_start:batch_end]

            self._log(f"Processing batch {batch_start + 1}-{batch_end} of {len(chunks)}")

            batch_results = transcriber.transcribe_batch(
                batch,
                transcription_config,
                progress_callback=lambda c, t: self._log(
                    f"  Chunk {batch_start + c}/{len(chunks)}: "
                    f"{batch[c-1].original_segment}"
                )
            )

            self.results.extend(batch_results)

            # Save intermediate results
            if self.config.save_intermediate:
                self._save_results(partial=True)

        # === Step 5: Save Final Results ===
        self._log("Step 5: Saving results...")
        output_file = self._save_results(partial=False)

        # === Cleanup ===
        if self.config.cleanup_after:
            self._log("Cleaning up downloaded files...")
            import shutil
            video_work_dir = os.path.join(self.config.work_dir, self.config.video_id)
            if os.path.exists(video_work_dir):
                shutil.rmtree(video_work_dir)

        # === Build Result ===
        total_time = time.time() - self.start_time
        completed_at = datetime.now().isoformat()

        total_audio = sum(r.duration_sec for r in self.results)

        result = PipelineResult(
            video_id=self.config.video_id,
            language=language,
            model=self.config.model,
            thinking_level=self.config.thinking_level,
            total_segments_found=stats.get('total_files', 0),
            segments_processed=len(set(r.segment_id for r in self.results)),
            chunks_created=len(chunks),
            transcriptions_completed=len(self.results),
            total_audio_duration_sec=total_audio,
            total_processing_time_sec=total_time,
            output_file=output_file,
            errors=self.errors,
            started_at=started_at,
            completed_at=completed_at
        )

        self._log(f"Pipeline completed in {total_time:.1f}s")
        self._log(f"Transcribed {total_audio:.1f}s of audio")
        self._log(f"Output saved to: {output_file}")


Output:
Exit code: 0
Wall time: 0 seconds
Output:

        return result

    def _save_results(self, partial: bool = False) -> str:
        """Save results to JSON file."""
        os.makedirs(self.config.output_dir, exist_ok=True)

        timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")
        suffix = "_partial" if partial else ""
        filename = f"{self.config.video_id}_{self.config.model.replace('-', '_')}_{timestamp}{suffix}.json"
        output_path = os.path.join(self.config.output_dir, filename)

        # Convert results to dict
        output_data = {
            "video_id": self.config.video_id,
            "model": self.config.model,
            "thinking_level": self.config.thinking_level,
            "language": self.results[0].language if self.results else self.config.default_language,
            "config": {
                "max_segment_duration_sec": self.config.max_segment_duration_sec,
                "min_segment_duration_sec": self.config.min_segment_duration_sec,
                "max_segments": self.config.max_segments,
            },
            "results_count": len(self.results),
            "total_duration_sec": sum(r.duration_sec for r in self.results),
            "results": [r.model_dump() for r in self.results]
        }

        with open(output_path, 'w', encoding='utf-8') as f:
            json.dump(output_data, f, ensure_ascii=False, indent=2)

        return output_path

def run_pipeline(
    video_id: str,
    language: Optional[str] = None,
    model: str = "gemini-3-flash-preview",
    thinking_level: str = "high",
    max_segments: Optional[int] = None,
    max_duration_sec: float = 10.0,
    output_dir: str = "./transcriptions"
) -> PipelineResult:
    """
    Convenience function to run the pipeline with common settings.

    Args:
        video_id: YouTube video ID
        language: Primary language (if None, fetched from Supabase)
        model: Gemini model to use
        thinking_level: Thinking level for Gemini 3
        max_segments: Limit segments for testing
        max_duration_sec: Max segment duration (hard limit)
        output_dir: Directory for output files

    Returns:
        PipelineResult with summary
    """
    config = PipelineConfig(
        video_id=video_id,
        language=language,
        model=model,
        thinking_level=thinking_level,
        max_segments=max_segments,
        max_segment_duration_sec=max_duration_sec,
        output_dir=output_dir
    )

    pipeline = TranscriptionPipeline(config)
    return pipeline.run()

def test_models(
    video_id: str,
    language: str,
    models: List[str],
    thinking_levels: List[str] = ["high"],
    max_segments: int = 3
) -> Dict[str, PipelineResult]:
    """
    Test multiple models on the same video for comparison.

    Args:
        video_id: Video ID to test
        language: Primary language
        models: List of model names to test
        thinking_levels: Thinking levels to try
        max_segments: Segments per test

    Returns:
        Dict mapping model_thinking to PipelineResult
    """
    results = {}

    for model in models:
        for thinking in thinking_levels:
            key = f"{model}_{thinking}"
            print(f"\n{'='*60}")
            print(f"Testing: {key}")
            print(f"{'='*60}")

            try:
                result = run_pipeline(
                    video_id=video_id,
                    language=language,
                    model=model,
                    thinking_level=thinking,
                    max_segments=max_segments,
                    output_dir=f"./transcriptions/test_{video_id}"
                )
                results[key] = result
            except Exception as e:
                print(f"ERROR testing {key}: {e}")

    return results

if __name__ == "__main__":
    import argparse

    parser = argparse.ArgumentParser(description="Transcription Pipeline")
    parser.add_argument("video_id", help="YouTube video ID")
    parser.add_argument("--language", "-l", help="Primary language (default: auto-detect)")
    parser.add_argument("--model", "-m", default="gemini-3-flash-preview",
                        help="Gemini model to use")
    parser.add_argument("--thinking", "-t", default="high",
                        choices=["minimal", "low", "medium", "high"],
                        help="Thinking level for Gemini 3")
    parser.add_argument("--max-segments", "-n", type=int,
                        help="Maximum segments to process")
    parser.add_argument("--max-duration", "-d", type=float, default=10.0,
                        help="Maximum segment duration in seconds")
    parser.add_argument("--output", "-o", default="./transcriptions",
                        help="Output directory")

    args = parser.parse_args()

    result = run_pipeline(
        video_id=args.video_id,
        language=args.language,
        model=args.model,
        thinking_level=args.thinking,
        max_segments=args.max_segments,
        max_duration_sec=args.max_duration,
        output_dir=args.output
    )

    print(f"\n{'='*60}")
    print("Pipeline Summary")
    print(f"{'='*60}")
    print(f"Video ID: {result.video_id}")
    print(f"Language: {result.language}")
    print(f"Model: {result.model} (thinking: {result.thinking_level})")
    print(f"Segments processed: {result.segments_processed}")
    print(f"Chunks transcribed: {result.transcriptions_completed}")
    print(f"Total audio: {result.total_audio_duration_sec:.1f}s")
    print(f"Total time: {result.total_processing_time_sec:.1f}s")
    print(f"Output: {result.output_file}")
    if result.errors:
        print(f"Errors: {result.errors}")


**Exploring data structures**

I'll investigate the transcription schema, gemini transcriber, and audio processor in the backend to understand core data structures relevant to the task.


rg -n "TranscriptionResult|AudioChunk|GeminiTranscriber|TranscriptionConfig|confidence|score|prob" -S src


Output:
Exit code: 0
Wall time: 0 seconds
Output:
src/backend/gemini_transcriber.py:16:    TranscriptionResult,
src/backend/gemini_transcriber.py:21:from .audio_processor import AudioChunk
src/backend/gemini_transcriber.py:25:class TranscriptionConfig:
src/backend/gemini_transcriber.py:34:class GeminiTranscriber:
src/backend/gemini_transcriber.py:70:    def _build_config(self, config: TranscriptionConfig) -> types.GenerateContentConfig:
src/backend/gemini_transcriber.py:100:        config: TranscriptionConfig
src/backend/gemini_transcriber.py:170:        chunk: AudioChunk,
src/backend/gemini_transcriber.py:171:        config: TranscriptionConfig
src/backend/gemini_transcriber.py:172:    ) -> TranscriptionResult:
src/backend/gemini_transcriber.py:174:        Transcribe an AudioChunk and return structured result.
src/backend/gemini_transcriber.py:177:            chunk: AudioChunk to transcribe
src/backend/gemini_transcriber.py:181:            TranscriptionResult with full metadata
src/backend/gemini_transcriber.py:200:                confidence=raw_result.get('confidence'),
src/backend/gemini_transcriber.py:204:        return TranscriptionResult(
src/backend/gemini_transcriber.py:218:        chunks: List[AudioChunk],
src/backend/gemini_transcriber.py:219:        config: TranscriptionConfig,
src/backend/gemini_transcriber.py:222:    ) -> List[TranscriptionResult]:
src/backend/gemini_transcriber.py:227:            chunks: List of AudioChunks to transcribe
src/backend/gemini_transcriber.py:233:            List of TranscriptionResults
src/backend/gemini_transcriber.py:262:    segments: List[AudioChunk],
src/backend/gemini_transcriber.py:267:) -> List[TranscriptionResult]:
src/backend/gemini_transcriber.py:272:        segments: List of AudioChunk objects
src/backend/gemini_transcriber.py:279:        List of TranscriptionResult objects
src/backend/gemini_transcriber.py:281:    config = TranscriptionConfig(
src/backend/gemini_transcriber.py:287:    transcriber = GeminiTranscriber()
src/backend/gemini_transcriber.py:298:        config = TranscriptionConfig(
src/backend/gemini_transcriber.py:304:        transcriber = GeminiTranscriber()
src/backend/audio_processor.py:14:class AudioChunk:
src/backend/audio_processor.py:68:    ) -> List[AudioChunk]:
src/backend/audio_processor.py:78:            List of AudioChunk objects
src/backend/audio_processor.py:92:            return [AudioChunk(
src/backend/audio_processor.py:128:            chunks.append(AudioChunk(
src/backend/audio_processor.py:146:    ) -> List[AudioChunk]:
src/backend/audio_processor.py:157:            List of AudioChunk objects ready for transcription
src/backend/audio_processor.py:196:                chunks = [AudioChunk(
src/backend/__init__.py:29:from .audio_processor import AudioProcessor, AudioChunk, get_segment_stats
src/backend/__init__.py:33:    TranscriptionResult,
src/backend/__init__.py:40:    GeminiTranscriber,
src/backend/__init__.py:41:    TranscriptionConfig,
src/backend/__init__.py:63:    'AudioChunk',
src/backend/__init__.py:67:    'TranscriptionResult',
src/backend/__init__.py:72:    'GeminiTranscriber',
src/backend/__init__.py:73:    'TranscriptionConfig',
src/backend/transcription_schema.py:39:    confidence: Optional[float] = Field(
src/backend/transcription_schema.py:41:        description="Optional confidence score for the transcription (0-1)"
src/backend/transcription_schema.py:50:class TranscriptionResult(BaseModel):
src/backend/transcription_schema.py:152:        "confidence": {
src/backend/transcription_schema.py:154:            "description": "Confidence score 0-1 (optional)"


sed -n '1,260p' src/backend/gemini_transcriber.py


Output:
Exit code: 0
Wall time: 0 seconds
Output:
"""
Gemini transcription module using Google AI Studio endpoints.
Handles audio transcription with structured output and multiple model support.
"""
import time
from pathlib import Path
from typing import List, Optional, Dict, Any
from dataclasses import dataclass

from google import genai
from google.genai import types

from .config import GEMINI_API_KEY, GEMINI_MODELS, get_model_name
from .transcription_schema import (
    TranscriptionOutput,
    TranscriptionResult,
    get_transcription_prompt,
    get_user_prompt,
    TRANSCRIPTION_JSON_SCHEMA
)
from .audio_processor import AudioChunk

@dataclass
class TranscriptionConfig:
    """Configuration for a transcription job."""
    model: str = "gemini-3-flash-preview"
    thinking_level: Optional[str] = "high"  # minimal, low, medium, high
    temperature: float = 1.0  # Keep at 1.0 for Gemini 3
    language: str = "Telugu"
    timeout_sec: int = 60

class GeminiTranscriber:
    """
    Handles audio transcription using Gemini models via Google AI Studio.
    """

    def __init__(self, api_key: Optional[str] = None):
        """
        Initialize the transcriber.

        Args:
            api_key: Gemini API key (defaults to env var)
        """
        self.api_key = api_key or GEMINI_API_KEY
        if not self.api_key:
            raise ValueError("GEMINI_API_KEY not set")

        self.client = genai.Client(api_key=self.api_key)

    def _get_mime_type(self, file_path: str) -> str:
        """Determine MIME type from file extension."""
        ext = Path(file_path).suffix.lower()
        mime_types = {
            '.flac': 'audio/flac',
            '.wav': 'audio/wav',
            '.mp3': 'audio/mpeg',
            '.ogg': 'audio/ogg',
            '.m4a': 'audio/mp4',
            '.aac': 'audio/aac'
        }
        return mime_types.get(ext, 'audio/flac')

    def _load_audio_bytes(self, file_path: str) -> bytes:
        """Load audio file as bytes."""
        with open(file_path, 'rb') as f:
            return f.read()

    def _build_config(self, config: TranscriptionConfig) -> types.GenerateContentConfig:
        """Build the generation config for the API call."""
        model_name = get_model_name(config.model)
        model_info = GEMINI_MODELS.get(model_name, {})

        # Build thinking config for Gemini 3 models
        thinking_config = None
        if model_info.get("supports_thinking") and config.thinking_level:
            if model_name.startswith("gemini-3"):
                thinking_config = types.ThinkingConfig(
                    thinking_level=config.thinking_level.upper()
                )

        gen_config = types.GenerateContentConfig(
            temperature=config.temperature,
            response_mime_type="application/json",
            response_json_schema=TRANSCRIPTION_JSON_SCHEMA,
            system_instruction=[
                types.Part.from_text(text=get_transcription_prompt(config.language))
            ]
        )

        if thinking_config:
            gen_config.thinking_config = thinking_config

        return gen_config

    def transcribe_audio(
        self,
        audio_path: str,
        config: TranscriptionConfig
    ) -> Dict[str, Any]:
        """
        Transcribe a single audio file.

        Args:
            audio_path: Path to the audio file
            config: Transcription configuration

        Returns:
            Dictionary with transcription results
        """
        model_name = get_model_name(config.model)

        # Load audio
        audio_bytes = self._load_audio_bytes(audio_path)
        mime_type = self._get_mime_type(audio_path)

        # Build content
        contents = [
            types.Content(
                role="user",
                parts=[
                    types.Part.from_bytes(
                        mime_type=mime_type,
                        data=audio_bytes
                    ),
                    types.Part.from_text(text=get_user_prompt())
                ]
            )
        ]

        # Build config
        gen_config = self._build_config(config)

        # Make API call
        start_time = time.time()
        try:
            response = self.client.models.generate_content(
                model=model_name,
                contents=contents,
                config=gen_config
            )
            processing_time = time.time() - start_time

            # Parse response
            if response.text:
                import json
                result = json.loads(response.text)
                result['_processing_time_sec'] = processing_time
                result['_model'] = model_name
                result['_thinking_level'] = config.thinking_level
                return result
            else:
                return {
                    "error": "Empty response from model",
                    "_processing_time_sec": processing_time,
                    "_model": model_name
                }

        except Exception as e:
            processing_time = time.time() - start_time
            return {
                "error": str(e),
                "_processing_time_sec": processing_time,
                "_model": model_name
            }

    def transcribe_chunk(
        self,
        chunk: AudioChunk,
        config: TranscriptionConfig
    ) -> TranscriptionResult:
        """
        Transcribe an AudioChunk and return structured result.

        Args:
            chunk: AudioChunk to transcribe
            config: Transcription configuration

        Returns:
            TranscriptionResult with full metadata
        """
        raw_result = self.transcribe_audio(chunk.file_path, config)

        # Handle errors
        if "error" in raw_result:
            transcription = TranscriptionOutput(
                native_transcription=f"[ERROR: {raw_result['error']}]",
                native_with_punctuation=f"[ERROR: {raw_result['error']}]",
                code_switch=f"[ERROR: {raw_result['error']}]",
                romanized=f"[ERROR: {raw_result['error']}]",
                notes=raw_result['error']
            )
        else:
            transcription = TranscriptionOutput(
                native_transcription=raw_result.get('native_transcription', ''),
                native_with_punctuation=raw_result.get('native_with_punctuation', ''),
                code_switch=raw_result.get('code_switch', ''),
                romanized=raw_result.get('romanized', ''),
                confidence=raw_result.get('confidence'),
                notes=raw_result.get('notes')
            )

        return TranscriptionResult(
            segment_id=chunk.original_segment,
            chunk_index=chunk.chunk_index,
            total_chunks=chunk.total_chunks,
            duration_sec=chunk.duration_sec,
            language=config.language,
            transcription=transcription,
            model_used=raw_result.get('_model', config.model),
            thinking_level=raw_result.get('_thinking_level'),
            processing_time_sec=raw_result.get('_processing_time_sec')
        )

    def transcribe_batch(
        self,
        chunks: List[AudioChunk],
        config: TranscriptionConfig,
        max_chunks: Optional[int] = None,
        progress_callback: Optional[callable] = None
    ) -> List[TranscriptionResult]:
        """
        Transcribe multiple audio chunks.

        Args:
            chunks: List of AudioChunks to transcribe
            config: Transcription configuration
            max_chunks: Maximum chunks to process (for testing)
            progress_callback: Optional callback(current, total) for progress updates

        Returns:
            List of TranscriptionResults
        """
        if max_chunks:
            chunks = chunks[:max_chunks]

        results = []
        total = len(chunks)

        print(f"[Transcriber] Processing {total} chunks with {config.model}...")

        for i, chunk in enumerate(chunks):
            if progress_callback:
                progress_callback(i + 1, total)
            else:
                print(f"[Transcriber] Processing {i + 1}/{total}: {chunk.original_segment} "
                      f"(chunk {chunk.chunk_index + 1}/{chunk.total_chunks})")

            result = self.transcribe_chunk(chunk, config)
            results.append(result)

            # Brief pause to avoid rate limiting
            if i < total - 1:
                time.sleep(0.5)

        print(f"[Transcriber] Completed {len(results)} transcriptions")
        return results


sed -n '1,240p' src/backend/transcription_schema.py


Output:
Exit code: 0
Wall time: 0 seconds
Output:
"""
Pydantic schemas for structured transcription output.
Defines the four output formats required for Indian language transcription.
"""
from typing import Optional
from pydantic import BaseModel, Field

class TranscriptionOutput(BaseModel):
    """
    Structured output for a single audio segment transcription.
    Contains four different transcription formats as per requirements.
    """

    native_transcription: str = Field(
        description="Complete native script transcription without any punctuation. "
                    "English words written phonetically in native script. "
                    "All repetitions, fillers, stammers preserved exactly as spoken."
    )

    native_with_punctuation: str = Field(
        description="Native script transcription with minimal punctuation based on speech flow. "
                    "Uses period, comma, question mark, exclamation only. "
                    "No grammar correction or sentence restructuring."
    )

    code_switch: str = Field(
        description="Mixed script transcription preserving original language switching. "
                    "Indian language parts in native script, English words in Latin script. "
                    "Minimal punctuation following primary language convention."
    )

    romanized: str = Field(
        description="Complete Roman/Latin script transcription (transliteration). "
                    "All speech converted to romanized form as pronounced. "
                    "Preserves pronunciation exactly even if spelling looks incorrect."
    )

    confidence: Optional[float] = Field(
        default=None,
        description="Optional confidence score for the transcription (0-1)"
    )

    notes: Optional[str] = Field(
        default=None,
        description="Any notes about audio quality, unclear portions, or special observations"
    )

class TranscriptionResult(BaseModel):
    """Complete result for a transcribed segment including metadata."""

    segment_id: str = Field(description="Identifier for the audio segment")
    chunk_index: int = Field(default=0, description="Chunk index if segment was split")
    total_chunks: int = Field(default=1, description="Total chunks for this segment")
    duration_sec: float = Field(description="Duration of audio in seconds")
    language: str = Field(description="Primary language of the audio")

    transcription: TranscriptionOutput = Field(description="The transcription outputs")

    model_used: str = Field(description="Gemini model used for transcription")
    thinking_level: Optional[str] = Field(default=None, description="Thinking level used")
    processing_time_sec: Optional[float] = Field(default=None, description="API call time in seconds")

def get_transcription_prompt(language: str) -> str:
    """
    Generate the system instruction prompt for transcription.

    Args:
        language: The primary language of the audio

    Returns:
        System instruction string
    """
    return f"""You are a strict, verbatim transcription engine for Indian languages.

Primary audio language: {language}

TASK:
Listen to the audio carefully and transcribe the speech exactly as spoken.
Produce a JSON object with four transcription outputs.

STRICT GLOBAL RULES (apply to all outputs):
1. VERBATIM ONLY: Include all repetitions, filler words, stammers, false starts, hesitations, and colloquial expressions exactly as spoken
2. NO NORMALIZATION: Do not correct grammar, pronunciation, spellings, or clean up dialect/accent/mixed language usage
3. NO INFERENCE: Do not add meaning, structure, or emphasis that is not clearly audible
4. SCRIPT FIDELITY: Follow the rules of each output strictly

OUTPUT DEFINITIONS:

native_transcription:
- Write everything ONLY in the native script of {language}
- NO punctuation marks at all
- Even if English words are spoken, write them phonetically in {language} script
- Preserve ALL repetitions, fillers, stammers exactly

native_with_punctuation:
- Write everything ONLY in the native script of {language}
- Use MINIMAL punctuation strictly based on speech flow:
  * Period (।) for clear sentence endings
  * Comma (,) only for clearly audible pauses
  * Question mark (?) only when question intonation is clearly audible
  * Exclamation mark (!) only when strong emphasis is clearly audible
- NO ellipses, quotation marks, brackets, colons, semicolons, or multiple punctuation
- Do NOT correct grammar or restructure sentences

code_switch:
- Preserve the original language switching exactly as spoken
- Write Indian language parts in their native script
- Write English words and phrases in English (Latin script) if spoken that way
- Do NOT translate, normalize, or rewrite across languages
- Use minimal punctuation based on audible cues

romanized:
- Write the ENTIRE transcription in Roman/Latin script only
- Convert all {language} speech into Romanized form as it sounds
- Preserve pronunciation exactly as spoken even if spelling looks incorrect
- Preserve ALL repetitions, fillers, stammers, false starts
- Do NOT standardize spellings or make them grammatically correct
- Use minimal punctuation only if clearly audible

If audio is unclear, transcribe what you can hear. Use the 'notes' field for quality issues.
If completely inaudible, set all transcription fields to "[INAUDIBLE]"."""

def get_user_prompt() -> str:
    """Get the user prompt to accompany the audio."""
    return "Transcribe this audio segment following the system instructions. Return a valid JSON object with the four transcription outputs."

# JSON Schema for Gemini's structured output
TRANSCRIPTION_JSON_SCHEMA = {
    "type": "object",
    "properties": {
        "native_transcription": {
            "type": "string",
            "description": "Native script transcription without punctuation"
        },
        "native_with_punctuation": {
            "type": "string",
            "description": "Native script transcription with minimal punctuation"
        },
        "code_switch": {
            "type": "string",
            "description": "Mixed script preserving language switching"
        },
        "romanized": {
            "type": "string",
            "description": "Complete Roman script transliteration"
        },
        "confidence": {
            "type": "number",
            "description": "Confidence score 0-1 (optional)"
        },
        "notes": {
            "type": "string",
            "description": "Notes about audio quality or unclear portions (optional)"
        }
    },
    "required": ["native_transcription", "native_with_punctuation", "code_switch", "romanized"]
}


**Identifying missing audio processor**


sed -n '1,260p' src/backend/audio_processor.py


Output:
Exit code: 0
Wall time: 0 seconds
Output:
"""
Audio processing module for handling segment duration and chunking.
Provides controls for splitting long segments to meet the 10s hard limit.
"""
import os
import math
from pathlib import Path
from typing import List, Dict, Optional
from dataclasses import dataclass
from pydub import AudioSegment

@dataclass
class AudioChunk:
    """Represents an audio chunk ready for transcription."""
    file_path: str
    original_segment: str  # Original segment filename
    chunk_index: int       # Index within the segment (0 if not split)
    total_chunks: int      # Total chunks for this segment
    start_ms: int          # Start time in milliseconds within original
    end_ms: int            # End time in milliseconds within original
    duration_sec: float    # Duration in seconds

    @property
    def is_split(self) -> bool:
        return self.total_chunks > 1

class AudioProcessor:
    """Handles audio segment processing with duration controls."""

    def __init__(
        self,
        max_duration_sec: float = 10.0,
        min_duration_sec: float = 1.0,
        overlap_sec: float = 0.0,
        output_format: str = "flac"
    ):
        """
        Initialize the audio processor.

        Args:
            max_duration_sec: Maximum allowed segment duration (hard limit)
            min_duration_sec: Minimum segment duration to process
            overlap_sec: Overlap between chunks when splitting (helps with word boundaries)
            output_format: Output format for split chunks (flac, wav, mp3)
        """
        self.max_duration_sec = max_duration_sec
        self.min_duration_sec = min_duration_sec
        self.overlap_sec = overlap_sec
        self.output_format = output_format

    def get_audio_duration(self, file_path: str) -> float:
        """Get duration of an audio file in seconds."""
        audio = AudioSegment.from_file(file_path)
        return len(audio) / 1000.0

    def needs_splitting(self, file_path: str) -> bool:
        """Check if a file needs to be split based on duration."""
        duration = self.get_audio_duration(file_path)
        return duration > self.max_duration_sec

    def split_audio(
        self,
        file_path: str,
        output_dir: str,
        chunk_duration_sec: Optional[float] = None
    ) -> List[AudioChunk]:
        """
        Split an audio file into chunks of specified duration.

        Args:
            file_path: Path to the audio file
            output_dir: Directory to save chunks
            chunk_duration_sec: Target chunk duration (defaults to max_duration_sec)

        Returns:
            List of AudioChunk objects
        """
        if chunk_duration_sec is None:
            chunk_duration_sec = self.max_duration_sec

        audio = AudioSegment.from_file(file_path)
        total_duration_ms = len(audio)
        total_duration_sec = total_duration_ms / 1000.0

        os.makedirs(output_dir, exist_ok=True)
        original_name = Path(file_path).stem

        # If already short enough, return as single chunk
        if total_duration_sec <= self.max_duration_sec:
            return [AudioChunk(
                file_path=file_path,
                original_segment=Path(file_path).name,
                chunk_index=0,
                total_chunks=1,
                start_ms=0,
                end_ms=total_duration_ms,
                duration_sec=total_duration_sec
            )]

        chunks = []
        chunk_duration_ms = int(chunk_duration_sec * 1000)
        overlap_ms = int(self.overlap_sec * 1000)

        # Calculate number of chunks
        effective_chunk_duration = chunk_duration_ms - overlap_ms
        num_chunks = math.ceil((total_duration_ms - overlap_ms) / effective_chunk_duration)

        for i in range(num_chunks):
            start_ms = i * effective_chunk_duration
            end_ms = min(start_ms + chunk_duration_ms, total_duration_ms)

            # Extract chunk
            chunk_audio = audio[start_ms:end_ms]

            # Skip very short chunks
            chunk_duration = len(chunk_audio) / 1000.0
            if chunk_duration < self.min_duration_sec and i > 0:
                # Extend previous chunk instead of creating tiny one
                continue

            # Save chunk
            chunk_filename = f"{original_name}_chunk{i:03d}.{self.output_format}"
            chunk_path = os.path.join(output_dir, chunk_filename)
            chunk_audio.export(chunk_path, format=self.output_format)

            chunks.append(AudioChunk(
                file_path=chunk_path,
                original_segment=Path(file_path).name,
                chunk_index=i,
                total_chunks=num_chunks,
                start_ms=start_ms,
                end_ms=end_ms,
                duration_sec=chunk_duration
            ))

        return chunks

    def process_segments_directory(
        self,
        segments_dir: str,
        output_dir: Optional[str] = None,
        max_segments: Optional[int] = None,
        skip_short: bool = True
    ) -> List[AudioChunk]:
        """
        Process all segments in a directory, splitting as needed.

        Args:
            segments_dir: Directory containing audio segments
            output_dir: Directory for split chunks (default: segments_dir/chunks)
            max_segments: Maximum number of segments to process (for testing)
            skip_short: Skip segments shorter than min_duration_sec

        Returns:
            List of AudioChunk objects ready for transcription
        """
        if output_dir is None:
            output_dir = os.path.join(segments_dir, "chunks")

        # Find all audio files
        audio_extensions = {'.flac', '.wav', '.mp3', '.ogg', '.m4a'}
        segment_files = []

        for ext in audio_extensions:
            segment_files.extend(Path(segments_dir).glob(f"*{ext}"))

        # Sort by filename for consistent ordering
        segment_files = sorted(segment_files, key=lambda x: x.name)

        if max_segments:
            segment_files = segment_files[:max_segments]

        print(f"[Audio] Processing {len(segment_files)} segments...")

        all_chunks = []
        split_count = 0
        skipped_count = 0

        for segment_path in segment_files:
            duration = self.get_audio_duration(str(segment_path))

            # Skip very short segments
            if skip_short and duration < self.min_duration_sec:
                skipped_count += 1
                continue

            # Split if needed
            if duration > self.max_duration_sec:
                chunks = self.split_audio(str(segment_path), output_dir)
                split_count += 1
                print(f"[Audio] Split {segment_path.name}: {duration:.1f}s -> {len(chunks)} chunks")
            else:
                # Use original file
                chunks = [AudioChunk(
                    file_path=str(segment_path),
                    original_segment=segment_path.name,
                    chunk_index=0,
                    total_chunks=1,
                    start_ms=0,
                    end_ms=int(duration * 1000),
                    duration_sec=duration
                )]

            all_chunks.extend(chunks)

        print(f"[Audio] Processed {len(segment_files)} segments -> {len(all_chunks)} chunks")
        print(f"[Audio] Split: {split_count}, Skipped (too short): {skipped_count}")

        return all_chunks

def get_segment_stats(segments_dir: str) -> Dict:
    """
    Get statistics about segments in a directory.

    Args:
        segments_dir: Directory containing audio segments

    Returns:
        Dictionary with duration statistics
    """
    processor = AudioProcessor()

    audio_extensions = {'.flac', '.wav', '.mp3', '.ogg', '.m4a'}
    durations = []

    for ext in audio_extensions:
        for f in Path(segments_dir).glob(f"*{ext}"):
            try:
                durations.append(processor.get_audio_duration(str(f)))
            except Exception as e:
                print(f"Error reading {f}: {e}")

    if not durations:
        return {"error": "No audio files found"}

    return {
        "total_files": len(durations),
        "total_duration_sec": sum(durations),
        "avg_duration_sec": sum(durations) / len(durations),
        "min_duration_sec": min(durations),
        "max_duration_sec": max(durations),
        "over_10s_count": sum(1 for d in durations if d > 10),
        "under_1s_count": sum(1 for d in durations if d < 1)
    }

if __name__ == "__main__":
    # Test with a sample directory
    import sys
    if len(sys.argv) > 1:
        segments_dir = sys.argv[1]
        stats = get_segment_stats(segments_dir)
        print("\nSegment Statistics:")
        for key, value in stats.items():
            print(f"  {key}: {value}")


**Inspecting transcription samples**

I'm exploring sample files in the transcriptions directory to understand the data structure and assess if it's useful for inferring the schema.


Output:
Exit code: 0
Wall time: 0 seconds
Output:
pF_BQpHaIdU_gemini_2.5_flash_20260203_225339.json
pF_BQpHaIdU_gemini_2.5_flash_20260203_225339_partial.json
pF_BQpHaIdU_gemini_3_flash_preview_20260203_225233.json
pF_BQpHaIdU_gemini_3_flash_preview_20260203_225233_partial.json
pF_BQpHaIdU_gemini_3_flash_preview_20260203_230246.json
pF_BQpHaIdU_gemini_3_flash_preview_20260203_230246_partial.json


sed -n '1,200p' transcriptions/pF_BQpHaIdU_gemini_3_flash_preview_20260203_225233.json


Output:
Exit code: 0
Wall time: 0 seconds
Output:
{
  "video_id": "pF_BQpHaIdU",
  "model": "gemini-3-flash-preview",
  "thinking_level": "high",
  "language": "Telugu",
  "config": {
    "max_segment_duration_sec": 10.0,
    "min_segment_duration_sec": 1.0,
    "max_segments": 3
  },
  "results_count": 4,
  "total_duration_sec": 24.819000000000003,
  "results": [
    {
      "segment_id": "SPEAKER_00_0000_0.03-2.61.flac",
      "chunk_index": 0,
      "total_chunks": 1,
      "duration_sec": 2.582,
      "language": "Telugu",
      "transcription": {
        "native_transcription": "నాకు కొన్ని యాడ్స్ గుర్తుంటాయ్ లైక్ ఎగ్జాంపుల్ కచ్చా మ్యాంగో బైట్",
        "native_with_punctuation": "నాకు కొన్ని యాడ్స్ గుర్తుంటాయ్, లైక్ ఎగ్జాంపుల్ కచ్చా మ్యాంగో బైట్।",
        "code_switch": "నాకు కొన్ని ads గుర్తుంటాయ్ like example కచ్చా mango bite.",
        "romanized": "naku konni ads gurtuntay like example kacha mango bite.",
        "confidence": null,
        "notes": null
      },
      "model_used": "gemini-3-flash-preview",
      "thinking_level": "high",
      "processing_time_sec": 12.866390705108643
    },
    {
      "segment_id": "SPEAKER_00_0001_3.05-13.40.flac",
      "chunk_index": 0,
      "total_chunks": 2,
      "duration_sec": 10.0,
      "language": "Telugu",
      "transcription": {
        "native_transcription": "ఆ కచ్చితంగా ఒక బ ఒక అబ్బాయి అయితే మా వాడు ఒకడున్నాడు ఐ విల్ నాట్ నేమ్ హిమ్ బట్ మరేంటంటే మ్యాంగోస్ తీసుకుని జిరాక్స్ షాప్ కి వెళ్ళిపోయాడు అన్నమాట ఇన్ ఫ్యాక్ట్ నాకు ఫోటో కాపీ అంటే వేరు జిరాక్స్ అంటే వేరు అని తెలిసింది అక్కడే",
        "native_with_punctuation": "ఆ కచ్చితంగా ఒక బ, ఒక అబ్బాయి అయితే మా వాడు ఒకడున్నాడు। ఐ విల్ నాట్ నేమ్ హిమ్, బట్ మరేంటంటే మ్యాంగోస్ తీసుకుని జిరాక్స్ షాప్ కి వెళ్ళిపోయాడు అన్నమాట। ఇన్ ఫ్యాక్ట్ నాకు ఫోటో కాపీ అంటే వేరు, జిరాక్స్ అంటే వేరు అని తెలిసింది అక్కడే।",
        "code_switch": "ఆ కచ్చితంగా ఒక బ, ఒక అబ్బాయి అయితే మా వాడు ఒకడున్నాడు। I'll not name him, but మరేంటంటే mangoes తీసుకుని Xerox shop కి వెళ్ళిపోయాడు అన్నమాట। In fact నాకు photo copy అంటే వేరు, Xerox అంటే వేరు అని తెలిసింది అక్కడే।",
        "romanized": "aa kachitangaa oka ba, oka abbayi aithe maa vaadu okadunnaadu. I'll not name him, but marentante mangoes theesukuni Xerox shop ki vellipoyaadu annamaata. In fact naaku photo copy ante veru, Xerox ante veru ani thelisindi akkade.",
        "confidence": 0.95,
        "notes": null
      },
      "model_used": "gemini-3-flash-preview",
      "thinking_level": "high",
      "processing_time_sec": 34.01047658920288
    },
    {
      "segment_id": "SPEAKER_00_0002_16.08-28.31.flac",
      "chunk_index": 0,
      "total_chunks": 2,
      "duration_sec": 10.0,
      "language": "Telugu",
      "transcription": {
        "native_transcription": "అదేంటది ప్రాసెస్ అని చెప్పి కరెక్ట్ వాడు రెండు మ్యాంగోస్ తీసుకెళ్లి అంటే క్యూరియాసిటీ అవును అవును తెలియకపోవడం కాదు అక్కడికెళ్ళి అరే ఇది ఇస్తే ఏ నిజంగా ఏమంటాడో చూద్దాం అవును అంత కనెక్ట్ అయిపోయారంటే ఇట్స్ లైక్",
        "native_with_punctuation": "అదేంటది ప్రాసెస్ అని చెప్పి। కరెక్ట్। వాడు రెండు మ్యాంగోస్ తీసుకెళ్లి అంటే క్యూరియాసిటీ। అవును అవును। తెలియకపోవడం కాదు అక్కడికెళ్ళి అరే ఇది ఇస్తే ఏ నిజంగా ఏమంటాడో చూద్దాం। అవును। అంత కనెక్ట్ అయిపోయారంటే ఇట్స్ లైక్।",
        "code_switch": "అదేంటది process అని చెప్పి। Correct। వాడు రెండు mangoes తీసుకెళ్లి అంటే curiosity। అవును అవును। తెలియకపోవడం కాదు అక్కడికెళ్ళి అరే ఇది ఇస్తే ఏ నిజంగా ఏమంటాడో చూద్దాం। అవును। అంత connect అయిపోయారంటే it's like।",
        "romanized": "adentadi process ani cheppi. correct. vaadu rendu mangoes theeskelli ante curiosity. avunu avunu. theliyakapovadam kaadu akkadikelli arey idi isthe ey nijanga emantado chuddam. avunu. antha connect ayipoyarante it's like.",
        "confidence": null,
        "notes": null
      },
      "model_used": "gemini-3-flash-preview",
      "thinking_level": "high",
      "processing_time_sec": 40.689393520355225
    },
    {
      "segment_id": "SPEAKER_00_0002_16.08-28.31.flac",
      "chunk_index": 1,
      "total_chunks": 2,
      "duration_sec": 2.237,
      "language": "Telugu",
      "transcription": {
        "native_transcription": "ఎవరైతే చూసుంటారో వాళ్ళకర్థమయ్యుంటది",
        "native_with_punctuation": "ఎవరైతే చూసుంటారో వాళ్ళకర్థమయ్యుంటది।",
        "code_switch": "ఎవరైతే చూసుంటారో వాళ్ళకర్థమయ్యుంటది",
        "romanized": "evaraithe choosuntaaro vallakarthamayyuntadi",
        "confidence": null,
        "notes": null
      },
      "model_used": "gemini-3-flash-preview",
      "thinking_level": "high",
      "processing_time_sec": 13.697313785552979
    }
  ]
}


**Proposing validation and scoring pipeline**

I'll suggest a 10-step validation and scoring pipeline after transcription, integrating forced alignment, confidence scoring, language/script checks, audio quality metrics, ensemble ASR comparisons, and GPT-based critiques to flag low-confidence segments for reprocessing or prompt refinement.


**Detailing comprehensive validation and scoring pipeline**

I'm outlining a 10-step pipeline post-transcription that includes audio quality checks, JSON schema and script validation, language/script ratio analysis, cross-model consistency with forced alignment and alternative ASR comparison, confidence estimation via alignment scores and ensemble agreement, and fluency metrics, culminating in flagging low-quality segments for reprocessing or human review with triage thresholds to ensure robust transcription quality assessment.


Output:
Exit code: 0
Wall time: 0 seconds
Output:
# Maya3 Transcription Pipeline

Audio transcription pipeline for Indian languages using Google Gemini AI models.

## Features

- **R2 Cloud Storage Integration**: Downloads audio tar files from Cloudflare R2
- **Supabase Integration**: Fetches video language metadata from database
- **Audio Processing**: Handles segment splitting with configurable duration limits
- **Multi-Model Support**: Works with Gemini 3 Pro, Flash, and 2.x models
- **Structured Output**: Four transcription formats per segment
- **Full Parameter Control**: Fine-grained control over all pipeline aspects

## Setup

```bash
# Create virtual environment
python3 -m venv venv
source venv/bin/activate

# Install dependencies
pip install -r requirements.txt

# Ensure .env file has required credentials:
# - R2_ENDPOINT_URL, R2_BUCKET, R2_ACCESS_KEY_ID, R2_SECRET_ACCESS_KEY
# - GEMINI_KEY
# - URL, SUPABASE_ADMIN (Supabase credentials)
```

## Quick Start

```bash
# Basic usage - process a video
python pipeline.py pF_BQpHaIdU --language Telugu

# Limit segments for testing
python pipeline.py pF_BQpHaIdU -n 5 --language Telugu

# Use different model
python pipeline.py pF_BQpHaIdU -m gemini-3-pro-preview -t high -n 3
```

## Pipeline Configuration

### Audio Processing Controls

| Parameter | Default | Description |
|-----------|---------|-------------|
| `max_segment_duration_sec` | 10.0 | Hard limit - segments longer than this are split |
| `min_segment_duration_sec` | 1.0 | Skip segments shorter than this |
| `chunk_overlap_sec` | 0.0 | Overlap when splitting (helps word boundaries) |

### Segment Selection Controls

| Parameter | Default | Description |
|-----------|---------|-------------|
| `max_segments` | None | Limit total segments processed (for testing) |
| `segment_start_index` | 0 | Start from this segment index |
| `skip_short_segments` | True | Skip segments < min_duration |

### Model Selection

| Model | Thinking Levels | Cost Tier |
|-------|----------------|-----------|
| `gemini-3-pro-preview` | low, high | Premium |
| `gemini-3-flash-preview` | minimal, low, medium, high | Standard |
| `gemini-2.5-pro` | N/A (budget-based) | Premium |
| `gemini-2.5-flash` | N/A (budget-based) | Standard |
| `gemini-2.5-flash-lite` | N/A (budget-based) | Lite |
| `gemini-2.0-flash` | N/A | Standard |

## Output Format

Each segment produces four transcription formats:

1. **native_transcription**: Pure native script, no punctuation
2. **native_with_punctuation**: Native script with minimal punctuation
3. **code_switch**: Mixed script preserving language switching
4. **romanized**: Complete Roman/Latin script transliteration

Example output structure:
```json
{
  "video_id": "pF_BQpHaIdU",
  "model": "gemini-3-flash-preview",
  "thinking_level": "high",
  "language": "Telugu",
  "results": [
    {
      "segment_id": "segment_001.flac",
      "chunk_index": 0,
      "duration_sec": 8.5,
      "transcription": {
        "native_transcription": "...",
        "native_with_punctuation": "...",
        "code_switch": "...",
        "romanized": "..."
      },
      "model_used": "gemini-3-flash-preview",
      "processing_time_sec": 2.3
    }
  ]
}
```

## Python API

```python
from pipeline import run_pipeline, PipelineConfig, TranscriptionPipeline

# Simple usage
result = run_pipeline(
    video_id="pF_BQpHaIdU",
    language="Telugu",
    model="gemini-3-flash-preview",
    thinking_level="high",
    max_segments=5
)

# Full control with config
config = PipelineConfig(
    video_id="pF_BQpHaIdU",
    language="Telugu",
    max_segment_duration_sec=10.0,
    min_segment_duration_sec=1.0,
    max_segments=10,
    model="gemini-3-flash-preview",
    thinking_level="high",
    save_intermediate=True,
    batch_size=5
)

pipeline = TranscriptionPipeline(config)
result = pipeline.run()

# Direct access to backend modules
from src.backend import (
    download_video_segments,
    get_video_language,
    AudioProcessor,
    GeminiTranscriber,
    TranscriptionConfig
)
```

## Testing Multiple Models

```python
from pipeline import test_models

# Compare models on same video
results = test_models(
    video_id="pF_BQpHaIdU",
    language="Telugu",
    models=["gemini-3-flash-preview", "gemini-3-pro-preview"],
    thinking_levels=["high"],
    max_segments=3
)

for model, result in results.items():
    print(f"{model}: {result.total_processing_time_sec:.1f}s")
```

## Module Structure

```
maya3_transcribe/
├── pipeline.py              # Main orchestration (entry point)
├── requirements.txt         # Dependencies
├── README.md
├── .env                     # Credentials (R2, Gemini, Supabase)
├── src/
│   └── backend/
│       ├── __init__.py      # Package exports
│       ├── config.py        # Configuration and environment loading
│       ├── r2_storage.py    # R2 cloud storage client
│       ├── supabase_client.py # Supabase client for metadata
│       ├── audio_processor.py # Audio segmentation and chunking
│       ├── transcription_schema.py # Pydantic schemas for output
│       └── gemini_transcriber.py # Gemini API integration
├── transcriptions/          # Output directory
└── docs/                    # Documentation
```

## Progress Log

### 2026-02-03
- Initial pipeline implementation
- R2 download module
- Supabase language lookup
- Audio processing with 10s hard limit
- Gemini transcription with structured output
- Multi-model support (Gemini 3 Pro/Flash, 2.x series)
- Tested successfully with video pF_BQpHaIdU (Telugu)
- Both gemini-3-flash-preview and gemini-2.5-flash working correctly
- Refactored into src/backend package structure
- Pipeline orchestrator remains at project root


**Proposing pipeline validation and scoring stages**

Suggest adding a "Validation & Scoring" stage after current steps to check each transcription result for errors, format correctness (using script-specific regex), and length consistency, then flag invalid outputs before optionally requeuing them.


**Proposing Validation & Scoring Pipeline**

I read that the current pipeline downloads, splits, and transcribes audio with Gemini, outputting JSON with optional confidence but no validation. Here’s a recommended validation and scoring pipeline to add after transcription:
- Step 1: Audio QC — check average energy, VAD speech percentage, silence detection to flag low-quality audio.
- Step 2: JSON/schema validation — verify required fields present and no errors like "INAUDIBLE".
- Step 3: Format compliance — enforce script and punctuation rules per output type using regex (e.g., Telugu Unicode block for native, ASCII for romanized).
- Step 4: Length and cross-field consistency — check token rate vs duration, detect duplication across fields, and approximate transliteration matches.
- Step 5: Confidence estimation — include forced alignment scores (CTC log-probabilities using open-source models), consensus WER/CER from alternative ASR models, and n-best variability.
- Step 6: Score aggregation — compute weighted quality_score combining above features plus flags for hard failures or warnings.
- Step 7: Triage — apply thresholds to decide if output is accepted, re-transcribed (with improved prompt or different model), or sent for manual review, recording reasons and flags in metadata.

**Additional Validation Ideas**
- Use language ID on tokens to confirm plausible code-switch ratios.
- Detect hallucinated content like unexpected "notes" or disclaimers.
- Run an optional "Verifier" LLM for second-pass review with a different model to avoid bias.
- Track segment-level metadata and handle chunk boundary overlaps to reduce duplication or missing content.
- Assess repetition or degeneracy in outputs with n-gram checks.
- For dataset-level performance, create a gold-standard test set and measure WER/CER to calibrate quality_score thresholds over time.

This setup ensures low-confidence or error-prone transcriptions get caught and improved systematically while integrating smoothly with the existing pipeline outputs and architecture.


Got it. I read the current pipeline: `pipeline.py` downloads R2 segments → fetches language → splits audio to <=10s → Gemini transcription → JSON output with 4 formats + optional `confidence`/`notes`. There’s no validation/scoring stage yet.

Here’s a proposed validation + scoring pipeline you can insert after transcription (and before final save / downstream use).

**Validation & Scoring Pipeline**
- Step 1 — Pre‑audio QC (before reprocessing decisions): compute VAD speech %, SNR, clipping %, silence %, duration sanity; flag segments with low speech or heavy noise.
- Step 2 — Schema + hard rules: JSON schema, required fields, non‑empty strings, no `[ERROR]`/`[INAUDIBLE]` everywhere; if fail → immediate requeue.
- Step 3 — Format compliance checks (per field):
  - `native_transcription`: only native script; no punctuation.
  - `native_with_punctuation`: only native script; punctuation limited to `। , ? !` (and minimal count).
  - `code_switch`: native + Latin mix allowed; reject other scripts.
  - `romanized`: ASCII/Latin only; no native script chars.
- Step 4 — Length/tempo heuristics: chars/sec or words/sec bounds; extremely high or low density → low score.
- Step 5 — Consistency checks across fields:
  - Native vs romanized approximate transliteration similarity (phonetic distance).
  - code_switch vs native: English tokens should appear in code_switch but not in native.
  - Detect degenerate outputs (same string in all 4 fields).
- Step 6 — External confidence via alignment/consensus:
  - Forced aligner (CTC‑based or word‑level) to compute alignment coverage + average log‑prob.
  - Independent ASR pass (Whisper/W2V2/etc) and compute CER/WER vs romanized or code_switch.
- Step 7 — Aggregate score:
  - `quality_score = w1*alignment + w2*consensus + w3*format + w4*tempo + w5*audio_QC - penalties`.
  - Store component scores + flags so you can explain why a segment was rejected.
- Step 8 — Triage rules:
  - Hard‑fail: schema/format errors, `[ERROR]`, almost‑silent audio.
  - Low score → re‑transcribe with stronger model or alternate prompt.
  - Mid score → retry with overlap or noise‑reduction.
  - High score → accept.

**Suggested scoring signals (quick list)**
- Audio: VAD speech %, SNR, clipping ratio, max amplitude saturation.
- Transcript structure: JSON validity, required fields present, no placeholders.
- Script rules: Unicode‑block validation per field.
- Speed sanity: chars/sec or syllables/sec.
- Alignment: % tokens aligned, avg CTC log‑prob, longest unaligned span.
- Consensus: WER/CER vs independent ASR.
- Variance: if 2–3 stochastic runs disagree strongly, penalize.

**What else can we do as validation steps (ideate)**
- Boundary checks on split chunks: if VAD shows speech right at start/end, likely cut mid‑word; mark low.
- Duplicate detection: identical transcription across different segments → likely hallucination or caching.
- Language ID per token: ensure Telugu dominates native fields; ensure English tokens appear only in code_switch.
- Lexicon sanity (optional): if romanized output is too “dictionary‑like” or too random, flag.
- Calibrate with a small gold set: hand‑transcribe ~1–2% of segments; use WER/CER to set thresholds and map `quality_score` to expected error.

If you want, I can sketch a concrete scoring schema + thresholds and where to hook it into `pipeline.py`. Then we can move to prompt improvements and re‑transcription strategy.


check the validations i have added sofar.  /home/ubuntu/maya3_transcribe/analysis_results/final_analysis.json

what do you think. of the current pipeline and usage of gemini models. priority is minimizing hallucination by increasing determinism and maintaining speed and efficiency in token usage and transcription. and also check the indicMFA usage and IndicConformer usage as well. need your opnion on current pipeline. and then i'll have a few concerns that needs to be designed first, then implement before testing. but sofar <10s samples, what do you think.


**Planning file read and usage check**

I'm preparing to read a file with `cat` and then search the repo for usage of indicMFA/IndicConformer with ripgrep to form an informed opinion on their integration.


sed -n '1,260p' analysis_results/final_analysis.json


Output:
Exit code: 0
Wall time: 0 seconds
Output:
{
  "metadata": {
    "video_id": "pF_BQpHaIdU",
    "language": "Telugu",
    "total_segments": 20,
    "models_tested": [
      "gemini-3-pro-preview",
      "gemini-3-flash-preview",
      "gemini-2.5-pro",
      "gemini-2.5-flash",
      "gemini-2.5-flash-lite"
    ],
    "configs_tested": [
      "temp1_high",
      "temp0_low"
    ],
    "created_at": "2026-02-05T00:29:19.723735",
    "notes": {
      "temp1_high": "Temperature=1.0 with thinking_level=high (Gemini 3) or default (2.5)",
      "temp0_low": "Temperature=0.0 with thinking_level=low (Gemini 3) or default (2.5)",
      "thinking_budget": "For Gemini 3 with temp=0, use thinking_budget=300 to prevent loops"
    }
  },
  "summary": {
    "speed_ranking": [
      {
        "model": "gemini-3-flash-preview",
        "temp1_high_avg_sec": 14.27,
        "temp0_low_avg_sec": 2.8
      },
      {
        "model": "gemini-2.5-flash",
        "temp1_high_avg_sec": 4.48,
        "temp0_low_avg_sec": 3.71
      },
      {
        "model": "gemini-3-pro-preview",
        "temp1_high_avg_sec": 30.17,
        "temp0_low_avg_sec": 6.76
      },
      {
        "model": "gemini-2.5-flash-lite",
        "temp1_high_avg_sec": 1.85,
        "temp0_low_avg_sec": 8.1
      },
      {
        "model": "gemini-2.5-pro",
        "temp1_high_avg_sec": 9.26,
        "temp0_low_avg_sec": 8.36
      }
    ],
    "recommendations": {
      "high_volume_production": "gemini-2.5-flash-lite (fastest, 1.5-2s avg)",
      "balanced_quality_speed": "gemini-2.5-flash (good quality, 3-4s avg)",
      "maximum_quality": "gemini-3-pro-preview (best quality, use thinking_budget=300 with temp=0)",
      "settings_notes": {
        "gemini_3_models": "Use temp=0 with thinking_budget=300 to prevent loops",
        "gemini_2.5_models": "temp=0 works fine, no thinking_budget needed"
      }
    }
  },
  "segments": [
    {
      "segment_id": "SPEAKER_00_0000_0.03-2.61.flac",
      "audio_path": "/tmp/maya3_transcribe/pF_BQpHaIdU/extracted/pF_BQpHaIdU/segments/SPEAKER_00_0000_0.03-2.61.flac",
      "duration_sec": 2.582,
      "models": {
        "gemini-3-pro-preview": {
          "temp1_high": {
            "native": "వస్తాదన్నమాట నాకు కొన్ని యాడ్స్ గుర్తుండేవి లైక్ ఎగ్జాంపుల్ కచ్చ మ్యాంగో బైట్",
            "punctuated": "వస్తాదన్నమాట. నాకు కొన్ని యాడ్స్ గుర్తుండేవి, లైక్ ఎగ్జాంపుల్ కచ్చ మ్యాంగో బైట్.",
            "code_switch": "వస్తాదన్నమాట. నాకు కొన్ని ads గుర్తుండేవి like example kaccha mango bite.",
            "romanized": "vasthadhannamata. naaku konni ads gurthundevi like example kaccha mango bite.",
            "time_sec": 20.036017417907715
          },
          "temp0_low": {
            "native": "నాకు కొన్ని యాడ్స్ గుర్తుండేవి లైక్ ఎగ్జాంపుల్ కచ్చా మ్యాంగో బైట్",
            "punctuated": "నాకు కొన్ని యాడ్స్ గుర్తుండేవి, లైక్ ఎగ్జాంపుల్ కచ్చా మ్యాంగో బైట్.",
            "code_switch": "నాకు కొన్ని ads గుర్తుండేవి, like example Kachcha Mango Bite.",
            "romanized": "naaku konni ads gurthundevi, like example kachcha mango bite.",
            "time_sec": 6.039599418640137
          }
        },
        "gemini-3-flash-preview": {
          "temp1_high": {
            "native": "నాకు కొన్ని యాడ్స్ గుర్తుంటాయి లైక్ ఎగ్జాంపుల్ కచ్చా మ్యాంగో బైట్",
            "punctuated": "నాకు కొన్ని యాడ్స్ గుర్తుంటాయి, లైక్ ఎగ్జాంపుల్ కచ్చా మ్యాంగో బైట్।",
            "code_switch": "నాకు కొన్ని ads గుర్తుంటాయి like example kaccha mango bite",
            "romanized": "naku konni ads gurtuntayi like example kaccha mango bite",
            "time_sec": 14.19179916381836
          },
          "temp0_low": {
            "native": "నాకు కొన్ని యాడ్స్ గుర్తుండేవి లైక్ ఎగ్జాంపుల్ కచ్చా మ్యాంగో బైట్",
            "punctuated": "నాకు కొన్ని యాడ్స్ గుర్తుండేవి, లైక్ ఎగ్జాంపుల్ కచ్చా మ్యాంగో బైట్.",
            "code_switch": "నాకు కొన్ని ads గుర్తుండేవి like example కచ్చా mango bite.",
            "romanized": "naaku konni ads gurtundevi like example kaccha mango bite",
            "time_sec": 4.253249168395996
          }
        },
        "gemini-2.5-pro": {
          "temp1_high": {
            "native": "అన్నమాట నాకు కొన్ని యాడ్స్ గుర్తుంటాయి లైక్ ఫర్ ఎక్సామ్పుల్ కచ్చా బాదం",
            "punctuated": "అన్నమాట, నాకు కొన్ని యాడ్స్ గుర్తుంటాయి, లైక్ ఫర్ ఎక్సామ్పుల్ కచ్చా బాదం।",
            "code_switch": "అన్నమాట, నాకు కొన్ని ads గుర్తుంటాయి, like for example కచ్చా బాదం।",
            "romanized": "annamaata, naaku konni ads gurtuntaayi, like for example kacha badam.",
            "time_sec": 10.94771671295166
          },
          "temp0_low": {
            "native": "నాకు కొన్ని యాడ్స్ గుర్తుంటాయి లైక్ ఎగ్జాంపుల్ కచ్చా బాదం",
            "punctuated": "నాకు కొన్ని యాడ్స్ గుర్తుంటాయి, లైక్ ఎగ్జాంపుల్ కచ్చా బాదం।",
            "code_switch": "నాకు కొన్ని ads గుర్తుంటాయి, like example కచ్చా బాదం।",
            "romanized": "naaku konni ads gurtuntaayi, like example kachcha badam.",
            "time_sec": 5.625661134719849
          }
        },
        "gemini-2.5-flash": {
          "temp1_high": {
            "native": "అన్నమాట నా కొన్ని యాడ్స్ గుర్తొస్తాయి లైక్ ఎగ్జాంపుల్ కచ్చా",
            "punctuated": "అన్నమాట. నా కొన్ని యాడ్స్ గుర్తొస్తాయి లైక్, ఎగ్జాంపుల్ కచ్చా.",
            "code_switch": "అన్నమాట. నా కొన్ని ads గుర్తొస్తాయి like, example కచ్చా.",
            "romanized": "annamaata. naa konni ads gurthostayi like, example kaccha.",
            "time_sec": 7.380277395248413
          },
          "temp0_low": {
            "native": "అన్నమాట నాకు కొన్ని యాడ్స్ గుర్తొస్తాయి లైక్ ఎగ్జాంపుల్ కచ్చా",
            "punctuated": "అన్నమాట. నాకు కొన్ని యాడ్స్ గుర్తొస్తాయి లైక్ ఎగ్జాంపుల్ కచ్చా.",
            "code_switch": "అన్నమాట నాకు కొన్ని ads గుర్తొస్తాయి like example కచ్చా",
            "romanized": "annamaata naaku konni ads gurthosthaayi like example kachchaa",
            "time_sec": 3.615159034729004
          }
        },
        "gemini-2.5-flash-lite": {
          "temp1_high": {
            "native": "అన్నమాట నా కొన్ని యాడ్స్ గుర్తుంటాయి లైక్ ఉదాహరణకు కచ్చా",
            "punctuated": "అన్నమాట। నా కొన్ని యాడ్స్ గుర్తుంటాయి లైక్ ఉదాహరణకు కచ్చా",
            "code_switch": "అన్నమాట నా కొన్ని యాడ్స్ గుర్తుంటాయి లైక్ ఉదాహరణకు కచ్చా",
            "romanized": "annamaata na konni ads gurtuntayi like udaaharanaku kachcha",
            "time_sec": 2.2073280811309814
          },
          "temp0_low": {
            "native": "అన్నమాట నా కొన్ని యాడ్స్ గుర్తుంటాయి లైక్ ఎగ్జాంపుల్ కచ్చాబాదం అన్నమాట",
            "punctuated": "అన్నమాట। నా కొన్ని యాడ్స్ గుర్తుంటాయి లైక్ ఎగ్జాంపుల్ కచ్చాబాదం అన్నమాట",
            "code_switch": "అన్నమాట నా కొన్ని యాడ్స్ గుర్తుంటాయి లైక్ ఎగ్జాంపుల్ కచ్చాబాదం అన్నమాట",
            "romanized": "annamaata na konni ads gurtuntaayi like example kachchabaadam annamaata",
            "time_sec": 1.9971544742584229
          }
        }
      },
      "validation": {
        "indicmfa": {
          "transcription": "అన్నమాట నా కొన్ని యాడ్స్ గుర్తుంటాయి లైక్ ఉదాహరణకు కచ్చా",
          "confidence": 0.8448988199234009,
          "word_count": 8,
          "time_sec": 0.0726473331451416
        },
        "indic_conformer": {
          "transcription": "నా కొన్ని యాడ్స్ గుర్తుండే లైక్ ఎగ్జాంపుల్ కచ్చా మ్యాంగో బయట",
          "confidence": null,
          "time_sec": 4.286128044128418
        }
      }
    },
    {
      "segment_id": "SPEAKER_00_0107_1343.96-1350.94.flac",
      "audio_path": "/tmp/maya3_transcribe/pF_BQpHaIdU/extracted/pF_BQpHaIdU/segments/SPEAKER_00_0107_1343.96-1350.94.flac",
      "duration_sec": 6.973,
      "models": {
        "gemini-3-pro-preview": {
          "temp1_high": {
            "native": "ఇప్పుడు మీరు పాపిన్స్ గురించి అని కిస్మీ గురించి చెప్తున్నప్పుడు రెండిటి గురించే రకరకాల స్టోరీస్ ఉంటాయి అవును నాకు స్కూల్లో ఉన్నప్పుడు ఐ గాట్ అ డేర్ ట్రూత్ ఆర్ డేర్ లో ఏదో",
            "punctuated": "ఇప్పుడు మీరు పాపిన్స్ గురించి అని కిస్మీ గురించి చెప్తున్నప్పుడు, రెండిటి గురించే రకరకాల స్టోరీస్ ఉంటాయి. అవును. నాకు స్కూల్లో ఉన్నప్పుడు, ఐ గాట్ అ డేర్. ట్రూత్ ఆర్ డేర్ లో ఏదో.",
            "code_switch": "ఇప్పుడు మీరు Poppins గురించి అని Kismi గురించి చెప్తున్నప్పుడు, రెండిటి గురించే రకరకాల stories ఉంటాయి. అవును. నాకు schoolలో ఉన్నప్పుడు, I got a dare. Truth or dareలో ఏదో.",
            "romanized": "ippudu meeru poppins gurinchi ani kismi gurinchi chepthunnappudu, renditi gurinche rakarakala stories untayi. avunu. naaku schoollo unnappudu, i got a dare. truth or dare lo edho.",
            "time_sec": 26.40731191635132
          },
          "temp0_low": {
            "native": "ఇప్పుడు మీరు పాపిన్స్ గురించి అని కిస్మి గురించి చెప్తున్నప్పుడు రెండిటి గురించి రకరకాల స్టోరీస్ ఉండేవి అవును నాకు స్కూల్లో ఉన్నప్పుడు ఐ గాట్ ఎ డేర్ ట్రూత్ ఆర్ డేర్ లో",
            "punctuated": "ఇప్పుడు మీరు పాపిన్స్ గురించి అని, కిస్మి గురించి చెప్తున్నప్పుడు, రెండిటి గురించి రకరకాల స్టోరీస్ ఉండేవి. అవును. నాకు స్కూల్లో ఉన్నప్పుడు ఐ గాట్ ఎ డేర్, ట్రూత్ ఆర్ డేర్ లో.",
            "code_switch": "ఇప్పుడు మీరు Poppins గురించి అని Kissmi గురించి చెప్తున్నప్పుడు, రెండిటి గురించి రకరకాల stories ఉండేవి. అవును. నాకు school లో ఉన్నప్పుడు I got a dare, truth or dare లో.",
            "romanized": "ippudu meeru poppins gurinchi ani kissmi gurinchi cheptunnappudu, renditi gurinchi rakarakala stories undevi. avunu. naaku school lo unnappudu i got a dare, truth or dare lo.",
            "time_sec": 6.233404159545898
          }
        },
        "gemini-3-flash-preview": {
          "temp1_high": {
            "native": "ఇప్పుడు మీరు పాపిన్స్ గురించి అండ్ కిస్మిస్ గురించి చెప్తున్నప్పుడు రెండిటి గురించి రకరకాల స్టోరీస్ ఉన్నాయి అవును నాకు స్కూల్ లో ఉన్నప్పుడు ఐ గాట్ ఏ డేర్ ట్రూత్ ఆర్ డేర్ లో",
            "punctuated": "ఇప్పుడు మీరు పాపిన్స్ గురించి అండ్ కిస్మిస్ గురించి చెప్తున్నప్పుడు, రెండిటి గురించి రకరకాల స్టోరీస్ ఉన్నాయి. అవును. నాకు స్కూల్ లో ఉన్నప్పుడు ఐ గాట్ ఏ డేర్, ట్రూత్ ఆర్ డేర్ లో.",
            "code_switch": "ఇప్పుడు మీరు Poppins గురించి and Kismiss గురించి చెప్తున్నప్పుడు, రెండిటి గురించి రకరకాల stories ఉన్నాయి. అవును. నాకు school లో ఉన్నప్పుడు I got a dare, truth or dare లో.",
            "romanized": "ippudu meeru poppins gurinchi and kismiss gurinchi cheptunnappudu renditi gurinchi rakkarakala stories unnayi avunu naaku school lo unnappudu i got a dare truth or dare lo",
            "time_sec": 17.55294895172119
          },
          "temp0_low": {
            "native": "ఇప్పుడు మీరు పాపిన్స్ గురించి ఇంకా కిస్మి గురించి చెప్తున్నప్పుడు రెండిటి గురించి రకరకాల స్టోరీస్ ఉన్నాయి అవును నాకు స్కూల్లో ఉన్నప్పుడు ఐ గాట్ ఏ డేర్ ట్రూత్ ఆర్ డేర్ లో",
            "punctuated": "ఇప్పుడు మీరు పాపిన్స్ గురించి ఇంకా కిస్మి గురించి చెప్తున్నప్పుడు, రెండిటి గురించి రకరకాల స్టోరీస్ ఉన్నాయి. అవును, నాకు స్కూల్లో ఉన్నప్పుడు ఐ గాట్ ఏ డేర్, ట్రూత్ ఆర్ డేర్ లో.",
            "code_switch": "ఇప్పుడు మీరు Poppins గురించి ఇంకా Kismi గురించి చెప్తున్నప్పుడు, రెండిటి గురించి రకరకాల stories ఉన్నాయి. అవును, నాకు school లో ఉన్నప్పుడు I got a dare, Truth or Dare లో.",
            "romanized": "ippudu meeru poppins gurinchi inka kismi gurinchi cheptunnappudu renditi gurinchi rakarakala stories unnaayi avunu naaku school lo unnappudu i got a dare truth or dare lo",
            "time_sec": 2.759925127029419
          }
        },
        "gemini-2.5-pro": {
          "temp1_high": {
            "native": "ఇప్పుడు మీరు పాపిన్స్ గురించి అండ్ కిస్మి గురించి చెప్తున్నప్పుడు రెండిటి గురించి రకరకాల స్టోరీస్ ఉండేవి నాకు స్కూల్ లో ఉన్నప్పుడు ఐ గాట్ ఏ డేర్ ట్రూత్ ఆర్ డేర్",
            "punctuated": "ఇప్పుడు మీరు పాపిన్స్ గురించి అండ్ కిస్మి గురించి చెప్తున్నప్పుడు, రెండిటి గురించి రకరకాల స్టోరీస్ ఉండేవి। నాకు స్కూల్ లో ఉన్నప్పుడు, ఐ గాట్ ఏ డేర్, ట్రూత్ ఆర్ డేర్।",
            "code_switch": "ఇప్పుడు మీరు poppins గురించి and కిస్మి గురించి చెప్తున్నప్పుడు, రెండిటి గురించి రకరకాల stories ఉండేవి। నాకు school లో ఉన్నప్పుడు, I got a dare, truth or dare।",
            "romanized": "ippudu meeru poppins gurinchi and kismi gurinchi cheptunnappudu, renditi gurinchi rakarakaala stories undevi. naaku school lo unnappudu, I got a dare, truth or dare.",
            "time_sec": 8.21396517753601
          },
          "temp0_low": {
            "native": "ఇప్పుడు మీరు పాపిన్స్ గురించి కిస్మి గురించి చెప్తున్నప్పుడు రెండిటి గురించి రకరకాల స్టోరీస్ ఉండేవి అవును నాకు స్కూల్లో ఉన్నప్పుడు ఐ గాట్ ఏ డేర్ ట్రూత్ ఆర్ డేర్",
            "punctuated": "ఇప్పుడు మీరు పాపిన్స్ గురించి, కిస్మి గురించి చెప్తున్నప్పుడు, రెండిటి గురించి రకరకాల స్టోరీస్ ఉండేవి। అవును। నాకు స్కూల్లో ఉన్నప్పుడు ఐ గాట్ ఏ డేర్, ట్రూత్ ఆర్ డేర్।",
            "code_switch": "ఇప్పుడు మీరు Poppins గురించి, Kismee గురించి చెప్తున్నప్పుడు, రెండిటి గురించి రకరకాల stories ఉండేవి। అవును। నాకు schoolలో ఉన్నప్పుడు I got a dare, Truth or Dare.",
            "romanized": "ippudu meeru Poppins gurinchi, Kismee gurinchi chepthunnappudu, renditi gurinchi rakarakaala stories undevi. avunu. naaku schoollo unnappudu I got a dare, Truth or Dare.",
            "time_sec": 8.039685010910034
          }
        },
        "gemini-2.5-flash": {
          "temp1_high": {
            "native": "ఇప్పుడు మీరు పాపెన్స్ గురించి కిస్మి గురించి చెప్తున్నప్పుడు రెండిటి గురించి రకరకాల స్టోరీస్ ఉండే నాకు స్కూల్లో ఉన్నప్పుడు ఐ గాట్ ఏ డేర్ ట్రూత్ ఆర్ డేర్",
            "punctuated": "ఇప్పుడు మీరు పాపెన్స్ గురించి, కిస్మి గురించి చెప్తున్నప్పుడు రెండిటి గురించి రకరకాల స్టోరీస్ ఉండే. నాకు స్కూల్లో ఉన్నప్పుడు ఐ గాట్ ఏ డేర్, ట్రూత్ ఆర్ డేర్.",
            "code_switch": "ఇప్పుడు మీరు పాపెన్స్ గురించి కిస్మి గురించి చెప్తున్నప్పుడు రెండిటి గురించి రకరకాల stories ఉండే. నాకు స్కూల్లో ఉన్నప్పుడు I got a dare, truth or dare.",
            "romanized": "ippuḍu mīru poppens gurinchi kismi gurinchi cheptunnappuḍu renḍiṭi gurinchi rakarākala stories unḍe. nāku school lo unnappuḍu I got a dare, truth or dare.",
            "time_sec": 6.689608097076416
          },
          "temp0_low": {
            "native": "ఇప్పుడు మీరు పాపిన్స్ గురించి కెస్మీ గురించి చెప్తున్నప్పుడు రెండిటి గురించి రకరకాల స్టోరీస్ ఉండేవి అవును నాకు స్కూల్లో ఉన్నప్పుడు ఐ గాట్ ఏ డేర్ ట్రూత్ ఆర్ డేర్",
            "punctuated": "ఇప్పుడు మీరు పాపిన్స్ గురించి, కెస్మీ గురించి చెప్తున్నప్పుడు రెండిటి గురించి రకరకాల స్టోరీస్ ఉండేవి. అవును, నాకు స్కూల్లో ఉన్నప్పుడు ఐ గాట్ ఏ డేర్, ట్రూత్ ఆర్ డేర్.",
            "code_switch": "ఇప్పుడు మీరు Popins గురించి Kesmi గురించి చెప్తున్నప్పుడు రెండిటి గురించి రకరకాల స్టోరీస్ ఉండేవి. అవును, నాకు స్కూల్లో ఉన్నప్పుడు I got a dare, Truth or Dare.",
            "romanized": "Ippudu meeru Popins gurinchi Kesmi gurinchi cheptunnappudu renditi gurinchi rakarakala stories undevi. Avunu, naaku schoollo unnappudu I got a dare, Truth or Dare.",
            "time_sec": 4.72485876083374
          }
        },
        "gemini-2.5-flash-lite": {
          "temp1_high": {
            "native": "ఇప్పుడు మీరు పాపిన్స్ గురించి ఐన్ కీస్మి గురించి చెబుతున్నారు అప్పుడు రెండిటి గురించి రకరకాల స్టోరీస్ ఉండేవి అవును నాకు స్కూల్లో ఉన్నప్పుడు ఐ గాట్ ఏ డేర్ ట్రూత్ ఆర్",
            "punctuated": "ఇప్పుడు మీరు పాపిన్స్ గురించి ఐన్ కీస్మి గురించి చెబుతున్నారు అప్పుడు రెండిటి గురించి రకరకాల స్టోరీస్ ఉండేవి। అవును। నాకు స్కూల్లో ఉన్నప్పుడు ఐ గాట్ ఏ డేర్ ట్రూత్ ఆర్",
            "code_switch": "ఇప్పుడు మీరు పాపిన్స్ గురించి ఐన్ కీస్మి గురించి చెబుతున్నారు అప్పుడు రెండిటి గురించి రకరకాల స్టోరీస్ ఉండేవి అవును నాకు స్కూల్లో ఉన్నప్పుడు I got a dare truth or",
            "romanized": "ippudu meeru paapins gurinchi ain kismis gurinchi chebutunnaru appudu renditi gurinchi rakarakala stories undevi avunu naaku school lo unnappudu I got a dare truth or",
            "time_sec": 1.5881335735321045
          },
          "temp0_low": {
            "native": "ఇప్పుడు మీరు పాపిన్స్ గురించి ఐస్ క్రీమ్ గురించి చెబుతున్నారు అప్పుడు రెండింటి గురించి రకరకాల స్టోరీలు ఉండేవి అవును నాకు స్కూల్లో ఉన్నప్పుడు ఐ గాట్ ఏ డేర్ ట్రూత్ ఆర్",
            "punctuated": "ఇప్పుడు మీరు పాపిన్స్ గురించి ఐస్ క్రీమ్ గురించి చెబుతున్నారు అప్పుడు రెండింటి గురించి రకరకాల స్టోరీలు ఉండేవి అవును నాకు స్కూల్లో ఉన్నప్పుడు ఐ గాట్ ఏ డేర్ ట్రూత్ ఆర్",
            "code_switch": "ఇప్పుడు మీరు పాపిన్స్ గురించి ఐస్ క్రీమ్ గురించి చెబుతున్నారు అప్పుడు రెండింటి గురించి రకరకాల స్టోరీలు ఉండేవి అవును నాకు స్కూల్లో ఉన్నప్పుడు I got a dare truth or",
            "romanized": "ippudu meeru papins gurinchi ice cream gurinchi chebutunnaru appudu rendintini gurinchi rakarakala storielu undevi avunu naaku school lo unnappudu I got a dare truth or",
            "time_sec": 1.7781872749328613
          }
        }
      },
      "validation": {
        "indicmfa": {
          "transcription": "ఇప్పుడు మీరు పాపిన్స్ గురించి ఐన్ కీస్మి గురించి చెబుతున్నారు అప్పుడు రెండిటి గురించి రకరకాల స్టోరీస్ ఉండేవి అవును నాకు స్కూల్లో ఉన్నప్పుడు ఐ గాట్ ఏ డేర్ ట్రూత్ ఆర్",
          "confidence": 0.8948860764503479,
          "word_count": 24,
          "time_sec": 0.13230586051940918
        },
        "indic_conformer": {
          "transcription": "నాకు స్కూల్లో ఉన్నప్పుడు ఐ గా డేర్ ట్రూట్ థౌట్ డేర్ అన్న",
          "confidence": null,
          "time_sec": 0.8548033237457275
        }


Summary: auto


sed -n '260,520p' analysis_results/final_analysis.json