Output:
Exit code: 0
Wall time: 0.1 seconds
Total output lines: 1811
Output:
/home/ubuntu/maya-transcribe/outputs/bench/maya_hf_v2/per_lang_vaani_limit20_routerpref/results.jsonl:1:{"schema_version": 1, "segment_id": "fleurs__as_in__test__102__1775-779fc9c34b257c32", "base_id": "fleurs__as_in__test__102__1775", "relpath": "asm/fleurs__as_in__test__102__1775.wav", "uid": "779fc9c34b257c32", "src_path": "data/bench/maya_hf_v2/audio/asm/fleurs__as_in__test__102__1775.wav", "wav_path": "outputs/bench/maya_hf_v2/per_lang_vaani_limit20_routerpref/audio/fleurs__as_in__test__102__1775-779fc9c34b257c32.wav", "src_ffprobe": {"path": "data/bench/maya_hf_v2/audio/asm/fleurs__as_in__test__102__1775.wav", "duration_sec": 8.58, "sample_rate": 16000, "channels": 1}, "ffprobe": {"path": "outputs/bench/maya_hf_v2/per_lang_vaani_limit20_routerpref/audio/fleurs__as_in__test__102__1775-779fc9c34b257c32.wav", "duration_sec": 2.462063, "sample_rate": 16000, "channels": 1}, "qc": {"duration_sec": 2.4620625, "rms_dbfs": -28.440811260172616, "peak_dbfs": -14.234395279703481, "clipping_fraction": 0.0}, "lid": {"model": "facebook/mms-lid-256", "predictions": [{"code": "asm", "prob": 0.9998915195465088}, {"code": "ben", "prob": 1.0081897016789299e-05}, {"code": "guj", "prob": 6.762039447494317e-06}, {"code": "npi", "prob": 6.628784376516705e-06}, {"code": "ory", "prob": 5.353842425392941e-06}]}, "route": {"language_hint": "asm", "language_hint_source": "path"}, "asr": {"faster-whisper:large-v3|det": {"model": "faster-whisper:large-v3", "text": "মাত্র কেটামান কেটামান কেটামান কেটামান কেটামান", "language": "bn", "language_probability": 1, "segments": [{"start": 0.0, "end": 2.44, "text": " মাত্র কেটামান কেটামান কেটামান কেটামান কেটামান", "avg_logprob": -0.2008223693621786, "no_speech_prob": 0.029144287109375, "compression_ratio": 2.9069767441860463}], "meta": {"device": "cuda", "compute_type": "float16", "beam_size": 5, "best_of": 5, "temperature": 0.0, "vad_filter": false}}, "faster-whisper:large-v3|samp_t0.2": {"model": "faster-whisper:large-v3", "text": "ক্যারানে ক্যারানে ক্যারানে ক্যারানে ক্যারানে ক্যারানে ক্যারানে ক্যারানে ক্যারানে ক্যারানে ক্যারানে ক্যারা", "language": "bn", "language_probability": 1, "segments": [{"start": 0.0, "end": 2.46, "text": " ক্যারানে ক্যারানে ক্যারানে ক্যারানে ক্যারানে ক্যারানে ক্যারানে ক্যারানে ক্যারানে ক্যারানে ক্যারানে ক্যারা", "avg_logprob": -0.11701179398430718, "no_speech_prob": 0.029144287109375, "compression_ratio": 8.13888888888889}], "meta": {"device": "cuda", "compute_type": "float16", "beam_size": 5, "best_of": 10, "temperature": 0.2, "vad_filter": false}}, "transformers:DrishtiSharma/whisper-large-v2-assamese-5k-steps": {"model": "transformers:DrishtiSharma/whisper-large-v2-assamese-5k-steps", "text": "ড্ৰিডে' ৰেকৰ্ডিংৰ ব্যৱহৰৰ ফৰত মাত্ৰক কিইটমান ত্বাৰেক্টিংৰ বিডিঅ' ৰেকৰ্ডিংৰ বিবহৰৰ ফৰত মাত্ৰক কিইটমান ত্বাৰেক্টিংৰ বিডি�", "language": "as", "language_probability": null, "segments": [], "meta": {"device": "cuda", "dtype": "float16", "max_new_tokens": 256, "num_beams": 5}}}, "asr_summary": {"faster-whisper:large-v3|det": {"language": "bn", "language_probability": 1, "avg_logprob_mean": -0.2008223693621786, "no_speech_prob_mean": 0.029144287109375, "compression_ratio_mean": 2.9069767441860463}, "faster-whisper:large-v3|samp_t0.2": {"language": "bn", "language_probability": 1, "avg_logprob_mean": -0.11701179398430718, "no_speech_prob_mean": 0.029144287109375, "compression_ratio_mean": 8.13888888888889}, "transformers:DrishtiSharma/whisper-large-v2-assamese-5k-steps": {"language": "as", "language_probability": null, "avg_logprob_mean": null, "no_speech_prob_mean": null, "compression_ratio_mean": null}}, "asr_errors": {}, "ensemble": {"unique_models": ["faster-whisper:large-v3", "transformers:DrishtiSharma/whisper-large-v2-assamese-5k-steps"], "unique_backends": ["faster-whisper", "transformers"], "warnings": []}, "normalized": {"faster-whisper:large-v3|det": "মাত্র কেটামান কেটামান কেটামান কেটামান কেটামান", "faster-whisper:large-v3|samp_t0.2": "ক্যারানে ক্যারানে ক্যারানে ক্যারানে ক্যারানে ক্যারানে ক্যারানে ক্যারানে ক্যারানে ক্যারানে ক্যারানে ক্যারা", "transformers:DrishtiSharma/whisper-large-v2-assamese-5k-steps": "ড্ৰিডে ৰেকৰ্ডিংৰ ব্যৱহৰৰ ফৰত মাত্ৰক কিইটমান ত্বাৰেক্টিংৰ বিডিঅ ৰেকৰ্ডিংৰ বিবহৰৰ ফৰত মাত্ৰক কিইটমান ত্বাৰেক্টিংৰ বিডি�"}, "rover": {"text": "ড্ৰিডে ৰেকৰ্ডিংৰ ব্যৱহৰৰ ফৰত মাত্ৰক কিইটমান ত্বাৰেক্টিংৰ বিডিঅ ৰেকৰ্ডিংৰ বিবহৰৰ ফৰত মাত্ৰক কিইটমান ত্বাৰেক্টিংৰ বিডি�", "skeleton_key": "transformers:DrishtiSharma/whisper-large-v2-assamese-5k-steps", "disagreement_rate": 0.1935483870967742, "inputs": ["faster-whisper:large-v3|det", "transformers:DrishtiSharma/whisper-large-v2-assamese-5k-steps"], "filtered_out": ["faster-whisper:large-v3|samp_t0.2"], "disagreements": [{"kind": "token", "index": 9, "chosen": "বিবহৰৰ", "options": {"বিবহৰৰ": 1.0, "মাত্র": 1.0}}, {"kind": "token", "index": 10, "chosen": "ফৰত", "options": {"কেটামান": 1.0, "ফৰত": 1.0}}, {"kind": "token", "index": 11, "chosen": "মাত্ৰক", "options": {"কেটামান": 1.0, "মাত্ৰক": 1.0}}, {"kind": "token", "index": 12, "chosen": "কিইটমান", "options": {"কিইটমান": 1.0, "কেটামান": 1.0}}, {"kind": "token", "index": 13, "chosen": "ত্বাৰেক্টিংৰ", "options": {"কেটামান": 1.0, "ত্বাৰেক্টিংৰ": 1.0}}, {"kind": "token", "index": 14, "chosen": "বিডি�", "options": {"কেটামান": 1.0, "বিডি�": 1.0}}]}, "ctc": null, "final": {"text": "ড্ৰিডে ৰেকৰ্ডিংৰ ব্যৱহৰৰ ফৰত মাত্ৰক কিইটমান ত্বাৰেক্টিংৰ বিডিঅ ৰেকৰ্ডিংৰ বিবহৰৰ ফৰত মাত্ৰক কিইটমান ত্বাৰেক্টিংৰ বিডি�", "source": "rover"}, "verification": {"decision": "human", "reasons": ["asr_disagreement", "high_compression_ratio"], "scores": {"duration_sec": 2.4620625, "clipping_fraction": 0.0, "rover_disagreement_rate": 0.1935483870967742, "lid": {"available": true, "top1": "asm", "top1_prob": 0.9998915195465088, "margin": 0.999881437649492, "confident": true}, "no_speech_prob_max": 0.029144287109375, "heuristics_asr_key": "faster-whisper:large-v3|det", "avg_logprob_mean": -0.2008223693621786, "compression_ratio_mean": 2.9069767441860463, "ctc": null}}, "config": {"trim_silence": true, "silence_threshold_db": -40.0, "min_silence_sec": 0.15, "gemini_model": null, "gemini_hard_only": true, "ctc_model": null}, "timing": {"preprocess_sec": 0.21457516076043248, "lid_sec": 0.3478547791019082, "asr_sec": 19.895464211236686, "normalize_sec": 0.00020983675494790077, "rover_sec": 0.0004512220621109009, "ctc_sec": 3.5064294934272766e-07, "verify_sec": 0.00011368421837687492, "total_sec": 20.458827154245228}}
/home/ubuntu/maya-transcribe/outputs/bench/maya_hf_v2/per_lang_vaani_limit20_routerpref/results.jsonl:2:{"schema_version": 1, "segment_id": "fleurs__as_in__test__105__1875-fc7f2738561720cf", "base_id": "fleurs__as_in__test__105__1875", "relpath": "asm/fleurs__as_in__test__105__1875.wav", "uid": "fc7f2738561720cf", "src_path": "data/bench/maya_hf_v2/audio/asm/fleurs__as_in__test__105__1875.wav", "wav_path": "outputs/bench/maya_hf_v2/per_lang_vaani_limit20_routerpref/audio/fleurs__as_in__test__105__1875-fc7f2738561720cf.wav", "src_ffprobe": {"path": "data/bench/maya_hf_v2/audio/asm/fleurs__as_in__test__105__1875.wav", "duration_sec": 9.0, "sample_rate": 16000, "channels": 1}, "ffprobe": {"path": "outputs/bench/maya_hf_v2/per_lang_vaani_limit20_routerpref/audio/fleurs__as_in__test__105__1875-fc7f2738561720cf.wav", "duration_sec": 6.06025, "sample_rate": 16000, "channels": 1}, "qc": {"duration_sec": 6.06025, "rms_dbfs": -20.180413637251753, "peak_dbfs": -4.625800291220733, "clipping_fraction": 0.0}, "lid": {"model": "facebook/mms-lid-256", "predictions": [{"code": "asm", "prob": 0.9998835325241089}, {"code": "ben", "prob": 3.67779030057136e-05}, {"code": "ory", "prob": 6.2822919062455185e-06}, {"code": "guj", "prob": 5.470949417940574e-06}, {"code": "npi", "prob": 3.2575569548498606e-06}]}, "route": {"language_hint": "asm", "language_hint_source": "path"}, "asr": {"faster-whisper:large-v3|det": {"model": "faster-whisper:large-v3", "text": "হাগোরোত পরা নদি হমুহর বিস্খতাং খো পানি এমাজনোর পরা আহে।", "language": "bn", "language_probability": 1, "segments": [{"start": 0.0, "end": 6.0, "text": " হাগোরোত পরা নদি হমুহর বিস্খতাং খো পানি এমাজনোর পরা আহে।", "avg_logprob": -0.11465371393405639, "no_speech_prob": 0.001155853271484375, "compression_ratio": 1.47}], "meta": {"device": "cuda", "compute_type": "float16", "beam_size": 5, "best_of": 5, "temperature": 0.0, "vad_filter": false}}, "faster-whisper:large-v3|samp_t0.2": {"model": "faster-whisper:large-v3", "text": "হাগোরোত পরা নদি হমুহর বিস্খতাং খো পানি এমাজনোর পরা আহে।", "language": "bn", "language_probability": 1, "segments": [{"start": 0.0, "end": 6.0, "text": " হাগোরোত পরা নদি হমুহর বিস্খতাং খো পানি এমাজনোর পরা আহে।", "avg_logprob": -0.11453371797059034, "no_speech_prob": 0.001155853271484375, "compression_ratio": 1.47}], "meta": {"device": "cuda", "compute_type": "float16", "beam_size": 5, "best_of": 10, "temperature": 0.2, "vad_filter": false}}, "transformers:DrishtiSharma/whisper-large-v2-assamese-5k-steps": {"model": "transformers:DrishtiSharma/whisper-large-v2-assamese-5k-steps", "text": "সাগৰত পৰা নদীসমূহৰ বিছসটাংশ পানী এমাজনৰ পৰা আহি।", "language": "as", "language_probability": null, "segments": [], "meta": {"device": "cuda", "dtype": "float16", "max_new_tokens": 256, "num_beams": 5}}}, "asr_summary": {"faster-whisper:large-v3|det": {"language": "bn", "language_probability": 1, "avg_logprob_mean": -0.11465371393405639, "no_speech_prob_mean": 0.001155853271484375, "compression_ratio_mean": 1.47}, "faster-whisper:large-v3|samp_t0.2": {"language": "bn", "language_probability": 1, "avg_logprob_mean": -0.11453371797059034, "no_speech_prob_mean": 0.001155853271484375, "compression_ratio_mean": 1.47}, "transformers:DrishtiSharma/whisper-large-v2-assamese-5k-steps": {"language": "as", "language_probability": null, "avg_logprob_mean": null, "no_speech_prob_mean": null, "compression_ratio_mean": null}}, "asr_errors": {}, "ensemble": {"unique_models": ["faster-whisper:large-v3", "transformers:DrishtiSharma/whisper-large-v2-assamese-5k-steps"], "unique_backends": ["faster-whisper", "transformers"], "warnings": []}, "normalized": {"faster-whisper:large-v3|det": "হাগোরোত পরা নদি হমুহর বিস্খতাং খো পানি এমাজনোর পরা আহে", "faster-whisper:large-v3|samp_t0.2": "হাগোরোত পরা নদি হমুহর বিস্খতাং খো পানি এমাজনোর পরা আহে", "transformers:DrishtiSharma/whisper-large-v2-assamese-5k-steps": "সাগৰত পৰা নদীসমূহৰ বিছসটাংশ পানী এমাজনৰ পৰা আহি"}, "rover": {"text": "হাগোরোত পরা সাগৰত পৰা নদীসমূহৰ বিছসটাংশ পানী এমাজনৰ পৰা আহি", "skeleton_key": "transformers:DrishtiSharma/whisper-large-v2-assamese-5k-steps", "disagreement_rate": 0.47058823529411764, "inputs": ["faster-whisper:large-v3|det", "transformers:DrishtiSharma/whisper-large-v2-assamese-5k-steps"], "filtered_out": ["faster-whisper:large-v3|samp_t0.2"], "disagreements": [{"kind": "token", "index": 0, "chosen": "সাগৰত", "options": {"নদি": 1.0, "সাগৰত": 1.0}}, {"kind": "token", "index": 1, "chosen": "পৰা", "options": {"পৰা": 1.0, "হমুহর": 1.0}}, {"kind": "token", "index": 2, "chosen": "নদীসমূহৰ", "options": {"নদীসমূহৰ": 1.0, "বিস্খতাং": 1.0}}, {"kind": "token", "index": 3, "chosen": "বিছসটাংশ", "options": {"খো": 1.0, "বিছসটাংশ": 1.0}}, {"kind": "token", "index": 4, "chosen": "পানী", "options": {"পানি": 1.0, "পানী": 1.0}}, {"kind": "token", "index": 5, "chosen": "এমাজনৰ", "options": {"এমাজনোর": 1.0, "এমাজনৰ": 1.0}}, {"kind": "token", "index": 6, "chosen": "পৰা", "options": {"পরা": 1.0, "পৰা": 1.0}}, {"kind": "token", "index": 7, "chosen": "আহি", "options": {"আহি": 1.0, "আহে": 1.0}}]}, "ctc": null, "final": {"text": "হাগোরোত পরা সাগৰত পৰা নদীসমূহৰ বিছসটাংশ পানী এমাজনৰ পৰা আহি", "source": "rover"}, "verification": {"decision": "human", "reasons": ["asr_disagreement"], "scores": {"duration_sec": 6.06025, "clipping_fraction": 0.0, "rover_disagreement_rate": 0.47058823529411764, "lid": {"available": true, "top1": "asm", "top1_prob": 0.9998835325241089, "margin": 0.9998467546211032, "confident": true}, "no_speech_prob_max": 0.001155853271484375, "heuristics_asr_key": "faster-whisper:large-v3|det", "avg_logprob_mean": -0.11465371393405639, "compression_ratio_mean": 1.47, "ctc": null}}, "config": {"trim_silence": true, "silence_threshold_db": -40.0, "min_silence_sec": 0.15, "gemini_model": null, "gemini_hard_only": true, "ctc_model": null}, "timing": {"preprocess_sec": 0.2178402771241963, "lid_sec": 0.122710513882339, "asr_sec": 6.707489777356386, "normalize_sec": 0.00013318192213773727, "rover_sec": 0.00040016695857048035, "ctc_sec": 6.41215592622757e-07, "verify_sec": 0.00011907517910003662, "total_sec": 7.048849356826395}}
/home/ubuntu/maya-transcribe/outputs/bench/maya_hf_v2/per_lang_vaani_limit20_routerpref/results.jsonl:3:{"schema_version": 1, "segment_id": "fleurs__as_in__test__117__1748-ae04a7dfdde8c0a6", "base_id": "fleurs__as_in__test__117__1748", "relpath": "asm/fleurs__as_in__test__117__1748.wav", "uid": "ae04a7dfdde8c0a6", "src_path": "data/bench/maya_hf_v2/audio/asm/fleurs__as_in__test__117__1748.wav", "wav_path": "outputs/bench/maya_hf_v2/per_lang_vaani_limit20_routerpref/audio/fleurs__as_in__test__117__1748-ae04a7dfdde8c0a6.wav", "src_ffprobe": {"path": "data/bench/maya_hf_v2/audio/asm/fleurs__as_in__test__117__1748.wav", "duration_sec": 10.02, "sample_rate": 16000, "channels": 1}, "ffprobe": {"path": "outputs/bench/maya_hf_v2/per_lang_vaani_limit20_routerpref/audio/fleurs__as_in__test__117__1748-ae04a7dfdde8c0a6.wav", "duration_sec": 2.276313, "sample_rate": 16000, "channels": 1}, "qc": {"duration_sec": 2.2763125, "rms_dbfs": -21.941688479934783, "peak_dbfs": -5.893243175876851, "clipping_fraction": 0.0}, "lid": {"model": "facebook/mms-lid-256", "predictions": [{"code": "asm", "prob": 0.9998373985290527}, {"code": "ben", "prob": 4.020015330752358e-05}, {"code": "guj", "prob": 1.0821251635206863e-05}, {"code": "mar", "prob": 5.7564052440284286e-06}, {"code": "npi", "prob": 5.523234904103447e-06}]}, "route": {"language_hint": "asm", "language_hint_source": "path"}, "asr": {"faster-whisper:large-v3|det": {"model": "faster-whisper:large-v3", "text": "অগ্ণি নির্বা পন বাহেনের লকে", "language": "bn", "language_probability": 1, "segments": [{"start": 0.0, "end": 2.2800000000000002, "text": " অগ্ণি নির্বা পন বাহেনের লকে", "avg_logprob": -0.1488347477832083, "no_speech_prob": 0.41357421875, "compression_ratio": 1.0895522388059702}], "meta": {"device": "cuda", "compute_type": "float16", "beam_size": 5, "best_of": 5, "temperature": 0.0, "vad_filter": false}}, "faster-whisper:large-v3|samp_t0.2": {"model": "faster-whisper:large-v3", "text": "অগ্ণি নির্বা পন বাহেনের লকে", "language": "bn", "language_probability": 1, "segments": [{"start": 0.0, "end": 2.2800000000000002, "text": " অগ্ণি নির্বা পন বাহেনের লকে", "avg_logprob": -0.1485828944181992, "no_speech_prob": 0.41357421875, "compression_ratio": 1.0895522388059702}], "meta": {"device": "cuda", "compute_type": "float16", "beam_size": 5, "best_of": 10, "temperature": 0.2, "vad_filter": false}}, "transformers:DrishtiSharma/whisper-large-v2-assamese-5k-steps": {"model": "transformers:DrishtiSharma/whisper-large-v2-assamese-5k-steps", "text": "অদ্নিৰ্বাপন বাহেনিৰ লগে", "language": "as", "language_probability": null, "segments": [], "meta": {"device": "cuda", "dtype": "float16", "max_new_tokens": 256, "num_beams": 5}}}, "asr_summary": {"faster-whisper:large-v3|det": {"language": "bn", "language_probability": 1, "avg_logprob_mean": -0.1488347477832083, "no_speech_prob_mean": 0.41357421875, "compression_ratio_mean": 1.0895522388059702}, "faster-whisper:large-v3|samp_t0.2": {"language": "bn", "language_probability": 1, "avg_logprob_mean": -0.1485828944181992, "no_speech_prob_mean": 0.41357421875, "compression_ratio_mean": 1.0895522388059702}, "transformers:DrishtiSharma/whisper-large-v2-assamese-5k-steps": {"language": "as", "language_probability": null, "avg_logprob_mean": null, "no_speech_prob_mean": null, "compression_ratio_mean": null}}, "asr_errors": {}, "ensemble": {"unique_models": ["faster-whisper:large-v3", "transformers:DrishtiSharma/whisper-large-v2-assamese-5k-steps"], "unique_backends": ["faster-whisper", "transformers"], "warnings": []}, "normalized": {"faster-whisper:large-v3|det": "অগ্ণি নির্বা পন বাহেনের লকে", "faster-whisper:large-v3|samp_t0.2": "অগ্ণি নির্বা পন বাহেনের লকে", "transformers:DrishtiSharma/whisper-large-v2-assamese-5k-steps": "অদ্নিৰ্বাপন বাহেনিৰ লগে"}, "rover": {"text": "অগ্ণি নির্বা অদ্নিৰ্বাপন বাহেনিৰ লগে", "skeleton_key": "transformers:DrishtiSharma/whisper-large-v2-assamese-5k-steps", "disagreement_rate": 0.42857142857142855, "inputs": ["faster-whisper:large-v3|det", "transformers:DrishtiSharma/whisper-large-v2-assamese-…3081691 tokens truncated… ద్వారా సాధించిన ముదటి విజేయంగా బావించబడింది"}, "rover": {"text": "ఇది బానిషా సాయన్యం విప్లవం ద్వారా సాధించిన ముదటి విజేయంగా బావించబడింది", "skeleton_key": "faster-whisper:large-v3|det", "disagreement_rate": 0.0, "disagreements": []}, "ctc": null, "final": {"text": "ఇది బానిషా సాయన్యం విప్లవం ద్వారా సాధించిన ముదటి విజేయంగా బావించబడింది", "source": "rover"}, "verification": {"decision": "accept", "reasons": [], "scores": {"duration_sec": 8.6431875, "clipping_fraction": 1.4462257124469416e-05, "rover_disagreement_rate": 0.0, "lid": {"available": true, "top1": "tel", "top1_prob": 0.9996296167373657, "margin": 0.9994722632254707, "confident": true}, "no_speech_prob_max": 0.00537109375, "heuristics_asr_key": "faster-whisper:large-v3|det", "avg_logprob_mean": -0.06818704361862996, "compression_ratio_mean": 1.7636363636363637, "ctc": null}}, "config": {"trim_silence": true, "silence_threshold_db": -40.0, "min_silence_sec": 0.15, "gemini_model": null, "gemini_hard_only": true, "ctc_model": null}, "timing": {"preprocess_sec": 0.21844988781958818, "lid_sec": 0.1833267747424543, "asr_sec": 4.820233981125057, "normalize_sec": 7.709534838795662e-05, "rover_sec": 8.245464414358139e-05, "ctc_sec": 6.41215592622757e-07, "verify_sec": 7.861992344260216e-05, "total_sec": 5.222352361772209}}
/home/ubuntu/maya-transcribe/outputs/bench/maya_hf_v2/per_lang_vaani/results.jsonl:2310:{"schema_version": 1, "segment_id": "Kathbath__telugu__valid__2148__844424933547655-514-m.m4a-57ee4b3e22863b1e", "base_id": "Kathbath__telugu__valid__2148__844424933547655-514-m.m4a", "relpath": "tel/Kathbath__telugu__valid__2148__844424933547655-514-m.m4a.wav", "uid": "57ee4b3e22863b1e", "src_path": "data/bench/maya_hf_v2/audio/tel/Kathbath__telugu__valid__2148__844424933547655-514-m.m4a.wav", "wav_path": "outputs/bench/maya_hf_v2/per_lang_vaani/audio/Kathbath__telugu__valid__2148__844424933547655-514-m.m4a-57ee4b3e22863b1e.wav", "src_ffprobe": {"path": "data/bench/maya_hf_v2/audio/tel/Kathbath__telugu__valid__2148__844424933547655-514-m.m4a.wav", "duration_sec": 11.21525, "sample_rate": 16000, "channels": 1}, "ffprobe": {"path": "outputs/bench/maya_hf_v2/per_lang_vaani/audio/Kathbath__telugu__valid__2148__844424933547655-514-m.m4a-57ee4b3e22863b1e.wav", "duration_sec": 11.168813, "sample_rate": 16000, "channels": 1}, "qc": {"duration_sec": 11.1688125, "rms_dbfs": -16.09931307968704, "peak_dbfs": -0.0005301608106181009, "clipping_fraction": 2.2383758344944907e-05}, "lid": {"model": "facebook/mms-lid-256", "predictions": [{"code": "tel", "prob": 0.99895179271698}, {"code": "kan", "prob": 0.0005066178855486214}, {"code": "mar", "prob": 0.00016265694284811616}, {"code": "mal", "prob": 9.31386457523331e-05}, {"code": "esg", "prob": 7.419912435580045e-05}]}, "route": {"language_hint": "tel", "language_hint_source": "lid"}, "asr": {"faster-whisper:large-v3|det": {"model": "faster-whisper:large-v3", "text": "సమిప జూయేర్ కలాసాల లక్షమి నర్సు పేటలోను ప్రభుత్వ ఆర్స్ సింస్ డిగ్రి కలాసాల హిరా మండలం లోను ఉన్నాయ్", "language": "te", "language_probability": 1, "segments": [{"start": 0.0, "end": 10.66, "text": " సమిప జూయేర్ కలాసాల లక్షమి నర్సు పేటలోను ప్రభుత్వ ఆర్స్ సింస్ డిగ్రి కలాసాల హిరా మండలం లోను ఉన్నాయ్", "avg_logprob": -0.09114583273248006, "no_speech_prob": 0.0078125, "compression_ratio": 1.8732394366197183}], "meta": {"device": "cuda", "compute_type": "float16", "beam_size": 5, "best_of": 5, "temperature": 0.0, "vad_filter": false}}}, "asr_summary": {"faster-whisper:large-v3|det": {"language": "te", "language_probability": 1, "avg_logprob_mean": -0.09114583273248006, "no_speech_prob_mean": 0.0078125, "compression_ratio_mean": 1.8732394366197183}}, "asr_errors": {"faster-whisper:large-v3|samp_t0.2": "RuntimeError:CUDA failed with error out of memory", "transformers_router:tel": "OutOfMemoryError:CUDA out of memory. Tried to allocate 14.00 MiB. GPU 0 has a total capacity of 79.25 GiB of which 11.19 MiB is free. Process 738802 has 60.16 GiB memory in use. Including non-PyTorch memory, this process has 19.07 GiB memory in use. Of the allocated memory 14.90 GiB is allocated by PyTorch, and 609.12 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation.  See documentation for Memory Management  (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables)"}, "ensemble": {"unique_models": ["faster-whisper:large-v3"], "unique_backends": ["faster-whisper"], "warnings": ["low_model_diversity"]}, "normalized": {"faster-whisper:large-v3|det": "సమిప జూయేర్ కలాసాల లక్షమి నర్సు పేటలోను ప్రభుత్వ ఆర్స్ సింస్ డిగ్రి కలాసాల హిరా మండలం లోను ఉన్నాయ్"}, "rover": {"text": "సమిప జూయేర్ కలాసాల లక్షమి నర్సు పేటలోను ప్రభుత్వ ఆర్స్ సింస్ డిగ్రి కలాసాల హిరా మండలం లోను ఉన్నాయ్", "skeleton_key": "faster-whisper:large-v3|det", "disagreement_rate": 0.0, "disagreements": []}, "ctc": null, "final": {"text": "సమిప జూయేర్ కలాసాల లక్షమి నర్సు పేటలోను ప్రభుత్వ ఆర్స్ సింస్ డిగ్రి కలాసాల హిరా మండలం లోను ఉన్నాయ్", "source": "rover"}, "verification": {"decision": "accept", "reasons": [], "scores": {"duration_sec": 11.1688125, "clipping_fraction": 2.2383758344944907e-05, "rover_disagreement_rate": 0.0, "lid": {"available": true, "top1": "tel", "top1_prob": 0.99895179271698, "margin": 0.9984451748314314, "confident": true}, "no_speech_prob_max": 0.0078125, "heuristics_asr_key": "faster-whisper:large-v3|det", "avg_logprob_mean": -0.09114583273248006, "compression_ratio_mean": 1.8732394366197183, "ctc": null}}, "config": {"trim_silence": true, "silence_threshold_db": -40.0, "min_silence_sec": 0.15, "gemini_model": null, "gemini_hard_only": true, "ctc_model": null}, "timing": {"preprocess_sec": 0.21938392100855708, "lid_sec": 0.2012883280403912, "asr_sec": 5.31257258169353, "normalize_sec": 0.00018982915207743645, "rover_sec": 8.528120815753937e-05, "ctc_sec": 4.908069968223572e-07, "verify_sec": 7.320800796151161e-05, "total_sec": 5.733695761766285}}
/home/ubuntu/maya-transcribe/outputs/bench/maya_hf_v2/per_lang_vaani/results.jsonl:2324:{"schema_version": 1, "segment_id": "Kathbath__telugu__valid__22__844424931097375-381-f.m4a-bc07e9724df253f4", "base_id": "Kathbath__telugu__valid__22__844424931097375-381-f.m4a", "relpath": "tel/Kathbath__telugu__valid__22__844424931097375-381-f.m4a.wav", "uid": "bc07e9724df253f4", "src_path": "data/bench/maya_hf_v2/audio/tel/Kathbath__telugu__valid__22__844424931097375-381-f.m4a.wav", "wav_path": "outputs/bench/maya_hf_v2/per_lang_vaani/audio/Kathbath__telugu__valid__22__844424931097375-381-f.m4a-bc07e9724df253f4.wav", "src_ffprobe": {"path": "data/bench/maya_hf_v2/audio/tel/Kathbath__telugu__valid__22__844424931097375-381-f.m4a.wav", "duration_sec": 6.501625, "sample_rate": 16000, "channels": 1}, "ffprobe": {"path": "outputs/bench/maya_hf_v2/per_lang_vaani/audio/Kathbath__telugu__valid__22__844424931097375-381-f.m4a-bc07e9724df253f4.wav", "duration_sec": null, "sample_rate": 16000, "channels": 1}, "qc": {"duration_sec": 0.0, "rms_dbfs": -Infinity, "peak_dbfs": -Infinity, "clipping_fraction": 0.0}, "lid": {"model": "facebook/mms-lid-256", "predictions": []}, "route": {"language_hint": "tel", "language_hint_source": "path"}, "asr": {"faster-whisper:large-v3|det": {"model": "faster-whisper:large-v3", "text": "", "language": "te", "language_probability": 1, "segments": [], "meta": {"device": "cuda", "compute_type": "float16", "beam_size": 5, "best_of": 5, "temperature": 0.0, "vad_filter": false}}, "faster-whisper:large-v3|samp_t0.2": {"model": "faster-whisper:large-v3", "text": "", "language": "te", "language_probability": 1, "segments": [], "meta": {"device": "cuda", "compute_type": "float16", "beam_size": 5, "best_of": 10, "temperature": 0.2, "vad_filter": false}}}, "asr_summary": {"faster-whisper:large-v3|det": {"language": "te", "language_probability": 1, "avg_logprob_mean": null, "no_speech_prob_mean": null, "compression_ratio_mean": null}, "faster-whisper:large-v3|samp_t0.2": {"language": "te", "language_probability": 1, "avg_logprob_mean": null, "no_speech_prob_mean": null, "compression_ratio_mean": null}}, "asr_errors": {"transformers_router:tel": "OutOfMemoryError:CUDA out of memory. Tried to allocate 14.00 MiB. GPU 0 has a total capacity of 79.25 GiB of which 11.19 MiB is free. Process 738802 has 60.16 GiB memory in use. Including non-PyTorch memory, this process has 19.07 GiB memory in use. Of the allocated memory 14.90 GiB is allocated by PyTorch, and 609.12 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation.  See documentation for Memory Management  (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables)"}, "ensemble": {"unique_models": ["faster-whisper:large-v3"], "unique_backends": ["faster-whisper"], "warnings": ["low_model_diversity"]}, "normalized": {"faster-whisper:large-v3|det": "", "faster-whisper:large-v3|samp_t0.2": ""}, "rover": {"text": "", "skeleton_key": "faster-whisper:large-v3|det", "disagreement_rate": 0.0, "disagreements": []}, "ctc": null, "final": {"text": "", "source": "rover"}, "verification": {"decision": "reject", "reasons": ["too_short", "lid_missing"], "scores": {"duration_sec": 0.0, "clipping_fraction": 0.0, "rover_disagreement_rate": 0.0, "lid": {"available": false, "top1": null, "top1_prob": null, "margin": null, "confident": false}, "no_speech_prob_max": null, "heuristics_asr_key": "faster-whisper:large-v3|det", "avg_logprob_mean": null, "compression_ratio_mean": null, "ctc": null}}, "config": {"trim_silence": true, "silence_threshold_db": -40.0, "min_silence_sec": 0.15, "gemini_model": null, "gemini_hard_only": true, "ctc_model": null}, "timing": {"preprocess_sec": 0.19830887904390693, "lid_sec": 0.0515102157369256, "asr_sec": 3.928018390201032, "normalize_sec": 1.1992640793323517e-05, "rover_sec": 9.163282811641693e-05, "ctc_sec": 5.709007382392883e-07, "verify_sec": 8.796714246273041e-05, "total_sec": 4.1781435129232705}}
/home/ubuntu/maya-transcribe/outputs/bench/maya_hf_v2/per_lang_vaani/results.jsonl:2335:{"schema_version": 1, "segment_id": "Kathbath__telugu__valid__313__844424931097425-381-f.m4a-d77bf363ca2370f1", "base_id": "Kathbath__telugu__valid__313__844424931097425-381-f.m4a", "relpath": "tel/Kathbath__telugu__valid__313__844424931097425-381-f.m4a.wav", "uid": "d77bf363ca2370f1", "src_path": "data/bench/maya_hf_v2/audio/tel/Kathbath__telugu__valid__313__844424931097425-381-f.m4a.wav", "wav_path": "outputs/bench/maya_hf_v2/per_lang_vaani/audio/Kathbath__telugu__valid__313__844424931097425-381-f.m4a-d77bf363ca2370f1.wav", "src_ffprobe": {"path": "data/bench/maya_hf_v2/audio/tel/Kathbath__telugu__valid__313__844424931097425-381-f.m4a.wav", "duration_sec": 6.106875, "sample_rate": 16000, "channels": 1}, "ffprobe": {"path": "outputs/bench/maya_hf_v2/per_lang_vaani/audio/Kathbath__telugu__valid__313__844424931097425-381-f.m4a-d77bf363ca2370f1.wav", "duration_sec": null, "sample_rate": 16000, "channels": 1}, "qc": {"duration_sec": 0.0, "rms_dbfs": -Infinity, "peak_dbfs": -Infinity, "clipping_fraction": 0.0}, "lid": {"model": "facebook/mms-lid-256", "predictions": []}, "route": {"language_hint": "tel", "language_hint_source": "path"}, "asr": {"faster-whisper:large-v3|det": {"model": "faster-whisper:large-v3", "text": "", "language": "te", "language_probability": 1, "segments": [], "meta": {"device": "cuda", "compute_type": "float16", "beam_size": 5, "best_of": 5, "temperature": 0.0, "vad_filter": false}}, "faster-whisper:large-v3|samp_t0.2": {"model": "faster-whisper:large-v3", "text": "", "language": "te", "language_probability": 1, "segments": [], "meta": {"device": "cuda", "compute_type": "float16", "beam_size": 5, "best_of": 10, "temperature": 0.2, "vad_filter": false}}}, "asr_summary": {"faster-whisper:large-v3|det": {"language": "te", "language_probability": 1, "avg_logprob_mean": null, "no_speech_prob_mean": null, "compression_ratio_mean": null}, "faster-whisper:large-v3|samp_t0.2": {"language": "te", "language_probability": 1, "avg_logprob_mean": null, "no_speech_prob_mean": null, "compression_ratio_mean": null}}, "asr_errors": {"transformers_router:tel": "OutOfMemoryError:CUDA out of memory. Tried to allocate 14.00 MiB. GPU 0 has a total capacity of 79.25 GiB of which 11.19 MiB is free. Process 738802 has 60.16 GiB memory in use. Including non-PyTorch memory, this process has 19.07 GiB memory in use. Of the allocated memory 14.90 GiB is allocated by PyTorch, and 609.12 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation.  See documentation for Memory Management  (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables)"}, "ensemble": {"unique_models": ["faster-whisper:large-v3"], "unique_backends": ["faster-whisper"], "warnings": ["low_model_diversity"]}, "normalized": {"faster-whisper:large-v3|det": "", "faster-whisper:large-v3|samp_t0.2": ""}, "rover": {"text": "", "skeleton_key": "faster-whisper:large-v3|det", "disagreement_rate": 0.0, "disagreements": []}, "ctc": null, "final": {"text": "", "source": "rover"}, "verification": {"decision": "reject", "reasons": ["too_short", "lid_missing"], "scores": {"duration_sec": 0.0, "clipping_fraction": 0.0, "rover_disagreement_rate": 0.0, "lid": {"available": false, "top1": null, "top1_prob": null, "margin": null, "confident": false}, "no_speech_prob_max": null, "heuristics_asr_key": "faster-whisper:large-v3|det", "avg_logprob_mean": null, "compression_ratio_mean": null, "ctc": null}}, "config": {"trim_silence": true, "silence_threshold_db": -40.0, "min_silence_sec": 0.15, "gemini_model": null, "gemini_hard_only": true, "ctc_model": null}, "timing": {"preprocess_sec": 0.19895822135731578, "lid_sec": 0.0450857519172132, "asr_sec": 3.5315909781493247, "normalize_sec": 8.406117558479309e-06, "rover_sec": 9.825685992836952e-05, "ctc_sec": 5.620531737804413e-07, "verify_sec": 7.442105561494827e-05, "total_sec": 3.775943853892386}}
/home/ubuntu/maya-transcribe/outputs/bench/maya_hf_v2/per_lang_vaani/results.jsonl:2357:{"schema_version": 1, "segment_id": "Kathbath__telugu__valid__53__844424931097432-381-f.m4a-111f57297fe28238", "base_id": "Kathbath__telugu__valid__53__844424931097432-381-f.m4a", "relpath": "tel/Kathbath__telugu__valid__53__844424931097432-381-f.m4a.wav", "uid": "111f57297fe28238", "src_path": "data/bench/maya_hf_v2/audio/tel/Kathbath__telugu__valid__53__844424931097432-381-f.m4a.wav", "wav_path": "outputs/bench/maya_hf_v2/per_lang_vaani/audio/Kathbath__telugu__valid__53__844424931097432-381-f.m4a-111f57297fe28238.wav", "src_ffprobe": {"path": "data/bench/maya_hf_v2/audio/tel/Kathbath__telugu__valid__53__844424931097432-381-f.m4a.wav", "duration_sec": 8.777125, "sample_rate": 16000, "channels": 1}, "ffprobe": {"path": "outputs/bench/maya_hf_v2/per_lang_vaani/audio/Kathbath__telugu__valid__53__844424931097432-381-f.m4a-111f57297fe28238.wav", "duration_sec": null, "sample_rate": 16000, "channels": 1}, "qc": {"duration_sec": 0.0, "rms_dbfs": -Infinity, "peak_dbfs": -Infinity, "clipping_fraction": 0.0}, "lid": {"model": "facebook/mms-lid-256", "predictions": []}, "route": {"language_hint": "tel", "language_hint_source": "path"}, "asr": {"faster-whisper:large-v3|det": {"model": "faster-whisper:large-v3", "text": "", "language": "te", "language_probability": 1, "segments": [], "meta": {"device": "cuda", "compute_type": "float16", "beam_size": 5, "best_of": 5, "temperature": 0.0, "vad_filter": false}}, "faster-whisper:large-v3|samp_t0.2": {"model": "faster-whisper:large-v3", "text": "", "language": "te", "language_probability": 1, "segments": [], "meta": {"device": "cuda", "compute_type": "float16", "beam_size": 5, "best_of": 10, "temperature": 0.2, "vad_filter": false}}}, "asr_summary": {"faster-whisper:large-v3|det": {"language": "te", "language_probability": 1, "avg_logprob_mean": null, "no_speech_prob_mean": null, "compression_ratio_mean": null}, "faster-whisper:large-v3|samp_t0.2": {"language": "te", "language_probability": 1, "avg_logprob_mean": null, "no_speech_prob_mean": null, "compression_ratio_mean": null}}, "asr_errors": {"transformers_router:tel": "OutOfMemoryError:CUDA out of memory. Tried to allocate 14.00 MiB. GPU 0 has a total capacity of 79.25 GiB of which 11.19 MiB is free. Process 738802 has 60.16 GiB memory in use. Including non-PyTorch memory, this process has 19.07 GiB memory in use. Of the allocated memory 14.90 GiB is allocated by PyTorch, and 609.12 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation.  See documentation for Memory Management  (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables)"}, "ensemble": {"unique_models": ["faster-whisper:large-v3"], "unique_backends": ["faster-whisper"], "warnings": ["low_model_diversity"]}, "normalized": {"faster-whisper:large-v3|det": "", "faster-whisper:large-v3|samp_t0.2": ""}, "rover": {"text": "", "skeleton_key": "faster-whisper:large-v3|det", "disagreement_rate": 0.0, "disagreements": []}, "ctc": null, "final": {"text": "", "source": "rover"}, "verification": {"decision": "reject", "reasons": ["too_short", "lid_missing"], "scores": {"duration_sec": 0.0, "clipping_fraction": 0.0, "rover_disagreement_rate": 0.0, "lid": {"available": false, "top1": null, "top1_prob": null, "margin": null, "confident": false}, "no_speech_prob_max": null, "heuristics_asr_key": "faster-whisper:large-v3|det", "avg_logprob_mean": null, "compression_ratio_mean": null, "ctc": null}}, "config": {"trim_silence": true, "silence_threshold_db": -40.0, "min_silence_sec": 0.15, "gemini_model": null, "gemini_hard_only": true, "ctc_model": null}, "timing": {"preprocess_sec": 0.21076451428234577, "lid_sec": 0.049231975339353085, "asr_sec": 3.725766504649073, "normalize_sec": 1.0970979928970337e-05, "rover_sec": 9.041000157594681e-05, "ctc_sec": 3.7997961044311523e-07, "verify_sec": 7.550418376922607e-05, "total_sec": 3.9860590221360326}}
/home/ubuntu/maya-transcribe/data/bench/model_maps/whisper_vaani_map.json:2:  "hin": "ARTPARK-IISc/whisper-large-v3-vaani-hindi",
/home/ubuntu/maya-transcribe/data/bench/model_maps/whisper_vaani_map.json:3:  "tel": "ARTPARK-IISc/whisper-large-v3-vaani-telugu",
/home/ubuntu/maya-transcribe/data/bench/model_maps/whisper_vaani_map.json:4:  "ory": "ARTPARK-IISc/whisper-large-v3-vaani-odia",
/home/ubuntu/maya-transcribe/data/bench/model_maps/whisper_vaani_map.json:5:  "ben": "ARTPARK-IISc/whisper-medium-vaani-bengali",
/home/ubuntu/maya-transcribe/data/bench/model_maps/whisper_vaani_map.json:6:  "kan": "ARTPARK-IISc/whisper-medium-vaani-kannada",


Summary: auto

## Skills
A skill is a set of local instructions to follow that is stored in a `SKILL.md` file. Below is the list of skills that can be used. Each entry includes a name, description, and file path so you can open the source for full instructions when using a specific skill.
### Available skills
- skill-creator: Guide for creating effective skills. This skill should be used when users want to create a new skill (or update an existing skill) that extends Codex's capabilities with specialized knowledge, workflows, or tool integrations. (file: /home/ubuntu/.codex/skills/.system/skill-creator/SKILL.md)
- skill-installer: Install Codex skills into $CODEX_HOME/skills from a curated list or a GitHub repo path. Use when a user asks to list installable skills, install a curated skill, or install a skill from another repo (including private repos). (file: /home/ubuntu/.codex/skills/.system/skill-installer/SKILL.md)
### How to use skills
- Discovery: The list above is the skills available in this session (name + description + file path). Skill bodies live on disk at the listed paths.
- Trigger rules: If the user names a skill (with `$SkillName` or plain text) OR the task clearly matches a skill's description shown above, you must use that skill for that turn. Multiple mentions mean use them all. Do not carry skills across turns unless re-mentioned.
- Missing/blocked: If a named skill isn't in the list or the path can't be read, say so briefly and continue with the best fallback.
- How to use a skill (progressive disclosure):
  1) After deciding to use a skill, open its `SKILL.md`. Read only enough to follow the workflow.
  2) If `SKILL.md` points to extra folders such as `references/`, load only the specific files needed for the request; don't bulk-load everything.
  3) If `scripts/` exist, prefer running or patching them instead of retyping large code blocks.
  4) If `assets/` or templates exist, reuse them instead of recreating from scratch.
- Coordination and sequencing:
  - If multiple skills apply, choose the minimal set that covers the request and state the order you'll use them.
  - Announce which skill(s) you're using and why (one short line). If you skip an obvious skill, say why.
- Context hygiene:
  - Keep context small: summarize long sections instead of pasting them; only load extra files when needed.
  - Avoid deep reference-chasing: prefer opening only files directly linked from `SKILL.md` unless you're blocked.
  - When variants exist (frameworks, providers, domains), pick only the relevant reference file(s) and note that choice.
- Safety and fallback: If a skill can't be applied cleanly (missing files, unclear instructions), state the issue, pick the next-best approach, and continue.


**Clarifying model access and APIs**

I'll confirm which models the user wants tested and request missing API keys or endpoint details, noting Gemini can run now but Voxtral and Gemma3n need integration or clarification before proceeding.


**Identifying benchmark dataset for WER comparison**

I'm verifying the likely benchmark as the 200-segment Hindi subset to align with prior results and planning to confirm this choice with the user before running any evaluations.


Output:
Exit code: 0
Wall time: 0 seconds
Output:
200 /home/ubuntu/maya-transcribe/data/bench/maya_hf_hin_v2/refs.jsonl


Summary: auto


**Planning benchmark runs with new models**

I confirmed the current dataset returns expected results, so I'll now explore the benchmark CLI options to integrate new models like Gemini and Whisper.


sed -n '330,480p' /home/ubuntu/maya-transcribe/maya_transcribe/cli.py


Output:
Exit code: 0
Wall time: 0 seconds
Output:
@bench_app.command("build")
def bench_build(
    out: Path = typer.Option(Path("data/bench/maya_hf_v0"), help="Output benchmark corpus directory."),
    samples_per_lang: int = typer.Option(200, help="Target samples per language."),
    seed: int = typer.Option(0, help="Shuffle seed."),
    shuffle: bool = typer.Option(True, help="Shuffle within each source before selecting."),
    lang: list[str] = typer.Option([], help="Limit to ISO-639-3 language codes (repeatable)."),
    min_duration_sec: float = typer.Option(1.0, help="Min audio duration."),
    max_duration_sec: float = typer.Option(30.0, help="Max audio duration."),
    force: bool = typer.Option(False, help="Delete and rebuild output directory if it exists."),
) -> None:
    """
    Build a local benchmark corpus from Hugging Face datasets.
    """
    from maya_transcribe.bench import build_benchmark_corpus

    manifest = build_benchmark_corpus(
        out_dir=out,
        samples_per_lang=samples_per_lang,
        seed=seed,
        shuffle=shuffle,
        langs=lang or None,
        min_duration_sec=min_duration_sec,
        max_duration_sec=max_duration_sec,
        force=force,
    )
    typer.echo(json.dumps({"wrote": str(out), "total_rows": manifest.get("total_rows"), "counts": manifest.get("counts")}, indent=2))

@bench_app.command("run")
def bench_run(
    bench_dir: Path = typer.Option(Path("data/bench/maya_hf_v0"), exists=True, file_okay=False, help="Benchmark corpus directory."),
    out: Path | None = typer.Option(None, help="Output directory (defaults under outputs/bench/)."),
    whisper_model: str = typer.Option("large-v3", help="Whisper model size/path for faster-whisper."),
    lid_model_id: str = typer.Option("facebook/mms-lid-256", help="HuggingFace MMS-LID model id."),
    require_lid_backend: bool = typer.Option(True, help="Fail if MMS-LID backend can't initialize."),
    transformers_whisper_model: list[str] = typer.Option(
        [], help="Extra whisper-family HF models to run via transformers (repeatable)."
    ),
    transformers_whisper_model_map: Path | None = typer.Option(
        None, help="JSON map {lang_code: hf_model_id} for per-language Whisper-family HF models (transformers)."
    ),
    # MMS ASR for CTC-based model diversity (per master.md plan).
    mms_asr_model: str | None = typer.Option(
        None, help="Optional MMS ASR model id (e.g., facebook/mms-1b-all) for CTC-based diversity."
    ),
    # IndicConformer for Conformer architecture diversity (per master.md plan).
    indicconformer_model_map: Path | None = typer.Option(
        None, help="JSON map {lang_code: nemo_model_name} for per-language IndicConformer models."
    ),
    indicconformer_default_model: str | None = typer.Option(
        None, help="Default IndicConformer NeMo model name for architecture diversity."
    ),
    ctc_model: str | None = typer.Option(None, help="Optional HF CTC model id for audio-grounded fit scoring."),
    ctc_model_map: Path | None = typer.Option(None, help="JSON map {lang_code: hf_model_id} for CTC scoring."),
    require_ctc: bool = typer.Option(True, help="When CTC is enabled, require a CTC score to accept."),
    min_ctc_margin: float = typer.Option(0.05, help="Min (second-best - best) CTC loss margin to auto-accept."),
    trim_silence: bool = typer.Option(
        False, help="Trim leading/trailing silence (ffmpeg silenceremove). Off by default for benchmarking."
    ),
    silence_threshold_db: float = typer.Option(-40.0, help="Silence threshold in dB for trimming."),
    min_silence_sec: float = typer.Option(0.15, help="Min silence duration for trimming."),
    limit: int | None = typer.Option(None, help="Process only the first N segments."),
    thresholds_json: Path | None = typer.Option(None, help="Optional JSON file of threshold overrides (verify + routing)."),
    hypothesis: str = typer.Option("final", help='Hypothesis to evaluate: "final", "rover", or "asr:<key>".'),
) -> None:
    """
    Run the full pipeline on a local benchmark corpus and write eval.json.
    """
    from datetime import datetime

    from maya_transcribe.eval import evaluate
    from maya_transcribe.pipeline import run_pipeline

    audio_dir = bench_dir / "audio"
    refs_path = bench_dir / "refs.jsonl"
    if not audio_dir.exists():
        raise typer.BadParameter(f"Missing: {audio_dir}")
    if not refs_path.exists():
        raise typer.BadParameter(f"Missing: {refs_path}")

    if out is None:
        ts = datetime.utcnow().strftime("%Y%m%d_%H%M%S")
        out = Path("outputs/bench") / bench_dir.name / f"run_{ts}"

    ctc_map = None
    if ctc_model_map is not None:
        ctc_map = json.loads(ctc_model_map.read_text(encoding="utf-8"))
        if not isinstance(ctc_map, dict):
            raise typer.BadParameter("ctc_model_map must be a JSON object {lang_code: model_id}")

    tf_map = None
    if transformers_whisper_model_map is not None:
        tf_map = json.loads(transformers_whisper_model_map.read_text(encoding="utf-8"))
        if not isinstance(tf_map, dict):
            raise typer.BadParameter("transformers_whisper_model_map must be a JSON object {lang_code: model_id}")

    ic_map = None
    if indicconformer_model_map is not None:
        ic_map = json.loads(indicconformer_model_map.read_text(encoding="utf-8"))
        if not isinstance(ic_map, dict):
            raise typer.BadParameter("indicconformer_model_map must be a JSON object {lang_code: nemo_model_name}")

    thresholds: dict[str, object] = {"route_lang_from_path": True, "route_lang_prefer_path": True}
    if thresholds_json is not None:
        loaded = json.loads(thresholds_json.read_text(encoding="utf-8"))
        if not isinstance(loaded, dict):
            raise typer.BadParameter("thresholds_json must be a JSON object")
        thresholds.update(loaded)
    if ctc_model or ctc_map:
        thresholds["require_ctc"] = require_ctc
        thresholds["min_ctc_margin"] = float(min_ctc_margin)

    results_path = run_pipeline(
        segments_dir=audio_dir,
        out_dir=out,
        lid_model_id=lid_model_id,
        require_lid_backend=require_lid_backend,
        whisper_model=whisper_model,
        extra_transformers_whisper_models=transformers_whisper_model,
        transformers_whisper_model_map=tf_map,
        mms_asr_model=mms_asr_model,
        indicconformer_model_map=ic_map,
        indicconformer_default_model=indicconformer_default_model,
        ctc_model=ctc_model,
        ctc_model_map=ctc_map,
        trim_silence=trim_silence,
        silence_threshold_db=silence_threshold_db,
        min_silence_sec=min_silence_sec,
        limit=limit,
        thresholds=thresholds,
    )

    eval_path = out / "eval.json"
    summary = evaluate(
        results_path=results_path,
        refs_path=refs_path,
        out_path=eval_path,
        hypothesis=hypothesis,
        oracle=True,
        language_source="refs",
    )
    typer.echo(f"Wrote: {results_path}")
    typer.echo(f"Wrote: {out / 'summary.json'}")
    typer.echo(f"Wrote: {eval_path}")
    typer.echo(json.dumps(summary, ensure_ascii=False, indent=2))

@bench_app.command("push")
def bench_push(
    bench_dir: Path = typer.Option(Path("data/bench/maya_hf_v0"), exists=True, file_okay=False, help="Benchmark corpus directory."),


**Inspecting pipeline.py for gemini support**


sed -n '1,240p' /home/ubuntu/maya-transcribe/maya_transcribe/pipeline.py


Output:
Exit code: 0
Wall time: 0 seconds
Output:
from __future__ import annotations

import json
import time
from dataclasses import asdict
from pathlib import Path
from typing import Any

from rich.progress import track

from maya_transcribe.asr import (
    ASRResult,
    FasterWhisperASR,
    IndicConformerASRRouter,
    MMSASR,
    TransformersWhisperASR,
    TransformersWhisperASRRouter,
)
from maya_transcribe.audio import compute_qc, ffprobe, normalize_audio
from maya_transcribe.ids import segment_identity
from maya_transcribe.language import LANG_CODE_TO_NAME
from maya_transcribe.lid import LIDResult, MMSLID
from maya_transcribe.rover import rover_combine
from maya_transcribe.textnorm import normalize_text
from maya_transcribe.verify import verify_segment

def _iter_audio_files(segments_dir: Path) -> list[Path]:
    audio_files: list[Path] = []
    for ext in ("*.wav", "*.mp3", "*.m4a", "*.flac", "*.ogg", "*.opus"):
        audio_files.extend(sorted(segments_dir.rglob(ext)))
    return sorted(set(audio_files))

def _asr_summary(result: ASRResult) -> dict[str, Any]:
    avg_logprobs = [s.avg_logprob for s in result.segments if s.avg_logprob is not None]
    no_speech_probs = [s.no_speech_prob for s in result.segments if s.no_speech_prob is not None]
    compression_ratios = [s.compression_ratio for s in result.segments if s.compression_ratio is not None]

    def mean(xs: list[float]) -> float | None:
        return (sum(xs) / len(xs)) if xs else None

    return {
        "language": result.language,
        "language_probability": result.language_probability,
        "avg_logprob_mean": mean([float(x) for x in avg_logprobs]),
        "no_speech_prob_mean": mean([float(x) for x in no_speech_probs]),
        "compression_ratio_mean": mean([float(x) for x in compression_ratios]),
    }

def _infer_lang_from_relpath(relpath: str) -> str | None:
    parts = [p.strip().lower() for p in (relpath or "").split("/") if p.strip()]
    for p in parts[:-1]:
        if p in LANG_CODE_TO_NAME:
            return p
    return None

def run_pipeline(
    *,
    segments_dir: Path,
    out_dir: Path,
    lid_model_id: str = "facebook/mms-lid-256",
    whisper_model: str = "large-v3",
    extra_transformers_whisper_models: list[str] | None = None,
    transformers_whisper_model_map: dict[str, str] | None = None,
    transformers_whisper_default_model: str | None = None,
    # MMS ASR for CTC-based diversity (per master.md plan).
    mms_asr_model: str | None = None,
    # IndicConformer for Conformer architecture diversity (per master.md plan).
    indicconformer_model_map: dict[str, str] | None = None,
    indicconformer_default_model: str | None = None,
    gemini_model: str | None = None,
    gemini_prompt_path: Path | None = None,
    gemini_max_output_tokens: int = 512,
    gemini_hard_only: bool = True,
    ctc_model: str | None = None,
    ctc_model_map: dict[str, str] | None = None,
    require_lid_backend: bool = True,
    trim_silence: bool = True,
    silence_threshold_db: float = -40.0,
    min_silence_sec: float = 0.15,
    limit: int | None = None,
    thresholds: dict[str, Any] | None = None,
) -> Path:
    """
    End-to-end run: preprocess -> LID -> ASR ensemble -> normalize -> ROVER -> verify.
    Writes `out_dir/results.jsonl` and returns that path.
    """
    thresholds = thresholds or {}
    if (ctc_model or ctc_model_map) and "require_ctc" not in thresholds:
        thresholds["require_ctc"] = True
    if (ctc_model or ctc_model_map) and "min_ctc_margin" not in thresholds:
        thresholds["min_ctc_margin"] = 0.05
    out_dir.mkdir(parents=True, exist_ok=True)

    audio_files = _iter_audio_files(segments_dir)
    if not audio_files:
        raise RuntimeError(f"No audio files found under: {segments_dir}")
    if limit is not None:
        audio_files = audio_files[: max(0, int(limit))]

    # Instantiate models once.
    lid_backend: MMSLID | None
    try:
        lid_backend = MMSLID(model_id=lid_model_id)
    except Exception as e:
        if require_lid_backend:
            raise RuntimeError(
                "Failed to initialize MMS-LID. Install torch/transformers deps (pip install -e '.[torch]') "
                "or run with require_lid_backend=False."
            ) from e
        lid_backend = None

    whisper_backend = FasterWhisperASR(model_size_or_path=whisper_model)

    extra_transformers_whisper_models = extra_transformers_whisper_models or []
    transformers_backends: list[TransformersWhisperASR] = []
    for model_id in extra_transformers_whisper_models:
        transformers_backends.append(TransformersWhisperASR(model_id=model_id))

    transformers_router = None
    if transformers_whisper_model_map:
        transformers_router = TransformersWhisperASRRouter(
            model_map=transformers_whisper_model_map, default_model=transformers_whisper_default_model
        )

    # MMS ASR backend for CTC-based diversity (per master.md: "Optional diversity: MMS ASR").
    mms_backend: MMSASR | None = None
    if mms_asr_model:
        try:
            mms_backend = MMSASR(model_id=mms_asr_model)
        except Exception:
            pass  # MMS optional; continue if unavailable.

    # IndicConformer router for architecture diversity (per master.md: "IndicConformer-600M multilingual").
    indicconformer_router: IndicConformerASRRouter | None = None
    if indicconformer_model_map or indicconformer_default_model:
        try:
            indicconformer_router = IndicConformerASRRouter(
                model_map=indicconformer_model_map,
                default_model=indicconformer_default_model,
            )
        except Exception:
            pass  # IndicConformer optional; continue if NeMo unavailable.

    ctc_scorer = None
    rank_by_ctc = None
    if ctc_model or ctc_model_map:
        from maya_transcribe.ctc_fit import CTCFitScorer, rank_candidates_by_ctc

        ctc_scorer = CTCFitScorer(model_map=ctc_model_map, default_model=ctc_model)
        rank_by_ctc = rank_candidates_by_ctc

    gemini_backend = None
    if gemini_model:
        try:
            from maya_transcribe.gemini_asr import GeminiASR, GeminiConfig, load_prompt_template

            prompt = load_prompt_template(gemini_prompt_path or Path("Prompt.md"))
            gemini_backend = GeminiASR(
                config=GeminiConfig(model=gemini_model, max_output_tokens=gemini_max_output_tokens),
                prompt_template=prompt,
            )
        except Exception as e:
            raise RuntimeError("Failed to initialize Gemini backend (check GEMINI_KEY and google-genai).") from e

    default_whisper_run_sampling = not bool(transformers_backends or transformers_router or gemini_backend is not None)

    results_path = out_dir / "results.jsonl"
    summary_path = out_dir / "summary.json"

    counts = {"accept": 0, "human": 0, "reject": 0, "error": 0}
    timings_total: list[float] = []

    with results_path.open("w", encoding="utf-8") as f:
        for src in track(audio_files, description="Transcribing"):
            ident = segment_identity(src, root=segments_dir)
            segment_id = ident.segment_id
            t0 = time.perf_counter()

            try:
                # 1) Preprocess
                t_pre0 = time.perf_counter()
                src_info = ffprobe(src)
                wav_path = out_dir / "audio" / f"{segment_id}.wav"
                normalize_audio(
                    src,
                    wav_path,
                    trim_silence=trim_silence,
                    silence_threshold_db=silence_threshold_db,
                    min_silence_sec=min_silence_sec,
                )
                info = ffprobe(wav_path)
                qc = compute_qc(wav_path)
                t_pre1 = time.perf_counter()

                # 2) LID
                t_lid0 = time.perf_counter()
                lid: LIDResult | None = None
                if lid_backend is not None:
                    lid = lid_backend.predict(wav_path, top_k=5)
                t_lid1 = time.perf_counter()

                # Route language hint (only if confident).
                route_lang = None
                route_lang_source = None
                prefer_path = bool(thresholds.get("route_lang_prefer_path", False))
                if prefer_path and bool(thresholds.get("route_lang_from_path", False)):
                    inferred = _infer_lang_from_relpath(ident.relpath)
                    if inferred:
                        route_lang = inferred
                        route_lang_source = "path"

                if route_lang is None and lid and lid.predictions:
                    top1 = lid.predictions[0]
                    top2p = lid.predictions[1].prob if len(lid.predictions) > 1 else 0.0
                    if float(top1.prob) >= float(thresholds.get("lid_min_prob", 0.90)) and (
                        float(top1.prob) - float(top2p)
                    ) >= float(thresholds.get("lid_min_margin", 0.20)):
                        route_lang = top1.code
                        route_lang_source = "lid"

                if route_lang is None and bool(thresholds.get("route_lang_from_path", False)):
                    inferred = _infer_lang_from_relpath(ident.relpath)
                    if inferred:
                        route_lang = inferred
                        route_lang_source = "path"

                # 3) ASR ensemble
                t_asr0 = time.perf_counter()
                asr_results: dict[str, ASRResult] = {}
                asr_errors: dict[str, str] = {}
                primary_asr_key = None
                det = None
                router_key: str | None = None

                # Whisper: deterministic + a small sampling diversity pass.
                try:


**Considering gemini support for run commands**

The current benchmark run doesn't accept gemini options, but the main `run` command does, so I'll check CLI options to verify this and plan to test LLMs using `maya-transcribe run` with gemini before deciding whether to extend `bench run`.


sed -n '200,320p' /home/ubuntu/maya-transcribe/maya_transcribe/cli.py


Output:
Exit code: 0
Wall time: 0 seconds
Output:

    prompt = load_prompt_template(prompt_file)
    backend = GeminiASR(config=GeminiConfig(model=model, max_output_tokens=max_output_tokens), prompt_template=prompt)
    res = backend.transcribe(audio_path=audio, language=language)
    typer.echo(res.text)

@app.command()
def run(
    segments_dir: Path = typer.Option(..., exists=True, file_okay=False, help="Directory containing audio segments."),
    out: Path = typer.Option(Path("outputs/run"), help="Output directory."),
    whisper_model: str = typer.Option("large-v3", help="Whisper model size/path for faster-whisper."),
    lid_model_id: str = typer.Option("facebook/mms-lid-256", help="HuggingFace MMS-LID model id."),
    require_lid_backend: bool = typer.Option(True, help="Fail if MMS-LID backend can't initialize."),
    transformers_whisper_model: list[str] = typer.Option(
        [], help="Extra whisper-family HF models to run via transformers (repeatable)."
    ),
    transformers_whisper_model_map: Path | None = typer.Option(
        None, help="JSON map {lang_code: hf_model_id} for per-language Whisper-family HF models (transformers)."
    ),
    # MMS ASR for CTC-based model diversity (per master.md plan).
    mms_asr_model: str | None = typer.Option(
        None, help="Optional MMS ASR model id (e.g., facebook/mms-1b-all) for CTC-based diversity."
    ),
    # IndicConformer for Conformer architecture diversity (per master.md plan).
    indicconformer_model_map: Path | None = typer.Option(
        None, help="JSON map {lang_code: nemo_model_name} for per-language IndicConformer models."
    ),
    indicconformer_default_model: str | None = typer.Option(
        None, help="Default IndicConformer NeMo model name for architecture diversity."
    ),
    gemini_model: str | None = typer.Option(None, help="Optional Gemini model name (adds Gemini as a hypothesis)."),
    gemini_prompt_file: Path | None = typer.Option(Path("Prompt.md"), help="Prompt file for Gemini."),
    gemini_hard_only: bool = typer.Option(True, help="Only call Gemini on 'hard' segments."),
    ctc_model: str | None = typer.Option(None, help="Optional HF CTC model id for audio-grounded fit scoring."),
    ctc_model_map: Path | None = typer.Option(None, help="JSON map {lang_code: hf_model_id} for CTC scoring."),
    require_ctc: bool = typer.Option(True, help="When CTC is enabled, require a CTC score to accept."),
    min_ctc_margin: float = typer.Option(0.05, help="Min (second-best - best) CTC loss margin to auto-accept."),
    trim_silence: bool = typer.Option(True, help="Trim leading/trailing silence (ffmpeg silenceremove)."),
    silence_threshold_db: float = typer.Option(-40.0, help="Silence threshold in dB for trimming."),
    min_silence_sec: float = typer.Option(0.15, help="Min silence duration for trimming."),
    limit: int | None = typer.Option(None, help="Process only the first N segments."),
    thresholds_json: Path | None = typer.Option(None, help="Optional JSON file of threshold overrides (verify + routing)."),
) -> None:
    """
    End-to-end pipeline: preprocess -> LID -> ASR ensemble -> ROVER -> verify.
    """
    from maya_transcribe.pipeline import run_pipeline

    ctc_map = None
    if ctc_model_map is not None:
        ctc_map = json.loads(ctc_model_map.read_text(encoding="utf-8"))
        if not isinstance(ctc_map, dict):
            raise typer.BadParameter("ctc_model_map must be a JSON object {lang_code: model_id}")

    tf_map = None
    if transformers_whisper_model_map is not None:
        tf_map = json.loads(transformers_whisper_model_map.read_text(encoding="utf-8"))
        if not isinstance(tf_map, dict):
            raise typer.BadParameter("transformers_whisper_model_map must be a JSON object {lang_code: model_id}")

    ic_map = None
    if indicconformer_model_map is not None:
        ic_map = json.loads(indicconformer_model_map.read_text(encoding="utf-8"))
        if not isinstance(ic_map, dict):
            raise typer.BadParameter("indicconformer_model_map must be a JSON object {lang_code: nemo_model_name}")

    thresholds: dict[str, object] = {}
    if thresholds_json is not None:
        loaded = json.loads(thresholds_json.read_text(encoding="utf-8"))
        if not isinstance(loaded, dict):
            raise typer.BadParameter("thresholds_json must be a JSON object")
        thresholds = loaded
    if ctc_model or ctc_map:
        thresholds["require_ctc"] = require_ctc
        thresholds["min_ctc_margin"] = float(min_ctc_margin)

    results_path = run_pipeline(
        segments_dir=segments_dir,
        out_dir=out,
        lid_model_id=lid_model_id,
        require_lid_backend=require_lid_backend,
        whisper_model=whisper_model,
        extra_transformers_whisper_models=transformers_whisper_model,
        transformers_whisper_model_map=tf_map,
        mms_asr_model=mms_asr_model,
        indicconformer_model_map=ic_map,
        indicconformer_default_model=indicconformer_default_model,
        gemini_model=gemini_model,
        gemini_prompt_path=gemini_prompt_file,
        gemini_hard_only=gemini_hard_only,
        ctc_model=ctc_model,
        ctc_model_map=ctc_map,
        trim_silence=trim_silence,
        silence_threshold_db=silence_threshold_db,
        min_silence_sec=min_silence_sec,
        limit=limit,
        thresholds=thresholds,
    )
    typer.echo(f"Wrote: {results_path}")
    typer.echo(f"Wrote: {out / 'summary.json'}")

@app.command()
def eval(
    results: Path = typer.Option(..., exists=True, dir_okay=False, help="Pipeline results JSONL (outputs/run/results.jsonl)."),
    refs: Path = typer.Option(..., exists=True, dir_okay=False, help="Reference transcripts JSONL."),
    out: Path | None = typer.Option(None, help="Write summary JSON to this path."),
    hypothesis: str = typer.Option('final', help='Hypothesis to evaluate: "final", "rover", or "asr:<key>".'),
    oracle: bool = typer.Option(True, help="Compute oracle-best and selection-gap stats across all candidates."),
    language_source: str = typer.Option("route", help="Use language from: 'route' (LID) or 'refs' (refs.jsonl)."),
    decision: list[str] = typer.Option([], help="Filter to verification decision(s): accept/human/reject (repeatable)."),
) -> None:
    """
    Compute WER/CER for a hypothesis against a reference JSONL keyed by `segment_id`.
    """
    from maya_transcribe.eval import evaluate

    summary = evaluate(
        results_path=results,
        refs_path=refs,


**Clarifying audio LLM options**

Gemini supports audio benchmarking now, while Gemma3n appears text-only and unsuitable for audio transcription; Voxtral integration is possible if API or weights are available. I'll propose running Gemini as a baseline and ask for Mistral API access or weights to support Voxtral.


**Checking for Mistral API key**

I'm planning to detect the Mistral API key in environment variables to proceed with Voxtral integration and benchmark runs without waiting for user input.


printenv | rg -n "MISTRAL|VOXTRAL|GEMMA|GEMINI"


Output:
Exit code: 1
Wall time: 0 seconds
Output:


**Checking dotenv usage in CLI**


sed -n '1,120p' /home/ubuntu/maya-transcribe/maya_transcribe/cli.py


Output:
Exit code: 0
Wall time: 0 seconds
Output:
from __future__ import annotations

import json
from pathlib import Path

import typer
from dotenv import load_dotenv

from maya_transcribe.audio import compute_qc, ffprobe, normalize_audio
from maya_transcribe.config import Settings
from maya_transcribe.dataset import find_segments_dir
from maya_transcribe.ids import segment_identity
from maya_transcribe.io import write_jsonl
from maya_transcribe.r2 import R2Client
from maya_transcribe.tar import safe_extract_tar

load_dotenv(dotenv_path=Path(".env"), override=False)

app = typer.Typer(add_completion=False, no_args_is_help=True)
bench_app = typer.Typer(add_completion=False, no_args_is_help=True, help="Benchmarking utilities.")
app.add_typer(bench_app, name="bench")

@app.command()
def download(
    video_id: str = typer.Option(..., help="YouTube video id (used to locate the tar in R2)."),
    out: Path = typer.Option(Path("data"), help="Output directory."),
    r2_key: str | None = typer.Option(None, help="Exact R2 object key (skips search)."),
    bucket: str | None = typer.Option(None, help="R2 bucket override (defaults to env R2_BUCKET)."),
    prefix: str | None = typer.Option(None, help="Optional R2 prefix override."),
    use_db: bool = typer.Option(False, help="Fetch `r2_tar_key` from Supabase Postgres (DATABASE_URL)."),
    dry_run: bool = typer.Option(False, help="Only show matched key(s); do not download."),
) -> None:
    """
    Download the cleaned/segmented tar for a video from R2 and extract it.
    """
    settings = Settings.load()
    r2 = R2Client.from_settings(settings)
    if bucket:
        r2 = R2Client(
            endpoint_url=r2.endpoint_url,
            access_key_id=r2.access_key_id,
            secret_access_key=r2.secret_access_key,
            bucket=bucket,
        )

    out.mkdir(parents=True, exist_ok=True)
    r2_prefix = prefix if prefix is not None else settings.r2_prefix

    matches: list[str]
    if use_db and not r2_key:
        from maya_transcribe.db import fetch_video_meta

        meta = fetch_video_meta(video_id)
        if meta and meta.r2_tar_key:
            r2_key = meta.r2_tar_key

    if r2_key:
        matches = [r2_key]
    else:
        best = r2.find_video_tar_key(video_id=video_id, prefix=r2_prefix)
        if not best:
            typer.echo("No matching objects found in R2.")
            raise typer.Exit(code=2)
        matches = [best]

    if dry_run:
        for key in matches:
            typer.echo(key)
        return

    key = matches[0]
    tar_path = r2.download_to(key=key, out_dir=out / video_id)
    extract_dir = out / video_id
    safe_extract_tar(tar_path, extract_dir=extract_dir)
    segments_dir = find_segments_dir(extract_dir)
    typer.echo(f"Downloaded: {tar_path}")
    typer.echo(f"Extracted to: {extract_dir}")
    if segments_dir:
        typer.echo(f"Segments dir: {segments_dir}")

@app.command()
def preprocess(
    segments_dir: Path = typer.Option(..., exists=True, file_okay=False, help="Directory containing audio segments."),
    out: Path = typer.Option(Path("outputs/preprocessed"), help="Output directory for normalized audio + QC."),
    sample_rate: int = typer.Option(16000, help="Target sample rate."),
    trim_silence: bool = typer.Option(True, help="Trim leading/trailing silence (ffmpeg silenceremove)."),
    silence_threshold_db: float = typer.Option(-40.0, help="Silence threshold in dB for trimming."),
    min_silence_sec: float = typer.Option(0.15, help="Min silence duration for trimming."),
) -> None:
    """
    Normalize audio to 16kHz mono WAV, optionally trim silence, and write QC metrics.
    """
    out.mkdir(parents=True, exist_ok=True)

    audio_files: list[Path] = []
    for ext in ("*.wav", "*.mp3", "*.m4a", "*.flac", "*.ogg", "*.opus"):
        audio_files.extend(sorted(segments_dir.rglob(ext)))
    audio_files = sorted(set(audio_files))

    if not audio_files:
        typer.echo(f"No audio files found under: {segments_dir}")
        raise typer.Exit(code=2)

    rows = []
    norm_dir = out / "audio"
    for src in audio_files:
        ident = segment_identity(src, root=segments_dir)
        segment_id = ident.segment_id
        dst = norm_dir / f"{segment_id}.wav"
        normalize_audio(
            src,
            dst,
            sample_rate=sample_rate,
            mono=True,
            trim_silence=trim_silence,
            silence_threshold_db=silence_threshold_db,
            min_silence_sec=min_silence_sec,
        )


**Searching for threshold file**

I'm searching the repo for any thresholds JSON or related files to match the previous gating settings for consistent evaluation.


rg -n "thresholds" /home/ubuntu/maya-transcribe -S


Output:
Exit code: 0
Wall time: 0.1 seconds
Output:
/home/ubuntu/maya-transcribe/docs/Reference1.md:114:* Whisper’s own transcription code supports fallback logic using thresholds like `compression_ratio_threshold`, `logprob_threshold`, and `no_speech_threshold`. ([GitHub][12])
/home/ubuntu/maya-transcribe/docs/Reference5.md:162:**Confidence thresholds for routing:**
/home/ubuntu/maya-transcribe/maya_transcribe/verify.py:40:    thresholds: dict[str, Any] | None = None,
/home/ubuntu/maya-transcribe/maya_transcribe/verify.py:45:    thresholds = thresholds or {}
/home/ubuntu/maya-transcribe/maya_transcribe/verify.py:46:    min_duration = float(thresholds.get("min_duration_sec", 0.8))
/home/ubuntu/maya-transcribe/maya_transcribe/verify.py:47:    max_clipping = float(thresholds.get("max_clipping_fraction", 0.01))
/home/ubuntu/maya-transcribe/maya_transcribe/verify.py:48:    max_disagreement = float(thresholds.get("max_rover_disagreement_rate", 0.03))
/home/ubuntu/maya-transcribe/maya_transcribe/verify.py:49:    lid_min_prob = float(thresholds.get("lid_min_prob", 0.90))
/home/ubuntu/maya-transcribe/maya_transcribe/verify.py:50:    lid_min_margin = float(thresholds.get("lid_min_margin", 0.20))
/home/ubuntu/maya-transcribe/maya_transcribe/verify.py:51:    require_lid = bool(thresholds.get("require_lid", True))
/home/ubuntu/maya-transcribe/maya_transcribe/verify.py:54:    min_avg_logprob = float(thresholds.get("min_avg_logprob_mean", -1.0))
/home/ubuntu/maya-transcribe/maya_transcribe/verify.py:55:    max_compression_ratio = float(thresholds.get("max_compression_ratio_mean", 2.4))
/home/ubuntu/maya-transcribe/maya_transcribe/verify.py:58:    require_ctc = bool(thresholds.get("require_ctc", False))
/home/ubuntu/maya-transcribe/maya_transcribe/verify.py:59:    min_ctc_margin = float(thresholds.get("min_ctc_margin", 0.0))
/home/ubuntu/maya-transcribe/maya_transcribe/verify.py:60:    min_ctc_token_coverage = float(thresholds.get("min_ctc_token_coverage", 0.85))
/home/ubuntu/maya-transcribe/maya_transcribe/verify.py:61:    min_ctc_token_score_mean = float(thresholds.get("min_ctc_token_score_mean", -3.0))
/home/ubuntu/maya-transcribe/maya_transcribe/verify.py:62:    fail_on_ctc_span_mismatch = bool(thresholds.get("fail_on_ctc_span_mismatch", True))
/home/ubuntu/maya-transcribe/maya_transcribe/cli.py:242:    thresholds_json: Path | None = typer.Option(None, help="Optional JSON file of threshold overrides (verify + routing)."),
/home/ubuntu/maya-transcribe/maya_transcribe/cli.py:267:    thresholds: dict[str, object] = {}
/home/ubuntu/maya-transcribe/maya_transcribe/cli.py:268:    if thresholds_json is not None:
/home/ubuntu/maya-transcribe/maya_transcribe/cli.py:269:        loaded = json.loads(thresholds_json.read_text(encoding="utf-8"))
/home/ubuntu/maya-transcribe/maya_transcribe/cli.py:271:            raise typer.BadParameter("thresholds_json must be a JSON object")
/home/ubuntu/maya-transcribe/maya_transcribe/cli.py:272:        thresholds = loaded
/home/ubuntu/maya-transcribe/maya_transcribe/cli.py:274:        thresholds["require_ctc"] = require_ctc
/home/ubuntu/maya-transcribe/maya_transcribe/cli.py:275:        thresholds["min_ctc_margin"] = float(min_ctc_margin)
/home/ubuntu/maya-transcribe/maya_transcribe/cli.py:297:        thresholds=thresholds,
/home/ubuntu/maya-transcribe/maya_transcribe/cli.py:393:    thresholds_json: Path | None = typer.Option(None, help="Optional JSON file of threshold overrides (verify + routing)."),
/home/ubuntu/maya-transcribe/maya_transcribe/cli.py:433:    thresholds: dict[str, object] = {"route_lang_from_path": True, "route_lang_prefer_path": True}
/home/ubuntu/maya-transcribe/maya_transcribe/cli.py:434:    if thresholds_json is not None:
/home/ubuntu/maya-transcribe/maya_transcribe/cli.py:435:        loaded = json.loads(thresholds_json.read_text(encoding="utf-8"))
/home/ubuntu/maya-transcribe/maya_transcribe/cli.py:437:            raise typer.BadParameter("thresholds_json must be a JSON object")
/home/ubuntu/maya-transcribe/maya_transcribe/cli.py:438:        thresholds.update(loaded)
/home/ubuntu/maya-transcribe/maya_transcribe/cli.py:440:        thresholds["require_ctc"] = require_ctc
/home/ubuntu/maya-transcribe/maya_transcribe/cli.py:441:        thresholds["min_ctc_margin"] = float(min_ctc_margin)
/home/ubuntu/maya-transcribe/maya_transcribe/cli.py:460:        thresholds=thresholds,
/home/ubuntu/maya-transcribe/maya_transcribe/pipeline.py:85:    thresholds: dict[str, Any] | None = None,
/home/ubuntu/maya-transcribe/maya_transcribe/pipeline.py:91:    thresholds = thresholds or {}
/home/ubuntu/maya-transcribe/maya_transcribe/pipeline.py:92:    if (ctc_model or ctc_model_map) and "require_ctc" not in thresholds:
/home/ubuntu/maya-transcribe/maya_transcribe/pipeline.py:93:        thresholds["require_ctc"] = True
/home/ubuntu/maya-transcribe/maya_transcribe/pipeline.py:94:    if (ctc_model or ctc_model_map) and "min_ctc_margin" not in thresholds:
/home/ubuntu/maya-transcribe/maya_transcribe/pipeline.py:95:        thresholds["min_ctc_margin"] = 0.05
/home/ubuntu/maya-transcribe/maya_transcribe/pipeline.py:209:                prefer_path = bool(thresholds.get("route_lang_prefer_path", False))
/home/ubuntu/maya-transcribe/maya_transcribe/pipeline.py:210:                if prefer_path and bool(thresholds.get("route_lang_from_path", False)):
/home/ubuntu/maya-transcribe/maya_transcribe/pipeline.py:219:                    if float(top1.prob) >= float(thresholds.get("lid_min_prob", 0.90)) and (
/home/ubuntu/maya-transcribe/maya_transcribe/pipeline.py:221:                    ) >= float(thresholds.get("lid_min_margin", 0.20)):
/home/ubuntu/maya-transcribe/maya_transcribe/pipeline.py:225:                if route_lang is None and bool(thresholds.get("route_lang_from_path", False)):
/home/ubuntu/maya-transcribe/maya_transcribe/pipeline.py:249:                run_sampling = bool(thresholds.get("whisper_run_sampling", default_whisper_run_sampling))
/home/ubuntu/maya-transcribe/maya_transcribe/pipeline.py:314:                            or (avg_lp is not None and float(avg_lp) < float(thresholds.get("min_avg_logprob_mean", -1.0)))
/home/ubuntu/maya-transcribe/maya_transcribe/pipeline.py:317:                                and float(comp) > float(thresholds.get("max_compression_ratio_mean", 2.4))
/home/ubuntu/maya-transcribe/maya_transcribe/pipeline.py:348:                if bool(thresholds.get("filter_hypotheses_by_script", False)) and route_lang:
/home/ubuntu/maya-transcribe/maya_transcribe/pipeline.py:367:                include_sampling_opt = thresholds.get("rover_include_sampling")
/home/ubuntu/maya-transcribe/maya_transcribe/pipeline.py:379:                prefer_router_opt = thresholds.get("rover_prefer_router")
/home/ubuntu/maya-transcribe/maya_transcribe/pipeline.py:398:                if router_key and router_key in rover_hyps and bool(thresholds.get("final_prefer_router", False)):
/home/ubuntu/maya-transcribe/maya_transcribe/pipeline.py:401:                elif bool(thresholds.get("final_prefer_transformers", False)):
/home/ubuntu/maya-transcribe/maya_transcribe/pipeline.py:431:                    thresholds=thresholds,
/home/ubuntu/maya-transcribe/master.md:136:   - Tune: LID thresholds, per-language ROVER weights, verification-gate thresholds.
/home/ubuntu/maya-transcribe/Agents.md:29:- A unified **Master Plan** (pipeline + thresholds + verification + human loop)
/home/ubuntu/maya-transcribe/Agents.md:59:- Must-meet thresholds:
/home/ubuntu/maya-transcribe/Agents.md:84:- **Adapt** = use but modify (thresholds, different model, narrowed scope)
/home/ubuntu/maya-transcribe/Agents.md:201:- **Must-meet thresholds (calibrate):**
/home/ubuntu/maya-transcribe/Agents.md:221:- **Adapt:** Use, but with constraints/thresholds or narrower scope.
/home/ubuntu/maya-transcribe/Agents.md:256:- Suggestion: MMS LID (+ VoxLingua107) and strict routing thresholds
/home/ubuntu/maya-transcribe/Agents.md:258:  - Why: Spoken LID routing is foundational; thresholds are sensible starting points but must be calibrated.
/home/ubuntu/maya-transcribe/Agents.md:352:  - Why: Good for precision; but make thresholds configurable so we don’t over-reject short segments; include an “unknown/mixed” route.
/home/ubuntu/maya-transcribe/Agents.md:387:- G1: No specific, audio-grounded verification metric definition (alignment/CTC score thresholds).
/home/ubuntu/maya-transcribe/Agents.md:438:- [ ] Implement LID: MMS-LID + thresholds.
/home/ubuntu/maya-transcribe/Agents.md:494:- Suggestion: Aggressive forced alignment discard thresholds
/home/ubuntu/maya-transcribe/Agents.md:496:  - Why: Keep the “discard on poor fit” principle, but tune thresholds on a gold set and prefer “reject segment” over “silently drop words.”
/home/ubuntu/maya-transcribe/Agents.md:510:- Compute audio QC metrics and reject early if bad (precision-first): clipping %, extreme loudness, very low speech ratio, suspiciously short effective speech, etc. (Inference: exact metrics/thresholds must be tuned.)
/home/ubuntu/maya-transcribe/Agents.md:660:3. **Calibrate:** Tune LID thresholds, ROVER weights, and verification gates on the gold set to maximize precision of accepted segments.
/home/ubuntu/maya-transcribe/Agents.md:673:- Make everything config-driven (`models`, `decode_params`, `thresholds`, `weights`), and record config snapshots with outputs.
/home/ubuntu/maya-transcribe/Agents.md:713:### Tune thresholds and weights
/home/ubuntu/maya-transcribe/Agents.md:715:  - LID “confident route” thresholds
/home/ubuntu/maya-transcribe/Agents.md:717:  - verification gate thresholds (CTC fit / alignment pass)
/home/ubuntu/maya-transcribe/Agents.md:723:  - recalibrate thresholds/weights
/home/ubuntu/maya-transcribe/Agents.md:730:4. Tune routing/verification thresholds on the gold set until the accepted pool hits your precision target.


**Locating threshold configuration**

Looking for a thresholds JSON file in outputs to verify if the "relaxed threshold" corresponds to the change in max_rover_disagreement_rate from 0.03 to 0.15.


find /home/ubuntu/maya-transcribe/outputs -type f -name "*thresholds*.json" -o -name "*thresholds*.yaml"


Output:
Exit code: 0
Wall time: 0 seconds
Output:


**Searching for threshold usage**

I'm searching the repo for references to the specific threshold key to uncover how and where it's configured or used.


rg -n "max_rover_disagreement_rate|max_disagreement|disagreement" /home/ubuntu/maya-transcribe -S