Output:
Exit code: 124
Wall time: 200.2 seconds
Output:
command timed out after 200197 milliseconds
/home/ubuntu/.local/lib/python3.10/site-packages/torch/cuda/__init__.py:63: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you.
  import pynvml  # type: ignore[import]

download:   0%|          | 0/100 [00:00<?, ?it/s]
download:   1%|          | 1/100 [00:00<01:27,  1.13it/s]
download:   2%|▏         | 2/100 [00:01<00:46,  2.10it/s]
download:   4%|▍         | 4/100 [00:01<00:22,  4.30it/s]
download:   5%|▌         | 5/100 [00:01<00:24,  3.85it/s]
download:   6%|▌         | 6/100 [00:01<00:21,  4.44it/s]
download:   7%|▋         | 7/100 [00:01<00:22,  4.14it/s]
download:   8%|▊         | 8/100 [00:02<00:19,  4.72it/s]
download:   9%|▉         | 9/100 [00:02<00:18,  4.99it/s]
download:  11%|█         | 11/100 [00:02<00:17,  5.12it/s]
download:  12%|█▏        | 12/100 [00:02<00:15,  5.75it/s]
download:  15%|█▌        | 15/100 [00:03<00:10,  8.47it/s]
download:  16%|█▌        | 16/100 [00:03<00:10,  8.04it/s]
download:  18%|█▊        | 18/100 [00:03<00:12,  6.70it/s]
download:  19%|█▉        | 19/100 [00:03<00:13,  6.19it/s]
download:  21%|██        | 21/100 [00:03<00:10,  7.43it/s]
download:  24%|██▍       | 24/100 [00:04<00:07, 10.42it/s]
download:  26%|██▌       | 26/100 [00:04<00:06, 10.74it/s]
download:  30%|███       | 30/100 [00:04<00:05, 11.72it/s]
download:  32%|███▏      | 32/100 [00:04<00:06,  9.76it/s]
download:  35%|███▌      | 35/100 [00:05<00:05, 11.40it/s]
download:  37%|███▋      | 37/100 [00:05<00:09,  6.79it/s]
download:  38%|███▊      | 38/100 [00:05<00:08,  6.99it/s]
download:  40%|████      | 40/100 [00:05<00:07,  8.16it/s]
download:  42%|████▏     | 42/100 [00:06<00:08,  7.03it/s]
download:  44%|████▍     | 44/100 [00:06<00:07,  7.71it/s]
download:  46%|████▌     | 46/100 [00:07<00:10,  5.28it/s]
download:  47%|████▋     | 47/100 [00:07<00:10,  5.21it/s]
download:  48%|████▊     | 48/100 [00:07<00:11,  4.71it/s]
download:  49%|████▉     | 49/100 [00:07<00:10,  5.08it/s]
download:  50%|█████     | 50/100 [00:08<00:10,  4.92it/s]
download:  52%|█████▏    | 52/100 [00:09<00:17,  2.78it/s]
download:  53%|█████▎    | 53/100 [00:09<00:15,  3.07it/s]
download:  55%|█████▌    | 55/100 [00:09<00:11,  4.08it/s]
download:  56%|█████▌    | 56/100 [00:09<00:11,  3.93it/s]
download:  57%|█████▋    | 57/100 [00:10<00:11,  3.89it/s]
download:  59%|█████▉    | 59/100 [00:10<00:07,  5.60it/s]
download:  60%|██████    | 60/100 [00:10<00:06,  5.86it/s]
download:  62%|██████▏   | 62/100 [00:10<00:04,  7.96it/s]
download:  65%|██████▌   | 65/100 [00:10<00:03, 11.53it/s]
download:  67%|██████▋   | 67/100 [00:10<00:02, 12.62it/s]
download:  69%|██████▉   | 69/100 [00:11<00:02, 10.47it/s]
download:  71%|███████   | 71/100 [00:11<00:03,  8.36it/s]
download:  73%|███████▎  | 73/100 [00:11<00:03,  7.38it/s]
download:  74%|███████▍  | 74/100 [00:11<00:03,  7.14it/s]
download:  77%|███████▋  | 77/100 [00:12<00:02, 10.31it/s]
download:  80%|████████  | 80/100 [00:12<00:01, 12.61it/s]
download:  82%|████████▏ | 82/100 [00:12<00:01, 11.03it/s]
download:  84%|████████▍ | 84/100 [00:12<00:01,  8.83it/s]
download:  86%|████████▌ | 86/100 [00:12<00:01, 10.22it/s]
download:  88%|████████▊ | 88/100 [00:13<00:01, 11.43it/s]
download:  90%|█████████ | 90/100 [00:13<00:01,  7.94it/s]
download:  93%|█████████▎| 93/100 [00:13<00:00, 10.09it/s]
download:  95%|█████████▌| 95/100 [00:13<00:00, 10.21it/s]
download:  97%|█████████▋| 97/100 [00:14<00:00,  9.91it/s]
download:  99%|█████████▉| 99/100 [00:14<00:00, 10.77it/s]
download: 100%|██████████| 100/100 [00:14<00:00,  7.00it/s]
Using GPU-accelerated batch processor (batch_size=32)
  GPU path uses EXACT same chunking/metrics as CPU - only resampling is GPU-accelerated
GPUResampler: Using NVIDIA A100-SXM4-80GB

process (GPU):   0%|          | 0/100 [00:00<?, ?it/s]
process (GPU):   4%|▍         | 4/100 [00:06<02:25,  1.52s/it]
process (GPU):   5%|▌         | 5/100 [00:09<03:23,  2.14s/it]
process (GPU):   7%|▋         | 7/100 [00:18<04:46,  3.08s/it]
process (GPU):   8%|▊         | 8/100 [00:20<04:19,  2.82s/it]
process (GPU):   9%|▉         | 9/100 [00:22<04:03,  2.67s/it]
process (GPU):  10%|█         | 10/100 [00:25<04:05,  2.73s/it]
process (GPU):  11%|█         | 11/100 [00:38<08:06,  5.46s/it]
process (GPU):  12%|█▏        | 12/100 [00:45<08:30,  5.80s/it]
process (GPU):  13%|█▎        | 13/100 [00:47<06:49,  4.71s/it]
process (GPU):  14%|█▍        | 14/100 [01:04<11:51,  8.28s/it]
process (GPU):  15%|█▌        | 15/100 [01:08<10:12,  7.21s/it]
process (GPU):  17%|█▋        | 17/100 [01:14<07:05,  5.13s/it]
process (GPU):  19%|█▉        | 19/100 [01:44<12:23,  9.18s/it]
process (GPU):  20%|██        | 20/100 [01:47<10:14,  7.68s/it]
process (GPU):  21%|██        | 21/100 [02:05<13:20, 10.13s/it]
process (GPU):  22%|██▏       | 22/100 [02:28<17:43, 13.63s/it]
process (GPU):  24%|██▍       | 24/100 [02:52<16:23, 12.94s/it]


Summary: auto

## Skills
A skill is a set of local instructions to follow that is stored in a `SKILL.md` file. Below is the list of skills that can be used. Each entry includes a name, description, and file path so you can open the source for full instructions when using a specific skill.
### Available skills
- skill-creator: Guide for creating effective skills. This skill should be used when users want to create a new skill (or update an existing skill) that extends Codex's capabilities with specialized knowledge, workflows, or tool integrations. (file: /home/ubuntu/.codex/skills/.system/skill-creator/SKILL.md)
- skill-installer: Install Codex skills into $CODEX_HOME/skills from a curated list or a GitHub repo path. Use when a user asks to list installable skills, install a curated skill, or install a skill from another repo (including private repos). (file: /home/ubuntu/.codex/skills/.system/skill-installer/SKILL.md)
### How to use skills
- Discovery: The list above is the skills available in this session (name + description + file path). Skill bodies live on disk at the listed paths.
- Trigger rules: If the user names a skill (with `$SkillName` or plain text) OR the task clearly matches a skill's description shown above, you must use that skill for that turn. Multiple mentions mean use them all. Do not carry skills across turns unless re-mentioned.
- Missing/blocked: If a named skill isn't in the list or the path can't be read, say so briefly and continue with the best fallback.
- How to use a skill (progressive disclosure):
  1) After deciding to use a skill, open its `SKILL.md`. Read only enough to follow the workflow.
  2) If `SKILL.md` points to extra folders such as `references/`, load only the specific files needed for the request; don't bulk-load everything.
  3) If `scripts/` exist, prefer running or patching them instead of retyping large code blocks.
  4) If `assets/` or templates exist, reuse them instead of recreating from scratch.
- Coordination and sequencing:
  - If multiple skills apply, choose the minimal set that covers the request and state the order you'll use them.
  - Announce which skill(s) you're using and why (one short line). If you skip an obvious skill, say why.
- Context hygiene:
  - Keep context small: summarize long sections instead of pasting them; only load extra files when needed.
  - Avoid deep reference-chasing: prefer opening only files directly linked from `SKILL.md` unless you're blocked.
  - When variants exist (frameworks, providers, domains), pick only the relevant reference file(s) and note that choice.
- Safety and fallback: If a skill can't be applied cleanly (missing files, unclear instructions), state the issue, pick the next-best approach, and continue.


**Planning longer timeout for processing**

The previous run timed out after 200 seconds and likely terminated mid-process, so I'll re-run with a much longer timeout (900000 ms) to accommodate slower GPU decoding and handle existing cached downloads without re-downloading.


**Confirming overwrite behavior and rerun parameters**

The output tar files will be overwritten on rerun since they open in write mode, so partial results won't accumulate. I'll rerun with a longer timeout (900000 ms), keep the GPU batch size at 32, and maintain worker counts while limiting to 100 videos.


time python3 scripts/prepare_shards.py --language hindi --limit 100 --output-dir /tmp/xcodec_validate/gpu_100 --tmp-dir /tmp/xcodec_validate/gpu_tmp_100 --download-workers 12 --gpu --gpu-batch-size 32 --gpu-decode-workers 8 --gpu-encode-workers 8


Output:
Exit code: 124
Wall time: 900.3 seconds
Output:
command timed out after 900294 milliseconds
/home/ubuntu/.local/lib/python3.10/site-packages/torch/cuda/__init__.py:63: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you.
  import pynvml  # type: ignore[import]

download:   0%|          | 0/100 [00:00<?, ?it/s]
download:   1%|          | 1/100 [00:00<01:37,  1.01it/s]
download:   2%|▏         | 2/100 [00:01<01:12,  1.35it/s]
download:   4%|▍         | 4/100 [00:01<00:32,  2.93it/s]
download:   5%|▌         | 5/100 [00:01<00:27,  3.52it/s]
download:   6%|▌         | 6/100 [00:02<00:21,  4.37it/s]
download:   8%|▊         | 8/100 [00:02<00:20,  4.51it/s]
download:   9%|▉         | 9/100 [00:02<00:19,  4.55it/s]
download:  11%|█         | 11/100 [00:03<00:18,  4.88it/s]
download:  13%|█▎        | 13/100 [00:03<00:14,  5.84it/s]
download:  15%|█▌        | 15/100 [00:03<00:11,  7.31it/s]
download:  18%|█▊        | 18/100 [00:03<00:08,  9.78it/s]
download:  20%|██        | 20/100 [00:03<00:07, 11.31it/s]
download:  24%|██▍       | 24/100 [00:03<00:05, 14.74it/s]
download:  26%|██▌       | 26/100 [00:04<00:05, 14.22it/s]
download:  28%|██▊       | 28/100 [00:04<00:06, 11.58it/s]
download:  30%|███       | 30/100 [00:04<00:08,  8.09it/s]
download:  32%|███▏      | 32/100 [00:04<00:07,  9.27it/s]
download:  34%|███▍      | 34/100 [00:05<00:06, 10.23it/s]
download:  36%|███▌      | 36/100 [00:05<00:08,  7.39it/s]
download:  38%|███▊      | 38/100 [00:05<00:08,  7.35it/s]
download:  40%|████      | 40/100 [00:05<00:07,  7.89it/s]
download:  41%|████      | 41/100 [00:06<00:07,  8.01it/s]
download:  43%|████▎     | 43/100 [00:06<00:06,  8.52it/s]
download:  45%|████▌     | 45/100 [00:06<00:06,  7.88it/s]
download:  46%|████▌     | 46/100 [00:06<00:07,  7.36it/s]
download:  48%|████▊     | 48/100 [00:06<00:06,  8.62it/s]
download:  50%|█████     | 50/100 [00:07<00:05,  9.86it/s]
download:  52%|█████▏    | 52/100 [00:07<00:05,  8.34it/s]
download:  54%|█████▍    | 54/100 [00:07<00:06,  6.76it/s]
download:  55%|█████▌    | 55/100 [00:08<00:07,  5.73it/s]
download:  56%|█████▌    | 56/100 [00:08<00:07,  6.24it/s]
download:  59%|█████▉    | 59/100 [00:08<00:05,  7.25it/s]
download:  60%|██████    | 60/100 [00:08<00:05,  7.22it/s]
download:  64%|██████▍   | 64/100 [00:08<00:03, 10.14it/s]
download:  66%|██████▌   | 66/100 [00:09<00:03, 10.04it/s]
download:  68%|██████▊   | 68/100 [00:09<00:03,  8.20it/s]
download:  70%|███████   | 70/100 [00:09<00:03,  9.28it/s]
download:  74%|███████▍  | 74/100 [00:09<00:02, 12.98it/s]
download:  77%|███████▋  | 77/100 [00:10<00:01, 12.59it/s]
download:  79%|███████▉  | 79/100 [00:10<00:01, 13.53it/s]
download:  81%|████████  | 81/100 [00:10<00:01, 10.06it/s]
download:  84%|████████▍ | 84/100 [00:10<00:01, 12.95it/s]
download:  86%|████████▌ | 86/100 [00:10<00:01, 12.88it/s]
download:  88%|████████▊ | 88/100 [00:10<00:01, 11.43it/s]
download:  90%|█████████ | 90/100 [00:11<00:00, 11.03it/s]
download:  92%|█████████▏| 92/100 [00:11<00:00, 10.88it/s]
download:  94%|█████████▍| 94/100 [00:11<00:00, 10.39it/s]
download:  96%|█████████▌| 96/100 [00:11<00:00, 10.62it/s]
download:  99%|█████████▉| 99/100 [00:12<00:00, 10.77it/s]
download: 100%|██████████| 100/100 [00:12<00:00,  8.15it/s]
Using GPU-accelerated batch processor (batch_size=32)
  GPU path uses EXACT same chunking/metrics as CPU - only resampling is GPU-accelerated
GPUResampler: Using NVIDIA A100-SXM4-80GB

process (GPU):   0%|          | 0/100 [00:00<?, ?it/s]
process (GPU):   2%|▏         | 2/100 [00:03<02:46,  1.70s/it]
process (GPU):   4%|▍         | 4/100 [00:08<03:32,  2.21s/it]
process (GPU):   5%|▌         | 5/100 [00:14<05:24,  3.42s/it]
process (GPU):   6%|▌         | 6/100 [00:18<05:23,  3.45s/it]
process (GPU):   7%|▋         | 7/100 [00:20<04:32,  2.93s/it]
process (GPU):   8%|▊         | 8/100 [00:26<06:08,  4.00s/it]
process (GPU):   9%|▉         | 9/100 [00:31<06:17,  4.15s/it]
process (GPU):  10%|█         | 10/100 [00:33<05:28,  3.65s/it]
process (GPU):  11%|█         | 11/100 [00:38<05:51,  3.95s/it]
process (GPU):  12%|█▏        | 12/100 [00:42<05:48,  3.97s/it]
process (GPU):  13%|█▎        | 13/100 [00:43<04:47,  3.31s/it]
process (GPU):  15%|█▌        | 15/100 [00:50<04:30,  3.19s/it]
process (GPU):  16%|█▌        | 16/100 [01:09<10:01,  7.16s/it]
process (GPU):  17%|█▋        | 17/100 [01:22<12:01,  8.69s/it]
process (GPU):  18%|█▊        | 18/100 [01:59<22:34, 16.52s/it]
process (GPU):  19%|█▉        | 19/100 [02:06<18:41, 13.85s/it]
process (GPU):  20%|██        | 20/100 [02:10<14:27, 10.85s/it]
process (GPU):  21%|██        | 21/100 [02:39<21:25, 16.27s/it]
process (GPU):  23%|██▎       | 23/100 [03:17<22:27, 17.50s/it]
process (GPU):  24%|██▍       | 24/100 [03:19<17:16, 13.64s/it]
process (GPU):  25%|██▌       | 25/100 [03:20<12:57, 10.36s/it]
process (GPU):  26%|██▌       | 26/100 [03:57<21:45, 17.64s/it]
process (GPU):  28%|██▊       | 28/100 [04:01<13:00, 10.84s/it]
process (GPU):  29%|██▉       | 29/100 [04:05<10:44,  9.08s/it]
process (GPU):  30%|███       | 30/100 [04:24<13:45, 11.80s/it]
process (GPU):  32%|███▏      | 32/100 [04:36<10:37,  9.38s/it]
process (GPU):  33%|███▎      | 33/100 [04:40<08:54,  7.97s/it]
process (GPU):  34%|███▍      | 34/100 [04:49<09:08,  8.31s/it]
process (GPU):  35%|███▌      | 35/100 [04:52<07:29,  6.92s/it]
process (GPU):  36%|███▌      | 36/100 [04:57<06:53,  6.47s/it]
process (GPU):  37%|███▋      | 37/100 [05:15<10:03,  9.58s/it]
process (GPU):  38%|███▊      | 38/100 [05:20<08:34,  8.30s/it]
process (GPU):  39%|███▉      | 39/100 [05:23<06:46,  6.67s/it]
process (GPU):  40%|████      | 40/100 [05:27<05:52,  5.88s/it]
process (GPU):  41%|████      | 41/100 [05:40<08:05,  8.22s/it]
process (GPU):  42%|████▏     | 42/100 [05:58<10:29, 10.86s/it]
process (GPU):  44%|████▍     | 44/100 [06:24<11:06, 11.90s/it]
process (GPU):  45%|████▌     | 45/100 [06:28<09:14, 10.07s/it]
process (GPU):  46%|████▌     | 46/100 [06:52<12:19, 13.69s/it]
process (GPU):  47%|████▋     | 47/100 [07:18<15:03, 17.05s/it]
process (GPU):  49%|████▉     | 49/100 [07:27<09:48, 11.54s/it]
process (GPU):  50%|█████     | 50/100 [07:47<11:15, 13.51s/it]
process (GPU):  51%|█████     | 51/100 [08:00<10:49, 13.26s/it]
process (GPU):  52%|█████▏    | 52/100 [08:02<08:13, 10.28s/it]
process (GPU):  53%|█████▎    | 53/100 [08:37<13:31, 17.26s/it]
process (GPU):  54%|█████▍    | 54/100 [08:43<10:40, 13.93s/it]
process (GPU):  56%|█████▌    | 56/100 [09:14<10:45, 14.66s/it]
process (GPU):  57%|█████▋    | 57/100 [09:15<08:11, 11.43s/it]
process (GPU):  58%|█████▊    | 58/100 [09:23<07:20, 10.48s/it]
process (GPU):  59%|█████▉    | 59/100 [09:56<11:18, 16.55s/it]
process (GPU):  60%|██████    | 60/100 [09:58<08:22, 12.57s/it]
process (GPU):  61%|██████    | 61/100 [10:19<09:38, 14.84s/it]
process (GPU):  62%|██████▏   | 62/100 [10:39<10:27, 16.51s/it]
process (GPU):  63%|██████▎   | 63/100 [10:52<09:29, 15.41s/it]
process (GPU):  64%|██████▍   | 64/100 [10:54<06:46, 11.30s/it]
process (GPU):  65%|██████▌   | 65/100 [11:05<06:39, 11.41s/it]
process (GPU):  67%|██████▋   | 67/100 [11:14<04:29,  8.17s/it]
process (GPU):  68%|██████▊   | 68/100 [11:19<03:59,  7.48s/it]
process (GPU):  69%|██████▉   | 69/100 [11:24<03:27,  6.68s/it]
process (GPU):  70%|███████   | 70/100 [11:49<05:51, 11.73s/it]
process (GPU):  71%|███████   | 71/100 [11:57<05:09, 10.66s/it]
process (GPU):  72%|███████▏  | 72/100 [12:22<06:55, 14.85s/it]
process (GPU):  73%|███████▎  | 73/100 [12:24<04:58, 11.04s/it]
process (GPU):  74%|███████▍  | 74/100 [12:35<04:48, 11.11s/it]
process (GPU):  75%|███████▌  | 75/100 [12:41<03:54,  9.39s/it]
process (GPU):  77%|███████▋  | 77/100 [12:43<02:09,  5.65s/it]
process (GPU):  78%|███████▊  | 78/100 [12:45<01:46,  4.83s/it]
process (GPU):  79%|███████▉  | 79/100 [12:50<01:38,  4.69s/it]
process (GPU):  80%|████████  | 80/100 [12:52<01:19,  3.99s/it]
process (GPU):  81%|████████  | 81/100 [12:53<01:03,  3.37s/it]
process (GPU):  82%|████████▏ | 82/100 [12:56<00:56,  3.15s/it]
process (GPU):  83%|████████▎ | 83/100 [12:59<00:54,  3.22s/it]
process (GPU):  84%|████████▍ | 84/100 [13:01<00:45,  2.81s/it]
process (GPU):  85%|████████▌ | 85/100 [13:34<02:56, 11.77s/it]
process (GPU):  86%|████████▌ | 86/100 [14:04<03:59, 17.11s/it]
process (GPU):  88%|████████▊ | 88/100 [14:09<02:03, 10.25s/it]
process (GPU):  89%|████████▉ | 89/100 [14:13<01:37,  8.84s/it]


Summary: auto


**Choosing timeout extension**

The 900s timeout cut off near the end of processing 100 videos, so I'll increase it to 2000s to allow completion despite reprocessing from scratch. Alternatively, a custom script skipping disk writes might speed things up, but full pipeline results are required.


**Estimating ETA from partial run**

The partial run reaching 89% in 14 minutes suggests a total of about 16 minutes for 100 videos, providing a solid ETA estimate without full completion. Next, I'll check the output directory for partial stats or shards to see if any usable metrics can be salvaged.


ls /tmp/xcodec_validate/gpu_100/stats || true


Output:
Exit code: 0
Wall time: 0 seconds
Output:


**Adjusting timeout and output format**

I'm increasing the timeout to 2,000,000 ms and setting the output format to FLAC to reduce overhead before rerunning the process.


Output:
Exit code: 0
Wall time: 1270.5 seconds
Output:
/home/ubuntu/.local/lib/python3.10/site-packages/torch/cuda/__init__.py:63: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you.
  import pynvml  # type: ignore[import]

download:   0%|          | 0/100 [00:00<?, ?it/s]
download:   1%|          | 1/100 [00:00<01:28,  1.12it/s]
download:   2%|▏         | 2/100 [00:01<00:48,  2.03it/s]
download:   3%|▎         | 3/100 [00:01<00:31,  3.12it/s]
download:   4%|▍         | 4/100 [00:01<00:22,  4.22it/s]
download:   5%|▌         | 5/100 [00:01<00:21,  4.34it/s]
download:   7%|▋         | 7/100 [00:01<00:18,  4.95it/s]
download:   8%|▊         | 8/100 [00:02<00:19,  4.77it/s]
download:  10%|█         | 10/100 [00:02<00:13,  6.82it/s]
download:  11%|█         | 11/100 [00:02<00:19,  4.55it/s]
download:  12%|█▏        | 12/100 [00:03<00:20,  4.20it/s]
download:  13%|█▎        | 13/100 [00:03<00:18,  4.70it/s]
download:  14%|█▍        | 14/100 [00:03<00:27,  3.10it/s]
download:  16%|█▌        | 16/100 [00:04<00:19,  4.21it/s]
download:  18%|█▊        | 18/100 [00:04<00:14,  5.83it/s]
download:  20%|██        | 20/100 [00:04<00:10,  7.41it/s]
download:  22%|██▏       | 22/100 [00:04<00:08,  9.30it/s]
download:  25%|██▌       | 25/100 [00:04<00:06, 11.27it/s]
download:  27%|██▋       | 27/100 [00:04<00:05, 12.21it/s]
download:  31%|███       | 31/100 [00:05<00:05, 12.52it/s]
download:  35%|███▌      | 35/100 [00:05<00:03, 16.67it/s]
download:  38%|███▊      | 38/100 [00:05<00:04, 14.10it/s]
download:  40%|████      | 40/100 [00:05<00:04, 12.43it/s]
download:  42%|████▏     | 42/100 [00:05<00:05, 11.02it/s]
download:  44%|████▍     | 44/100 [00:06<00:04, 11.60it/s]
download:  46%|████▌     | 46/100 [00:06<00:04, 10.97it/s]
download:  48%|████▊     | 48/100 [00:06<00:06,  7.78it/s]
download:  49%|████▉     | 49/100 [00:06<00:06,  7.83it/s]
download:  52%|█████▏    | 52/100 [00:06<00:04, 11.09it/s]
download:  54%|█████▍    | 54/100 [00:07<00:04, 10.85it/s]
download:  56%|█████▌    | 56/100 [00:07<00:04,  8.88it/s]
download:  59%|█████▉    | 59/100 [00:07<00:03, 12.12it/s]
download:  61%|██████    | 61/100 [00:07<00:04,  9.53it/s]
download:  63%|██████▎   | 63/100 [00:08<00:03, 10.54it/s]
download:  65%|██████▌   | 65/100 [00:08<00:03,  9.55it/s]
download:  68%|██████▊   | 68/100 [00:08<00:02, 12.85it/s]
download:  70%|███████   | 70/100 [00:08<00:02, 11.01it/s]
download:  72%|███████▏  | 72/100 [00:08<00:02, 10.89it/s]
download:  74%|███████▍  | 74/100 [00:09<00:02,  9.42it/s]
download:  76%|███████▌  | 76/100 [00:09<00:02,  9.49it/s]
download:  78%|███████▊  | 78/100 [00:09<00:02, 10.19it/s]
download:  82%|████████▏ | 82/100 [00:09<00:01,  9.40it/s]
download:  84%|████████▍ | 84/100 [00:10<00:01,  9.12it/s]
download:  85%|████████▌ | 85/100 [00:10<00:02,  6.47it/s]
download:  87%|████████▋ | 87/100 [00:10<00:01,  6.80it/s]
download:  88%|████████▊ | 88/100 [00:11<00:02,  5.18it/s]
download:  91%|█████████ | 91/100 [00:11<00:01,  7.24it/s]
download:  92%|█████████▏| 92/100 [00:11<00:01,  5.47it/s]
download:  94%|█████████▍| 94/100 [00:12<00:00,  6.33it/s]
download:  95%|█████████▌| 95/100 [00:12<00:00,  6.50it/s]
download:  98%|█████████▊| 98/100 [00:12<00:00,  8.33it/s]
download:  99%|█████████▉| 99/100 [00:12<00:00,  7.18it/s]
download: 100%|██████████| 100/100 [00:12<00:00,  6.90it/s]
download: 100%|██████████| 100/100 [00:12<00:00,  7.78it/s]
Using GPU-accelerated batch processor (batch_size=32)
  GPU path uses EXACT same chunking/metrics as CPU - only resampling is GPU-accelerated
GPUResampler: Using NVIDIA A100-SXM4-80GB

process (GPU):   0%|          | 0/100 [00:00<?, ?it/s]
process (GPU):   3%|▎         | 3/100 [00:02<01:25,  1.13it/s]
process (GPU):   4%|▍         | 4/100 [00:07<03:17,  2.06s/it]
process (GPU):   5%|▌         | 5/100 [00:09<03:15,  2.06s/it]
process (GPU):   6%|▌         | 6/100 [00:13<04:23,  2.80s/it]
process (GPU):   7%|▋         | 7/100 [00:22<07:13,  4.66s/it]
process (GPU):   8%|▊         | 8/100 [00:29<08:15,  5.38s/it]
process (GPU):   9%|▉         | 9/100 [00:33<07:23,  4.88s/it]
process (GPU):  10%|█         | 10/100 [00:41<09:01,  6.02s/it]
process (GPU):  11%|█         | 11/100 [00:56<12:34,  8.48s/it]
process (GPU):  13%|█▎        | 13/100 [01:01<08:21,  5.76s/it]
process (GPU):  14%|█▍        | 14/100 [01:05<07:35,  5.29s/it]
process (GPU):  15%|█▌        | 15/100 [01:22<11:55,  8.42s/it]
process (GPU):  16%|█▌        | 16/100 [01:26<10:04,  7.20s/it]
process (GPU):  17%|█▋        | 17/100 [02:12<24:57, 18.04s/it]
process (GPU):  18%|█▊        | 18/100 [02:36<27:12, 19.91s/it]
process (GPU):  19%|█▉        | 19/100 [02:56<26:55, 19.95s/it]
process (GPU):  21%|██        | 21/100 [03:33<25:14, 19.17s/it]
process (GPU):  22%|██▏       | 22/100 [03:58<26:57, 20.74s/it]
process (GPU):  23%|██▎       | 23/100 [04:33<31:05, 24.22s/it]
process (GPU):  24%|██▍       | 24/100 [04:34<22:54, 18.09s/it]
process (GPU):  25%|██▌       | 25/100 [05:10<28:44, 23.00s/it]
process (GPU):  27%|██▋       | 27/100 [05:43<24:24, 20.05s/it]
process (GPU):  28%|██▊       | 28/100 [05:49<20:09, 16.79s/it]
process (GPU):  29%|██▉       | 29/100 [05:51<15:28, 13.08s/it]
process (GPU):  30%|███       | 30/100 [06:00<13:56, 11.95s/it]
process (GPU):  31%|███       | 31/100 [06:32<20:10, 17.55s/it]
process (GPU):  33%|███▎      | 33/100 [06:36<11:57, 10.71s/it]
process (GPU):  35%|███▌      | 35/100 [06:41<08:06,  7.48s/it]
process (GPU):  36%|███▌      | 36/100 [06:54<09:08,  8.57s/it]
process (GPU):  37%|███▋      | 37/100 [06:56<07:25,  7.06s/it]
process (GPU):  38%|███▊      | 38/100 [07:00<06:29,  6.27s/it]
process (GPU):  39%|███▉      | 39/100 [07:07<06:41,  6.57s/it]
process (GPU):  41%|████      | 41/100 [07:11<04:31,  4.61s/it]
process (GPU):  42%|████▏     | 42/100 [07:17<04:45,  4.93s/it]
process (GPU):  43%|████▎     | 43/100 [07:19<04:01,  4.24s/it]
process (GPU):  44%|████▍     | 44/100 [07:23<03:53,  4.17s/it]
process (GPU):  46%|████▌     | 46/100 [07:30<03:20,  3.71s/it]
process (GPU):  47%|████▋     | 47/100 [07:54<07:33,  8.56s/it]
process (GPU):  48%|████▊     | 48/100 [08:06<08:13,  9.48s/it]
process (GPU):  49%|████▉     | 49/100 [08:35<12:24, 14.59s/it]
process (GPU):  50%|█████     | 50/100 [09:01<14:50, 17.82s/it]
process (GPU):  51%|█████     | 51/100 [09:06<11:28, 14.05s/it]
process (GPU):  52%|█████▏    | 52/100 [09:19<11:07, 13.90s/it]
process (GPU):  53%|█████▎    | 53/100 [09:25<08:55, 11.40s/it]
process (GPU):  55%|█████▌    | 55/100 [09:46<08:18, 11.07s/it]
process (GPU):  56%|█████▌    | 56/100 [09:50<06:54,  9.43s/it]
process (GPU):  57%|█████▋    | 57/100 [09:55<05:55,  8.27s/it]
process (GPU):  58%|█████▊    | 58/100 [09:59<04:58,  7.11s/it]
process (GPU):  59%|█████▉    | 59/100 [10:05<04:35,  6.71s/it]
process (GPU):  60%|██████    | 60/100 [10:07<03:34,  5.37s/it]
process (GPU):  61%|██████    | 61/100 [11:05<13:22, 20.59s/it]
process (GPU):  62%|██████▏   | 62/100 [11:12<10:30, 16.59s/it]
process (GPU):  63%|██████▎   | 63/100 [11:15<07:44, 12.55s/it]
process (GPU):  64%|██████▍   | 64/100 [11:22<06:40, 11.13s/it]
process (GPU):  65%|██████▌   | 65/100 [11:27<05:19,  9.12s/it]
process (GPU):  66%|██████▌   | 66/100 [11:30<04:05,  7.21s/it]
process (GPU):  67%|██████▋   | 67/100 [11:30<02:56,  5.34s/it]
process (GPU):  68%|██████▊   | 68/100 [11:56<06:05, 11.43s/it]
process (GPU):  70%|███████   | 70/100 [12:00<03:29,  6.97s/it]
process (GPU):  71%|███████   | 71/100 [12:01<02:41,  5.58s/it]
process (GPU):  72%|███████▏  | 72/100 [12:03<02:09,  4.62s/it]
process (GPU):  73%|███████▎  | 73/100 [12:20<03:37,  8.06s/it]
process (GPU):  74%|███████▍  | 74/100 [12:25<03:04,  7.10s/it]
process (GPU):  76%|███████▌  | 76/100 [12:27<01:45,  4.38s/it]
process (GPU):  77%|███████▋  | 77/100 [12:37<02:11,  5.73s/it]
process (GPU):  78%|███████▊  | 78/100 [12:52<02:59,  8.14s/it]
process (GPU):  79%|███████▉  | 79/100 [13:29<05:34, 15.93s/it]
process (GPU):  80%|████████  | 80/100 [13:35<04:21, 13.10s/it]
process (GPU):  81%|████████  | 81/100 [13:50<04:20, 13.72s/it]
process (GPU):  82%|████████▏ | 82/100 [13:52<03:05, 10.30s/it]
process (GPU):  83%|████████▎ | 83/100 [13:55<02:19,  8.22s/it]
process (GPU):  84%|████████▍ | 84/100 [14:39<04:57, 18.60s/it]
process (GPU):  86%|████████▌ | 86/100 [15:14<04:14, 18.19s/it]
process (GPU):  87%|████████▋ | 87/100 [15:19<03:14, 14.99s/it]
process (GPU):  88%|████████▊ | 88/100 [15:32<02:53, 14.42s/it]
process (GPU):  89%|████████▉ | 89/100 [15:48<02:41, 14.70s/it]
process (GPU):  90%|█████████ | 90/100 [15:54<02:02, 12.30s/it]
process (GPU):  91%|█████████ | 91/100 [16:20<02:27, 16.44s/it]
process (GPU):  92%|█████████▏| 92/100 [16:56<02:56, 22.07s/it]
process (GPU):  93%|█████████▎| 93/100 [17:24<02:46, 23.74s/it]
process (GPU):  94%|█████████▍| 94/100 [17:56<02:37, 26.28s/it]
process (GPU):  95%|█████████▌| 95/100 [18:18<02:04, 25.00s/it]
process (GPU):  96%|█████████▌| 96/100 [18:49<01:46, 26.63s/it]
process (GPU):  97%|█████████▋| 97/100 [19:03<01:09, 23.02s/it]
process (GPU):  98%|█████████▊| 98/100 [19:28<00:47, 23.59s/it]
process (GPU):  99%|█████████▉| 99/100 [20:05<00:27, 27.58s/it]
process (GPU): 100%|██████████| 100/100 [20:44<00:00, 30.97s/it]
process (GPU): 100%|██████████| 100/100 [20:44<00:00, 12.45s/it]
GPU processor stats: {'decoded': 18406, 'chunks_emitted': 13468, 'segments_chunked': 1576, 'dropped_too_short': 14659, 'dropped_too_long': 43, 'dropped_clipped': 0}
Done. Stats written to /tmp/xcodec_validate/gpu_100/stats/hindi_stats.json

real	21m10.537s
user	23m25.514s
sys	0m37.440s


**Preparing to read stats file**


Output:
Exit code: 0
Wall time: 0 seconds
Output:
{
  "videos": 100,
  "segments_total": 18406,
  "segments_kept": {
    "16000": 13468,
    "24000": 10310,
    "32000": 8805,
    "44100": 2877
  },
  "dropped_too_short": 0,
  "dropped_too_long": 0,
  "dropped_clipped": 0,
  "segments_chunked": 0,
  "chunks_emitted": 0,
  "native_sr_buckets": {
    "32-44k": 3434,
    "24-32k": 7499,
    "16-24k": 1882,
    "<16k": 638,
    ">=44k": 15
  }
}


**Evaluating 500-sample feasibility**

The 500-video sample run may take about 1.8 hours based on scaling from 100 samples, which is potentially long but possibly manageable; I'll note the GPU stats issue and consider reporting initial 100 results before proceeding with 500.


**Preparing full 500-sample run**

I'm setting up the 500-video run with default config (GPU batch 32, 8 workers), Hindi language, a long timeout (2.2 hours), and specific output/temp paths, accepting the longer runtime to fully validate the pipeline as requested.


time python3 scripts/prepare_shards.py --language hindi --limit 500 --output-dir /tmp/xcodec_validate/gpu_500 --tmp-dir /tmp/xcodec_validate/gpu_tmp_500 --download-workers 16 --gpu --gpu-batch-size 32 --gpu-decode-workers 8 --gpu-encode-workers 8


Output:
Exit code: 0
Wall time: 6549.8 seconds
Output:
/home/ubuntu/.local/lib/python3.10/site-packages/torch/cuda/__init__.py:63: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you.
  import pynvml  # type: ignore[import]

download:   0%|          | 0/500 [00:00<?, ?it/s]
download:   0%|          | 1/500 [00:01<11:23,  1.37s/it]
download:   0%|          | 2/500 [00:01<06:51,  1.21it/s]
download:   1%|          | 3/500 [00:01<04:07,  2.01it/s]
download:   1%|          | 5/500 [00:02<02:11,  3.77it/s]
download:   1%|          | 6/500 [00:02<02:22,  3.47it/s]
download:   2%|▏         | 9/500 [00:02<01:27,  5.59it/s]
download:   2%|▏         | 10/500 [00:02<01:33,  5.26it/s]
download:   2%|▏         | 12/500 [00:03<01:11,  6.82it/s]
download:   3%|▎         | 13/500 [00:03<01:08,  7.12it/s]
download:   3%|▎         | 15/500 [00:03<01:13,  6.57it/s]
download:   3%|▎         | 16/500 [00:04<01:45,  4.58it/s]
download:   3%|▎         | 17/500 [00:04<01:36,  4.99it/s]
download:   4%|▎         | 18/500 [00:04<02:10,  3.68it/s]
download:   4%|▍         | 21/500 [00:04<01:22,  5.82it/s]
download:   5%|▍         | 23/500 [00:05<01:12,  6.55it/s]
download:   5%|▌         | 25/500 [00:05<01:04,  7.32it/s]
download:   5%|▌         | 26/500 [00:05<01:26,  5.45it/s]
download:   6%|▌         | 28/500 [00:05<01:20,  5.85it/s]
download:   6%|▌         | 29/500 [00:06<01:29,  5.26it/s]
download:   6%|▌         | 30/500 [00:06<02:01,  3.86it/s]
download:   6%|▌         | 31/500 [00:06<01:46,  4.42it/s]
download:   7%|▋         | 33/500 [00:07<01:22,  5.63it/s]
download:   7%|▋         | 34/500 [00:07<01:46,  4.39it/s]
download:   7%|▋         | 36/500 [00:07<01:20,  5.77it/s]
download:   7%|▋         | 37/500 [00:07<01:26,  5.36it/s]
download:   8%|▊         | 39/500 [00:08<01:11,  6.44it/s]
download:   8%|▊         | 40/500 [00:08<01:18,  5.86it/s]
download:   9%|▉         | 44/500 [00:08<00:42, 10.82it/s]
download:   9%|▉         | 46/500 [00:08<00:52,  8.72it/s]
download:  10%|▉         | 49/500 [00:08<00:40, 11.27it/s]
download:  10%|█         | 51/500 [00:09<00:55,  8.11it/s]
download:  11%|█         | 53/500 [00:09<00:52,  8.45it/s]
download:  11%|█         | 55/500 [00:09<00:50,  8.90it/s]
download:  11%|█▏        | 57/500 [00:10<00:58,  7.56it/s]
download:  12%|█▏        | 58/500 [00:10<01:02,  7.11it/s]
download:  12%|█▏        | 60/500 [00:10<00:57,  7.64it/s]
download:  12%|█▏        | 61/500 [00:10<01:04,  6.79it/s]
download:  12%|█▏        | 62/500 [00:11<01:17,  5.62it/s]
download:  13%|█▎        | 64/500 [00:11<01:34,  4.63it/s]
download:  13%|█▎        | 65/500 [00:11<01:46,  4.09it/s]
download:  13%|█▎        | 66/500 [00:12<01:34,  4.59it/s]
download:  14%|█▎        | 68/500 [00:12<01:45,  4.11it/s]
download:  14%|█▍        | 70/500 [00:12<01:28,  4.88it/s]
download:  14%|█▍        | 72/500 [00:13<01:14,  5.75it/s]
download:  15%|█▌        | 77/500 [00:13<00:48,  8.72it/s]
download:  16%|█▌        | 80/500 [00:13<00:38, 10.85it/s]
download:  16%|█▋        | 82/500 [00:14<00:55,  7.60it/s]
download:  17%|█▋        | 84/500 [00:14<00:49,  8.33it/s]
download:  17%|█▋        | 87/500 [00:14<00:39, 10.55it/s]
download:  18%|█▊        | 91/500 [00:14<00:28, 14.32it/s]
download:  19%|█▊        | 93/500 [00:14<00:31, 12.80it/s]
download:  19%|█▉        | 95/500 [00:15<00:48,  8.35it/s]
download:  19%|█▉        | 97/500 [00:15<00:47,  8.43it/s]
download:  20%|██        | 101/500 [00:15<00:41,  9.68it/s]
download:  21%|██        | 103/500 [00:16<00:40,  9.75it/s]
download:  21%|██        | 105/500 [00:16<01:00,  6.57it/s]
download:  21%|██▏       | 107/500 [00:16<01:00,  6.49it/s]
download:  22%|██▏       | 109/500 [00:17<01:01,  6.33it/s]
download:  22%|██▏       | 110/500 [00:17<00:58,  6.64it/s]
download:  23%|██▎       | 113/500 [00:17<00:43,  8.85it/s]
download:  23%|██▎       | 115/500 [00:18<00:51,  7.44it/s]
download:  23%|██▎       | 116/500 [00:18<00:53,  7.19it/s]
download:  23%|██▎       | 117/500 [00:18<01:07,  5.66it/s]
download:  24%|██▍       | 119/500 [00:18<01:11,  5.33it/s]
download:  24%|██▍       | 120/500 [00:19<01:23,  4.53it/s]
download:  24%|██▍       | 121/500 [00:19<01:30,  4.21it/s]
download:  25%|██▍       | 123/500 [00:20<01:30,  4.15it/s]
download:  25%|██▌       | 125/500 [00:20<01:14,  5.02it/s]
download:  25%|██▌       | 126/500 [00:20<01:37,  3.85it/s]
download:  25%|██▌       | 127/500 [00:20<01:27,  4.25it/s]
download:  26%|██▌       | 130/500 [00:21<00:52,  7.03it/s]
download:  26%|██▋       | 132/500 [00:21<00:53,  6.94it/s]
download:  27%|██▋       | 133/500 [00:21<00:54,  6.76it/s]
download:  27%|██▋       | 134/500 [00:21<01:00,  6.08it/s]
download:  27%|██▋       | 135/500 [00:21<01:00,  6.08it/s]
download:  28%|██▊       | 138/500 [00:22<00:53,  6.71it/s]
download:  28%|██▊       | 139/500 [00:22<01:02,  5.82it/s]
download:  28%|██▊       | 140/500 [00:22<00:57,  6.25it/s]
download:  28%|██▊       | 142/500 [00:22<00:46,  7.77it/s]
download:  29%|██▉       | 144/500 [00:23<00:45,  7.88it/s]
download:  29%|██▉       | 146/500 [00:23<00:38,  9.11it/s]
download:  29%|██▉       | 147/500 [00:23<00:59,  5.90it/s]
download:  30%|██▉       | 148/500 [00:24<01:11,  4.94it/s]
download:  30%|██▉       | 149/500 [00:24<01:05,  5.39it/s]
download:  30%|███       | 151/500 [00:24<00:46,  7.46it/s]
download:  30%|███       | 152/500 [00:24<00:45,  7.64it/s]
download:  31%|███       | 153/500 [00:24<00:44,  7.82it/s]
download:  31%|███       | 155/500 [00:24<00:57,  6.04it/s]
download:  32%|███▏      | 158/500 [00:25<00:42,  8.03it/s]
download:  32%|███▏      | 159/500 [00:25<00:43,  7.75it/s]
download:  32%|███▏      | 162/500 [00:25<00:44,  7.57it/s]
download:  33%|███▎      | 165/500 [00:25<00:37,  8.94it/s]
download:  33%|███▎      | 167/500 [00:26<00:51,  6.48it/s]
download:  34%|███▍      | 169/500 [00:26<00:42,  7.88it/s]
download:  34%|███▍      | 171/500 [00:26<00:43,  7.54it/s]
download:  35%|███▍      | 173/500 [00:27<00:46,  7.02it/s]
download:  35%|███▍      | 174/500 [00:27<00:45,  7.10it/s]
download:  35%|███▌      | 175/500 [00:27<00:45,  7.12it/s]
download:  36%|███▌      | 179/500 [00:27<00:32,  9.93it/s]
download:  36%|███▌      | 181/500 [00:28<00:44,  7.19it/s]
download:  37%|███▋      | 183/500 [00:28<00:42,  7.39it/s]
download:  37%|███▋      | 184/500 [00:28<00:46,  6.72it/s]
download:  37%|███▋      | 185/500 [00:28<00:47,  6.60it/s]
download:  37%|███▋      | 186/500 [00:29<01:02,  5.06it/s]
download:  38%|███▊      | 190/500 [00:29<00:32,  9.48it/s]
download:  38%|███▊      | 192/500 [00:29<00:38,  8.08it/s]
download:  39%|███▉      | 194/500 [00:29<00:38,  7.92it/s]
download:  40%|███▉      | 198/500 [00:30<00:26, 11.32it/s]
download:  40%|████      | 200/500 [00:30<00:25, 11.84it/s]
download:  40%|████      | 202/500 [00:30<00:32,  9.25it/s]
download:  41%|████      | 204/500 [00:30<00:29,  9.92it/s]
download:  41%|████      | 206/500 [00:31<00:49,  5.96it/s]
download:  41%|████▏     | 207/500 [00:31<00:46,  6.36it/s]
download:  42%|████▏     | 208/500 [00:31<00:57,  5.04it/s]
download:  42%|████▏     | 209/500 [00:32<00:53,  5.40it/s]
download:  42%|████▏     | 211/500 [00:32<00:40,  7.21it/s]
download:  43%|████▎     | 214/500 [00:32<00:35,  8.13it/s]
download:  43%|████▎     | 215/500 [00:32<00:49,  5.77it/s]
download:  44%|████▎     | 218/500 [00:33<00:34,  8.09it/s]
download:  44%|████▍     | 220/500 [00:33<00:35,  7.91it/s]
download:  45%|████▍     | 223/500 [00:33<00:25, 10.95it/s]
download:  45%|████▌     | 225/500 [00:33<00:24, 11.02it/s]
download:  45%|████▌     | 227/500 [00:33<00:22, 12.01it/s]
download:  46%|████▌     | 230/500 [00:34<00:25, 10.61it/s]
download:  46%|████▋     | 232/500 [00:34<00:22, 11.91it/s]
download:  47%|████▋     | 234/500 [00:34<00:25, 10.37it/s]
download:  47%|████▋     | 236/500 [00:34<00:24, 10.58it/s]
download:  48%|████▊     | 238/500 [00:34<00:24, 10.81it/s]
download:  48%|████▊     | 241/500 [00:35<00:26,  9.87it/s]
download:  49%|████▊     | 243/500 [00:35<00:31,  8.20it/s]
download:  49%|████▉     | 245/500 [00:35<00:36,  6.93it/s]
download:  49%|████▉     | 246/500 [00:36<00:41,  6.16it/s]
download:  49%|████▉     | 247/500 [00:36<00:40,  6.23it/s]
download:  50%|████▉     | 248/500 [00:36<00:43,  5.84it/s]
download:  50%|████▉     | 249/500 [00:36<00:55,  4.52it/s]
download:  50%|█████     | 250/500 [00:37<00:50,  4.91it/s]
download:  50%|█████     | 252/500 [00:37<00:39,  6.26it/s]
download:  51%|█████     | 255/500 [00:37<00:24,  9.95it/s]
download:  52%|█████▏    | 260/500 [00:37<00:15, 15.81it/s]
download:  52%|█████▏    | 262/500 [00:38<00:25,  9.26it/s]
download:  53%|█████▎    | 264/500 [00:38<00:25,  9.27it/s]
download:  53%|█████▎    | 266/500 [00:38<00:23,  9.82it/s]
download:  54%|█████▍    | 269/500 [00:38<00:27,  8.55it/s]
download:  54%|█████▍    | 271/500 [00:39<00:31,  7.21it/s]
download:  55%|█████▍    | 273/500 [00:39<00:26,  8.44it/s]
download:  55%|█████▌    | 275/500 [00:39<00:27,  8.19it/s]
download:  55%|█████▌    | 277/500 [00:39<00:27,  8.21it/s]
download:  56%|█████▌    | 278/500 [00:40<00:28,  7.74it/s]
download:  56%|█████▌    | 281/500 [00:40<00:24,  8.77it/s]
download:  56%|█████▋    | 282/500 [00:40<00:33,  6.54it/s]
download:  57%|█████▋    | 283/500 [00:40<00:33,  6.50it/s]
download:  57%|█████▋    | 286/500 [00:41<00:21, 10.02it/s]
download:  58%|█████▊    | 289/500 [00:41<00:18, 11.37it/s]
download:  58%|█████▊    | 291/500 [00:41<00:18, 11.32it/s]
download:  59%|█████▉    | 294/500 [00:41<00:14, 13.90it/s]
download:  59%|█████▉    | 296/500 [00:41<00:16, 12.35it/s]
download:  60%|██████    | 300/500 [00:42<00:18, 10.78it/s]
download:  60%|██████    | 302/500 [00:42<00:18, 10.51it/s]
download:  61%|██████    | 306/500 [00:42<00:15, 12.18it/s]
download:  62%|██████▏   | 308/500 [00:42<00:19,  9.67it/s]
download:  62%|██████▏   | 310/500 [00:43<00:18, 10.05it/s]
download:  62%|██████▏   | 312/500 [00:43<00:26,  7.15it/s]
download:  63%|██████▎   | 313/500 [00:44<00:36,  5.19it/s]
download:  63%|██████▎   | 316/500 [00:44<00:29,  6.26it/s]
download:  63%|██████▎   | 317/500 [00:44<00:29,  6.29it/s]
download:  64%|██████▍   | 319/500 [00:44<00:28,  6.38it/s]
download:  64%|██████▍   | 320/500 [00:45<00:30,  5.89it/s]
download:  64%|██████▍   | 322/500 [00:45<00:26,  6.68it/s]
download:  65%|██████▍   | 323/500 [00:45<00:28,  6.23it/s]
download:  65%|██████▌   | 325/500 [00:45<00:22,  7.67it/s]
download:  65%|██████▌   | 326/500 [00:45<00:23,  7.51it/s]
download:  66%|██████▌   | 329/500 [00:46<00:16, 10.34it/s]
download:  66%|██████▌   | 331/500 [00:46<00:16, 10.44it/s]
download:  67%|██████▋   | 333/500 [00:46<00:13, 12.16it/s]
download:  67%|██████▋   | 336/500 [00:46<00:11, 14.11it/s]
download:  68%|██████▊   | 338/500 [00:46<00:11, 13.84it/s]
download:  68%|██████▊   | 340/500 [00:47<00:20,  7.72it/s]
download:  68%|██████▊   | 342/500 [00:47<00:28,  5.47it/s]
download:  69%|██████▉   | 345/500 [00:48<00:24,  6.30it/s]
download:  70%|██████▉   | 348/500 [00:48<00:21,  7.05it/s]
download:  70%|██████▉   | 349/500 [00:48<00:24,  6.20it/s]
download:  70%|███████   | 351/500 [00:49<00:21,  6.93it/s]
download:  71%|███████   | 353/500 [00:49<00:21,  6.91it/s]
download:  71%|███████   | 355/500 [00:49<00:22,  6.41it/s]
download:  71%|███████▏  | 357/500 [00:49<00:19,  7.26it/s]
download:  72%|███████▏  | 361/500 [00:50<00:12, 11.00it/s]
download:  73%|███████▎  | 366/500 [00:50<00:08, 16.03it/s]
download:  74%|███████▍  | 369/500 [00:50<00:09, 14.30it/s]
download:  74%|███████▍  | 371/500 [00:50<00:08, 15.20it/s]
download:  75%|███████▍  | 373/500 [00:50<00:08, 14.53it/s]
download:  75%|███████▌  | 375/500 [00:51<00:12, 10.32it/s]
download:  75%|███████▌  | 377/500 [00:51<00:15,  8.01it/s]
download:  76%|███████▌  | 379/500 [00:51<00:14,  8.49it/s]
download:  76%|███████▌  | 381/500 [00:51<00:12,  9.70it/s]
download:  77%|███████▋  | 383/500 [00:52<00:15,  7.62it/s]
download:  77%|███████▋  | 385/500 [00:52<00:17,  6.42it/s]
download:  78%|███████▊  | 388/500 [00:52<00:12,  8.64it/s]
download:  78%|███████▊  | 390/500 [00:53<00:14,  7.43it/s]
download:  78%|███████▊  | 391/500 [00:53<00:19,  5.67it/s]
download:  78%|███████▊  | 392/500 [00:53<00:18,  5.96it/s]
download:  79%|███████▉  | 395/500 [00:53<00:11,  8.84it/s]
download:  79%|███████▉  | 397/500 [00:54<00:11,  8.86it/s]
download:  80%|███████▉  | 399/500 [00:54<00:10,  9.62it/s]
download:  80%|████████  | 402/500 [00:54<00:08, 11.05it/s]
download:  81%|████████  | 404/500 [00:54<00:09, 10.04it/s]
download:  81%|████████  | 406/500 [00:54<00:09, 10.43it/s]
download:  82%|████████▏ | 408/500 [00:55<00:10,  8.97it/s]
download:  82%|████████▏ | 409/500 [00:55<00:12,  7.28it/s]
download:  82%|████████▏ | 412/500 [00:55<00:08, 10.12it/s]
download:  83%|████████▎ | 414/500 [00:56<00:12,  7.14it/s]
download:  83%|████████▎ | 415/500 [00:56<00:11,  7.15it/s]
download:  83%|████████▎ | 417/500 [00:56<00:11,  7.46it/s]
download:  84%|████████▍ | 419/500 [00:56<00:09,  8.96it/s]
download:  84%|████████▍ | 421/500 [00:56<00:09,  8.50it/s]
download:  84%|████████▍ | 422/500 [00:57<00:11,  6.92it/s]
download:  85%|████████▍ | 423/500 [00:57<00:16,  4.60it/s]
download:  85%|████████▌ | 425/500 [00:57<00:13,  5.70it/s]
download:  85%|████████▌ | 426/500 [00:57<00:13,  5.59it/s]
download:  85%|████████▌ | 427/500 [00:58<00:12,  5.95it/s]
download:  86%|████████▌ | 429/500 [00:58<00:11,  6.38it/s]
download:  86%|████████▌ | 430/500 [00:58<00:13,  5.01it/s]
download:  86%|████████▌ | 431/500 [00:58<00:14,  4.67it/s]
download:  86%|████████▋ | 432/500 [00:59<00:13,  4.94it/s]
download:  87%|████████▋ | 435/500 [00:59<00:08,  8.07it/s]
download:  87%|████████▋ | 436/500 [00:59<00:10,  6.40it/s]
download:  88%|████████▊ | 438/500 [00:59<00:09,  6.33it/s]
download:  88%|████████▊ | 439/500 [01:00<00:11,  5.39it/s]
download:  88%|████████▊ | 441/500 [01:00<00:08,  6.75it/s]
download:  89%|████████▉ | 444/500 [01:00<00:07,  7.90it/s]
download:  89%|████████▉ | 445/500 [01:00<00:08,  6.49it/s]
download:  89%|████████▉ | 446/500 [01:01<00:10,  5.25it/s]
download:  90%|████████▉ | 448/500 [01:01<00:10,  4.95it/s]
download:  90%|████████▉ | 449/500 [01:01<00:09,  5.28it/s]
download:  90%|█████████ | 452/500 [01:02<00:06,  7.99it/s]
download:  91%|█████████ | 453/500 [01:02<00:07,  6.07it/s]
download:  91%|█████████ | 456/500 [01:02<00:04,  9.26it/s]
download:  92%|█████████▏| 458/500 [01:02<00:04,  9.94it/s]
download:  92%|█████████▏| 460/500 [01:02<00:04,  9.42it/s]
download:  92%|█████████▏| 462/500 [01:03<00:04,  8.78it/s]
download:  93%|█████████▎| 466/500 [01:03<00:02, 13.60it/s]
download:  94%|█████████▎| 468/500 [01:03<00:03,  9.69it/s]
download:  94%|█████████▍| 472/500 [01:03<00:02, 13.98it/s]
download:  95%|█████████▌| 476/500 [01:03<00:01, 17.16it/s]
download:  96%|█████████▌| 479/500 [01:04<00:01, 12.80it/s]
download:  96%|█████████▌| 481/500 [01:04<00:02,  8.78it/s]
download:  97%|█████████▋| 486/500 [01:05<00:01, 11.50it/s]
download:  98%|█████████▊| 488/500 [01:05<00:01,  7.97it/s]
download:  98%|█████████▊| 490/500 [01:05<00:01,  8.60it/s]
download:  99%|█████████▊| 493/500 [01:06<00:00,  9.05it/s]
download:  99%|█████████▉| 495/500 [01:06<00:00,  9.81it/s]
download: 100%|██████████| 500/500 [01:06<00:00,  7.55it/s]
Using GPU-accelerated batch processor (batch_size=32)
  GPU path uses EXACT same chunking/metrics as CPU - only resampling is GPU-accelerated
GPUResampler: Using NVIDIA A100-SXM4-80GB

process (GPU):   0%|          | 0/500 [00:00<?, ?it/s]
process (GPU):   0%|          | 2/500 [00:02<10:34,  1.27s/it]
process (GPU):   1%|          | 3/500 [00:09<30:39,  3.70s/it]
process (GPU):   1%|          | 4/500 [00:15<37:50,  4.58s/it]
process (GPU):   1%|          | 5/500 [00:21<40:13,  4.88s/it]
process (GPU):   1%|          | 6/500 [00:27<43:49,  5.32s/it]
process (GPU):   1%|▏         | 7/500 [00:33<44:53,  5.46s/it]
process (GPU):   2%|▏         | 8/500 [00:38<44:30,  5.43s/it]
process (GPU):   2%|▏         | 9/500 [00:46<49:47,  6.09s/it]
process (GPU):   2%|▏         | 11/500 [00:51<36:59,  4.54s/it]
process (GPU):   3%|▎         | 14/500 [00:57<26:02,  3.22s/it]
process (GPU):   3%|▎         | 15/500 [01:08<39:06,  4.84s/it]
process (GPU):   3%|▎         | 16/500 [01:26<1:00:19,  7.48s/it]
process (GPU):   3%|▎         | 17/500 [01:43<1:19:50,  9.92s/it]
process (GPU):   4%|▎         | 18/500 [02:01<1:35:32, 11.89s/it]
process (GPU):   4%|▍         | 20/500 [02:27<1:38:40, 12.33s/it]
process (GPU):   4%|▍         | 21/500 [02:38<1:36:15, 12.06s/it]
process (GPU):   4%|▍         | 22/500 [03:20<2:34:58, 19.45s/it]
process (GPU):   5%|▍         | 23/500 [03:33<2:20:57, 17.73s/it]
process (GPU):   5%|▍         | 24/500 [03:36<1:49:04, 13.75s/it]
process (GPU):   5%|▌         | 25/500 [03:47<1:41:51, 12.87s/it]
process (GPU):   5%|▌         | 26/500 [03:56<1:33:38, 11.85s/it]
process (GPU):   5%|▌         | 27/500 [04:00<1:15:07,  9.53s/it]
process (GPU):   6%|▌         | 28/500 [04:23<1:46:58, 13.60s/it]
process (GPU):   6%|▌         | 29/500 [05:06<2:55:09, 22.31s/it]
process (GPU):   6%|▌         | 30/500 [05:15<2:22:36, 18.21s/it]
process (GPU):   6%|▌         | 31/500 [05:31<2:18:43, 17.75s/it]
process (GPU):   6%|▋         | 32/500 [06:13<3:14:46, 24.97s/it]
…2867 tokens truncated…▎     | 214/500 [48:11<59:03, 12.39s/it]
process (GPU):  43%|████▎     | 215/500 [50:43<3:37:36, 45.81s/it]
process (GPU):  43%|████▎     | 216/500 [50:50<2:49:48, 35.87s/it]
process (GPU):  43%|████▎     | 217/500 [50:52<2:06:30, 26.82s/it]
process (GPU):  44%|████▎     | 218/500 [51:13<1:58:21, 25.18s/it]
process (GPU):  44%|████▍     | 219/500 [51:16<1:28:22, 18.87s/it]
process (GPU):  44%|████▍     | 220/500 [51:19<1:07:31, 14.47s/it]
process (GPU):  44%|████▍     | 222/500 [51:22<39:11,  8.46s/it]
process (GPU):  45%|████▍     | 223/500 [51:27<35:41,  7.73s/it]
process (GPU):  45%|████▍     | 224/500 [51:29<28:04,  6.10s/it]
process (GPU):  45%|████▌     | 225/500 [51:53<50:08, 10.94s/it]
process (GPU):  45%|████▌     | 226/500 [52:14<1:03:30, 13.91s/it]
process (GPU):  45%|████▌     | 227/500 [52:40<1:19:08, 17.39s/it]
process (GPU):  46%|████▌     | 229/500 [52:50<52:54, 11.72s/it]
process (GPU):  46%|████▋     | 232/500 [52:56<31:06,  6.97s/it]
process (GPU):  47%|████▋     | 233/500 [53:19<44:11,  9.93s/it]
process (GPU):  47%|████▋     | 234/500 [53:44<58:22, 13.17s/it]
process (GPU):  47%|████▋     | 236/500 [54:25<1:10:04, 15.93s/it]
process (GPU):  47%|████▋     | 237/500 [54:49<1:17:12, 17.62s/it]
process (GPU):  48%|████▊     | 238/500 [54:50<1:00:41, 13.90s/it]
process (GPU):  48%|████▊     | 239/500 [54:52<47:20, 10.88s/it]
process (GPU):  48%|████▊     | 240/500 [54:54<36:57,  8.53s/it]
process (GPU):  48%|████▊     | 241/500 [55:04<38:55,  9.02s/it]
process (GPU):  48%|████▊     | 242/500 [55:07<31:30,  7.33s/it]
process (GPU):  49%|████▊     | 243/500 [55:25<43:47, 10.22s/it]
process (GPU):  49%|████▉     | 244/500 [55:27<33:25,  7.84s/it]
process (GPU):  49%|████▉     | 245/500 [55:34<32:55,  7.75s/it]
process (GPU):  49%|████▉     | 246/500 [55:37<26:09,  6.18s/it]
process (GPU):  49%|████▉     | 247/500 [55:49<34:16,  8.13s/it]
process (GPU):  50%|████▉     | 248/500 [55:54<29:13,  6.96s/it]
process (GPU):  50%|████▉     | 249/500 [56:20<52:58, 12.66s/it]
process (GPU):  50%|█████     | 250/500 [56:25<43:20, 10.40s/it]
process (GPU):  50%|█████     | 251/500 [56:27<32:57,  7.94s/it]
process (GPU):  51%|█████     | 253/500 [56:56<45:09, 10.97s/it]
process (GPU):  51%|█████     | 254/500 [57:12<50:25, 12.30s/it]
process (GPU):  51%|█████     | 255/500 [57:30<56:25, 13.82s/it]
process (GPU):  51%|█████     | 256/500 [58:00<1:13:45, 18.14s/it]
process (GPU):  51%|█████▏    | 257/500 [58:09<1:02:50, 15.51s/it]
process (GPU):  52%|█████▏    | 258/500 [58:40<1:20:33, 19.97s/it]
process (GPU):  52%|█████▏    | 259/500 [58:53<1:12:24, 18.03s/it]
process (GPU):  52%|█████▏    | 260/500 [59:23<1:25:51, 21.46s/it]
process (GPU):  52%|█████▏    | 262/500 [59:26<49:10, 12.40s/it]
process (GPU):  53%|█████▎    | 264/500 [59:53<50:13, 12.77s/it]
process (GPU):  53%|█████▎    | 265/500 [59:55<40:32, 10.35s/it]
process (GPU):  53%|█████▎    | 266/500 [1:00:18<52:36, 13.49s/it]
process (GPU):  53%|█████▎    | 267/500 [1:00:26<47:09, 12.15s/it]
process (GPU):  54%|█████▎    | 268/500 [1:00:52<1:00:25, 15.63s/it]
process (GPU):  54%|█████▍    | 269/500 [1:00:54<45:42, 11.87s/it]
process (GPU):  54%|█████▍    | 270/500 [1:00:58<37:22,  9.75s/it]
process (GPU):  54%|█████▍    | 271/500 [1:01:09<38:27, 10.08s/it]
process (GPU):  55%|█████▍    | 273/500 [1:02:08<1:11:26, 18.88s/it]
process (GPU):  55%|█████▍    | 274/500 [1:02:13<58:22, 15.50s/it]
process (GPU):  55%|█████▌    | 275/500 [1:02:38<1:07:11, 17.92s/it]
process (GPU):  55%|█████▌    | 276/500 [1:02:47<58:19, 15.62s/it]
process (GPU):  55%|█████▌    | 277/500 [1:02:50<44:27, 11.96s/it]
process (GPU):  56%|█████▌    | 278/500 [1:02:51<32:42,  8.84s/it]
process (GPU):  56%|█████▌    | 279/500 [1:03:06<39:13, 10.65s/it]
process (GPU):  56%|█████▌    | 281/500 [1:03:26<38:14, 10.48s/it]
process (GPU):  56%|█████▋    | 282/500 [1:03:28<30:09,  8.30s/it]
process (GPU):  57%|█████▋    | 283/500 [1:03:41<34:12,  9.46s/it]
process (GPU):  57%|█████▋    | 284/500 [1:04:05<48:30, 13.47s/it]
process (GPU):  57%|█████▋    | 285/500 [1:04:53<1:22:30, 23.02s/it]
process (GPU):  57%|█████▋    | 286/500 [1:04:56<1:02:15, 17.46s/it]
process (GPU):  57%|█████▋    | 287/500 [1:04:59<47:18, 13.32s/it]
process (GPU):  58%|█████▊    | 289/500 [1:05:35<53:55, 15.34s/it]
process (GPU):  58%|█████▊    | 291/500 [1:05:47<40:49, 11.72s/it]
process (GPU):  58%|█████▊    | 292/500 [1:06:08<47:30, 13.70s/it]
process (GPU):  59%|█████▊    | 293/500 [1:06:11<38:41, 11.22s/it]
process (GPU):  59%|█████▉    | 294/500 [1:06:14<31:25,  9.15s/it]
process (GPU):  59%|█████▉    | 296/500 [1:06:20<22:10,  6.52s/it]
process (GPU):  59%|█████▉    | 297/500 [1:06:35<28:28,  8.41s/it]
process (GPU):  60%|█████▉    | 298/500 [1:06:53<36:48, 10.93s/it]
process (GPU):  60%|█████▉    | 299/500 [1:06:57<29:44,  8.88s/it]
process (GPU):  60%|██████    | 301/500 [1:07:09<25:25,  7.66s/it]
process (GPU):  60%|██████    | 302/500 [1:07:12<22:21,  6.78s/it]
process (GPU):  61%|██████    | 303/500 [1:07:30<31:26,  9.57s/it]
process (GPU):  61%|██████    | 304/500 [1:07:58<46:31, 14.24s/it]
process (GPU):  61%|██████    | 305/500 [1:08:03<38:19, 11.79s/it]
process (GPU):  61%|██████    | 306/500 [1:08:05<29:19,  9.07s/it]
process (GPU):  61%|██████▏   | 307/500 [1:08:23<37:12, 11.57s/it]
process (GPU):  62%|██████▏   | 308/500 [1:08:25<28:27,  8.89s/it]
process (GPU):  62%|██████▏   | 309/500 [1:08:27<21:31,  6.76s/it]
process (GPU):  62%|██████▏   | 310/500 [1:08:29<17:18,  5.47s/it]
process (GPU):  62%|██████▏   | 311/500 [1:08:37<19:33,  6.21s/it]
process (GPU):  62%|██████▏   | 312/500 [1:08:46<21:45,  6.94s/it]
process (GPU):  63%|██████▎   | 313/500 [1:08:51<20:10,  6.48s/it]
process (GPU):  63%|██████▎   | 314/500 [1:08:59<20:51,  6.73s/it]
process (GPU):  63%|██████▎   | 315/500 [1:09:18<32:53, 10.67s/it]
process (GPU):  63%|██████▎   | 316/500 [1:09:20<24:12,  7.90s/it]
process (GPU):  63%|██████▎   | 317/500 [1:09:26<22:00,  7.22s/it]
process (GPU):  64%|██████▎   | 318/500 [1:09:54<40:49, 13.46s/it]
process (GPU):  64%|██████▍   | 319/500 [1:09:56<30:45, 10.20s/it]
process (GPU):  64%|██████▍   | 320/500 [1:10:01<25:47,  8.60s/it]
process (GPU):  64%|██████▍   | 321/500 [1:10:27<41:34, 13.94s/it]
process (GPU):  64%|██████▍   | 322/500 [1:10:35<35:42, 12.03s/it]
process (GPU):  65%|██████▍   | 323/500 [1:10:42<30:50, 10.45s/it]
process (GPU):  65%|██████▌   | 325/500 [1:11:03<30:46, 10.55s/it]
process (GPU):  65%|██████▌   | 326/500 [1:11:45<52:51, 18.23s/it]
process (GPU):  66%|██████▌   | 328/500 [1:11:53<35:18, 12.32s/it]
process (GPU):  66%|██████▌   | 329/500 [1:12:04<34:32, 12.12s/it]
process (GPU):  66%|██████▌   | 330/500 [1:12:06<26:59,  9.53s/it]
process (GPU):  66%|██████▌   | 331/500 [1:12:17<27:56,  9.92s/it]
process (GPU):  66%|██████▋   | 332/500 [1:12:32<32:02, 11.44s/it]
process (GPU):  67%|██████▋   | 334/500 [1:12:36<20:03,  7.25s/it]
process (GPU):  67%|██████▋   | 335/500 [1:12:40<17:41,  6.44s/it]
process (GPU):  67%|██████▋   | 336/500 [1:13:13<36:33, 13.37s/it]
process (GPU):  67%|██████▋   | 337/500 [1:13:55<56:43, 20.88s/it]
process (GPU):  68%|██████▊   | 338/500 [1:14:29<1:05:59, 24.44s/it]
process (GPU):  68%|██████▊   | 339/500 [1:14:32<49:34, 18.47s/it]
process (GPU):  68%|██████▊   | 340/500 [1:15:04<59:46, 22.42s/it]
process (GPU):  68%|██████▊   | 341/500 [1:15:55<1:21:15, 30.66s/it]
process (GPU):  68%|██████▊   | 342/500 [1:16:00<1:01:04, 23.19s/it]
process (GPU):  69%|██████▊   | 343/500 [1:16:02<44:16, 16.92s/it]
process (GPU):  69%|██████▉   | 344/500 [1:16:11<37:23, 14.38s/it]
process (GPU):  69%|██████▉   | 346/500 [1:16:14<21:57,  8.56s/it]
process (GPU):  69%|██████▉   | 347/500 [1:16:30<26:26, 10.37s/it]
process (GPU):  70%|██████▉   | 348/500 [1:16:36<23:09,  9.14s/it]
process (GPU):  70%|██████▉   | 349/500 [1:16:59<32:28, 12.90s/it]
process (GPU):  70%|███████   | 350/500 [1:17:11<31:51, 12.75s/it]
process (GPU):  70%|███████   | 352/500 [1:17:14<18:53,  7.66s/it]
process (GPU):  71%|███████   | 354/500 [1:17:32<20:00,  8.22s/it]
process (GPU):  71%|███████   | 355/500 [1:17:56<28:01, 11.59s/it]
process (GPU):  71%|███████   | 356/500 [1:18:03<25:19, 10.55s/it]
process (GPU):  71%|███████▏  | 357/500 [1:18:05<20:16,  8.50s/it]
process (GPU):  72%|███████▏  | 358/500 [1:18:16<21:41,  9.16s/it]
process (GPU):  72%|███████▏  | 360/500 [1:18:43<25:29, 10.92s/it]
process (GPU):  72%|███████▏  | 361/500 [1:18:52<24:31, 10.59s/it]
process (GPU):  72%|███████▏  | 362/500 [1:18:56<20:12,  8.78s/it]
process (GPU):  73%|███████▎  | 363/500 [1:19:20<29:41, 13.01s/it]
process (GPU):  73%|███████▎  | 364/500 [1:19:58<44:44, 19.74s/it]
process (GPU):  73%|███████▎  | 365/500 [1:20:04<35:59, 16.00s/it]
process (GPU):  73%|███████▎  | 367/500 [1:20:07<20:49,  9.40s/it]
process (GPU):  74%|███████▍  | 369/500 [1:20:47<29:29, 13.51s/it]
process (GPU):  74%|███████▍  | 370/500 [1:20:52<25:36, 11.82s/it]
process (GPU):  74%|███████▍  | 371/500 [1:21:23<35:09, 16.35s/it]
process (GPU):  74%|███████▍  | 372/500 [1:21:29<29:01, 13.60s/it]
process (GPU):  75%|███████▍  | 374/500 [1:21:31<17:24,  8.29s/it]
process (GPU):  75%|███████▌  | 375/500 [1:22:06<30:13, 14.51s/it]
process (GPU):  75%|███████▌  | 376/500 [1:22:12<25:47, 12.48s/it]
process (GPU):  75%|███████▌  | 377/500 [1:22:20<23:16, 11.36s/it]
process (GPU):  76%|███████▌  | 378/500 [1:22:29<21:26, 10.55s/it]
process (GPU):  76%|███████▌  | 380/500 [1:22:36<14:52,  7.44s/it]
process (GPU):  76%|███████▌  | 381/500 [1:22:46<16:18,  8.22s/it]
process (GPU):  77%|███████▋  | 383/500 [1:22:55<13:04,  6.71s/it]
process (GPU):  77%|███████▋  | 385/500 [1:23:04<11:13,  5.86s/it]
process (GPU):  77%|███████▋  | 386/500 [1:23:10<11:02,  5.81s/it]
process (GPU):  77%|███████▋  | 387/500 [1:23:12<09:25,  5.01s/it]
process (GPU):  78%|███████▊  | 388/500 [1:23:16<09:00,  4.82s/it]
process (GPU):  78%|███████▊  | 389/500 [1:23:20<08:15,  4.46s/it]
process (GPU):  78%|███████▊  | 390/500 [1:23:31<11:27,  6.25s/it]
process (GPU):  78%|███████▊  | 392/500 [1:23:37<08:51,  4.92s/it]
process (GPU):  79%|███████▊  | 393/500 [1:23:43<09:05,  5.10s/it]
process (GPU):  79%|███████▉  | 394/500 [1:23:51<10:13,  5.79s/it]
process (GPU):  79%|███████▉  | 395/500 [1:24:02<12:25,  7.10s/it]
process (GPU):  79%|███████▉  | 396/500 [1:24:08<11:56,  6.89s/it]
process (GPU):  79%|███████▉  | 397/500 [1:24:13<10:55,  6.37s/it]
process (GPU):  80%|███████▉  | 398/500 [1:24:25<13:43,  8.07s/it]
process (GPU):  80%|████████  | 400/500 [1:25:15<26:12, 15.73s/it]
process (GPU):  80%|████████  | 401/500 [1:25:51<33:48, 20.49s/it]
process (GPU):  80%|████████  | 402/500 [1:25:53<25:57, 15.90s/it]
process (GPU):  81%|████████  | 403/500 [1:25:56<19:57, 12.35s/it]
process (GPU):  81%|████████  | 404/500 [1:25:57<14:38,  9.15s/it]
process (GPU):  81%|████████  | 405/500 [1:26:17<19:31, 12.33s/it]
process (GPU):  81%|████████▏ | 407/500 [1:26:25<13:13,  8.53s/it]
process (GPU):  82%|████████▏ | 408/500 [1:26:30<11:40,  7.61s/it]
process (GPU):  82%|████████▏ | 410/500 [1:26:35<08:22,  5.58s/it]
process (GPU):  82%|████████▏ | 411/500 [1:27:27<23:53, 16.11s/it]
process (GPU):  82%|████████▏ | 412/500 [1:27:41<22:42, 15.48s/it]
process (GPU):  83%|████████▎ | 413/500 [1:27:43<17:34, 12.12s/it]
process (GPU):  83%|████████▎ | 415/500 [1:28:00<15:06, 10.66s/it]
process (GPU):  83%|████████▎ | 416/500 [1:28:05<12:58,  9.27s/it]
process (GPU):  83%|████████▎ | 417/500 [1:28:06<10:05,  7.30s/it]
process (GPU):  84%|████████▍ | 420/500 [1:28:14<06:20,  4.76s/it]
process (GPU):  84%|████████▍ | 421/500 [1:28:22<07:14,  5.50s/it]
process (GPU):  84%|████████▍ | 422/500 [1:28:23<05:56,  4.58s/it]
process (GPU):  85%|████████▍ | 424/500 [1:28:32<05:38,  4.46s/it]
process (GPU):  85%|████████▌ | 425/500 [1:28:35<05:12,  4.17s/it]
process (GPU):  85%|████████▌ | 426/500 [1:30:04<29:46, 24.14s/it]
process (GPU):  86%|████████▌ | 429/500 [1:30:22<17:47, 15.03s/it]
process (GPU):  86%|████████▌ | 430/500 [1:30:27<15:16, 13.10s/it]
process (GPU):  86%|████████▌ | 431/500 [1:30:29<12:13, 10.64s/it]
process (GPU):  86%|████████▋ | 432/500 [1:30:39<12:03, 10.64s/it]
process (GPU):  87%|████████▋ | 434/500 [1:30:47<08:46,  7.98s/it]
process (GPU):  87%|████████▋ | 435/500 [1:31:08<11:36, 10.72s/it]
process (GPU):  87%|████████▋ | 437/500 [1:32:00<17:31, 16.70s/it]
process (GPU):  88%|████████▊ | 438/500 [1:32:41<22:45, 22.02s/it]
process (GPU):  88%|████████▊ | 439/500 [1:32:43<17:40, 17.39s/it]
process (GPU):  88%|████████▊ | 440/500 [1:32:55<15:50, 15.84s/it]
process (GPU):  88%|████████▊ | 442/500 [1:33:25<14:55, 15.44s/it]
process (GPU):  89%|████████▊ | 443/500 [1:34:02<19:36, 20.64s/it]
process (GPU):  89%|████████▉ | 445/500 [1:34:16<13:52, 15.13s/it]
process (GPU):  89%|████████▉ | 446/500 [1:34:23<12:04, 13.41s/it]
process (GPU):  90%|████████▉ | 448/500 [1:34:51<11:49, 13.64s/it]
process (GPU):  90%|████████▉ | 449/500 [1:34:59<10:30, 12.36s/it]
process (GPU):  90%|█████████ | 450/500 [1:35:04<08:46, 10.53s/it]
process (GPU):  90%|█████████ | 451/500 [1:36:29<24:05, 29.50s/it]
process (GPU):  90%|█████████ | 452/500 [1:36:34<18:19, 22.90s/it]
process (GPU):  91%|█████████ | 453/500 [1:36:41<14:35, 18.64s/it]
process (GPU):  91%|█████████ | 454/500 [1:36:43<10:34, 13.80s/it]
process (GPU):  91%|█████████ | 455/500 [1:36:53<09:31, 12.69s/it]
process (GPU):  91%|█████████ | 456/500 [1:36:56<07:18,  9.97s/it]
process (GPU):  91%|█████████▏| 457/500 [1:37:03<06:23,  8.93s/it]
process (GPU):  92%|█████████▏| 458/500 [1:37:15<06:59,  9.98s/it]
process (GPU):  92%|█████████▏| 459/500 [1:37:56<13:02, 19.09s/it]
process (GPU):  92%|█████████▏| 460/500 [1:37:58<09:20, 14.01s/it]
process (GPU):  92%|█████████▏| 461/500 [1:38:36<13:47, 21.21s/it]
process (GPU):  92%|█████████▏| 462/500 [1:39:00<14:03, 22.20s/it]
process (GPU):  93%|█████████▎| 464/500 [1:39:25<10:38, 17.75s/it]
process (GPU):  93%|█████████▎| 465/500 [1:39:50<11:20, 19.45s/it]
process (GPU):  93%|█████████▎| 466/500 [1:39:53<08:34, 15.12s/it]
process (GPU):  93%|█████████▎| 467/500 [1:40:00<07:05, 12.88s/it]
process (GPU):  94%|█████████▍| 469/500 [1:40:02<04:02,  7.81s/it]
process (GPU):  94%|█████████▍| 471/500 [1:40:05<02:36,  5.39s/it]
process (GPU):  94%|█████████▍| 472/500 [1:40:18<03:14,  6.96s/it]
process (GPU):  95%|█████████▍| 474/500 [1:40:25<02:26,  5.65s/it]
process (GPU):  95%|█████████▌| 475/500 [1:40:26<01:56,  4.67s/it]
process (GPU):  95%|█████████▌| 476/500 [1:40:31<01:54,  4.77s/it]
process (GPU):  95%|█████████▌| 477/500 [1:40:36<01:50,  4.81s/it]
process (GPU):  96%|█████████▌| 478/500 [1:40:38<01:25,  3.88s/it]
process (GPU):  96%|█████████▌| 479/500 [1:40:42<01:23,  3.96s/it]
process (GPU):  96%|█████████▌| 480/500 [1:41:04<02:59,  8.98s/it]
process (GPU):  96%|█████████▌| 481/500 [1:41:11<02:43,  8.63s/it]
process (GPU):  96%|█████████▋| 482/500 [1:41:16<02:14,  7.46s/it]
process (GPU):  97%|█████████▋| 483/500 [1:41:20<01:52,  6.61s/it]
process (GPU):  97%|█████████▋| 484/500 [1:43:25<11:02, 41.40s/it]
process (GPU):  97%|█████████▋| 485/500 [1:43:35<08:00, 32.04s/it]
process (GPU):  97%|█████████▋| 487/500 [1:44:07<05:22, 24.84s/it]
process (GPU):  98%|█████████▊| 488/500 [1:44:16<04:10, 20.89s/it]
process (GPU):  98%|█████████▊| 489/500 [1:44:31<03:31, 19.23s/it]
process (GPU):  98%|█████████▊| 490/500 [1:44:54<03:24, 20.41s/it]
process (GPU):  98%|█████████▊| 491/500 [1:45:01<02:28, 16.50s/it]
process (GPU):  98%|█████████▊| 492/500 [1:45:15<02:06, 15.78s/it]
process (GPU):  99%|█████████▊| 493/500 [1:45:25<01:39, 14.16s/it]
process (GPU):  99%|█████████▉| 494/500 [1:45:42<01:30, 15.01s/it]
process (GPU):  99%|█████████▉| 495/500 [1:46:01<01:20, 16.17s/it]
process (GPU):  99%|█████████▉| 496/500 [1:46:15<01:02, 15.54s/it]
process (GPU):  99%|█████████▉| 497/500 [1:46:42<00:56, 18.97s/it]
process (GPU): 100%|█████████▉| 498/500 [1:47:09<00:42, 21.24s/it]
process (GPU): 100%|█████████▉| 499/500 [1:47:30<00:21, 21.23s/it]
process (GPU): 100%|██████████| 500/500 [1:47:50<00:00, 20.81s/it]
process (GPU): 100%|██████████| 500/500 [1:47:50<00:00, 12.94s/it]
GPU processor stats: {'decoded': 88103, 'chunks_emitted': 67652, 'segments_chunked': 8196, 'dropped_too_short': 72083, 'dropped_too_long': 458, 'dropped_clipped': 12}
Done. Stats written to /tmp/xcodec_validate/gpu_500/stats/hindi_stats.json

real	109m9.824s
user	120m17.028s
sys	3m10.795s


**Reading stats JSON for 500 run**


Output:
Exit code: 0
Wall time: 0 seconds
Output:
{
  "videos": 500,
  "segments_total": 88103,
  "segments_kept": {
    "16000": 67652,
    "24000": 50137,
    "32000": 42660,
    "44100": 12336
  },
  "dropped_too_short": 0,
  "dropped_too_long": 0,
  "dropped_clipped": 0,
  "segments_chunked": 0,
  "chunks_emitted": 0,
  "native_sr_buckets": {
    "32-44k": 16016,
    "24-32k": 40974,
    "16-24k": 9495,
    "<16k": 1158,
    ">=44k": 9
  }
}


**Planning additional stats analysis**

I'm outlining how to compute bucket distributions from native stats and drop rates from GPU processor stats in run outputs, which aren't saved automatically and need manual parsing. I'll also validate shard integrity by sampling JSONL entries in tar files for selected runs to ensure data completeness.


**Planning shard sample checks**

I'll create a small script to verify the first 10 entries in the first shard of each SR path for both 100 and 500 runs to efficiently validate shard integrity without heavy processing.


python3 - <<'PY'
import io
import json
import tarfile
from pathlib import Path
import soundfile as sf

base = Path('/tmp/xcodec_validate/gpu_100')

for sr_dir in ['hindi_16','hindi_24','hindi_32','hindi_44']:
    tar_path = base / sr_dir / 'shard-000000.tar'
    jsonl_path = base / sr_dir / 'shard-000000.jsonl'
    if not tar_path.exists():
        continue
    with tarfile.open(tar_path, 'r') as tf:
        tar_names = set(tf.getnames())
        lines = [json.loads(l) for l in jsonl_path.read_text().splitlines() if l.strip()]
        missing = 0
        mismatched_sr = 0
        checked = 0
        for entry in lines[:10]:
            checked += 1
            audio_name = f"{entry['id']}.flac"
            if audio_name not in tar_names:
                missing += 1
                continue
            member = tf.getmember(audio_name)
            audio_bytes = tf.extractfile(member).read()
            audio, sr = sf.read(io.BytesIO(audio_bytes))
            if sr != entry['sample_rate']:
                mismatched_sr += 1
        print(f"gpu_100 {sr_dir}: entries={len(lines)} checked={checked} missing={missing} sr_mismatch={mismatched_sr}")
PY


Output:
Exit code: 0
Wall time: 7.1 seconds
Output:
gpu_100 hindi_16: entries=13468 checked=10 missing=0 sr_mismatch=0
gpu_100 hindi_24: entries=10310 checked=10 missing=0 sr_mismatch=0
gpu_100 hindi_32: entries=8805 checked=10 missing=0 sr_mismatch=0
gpu_100 hindi_44: entries=2877 checked=10 missing=0 sr_mismatch=0


**Adding check for 500 status**


python3 - <<'PY'
import io
import json
import tarfile
from pathlib import Path
import soundfile as sf

base = Path('/tmp/xcodec_validate/gpu_500')

for sr_dir in ['hindi_16','hindi_24','hindi_32','hindi_44']:
    tar_path = base / sr_dir / 'shard-000000.tar'
    jsonl_path = base / sr_dir / 'shard-000000.jsonl'
    if not tar_path.exists():
        continue
    with tarfile.open(tar_path, 'r') as tf:
        tar_names = set(tf.getnames())
        lines = [json.loads(l) for l in jsonl_path.read_text().splitlines() if l.strip()]
        missing = 0
        mismatched_sr = 0
        checked = 0
        for entry in lines[:10]:
            checked += 1
            audio_name = f"{entry['id']}.flac"
            if audio_name not in tar_names:
                missing += 1
                continue
            member = tf.getmember(audio_name)
            audio_bytes = tf.extractfile(member).read()
            audio, sr = sf.read(io.BytesIO(audio_bytes))
            if sr != entry['sample_rate']:
                mismatched_sr += 1
        print(f"gpu_500 {sr_dir}: entries={len(lines)} checked={checked} missing={missing} sr_mismatch={mismatched_sr}")
PY


Output:
Exit code: 124
Wall time: 10 seconds
Output:
command timed out after 10028 milliseconds


**Extending timeout for tar reading**

I'm increasing the timeout to 60 seconds to accommodate possible slow decompression when reading multiple entries from a large tar archive.


Output:
Exit code: 0
Wall time: 16.4 seconds
Output:
gpu_500 hindi_16: entries=42511 checked=10 missing=0 sr_mismatch=0
gpu_500 hindi_24: entries=24761 checked=10 missing=0 sr_mismatch=0
gpu_500 hindi_32: entries=19711 checked=10 missing=0 sr_mismatch=0
gpu_500 hindi_44: entries=12336 checked=10 missing=0 sr_mismatch=0


**Planning shard entry count validation**

Planning to sum entries across all shards per bucket to verify total matches the stats, clarifying shard distribution and ensuring output counts align for 100 and 500 runs.


**Initiating shard count script run**


python3 - <<'PY'
import json
from pathlib import Path

for label in ['gpu_100','gpu_500']:
    base = Path('/tmp/xcodec_validate') / label
    stats_path = base / 'stats' / 'hindi_stats.json'
    stats = json.loads(stats_path.read_text())
    print(f"{label} stats segments_kept: {stats['segments_kept']}")
    for sr_dir, sr in [('hindi_16',16000),('hindi_24',24000),('hindi_32',32000),('hindi_44',44100)]:
        dir_path = base / sr_dir
        if not dir_path.exists():
            continue
        total = 0
        for jsonl in dir_path.glob('*.jsonl'):
            total += sum(1 for _ in jsonl.open('r', encoding='utf-8'))
        print(f"  {label} {sr_dir} total entries: {total}")
    print('')
PY


Output:
Exit code: 0
Wall time: 0.2 seconds
Output:
gpu_100 stats segments_kept: {'16000': 13468, '24000': 10310, '32000': 8805, '44100': 2877}
  gpu_100 hindi_16 total entries: 13468
  gpu_100 hindi_24 total entries: 10310
  gpu_100 hindi_32 total entries: 8805
  gpu_100 hindi_44 total entries: 2877

gpu_500 stats segments_kept: {'16000': 67652, '24000': 50137, '32000': 42660, '44100': 12336}
  gpu_500 hindi_16 total entries: 67652
  gpu_500 hindi_24 total entries: 50137
  gpu_500 hindi_32 total entries: 42660
  gpu_500 hindi_44 total entries: 12336


Summary: auto


**Planning detailed prefix validation**

I'm planning a script to sample around 500 Hindi video records and run a custom key resolution that tracks which candidate key (exact, prefix, or basename) matches via S3 head requests, enabling analysis of prefix handling accuracy.


**Confirming sample approach for prefix check**

The plan is to query 500 random Hindi records and test candidate keys sequentially to identify matches and count not-found cases, improving prefix resolution accuracy.


python3 - <<'PY'
import os
from pathlib import Path
import psycopg
import boto3

# load env
env = {}
for line in Path('.env').read_text().splitlines():
    line = line.strip()
    if not line or line.startswith('#') or '=' not in line:
        continue
    if line.startswith('export '):
        line = line[len('export '):]
    k, v = line.split('=', 1)
    env[k.strip()] = v.strip()

endpoint = env.get('R2_ENDPOINT_URL')
access_key = env.get('R2_ACCESS_KEY_ID')
secret_key = env.get('R2_SECRET_ACCESS_KEY')
cleaned_bucket = env.get('R2_ClEANED_BUCKET') or env.get('R2_CLEANED_BUCKET')
prefix = env.get('R2_PREFIX')

s3 = boto3.client(
    's3',
    endpoint_url=endpoint,
    aws_access_key_id=access_key,
    aws_secret_access_key=secret_key,
)

db_url = env.get('DATABASE_URL')

# fetch 500 random hindi records
with psycopg.connect(db_url) as conn:
    with conn.cursor() as cur:
        cur.execute("""
            select youtube_id, r2_tar_key
            from videos
            where lower(language) = 'hindi'
              and r2_tar_key is not null
              and status = 'COMPLETED'
            order by random()
            limit 500
        """)
        rows = cur.fetchall()

counts = {
    'exact': 0,
    'prefix_added': 0,
    'basename': 0,
    'prefix_basename': 0,
    'missing': 0,
}

for _, r2_key in rows:
    if not r2_key:
        counts['missing'] += 1
        continue
    key = r2_key.lstrip('/')
    candidates = [key]
    if prefix:
        cleaned_prefix = prefix.strip('/')
        if not key.startswith(cleaned_prefix):
            candidates.append(f"{cleaned_prefix}/{key}")
    base = os.path.basename(key)
    if base != key:
        candidates.append(base)
        if prefix:
            candidates.append(f"{prefix.strip('/')}/{base}")

    matched = None
    for i, cand in enumerate(candidates):
        try:
            s3.head_object(Bucket=cleaned_bucket, Key=cand)
            matched = i
            break
        except Exception:
            continue

    if matched is None:
        counts['missing'] += 1
    else:
        if matched == 0:
            counts['exact'] += 1
        elif matched == 1:
            counts['prefix_added'] += 1
        elif matched == 2:
            counts['basename'] += 1
        elif matched == 3:
            counts['prefix_basename'] += 1

print(counts)
PY


Output:
Exit code: 0
Wall time: 31.7 seconds
Output:
{'exact': 500, 'prefix_added': 0, 'basename': 0, 'prefix_basename': 0, 'missing': 0}


**Planning explicit segment-metadata validation**

I'm considering creating a separate script to randomly validate 100 segments by decoding and matching metadata, ensuring no mismatches or dropped segments beyond existing implicit checks in the pipeline. This standalone validation complements pipeline processing and verifies segment integrity more robustly.


**Detailing validation script for sample tars**

I'm planning a script to validate 100 sample tars by matching metadata segments to flac files, checking for mismatches, decoding audio to catch failures, and comparing durations and sample rates—all using local tar files from the GPU run to ensure thorough segment integrity.


python3 - <<'PY'
import io
import json
import tarfile
from pathlib import Path
import numpy as np
import soundfile as sf
import sys

sys.path.insert(0, str(Path('scripts').resolve()))
from prepare_shards import parse_segment_filename

base = Path('/tmp/xcodec_validate/gpu_tmp_100/downloads')
if not base.exists():
    raise SystemExit('download dir not found')

stats = {
    'videos': 0,
    'segments_in_tar': 0,
    'segments_usable_meta': 0,
    'parse_fail': 0,
    'meta_mismatch': 0,
    'decode_fail': 0,
    'duration_diff_count': 0,
    'duration_diff_max': 0.0,
    'duration_diff_mean': 0.0,
}

duration_diffs = []

sample_rates = {}

for tar_path in sorted(base.glob('*.tar')):
    video_id = tar_path.stem
    stats['videos'] += 1
    with tarfile.open(tar_path, 'r') as tf:
        try:
            meta = json.loads(tf.extractfile(f"{video_id}/metadata.json").read())
        except Exception:
            continue
        segments = meta.get('segments', [])
        usable = [s for s in segments if s.get('status') == 'usable']
        stats['segments_usable_meta'] += len(usable)
        segment_map = {
            (s.get('speaker'), round(s.get('start', 0.0), 2)): s
            for s in usable
        }
        members = [m for m in tf.getmembers() if m.name.endswith('.flac')]
        stats['segments_in_tar'] += len(members)
        for member in members:
            parsed = parse_segment_filename(member.name)
            if not parsed:
                stats['parse_fail'] += 1
                continue
            speaker, start_time = parsed
            seg_meta = segment_map.get((speaker, round(start_time, 2)))
            if not seg_meta:
                matches = [
                    s for s in usable
                    if s.get('speaker') == speaker
                    and abs(s.get('start', 0.0) - start_time) < 0.1
                ]
                if not matches:
                    stats['meta_mismatch'] += 1
                    continue
                seg_meta = matches[0]
            try:
                audio, sr = sf.read(io.BytesIO(tf.extractfile(member).read()), dtype='float32')
            except Exception:
                stats['decode_fail'] += 1
                continue
            if audio.ndim > 1:
                audio = np.mean(audio, axis=1)
            dur = float(audio.shape[0] / sr)
            sample_rates[sr] = sample_rates.get(sr, 0) + 1
            expected = None
            if seg_meta.get('duration') is not None:
                expected = float(seg_meta['duration'])
            elif seg_meta.get('end') is not None and seg_meta.get('start') is not None:
                expected = float(seg_meta['end'] - seg_meta['start'])
            if expected is not None:
                diff = abs(dur - expected)
                duration_diffs.append(diff)

if duration_diffs:
    stats['duration_diff_count'] = len(duration_diffs)
    stats['duration_diff_max'] = float(np.max(duration_diffs))
    stats['duration_diff_mean'] = float(np.mean(duration_diffs))

print('validation_stats', stats)
print('sample_rates', sample_rates)
PY


Output:
Exit code: 124
Wall time: 200 seconds
Output:
command timed out after 200018 milliseconds


**Extending validation timeout**

The validation script timed out at 200 seconds, so I'm increasing the timeout to 10 minutes and rerunning it from the start.


Output:
Exit code: 0
Wall time: 526.1 seconds
Output:
validation_stats {'videos': 300, 'segments_in_tar': 57503, 'segments_usable_meta': 57503, 'parse_fail': 0, 'meta_mismatch': 0, 'decode_fail': 0, 'duration_diff_count': 57503, 'duration_diff_max': 0.26041666666671226, 'duration_diff_mean': 1.763163956299632e-05}
sample_rates {48000: 57057, 44100: 446}


**Analyzing video validation details**

The validation currently covers 300 videos due to accumulated runs, which is more robust than expected. Segment counts match perfectly, decoding errors are zero, and timing differences are minimal, likely just rounding noise. Some videos use 44.1kHz audio instead of the expected 48kHz, which is acceptable, though sampling rate stats come only from 16k outputs; further nyquist distribution analysis could be helpful.


**Summarizing GPU validation results and proposing next steps**

Validation covered 300 videos from accumulated runs with perfect segment matching and no decode errors; duration differences were minimal (0.26s max), mostly rounding noise. GPU stats reflect outputs (post-filtering) rather than raw segments; about 81% of segments were dropped for being too short (<3s), consistent across runs. Audio sample rates were mostly 48kHz with a small 44.1kHz portion. Throughput averaged 13s per video, projecting 75 days for 500k videos on a single GPU machine. Bandwidth and sample rate distributions largely align with the dataset strategy, but cross-language bandwidth validation and upload verification remain as pending tasks. GPU stats reporting shows some zeros due to known issues but do not affect pipeline output integrity. Next steps include (1) optional multi-language sample for bandwidth, (2) upload integrity verification, and (3) addressing GPU stats reporting bugs.


**GPU Pipeline Validation (Hindi only)**

- Ran GPU-path validation for 100 and 500 video batches in `/tmp/xcodec_validate/gpu_100` and `/tmp/xcodec_validate/gpu_500`.
- 100 run: 21m10s, 18,406 segments, 13,468 16k outputs; dropped 79.6% segments (too short), 0.23% too long; 24k outputs 76.5%, 32k 65.4%, 44k 21.4%.
- 500 run: 109m10s, 88,103 segments, 67,652 16k outputs; dropped 81.8% segments too short; 0.52% too long; 24k outputs 74.1%, 32k 63.1%, 44k 18.2%.
- Shard integrity verified with first shard JSONL counts matching stats; no missing audio detected.

**Raw Tar Integrity (300 videos)**
- Validated 300 videos from cached downloads (`/tmp/xcodec_validate/gpu_tmp_100/downloads`).
- No parse, decode, or metadata mismatches; durations match closely (mean diff 1.8e-5s, max 0.26s).
- Sample rates mostly 48kHz (99.2%), small 44.1kHz portion (0.8%).

**R2 Key Resolution**
- In 500-sample Hindi validation, all tar keys resolved exactly; no prefix fallback needed.

**Bandwidth and Sample Rate Distribution**
- Native sample rate bucket distribution consistent between runs and dataset strategy (e.g., 500 run: 24-32k 60.6%, 32-44k 23.7%).
- Segment sample rate distribution confirms mostly 48kHz with small 44.1kHz segments.

**ETA Estimate**
- Average throughput 12.9 seconds per video from runs.
- Extrapolates to 75 days for full 500k-video dataset on a single A100 GPU host; scales linearly with additional machines.
- Segment decode throughput 13.5 segments/sec (500 run).

**Known Issues and Remaining Tasks**
- GPU stats JSON lacks drop counts; only available from GPU logs (known reporting issue).
- Cross-language bandwidth distribution and upload verification not yet performed.
- Training pipeline not changed; outputs differ from plan’s expected WebDataset format, using file lists instead.
- Validation only covers GPU path per request; CPU parity checks skipped.

**Next Steps Suggestions**
1. Run smaller GPU validation samples for at least one additional language to confirm cross-language bandwidth distribution.
2. Perform R2 upload integrity verification on a sample batch.
3. Address GPU stats reporting to write drop and processing counts into stats JSON for completeness.


Output:
Exit code: 0
Wall time: 0 seconds
Output:
100 pct 24/16 76.6
100 pct 32/16 65.4
100 pct 44/16 21.4
100 32-44k 25.5
100 24-32k 55.7
100 16-24k 14.0
100 <16k 4.7
100 >=44k 0.1
500 pct 24/16 74.1
500 pct 32/16 63.1
500 pct 44/16 18.2
500 32-44k 23.7
500 24-32k 60.6
500 16-24k 14.0
500 <16k 1.7
500 >=44k 0.0
sec per video 100 12.705
sec per video 500 13.0996
eta days 500k 75.80787037037037
segs/s 100 14.487209759937032
segs/s 500 13.451250419860148


**Computing drop rates**

I'm calculating drop rates from run data, finding about 79.6% dropped in the 100 run and 81.8% in the 500 run.


python3 - <<'PY'
from math import floor

def pct(n,d):
    return round(100*n/d,1)

print('100 drop_short', pct(14659,18406))
print('100 drop_long', pct(43,18406))
print('500 drop_short', pct(72083,88103))
print('500 drop_long', pct(458,88103))
print('500 drop_clip', pct(12,88103))
PY


Output:
Exit code: 0
Wall time: 0 seconds
Output:
100 drop_short 79.6
100 drop_long 0.2
500 drop_short 81.8
500 drop_long 0.5
500 drop_clip 0.0


**Calculating drop clip ratios**

I'm computing precise drop clip ratios for different sample rates and will present counts alongside percentages for accuracy and clarity.


python3 - <<'PY'
from math import floor

def pct(n,d):
    return round(100*n/d,1)

print('48k', pct(57057,57503))
print('44.1k', pct(446,57503))
PY


Output:
Exit code: 0
Wall time: 0 seconds
Output:
48k 99.2
44.1k 0.8


**Computing average segments per video**

The average number of segments per video in the 300-video validation set is about 191.7.


Output:
Exit code: 0
Wall time: 0 seconds
Output:
191.67666666666668


**Summarizing final validation and next steps**

The summary covers GPU run results, raw segment integrity, shard consistency, bandwidth eligibility, and ETA estimates, while noting gaps like missing GPU drop stats and limited language sampling. Suggested next actions include running GPU for another language, verifying R2 uploads, and optionally patching stats logging without code changes.


**Detailing GPU and data validation results**

The summary lists comprehensive GPU run stats for Hindi, including segment counts, drop reasons, bandwidth buckets, and raw tar integrity with zero parse failures. It confirms R2 keys resolve exactly and estimates ETA on a single A100. Notable gaps include GPU drop stats only in stdout, unvalidated cross-language runs, and training pipeline still requiring filelists—not shards. Optional next steps cover additional language sampling, upload verification, and GPU stats patching without altering training code.


Ran the remaining GPU‑only validations and larger samples; no code changes made, and I kept everything confined to `/tmp`.

**GPU Pipeline Runs (Hindi, GPU path)**
- `--limit 100`: 21m10s total, 18,406 segments decoded; 16k outputs 13,468; 24k/32k/44k eligibility 76.6%/65.4%/21.4% of 16k outputs.
- GPU processor stats (stdout): dropped_too_short 14,659 (79.6%), dropped_too_long 43 (0.2%), dropped_clipped 0; segments_chunked 1,576; chunks_emitted 13,468.
- `--limit 500`: 109m10s total, 88,103 segments decoded; 16k outputs 67,652; 24k/32k/44k eligibility 74.1%/63.1%/18.2%.
- GPU processor stats (stdout): dropped_too_short 72,083 (81.8%), dropped_too_long 458 (0.5%), dropped_clipped 12; segments_chunked 8,196; chunks_emitted 67,652.
- Native bandwidth buckets from the 500 run (based on 16k outputs): 24–32k 60.6%, 32–44k 23.7%, 16–24k 14.0%, <16k 1.7%, ≥44k ~0.0%.
- Artifacts: `/tmp/xcodec_validate/gpu_100`, `/tmp/xcodec_validate/gpu_500`.

**Raw Tar Integrity (300 Hindi videos from cached downloads)**
- 57,503 segment files matched 57,503 usable metadata segments; parse_fail 0, meta_mismatch 0, decode_fail 0.
- Duration diff mean 1.76e‑5s, max 0.2604s (likely rounding/segment boundary).
- Container SRs: 99.2% 48k, 0.8% 44.1k.
- Download cache used: `/tmp/xcodec_validate/gpu_tmp_100/downloads`.

**Shard/Manifest Integrity**
- JSONL line counts across all shards equal `stats` totals for 100 and 500 runs.
- Spot‑checked `shard-000000.*` in each SR bucket: 0 missing audio entries, 0 SR mismatches.

**R2 Key Resolution (Hindi)**
- 500 random rows: `r2_tar_key` resolved directly 100% of the time; prefix fallback never needed; no missing objects.

**ETA (Single A100, current config)**
- ~13.1s/video (500‑run), ~4.6 videos/min, ~13.5 segments/sec.
- 500k videos ≈ 75.8 days on one node; scale linearly with more nodes.

**Known Gaps / Reporting Issues**
- GPU runs write drop/chunk stats only to stdout; `stats/*.json` shows zeros for those fields (outputs are correct, reporting is incomplete).
- Only Hindi sampled at 100/500; cross‑language bandwidth distribution still unverified.
- Training pipeline still consumes filelists (`X-Codec-2.0/data_module.py`), not the WebDataset shards/manifests yet (plan integration gap, unchanged).

If you want me to continue, pick one:
1) Run the same 100/500 GPU validation for another language (e.g., Telugu) to confirm cross‑language bandwidth distribution.
2) Run an R2 upload + `--verify-upload` smoke test for one small batch.
3) Patch `scripts/prepare_shards.py` to persist GPU drop/chunk stats into `stats/*.json` (no training changes).