⚠ benchmark_cohere.py compatibility: the script expects a flat checkpoint dir (model.safetensors + config.json +
  tokenizer.json all in one folder). Our new ckpts are split into step-N/model/ and step-N/processor/. You'll need to either:
  - Pass step-N/model/ as --checkpoint and cp step-N/processor/tokenizer*.json step-N/model/ first, OR
  - Patch benchmark_cohere.py to load processor from <ckpt>/processor/ and model from <ckpt>/model/

  The KEY_REMAP block is now harmless (our new ckpts already have correct keys), so it won't break — just won't match anything.

  Now committing + pushing.

● Bash(cd /workspace/training; ls .gitignore 2>/dev/null && cat .gitignore || echo "no .gitignore")
  ⎿  .gitignore
     __pycache__/
     *.pyc
     … +6 lines (ctrl+o to expand)

● Bash(cd /workspace/training
      git add config.yaml train.py push_to_r2.sh…)
  ⎿  rm 'config_smoke.yaml'
     rm 'dataset_fast.py'
     rm 'merge_tokenizer_bpe.py'
     … +27 lines (ctrl+o to expand)

● Bash(cd /workspace/training; git commit -m "$(cat <<'EOF'
      Phase 1 production training: dataset v2, tiny eval, launch + auto-resume…)
  ⎿   create mode 100755 launch_phase1.sh
      delete mode 100644 merge_tokenizer_bpe.py
      delete mode 100644 merge_tokenizer_v2.py
     … +2 lines (ctrl+o to expand)

● Bash(cd /workspace/training; git push 2>&1 | tail -10)
  ⎿  To https://github.com/Maya-AI-ML-Trainings-Cookbooks/cohere-transcribe-indic.git
        d619b3a..455d77c  main -> main

● Pushed: d619b3a..455d77c to Maya-AI-ML-Trainings-Cookbooks/cohere-transcribe-indic main.

  Summary of commit 455d77c:
  - ✅ New: dataset_v2.py, tiny_eval.py, build_eval_set.py, launch_phase1.sh, auto_resume_phase1.sh
  - ✅ Modified: train.py (env-based wandb, tiny_eval wiring, importlib.reload, dropped dead dataset_fast import), config.yaml
  (validated 32/400k batch + eval_every=5000), push_to_r2.sh (date)
  - ✅ Removed: config_smoke.yaml, dataset_fast.py, merge_tokenizer_*.py, smoke_test.py

  For your inference work:
  1. Rotate the GitHub PAT in the remote URL (security)
  2. Patch benchmark_cohere.py to handle the new split layout — needs <ckpt>/model/ for safetensors+config and
  <ckpt>/processor/ for tokenizer. Quickest hack: python two cp calls before invoking it on a downloaded R2 ckpt.
  3. R2 ckpts available now: r2:ptcheckpoints/cohere-transcribe-mixed/04-07-2026/ckpt-{20000,40000,60000}/. Each has model/ +
  processor/ + training_state.pt.