<persisted-output>
Output too large (4.2MB). Full output saved to: /home/ubuntu/.claude/projects/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tool-results/b1o8quhc4.txt

Preview (first 2KB):
[3]   Done                    AWS_ACCESS_KEY_ID=$R2_ACCESS_KEY_ID AWS_SECRET_ACCESS_KEY=$R2_SECRET_ACCESS_KEY aws s3 cp "s3://ptcheckpoints/gemma3n-E2B-asr/03-28-2026/ckpt-20000/$f" "/home/ubuntu/training/checkpoints/gemma3n-e2b-ckpt-20000/$f" --endpoint-url "$R2_ENDPOINT_URL" --region auto
Completed 3.5 MiB/31.9 MiB (7.1 MiB/s) with 1 file(s) remaining
Completed 3.8 MiB/31.9 MiB (7.5 MiB/s) with 1 file(s) remaining
Completed 4.0 MiB/31.9 MiB (8.0 MiB/s) with 1 file(s) remaining
Completed 4.2 MiB/31.9 MiB (8.5 MiB/s) with 1 file(s) remaining
Completed 4.5 MiB/31.9 MiB (8.9 MiB/s) with 1 file(s) remaining
Completed 4.8 MiB/31.9 MiB (9.3 MiB/s) with 1 file(s) remaining
Completed 5.0 MiB/31.9 MiB (9.8 MiB/s) with 1 file(s) remaining
Completed 5.2 MiB/31.9 MiB (10.2 MiB/s) with 1 file(s) remaining
Completed 5.5 MiB/31.9 MiB (10.7 MiB/s) with 1 file(s) remaining
Completed 5.8 MiB/31.9 MiB (11.1 MiB/s) with 1 file(s) remaining
Completed 2.2 KiB/2.2 KiB (4.3 KiB/s) with 1 file(s) remaining
download: s3://ptcheckpoints/gemma3n-E2B-asr/03-28-2026/ckpt-20000/processor_config.json to ../checkpoints/gemma3n-e2b-ckpt-20000/processor_config.json
...
</persisted-output>

[3]   Done                    AWS_ACCESS_KEY_ID=$R2_ACCESS_KEY_ID AWS_SECRET_ACCESS_KEY=$R2_SECRET_ACCESS_KEY aws s3 cp "s3://ptcheckpoints/gemma3n-E2B-asr/03-28-2026/ckpt-20000/$f" "/home/ubuntu/training/checkpoints/gemma3n-e2b-ckpt-20000/$f" --endpoint-url "$R2_ENDPOINT_URL" --region auto
Completed 3.5 MiB/31.9 MiB (7.1 MiB/s) with 1 file(s) remaining
Completed 3.8 MiB/31.9 MiB (7.5 MiB/s) with 1 file(s) remaining
Completed 4.0 MiB/31.9 MiB (8.0 MiB/s) with 1 file(s) remaining
Completed 4.2 MiB/31.9 MiB (8.5 MiB/s) with 1 file(s) remaining
Completed 4.5 MiB/31.9 MiB (8.9 MiB/s) with 1 file(s) remaining
Completed 4.8 MiB/31.9 MiB (9.3 MiB/s) with 1 file(s) remaining
Completed 5.0 MiB/31.9 MiB (9.8 MiB/s) with 1 file(s) remaining
Completed 5.2 MiB/31.9 MiB (10.2 MiB/s) with 1 file(s) remaining
Completed 5.5 MiB/31.9 MiB (10.7 MiB/s) with 1 file(s) remaining
Completed 5.8 MiB/31.9 MiB (11.1 MiB/s) with 1 file(s) remaining
Completed 2.2 KiB/2.2 KiB (4.3 KiB/s) with 1 file(s) remaining
download: s3://ptcheckpoints/gemma3n-E2B-asr/03-28-2026/ckpt-20000/processor_config.json to ../checkpoints/gemma3n-e2b-ckpt-20000/processor_config.json
Completed 6.0 MiB/31.9 MiB (11.6 MiB/s) with 1 file(s) remaining
Completed 6.2 MiB/31.9 MiB (12.0 MiB/s) with 1 file(s) remaining
Completed 6.5 MiB/31.9 MiB (12.5 MiB/s) with 1 file(s) remaining
Completed 6.8 MiB/31.9 MiB (12.9 MiB/s) with 1 file(s) remaining
Completed 7.0 MiB/31.9 MiB (13.4 MiB/s) with 1 file(s) remaining
Completed 7.2 MiB/31.9 MiB (13.8 MiB/s) with 1 file(s) remaining
Completed 7.5 MiB/31.9 MiB (14.3 MiB/s) with 1 file(s) remaining
Completed 7.8 MiB/31.9 MiB (14.7 MiB/s) with 1 file(s) remaining
Completed 8.0 MiB/31.9 MiB (15.2 MiB/s) with 1 file(s) remaining
Completed 8.2 MiB/31.9 MiB (15.6 MiB/s) with 1 file(s) remaining
Completed 8.5 MiB/31.9 MiB (16.0 MiB/s) with 1 file(s) remaining
Completed 8.8 MiB/31.9 MiB (16.5 MiB/s) with 1 file(s) remaining
Completed 9.0 MiB/31.9 MiB (16.9 MiB/s) with 1 file(s) remaining
Completed 9.2 MiB/31.9 MiB (17.4 MiB/s) with 1 file(s) remaining
Completed 8.2 MiB/22.3 GiB (15.4 MiB/s) with 1 file(s) remaining
Completed 9.5 MiB/31.9 MiB (17.8 MiB/s) with 1 file(s) remaining
Completed 9.8 MiB/31.9 MiB (18.2 MiB/s) with 1 file(s) remaining
Completed 10.0 MiB/31.9 MiB (18.5 MiB/s) with 1 file(s) remaining
Completed 10.2 MiB/31.9 MiB (18.9 MiB/s) with 1 file(s) remaining
Completed 10.5 MiB/31.9 MiB (19.3 MiB/s) with 1 file(s) remaining
Completed 10.8 MiB/31.9 MiB (19.7 MiB/s) with 1 file(s) remaining
Completed 11.0 MiB/31.9 MiB (20.1 MiB/s) with 1 file(s) remaining
Completed 11.2 MiB/31.9 MiB (20.5 MiB/s) with 1 file(s) remaining
Completed 11.5 MiB/31.9 MiB (20.9 MiB/s) with 1 file(s) remaining
Completed 11.8 MiB/31.9 MiB (21.3 MiB/s) with 1 file(s) remaining
Completed 12.0 MiB/31.9 MiB (21.7 MiB/s) with 1 file(s) remaining
Completed 12.2 MiB/31.9 MiB (22.1 MiB/s) with 1 file(s) remaining
Completed 12.5 MiB/31.9 MiB (22.6 MiB/s) with 1 file(s) remaining
Completed 12.8 MiB/31.9 MiB (23.0 MiB/s) with 1 file(s) remaining
Completed 13.0 MiB/31.9 MiB (23.4 MiB/s) with 1 file(s) remaining
Completed 13.2 MiB/31.9 MiB (23.8 MiB/s) with 1 file(s) remaining
Completed 13.5 MiB/31.9 MiB (24.1 MiB/s) with 1 file(s) remaining
Completed 13.8 MiB/31.9 MiB (24.4 MiB/s) with 1 file(s) remaining
Completed 14.0 MiB/31.9 MiB (24.8 MiB/s) with 1 file(s) remaining
Completed 14.2 MiB/31.9 MiB (25.2 MiB/s) with 1 file(s) remaining
Completed 14.5 MiB/31.9 MiB (25.6 MiB/s) with 1 file(s) remaining
Completed 8.5 MiB/22.3 GiB (15.0 MiB/s) with 1 file(s) remaining
Completed 14.8 MiB/31.9 MiB (25.9 MiB/s) with 1 file(s) remaining
Completed 15.0 MiB/31.9 MiB (26.3 MiB/s) with 1 file(s) remaining
Completed 15.2 MiB/31.9 MiB (26.7 MiB/s) with 1 file(s) remaining
Completed 8.8 MiB/22.3 GiB (15.3 MiB/s) with 1 file(s) remaining
Completed 15.5 MiB/31.9 MiB (27.0 MiB/s) with 1 file(s) remaining
Completed 15.8 MiB/31.9 MiB (27.4 MiB/s) with 1 file(s) remaining
Completed 16.0 MiB/31.9 MiB (27.9 MiB/s) with 1 file(s) remaining
Completed 16.2 MiB/31.9 MiB (28.2 MiB/s) with 1 file(s) remaining
Completed 16.5 MiB/31.9 MiB (28.6 MiB/s) with 1 file(s) remaining
Completed 9.0 MiB/22.3 GiB (15.5 MiB/s) with 1 file(s) remaining
Completed 16.8 MiB/31.9 MiB (28.8 MiB/s) with 1 file(s) remaining
Completed 1004 Bytes/1004 Bytes (1.7 KiB/s) with 1 file(s) remaining
Completed 9.2 MiB/22.3 GiB (15.8 MiB/s) with 1 file(s) remaining
download: s3://ptcheckpoints/gemma3n-E2B-asr/03-28-2026/ckpt-20000/tokenizer_config.json to ../checkpoints/gemma3n-e2b-ckpt-20000/tokenizer_config.json
Completed 17.0 MiB/31.9 MiB (29.0 MiB/s) with 1 file(s) remaining
Completed 17.2 MiB/31.9 MiB (29.4 MiB/s) with 1 file(s) remaining
Completed 17.5 MiB/31.9 MiB (29.7 MiB/s) with 1 file(s) remaining
Completed 17.8 MiB/31.9 MiB (30.1 MiB/s) with 1 file(s) remaining
Completed 9.5 MiB/22.3 GiB (16.1 MiB/s) with 1 file(s) remaining
Completed 18.0 MiB/31.9 MiB (30.5 MiB/s) with 1 file(s) remaining
Completed 18.2 MiB/31.9 MiB (30.8 MiB/s) with 1 file(s) remaining
Completed 18.5 MiB/31.9 MiB (31.2 MiB/s) with 1 file(s) remaining
Completed 18.8 MiB/31.9 MiB (31.5 MiB/s) with 1 file(s) remaining
Completed 19.0 MiB/31.9 MiB (31.9 MiB/s) with 1 file(s) remaining
Completed 19.2 MiB/31.9 MiB (32.2 MiB/s) with 1 file(s) remaining
Completed 19.5 MiB/31.9 MiB (32.6 MiB/s) with 1 file(s) remaining
Completed 19.8 MiB/31.9 MiB (32.9 MiB/s) with 1 file(s) remaining
Completed 20.0 MiB/31.9 MiB (33.2 MiB/s) with 1 file(s) remaining
Completed 20.2 MiB/31.9 MiB (33.5 MiB/s) with 1 file(s) remaining
Completed 20.5 MiB/31.9 MiB (33.8 MiB/s) with 1 file(s) remaining
Completed 20.8 MiB/31.9 MiB (34.2 MiB/s) with 1 file(s) remaining
Completed 21.0 MiB/31.9 MiB (34.6 MiB/s) with 1 file(s) remaining
Completed 9.8 MiB/22.3 GiB (16.0 MiB/s) with 1 file(s) remaining
Completed 21.2 MiB/31.9 MiB (34.9 MiB/s) with 1 file(s) remaining
Completed 21.5 MiB/31.9 MiB (35.3 MiB/s) with 1 file(s) remaining
Completed 21.8 MiB/31.9 MiB (35.6 MiB/s) with 1 file(s) remaining
Completed 22.0 MiB/31.9 MiB (35.9 MiB/s) with 1 file(s) remaining
Completed 22.2 MiB/31.9 MiB (36.1 MiB/s) with 1 file(s) remaining
Completed 22.5 MiB/31.9 MiB (36.4 MiB/s) with 1 file(s) remaining
Completed 22.8 MiB/31.9 MiB (36.8 MiB/s) with 1 file(s) remaining
Completed 23.0 MiB/31.9 MiB (37.1 MiB/s) with 1 file(s) remaining
Completed 23.2 MiB/31.9 MiB (37.4 MiB/s) with 1 file(s) remaining
Completed 23.5 MiB/31.9 MiB (37.8 MiB/s) with 1 file(s) remaining
Completed 10.0 MiB/22.3 GiB (16.0 MiB/s) with 1 file(s) remaining
Completed 23.8 MiB/31.9 MiB (38.0 MiB/s) with 1 file(s) remaining
Completed 23.9 MiB/31.9 MiB (38.1 MiB/s) with 1 file(s) remaining
Completed 10.2 MiB/22.3 GiB (16.3 MiB/s) with 1 file(s) remaining
Completed 10.5 MiB/22.3 GiB (16.6 MiB/s) with 1 file(s) remaining
Completed 10.8 MiB/22.3 GiB (16.9 MiB/s) with 1 file(s) remaining
Completed 11.0 MiB/22.3 GiB (17.3 MiB/s) with 1 file(s) remaining
Completed 11.2 MiB/22.3 GiB (17.7 MiB/s) with 1 file(s) remaining
Completed 11.5 MiB/22.3 GiB (17.9 MiB/s) with 1 file(s) remaining
Completed 11.8 MiB/22.3 GiB (17.7 MiB/s) with 1 file(s) remaining
Completed 12.0 MiB/22.3 GiB (18.0 MiB/s) with 1 file(s) remaining
Completed 12.2 MiB/22.3 GiB (18.1 MiB/s) with 1 file(s) remaining
Completed 12.5 MiB/22.3 GiB (18.4 MiB/s) with 1 file(s) remaining
Completed 12.8 MiB/22.3 GiB (18.7 MiB/s) with 1 file(s) remaining
Completed 13.0 MiB/22.3 GiB (19.0 MiB/s) with 1 file(s) remaining
Completed 13.2 MiB/22.3 GiB (19.4 MiB/s) with 1 file(s) remaining
Completed 13.5 MiB/22.3 GiB (19.7 MiB/s) with 1 file(s) remaining
Completed 13.8 MiB/22.3 GiB (20.0 MiB/s) with 1 file(s) remaining
Completed 14.0 MiB/22.3 GiB (20.3 MiB/s) with 1 file(s) remaining
Completed 14.2 MiB/22.3 GiB (20.7 MiB/s) with 1 file(s) remaining
Completed 14.5 MiB/22.3 GiB (20.9 MiB/s) with 1 file(s) remaining
Completed 14.8 MiB/22.3 GiB (21.2 MiB/s) with 1 file(s) remaining
Completed 15.0 MiB/22.3 GiB (21.4 MiB/s) with 1 file(s) remaining
Completed 15.2 MiB/22.3 GiB (21.7 MiB/s) with 1 file(s) remaining
Completed 15.5 MiB/22.3 GiB (21.9 MiB/s) with 1 file(s) remaining
Completed 15.8 MiB/22.3 GiB (21.9 MiB/s) with 1 file(s) remaining
Completed 16.0 MiB/22.3 GiB (22.0 MiB/s) with 1 file(s) remaining
Completed 16.2 MiB/22.3 GiB (22.3 MiB/s) with 1 file(s) remaining
Completed 16.5 MiB/22.3 GiB (22.5 MiB/s) with 1 file(s) remaining
Completed 16.8 MiB/22.3 GiB (22.7 MiB/s) with 1 file(s) remaining
Completed 17.0 MiB/22.3 GiB (22.8 MiB/s) with 1 file(s) remaining
Completed 17.2 MiB/22.3 GiB (23.1 MiB/s) with 1 file(s) remaining
Completed 17.5 MiB/22.3 GiB (23.3 MiB/s) with 1 file(s) remaining
Completed 17.8 MiB/22.3 GiB (23.7 MiB/s) with 1 file(s) remaining
Completed 18.0 MiB/22.3 GiB (24.0 MiB/s) with 1 file(s) remaining
Completed 18.2 MiB/22.3 GiB (24.3 MiB/s) with 1 file(s) remaining
Completed 18.5 MiB/22.3 GiB (24.5 MiB/s) with 1 file(s) remaining
Completed 18.8 MiB/22.3 GiB (24.8 MiB/s) with 1 file(s) remaining
Completed 19.0 MiB/22.3 GiB (25.0 MiB/s) with 1 file(s) remaining
Completed 19.2 MiB/22.3 GiB (25.3 MiB/s) with 1 file(s) remaining
Completed 19.5 MiB/22.3 GiB (25.6 MiB/s) with 1 file(s) remaining
Completed 19.8 MiB/22.3 GiB (25.9 MiB/s) with 1 file(s) remaining
Completed 20.0 MiB/22.3 GiB (26.2 MiB/s) with 1 file(s) remaining
Completed 20.2 MiB/22.3 GiB (26.5 MiB/s) with 1 file(s) remaining
Completed 20.5 MiB/22.3 GiB (26.8 MiB/s) with 1 file(s) remaining
Completed 20.8 MiB/22.3 GiB (27.1 MiB/s) with 1 file(s) remaining
Completed 21.0 MiB/22.3 GiB (27.4 MiB/s) with 1 file(s) remaining
Completed 21.2 MiB/22.3 GiB (27.7 MiB/s) with 1 file(s) remaining
Completed 21.5 MiB/22.3 GiB (28.0 MiB/s) with 1 file(s) remaining
Completed 21.8 MiB/22.3 GiB (28.3 MiB/s) with 1 file(s) remaining
Completed 22.0 MiB/22.3 GiB (28.6 MiB/s) with 1 file(s) remaining
Completed 22.2 MiB/22.3 GiB (28.9 MiB/s) with 1 file(s) remaining
Completed 22.5 MiB/22.3 GiB (29.2 MiB/s) with 1 file(s) remaining
Completed 22.8 MiB/22.3 GiB (29.5 MiB/s) with 1 file(s) remaining
Completed 23.0 MiB/22.3 GiB (29.8 MiB/s) with 1 file(s) remaining
Completed 23.2 MiB/22.3 GiB (29.9 MiB/s) with 1 file(s) remaining
Completed 23.5 MiB/22.3 GiB (30.2 MiB/s) with 1 file(s) remaining
Completed 23.8 MiB/22.3 GiB (30.5 MiB/s) with 1 file(s) remaining
Completed 24.0 MiB/22.3 GiB (30.8 MiB/s) with 1 file(s) remaining
Completed 24.2 MiB/22.3 GiB (31.1 MiB/s) with 1 file(s) remaining
Completed 24.5 MiB/22.3 GiB (31.4 MiB/s) with 1 file(s) remaining
Completed 24.8 MiB/22.3 GiB (31.7 MiB/s) with 1 file(s) remaining
Completed 25.0 MiB/22.3 GiB (31.9 MiB/s) with 1 file(s) remaining
Completed 25.2 MiB/22.3 GiB (32.2 MiB/s) with 1 file(s) remaining
Completed 25.5 MiB/22.3 GiB (32.5 MiB/s) with 1 file(s) remaining
Completed 25.8 MiB/22.3 GiB (32.8 MiB/s) with 1 file(s) remaining
Completed 26.0 MiB/22.3 GiB (33.1 MiB/s) with 1 file(s) remaining
Completed 26.2 MiB/22.3 GiB (33.4 MiB/s) with 1 file(s) remaining
Completed 26.5 MiB/22.3 GiB (33.7 MiB/s) with 1 file(s) remaining
Completed 26.8 MiB/22.3 GiB (34.0 MiB/s) with 1 file(s) remaining
Completed 27.0 MiB/22.3 GiB (34.2 MiB/s) with 1 file(s) remaining
Completed 27.2 MiB/22.3 GiB (34.5 MiB/s) with 1 file(s) remaining
Completed 27.5 MiB/22.3 GiB (34.8 MiB/s) with 1 file(s) remaining
Completed 27.8 MiB/22.3 GiB (35.1 MiB/s) with 1 file(s) remaining
Completed 24.1 MiB/31.9 MiB (30.6 MiB/s) with 1 file(s) remaining
Completed 28.0 MiB/22.3 GiB (35.4 MiB/s) with 1 file(s) remaining
Completed 28.2 MiB/22.3 GiB (35.7 MiB/s) with 1 file(s) remaining
Completed 28.5 MiB/22.3 GiB (36.0 MiB/s) with 1 file(s) remaining
Completed 28.8 MiB/22.3 GiB (36.2 MiB/s) with 1 file(s) remaining
Completed 29.0 MiB/22.3 GiB (36.5 MiB/s) with 1 file(s) remaining
Completed 29.2 MiB/22.3 GiB (36.8 MiB/s) with 1 file(s) remaining
Completed 29.5 MiB/22.3 GiB (37.1 MiB/s) with 1 file(s) remaining
Completed 29.8 MiB/22.3 GiB (37.4 MiB/s) with 1 file(s) remaining
Completed 30.0 MiB/22.3 GiB (37.6 MiB/s) with 1 file(s) remaining
Completed 30.2 MiB/22.3 GiB (37.9 MiB/s) with 1 file(s) remaining
Completed 30.5 MiB/22.3 GiB (38.2 MiB/s) with 1 file(s) remaining
Completed 30.8 MiB/22.3 GiB (38.4 MiB/s) with 1 file(s) remaining
Completed 31.0 MiB/22.3 GiB (38.7 MiB/s) with 1 file(s) remaining
Completed 31.2 MiB/22.3 GiB (39.0 MiB/s) with 1 file(s) remaining
Completed 31.5 MiB/22.3 GiB (39.3 MiB/s) with 1 file(s) remaining
Completed 31.8 MiB/22.3 GiB (39.6 MiB/s) with 1 file(s) remaining
Completed 32.0 MiB/22.3 GiB (39.9 MiB/s) with 1 file(s) remaining
Completed 32.2 MiB/22.3 GiB (40.1 MiB/s) with 1 file(s) remaining
Completed 32.5 MiB/22.3 GiB (40.4 MiB/s) with 1 file(s) remaining
Completed 32.8 MiB/22.3 GiB (40.7 MiB/s) with 1 file(s) remaining
Completed 33.0 MiB/22.3 GiB (40.9 MiB/s) with 1 file(s) remaining
Completed 33.2 MiB/22.3 GiB (41.2 MiB/s) with 1 file(s) remaining
Completed 33.5 MiB/22.3 GiB (41.5 MiB/s) with 1 file(s) remaining
Completed 33.8 MiB/22.3 GiB (41.8 MiB/s) with 1 file(s) remaining
Completed 34.0 MiB/22.3 GiB (42.0 MiB/s) with 1 file(s) remaining
Completed 34.2 MiB/22.3 GiB (42.3 MiB/s) with 1 file(s) remaining
Completed 34.5 MiB/22.3 GiB (42.6 MiB/s) with 1 file(s) remaining
Completed 34.8 MiB/22.3 GiB (42.7 MiB/s) with 1 file(s) remaining
Completed 35.0 MiB/22.3 GiB (42.9 MiB/s) with 1 file(s) remaining
Completed 35.2 MiB/22.3 GiB (43.2 MiB/s) with 1 file(s) remaining
Completed 35.5 MiB/22.3 GiB (43.4 MiB/s) with 1 file(s) remaining
Completed 35.8 MiB/22.3 GiB (43.5 MiB/s) with 1 file(s) remaining
Completed 36.0 MiB/22.3 GiB (43.5 MiB/s) with 1 file(s) remaining
Completed 36.2 MiB/22.3 GiB (43.8 MiB/s) with 1 file(s) remaining
Completed 36.5 MiB/22.3 GiB (43.9 MiB/s) with 1 file(s) remaining
Completed 36.8 MiB/22.3 GiB (44.1 MiB/s) with 1 file(s) remaining
Completed 37.0 MiB/22.3 GiB (44.4 MiB/s) with 1 file(s) remaining
Completed 37.2 MiB/22.3 GiB (44.6 MiB/s) with 1 file(s) remaining
Completed 37.5 MiB/22.3 GiB (44.9 MiB/s) with 1 file(s) remaining
Completed 37.8 MiB/22.3 GiB (45.2 MiB/s) with 1 file(s) remaining
Completed 38.0 MiB/22.3 GiB (45.4 MiB/s) with 1 file(s) remaining
Completed 38.2 MiB/22.3 GiB (45.7 MiB/s) with 1 file(s) remaining
Completed 38.5 MiB/22.3 GiB (46.0 MiB/s) with 1 file(s) remaining
Completed 38.8 MiB/22.3 GiB (46.2 MiB/s) with 1 file(s) remaining
Completed 39.0 MiB/22.3 GiB (46.4 MiB/s) with 1 file(s) remaining
Completed 39.2 MiB/22.3 GiB (46.7 MiB/s) with 1 file(s) remaining
Completed 39.5 MiB/22.3 GiB (46.9 MiB/s) with 1 file(s) remaining
Completed 39.8 MiB/22.3 GiB (47.1 MiB/s) with 1 file(s) remaining
Completed 40.0 MiB/22.3 GiB (47.3 MiB/s) with 1 file(s) remaining
Completed 24.4 MiB/31.9 MiB (28.9 MiB/s) with 1 file(s) remaining
Completed 40.2 MiB/22.3 GiB (47.5 MiB/s) with 1 file(s) remaining
Completed 40.5 MiB/22.3 GiB (47.8 MiB/s) with 1 file(s) remaining
Completed 40.8 MiB/22.3 GiB (47.8 MiB/s) with 1 file(s) remaining
Completed 41.0 MiB/22.3 GiB (47.8 MiB/s) with 1 file(s) remaining
Completed 41.2 MiB/22.3 GiB (47.9 MiB/s) with 1 file(s) remaining
Completed 41.5 MiB/22.3 GiB (48.2 MiB/s) with 1 file(s) remaining
Completed 41.8 MiB/22.3 GiB (48.4 MiB/s) with 1 file(s) remaining
Completed 42.0 MiB/22.3 GiB (48.6 MiB/s) with 1 file(s) remaining
Completed 42.2 MiB/22.3 GiB (48.8 MiB/s) with 1 file(s) remaining
Completed 42.5 MiB/22.3 GiB (48.9 MiB/s) with 1 file(s) remaining
Completed 42.8 MiB/22.3 GiB (49.2 MiB/s) with 1 file(s) remaining
Completed 43.0 MiB/22.3 GiB (49.2 MiB/s) with 1 file(s) remaining
Completed 43.2 MiB/22.3 GiB (49.4 MiB/s) with 1 file(s) remaining
Completed 43.5 MiB/22.3 GiB (49.6 MiB/s) with 1 file(s) remaining
Completed 43.8 MiB/22.3 GiB (49.9 MiB/s) with 1 file(s) remaining
Completed 44.0 MiB/22.3 GiB (50.1 MiB/s) with 1 file(s) remaining
Completed 44.2 MiB/22.3 GiB (50.3 MiB/s) with 1 file(s) remaining
Completed 44.5 MiB/22.3 GiB (50.5 MiB/s) with 1 file(s) remaining
Completed 44.8 MiB/22.3 GiB (50.7 MiB/s) with 1 file(s) remaining
Completed 45.0 MiB/22.3 GiB (51.0 MiB/s) with 1 file(s) remaining
Completed 45.2 MiB/22.3 GiB (51.2 MiB/s) with 1 file(s) remaining
Completed 24.6 MiB/31.9 MiB (27.9 MiB/s) with 1 file(s) remaining
Completed 45.5 MiB/22.3 GiB (51.4 MiB/s) with 1 file(s) remaining
Completed 45.8 MiB/22.3 GiB (51.7 MiB/s) with 1 file(s) remaining
Completed 46.0 MiB/22.3 GiB (51.9 MiB/s) with 1 file(s) remaining
Completed 46.2 MiB/22.3 GiB (52.2 MiB/s) with 1 file(s) remaining
Completed 46.5 MiB/22.3 GiB (52.4 MiB/s) with 1 file(s) remaining
Completed 46.8 MiB/22.3 GiB (52.6 MiB/s) with 1 file(s) remaining
Completed 47.0 MiB/22.3 GiB (52.9 MiB/s) with 1 file(s) remaining
Completed 47.2 MiB/22.3 GiB (53.1 MiB/s) with 1 file(s) remaining
Completed 47.5 MiB/22.3 GiB (53.4 MiB/s) with 1 file(s) remaining
Completed 47.8 MiB/22.3 GiB (53.6 MiB/s) with 1 file(s) remaining
Completed 48.0 MiB/22.3 GiB (53.8 MiB/s) with 1 file(s) remaining
Completed 48.2 MiB/22.3 GiB (54.0 MiB/s) with 1 file(s) remaining
Completed 48.5 MiB/22.3 GiB (54.2 MiB/s) with 1 file(s) remaining
Completed 48.8 MiB/22.3 GiB (54.2 MiB/s) with 1 file(s) remaining
Completed 49.0 MiB/22.3 GiB (54.5 MiB/s) with 1 file(s) remaining
Completed 49.2 MiB/22.3 GiB (54.7 MiB/s) with 1 file(s) remaining
Completed 49.5 MiB/22.3 GiB (55.0 MiB/s) with 1 file(s) remaining
Completed 49.8 MiB/22.3 GiB (54.9 MiB/s) with 1 file(s) remaining
Completed 24.9 MiB/31.9 MiB (27.4 MiB/s) with 1 file(s) remaining
Completed 50.0 MiB/22.3 GiB (54.7 MiB/s) with 1 file(s) remaining
Completed 50.2 MiB/22.3 GiB (54.9 MiB/s) with 1 file(s) remaining
Completed 50.5 MiB/22.3 GiB (55.1 MiB/s) with 1 file(s) remaining
Completed 50.8 MiB/22.3 GiB (55.3 MiB/s) with 1 file(s) remaining
Completed 51.0 MiB/22.3 GiB (55.6 MiB/s) with 1 file(s) remaining
Completed 51.2 MiB/22.3 GiB (55.8 MiB/s) with 1 file(s) remaining
Completed 51.5 MiB/22.3 GiB (56.0 MiB/s) with 1 file(s) remaining
Completed 51.8 MiB/22.3 GiB (56.2 MiB/s) with 1 file(s) remaining
Completed 52.0 MiB/22.3 GiB (56.5 MiB/s) with 1 file(s) remaining
Completed 52.2 MiB/22.3 GiB (56.7 MiB/s) with 1 file(s) remaining
Completed 52.5 MiB/22.3 GiB (56.9 MiB/s) with 1 file(s) remaining
Completed 52.8 MiB/22.3 GiB (57.1 MiB/s) with 1 file(s) remaining
Completed 53.0 MiB/22.3 GiB (57.4 MiB/s) with 1 file(s) remaining
Completed 53.2 MiB/22.3 GiB (57.6 MiB/s) with 1 file(s) remaining
Completed 53.5 MiB/22.3 GiB (57.8 MiB/s) with 1 file(s) remaining
Completed 25.1 MiB/31.9 MiB (27.2 MiB/s) with 1 file(s) remaining
Completed 53.8 MiB/22.3 GiB (58.0 MiB/s) with 1 file(s) remaining
Completed 54.0 MiB/22.3 GiB (58.3 MiB/s) with 1 file(s) remaining
Completed 54.2 MiB/22.3 GiB (58.5 MiB/s) with 1 file(s) remaining
Completed 54.5 MiB/22.3 GiB (58.7 MiB/s) with 1 file(s) remaining
Completed 54.8 MiB/22.3 GiB (58.9 MiB/s) with 1 file(s) remaining
Completed 55.0 MiB/22.3 GiB (59.1 MiB/s) with 1 file(s) remaining
Completed 55.2 MiB/22.3 GiB (59.3 MiB/s) with 1 file(s) remaining
Completed 55.5 MiB/22.3 GiB (59.4 MiB/s) with 1 file(s) remaining
Completed 55.8 MiB/22.3 GiB (59.7 MiB/s) with 1 file(s) remaining
Completed 56.0 MiB/22.3 GiB (59.8 MiB/s) with 1 file(s) remaining
Completed 56.2 MiB/22.3 GiB (60.1 MiB/s) with 1 file(s) remaining
Completed 56.5 MiB/22.3 GiB (60.3 MiB/s) with 1 file(s) remaining
Completed 56.8 MiB/22.3 GiB (60.5 MiB/s) with 1 file(s) remaining
Completed 57.0 MiB/22.3 GiB (60.7 MiB/s) with 1 file(s) remaining
Completed 57.2 MiB/22.3 GiB (60.9 MiB/s) with 1 file(s) remaining
Completed 57.5 MiB/22.3 GiB (61.1 MiB/s) with 1 file(s) remaining
Completed 57.8 MiB/22.3 GiB (61.3 MiB/s) with 1 file(s) remaining
Completed 25.4 MiB/31.9 MiB (27.0 MiB/s) with 1 file(s) remaining
Completed 58.0 MiB/22.3 GiB (61.5 MiB/s) with 1 file(s) remaining
Completed 58.2 MiB/22.3 GiB (61.7 MiB/s) with 1 file(s) remaining
Completed 58.5 MiB/22.3 GiB (62.0 MiB/s) with 1 file(s) remaining
Completed 58.8 MiB/22.3 GiB (62.2 MiB/s) with 1 file(s) remaining
Completed 59.0 MiB/22.3 GiB (62.4 MiB/s) with 1 file(s) remaining
Completed 59.2 MiB/22.3 GiB (62.6 MiB/s) with 1 file(s) remaining
Completed 59.5 MiB/22.3 GiB (62.8 MiB/s) with 1 file(s) remaining
Completed 59.8 MiB/22.3 GiB (63.1 MiB/s) with 1 file(s) remaining
Completed 60.0 MiB/22.3 GiB (63.3 MiB/s) with 1 file(s) remaining
Completed 60.2 MiB/22.3 GiB (63.5 MiB/s) with 1 file(s) remaining
Completed 60.5 MiB/22.3 GiB (63.8 MiB/s) with 1 file(s) remaining
Completed 60.8 MiB/22.3 GiB (63.9 MiB/s) with 1 file(s) remaining
Completed 61.0 MiB/22.3 GiB (64.2 MiB/s) with 1 file(s) remaining
Completed 61.2 MiB/22.3 GiB (64.4 MiB/s) with 1 file(s) remaining
Completed 61.5 MiB/22.3 GiB (64.6 MiB/s) with 1 file(s) remaining
Completed 61.8 MiB/22.3 GiB (64.8 MiB/s) with 1 file(s) remaining
Completed 62.0 MiB/22.3 GiB (65.1 MiB/s) with 1 file(s) remaining
Completed 62.2 MiB/22.3 GiB (65.3 MiB/s) with 1 file(s) remaining
Completed 62.5 MiB/22.3 GiB (65.5 MiB/s) with 1 file(s) remaining
Completed 62.8 MiB/22.3 GiB (65.7 MiB/s) with 1 file(s) remaining
Completed 63.0 MiB/22.3 GiB (65.9 MiB/s) with 1 file(s) remaining
Completed 63.2 MiB/22.3 GiB (66.1 MiB/s) with 1 file(s) remaining
Completed 63.5 MiB/22.3 GiB (66.3 MiB/s) with 1 file(s) remaining
Completed 63.8 MiB/22.3 GiB (66.5 MiB/s) with 1 file(s) remaining
Completed 64.0 MiB/22.3 GiB (66.8 MiB/s) with 1 file(s) remaining
Completed 64.2 MiB/22.3 GiB (66.9 MiB/s) with 1 file(s) remaining
Completed 25.6 MiB/31.9 MiB (26.6 MiB/s) with 1 file(s) remaining
Completed 64.5 MiB/22.3 GiB (66.8 MiB/s) with 1 file(s) remaining
Completed 64.8 MiB/22.3 GiB (67.0 MiB/s) with 1 file(s) remaining
Completed 65.0 MiB/22.3 GiB (66.9 MiB/s) with 1 file(s) remaining
Completed 65.2 MiB/22.3 GiB (67.1 MiB/s) with 1 file(s) remaining
Completed 65.5 MiB/22.3 GiB (67.3 MiB/s) with 1 file(s) remaining
Completed 65.8 MiB/22.3 GiB (67.5 MiB/s) with 1 file(s) remaining
Completed 66.0 MiB/22.3 GiB (67.8 MiB/s) with 1 file(s) remaining
Completed 66.2 MiB/22.3 GiB (67.9 MiB/s) with 1 file(s) remaining
Completed 66.5 MiB/22.3 GiB (68.1 MiB/s) with 1 file(s) remaining
Completed 66.8 MiB/22.3 GiB (68.2 MiB/s) with 1 file(s) remaining
Completed 67.0 MiB/22.3 GiB (68.4 MiB/s) with 1 file(s) remaining
Completed 67.2 MiB/22.3 GiB (68.5 MiB/s) with 1 file(s) remaining
Completed 67.5 MiB/22.3 GiB (68.7 MiB/s) with 1 file(s) remaining
Completed 67.8 MiB/22.3 GiB (68.8 MiB/s) with 1 file(s) remaining
Completed 68.0 MiB/22.3 GiB (69.0 MiB/s) with 1 file(s) remaining
Completed 68.2 MiB/22.3 GiB (69.2 MiB/s) with 1 file(s) remaining
Completed 68.5 MiB/22.3 GiB (69.5 MiB/s) with 1 file(s) remaining
Completed 68.8 MiB/22.3 GiB (69.5 MiB/s) with 1 file(s) remaining
Completed 69.0 MiB/22.3 GiB (69.7 MiB/s) with 1 file(s) remaining
Completed 69.2 MiB/22.3 GiB (69.9 MiB/s) with 1 file(s) remaining
Completed 69.5 MiB/22.3 GiB (70.1 MiB/s) with 1 file(s) remaining
Completed 69.8 MiB/22.3 GiB (70.3 MiB/s) with 1 file(s) remaining
Completed 70.0 MiB/22.3 GiB (70.5 MiB/s) with 1 file(s) remaining
Completed 70.2 MiB/22.3 GiB (70.5 MiB/s) with 1 file(s) remaining
Completed 70.5 MiB/22.3 GiB (70.8 MiB/s) with 1 file(s) remaining
Completed 70.8 MiB/22.3 GiB (71.0 MiB/s) with 1 file(s) remaining
Completed 71.0 MiB/22.3 GiB (71.1 MiB/s) with 1 file(s) remaining
Completed 71.2 MiB/22.3 GiB (71.3 MiB/s) with 1 file(s) remaining
Completed 71.5 MiB/22.3 GiB (71.6 MiB/s) with 1 file(s) remaining
Completed 71.8 MiB/22.3 GiB (71.7 MiB/s) with 1 file(s) remaining
Completed 72.0 MiB/22.3 GiB (71.9 MiB/s) with 1 file(s) remaining
Completed 72.2 MiB/22.3 GiB (72.1 MiB/s) with 1 file(s) remaining
Completed 25.9 MiB/31.9 MiB (25.8 MiB/s) with 1 file(s) remaining
Completed 72.5 MiB/22.3 GiB (72.2 MiB/s) with 1 file(s) remaining
Completed 72.8 MiB/22.3 GiB (72.2 MiB/s) with 1 file(s) remaining
Completed 73.0 MiB/22.3 GiB (72.5 MiB/s) with 1 file(s) remaining
Completed 73.2 MiB/22.3 GiB (72.7 MiB/s) with 1 file(s) remaining
Completed 73.5 MiB/22.3 GiB (72.9 MiB/s) with 1 file(s) remaining
Completed 73.8 MiB/22.3 GiB (73.1 MiB/s) with 1 file(s) remaining
Completed 74.0 MiB/22.3 GiB (73.2 MiB/s) with 1 file(s) remaining
Completed 74.2 MiB/22.3 GiB (73.3 MiB/s) with 1 file(s) remaining
Completed 74.5 MiB/22.3 GiB (73.4 MiB/s) with 1 file(s) remaining
Completed 74.8 MiB/22.3 GiB (73.6 MiB/s) with 1 file(s) remaining
Completed 75.0 MiB/22.3 GiB (73.8 MiB/s) with 1 file(s) remaining
Completed 75.2 MiB/22.3 GiB (74.0 MiB/s) with 1 file(s) remaining
Completed 75.5 MiB/22.3 GiB (74.1 MiB/s) with 1 file(s) remaining
Completed 75.8 MiB/22.3 GiB (74.3 MiB/s) with 1 file(s) remaining
Completed 76.0 MiB/22.3 GiB (74.5 MiB/s) with 1 file(s) remaining
Completed 76.2 MiB/22.3 GiB (74.6 MiB/s) with 1 file(s) remaining
Completed 26.1 MiB/31.9 MiB (25.0 MiB/s) with 1 file(s) remaining
Completed 76.5 MiB/22.3 GiB (72.8 MiB/s) with 1 file(s) remaining
Completed 76.8 MiB/22.3 GiB (73.0 MiB/s) with 1 file(s) remaining
Completed 77.0 MiB/22.3 GiB (73.1 MiB/s) with 1 file(s) remaining
Completed 77.2 MiB/22.3 GiB (73.3 MiB/s) with 1 file(s) remaining
Completed 77.5 MiB/22.3 GiB (73.5 MiB/s) with 1 file(s) remaining
Completed 77.8 MiB/22.3 GiB (72.1 MiB/s) with 1 file(s) remaining
Completed 26.4 MiB/31.9 MiB (24.5 MiB/s) with 1 file(s) remaining
Completed 26.6 MiB/31.9 MiB (23.7 MiB/s) with 1 file(s) remaining
Completed 78.0 MiB/22.3 GiB (67.7 MiB/s) with 1 file(s) remaining
Completed 78.2 MiB/22.3 GiB (67.7 MiB/s) with 1 file(s) remaining
Completed 78.5 MiB/22.3 GiB (67.8 MiB/s) with 1 file(s) remaining
Completed 78.8 MiB/22.3 GiB (67.9 MiB/s) with 1 file(s) remaining
Completed 79.0 MiB/22.3 GiB (67.8 MiB/s) with 1 file(s) remaining
Completed 79.2 MiB/22.3 GiB (67.4 MiB/s) with 1 file(s) remaining
Completed 79.5 MiB/22.3 GiB (67.5 MiB/s) with 1 file(s) remaining
Completed 79.8 MiB/22.3 GiB (67.7 MiB/s) with 1 file(s) remaining
Completed 80.0 MiB/22.3 GiB (67.9 MiB/s) with 1 file(s) remaining
Completed 26.9 MiB/31.9 MiB (22.8 MiB/s) with 1 file(s) remaining
Completed 80.2 MiB/22.3 GiB (68.0 MiB/s) with 1 file(s) remaining
Completed 80.5 MiB/22.3 GiB (68.2 MiB/s) with 1 file(s) remaining
Completed 80.8 MiB/22.3 GiB (68.4 MiB/s) with 1 file(s) remaining
Completed 27.1 MiB/31.9 MiB (22.1 MiB/s) with 1 file(s) remaining
Completed 81.0 MiB/22.3 GiB (64.4 MiB/s) with 1 file(s) remaining
Completed 81.2 MiB/22.3 GiB (64.6 MiB/s) with 1 file(s) remaining
Completed 81.5 MiB/22.3 GiB (64.3 MiB/s) with 1 file(s) remaining
Completed 81.8 MiB/22.3 GiB (64.5 MiB/s) with 1 file(s) remaining
Completed 27.4 MiB/31.9 MiB (21.5 MiB/s) with 1 file(s) remaining
Completed 82.0 MiB/22.3 GiB (62.8 MiB/s) with 1 file(s) remaining
Completed 27.6 MiB/31.9 MiB (20.9 MiB/s) with 1 file(s) remaining
Completed 27.9 MiB/31.9 MiB (20.4 MiB/s) with 1 file(s) remaining
Completed 28.1 MiB/31.9 MiB (19.9 MiB/s) with 1 file(s) remaining
Completed 28.4 MiB/31.9 MiB (19.5 MiB/s) with 1 file(s) remaining
Completed 82.2 MiB/22.3 GiB (55.0 MiB/s) with 1 file(s) remaining
Completed 28.6 MiB/31.9 MiB (19.1 MiB/s) with 1 file(s) remaining
Completed 82.5 MiB/22.3 GiB (55.1 MiB/s) with 1 file(s) remaining
Completed 82.8 MiB/22.3 GiB (55.1 MiB/s) with 1 file(s) remaining
Completed 83.0 MiB/22.3 GiB (54.8 MiB/s) with 1 file(s) remaining
Completed 83.2 MiB/22.3 GiB (54.9 MiB/s) with 1 file(s) remaining
Completed 83.5 MiB/22.3 GiB (54.9 MiB/s) with 1 file(s) remaining
Completed 83.8 MiB/22.3 GiB (55.0 MiB/s) with 1 file(s) remaining
Completed 84.0 MiB/22.3 GiB (55.2 MiB/s) with 1 file(s) remaining
Completed 84.2 MiB/22.3 GiB (55.3 MiB/s) with 1 file(s) remaining
Completed 84.5 MiB/22.3 GiB (55.5 MiB/s) with 1 file(s) remaining
Completed 84.8 MiB/22.3 GiB (55.6 MiB/s) with 1 file(s) remaining
Completed 85.0 MiB/22.3 GiB (55.4 MiB/s) with 1 file(s) remaining
Completed 85.2 MiB/22.3 GiB (55.4 MiB/s) with 1 file(s) remaining
Completed 85.5 MiB/22.3 GiB (55.5 MiB/s) with 1 file(s) remaining
Completed 28.9 MiB/31.9 MiB (18.8 MiB/s) with 1 file(s) remaining
Completed 85.8 MiB/22.3 GiB (55.6 MiB/s) with 1 file(s) remaining
Completed 86.0 MiB/22.3 GiB (55.7 MiB/s) with 1 file(s) remaining
Completed 86.2 MiB/22.3 GiB (55.7 MiB/s) with 1 file(s) remaining
Completed 86.5 MiB/22.3 GiB (55.9 MiB/s) with 1 file(s) remaining
Completed 86.8 MiB/22.3 GiB (56.0 MiB/s) with 1 file(s) remaining
Completed 87.0 MiB/22.3 GiB (56.1 MiB/s) with 1 file(s) remaining
Completed 87.2 MiB/22.3 GiB (55.8 MiB/s) with 1 file(s) remaining
Completed 87.5 MiB/22.3 GiB (55.9 MiB/s) with 1 file(s) remaining
Completed 87.8 MiB/22.3 GiB (56.0 MiB/s) with 1 file(s) remaining
Completed 88.0 MiB/22.3 GiB (56.1 MiB/s) with 1 file(s) remaining
Completed 88.2 MiB/22.3 GiB (56.2 MiB/s) with 1 file(s) remaining
Completed 88.5 MiB/22.3 GiB (56.3 MiB/s) with 1 file(s) remaining
Completed 88.8 MiB/22.3 GiB (56.5 MiB/s) with 1 file(s) remaining
Completed 29.1 MiB/31.9 MiB (18.5 MiB/s) with 1 file(s) remaining
Completed 89.0 MiB/22.3 GiB (56.3 MiB/s) with 1 file(s) remaining
Completed 89.2 MiB/22.3 GiB (56.4 MiB/s) with 1 file(s) remaining
Completed 89.5 MiB/22.3 GiB (56.2 MiB/s) with 1 file(s) remaining
Completed 89.8 MiB/22.3 GiB (56.


Bash

grep -c "DONE\|error\|fail\|model.safetensors" /home/ubuntu/.claude/projects/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tool-results/b1o8quhc4.txt 2>&1 | head -5; grep "model.safetensors" /home/ubuntu/.claude/projects/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tool-results/b1o8quhc4.txt 2>&1 | tail -3

Check if model.safetensors was downloaded for ckpt-20000


<persisted-output>
Output too large (4.2MB). Full output saved to: /home/ubuntu/.claude/projects/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tool-results/bsy85qxvo.txt

Preview (first 2KB):
2
Completed 140.5 MiB/22.3 GiB (71.8 MiB/s) with 1 file(s) remaining
Completed 140.8 MiB/22.3 GiB (71.9 MiB/s) with 1 file(s) remaining
Completed 141.0 MiB/22.3 GiB (72.0 MiB/s) with 1 file(s) remaining
Completed 141.2 MiB/22.3 GiB (71.6 MiB/s) with 1 file(s) remaining
Completed 141.5 MiB/22.3 GiB (71.7 MiB/s) with 1 file(s) remaining
Completed 141.8 MiB/22.3 GiB (71.8 MiB/s) with 1 file(s) remaining
Completed 142.0 MiB/22.3 GiB (71.8 MiB/s) with 1 file(s) remaining
Completed 142.2 MiB/22.3 GiB (71.9 MiB/s) with 1 file(s) remaining
Completed 142.5 MiB/22.3 GiB (72.0 MiB/s) with 1 file(s) remaining
Completed 142.8 MiB/22.3 GiB (72.1 MiB/s) with 1 file(s) remaining
Completed 143.0 MiB/22.3 GiB (72.2 MiB/s) with 1 file(s) remaining
Completed 143.2 MiB/22.3 GiB (72.2 MiB/s) with 1 file(s) remaining
Completed 143.5 MiB/22.3 GiB (72.3 MiB/s) with 1 file(s) remaining
Completed 143.8 MiB/22.3 GiB (72.4 MiB/s) with 1 file(s) remaining
Completed 144.0 MiB/22.3 GiB (72.5 MiB/s) with 1 file(s) remaining
Completed 144.2 MiB/22.3 GiB (72.6 MiB/s) with 1 file(s) remaining
Completed 144.5 MiB/22.3 GiB (72.5 MiB/s) with 1 file(s) remaining
Completed 144.8 MiB/22.3 GiB (72.6 MiB/s) with 1 file(s) remaining
Completed 145.0 MiB/22.3 GiB (72.7 MiB/s) with 1 file(s) remaining
Completed 145.2 MiB/22.3 GiB (72.3 MiB/s) with 1 file(s) remaining
Completed 145.5 MiB/22.3 GiB (72.3 MiB/s) with 1 file(s) remaining
Completed 145.8 MiB/22.3 GiB (69.4 MiB/s) with 1 file(s) remaining
Completed 146.0 MiB/22.3 GiB (69.1 MiB/s) with 1 file(s) remaining
Completed 146.2 MiB/22.3 GiB (69.2 MiB/s) with 1 file(s) remaining
Completed 146.5 MiB/22.3 GiB (69.2 MiB/s) with 1 file(s) remaining
Completed 146.8 MiB/22.3 GiB (69.3 MiB/s) with 1 file(s) remaining
Completed 147.0 MiB/22.3 GiB (68.2 MiB/s) with 1 file(s) remaining
Completed 147.2 MiB/22.3 GiB (66.8 MiB/s) with 1 file(s) remaining
Completed 147.5 MiB/22.3 GiB (66.9 MiB/s) with 1 file(s) remaining
Completed 147.8 MiB/22.3 G
...
</persisted-output>

2
Completed 140.5 MiB/22.3 GiB (71.8 MiB/s) with 1 file(s) remaining
Completed 140.8 MiB/22.3 GiB (71.9 MiB/s) with 1 file(s) remaining
Completed 141.0 MiB/22.3 GiB (72.0 MiB/s) with 1 file(s) remaining
Completed 141.2 MiB/22.3 GiB (71.6 MiB/s) with 1 file(s) remaining
Completed 141.5 MiB/22.3 GiB (71.7 MiB/s) with 1 file(s) remaining
Completed 141.8 MiB/22.3 GiB (71.8 MiB/s) with 1 file(s) remaining
Completed 142.0 MiB/22.3 GiB (71.8 MiB/s) with 1 file(s) remaining
Completed 142.2 MiB/22.3 GiB (71.9 MiB/s) with 1 file(s) remaining
Completed 142.5 MiB/22.3 GiB (72.0 MiB/s) with 1 file(s) remaining
Completed 142.8 MiB/22.3 GiB (72.1 MiB/s) with 1 file(s) remaining
Completed 143.0 MiB/22.3 GiB (72.2 MiB/s) with 1 file(s) remaining
Completed 143.2 MiB/22.3 GiB (72.2 MiB/s) with 1 file(s) remaining
Completed 143.5 MiB/22.3 GiB (72.3 MiB/s) with 1 file(s) remaining
Completed 143.8 MiB/22.3 GiB (72.4 MiB/s) with 1 file(s) remaining
Completed 144.0 MiB/22.3 GiB (72.5 MiB/s) with 1 file(s) remaining
Completed 144.2 MiB/22.3 GiB (72.6 MiB/s) with 1 file(s) remaining
Completed 144.5 MiB/22.3 GiB (72.5 MiB/s) with 1 file(s) remaining
Completed 144.8 MiB/22.3 GiB (72.6 MiB/s) with 1 file(s) remaining
Completed 145.0 MiB/22.3 GiB (72.7 MiB/s) with 1 file(s) remaining
Completed 145.2 MiB/22.3 GiB (72.3 MiB/s) with 1 file(s) remaining
Completed 145.5 MiB/22.3 GiB (72.3 MiB/s) with 1 file(s) remaining
Completed 145.8 MiB/22.3 GiB (69.4 MiB/s) with 1 file(s) remaining
Completed 146.0 MiB/22.3 GiB (69.1 MiB/s) with 1 file(s) remaining
Completed 146.2 MiB/22.3 GiB (69.2 MiB/s) with 1 file(s) remaining
Completed 146.5 MiB/22.3 GiB (69.2 MiB/s) with 1 file(s) remaining
Completed 146.8 MiB/22.3 GiB (69.3 MiB/s) with 1 file(s) remaining
Completed 147.0 MiB/22.3 GiB (68.2 MiB/s) with 1 file(s) remaining
Completed 147.2 MiB/22.3 GiB (66.8 MiB/s) with 1 file(s) remaining
Completed 147.5 MiB/22.3 GiB (66.9 MiB/s) with 1 file(s) remaining
Completed 147.8 MiB/22.3 GiB (65.6 MiB/s) with 1 file(s) remaining
Completed 148.0 MiB/22.3 GiB (65.7 MiB/s) with 1 file(s) remaining
Completed 148.2 MiB/22.3 GiB (65.7 MiB/s) with 1 file(s) remaining
Completed 148.5 MiB/22.3 GiB (65.8 MiB/s) with 1 file(s) remaining
Completed 148.8 MiB/22.3 GiB (65.9 MiB/s) with 1 file(s) remaining
Completed 149.0 MiB/22.3 GiB (65.9 MiB/s) with 1 file(s) remaining
Completed 149.2 MiB/22.3 GiB (64.9 MiB/s) with 1 file(s) remaining
Completed 149.5 MiB/22.3 GiB (64.9 MiB/s) with 1 file(s) remaining
Completed 149.8 MiB/22.3 GiB (65.0 MiB/s) with 1 file(s) remaining
Completed 150.0 MiB/22.3 GiB (65.1 MiB/s) with 1 file(s) remaining
Completed 150.2 MiB/22.3 GiB (65.2 MiB/s) with 1 file(s) remaining
Completed 150.5 MiB/22.3 GiB (65.3 MiB/s) with 1 file(s) remaining
Completed 150.8 MiB/22.3 GiB (65.3 MiB/s) with 1 file(s) remaining
Completed 151.0 MiB/22.3 GiB (65.4 MiB/s) with 1 file(s) remaining
Completed 151.2 MiB/22.3 GiB (65.5 MiB/s) with 1 file(s) remaining
Completed 151.5 MiB/22.3 GiB (65.6 MiB/s) with 1 file(s) remaining
Completed 151.8 MiB/22.3 GiB (65.6 MiB/s) with 1 file(s) remaining
Completed 152.0 MiB/22.3 GiB (64.7 MiB/s) with 1 file(s) remaining
Completed 152.2 MiB/22.3 GiB (64.8 MiB/s) with 1 file(s) remaining
Completed 152.5 MiB/22.3 GiB (64.9 MiB/s) with 1 file(s) remaining
Completed 152.8 MiB/22.3 GiB (64.9 MiB/s) with 1 file(s) remaining
Completed 153.0 MiB/22.3 GiB (65.0 MiB/s) with 1 file(s) remaining
Completed 153.2 MiB/22.3 GiB (65.1 MiB/s) with 1 file(s) remaining
Completed 153.5 MiB/22.3 GiB (65.2 MiB/s) with 1 file(s) remaining
Completed 153.8 MiB/22.3 GiB (65.3 MiB/s) with 1 file(s) remaining
Completed 154.0 MiB/22.3 GiB (65.3 MiB/s) with 1 file(s) remaining
Completed 154.2 MiB/22.3 GiB (65.4 MiB/s) with 1 file(s) remaining
Completed 154.5 MiB/22.3 GiB (65.5 MiB/s) with 1 file(s) remaining
Completed 154.8 MiB/22.3 GiB (65.6 MiB/s) with 1 file(s) remaining
Completed 155.0 MiB/22.3 GiB (65.7 MiB/s) with 1 file(s) remaining
Completed 155.2 MiB/22.3 GiB (65.8 MiB/s) with 1 file(s) remaining
Completed 155.5 MiB/22.3 GiB (65.9 MiB/s) with 1 file(s) remaining
Completed 155.8 MiB/22.3 GiB (65.9 MiB/s) with 1 file(s) remaining
Completed 156.0 MiB/22.3 GiB (66.0 MiB/s) with 1 file(s) remaining
Completed 156.2 MiB/22.3 GiB (66.1 MiB/s) with 1 file(s) remaining
Completed 156.5 MiB/22.3 GiB (66.2 MiB/s) with 1 file(s) remaining
Completed 156.8 MiB/22.3 GiB (66.2 MiB/s) with 1 file(s) remaining
Completed 157.0 MiB/22.3 GiB (66.3 MiB/s) with 1 file(s) remaining
Completed 157.2 MiB/22.3 GiB (66.2 MiB/s) with 1 file(s) remaining
Completed 157.5 MiB/22.3 GiB (65.7 MiB/s) with 1 file(s) remaining
Completed 157.8 MiB/22.3 GiB (65.7 MiB/s) with 1 file(s) remaining
Completed 158.0 MiB/22.3 GiB (65.8 MiB/s) with 1 file(s) remaining
Completed 158.2 MiB/22.3 GiB (65.9 MiB/s) with 1 file(s) remaining
Completed 158.5 MiB/22.3 GiB (65.9 MiB/s) with 1 file(s) remaining
Completed 158.8 MiB/22.3 GiB (66.0 MiB/s) with 1 file(s) remaining
Completed 159.0 MiB/22.3 GiB (66.0 MiB/s) with 1 file(s) remaining
Completed 159.2 MiB/22.3 GiB (66.1 MiB/s) with 1 file(s) remaining
Completed 159.5 MiB/22.3 GiB (66.1 MiB/s) with 1 file(s) remaining
Completed 159.8 MiB/22.3 GiB (66.2 MiB/s) with 1 file(s) remaining
Completed 160.0 MiB/22.3 GiB (66.2 MiB/s) with 1 file(s) remaining
Completed 160.2 MiB/22.3 GiB (66.3 MiB/s) with 1 file(s) remaining
Completed 160.5 MiB/22.3 GiB (66.3 MiB/s) with 1 file(s) remaining
Completed 160.8 MiB/22.3 GiB (66.4 MiB/s) with 1 file(s) remaining
Completed 161.0 MiB/22.3 GiB (66.2 MiB/s) with 1 file(s) remaining
Completed 161.2 MiB/22.3 GiB (66.3 MiB/s) with 1 file(s) remaining
Completed 161.5 MiB/22.3 GiB (66.4 MiB/s) with 1 file(s) remaining
Completed 161.8 MiB/22.3 GiB (66.2 MiB/s) with 1 file(s) remaining
Completed 162.0 MiB/22.3 GiB (66.3 MiB/s) with 1 file(s) remaining
Completed 162.2 MiB/22.3 GiB (66.0 MiB/s) with 1 file(s) remaining
Completed 162.5 MiB/22.3 GiB (66.1 MiB/s) with 1 file(s) remaining
Completed 162.8 MiB/22.3 GiB (66.1 MiB/s) with 1 file(s) remaining
Completed 163.0 MiB/22.3 GiB (66.2 MiB/s) with 1 file(s) remaining
Completed 163.2 MiB/22.3 GiB (66.2 MiB/s) with 1 file(s) remaining
Completed 163.5 MiB/22.3 GiB (66.3 MiB/s) with 1 file(s) remaining
Completed 163.8 MiB/22.3 GiB (66.3 MiB/s) with 1 file(s) remaining
Completed 164.0 MiB/22.3 GiB (66.4 MiB/s) with 1 file(s) remaining
Completed 164.2 MiB/22.3 GiB (66.5 MiB/s) with 1 file(s) remaining
Completed 164.5 MiB/22.3 GiB (66.4 MiB/s) with 1 file(s) remaining
Completed 164.8 MiB/22.3 GiB (66.4 MiB/s) with 1 file(s) remaining
Completed 165.0 MiB/22.3 GiB (66.5 MiB/s) with 1 file(s) remaining
Completed 165.2 MiB/22.3 GiB (66.5 MiB/s) with 1 file(s) remaining
Completed 165.5 MiB/22.3 GiB (66.6 MiB/s) with 1 file(s) remaining
Completed 165.8 MiB/22.3 GiB (66.6 MiB/s) with 1 file(s) remaining
Completed 166.0 MiB/22.3 GiB (66.7 MiB/s) with 1 file(s) remaining
Completed 166.2 MiB/22.3 GiB (66.8 MiB/s) with 1 file(s) remaining
Completed 166.5 MiB/22.3 GiB (66.8 MiB/s) with 1 file(s) remaining
Completed 166.8 MiB/22.3 GiB (66.8 MiB/s) with 1 file(s) remaining
Completed 167.0 MiB/22.3 GiB (66.9 MiB/s) with 1 file(s) remaining
Completed 167.2 MiB/22.3 GiB (66.9 MiB/s) with 1 file(s) remaining
Completed 167.5 MiB/22.3 GiB (67.0 MiB/s) with 1 file(s) remaining
Completed 167.8 MiB/22.3 GiB (67.1 MiB/s) with 1 file(s) remaining
Completed 168.0 MiB/22.3 GiB (67.1 MiB/s) with 1 file(s) remaining
Completed 168.2 MiB/22.3 GiB (67.2 MiB/s) with 1 file(s) remaining
Completed 168.5 MiB/22.3 GiB (66.8 MiB/s) with 1 file(s) remaining
Completed 168.8 MiB/22.3 GiB (66.9 MiB/s) with 1 file(s) remaining
Completed 169.0 MiB/22.3 GiB (66.9 MiB/s) with 1 file(s) remaining
Completed 169.2 MiB/22.3 GiB (67.0 MiB/s) with 1 file(s) remaining
Completed 169.5 MiB/22.3 GiB (67.0 MiB/s) with 1 file(s) remaining
Completed 169.8 MiB/22.3 GiB (67.1 MiB/s) with 1 file(s) remaining
Completed 170.0 MiB/22.3 GiB (67.2 MiB/s) with 1 file(s) remaining
Completed 170.2 MiB/22.3 GiB (67.2 MiB/s) with 1 file(s) remaining
Completed 170.5 MiB/22.3 GiB (67.3 MiB/s) with 1 file(s) remaining
Completed 170.8 MiB/22.3 GiB (67.4 MiB/s) with 1 file(s) remaining
Completed 171.0 MiB/22.3 GiB (67.5 MiB/s) with 1 file(s) remaining
Completed 171.2 MiB/22.3 GiB (67.6 MiB/s) with 1 file(s) remaining
Completed 171.5 MiB/22.3 GiB (67.6 MiB/s) with 1 file(s) remaining
Completed 171.8 MiB/22.3 GiB (67.7 MiB/s) with 1 file(s) remaining
Completed 172.0 MiB/22.3 GiB (67.7 MiB/s) with 1 file(s) remaining
Completed 172.2 MiB/22.3 GiB (67.8 MiB/s) with 1 file(s) remaining
Completed 172.5 MiB/22.3 GiB (67.7 MiB/s) with 1 file(s) remaining
Completed 172.8 MiB/22.3 GiB (67.8 MiB/s) with 1 file(s) remaining
Completed 173.0 MiB/22.3 GiB (67.9 MiB/s) with 1 file(s) remaining
Completed 173.2 MiB/22.3 GiB (68.0 MiB/s) with 1 file(s) remaining
Completed 173.5 MiB/22.3 GiB (68.0 MiB/s) with 1 file(s) remaining
Completed 173.8 MiB/22.3 GiB (68.1 MiB/s) with 1 file(s) remaining
Completed 174.0 MiB/22.3 GiB (68.1 MiB/s) with 1 file(s) remaining
Completed 174.2 MiB/22.3 GiB (68.2 MiB/s) with 1 file(s) remaining
Completed 174.5 MiB/22.3 GiB (67.8 MiB/s) with 1 file(s) remaining
Completed 174.8 MiB/22.3 GiB (67.7 MiB/s) with 1 file(s) remaining
Completed 175.0 MiB/22.3 GiB (67.6 MiB/s) with 1 file(s) remaining
Completed 175.2 MiB/22.3 GiB (67.6 MiB/s) with 1 file(s) remaining
Completed 175.5 MiB/22.3 GiB (67.6 MiB/s) with 1 file(s) remaining
Completed 175.8 MiB/22.3 GiB (67.7 MiB/s) with 1 file(s) remaining
Completed 176.0 MiB/22.3 GiB (67.6 MiB/s) with 1 file(s) remaining
Completed 176.2 MiB/22.3 GiB (67.6 MiB/s) with 1 file(s) remaining
Completed 176.5 MiB/22.3 GiB (67.0 MiB/s) with 1 file(s) remaining
Completed 176.8 MiB/22.3 GiB (66.8 MiB/s) with 1 file(s) remaining
Completed 177.0 MiB/22.3 GiB (66.2 MiB/s) with 1 file(s) remaining
Completed 177.2 MiB/22.3 GiB (66.3 MiB/s) with 1 file(s) remaining
Completed 177.5 MiB/22.3 GiB (66.3 MiB/s) with 1 file(s) remaining
Completed 177.8 MiB/22.3 GiB (66.3 MiB/s) with 1 file(s) remaining
Completed 178.0 MiB/22.3 GiB (66.1 MiB/s) with 1 file(s) remaining
Completed 178.2 MiB/22.3 GiB (66.2 MiB/s) with 1 file(s) remaining
Completed 178.5 MiB/22.3 GiB (66.1 MiB/s) with 1 file(s) remaining
Completed 178.8 MiB/22.3 GiB (66.1 MiB/s) with 1 file(s) remaining
Completed 179.0 MiB/22.3 GiB (66.1 MiB/s) with 1 file(s) remaining
Completed 179.2 MiB/22.3 GiB (66.1 MiB/s) with 1 file(s) remaining
Completed 179.5 MiB/22.3 GiB (66.1 MiB/s) with 1 file(s) remaining
Completed 179.8 MiB/22.3 GiB (66.1 MiB/s) with 1 file(s) remaining
Completed 180.0 MiB/22.3 GiB (66.2 MiB/s) with 1 file(s) remaining
Completed 180.2 MiB/22.3 GiB (66.3 MiB/s) with 1 file(s) remaining
Completed 180.5 MiB/22.3 GiB (66.2 MiB/s) with 1 file(s) remaining
Completed 180.8 MiB/22.3 GiB (66.2 MiB/s) with 1 file(s) remaining
Completed 181.0 MiB/22.3 GiB (66.1 MiB/s) with 1 file(s) remaining
Completed 181.2 MiB/22.3 GiB (66.2 MiB/s) with 1 file(s) remaining
Completed 181.5 MiB/22.3 GiB (66.2 MiB/s) with 1 file(s) remaining
Completed 181.8 MiB/22.3 GiB (66.1 MiB/s) with 1 file(s) remaining
Completed 182.0 MiB/22.3 GiB (66.2 MiB/s) with 1 file(s) remaining
Completed 182.2 MiB/22.3 GiB (66.3 MiB/s) with 1 file(s) remaining
Completed 182.5 MiB/22.3 GiB (66.4 MiB/s) with 1 file(s) remaining
Completed 182.8 MiB/22.3 GiB (66.3 MiB/s) with 1 file(s) remaining
Completed 183.0 MiB/22.3 GiB (66.4 MiB/s) with 1 file(s) remaining
Completed 183.2 MiB/22.3 GiB (66.5 MiB/s) with 1 file(s) remaining
Completed 183.5 MiB/22.3 GiB (66.5 MiB/s) with 1 file(s) remaining
Completed 183.8 MiB/22.3 GiB (66.6 MiB/s) with 1 file(s) remaining
Completed 184.0 MiB/22.3 GiB (66.6 MiB/s) with 1 file(s) remaining
Completed 184.2 MiB/22.3 GiB (66.7 MiB/s) with 1 file(s) remaining
Completed 184.5 MiB/22.3 GiB (66.8 MiB/s) with 1 file(s) remaining
Completed 184.8 MiB/22.3 GiB (66.8 MiB/s) with 1 file(s) remaining
Completed 185.0 MiB/22.3 GiB (66.8 MiB/s) with 1 file(s) remaining
Completed 185.2 MiB/22.3 GiB (66.8 MiB/s) with 1 file(s) remaining
Completed 185.5 MiB/22.3 GiB (66.7 MiB/s) with 1 file(s) remaining
Completed 185.8 MiB/22.3 GiB (66.7 MiB/s) with 1 file(s) remaining
Completed 186.0 MiB/22.3 GiB (66.7 MiB/s) with 1 file(s) remaining
Completed 186.2 MiB/22.3 GiB (66.6 MiB/s) with 1 file(s) remaining
Completed 186.5 MiB/22.3 GiB (66.6 MiB/s) with 1 file(s) remaining
Completed 186.8 MiB/22.3 GiB (66.6 MiB/s) with 1 file(s) remaining
Completed 187.0 MiB/22.3 GiB (66.7 MiB/s) with 1 file(s) remaining
Completed 187.2 MiB/22.3 GiB (66.8 MiB/s) with 1 file(s) remaining
Completed 187.5 MiB/22.3 GiB (66.8 MiB/s) with 1 file(s) remaining
Completed 187.8 MiB/22.3 GiB (66.9 MiB/s) with 1 file(s) remaining
Completed 188.0 MiB/22.3 GiB (66.9 MiB/s) with 1 file(s) remaining
Completed 188.2 MiB/22.3 GiB (66.9 MiB/s) with 1 file(s) remaining
Completed 188.5 MiB/22.3 GiB (67.0 MiB/s) with 1 file(s) remaining
Completed 188.8 MiB/22.3 GiB (67.0 MiB/s) with 1 file(s) remaining
Completed 189.0 MiB/22.3 GiB (66.7 MiB/s) with 1 file(s) remaining
Completed 189.2 MiB/22.3 GiB (66.3 MiB/s) with 1 file(s) remaining
Completed 189.5 MiB/22.3 GiB (66.3 MiB/s) with 1 file(s) remaining
Completed 189.8 MiB/22.3 GiB (66.4 MiB/s) with 1 file(s) remaining
Completed 190.0 MiB/22.3 GiB (66.4 MiB/s) with 1 file(s) remaining
Completed 190.2 MiB/22.3 GiB (66.5 MiB/s) with 1 file(s) remaining
Completed 190.5 MiB/22.3 GiB (66.5 MiB/s) with 1 file(s) remaining
Completed 190.8 MiB/22.3 GiB (66.6 MiB/s) with 1 file(s) remaining
Completed 191.0 MiB/22.3 GiB (66.7 MiB/s) with 1 file(s) remaining
Completed 191.2 MiB/22.3 GiB (66.7 MiB/s) with 1 file(s) remaining
Completed 191.5 MiB/22.3 GiB (66.8 MiB/s) with 1 file(s) remaining
Completed 191.8 MiB/22.3 GiB (66.2 MiB/s) with 1 file(s) remaining
Completed 192.0 MiB/22.3 GiB (66.1 MiB/s) with 1 file(s) remaining
Completed 192.2 MiB/22.3 GiB (66.1 MiB/s) with 1 file(s) remaining
Completed 192.5 MiB/22.3 GiB (66.2 MiB/s) with 1 file(s) remaining
Completed 192.8 MiB/22.3 GiB (66.2 MiB/s) with 1 file(s) remaining
Completed 193.0 MiB/22.3 GiB (66.3 MiB/s) with 1 file(s) remaining
Completed 193.2 MiB/22.3 GiB (66.4 MiB/s) with 1 file(s) remaining
Completed 193.5 MiB/22.3 GiB (66.4 MiB/s) with 1 file(s) remaining
Completed 193.8 MiB/22.3 GiB (66.5 MiB/s) with 1 file(s) remaining
Completed 194.0 MiB/22.3 GiB (66.6 MiB/s) with 1 file(s) remaining
Completed 194.2 MiB/22.3 GiB (66.6 MiB/s) with 1 file(s) remaining
Completed 194.5 MiB/22.3 GiB (66.7 MiB/s) with 1 file(s) remaining
Completed 194.8 MiB/22.3 GiB (66.8 MiB/s) with 1 file(s) remaining
Completed 195.0 MiB/22.3 GiB (66.9 MiB/s) with 1 file(s) remaining
Completed 195.2 MiB/22.3 GiB (66.9 MiB/s) with 1 file(s) remaining
Completed 195.5 MiB/22.3 GiB (67.0 MiB/s) with 1 file(s) remaining
Completed 195.8 MiB/22.3 GiB (67.1 MiB/s) with 1 file(s) remaining
Completed 196.0 MiB/22.3 GiB (67.1 MiB/s) with 1 file(s) remaining
Completed 196.2 MiB/22.3 GiB (67.2 MiB/s) with 1 file(s) remaining
Completed 196.5 MiB/22.3 GiB (67.2 MiB/s) with 1 file(s) remaining
Completed 196.8 MiB/22.3 GiB (66.9 MiB/s) with 1 file(s) remaining
Completed 197.0 MiB/22.3 GiB (66.9 MiB/s) with 1 file(s) remaining
Completed 197.2 MiB/22.3 GiB (67.0 MiB/s) with 1 file(s) remaining
Completed 197.5 MiB/22.3 GiB (67.0 MiB/s) with 1 file(s) remaining
Completed 197.8 MiB/22.3 GiB (67.0 MiB/s) with 1 file(s) remaining
Completed 198.0 MiB/22.3 GiB (66.9 MiB/s) with 1 file(s) remaining
Completed 198.2 MiB/22.3 GiB (67.0 MiB/s) with 1 file(s) remaining
Completed 198.5 MiB/22.3 GiB (67.0 MiB/s) with 1 file(s) remaining
Completed 198.8 MiB/22.3 GiB (66.9 MiB/s) with 1 file(s) remaining
Completed 199.0 MiB/22.3 GiB (66.7 MiB/s) with 1 file(s) remaining
Completed 199.2 MiB/22.3 GiB (66.6 MiB/s) with 1 file(s) remaining
Completed 199.5 MiB/22.3 GiB (66.6 MiB/s) with 1 file(s) remaining
Completed 199.8 MiB/22.3 GiB (66.7 MiB/s) with 1 file(s) remaining
Completed 200.0 MiB/22.3 GiB (66.1 MiB/s) with 1 file(s) remaining
Completed 200.2 MiB/22.3 GiB (65.9 MiB/s) with 1 file(s) remaining
Completed 200.5 MiB/22.3 GiB (65.9 MiB/s) with 1 file(s) remaining
Completed 200.8 MiB/22.3 GiB (65.9 MiB/s) with 1 file(s) remaining
Completed 201.0 MiB/22.3 GiB (66.0 MiB/s) with 1 file(s) remaining
Completed 201.2 MiB/22.3 GiB (66.0 MiB/s) with 1 file(s) remaining
Completed 201.5 MiB/22.3 GiB (66.1 MiB/s) with 1 file(s) remaining
Completed 201.8 MiB/22.3 GiB (66.1 MiB/s) with 1 file(s) remaining
Completed 202.0 MiB/22.3 GiB (66.2 MiB/s) with 1 file(s) remaining
Completed 202.2 MiB/22.3 GiB (66.2 MiB/s) with 1 file(s) remaining
Completed 202.5 MiB/22.3 GiB (66.3 MiB/s) with 1 file(s) remaining
Completed 202.8 MiB/22.3 GiB (66.3 MiB/s) with 1 file(s) remaining
Completed 203.0 MiB/22.3 GiB (66.4 MiB/s) with 1 file(s) remaining
Completed 203.2 MiB/22.3 GiB (66.4 MiB/s) with 1 file(s) remaining
Completed 203.5 MiB/22.3 GiB (66.5 MiB/s) with 1 file(s) remaining
Completed 203.8 MiB/22.3 GiB (66.6 MiB/s) with 1 file(s) remaining
Completed 204.0 MiB/22.3 GiB (66.6 MiB/s) with 1 file(s) remaining
Completed 204.2 MiB/22.3 GiB (66.7 MiB/s) with 1 file(s) remaining
Completed 204.5 MiB/22.3 GiB (66.2 MiB/s) with 1 file(s) remaining
Completed 204.8 MiB/22.3 GiB (66.3 MiB/s) with 1 file(s) remaining
Completed 205.0 MiB/22.3 GiB (66.3 MiB/s) with 1 file(s) remaining
Completed 205.2 MiB/22.3 GiB (66.4 MiB/s) with 1 file(s) remaining
Completed 205.5 MiB/22.3 GiB (66.4 MiB/s) with 1 file(s) remaining
Completed 205.8 MiB/22.3 GiB (66.5 MiB/s) with 1 file(s) remaining
Completed 206.0 MiB/22.3 GiB (66.5 MiB/s) with 1 file(s) remaining
Completed 206.2 MiB/22.3 GiB (65.8 MiB/s) with 1 file(s) remaining
Completed 206.5 MiB/22.3 GiB (65.8 MiB/s) with 1 file(s) remaining
Completed 206.8 MiB/22.3 GiB (65.8 MiB/s) with 1 file(s) remaining
Completed 207.0 MiB/22.3 GiB (65.9 MiB/s) with 1 file(s) remaining
Completed 207.2 MiB/22.3 GiB (65.9 MiB/s) with 1 file(s) remaining
Completed 207.5 MiB/22.3 GiB (66.0 MiB/s) with 1 file(s) remaining
Completed 207.8 MiB/22.3 GiB (66.0 MiB/s) with 1 file(s) remaining
Completed 208.0 MiB/22.3 GiB (66.1 MiB/s) with 1 file(s) remaining
Completed 208.2 MiB/22.3 GiB (66.1 MiB/s) with 1 file(s) remaining
Completed 208.5 MiB/22.3 GiB (66.2 MiB/s) with 1 file(s) remaining
Completed 208.8 MiB/22.3 GiB (66.2 MiB/s) with 1 file(s) remaining
Completed 209.0 MiB/22.3 GiB (66.3 MiB/s) with 1 file(s) remaining
Completed 209.2 MiB/22.3 GiB (66.2 MiB/s) with 1 file(s) remaining
Completed 209.5 MiB/22.3 GiB (66.3 MiB/s) with 1 file(s) remaining
Completed 209.8 MiB/22.3 GiB (65.9 MiB/s) with 1 file(s) remaining
Completed 210.0 MiB/22.3 GiB (65.2 MiB/s) with 1 file(s) remaining
Completed 210.2 MiB/22.3 GiB (64.9 MiB/s) with 1 file(s) remaining
Completed 210.5 MiB/22.3 GiB (65.0 MiB/s) with 1 file(s) remaining
Completed 210.8 MiB/22.3 GiB (65.0 MiB/s) with 1 file(s) remaining
Completed 211.0 MiB/22.3 GiB (64.8 MiB/s) with 1 file(s) remaining
Completed 211.2 MiB/22.3 GiB (64.9 MiB/s) with 1 file(s) remaining
Completed 211.5 MiB/22.3 GiB (64.9 MiB/s) with 1 file(s) remaining
Completed 211.8 MiB/22.3 GiB (64.9 MiB/s) with 1 file(s) remaining
Completed 212.0 MiB/22.3 GiB (65.0 MiB/s) with 1 file(s) remaining
Completed 212.2 MiB/22.3 GiB (65.0 MiB/s) with 1 file(s) remaining
Completed 212.5 MiB/22.3 GiB (65.1 MiB/s) with 1 file(s) remaining
Completed 212.8 MiB/22.3 GiB (65.1 MiB/s) with 1 file(s) remaining
Completed 213.0 MiB/22.3 GiB (65.1 MiB/s) with 1 file(s) remaining
Completed 213.2 MiB/22.3 GiB (65.1 MiB/s) with 1 file(s) remaining
Completed 213.5 MiB/22.3 GiB (65.2 MiB/s) with 1 file(s) remaining
Completed 213.8 MiB/22.3 GiB (65.2 MiB/s) with 1 file(s) remaining
Completed 214.0 MiB/22.3 GiB (65.2 MiB/s) with 1 file(s) remaining
Completed 214.2 MiB/22.3 GiB (65.3 MiB/s) with 1 file(s) remaining
Completed 214.5 MiB/22.3 GiB (65.3 MiB/s) with 1 file(s) remaining
Completed 214.8 MiB/22.3 GiB (65.4 MiB/s) with 1 file(s) remaining
Completed 215.0 MiB/22.3 GiB (65.4 MiB/s) with 1 file(s) remaining
Completed 215.2 MiB/22.3 GiB (65.5 MiB/s) with 1 file(s) remaining
Completed 215.5 MiB/22.3 GiB (65.4 MiB/s) with 1 file(s) remaining
Completed 215.8 MiB/22.3 GiB (65.5 MiB/s) with 1 file(s) remaining
Completed 216.0 MiB/22.3 GiB (65.5 MiB/s) with 1 file(s) remaining
Completed 216.2 MiB/22.3 GiB (65.5 MiB/s) with 1 file(s) remaining
Completed 216.5 MiB/22.3 GiB (64.8 MiB/s) with 1 file(s) remaining
Completed 216.8 MiB/22.3 GiB (64.8 MiB/s) with 1 file(s) remaining
Completed 217.0 MiB/22.3 GiB (64.8 MiB/s) with 1 file(s) remaining
Completed 217.2 MiB/22.3 GiB (64.9 MiB/s) with 1 file(s) remaining
Completed 217.5 MiB/22.3 GiB (64.9 MiB/s) with 1 file(s) remaining
Completed 217.8 MiB/22.3 GiB (65.0 MiB/s) with 1 file(s) remaining
Completed 218.0 MiB/22.3 GiB (65.0 MiB/s) with 1 file(s) remaining
Completed 218.2 MiB/22.3 GiB (65.1 MiB/s) with 1 file(s) remaining
Completed 218.5 MiB/22.3 GiB (65.2 MiB/s) with 1 file(s) remaining
Completed 218.8 MiB/22.3 GiB (64.9 MiB/s) with 1 file(s) remaining
Completed 219.0 MiB/22.3 GiB (64.9 MiB/s) with 1 file(s) remaining
Completed 219.2 MiB/22.3 GiB (64.8 MiB/s) with 1 file(s) remaining
Completed 219.5 MiB/22.3 GiB (64.5 MiB/s) with 1 file(s) remaining
Completed 219.8 MiB/22.3 GiB (64.6 MiB/s) with 1 file(s) remaining
Completed 220.0 MiB/22.3 GiB (64.6 MiB/s) with 1 file(s) remaining
Completed 220.2 MiB/22.3 GiB (64.7 MiB/s) with 1 file(s) remaining
Completed 220.5 MiB/22.3 GiB (64.7 MiB/s) with 1 file(s) remaining
Completed 220.8 MiB/22.3 GiB (64.8 MiB/s) with 1 file(s) remaining
Completed 221.0 MiB/22.3 GiB (64.7 MiB/s) with 1 file(s) remaining
Completed 221.2 MiB/22.3 GiB (64.7 MiB/s) with 1 file(s) remaining
Completed 221.5 MiB/22.3 GiB (64.8 MiB/s) with 1 file(s) remaining
Completed 221.8 MiB/22.3 GiB (64.8 MiB/s) with 1 file(s) remaining
Completed 222.0 MiB/22.3 GiB (64.9 MiB/s) with 1 file(s) remaining
Completed 222.2 MiB/22.3 GiB (64.9 MiB/s) with 1 file(s) remaining
Completed 222.5 MiB/22.3 GiB (65.0 MiB/s) with 1 file(s) remaining
Completed 222.8 MiB/22.3 GiB (65.0 MiB/s) with 1 file(s) remaining
Completed 223.0 MiB/22.3 GiB (65.1 MiB/s) with 1 file(s) remaining
Completed 223.2 MiB/22.3 GiB (65.1 MiB/s) with 1 file(s) remaining
Completed 223.5 MiB/22.3 GiB (65.1 MiB/s) with 1 file(s) remaining
Completed 223.8 MiB/22.3 GiB (64.7 MiB/s) with 1 file(s) remaining
Completed 224.0 MiB/22.3 GiB (64.7 MiB/s) with 1 file(s) remaining
Completed 224.2 MiB/22.3 GiB (64.8 MiB/s) with 1 file(s) remaining
Completed 224.5 MiB/22.3 GiB (64.9 MiB/s) with 1 file(s) remaining
Completed 224.8 MiB/22.3 GiB (64.9 MiB/s) with 1 file(s) remaining
Completed 225.0 MiB/22.3 GiB (65.0 MiB/s) with 1 file(s) remaining
Completed 225.2 MiB/22.3 GiB (65.0 MiB/s) with 1 file(s) remaining
Completed 225.5 MiB/22.3 GiB (65.0 MiB/s) with 1 file(s) remaining
Completed 225.8 MiB/22.3 GiB (65.0 MiB/s) with 1 file(s) remaining
Completed 226.0 MiB/22.3 GiB (64.9 MiB/s) with 1 file(s) remaining
Completed 226.2 MiB/22.3 GiB (64.9 MiB/s) with 1 file(s) remaining
Completed 226.5 MiB/22.3 GiB (64.9 MiB/s) with 1 file(s) remaining
Completed 226.8 MiB/22.3 GiB (64.9 MiB/s) with 1 file(s) remaining
Completed 227.0 MiB/22.3 GiB (64.9 MiB/s) with 1 file(s) remaining
Completed 227.2 MiB/22.3 GiB (64.9 MiB/s) with 1 file(s) remaining
Completed 227.5 MiB/22.3 GiB (64.8 MiB/s) with 1 file(s) remaining
Completed 227.8 MiB/22.3 GiB (64.8 MiB/s) with 1 file(s) remaining
Completed 228.0 MiB/22.3 GiB (64.7 MiB/s) with 1 file(s) remaining
Completed 228.2 MiB/22.3 GiB (64.7 MiB/s) with 1 file(s) remaining
Completed 228.5 MiB/22.3 GiB (64.7 MiB/s) with 1 file(s) remaining
Completed 228.8 MiB/22.3 GiB (64.7 MiB/s) with 1 file(s) remaining
Completed 229.0 MiB/22.3 GiB (64.8 MiB/s) with 1 file(s) remaining
Completed 229.2 MiB/22.3 GiB (64.8 MiB/s) with 1 file(s) remaining
Completed 229.5 MiB/22.3 GiB (64.8 MiB/s) with 1 file(s) remaining
Completed 229.8 MiB/22.3 GiB (64.9 MiB/s) with 1 file(s) remaining
Completed 230.0 MiB/22.3 GiB (64.9 MiB/s) with 1 file(s) remaining
Completed 230.2 MiB/22.3 GiB (65.0 MiB/s) with 1 file(s) remaining
Completed 230.5 MiB/22.3 GiB (65.0 MiB/s) with 1 file(s) remaining
Completed 230.8 MiB/22.3 GiB (65.1 MiB/s) with 1 file(s) remaining
Completed 231.0 MiB/22.3 GiB (65.1 MiB/s) with 1 file(s) remaining
Completed 231.2 MiB/22.3 GiB (65.2 MiB/s) with 1 file(s) remaining
Completed 231.5 MiB/22.3 GiB (65.1 MiB/s) with 1 file(s) remaining
Completed 231.8 MiB/22.3 GiB (65.2 MiB/s) with 1 file(s) remaining
Completed 232.0 MiB/22.3 GiB (65.2 MiB/s) with 1 file(s) remaining
Completed 232.2 MiB/22.3 GiB (64.5 MiB/s) with 1 file(s) remaining
Completed 232.5 MiB/22.3 GiB (64.6 MiB/s) with 1 file(s) remaining
Completed 232.8 MiB/22.3 GiB (64.6 MiB/s) with 1 file(s) remaining
Completed 233.0 MiB/22.3 GiB (64.5 MiB/s) with 1 file(s) remaining
Completed 233.2 MiB/22.3 GiB (64.5 MiB/s) with 1 file(s) remaining
Completed 233.5 MiB/22.3 GiB (63.9 MiB/s) with 1 file(s) remaining
Completed 233.8 MiB/22.3 GiB (64.0 MiB/s) with 1 file(s) remaining
Completed 234.0 MiB/22.3 GiB (64.0 MiB/s) with 1 file(s) remaining
Completed 234.2 MiB/22.3 GiB (64.1 MiB/s) with 1 file(s) remaining
Completed 234.5 MiB/22.3 GiB (64.1 MiB/s) with 1 file(s) remaining
Completed 234.8 MiB/22.3 GiB (64.2 MiB/s) with 1 file(s) remaining
Completed 235.0 MiB/22.3 GiB (64.2 MiB/s) with 1 file(s) remaining
Completed 235.2 MiB/22.3 GiB (64.2 MiB/s) with 1 file(s) remaining
Completed 235.5 MiB/22.3 GiB (64.2 MiB/s) with 1 file(s) remaining
Completed 235.8 MiB/22.3 GiB (64.3 MiB/s) with 1 file(s) remaining
Completed 236.0 MiB/22.3 GiB (64.0 MiB/s) with 1 file(s) remaining
Completed 236.2 MiB/22.3 GiB (63.8 MiB/s) with 1 file(s) remaining
Completed 236.5 MiB/22.3 GiB (63.8 MiB/s) with 1 file(s) remaining
Completed 236.8 MiB/22.3 GiB (63.9 MiB/s) with 1 file(s) remaining
Completed 237.0 MiB/22.3 GiB (63.9 MiB/s) with 1 file(s) remaining
Completed 237.2 MiB/22.3 GiB (63.9 MiB/s) with 1 file(s) remaining
Completed 237.5 MiB/22.3 GiB (64.0 MiB/s) with 1 file(s) remaining
Completed 237.8 MiB/22.3 GiB (64.0 MiB/s) with 1 file(s) remaining
Completed 238.0 MiB/22.3 GiB (64.1 MiB/s) with 1 file(s) remaining
Completed 238.2 MiB/22.3 GiB (64.1 MiB/s) with 1 file(s) remaining
Completed 238.5 MiB/22.3 GiB (64.2 MiB/s) with 1 file(s) remaining
Completed 238.8 MiB/22.3 GiB (64.2 MiB/s) with 1 file(s) remaining
Completed 239.0 MiB/22.3 GiB (64.3 MiB/s) with 1 file(s) remaining
Completed 239.2 MiB/22.3 GiB (64.3 MiB/s) with 1 file(s) remaining
Completed 239.5 MiB/22.3 GiB (64.4 MiB/s) with 1 file(s) remaining
Completed 239.8 MiB/22.3 GiB (64.4 MiB/s) with 1 file(s) remaining
Completed 240.0 MiB/22.3 GiB (64.5 MiB/s) with 1 file(s) remaining
Completed 240.2 MiB/22.3 GiB (64.5 MiB/s) with 1 file(s) remaining
Completed 240.5 MiB/22.3 GiB (64.5 MiB/s) with 1 file(s) remaining
Completed 240.8 MiB/22.3 GiB (64.5 MiB/s) with 1 file(s) remaining
Completed 241.0 MiB/22.3 GiB (64.5 MiB/s) with 1 file(s) remaining
Completed 241.2 MiB/22.3 GiB (64.5 MiB/s) with 1 file(s) remaining
Completed 241.5 MiB/22.3 GiB (64.5 MiB/s) with 1 file(s) remaining
Completed 241.8 MiB/22.3 GiB (64.5 MiB/s) with 1 file(s) remaining
Completed 242.0 MiB/22.3 GiB (64.4 MiB/s) with 1 file(s) remaining
Completed 242.2 MiB/22.3 GiB (64.4 MiB/s) with 1 file(s) remaining
Completed 242.5 MiB/22.3 GiB (64.4 MiB/s) with 1 file(s) remaining
Completed 242.8 MiB/22.3 GiB (64.5 MiB/s) with 1 file(s) remaining
Completed 243.0 MiB/22.3 GiB (64.5 MiB/s) with 1 file(s) remaining
Completed 243.2 MiB/22.3 GiB (64.5 MiB/s) with 1 file(s) remaining
Completed 243.5 MiB/22.3 GiB (64.6 MiB/s) with 1 file(s) remaining
Completed 243.8 MiB/22.3 GiB (64.6 MiB/s) with 1 file(s) remaining
Completed 244.0 MiB/22.3 GiB (64.6 MiB/s) with 1 file(s) remaining
Completed 244.2 MiB/22.3 GiB (64.7 MiB/s) with 1 file(s) remaining
Completed 244.5 MiB/22.3 GiB (64.7 MiB/s) with 1 file(s) remaining
Completed 244.8 MiB/22.3 GiB (64.7 MiB/s) with 1 file(s) remaining
Completed 245.0 MiB/22.3 GiB (64.8 MiB/s) with 1 file(s) remaining
Completed 245.2 MiB/22.3 GiB (64.7 MiB/s) with 1 file(s) remaining
Completed 245.5 MiB/22.3 GiB (64.7 MiB/s) with 1 file(s) remaining
Completed 245.8 MiB/22.3 GiB (64.7 MiB/s) with 1 file(s) remaining
Completed 246.0 MiB/22.3 GiB (64.7 MiB/s) with 1 file(s) remaining
Completed 246.2 MiB/22.3 GiB (64.3 MiB/s) with 1 file(s) remaining
Completed 246.5 MiB/22.3 GiB (64.3 MiB/s) with 1 file(s) remaining
Completed 246.8 MiB/22.3 GiB (64.2 MiB/s) with 1 file(s) remaining
Completed 247.0 MiB/22.3 GiB (64.3 MiB/s) with 1 file(s) remaining
Completed 247.2 MiB/22.3 GiB (64.3 MiB/s) with 1 file(s) remaining
Completed 247.5 MiB/22.3 GiB (64.3 MiB/s) with 1 file(s) remaining
Completed 247.8 MiB/22.3 GiB (64.3 MiB/s) with 1 file(s) remaining
Completed 248.0 MiB/22.3 GiB (64.3 MiB/s) with 1 file(s) remaining
Completed 248.2 MiB/22.3 GiB (64.4 MiB/s) with 1 file(s) remaining
Completed 248.5 MiB/22.3 GiB (64.3 MiB/s) with 1 file(s) remaining
Completed 248.8 MiB/22.3 GiB (63.5 MiB/s) with 1 file(s) remaining
Completed 249.0 MiB/22.3 GiB (63.5 MiB/s) with 1 file(s) remaining
Completed 249.2 MiB/22.3 GiB (62.7 MiB/s) with 1 file(s) remaining
Completed 249.5 MiB/22.3 GiB (62.8 MiB/s) with 1 file(s) remaining
Completed 249.8 MiB/22.3 GiB (62.8 MiB/s) with 1 file(s) remaining
Completed 250.0 MiB/22.3 GiB (62.9 MiB/s) with 1 file(s) remaining
Completed 250.2 MiB/22.3 GiB (62.7 MiB/s) with 1 file(s) remaining
Completed 250.5 MiB/22.3 GiB (62.7 MiB/s) with 1 file(s) remaining
Completed


The ckpt-20000 download is still running (22.3 GiB model.safetensors). The `du` was misleading because the temp file was counting. Let me check the actual temp file.


ls -lh /home/ubuntu/training/checkpoints/gemma3n-e2b-ckpt-20000/model.safetensors* 2>&1

Check model file status


Error: Exit code 2
ls: cannot access '/home/ubuntu/training/checkpoints/gemma3n-e2b-ckpt-20000/model.safetensors*': No such file or directory


ls -lha /home/ubuntu/training/checkpoints/gemma3n-e2b-ckpt-20000/ 2>&1

List all files including hidden


total 32M
drwxrwxr-x  2 ubuntu ubuntu 4.0K Mar 29 07:40 .
drwxrwxr-x 21 ubuntu ubuntu 4.0K Mar 29 07:32 ..
-rw-rw-r--  1 ubuntu ubuntu  613 Mar 28 06:31 chat_template.jinja
-rw-rw-r--  1 ubuntu ubuntu 4.5K Mar 28 06:31 config.json
-rw-rw-r--  1 ubuntu ubuntu  209 Mar 28 06:31 generation_config.json
-rw-rw-r--  1 ubuntu ubuntu 2.3K Mar 28 06:31 processor_config.json
-rw-rw-r--  1 ubuntu ubuntu 1004 Mar 28 06:31 tokenizer_config.json
-rw-rw-r--  1 ubuntu ubuntu  32M Mar 28 06:31 tokenizer.json


ckpt-20000 model.safetensors download failed. Let me re-download it. ckpt-40000 is still in progress.


source /home/ubuntu/training/.env && AWS_ACCESS_KEY_ID=$R2_ACCESS_KEY_ID AWS_SECRET_ACCESS_KEY=$R2_SECRET_ACCESS_KEY aws s3 cp "s3://ptcheckpoints/gemma3n-E2B-asr/03-28-2026/ckpt-20000/model.safetensors" "/home/ubuntu/training/checkpoints/gemma3n-e2b-ckpt-20000/model.safetensors" --endpoint-url "$R2_ENDPOINT_URL" --region auto 2>&1

Re-download ckpt-20000 model.safetensors


Command running in background with ID: bjyyto5zb. Output is being written to: /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/bjyyto5zb.output


ls -lh /home/ubuntu/training/checkpoints/gemma3n-e2b-ckpt-40000/model.safetensors* 2>&1

Check ckpt-40000 model status


-rw-rw-r-- 1 ubuntu ubuntu 22G Mar 29 07:41 /home/ubuntu/training/checkpoints/gemma3n-e2b-ckpt-40000/model.safetensors.23be8a9f


ls -lh /home/ubuntu/training/checkpoints/gemma3n-e2b-ckpt-40000/model.safetensors* 2>&1

Check ckpt-40000 model status


-rw-rw-r-- 1 ubuntu ubuntu 23G Mar 29 07:42 /home/ubuntu/training/checkpoints/gemma3n-e2b-ckpt-40000/model.safetensors.23be8a9f


source /home/ubuntu/training/.env && AWS_ACCESS_KEY_ID=$R2_ACCESS_KEY_ID AWS_SECRET_ACCESS_KEY=$R2_SECRET_ACCESS_KEY aws s3 ls "s3://ptcheckpoints/gemma3n-E2B-asr/03-28-2026/ckpt-40000/model.safetensors" --endpoint-url "$R2_ENDPOINT_URL" --region auto 2>&1

Check R2 model.safetensors size


2026-03-29 07:23:01 23907545024 model.safetensors


~23.9GB. ckpt-40000 is at 23GB already — almost done.


ls -lh /home/ubuntu/training/checkpoints/gemma3n-e2b-ckpt-40000/model.safetensors* 2>&1

Check ckpt-40000 final


<task-notification>
<task-id>bosyl0g8u</task-id>
<tool-use-id>toolu_01Nqnv52vG1yQ8BWc12WTeBo</tool-use-id>
<output-file>/tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/bosyl0g8u.output</output-file>
<status>completed</status>
<summary>Background command "Download ckpt-40000 model files" completed (exit code 0)</summary>
</task-notification>


ls -lh /home/ubuntu/training/checkpoints/gemma3n-e2b-ckpt-40000/model.safetensors* 2>&1

Check ckpt-40000 final


-rw-rw-r-- 1 ubuntu ubuntu 23G Mar 29 07:23 /home/ubuntu/training/checkpoints/gemma3n-e2b-ckpt-40000/model.safetensors


ckpt-40000 done (23GB, temp suffix gone). Let me check ckpt-20000 re-download progress and check GPU availability to start the first benchmark.


ls -lh /home/ubuntu/training/checkpoints/gemma3n-e2b-ckpt-20000/model.safetensors* 2>&1 && nvidia-smi --query-compute-apps=pid,name,used_gpu_memory --format=csv 2>&1

Check ckpt-20000 download and GPU status


-rw-rw-r-- 1 ubuntu ubuntu 2.7G Mar 29 07:42 /home/ubuntu/training/checkpoints/gemma3n-e2b-ckpt-20000/model.safetensors.cf0310fa
pid, process_name, used_gpu_memory [MiB]


GPU is free. ckpt-40000 is ready — let me start its benchmark now while ckpt-20000 finishes downloading.


cd /home/ubuntu/training && python3 benchmark_gemma3n_asr.py \
    --checkpoint /home/ubuntu/training/checkpoints/gemma3n-e2b-ckpt-40000 \
    --checkpoint-name ckpt-40000 \
    --output-dir /home/ubuntu/training/benchmark_results_gemma3n \
    --batch-size 128 2>&1

Run ckpt-40000 benchmark (BS=128, full GPU)


Command running in background with ID: blh4bku6v. Output is being written to: /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/blh4bku6v.output


tail -5 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/blh4bku6v.output 2>&1

Check benchmark start


(Bash completed with no output)


tail -5 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/blh4bku6v.output 2>&1

Check benchmark progress


Read output


`torch_dtype` is deprecated! Use `dtype` instead!


Loading model. Let me wait.


tail -5 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/blh4bku6v.output 2>&1

Check benchmark progress


Using the latest cached version of the dataset since BayAreaBoys/indic-asr-benchmark-6k couldn't be found on the Hugging Face Hub
WARNING:datasets.load:Using the latest cached version of the dataset since BayAreaBoys/indic-asr-benchmark-6k couldn't be found on the Hugging Face Hub
Found the latest cached dataset configuration 'default' at /home/ubuntu/training/datasets/indic-asr-benchmark-6k/BayAreaBoys___indic-asr-benchmark-6k/default/0.0.0/4d4bc89b7c915a6d80a1efbee3a708d58e688c81 (last modified on Thu Mar 26 01:11:34 2026).
WARNING:datasets.packaged_modules.cache.cache:Found the latest cached dataset configuration 'default' at /home/ubuntu/training/datasets/indic-asr-benchmark-6k/BayAreaBoys___indic-asr-benchmark-6k/default/0.0.0/4d4bc89b7c915a6d80a1efbee3a708d58e688c81 (last modified on Thu Mar 26 01:11:34 2026).
The following generation flags are not valid and may be ignored: ['top_p', 'top_k']. Set `TRANSFORMERS_VERBOSITY=info` for more details.


tail -5 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/blh4bku6v.output 2>&1

Check benchmark progress


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/blh4bku6v.output 2>&1

Check benchmark progress


Running inference (batch_size=128)...
  [128/6000] (2%) processed


ckpt-40000 benchmark running. Let me also check ckpt-20000 download.


ls -lh /home/ubuntu/training/checkpoints/gemma3n-e2b-ckpt-20000/model.safetensors* 2>&1

Check ckpt-20000 download


-rw-rw-r-- 1 ubuntu ubuntu 4.8G Mar 29 07:43 /home/ubuntu/training/checkpoints/gemma3n-e2b-ckpt-20000/model.safetensors.cf0310fa


4.8G of ~24G. Still downloading. Let me monitor both.


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/blh4bku6v.output 2>&1

Check ckpt-40000 benchmark


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/blh4bku6v.output 2>&1

Check ckpt-40000 benchmark


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/blh4bku6v.output 2>&1

Check ckpt-40000 benchmark


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/blh4bku6v.output 2>&1

Check ckpt-40000 benchmark


Running inference (batch_size=128)...
  [128/6000] (2%) processed
  [256/6000] (4%) processed


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/blh4bku6v.output 2>&1

Check ckpt-40000 benchmark


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/blh4bku6v.output 2>&1

Check progress


  [128/6000] (2%) processed
  [256/6000] (4%) processed
  [384/6000] (6%) processed


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/blh4bku6v.output 2>&1

Check progress


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/blh4bku6v.output 2>&1

Check progress


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/blh4bku6v.output 2>&1

Check progress


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/blh4bku6v.output 2>&1

Check progress


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/blh4bku6v.output 2>&1

Check progress


  [256/6000] (4%) processed
  [384/6000] (6%) processed
  [512/6000] (9%) processed


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/blh4bku6v.output 2>&1

Check progress


  [384/6000] (6%) processed
  [512/6000] (9%) processed
  [640/6000] (11%) processed


Good, picking up speed. Let me check less frequently.


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/blh4bku6v.output 2>&1

Check progress


  [512/6000] (9%) processed
  [640/6000] (11%) processed
  [768/6000] (13%) processed


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/blh4bku6v.output 2>&1

Check progress


  [640/6000] (11%) processed
  [768/6000] (13%) processed
  [896/6000] (15%) processed


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/blh4bku6v.output 2>&1

Check progress


  [768/6000] (13%) processed
  [896/6000] (15%) processed
  [1024/6000] (17%) processed


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/blh4bku6v.output 2>&1

Check progress


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/blh4bku6v.output 2>&1

Check progress


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/blh4bku6v.output 2>&1

Check progress


  [896/6000] (15%) processed
  [1024/6000] (17%) processed
  [1152/6000] (19%) processed


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/blh4bku6v.output 2>&1

Check progress


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/blh4bku6v.output 2>&1

Check progress


  [1024/6000] (17%) processed
  [1152/6000] (19%) processed
  [1280/6000] (21%) processed


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/blh4bku6v.output 2>&1

Check progress


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/blh4bku6v.output 2>&1

Check progress


  [1152/6000] (19%) processed
  [1280/6000] (21%) processed
  [1408/6000] (23%) processed


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/blh4bku6v.output 2>&1

Check progress


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/blh4bku6v.output 2>&1

Check progress


  [1280/6000] (21%) processed
  [1408/6000] (23%) processed
  [1536/6000] (26%) processed


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/blh4bku6v.output 2>&1

Check progress


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/blh4bku6v.output 2>&1

Check progress


  [1408/6000] (23%) processed
  [1536/6000] (26%) processed
  [1664/6000] (28%) processed


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/blh4bku6v.output 2>&1

Check progress


  [1536/6000] (26%) processed
  [1664/6000] (28%) processed
  [1792/6000] (30%) processed


30%. Also check ckpt-20000 download.


ls -lh /home/ubuntu/training/checkpoints/gemma3n-e2b-ckpt-20000/model.safetensors* 2>&1

Check ckpt-20000 download


-rw-rw-r-- 1 ubuntu ubuntu 18G Mar 29 07:47 /home/ubuntu/training/checkpoints/gemma3n-e2b-ckpt-20000/model.safetensors.cf0310fa


18G of 24G — about 6G left. Should finish in ~5 minutes.


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/blh4bku6v.output 2>&1

Check ckpt-40000 progress


  [1792/6000] (30%) processed
  [1920/6000] (32%) processed
  [2048/6000] (34%) processed


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/blh4bku6v.output 2>&1

Check progress


  [1920/6000] (32%) processed
  [2048/6000] (34%) processed
  [2176/6000] (36%) processed


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/blh4bku6v.output 2>&1

Check progress


  [2048/6000] (34%) processed
  [2176/6000] (36%) processed
  [2304/6000] (38%) processed


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/blh4bku6v.output 2>&1

Check progress


  [2176/6000] (36%) processed
  [2304/6000] (38%) processed
  [2432/6000] (41%) processed


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/blh4bku6v.output 2>&1

Check progress


  [2304/6000] (38%) processed
  [2432/6000] (41%) processed
  [2560/6000] (43%) processed


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/blh4bku6v.output 2>&1 && ls -lh /home/ubuntu/training/checkpoints/gemma3n-e2b-ckpt-20000/model.safetensors* 2>&1

Check both


  [2304/6000] (38%) processed
  [2432/6000] (41%) processed
  [2560/6000] (43%) processed
-rw-rw-r-- 1 ubuntu ubuntu 21G Mar 29 07:48 /home/ubuntu/training/checkpoints/gemma3n-e2b-ckpt-20000/model.safetensors.cf0310fa


ckpt-20000 at 21G — very close.


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/blh4bku6v.output 2>&1 && ls -lh /home/ubuntu/training/checkpoints/gemma3n-e2b-ckpt-20000/model.safetensors* 2>&1

Check both


  [2432/6000] (41%) processed
  [2560/6000] (43%) processed
  [2688/6000] (45%) processed
-rw-rw-r-- 1 ubuntu ubuntu 21G Mar 29 07:48 /home/ubuntu/training/checkpoints/gemma3n-e2b-ckpt-20000/model.safetensors.cf0310fa


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/blh4bku6v.output 2>&1 && ls -lh /home/ubuntu/training/checkpoints/gemma3n-e2b-ckpt-20000/model.safetensors* 2>&1

Check both


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/blh4bku6v.output 2>&1 && ls /home/ubuntu/training/checkpoints/gemma3n-e2b-ckpt-20000/model.safetensors 2>&1

Check both


Error: Exit code 2
  [2560/6000] (43%) processed
  [2688/6000] (45%) processed
  [2816/6000] (47%) processed
ls: cannot access '/home/ubuntu/training/checkpoints/gemma3n-e2b-ckpt-20000/model.safetensors': No such file or directory


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/blh4bku6v.output 2>&1 && ls -lh /home/ubuntu/training/checkpoints/gemma3n-e2b-ckpt-20000/model.safetensors* 2>&1

Check both


  [2688/6000] (45%) processed
  [2816/6000] (47%) processed
  [2944/6000] (49%) processed
-rw-rw-r-- 1 ubuntu ubuntu 22G Mar 29 07:48 /home/ubuntu/training/checkpoints/gemma3n-e2b-ckpt-20000/model.safetensors.cf0310fa


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/blh4bku6v.output 2>&1

Check ckpt-40000 progress


  [2688/6000] (45%) processed
  [2816/6000] (47%) processed
  [2944/6000] (49%) processed


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/blh4bku6v.output 2>&1

Check progress


  [2816/6000] (47%) processed
  [2944/6000] (49%) processed
  [3072/6000] (51%) processed


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/blh4bku6v.output 2>&1

Check progress


  [2944/6000] (49%) processed
  [3072/6000] (51%) processed
  [3200/6000] (53%) processed


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/blh4bku6v.output 2>&1

Check progress


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/blh4bku6v.output 2>&1

Check progress


  [3072/6000] (51%) processed
  [3200/6000] (53%) processed
  [3328/6000] (55%) processed


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/blh4bku6v.output 2>&1

Check progress


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/blh4bku6v.output 2>&1

Check progress


  [3200/6000] (53%) processed
  [3328/6000] (55%) processed
  [3456/6000] (58%) processed


<task-notification>
<task-id>bjyyto5zb</task-id>
<tool-use-id>toolu_0123bcXw57FtMQRpTeF9t3em</tool-use-id>
<output-file>/tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/bjyyto5zb.output</output-file>
<status>completed</status>
<summary>Background command "Re-download ckpt-20000 model.safetensors" completed (exit code 0)</summary>
</task-notification>


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/blh4bku6v.output 2>&1 && ls /home/ubuntu/training/checkpoints/gemma3n-e2b-ckpt-20000/model.safetensors 2>&1

Check both


  [3328/6000] (55%) processed
  [3456/6000] (58%) processed
  [3584/6000] (60%) processed
/home/ubuntu/training/checkpoints/gemma3n-e2b-ckpt-20000/model.safetensors


ckpt-20000 download complete. ckpt-40000 benchmark at 60%. About 10 more minutes for the benchmark. Let me wait for it to finish, then immediately run ckpt-20000.


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/blh4bku6v.output 2>&1

Check progress


  [3456/6000] (58%) processed
  [3584/6000] (60%) processed
  [3712/6000] (62%) processed


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/blh4bku6v.output 2>&1

Check progress


  [3584/6000] (60%) processed
  [3712/6000] (62%) processed
  [3840/6000] (64%) processed


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/blh4bku6v.output 2>&1

Check progress


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/blh4bku6v.output 2>&1

Check progress


  [3712/6000] (62%) processed
  [3840/6000] (64%) processed
  [3968/6000] (66%) processed


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/blh4bku6v.output 2>&1

Check progress


  [3840/6000] (64%) processed
  [3968/6000] (66%) processed
  [4096/6000] (68%) processed


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/blh4bku6v.output 2>&1

Check progress


  [3968/6000] (66%) processed
  [4096/6000] (68%) processed
  [4224/6000] (70%) processed


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/blh4bku6v.output 2>&1

Check progress


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/blh4bku6v.output 2>&1

Check progress


  [4096/6000] (68%) processed
  [4224/6000] (70%) processed
  [4352/6000] (73%) processed


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/blh4bku6v.output 2>&1

Check progress


  [4224/6000] (70%) processed
  [4352/6000] (73%) processed
  [4480/6000] (75%) processed


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/blh4bku6v.output 2>&1

Check progress


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/blh4bku6v.output 2>&1

Check progress


  [4352/6000] (73%) processed
  [4480/6000] (75%) processed
  [4608/6000] (77%) processed


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/blh4bku6v.output 2>&1

Check progress


  [4480/6000] (75%) processed
  [4608/6000] (77%) processed
  [4736/6000] (79%) processed


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/blh4bku6v.output 2>&1

Check progress


  [4608/6000] (77%) processed
  [4736/6000] (79%) processed
  [4864/6000] (81%) processed


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/blh4bku6v.output 2>&1

Check progress


  [4736/6000] (79%) processed
  [4864/6000] (81%) processed
  [4992/6000] (83%) processed


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/blh4bku6v.output 2>&1

Check progress


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/blh4bku6v.output 2>&1

Check progress


  [4864/6000] (81%) processed
  [4992/6000] (83%) processed
  [5120/6000] (85%) processed


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/blh4bku6v.output 2>&1

Check progress


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/blh4bku6v.output 2>&1

Check progress


  [4992/6000] (83%) processed
  [5120/6000] (85%) processed
  [5248/6000] (87%) processed


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/blh4bku6v.output 2>&1

Check progress


  [5120/6000] (85%) processed
  [5248/6000] (87%) processed
  [5376/6000] (90%) processed


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/blh4bku6v.output 2>&1

Check progress


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/blh4bku6v.output 2>&1

Check progress


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/blh4bku6v.output 2>&1

Check progress


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/blh4bku6v.output 2>&1

Check progress


  [5248/6000] (87%) processed
  [5376/6000] (90%) processed
  [5504/6000] (92%) processed


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/blh4bku6v.output 2>&1

Check progress


  [5376/6000] (90%) processed
  [5504/6000] (92%) processed
  [5632/6000] (94%) processed


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/blh4bku6v.output 2>&1

Check progress


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/blh4bku6v.output 2>&1

Check progress


  [5504/6000] (92%) processed
  [5632/6000] (94%) processed
  [5760/6000] (96%) processed


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/blh4bku6v.output 2>&1

Check progress


  [5632/6000] (94%) processed
  [5760/6000] (96%) processed
  [5888/6000] (98%) processed


tail -20 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/blh4bku6v.output 2>&1

Check for results


  [3456/6000] (58%) processed
  [3584/6000] (60%) processed
  [3712/6000] (62%) processed
  [3840/6000] (64%) processed
  [3968/6000] (66%) processed
  [4096/6000] (68%) processed
  [4224/6000] (70%) processed
  [4352/6000] (73%) processed
  [4480/6000] (75%) processed
  [4608/6000] (77%) processed
  [4736/6000] (79%) processed
  [4864/6000] (81%) processed
  [4992/6000] (83%) processed
  [5120/6000] (85%) processed
  [5248/6000] (87%) processed
  [5376/6000] (90%) processed
  [5504/6000] (92%) processed
  [5632/6000] (94%) processed
  [5760/6000] (96%) processed
  [5888/6000] (98%) processed


<task-notification>
<task-id>blh4bku6v</task-id>
<tool-use-id>toolu_01GdnVoPWDe8dwWMg2SAu5BA</tool-use-id>
<output-file>/tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/blh4bku6v.output</output-file>
<status>completed</status>
<summary>Background command "Run ckpt-40000 benchmark (BS=128, full GPU)" completed (exit code 0)</summary>
</task-notification>


tail -20 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/blh4bku6v.output 2>&1

Check for final results


Language           N     WER%     CER%   WER-n%   CER-n%
-------------- ----- -------- -------- -------- --------
assamese         500    21.67     9.93    18.60     9.88
bengali          500    21.82     5.67    16.83     4.97
english          500    27.34    14.17    17.33    11.44
gujarati         500    19.84     6.77    15.39     5.94
hindi            500    13.34     4.63     8.90     3.63
kannada          500    43.16    23.62    40.26    22.56
malayalam        500    42.17     9.06    37.61     8.30
marathi          500    25.77     8.96    20.96     8.01
odia             500    28.49     9.93    22.79     8.64
punjabi          500    23.18    11.07    17.17     9.10
tamil            500    35.41    12.22    33.12    11.97
telugu           500    27.12     6.11    22.11     5.71
-------------- ----- -------- -------- -------- --------
OVERALL         6000    27.15    10.43    22.35     9.56
MACRO-AVG         12    27.44    10.18    22.59     9.18
=====================================================================================

Sample analysis saved: /home/ubuntu/training/benchmark_results_gemma3n/ckpt-40000/sample_analysis.json


**ckpt-40000 benchmark done!** Massive improvement. Now let me start ckpt-20000 and generate dashboard JSONs for ckpt-40000 in parallel.


cd /home/ubuntu/training && python3 benchmark_gemma3n_asr.py \
    --checkpoint /home/ubuntu/training/checkpoints/gemma3n-e2b-ckpt-20000 \
    --checkpoint-name ckpt-20000 \
    --output-dir /home/ubuntu/training/benchmark_results_gemma3n \
    --batch-size 128 2>&1

Run ckpt-20000 benchmark


Command running in background with ID: bct7cipzt. Output is being written to: /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/bct7cipzt.output


cd /home/ubuntu/training && python3 generate_schema_v1.py --src benchmark_results_gemma3n/ckpt-40000 --ckpt ckpt-40000 2>&1

Generate dashboard JSON for ckpt-40000


============================================================
  ckpt-40000
============================================================
  Loaded 6000 predictions
  Overall: wer_raw=27.01 wer_norm=26.14 wer_nc=26.1 snwer=19.15 mer=10.61 cer_norm=10.21
  Diagnosis: recognition-limited


ckpt-40000 dashboard JSON generated. ckpt-20000 benchmark running. Let me monitor.


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/bct7cipzt.output 2>&1

Check ckpt-20000 benchmark


Found the latest cached dataset configuration 'default' at /home/ubuntu/training/datasets/indic-asr-benchmark-6k/BayAreaBoys___indic-asr-benchmark-6k/default/0.0.0/4d4bc89b7c915a6d80a1efbee3a708d58e688c81 (last modified on Thu Mar 26 01:11:34 2026).
WARNING:datasets.packaged_modules.cache.cache:Found the latest cached dataset configuration 'default' at /home/ubuntu/training/datasets/indic-asr-benchmark-6k/BayAreaBoys___indic-asr-benchmark-6k/default/0.0.0/4d4bc89b7c915a6d80a1efbee3a708d58e688c81 (last modified on Thu Mar 26 01:11:34 2026).
The following generation flags are not valid and may be ignored: ['top_p', 'top_k']. Set `TRANSFORMERS_VERBOSITY=info` for more details.


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/bct7cipzt.output 2>&1

Check progress


Running inference (batch_size=128)...
  [128/6000] (2%) processed


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/bct7cipzt.output 2>&1

Check progress


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/bct7cipzt.output 2>&1

Check progress


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/bct7cipzt.output 2>&1

Check progress


Running inference (batch_size=128)...
  [128/6000] (2%) processed
  [256/6000] (4%) processed


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/bct7cipzt.output 2>&1

Check progress


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/bct7cipzt.output 2>&1

Check progress


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/bct7cipzt.output 2>&1

Check progress


  [128/6000] (2%) processed
  [256/6000] (4%) processed
  [384/6000] (6%) processed


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/bct7cipzt.output 2>&1

Check progress


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/bct7cipzt.output 2>&1

Check progress


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/bct7cipzt.output 2>&1

Check progress


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/bct7cipzt.output 2>&1

Check progress


  [256/6000] (4%) processed
  [384/6000] (6%) processed
  [512/6000] (9%) processed


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/bct7cipzt.output 2>&1

Check progress


  [384/6000] (6%) processed
  [512/6000] (9%) processed
  [640/6000] (11%) processed


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/bct7cipzt.output 2>&1

Check progress


  [512/6000] (9%) processed
  [640/6000] (11%) processed
  [768/6000] (13%) processed


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/bct7cipzt.output 2>&1

Check progress


  [640/6000] (11%) processed
  [768/6000] (13%) processed
  [896/6000] (15%) processed


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/bct7cipzt.output 2>&1

Check progress


  [768/6000] (13%) processed
  [896/6000] (15%) processed
  [1024/6000] (17%) processed


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/bct7cipzt.output 2>&1

Check progress


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/bct7cipzt.output 2>&1

Check progress


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/bct7cipzt.output 2>&1

Check progress


  [896/6000] (15%) processed
  [1024/6000] (17%) processed
  [1152/6000] (19%) processed


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/bct7cipzt.output 2>&1

Check progress


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/bct7cipzt.output 2>&1

Check progress


  [1024/6000] (17%) processed
  [1152/6000] (19%) processed
  [1280/6000] (21%) processed


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/bct7cipzt.output 2>&1

Check progress


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/bct7cipzt.output 2>&1

Check progress


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/bct7cipzt.output 2>&1

Check progress


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/bct7cipzt.output 2>&1

Check progress


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/bct7cipzt.output 2>&1

Check progress


  [1152/6000] (19%) processed
  [1280/6000] (21%) processed
  [1408/6000] (23%) processed


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/bct7cipzt.output 2>&1

Check progress


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/bct7cipzt.output 2>&1

Check progress


  [1280/6000] (21%) processed
  [1408/6000] (23%) processed
  [1536/6000] (26%) processed


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/bct7cipzt.output 2>&1

Check progress


  [1408/6000] (23%) processed
  [1536/6000] (26%) processed
  [1664/6000] (28%) processed


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/bct7cipzt.output 2>&1

Check progress


  [1536/6000] (26%) processed
  [1664/6000] (28%) processed
  [1792/6000] (30%) processed


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/bct7cipzt.output 2>&1

Check progress


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/bct7cipzt.output 2>&1

Check progress


  [1664/6000] (28%) processed
  [1792/6000] (30%) processed
  [1920/6000] (32%) processed


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/bct7cipzt.output 2>&1

Check progress


  [1792/6000] (30%) processed
  [1920/6000] (32%) processed
  [2048/6000] (34%) processed


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/bct7cipzt.output 2>&1

Check progress


  [1920/6000] (32%) processed
  [2048/6000] (34%) processed
  [2176/6000] (36%) processed


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/bct7cipzt.output 2>&1

Check progress


  [2048/6000] (34%) processed
  [2176/6000] (36%) processed
  [2304/6000] (38%) processed


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/bct7cipzt.output 2>&1

Check progress


  [2176/6000] (36%) processed
  [2304/6000] (38%) processed
  [2432/6000] (41%) processed


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/bct7cipzt.output 2>&1

Check progress


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/bct7cipzt.output 2>&1

Check progress


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/bct7cipzt.output 2>&1

Check progress


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/bct7cipzt.output 2>&1

Check progress


  [2304/6000] (38%) processed
  [2432/6000] (41%) processed
  [2560/6000] (43%) processed


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/bct7cipzt.output 2>&1

Check progress


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/bct7cipzt.output 2>&1

Check progress


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/bct7cipzt.output 2>&1

Check progress


  [2432/6000] (41%) processed
  [2560/6000] (43%) processed
  [2688/6000] (45%) processed


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/bct7cipzt.output 2>&1

Check progress


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/bct7cipzt.output 2>&1

Check progress


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/bct7cipzt.output 2>&1

Check progress


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/bct7cipzt.output 2>&1

Check progress


  [2560/6000] (43%) processed
  [2688/6000] (45%) processed
  [2816/6000] (47%) processed


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/bct7cipzt.output 2>&1

Check progress


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/bct7cipzt.output 2>&1

Check progress


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/bct7cipzt.output 2>&1

Check progress


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/bct7cipzt.output 2>&1

Check progress


  [2688/6000] (45%) processed
  [2816/6000] (47%) processed
  [2944/6000] (49%) processed


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/bct7cipzt.output 2>&1

Check progress


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/bct7cipzt.output 2>&1

Check progress


  [2816/6000] (47%) processed
  [2944/6000] (49%) processed
  [3072/6000] (51%) processed


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/bct7cipzt.output 2>&1

Check progress


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/bct7cipzt.output 2>&1

Check progress


  [2944/6000] (49%) processed
  [3072/6000] (51%) processed
  [3200/6000] (53%) processed


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/bct7cipzt.output 2>&1

Check progress


  [3072/6000] (51%) processed
  [3200/6000] (53%) processed
  [3328/6000] (55%) processed


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/bct7cipzt.output 2>&1

Check progress


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/bct7cipzt.output 2>&1

Check progress


  [3200/6000] (53%) processed
  [3328/6000] (55%) processed
  [3456/6000] (58%) processed


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/bct7cipzt.output 2>&1

Check progress


  [3328/6000] (55%) processed
  [3456/6000] (58%) processed
  [3584/6000] (60%) processed


60%. About 10 more minutes.


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/bct7cipzt.output 2>&1

Check progress


  [3456/6000] (58%) processed
  [3584/6000] (60%) processed
  [3712/6000] (62%) processed


tail -3 /tmp/claude-1001/-home-ubuntu-training/5e207b1e-98a7-43a4-9947-9579680cf6bd/tasks/bct7cipzt.output 2>&1

Check progress