     1→Device: cuda
     2→LeWM TTS model: 21.11M parameters
     3→Dataset: 6959 samples loaded
     4→Training: 6959 samples, 27 batches/epoch
     5→Epoch 1/200 | avg_loss=11.9772 pred=1.4899 kl=0.2497 recon=2.3543 spec=3.8524 | 42.9s
     6→  New best model (loss=11.9772)
     7→  [2/200] step 50 | loss=11.6261 pred=1.2251 kl=0.2035 recon=2.3549 spec=3.7941 | grad=3.04 | 35.2s
     8→Epoch 2/200 | avg_loss=11.7592 pred=1.3041 kl=0.2161 recon=2.3541 spec=3.8312 | 41.1s
     9→  New best model (loss=11.7592)
    10→Epoch 3/200 | avg_loss=11.3993 pred=1.0685 kl=0.1916 recon=2.3521 spec=3.7509 | 40.2s
    11→  New best model (loss=11.3993)
    12→  [4/200] step 100 | loss=10.4622 pred=0.8959 kl=0.2493 recon=2.3514 spec=3.2423 | grad=3.24 | 28.7s
    13→Epoch 4/200 | avg_loss=10.6588 pred=0.9144 kl=0.2344 recon=2.3339 spec=3.3843 | 40.8s
    14→  New best model (loss=10.6588)
    15→Epoch 5/200 | avg_loss=9.0671 pred=0.8269 kl=0.3122 recon=2.2122 spec=2.5436 | 40.3s
    16→  New best model (loss=9.0671)
    17→  [6/200] step 150 | loss=7.2285 pred=0.8280 kl=0.2964 recon=1.6269 spec=2.0975 | grad=7.31 | 22.7s
    18→Epoch 6/200 | avg_loss=7.2588 pred=0.7820 kl=0.2954 recon=1.6706 spec=2.0902 | 40.6s
    19→  New best model (loss=7.2588)
    20→Epoch 7/200 | avg_loss=5.9076 pred=0.6632 kl=0.3169 recon=1.3503 spec=1.6955 | 40.6s
    21→  New best model (loss=5.9076)
    22→  [8/200] step 200 | loss=4.9208 pred=0.5572 kl=0.3621 recon=1.1829 spec=1.3314 | grad=4.71 | 16.9s
    23→Epoch 8/200 | avg_loss=4.8213 pred=0.5439 kl=0.3731 recon=1.1619 spec=1.3019 | 40.8s
    24→  New best model (loss=4.8213)
    25→Epoch 9/200 | avg_loss=4.0268 pred=0.4353 kl=0.4836 recon=1.0010 spec=1.0590 | 40.4s
    26→  New best model (loss=4.0268)
    27→  [10/200] step 250 | loss=3.8342 pred=0.3734 kl=0.5448 recon=0.9780 spec=1.0024 | grad=32.58 | 10.9s
    28→Epoch 10/200 | avg_loss=3.7727 pred=0.3652 kl=0.5560 recon=0.9683 spec=0.9797 | 40.8s
    29→  Saved checkpoint: /home/ubuntu/lewm-tts/output_v3c/checkpoints/epoch_0010.pt
    30→  New best model (loss=3.7727)
    31→Epoch 11/200 | avg_loss=3.6309 pred=0.3330 kl=0.5902 recon=0.9486 spec=0.9328 | 40.5s
    32→  New best model (loss=3.6309)
    33→  [12/200] step 300 | loss=3.6239 pred=0.3101 kl=0.6098 recon=0.9584 spec=0.9303 | grad=26.33 | 4.9s
    34→Epoch 12/200 | avg_loss=3.4695 pred=0.2884 kl=0.6412 recon=0.9204 spec=0.8922 | 40.6s
    35→  New best model (loss=3.4695)
    36→  [13/200] step 350 | loss=3.2157 pred=0.2090 kl=0.7828 recon=0.8850 spec=0.8227 | grad=14.26 | 39.2s
    37→Epoch 13/200 | avg_loss=3.3178 pred=0.2341 kl=0.7302 recon=0.8977 spec=0.8574 | 40.8s
    38→  New best model (loss=3.3178)
    39→Epoch 14/200 | avg_loss=3.2315 pred=0.1837 kl=0.8377 recon=0.8898 spec=0.8436 | 40.9s
    40→  New best model (loss=3.2315)
    41→  [15/200] step 400 | loss=3.1591 pred=0.1422 kl=0.9733 recon=0.8933 spec=0.8179 | grad=22.70 | 33.5s
    42→Epoch 15/200 | avg_loss=3.1433 pred=0.1514 kl=0.9374 recon=0.8741 spec=0.8269 | 40.9s
    43→  New best model (loss=3.1433)
    44→Epoch 16/200 | avg_loss=3.0754 pred=0.1303 kl=1.0223 recon=0.8569 spec=0.8182 | 40.5s
    45→  New best model (loss=3.0754)
    46→  [17/200] step 450 | loss=2.9534 pred=0.1163 kl=1.1075 recon=0.8214 spec=0.7931 | grad=13.75 | 27.6s
    47→Epoch 17/200 | avg_loss=2.9850 pred=0.1171 kl=1.0986 recon=0.8310 spec=0.8009 | 40.9s
    48→  New best model (loss=2.9850)
    49→Epoch 18/200 | avg_loss=2.9984 pred=0.1021 kl=1.1966 recon=0.8366 spec=0.8119 | 40.5s
    50→  [19/200] step 500 | loss=2.9102 pred=0.0872 kl=1.2991 recon=0.7968 spec=0.8156 | grad=12.00 | 21.1s
    51→Epoch 19/200 | avg_loss=2.8936 pred=0.0886 kl=1.2971 recon=0.8056 spec=0.7920 | 40.6s
    52→  New best model (loss=2.8936)
    53→Epoch 20/200 | avg_loss=2.8444 pred=0.0758 kl=1.4101 recon=0.7931 spec=0.7837 | 40.7s
    54→  Saved checkpoint: /home/ubuntu/lewm-tts/output_v3c/checkpoints/epoch_0020.pt
    55→  New best model (loss=2.8444)
    56→  [21/200] step 550 | loss=2.7382 pred=0.0664 kl=1.5054 recon=0.7477 spec=0.7792 | grad=7.86 | 15.4s
    57→Epoch 21/200 | avg_loss=2.8097 pred=0.0648 kl=1.5170 recon=0.7812 spec=0.7832 | 40.7s
    58→  New best model (loss=2.8097)
    59→Epoch 22/200 | avg_loss=2.7211 pred=0.0561 kl=1.6157 recon=0.7555 spec=0.7635 | 40.8s
    60→  New best model (loss=2.7211)
    61→  [23/200] step 600 | loss=2.7167 pred=0.0525 kl=1.6758 recon=0.7557 spec=0.7624 | grad=10.03 | 9.4s
    62→Epoch 23/200 | avg_loss=2.6877 pred=0.0490 kl=1.7129 recon=0.7431 spec=0.7620 | 40.4s
    63→  New best model (loss=2.6877)
    64→Epoch 24/200 | avg_loss=2.6451 pred=0.0435 kl=1.8101 recon=0.7303 spec=0.7536 | 40.8s
    65→  New best model (loss=2.6451)
    66→  [25/200] step 650 | loss=2.6094 pred=0.0415 kl=1.8704 recon=0.7259 spec=0.7366 | grad=9.57 | 3.6s
    67→Epoch 25/200 | avg_loss=2.6333 pred=0.0396 kl=1.9033 recon=0.7274 spec=0.7515 | 40.7s
    68→  New best model (loss=2.6333)
    69→  [26/200] step 700 | loss=2.6726 pred=0.0363 kl=2.0206 recon=0.7374 spec=0.7656 | grad=9.21 | 37.5s
    70→Epoch 26/200 | avg_loss=2.6300 pred=0.0374 kl=1.9832 recon=0.7267 spec=0.7511 | 40.5s
    71→  New best model (loss=2.6300)
    72→Epoch 27/200 | avg_loss=2.6164 pred=0.0357 kl=2.0697 recon=0.7212 spec=0.7497 | 40.7s
    73→  New best model (loss=2.6164)
    74→  [28/200] step 750 | loss=2.5593 pred=0.0320 kl=2.1418 recon=0.7061 spec=0.7335 | grad=13.84 | 31.9s
    75→Epoch 28/200 | avg_loss=2.6100 pred=0.0332 kl=2.1325 recon=0.7218 spec=0.7457 | 40.8s
    76→  New best model (loss=2.6100)
    77→Epoch 29/200 | avg_loss=2.5911 pred=0.0311 kl=2.1768 recon=0.7150 spec=0.7429 | 40.5s
    78→  New best model (loss=2.5911)
    79→  [30/200] step 800 | loss=2.5686 pred=0.0305 kl=2.2163 recon=0.7134 spec=0.7299 | grad=10.11 | 25.6s
    80→Epoch 30/200 | avg_loss=2.5796 pred=0.0301 kl=2.2186 recon=0.7101 spec=0.7420 | 40.7s
    81→  Saved checkpoint: /home/ubuntu/lewm-tts/output_v3c/checkpoints/epoch_0030.pt
    82→  New best model (loss=2.5796)
    83→Epoch 31/200 | avg_loss=2.5567 pred=0.0290 kl=2.2608 recon=0.7027 spec=0.7367 | 40.5s
    84→  New best model (loss=2.5567)
    85→  [32/200] step 850 | loss=2.5136 pred=0.0283 kl=2.2881 recon=0.6939 spec=0.7197 | grad=6.13 | 20.0s
    86→Epoch 32/200 | avg_loss=2.5509 pred=0.0285 kl=2.2890 recon=0.7021 spec=0.7334 | 40.8s
    87→  New best model (loss=2.5509)
    88→Epoch 33/200 | avg_loss=2.5990 pred=0.0287 kl=2.3147 recon=0.7185 spec=0.7430 | 40.6s
    89→  [34/200] step 900 | loss=2.6683 pred=0.0284 kl=2.3074 recon=0.7330 spec=0.7698 | grad=14.52 | 13.8s
    90→Epoch 34/200 | avg_loss=2.6028 pred=0.0286 kl=2.3078 recon=0.7215 spec=0.7412 | 40.8s
    91→Epoch 35/200 | avg_loss=2.5449 pred=0.0293 kl=2.3040 recon=0.6980 spec=0.7332 | 40.6s
    92→  New best model (loss=2.5449)
    93→  [36/200] step 950 | loss=2.5300 pred=0.0277 kl=2.3304 recon=0.6851 spec=0.7410 | grad=5.92 | 8.0s
    94→Epoch 36/200 | avg_loss=2.5258 pred=0.0275 kl=2.3351 recon=0.6917 spec=0.7295 | 40.6s
    95→  New best model (loss=2.5258)
    96→Epoch 37/200 | avg_loss=2.5108 pred=0.0268 kl=2.3397 recon=0.6855 spec=0.7278 | 40.4s
    97→  New best model (loss=2.5108)
    98→  [38/200] step 1000 | loss=2.5287 pred=0.0251 kl=2.3660 recon=0.6945 spec=0.7285 | grad=6.34 | 2.0s
    99→Epoch 38/200 | avg_loss=2.5070 pred=0.0257 kl=2.3479 recon=0.6866 spec=0.7241 | 40.3s
   100→  New best model (loss=2.5070)
   101→  [39/200] step 1050 | loss=2.5116 pred=0.0256 kl=2.3389 recon=0.6930 spec=0.7182 | grad=8.98 | 36.3s
   102→Epoch 39/200 | avg_loss=2.5086 pred=0.0256 kl=2.3421 recon=0.6875 spec=0.7236 | 40.8s
   103→Epoch 40/200 | avg_loss=2.4941 pred=0.0256 kl=2.3290 recon=0.6813 spec=0.7219 | 40.8s
   104→  Saved checkpoint: /home/ubuntu/lewm-tts/output_v3c/checkpoints/epoch_0040.pt
   105→  New best model (loss=2.4941)
   106→  [41/200] step 1100 | loss=2.5019 pred=0.0257 kl=2.3210 recon=0.6767 spec=0.7330 | grad=6.34 | 30.0s
   107→Epoch 41/200 | avg_loss=2.4917 pred=0.0259 kl=2.3225 recon=0.6794 spec=0.7225 | 40.6s
   108→  New best model (loss=2.4917)
   109→Epoch 42/200 | avg_loss=2.4907 pred=0.0258 kl=2.3181 recon=0.6807 spec=0.7202 | 41.4s
   110→  New best model (loss=2.4907)
   111→  [43/200] step 1150 | loss=2.4448 pred=0.0256 kl=2.3266 recon=0.6714 spec=0.7022 | grad=3.72 | 24.4s
   112→Epoch 43/200 | avg_loss=2.4890 pred=0.0260 kl=2.3112 recon=0.6805 spec=0.7192 | 40.9s
   113→  New best model (loss=2.4890)
   114→Epoch 44/200 | avg_loss=2.4703 pred=0.0259 kl=2.3046 recon=0.6737 spec=0.7160 | 41.1s
   115→  New best model (loss=2.4703)
   116→  [45/200] step 1200 | loss=2.4462 pred=0.0260 kl=2.3079 recon=0.6681 spec=0.7073 | grad=6.64 | 18.2s
   117→Epoch 45/200 | avg_loss=2.4556 pred=0.0256 kl=2.3063 recon=0.6681 spec=0.7138 | 40.7s
   118→  New best model (loss=2.4556)
   119→Epoch 46/200 | avg_loss=2.4424 pred=0.0255 kl=2.3130 recon=0.6638 spec=0.7108 | 40.7s
   120→  New best model (loss=2.4424)
   121→  [47/200] step 1250 | loss=2.4447 pred=0.0262 kl=2.3129 recon=0.6634 spec=0.7124 | grad=4.21 | 12.8s
   122→Epoch 47/200 | avg_loss=2.4271 pred=0.0265 kl=2.3131 recon=0.6583 spec=0.7072 | 41.5s
   123→  New best model (loss=2.4271)
   124→Epoch 48/200 | avg_loss=2.3963 pred=0.0274 kl=2.3100 recon=0.6461 spec=0.7023 | 40.9s
   125→  New best model (loss=2.3963)
   126→  [49/200] step 1300 | loss=2.3802 pred=0.0273 kl=2.3204 recon=0.6421 spec=0.6971 | grad=7.82 | 6.4s
   127→Epoch 49/200 | avg_loss=2.3919 pred=0.0277 kl=2.3083 recon=0.6449 spec=0.7009 | 40.8s
   128→  New best model (loss=2.3919)
   129→  [50/200] step 1350 | loss=2.3420 pred=0.0264 kl=2.3338 recon=0.6256 spec=0.6940 | grad=4.91 | 40.8s
   130→Epoch 50/200 | avg_loss=2.3553 pred=0.0277 kl=2.3177 recon=0.6307 spec=0.6954 | 40.9s
   131→  Saved checkpoint: /home/ubuntu/lewm-tts/output_v3c/checkpoints/epoch_0050.pt
   132→  New best model (loss=2.3553)
   133→Epoch 51/200 | avg_loss=2.3503 pred=0.0266 kl=2.3342 recon=0.6285 spec=0.6956 | 40.8s
   134→  New best model (loss=2.3503)
   135→  [52/200] step 1400 | loss=2.3157 pred=0.0269 kl=2.3268 recon=0.6164 spec=0.6885 | grad=4.67 | 34.3s
   136→Epoch 52/200 | avg_loss=2.3292 pred=0.0267 kl=2.3327 recon=0.6190 spec=0.6942 | 40.4s
   137→  New best model (loss=2.3292)
   138→Epoch 53/200 | avg_loss=2.3206 pred=0.0264 kl=2.3388 recon=0.6175 spec=0.6906 | 40.6s
   139→  New best model (loss=2.3206)
   140→  [54/200] step 1450 | loss=2.2774 pred=0.0256 kl=2.3475 recon=0.6093 spec=0.6731 | grad=6.15 | 28.8s
   141→Epoch 54/200 | avg_loss=2.3003 pred=0.0258 kl=2.3416 recon=0.6098 spec=0.6877 | 40.7s
   142→  New best model (loss=2.3003)
   143→Epoch 55/200 | avg_loss=2.2905 pred=0.0255 kl=2.3496 recon=0.6073 spec=0.6845 | 40.6s
   144→  New best model (loss=2.2905)
   145→  [56/200] step 1500 | loss=2.3161 pred=0.0254 kl=2.3528 recon=0.6174 spec=0.6882 | grad=7.04 | 23.0s
   146→Epoch 56/200 | avg_loss=2.3315 pred=0.0255 kl=2.3495 recon=0.6230 spec=0.6911 | 41.0s
   147→Epoch 57/200 | avg_loss=2.2962 pred=0.0260 kl=2.3402 recon=0.6096 spec=0.6851 | 40.8s
   148→  [58/200] step 1550 | loss=2.2628 pred=0.0259 kl=2.3417 recon=0.5985 spec=0.6777 | grad=10.75 | 16.8s
   149→Epoch 58/200 | avg_loss=2.2690 pred=0.0254 kl=2.3435 recon=0.5994 spec=0.6809 | 40.7s
   150→  New best model (loss=2.2690)
   151→Epoch 59/200 | avg_loss=2.2864 pred=0.0249 kl=2.3518 recon=0.6068 spec=0.6829 | 41.0s
   152→  [60/200] step 1600 | loss=2.2822 pred=0.0250 kl=2.3495 recon=0.6046 spec=0.6830 | grad=6.96 | 11.0s
   153→Epoch 60/200 | avg_loss=2.2586 pred=0.0249 kl=2.3509 recon=0.5965 spec=0.6782 | 40.7s
   154→  Saved checkpoint: /home/ubuntu/lewm-tts/output_v3c/checkpoints/epoch_0060.pt
   155→  New best model (loss=2.2586)
   156→Epoch 61/200 | avg_loss=2.2728 pred=0.0247 kl=2.3483 recon=0.6014 spec=0.6813 | 40.8s
   157→  [62/200] step 1650 | loss=2.3741 pred=0.0253 kl=2.3433 recon=0.6319 spec=0.7076 | grad=14.34 | 4.8s
   158→Epoch 62/200 | avg_loss=2.2717 pred=0.0245 kl=2.3454 recon=0.6001 spec=0.6823 | 40.5s
   159→  [63/200] step 1700 | loss=2.2464 pred=0.0251 kl=2.3397 recon=0.5924 spec=0.6755 | grad=2.79 | 39.3s
   160→Epoch 63/200 | avg_loss=2.2423 pred=0.0248 kl=2.3390 recon=0.5908 spec=0.6750 | 40.9s
   161→  New best model (loss=2.2423)
   162→Epoch 64/200 | avg_loss=2.2273 pred=0.0242 kl=2.3472 recon=0.5848 spec=0.6734 | 40.8s
   163→  New best model (loss=2.2273)
   164→  [65/200] step 1750 | loss=2.2269 pred=0.0237 kl=2.3583 recon=0.5934 spec=0.6619 | grad=5.92 | 33.1s
   165→Epoch 65/200 | avg_loss=2.2275 pred=0.0233 kl=2.3628 recon=0.5839 spec=0.6752 | 40.5s
   166→Epoch 66/200 | avg_loss=2.2466 pred=0.0232 kl=2.3605 recon=0.5930 spec=0.6759 | 40.7s
   167→  [67/200] step 1800 | loss=2.2083 pred=0.0233 kl=2.3615 recon=0.5825 spec=0.6643 | grad=4.31 | 27.3s
   168→Epoch 67/200 | avg_loss=2.2201 pred=0.0232 kl=2.3607 recon=0.5833 spec=0.6711 | 40.6s
   169→  New best model (loss=2.2201)
   170→Epoch 68/200 | avg_loss=2.2320 pred=0.0230 kl=2.3627 recon=0.5857 spec=0.6760 | 40.3s
   171→  [69/200] step 1850 | loss=2.1959 pred=0.0225 kl=2.3655 recon=0.5727 spec=0.6696 | grad=4.18 | 21.2s
   172→Epoch 69/200 | avg_loss=2.2099 pred=0.0226 kl=2.3653 recon=0.5805 spec=0.6685 | 41.0s
   173→  New best model (loss=2.2099)
   174→Epoch 70/200 | avg_loss=2.1966 pred=0.0221 kl=2.3680 recon=0.5759 spec=0.6661 | 40.7s
   175→  Saved checkpoint: /home/ubuntu/lewm-tts/output_v3c/checkpoints/epoch_0070.pt
   176→  New best model (loss=2.1966)
   177→  [71/200] step 1900 | loss=2.2263 pred=0.0214 kl=2.3772 recon=0.5734 spec=0.6896 | grad=7.18 | 15.2s
   178→Epoch 71/200 | avg_loss=2.2085 pred=0.0219 kl=2.3681 recon=0.5796 spec=0.6692 | 40.6s
   179→Epoch 72/200 | avg_loss=2.1989 pred=0.0224 kl=2.3601 recon=0.5753 spec=0.6682 | 40.5s
   180→  [73/200] step 1950 | loss=2.2031 pred=0.0222 kl=2.3601 recon=0.5749 spec=0.6716 | grad=4.75 | 9.5s
   181→Epoch 73/200 | avg_loss=2.1931 pred=0.0220 kl=2.3632 recon=0.5743 spec=0.6659 | 40.8s
   182→  New best model (loss=2.1931)
   183→Epoch 74/200 | avg_loss=2.2029 pred=0.0218 kl=2.3658 recon=0.5777 spec=0.6681 | 41.1s
   184→  [75/200] step 2000 | loss=2.1799 pred=0.0212 kl=2.3802 recon=0.5697 spec=0.6636 | grad=3.17 | 3.6s
   185→Epoch 75/200 | avg_loss=2.1929 pred=0.0215 kl=2.3763 recon=0.5744 spec=0.6658 | 40.8s
   186→  New best model (loss=2.1929)
   187→  [76/200] step 2050 | loss=2.2045 pred=0.0210 kl=2.3783 recon=0.5762 spec=0.6716 | grad=5.58 | 38.4s
   188→Epoch 76/200 | avg_loss=2.1819 pred=0.0214 kl=2.3794 recon=0.5706 spec=0.6637 | 41.4s
   189→  New best model (loss=2.1819)
   190→Epoch 77/200 | avg_loss=2.1824 pred=0.0212 kl=2.3782 recon=0.5712 spec=0.6633 | 41.1s
   191→  [78/200] step 2100 | loss=2.1417 pred=0.0210 kl=2.3817 recon=0.5640 spec=0.6459 | grad=2.90 | 31.8s
   192→Epoch 78/200 | avg_loss=2.1719 pred=0.0213 kl=2.3791 recon=0.5675 spec=0.6612 | 40.7s
   193→  New best model (loss=2.1719)
   194→Epoch 79/200 | avg_loss=2.1734 pred=0.0212 kl=2.3740 recon=0.5685 spec=0.6609 | 41.1s
   195→  [80/200] step 2150 | loss=2.1445 pred=0.0206 kl=2.3876 recon=0.5638 spec=0.6483 | grad=4.33 | 25.4s
   196→Epoch 80/200 | avg_loss=2.1680 pred=0.0208 kl=2.3801 recon=0.5645 spec=0.6629 | 40.4s
   197→  Saved checkpoint: /home/ubuntu/lewm-tts/output_v3c/checkpoints/epoch_0080.pt
   198→  New best model (loss=2.1680)
   199→Epoch 81/200 | avg_loss=2.1702 pred=0.0204 kl=2.3827 recon=0.5673 spec=0.6609 | 40.8s
   200→  [82/200] step 2200 | loss=2.1877 pred=0.0204 kl=2.3846 recon=0.5756 spec=0.6616 | grad=7.33 | 19.9s
   201→Epoch 82/200 | avg_loss=2.1934 pred=0.0203 kl=2.3808 recon=0.5762 spec=0.6645 | 40.9s
   202→Epoch 83/200 | avg_loss=2.1657 pred=0.0206 kl=2.3762 recon=0.5658 spec=0.6599 | 40.8s
   203→  New best model (loss=2.1657)
   204→  [84/200] step 2250 | loss=2.1685 pred=0.0209 kl=2.3816 recon=0.5681 spec=0.6584 | grad=5.59 | 14.2s
   205→Epoch 84/200 | avg_loss=2.1576 pred=0.0207 kl=2.3773 recon=0.5625 spec=0.6588 | 40.7s
   206→  New best model (loss=2.1576)
   207→Epoch 85/200 | avg_loss=2.1639 pred=0.0205 kl=2.3841 recon=0.5651 spec=0.6597 | 40.5s
   208→  [86/200] step 2300 | loss=2.1748 pred=0.0203 kl=2.3869 recon=0.5637 spec=0.6688 | grad=4.85 | 8.0s
   209→Epoch 86/200 | avg_loss=2.1583 pred=0.0205 kl=2.3799 recon=0.5634 spec=0.6582 | 40.5s
   210→Epoch 87/200 | avg_loss=2.1500 pred=0.0201 kl=2.3894 recon=0.5606 spec=0.6565 | 40.7s
   211→  New best model (loss=2.1500)
   212→  [88/200] step 2350 | loss=2.1683 pred=0.0197 kl=2.4012 recon=0.5673 spec=0.6600 | grad=6.14 | 2.0s
   213→Epoch 88/200 | avg_loss=2.1607 pred=0.0198 kl=2.3925 recon=0.5634 spec=0.6601 | 40.6s
   214→  [89/200] step 2400 | loss=2.1479 pred=0.0201 kl=2.3842 recon=0.5575 spec=0.6593 | grad=4.23 | 36.3s
   215→Epoch 89/200 | avg_loss=2.1456 pred=0.0202 kl=2.3839 recon=0.5593 spec=0.6554 | 40.9s
   216→  New best model (loss=2.1456)
   217→Epoch 90/200 | avg_loss=2.1441 pred=0.0201 kl=2.3830 recon=0.5595 spec=0.6541 | 41.1s
   218→  Saved checkpoint: /home/ubuntu/lewm-tts/output_v3c/checkpoints/epoch_0090.pt
   219→  New best model (loss=2.1441)
   220→  [91/200] step 2450 | loss=2.1513 pred=0.0195 kl=2.3962 recon=0.5631 spec=0.6545 | grad=3.39 | 30.2s
   221→Epoch 91/200 | avg_loss=2.1518 pred=0.0199 kl=2.3881 recon=0.5618 spec=0.6562 | 40.7s
   222→Epoch 92/200 | avg_loss=2.1548 pred=0.0200 kl=2.3849 recon=0.5619 spec=0.6581 | 40.6s
   223→  [93/200] step 2500 | loss=2.1726 pred=0.0203 kl=2.3807 recon=0.5622 spec=0.6695 | grad=3.79 | 24.3s
   224→Epoch 93/200 | avg_loss=2.1408 pred=0.0202 kl=2.3859 recon=0.5581 spec=0.6537 | 40.7s
   225→  New best model (loss=2.1408)
   226→Epoch 94/200 | avg_loss=2.1370 pred=0.0201 kl=2.3889 recon=0.5554 spec=0.6548 | 40.6s
   227→  New best model (loss=2.1370)
   228→  [95/200] step 2550 | loss=2.1540 pred=0.0201 kl=2.3872 recon=0.5557 spec=0.6657 | grad=3.94 | 18.3s
   229→Epoch 95/200 | avg_loss=2.1514 pred=0.0201 kl=2.3882 recon=0.5616 spec=0.6561 | 40.8s
   230→Epoch 96/200 | avg_loss=2.1444 pred=0.0199 kl=2.3963 recon=0.5598 spec=0.6540 | 40.8s
   231→  [97/200] step 2600 | loss=2.1690 pred=0.0197 kl=2.3967 recon=0.5676 spec=0.6601 | grad=10.26 | 12.4s
   232→Epoch 97/200 | avg_loss=2.1384 pred=0.0199 kl=2.3951 recon=0.5571 spec=0.6536 | 40.6s
   233→Epoch 98/200 | avg_loss=2.1370 pred=0.0203 kl=2.3892 recon=0.5565 spec=0.6532 | 41.2s
   234→  New best model (loss=2.1370)
   235→  [99/200] step 2650 | loss=2.1515 pred=0.0200 kl=2.3915 recon=0.5504 spec=0.6712 | grad=3.93 | 6.6s
   236→Epoch 99/200 | avg_loss=2.1327 pred=0.0198 kl=2.3979 recon=0.5542 spec=0.6536 | 41.1s
   237→  New best model (loss=2.1327)
   238→  [100/200] step 2700 | loss=2.0931 pred=0.0195 kl=2.4133 recon=0.5473 spec=0.6366 | grad=1.89 | 40.9s
   239→Epoch 100/200 | avg_loss=2.1194 pred=0.0195 kl=2.4127 recon=0.5508 spec=0.6495 | 41.0s
   240→  Saved checkpoint: /home/ubuntu/lewm-tts/output_v3c/checkpoints/epoch_0100.pt
   241→  New best model (loss=2.1194)
   242→Epoch 101/200 | avg_loss=2.1080 pred=0.0195 kl=2.4126 recon=0.5469 spec=0.6471 | 41.0s
   243→  New best model (loss=2.1080)
   244→  [102/200] step 2750 | loss=2.0846 pred=0.0197 kl=2.3963 recon=0.5433 spec=0.6363 | grad=4.95 | 35.3s
   245→Epoch 102/200 | avg_loss=2.0944 pred=0.0197 kl=2.4024 recon=0.5440 spec=0.6418 | 41.2s
   246→  New best model (loss=2.0944)
   247→Epoch 103/200 | avg_loss=2.0998 pred=0.0196 kl=2.3997 recon=0.5427 spec=0.6472 | 40.5s
   248→  [104/200] step 2800 | loss=2.0818 pred=0.0193 kl=2.4141 recon=0.5474 spec=0.6291 | grad=3.10 | 28.7s
   249→Epoch 104/200 | avg_loss=2.1022 pred=0.0195 kl=2.4072 recon=0.5449 spec=0.6458 | 40.7s
   250→Epoch 105/200 | avg_loss=2.0867 pred=0.0194 kl=2.4118 recon=0.5408 spec=0.6410 | 40.8s
   251→  New best model (loss=2.0867)
   252→  [106/200] step 2850 | loss=2.1014 pred=0.0189 kl=2.4173 recon=0.5482 spec=0.6413 | grad=4.48 | 22.7s
   253→Epoch 106/200 | avg_loss=2.1020 pred=0.0192 kl=2.4132 recon=0.5458 spec=0.6447 | 40.8s
   254→Epoch 107/200 | avg_loss=2.0944 pred=0.0192 kl=2.4109 recon=0.5430 spec=0.6433 | 40.8s
   255→  [108/200] step 2900 | loss=2.0839 pred=0.0196 kl=2.3993 recon=0.5391 spec=0.6413 | grad=1.92 | 17.0s
   256→Epoch 108/200 | avg_loss=2.0772 pred=0.0193 kl=2.4077 recon=0.5370 spec=0.6399 | 40.8s
   257→  New best model (loss=2.0772)
   258→Epoch 109/200 | avg_loss=2.0766 pred=0.0193 kl=2.4100 recon=0.5366 spec=0.6400 | 40.6s
   259→  New best model (loss=2.0766)
   260→  [110/200] step 2950 | loss=2.0721 pred=0.0192 kl=2.4132 recon=0.5361 spec=0.6377 | grad=3.52 | 11.0s
   261→Epoch 110/200 | avg_loss=2.0798 pred=0.0191 kl=2.4096 recon=0.5381 spec=0.6403 | 40.7s
   262→  Saved checkpoint: /home/ubuntu/lewm-tts/output_v3c/checkpoints/epoch_0110.pt
   263→Epoch 111/200 | avg_loss=2.0729 pred=0.0191 kl=2.4085 recon=0.5355 spec=0.6392 | 40.6s
   264→  New best model (loss=2.0729)
   265→  [112/200] step 3000 | loss=2.1085 pred=0.0196 kl=2.4101 recon=0.5382 spec=0.6589 | grad=2.72 | 4.9s
   266→Epoch 112/200 | avg_loss=2.0696 pred=0.0188 kl=2.4096 recon=0.5343 spec=0.6388 | 40.6s
   267→  New best model (loss=2.0696)
   268→  [113/200] step 3050 | loss=2.0915 pred=0.0185 kl=2.4134 recon=0.5299 spec=0.6594 | grad=2.52 | 39.2s
   269→Epoch 113/200 | avg_loss=2.0668 pred=0.0186 kl=2.4093 recon=0.5323 spec=0.6396 | 40.6s
   270→  New best model (loss=2.0668)
   271→Epoch 114/200 | avg_loss=2.0673 pred=0.0185 kl=2.4170 recon=0.5319 spec=0.6405 | 40.5s
   272→  [115/200] step 3100 | loss=2.0705 pred=0.0189 kl=2.4155 recon=0.5384 spec=0.6337 | grad=6.95 | 33.1s
   273→Epoch 115/200 | avg_loss=2.0717 pred=0.0186 kl=2.4176 recon=0.5347 spec=0.6398 | 40.6s
   274→Epoch 116/200 | avg_loss=2.0661 pred=0.0187 kl=2.4103 recon=0.5325 spec=0.6389 | 40.5s
   275→  New best model (loss=2.0661)
   276→  [117/200] step 3150 | loss=2.0836 pred=0.0187 kl=2.4056 recon=0.5354 spec=0.6467 | grad=1.59 | 27.4s
   277→Epoch 117/200 | avg_loss=2.0594 pred=0.0186 kl=2.4083 recon=0.5308 spec=0.6368 | 40.6s
   278→  New best model (loss=2.0594)
   279→Epoch 118/200 | avg_loss=2.0511 pred=0.0183 kl=2.4134 recon=0.5284 spec=0.6346 | 40.7s
   280→  New best model (loss=2.0511)
   281→  [119/200] step 3200 | loss=2.0833 pred=0.0181 kl=2.4119 recon=0.5289 spec=0.6555 | grad=3.95 | 21.4s
   282→Epoch 119/200 | avg_loss=2.0593 pred=0.0183 kl=2.4101 recon=0.5310 spec=0.6365 | 40.6s
   283→Epoch 120/200 | avg_loss=2.0569 pred=0.0185 kl=2.4065 recon=0.5308 spec=0.6351 | 40.9s
   284→  Saved checkpoint: /home/ubuntu/lewm-tts/output_v3c/checkpoints/epoch_0120.pt
   285→  [121/200] step 3250 | loss=2.0750 pred=0.0185 kl=2.4085 recon=0.5315 spec=0.6463 | grad=2.07 | 15.3s
   286→Epoch 121/200 | avg_loss=2.0463 pred=0.0184 kl=2.4109 recon=0.5264 spec=0.6341 | 40.8s
   287→  New best model (loss=2.0463)
   288→Epoch 122/200 | avg_loss=2.0506 pred=0.0182 kl=2.4147 recon=0.5275 spec=0.6355 | 40.7s
   289→  [123/200] step 3300 | loss=2.0926 pred=0.0180 kl=2.4286 recon=0.5284 spec=0.6624 | grad=2.93 | 9.5s
   290→Epoch 123/200 | avg_loss=2.0510 pred=0.0181 kl=2.4289 recon=0.5268 spec=0.6368 | 40.5s
   291→Epoch 124/200 | avg_loss=2.0472 pred=0.0180 kl=2.4319 recon=0.5261 spec=0.6351 | 40.5s
   292→  [125/200] step 3350 | loss=2.0537 pred=0.0182 kl=2.4239 recon=0.5326 spec=0.6307 | grad=2.51 | 3.6s
   293→Epoch 125/200 | avg_loss=2.0495 pred=0.0179 kl=2.4288 recon=0.5270 spec=0.6354 | 40.5s
   294→  [126/200] step 3400 | loss=2.0151 pred=0.0179 kl=2.4286 recon=0.5211 spec=0.6205 | grad=1.61 | 37.7s
   295→Epoch 126/200 | avg_loss=2.0393 pred=0.0179 kl=2.4295 recon=0.5242 spec=0.6325 | 40.7s
   296→  New best model (loss=2.0393)
   297→Epoch 127/200 | avg_loss=2.0461 pred=0.0180 kl=2.4290 recon=0.5256 spec=0.6352 | 40.3s
   298→  [128/200] step 3450 | loss=2.0332 pred=0.0179 kl=2.4242 recon=0.5251 spec=0.6273 | grad=1.94 | 31.6s
   299→Epoch 128/200 | avg_loss=2.0379 pred=0.0179 kl=2.4275 recon=0.5237 spec=0.6322 | 40.6s
   300→  New best model (loss=2.0379)
   301→Epoch 129/200 | avg_loss=2.0379 pred=0.0180 kl=2.4192 recon=0.5232 spec=0.6329 | 40.8s
   302→  [130/200] step 3500 | loss=2.0166 pred=0.0177 kl=2.4200 recon=0.5200 spec=0.6231 | grad=2.40 | 26.1s
   303→Epoch 130/200 | avg_loss=2.0319 pred=0.0180 kl=2.4172 recon=0.5228 spec=0.6295 | 41.0s
   304→  Saved checkpoint: /home/ubuntu/lewm-tts/output_v3c/checkpoints/epoch_0130.pt
   305→  New best model (loss=2.0319)
   306→Epoch 131/200 | avg_loss=2.0348 pred=0.0179 kl=2.4194 recon=0.5220 spec=0.6324 | 40.5s
   307→  [132/200] step 3550 | loss=2.0231 pred=0.0180 kl=2.4095 recon=0.5227 spec=0.6238 | grad=5.60 | 19.7s
   308→Epoch 132/200 | avg_loss=2.0360 pred=0.0181 kl=2.4107 recon=0.5225 spec=0.6326 | 40.7s
   309→Epoch 133/200 | avg_loss=2.0322 pred=0.0179 kl=2.4092 recon=0.5228 spec=0.6297 | 40.9s
   310→  [134/200] step 3600 | loss=2.0284 pred=0.0179 kl=2.4118 recon=0.5230 spec=0.6270 | grad=2.02 | 14.0s
   311→Epoch 134/200 | avg_loss=2.0282 pred=0.0179 kl=2.4178 recon=0.5205 spec=0.6302 | 40.8s
   312→  New best model (loss=2.0282)
   313→Epoch 135/200 | avg_loss=2.0333 pred=0.0178 kl=2.4273 recon=0.5218 spec=0.6318 | 40.6s
   314→  [136/200] step 3650 | loss=2.0110 pred=0.0177 kl=2.4286 recon=0.5181 spec=0.6219 | grad=3.39 | 8.0s
   315→Epoch 136/200 | avg_loss=2.0266 pred=0.0177 kl=2.4307 recon=0.5195 spec=0.6304 | 40.8s
   316→  New best model (loss=2.0266)
   317→Epoch 137/200 | avg_loss=2.0270 pred=0.0177 kl=2.4306 recon=0.5194 spec=0.6308 | 40.7s
   318→  [138/200] step 3700 | loss=2.0429 pred=0.0177 kl=2.4280 recon=0.5166 spec=0.6451 | grad=3.94 | 2.0s
   319→Epoch 138/200 | avg_loss=2.0252 pred=0.0177 kl=2.4298 recon=0.5192 spec=0.6298 | 40.6s
   320→  New best model (loss=2.0252)
   321→  [139/200] step 3750 | loss=2.0221 pred=0.0178 kl=2.4218 recon=0.5182 spec=0.6292 | grad=1.43 | 36.2s
   322→Epoch 139/200 | avg_loss=2.0215 pred=0.0178 kl=2.4248 recon=0.5175 spec=0.6296 | 40.6s
   323→  New best model (loss=2.0215)
   324→Epoch 140/200 | avg_loss=2.0212 pred=0.0179 kl=2.4269 recon=0.5178 spec=0.6290 | 40.8s
   325→  Saved checkpoint: /home/ubuntu/lewm-tts/output_v3c/checkpoints/epoch_0140.pt
   326→  New best model (loss=2.0212)
   327→  [141/200] step 3800 | loss=2.0163 pred=0.0179 kl=2.4324 recon=0.5182 spec=0.6251 | grad=1.71 | 30.2s
   328→Epoch 141/200 | avg_loss=2.0192 pred=0.0178 kl=2.4313 recon=0.5173 spec=0.6284 | 40.7s
   329→  New best model (loss=2.0192)
   330→Epoch 142/200 | avg_loss=2.0169 pred=0.0179 kl=2.4289 recon=0.5162 spec=0.6282 | 40.7s
   331→  New best model (loss=2.0169)
   332→  [143/200] step 3850 | loss=2.0133 pred=0.0178 kl=2.4366 recon=0.5199 spec=0.6209 | grad=1.36 | 24.1s
   333→Epoch 143/200 | avg_loss=2.0144 pred=0.0178 kl=2.4356 recon=0.5156 spec=0.6273 | 40.6s
   334→  New best model (loss=2.0144)
   335→Epoch 144/200 | avg_loss=2.0149 pred=0.0177 kl=2.4383 recon=0.5152 spec=0.6284 | 40.4s
   336→  [145/200] step 3900 | loss=2.0197 pred=0.0177 kl=2.4395 recon=0.5171 spec=0.6290 | grad=3.49 | 18.2s
   337→Epoch 145/200 | avg_loss=2.0141 pred=0.0177 kl=2.4402 recon=0.5151 spec=0.6280 | 40.6s
   338→  New best model (loss=2.0141)
   339→Epoch 146/200 | avg_loss=2.0110 pred=0.0176 kl=2.4478 recon=0.5139 spec=0.6274 | 40.7s
   340→  New best model (loss=2.0110)
   341→  [147/200] step 3950 | loss=2.0148 pred=0.0180 kl=2.4460 recon=0.5153 spec=0.6278 | grad=1.23 | 12.5s
   342→Epoch 147/200 | avg_loss=2.0125 pred=0.0176 kl=2.4460 recon=0.5143 spec=0.6279 | 40.7s
   343→Epoch 148/200 | avg_loss=2.0090 pred=0.0176 kl=2.4441 recon=0.5137 spec=0.6263 | 40.7s
   344→  New best model (loss=2.0090)
   345→  [149/200] step 4000 | loss=1.9978 pred=0.0176 kl=2.4428 recon=0.5099 spec=0.6240 | grad=3.30 | 6.5s
   346→Epoch 149/200 | avg_loss=2.0047 pred=0.0176 kl=2.4410 recon=0.5126 spec=0.6250 | 40.7s
   347→  New best model (loss=2.0047)
   348→  [150/200] step 4050 | loss=2.0125 pred=0.0178 kl=2.4379 recon=0.5127 spec=0.6299 | grad=1.22 | 40.8s
   349→Epoch 150/200 | avg_loss=2.0024 pred=0.0177 kl=2.4377 recon=0.5120 spec=0.6242 | 40.9s
   350→  Saved checkpoint: /home/ubuntu/lewm-tts/output_v3c/checkpoints/epoch_0150.pt
   351→  New best model (loss=2.0024)
   352→Epoch 151/200 | avg_loss=2.0051 pred=0.0177 kl=2.4353 recon=0.5112 spec=0.6272 | 40.6s
   353→  [152/200] step 4100 | loss=2.0031 pred=0.0178 kl=2.4328 recon=0.5134 spec=0.6228 | grad=4.07 | 34.4s
   354→Epoch 152/200 | avg_loss=2.0020 pred=0.0176 kl=2.4342 recon=0.5107 spec=0.6257 | 40.6s
   355→  New best model (loss=2.0020)
   356→Epoch 153/200 | avg_loss=2.0001 pred=0.0176 kl=2.4368 recon=0.5102 spec=0.6251 | 40.7s
   357→  New best model (loss=2.0001)
   358→  [154/200] step 4150 | loss=1.9898 pred=0.0173 kl=2.4485 recon=0.5088 spec=0.6202 | grad=1.91 | 28.8s
   359→Epoch 154/200 | avg_loss=1.9991 pred=0.0176 kl=2.4422 recon=0.5099 spec=0.6248 | 40.7s
   360→  New best model (loss=1.9991)
   361→Epoch 155/200 | avg_loss=1.9954 pred=0.0176 kl=2.4446 recon=0.5092 spec=0.6234 | 40.8s
   362→  New best model (loss=1.9954)
   363→  [156/200] step 4200 | loss=2.0261 pred=0.0174 kl=2.4505 recon=0.5082 spec=0.6452 | grad=3.33 | 22.7s
   364→Epoch 156/200 | avg_loss=1.9970 pred=0.0175 kl=2.4491 recon=0.5090 spec=0.6247 | 40.5s
   365→Epoch 157/200 | avg_loss=1.9939 pred=0.0175 kl=2.4483 recon=0.5087 spec=0.6229 | 40.7s
   366→  New best model (loss=1.9939)
   367→  [158/200] step 4250 | loss=2.0006 pred=0.0174 kl=2.4530 recon=0.5037 spec=0.6342 | grad=1.74 | 16.7s
   368→Epoch 158/200 | avg_loss=1.9965 pred=0.0175 kl=2.4532 recon=0.5082 spec=0.6254 | 40.5s
   369→Epoch 159/200 | avg_loss=1.9919 pred=0.0174 kl=2.4545 recon=0.5080 spec=0.6226 | 40.6s
   370→  New best model (loss=1.9919)
   371→  [160/200] step 4300 | loss=2.0142 pred=0.0172 kl=2.4594 recon=0.5066 spec=0.6394 | grad=7.58 | 10.6s
   372→Epoch 160/200 | avg_loss=1.9945 pred=0.0173 kl=2.4571 recon=0.5076 spec=0.6250 | 40.4s
   373→  Saved checkpoint: /home/ubuntu/lewm-tts/output_v3c/checkpoints/epoch_0160.pt
   374→Epoch 161/200 | avg_loss=1.9903 pred=0.0173 kl=2.4519 recon=0.5076 spec=0.6222 | 40.8s
   375→  New best model (loss=1.9903)
   376→  [162/200] step 4350 | loss=2.0074 pred=0.0172 kl=2.4526 recon=0.5113 spec=0.6286 | grad=2.33 | 5.0s
   377→Epoch 162/200 | avg_loss=1.9896 pred=0.0173 kl=2.4500 recon=0.5070 spec=0.6224 | 40.8s
   378→  New best model (loss=1.9896)
   379→  [163/200] step 4400 | loss=1.9792 pred=0.0172 kl=2.4542 recon=0.5023 spec=0.6219 | grad=0.99 | 39.2s
   380→Epoch 163/200 | avg_loss=1.9893 pred=0.0173 kl=2.4517 recon=0.5065 spec=0.6230 | 40.7s
   381→  New best model (loss=1.9893)
   382→Epoch 164/200 | avg_loss=1.9887 pred=0.0173 kl=2.4511 recon=0.5063 spec=0.6228 | 40.5s
   383→  New best model (loss=1.9887)
   384→  [165/200] step 4450 | loss=1.9915 pred=0.0175 kl=2.4437 recon=0.5096 spec=0.6202 | grad=1.65 | 33.2s
   385→Epoch 165/200 | avg_loss=1.9914 pred=0.0174 kl=2.4465 recon=0.5065 spec=0.6244 | 40.6s
   386→Epoch 166/200 | avg_loss=1.9860 pred=0.0174 kl=2.4440 recon=0.5061 spec=0.6213 | 40.7s
   387→  New best model (loss=1.9860)
   388→  [167/200] step 4500 | loss=2.0048 pred=0.0173 kl=2.4415 recon=0.5059 spec=0.6342 | grad=1.11 | 27.4s
   389→Epoch 167/200 | avg_loss=1.9864 pred=0.0174 kl=2.4411 recon=0.5057 spec=0.6222 | 40.7s
   390→Epoch 168/200 | avg_loss=1.9844 pred=0.0173 kl=2.4448 recon=0.5056 spec=0.6209 | 40.8s
   391→  New best model (loss=1.9844)
   392→  [169/200] step 4550 | loss=1.9810 pred=0.0174 kl=2.4455 recon=0.5035 spec=0.6214 | grad=1.37 | 21.4s
   393→Epoch 169/200 | avg_loss=1.9870 pred=0.0174 kl=2.4474 recon=0.5053 spec=0.6230 | 40.5s
   394→Epoch 170/200 | avg_loss=1.9853 pred=0.0173 kl=2.4464 recon=0.5050 spec=0.6223 | 40.5s
   395→  Saved checkpoint: /home/ubuntu/lewm-tts/output_v3c/checkpoints/epoch_0170.pt
   396→  [171/200] step 4600 | loss=1.9547 pred=0.0172 kl=2.4498 recon=0.5016 spec=0.6065 | grad=2.12 | 15.4s
   397→Epoch 171/200 | avg_loss=1.9857 pred=0.0173 kl=2.4483 recon=0.5048 spec=0.6229 | 40.7s
   398→Epoch 172/200 | avg_loss=1.9834 pred=0.0173 kl=2.4491 recon=0.5044 spec=0.6218 | 40.7s
   399→  New best model (loss=1.9834)
   400→  [173/200] step 4650 | loss=1.9600 pred=0.0172 kl=2.4502 recon=0.4995 spec=0.6129 | grad=1.70 | 9.3s
   401→Epoch 173/200 | avg_loss=1.9846 pred=0.0173 kl=2.4477 recon=0.5045 spec=0.6225 | 40.6s
   402→Epoch 174/200 | avg_loss=1.9808 pred=0.0172 kl=2.4492 recon=0.5043 spec=0.6203 | 40.9s
   403→  New best model (loss=1.9808)
   404→  [175/200] step 4700 | loss=1.9886 pred=0.0172 kl=2.4495 recon=0.4998 spec=0.6315 | grad=1.89 | 3.5s
   405→Epoch 175/200 | avg_loss=1.9818 pred=0.0173 kl=2.4466 recon=0.5041 spec=0.6213 | 40.7s
   406→  [176/200] step 4750 | loss=1.9816 pred=0.0171 kl=2.4488 recon=0.5050 spec=0.6199 | grad=1.24 | 37.4s
   407→Epoch 176/200 | avg_loss=1.9836 pred=0.0173 kl=2.4470 recon=0.5038 spec=0.6228 | 40.4s
   408→Epoch 177/200 | avg_loss=1.9806 pred=0.0172 kl=2.4478 recon=0.5038 spec=0.6209 | 40.7s
   409→  New best model (loss=1.9806)
   410→  [178/200] step 4800 | loss=1.9574 pred=0.0172 kl=2.4515 recon=0.5023 spec=0.6074 | grad=5.63 | 31.5s
   411→Epoch 178/200 | avg_loss=1.9820 pred=0.0172 kl=2.4502 recon=0.5035 spec=0.6222 | 40.4s
   412→Epoch 179/200 | avg_loss=1.9786 pred=0.0172 kl=2.4505 recon=0.5036 spec=0.6197 | 40.8s
   413→  New best model (loss=1.9786)
   414→  [180/200] step 4850 | loss=1.9658 pred=0.0171 kl=2.4504 recon=0.5016 spec=0.6140 | grad=1.18 | 25.9s
   415→Epoch 180/200 | avg_loss=1.9799 pred=0.0172 kl=2.4502 recon=0.5032 spec=0.6212 | 40.7s
   416→  Saved checkpoint: /home/ubuntu/lewm-tts/output_v3c/checkpoints/epoch_0180.pt
   417→Epoch 181/200 | avg_loss=1.9833 pred=0.0172 kl=2.4504 recon=0.5033 spec=0.6234 | 40.6s
   418→  [182/200] step 4900 | loss=1.9646 pred=0.0171 kl=2.4547 recon=0.5069 spec=0.6062 | grad=1.04 | 20.0s
   419→Epoch 182/200 | avg_loss=1.9778 pred=0.0172 kl=2.4518 recon=0.5033 spec=0.6197 | 40.9s
   420→  New best model (loss=1.9778)
   421→Epoch 183/200 | avg_loss=1.9787 pred=0.0172 kl=2.4520 recon=0.5033 spec=0.6203 | 40.8s
   422→  [184/200] step 4950 | loss=1.9813 pred=0.0172 kl=2.4491 recon=0.5088 spec=0.6146 | grad=7.07 | 14.0s
   423→Epoch 184/200 | avg_loss=1.9772 pred=0.0172 kl=2.4519 recon=0.5030 spec=0.6197 | 40.9s
   424→  New best model (loss=1.9772)
   425→Epoch 185/200 | avg_loss=1.9803 pred=0.0172 kl=2.4521 recon=0.5030 spec=0.6218 | 40.5s
   426→  [186/200] step 5000 | loss=1.9795 pred=0.0171 kl=2.4528 recon=0.5049 spec=0.6187 | grad=1.56 | 7.9s
   427→Epoch 186/200 | avg_loss=1.9789 pred=0.0171 kl=2.4531 recon=0.5029 spec=0.6210 | 40.7s
   428→Epoch 187/200 | avg_loss=1.9771 pred=0.0171 kl=2.4528 recon=0.5028 spec=0.6199 | 40.8s
   429→  New best model (loss=1.9771)
   430→  [188/200] step 5050 | loss=1.9892 pred=0.0170 kl=2.4546 recon=0.5050 spec=0.6252 | grad=0.89 | 2.0s
   431→Epoch 188/200 | avg_loss=1.9775 pred=0.0172 kl=2.4517 recon=0.5027 spec=0.6203 | 40.7s
   432→  [189/200] step 5100 | loss=1.9634 pred=0.0172 kl=2.4512 recon=0.5033 spec=0.6101 | grad=3.41 | 36.1s
   433→Epoch 189/200 | avg_loss=1.9782 pred=0.0172 kl=2.4519 recon=0.5027 spec=0.6207 | 40.5s
   434→Epoch 190/200 | avg_loss=1.9801 pred=0.0171 kl=2.4521 recon=0.5027 spec=0.6219 | 40.7s
   435→  Saved checkpoint: /home/ubuntu/lewm-tts/output_v3c/checkpoints/epoch_0190.pt
   436→  [191/200] step 5150 | loss=1.9591 pred=0.0170 kl=2.4533 recon=0.5012 spec=0.6101 | grad=1.52 | 30.2s
   437→Epoch 191/200 | avg_loss=1.9790 pred=0.0171 kl=2.4524 recon=0.5026 spec=0.6214 | 40.6s
   438→Epoch 192/200 | avg_loss=1.9759 pred=0.0171 kl=2.4522 recon=0.5026 spec=0.6193 | 40.9s
   439→  New best model (loss=1.9759)
   440→  [193/200] step 5200 | loss=1.9515 pred=0.0170 kl=2.4544 recon=0.5019 spec=0.6041 | grad=1.74 | 24.2s
   441→Epoch 193/200 | avg_loss=1.9763 pred=0.0171 kl=2.4524 recon=0.5025 spec=0.6197 | 40.5s
   442→Epoch 194/200 | avg_loss=1.9755 pred=0.0171 kl=2.4521 recon=0.5025 spec=0.6192 | 40.6s
   443→  New best model (loss=1.9755)
   444→  [195/200] step 5250 | loss=1.9719 pred=0.0171 kl=2.4506 recon=0.5019 spec=0.6177 | grad=1.07 | 18.4s
   445→Epoch 195/200 | avg_loss=1.9780 pred=0.0171 kl=2.4521 recon=0.5026 spec=0.6208 | 40.6s
   446→Epoch 196/200 | avg_loss=1.9765 pred=0.0171 kl=2.4522 recon=0.5025 spec=0.6199 | 40.7s
   447→  [197/200] step 5300 | loss=1.9615 pred=0.0171 kl=2.4543 recon=0.5012 spec=0.6117 | grad=0.89 | 12.5s
   448→Epoch 197/200 | avg_loss=1.9785 pred=0.0171 kl=2.4519 recon=0.5026 spec=0.6211 | 40.6s
   449→Epoch 198/200 | avg_loss=1.9734 pred=0.0171 kl=2.4519 recon=0.5026 spec=0.6177 | 40.9s
   450→  New best model (loss=1.9734)
   451→  [199/200] step 5350 | loss=1.9805 pred=0.0172 kl=2.4502 recon=0.5032 spec=0.6216 | grad=1.39 | 6.6s
   452→Epoch 199/200 | avg_loss=1.9756 pred=0.0171 kl=2.4519 recon=0.5025 spec=0.6193 | 40.9s
   453→  [200/200] step 5400 | loss=1.9890 pred=0.0171 kl=2.4549 recon=0.5023 spec=0.6285 | grad=1.26 | 40.6s
   454→Epoch 200/200 | avg_loss=1.9766 pred=0.0171 kl=2.4520 recon=0.5026 spec=0.6199 | 40.6s
   455→  Saved checkpoint: /home/ubuntu/lewm-tts/output_v3c/checkpoints/epoch_0200.pt
   456→
   457→Training complete. Best loss: 1.9734
   458→

Device: cuda
LeWM TTS model: 21.11M parameters
Dataset: 6959 samples loaded
Training: 6959 samples, 27 batches/epoch
Epoch 1/200 | avg_loss=11.9772 pred=1.4899 kl=0.2497 recon=2.3543 spec=3.8524 | 42.9s
  New best model (loss=11.9772)
  [2/200] step 50 | loss=11.6261 pred=1.2251 kl=0.2035 recon=2.3549 spec=3.7941 | grad=3.04 | 35.2s
Epoch 2/200 | avg_loss=11.7592 pred=1.3041 kl=0.2161 recon=2.3541 spec=3.8312 | 41.1s
  New best model (loss=11.7592)
Epoch 3/200 | avg_loss=11.3993 pred=1.0685 kl=0.1916 recon=2.3521 spec=3.7509 | 40.2s
  New best model (loss=11.3993)
  [4/200] step 100 | loss=10.4622 pred=0.8959 kl=0.2493 recon=2.3514 spec=3.2423 | grad=3.24 | 28.7s
Epoch 4/200 | avg_loss=10.6588 pred=0.9144 kl=0.2344 recon=2.3339 spec=3.3843 | 40.8s
  New best model (loss=10.6588)
Epoch 5/200 | avg_loss=9.0671 pred=0.8269 kl=0.3122 recon=2.2122 spec=2.5436 | 40.3s
  New best model (loss=9.0671)
  [6/200] step 150 | loss=7.2285 pred=0.8280 kl=0.2964 recon=1.6269 spec=2.0975 | grad=7.31 | 22.7s
Epoch 6/200 | avg_loss=7.2588 pred=0.7820 kl=0.2954 recon=1.6706 spec=2.0902 | 40.6s
  New best model (loss=7.2588)
Epoch 7/200 | avg_loss=5.9076 pred=0.6632 kl=0.3169 recon=1.3503 spec=1.6955 | 40.6s
  New best model (loss=5.9076)
  [8/200] step 200 | loss=4.9208 pred=0.5572 kl=0.3621 recon=1.1829 spec=1.3314 | grad=4.71 | 16.9s
Epoch 8/200 | avg_loss=4.8213 pred=0.5439 kl=0.3731 recon=1.1619 spec=1.3019 | 40.8s
  New best model (loss=4.8213)
Epoch 9/200 | avg_loss=4.0268 pred=0.4353 kl=0.4836 recon=1.0010 spec=1.0590 | 40.4s
  New best model (loss=4.0268)
  [10/200] step 250 | loss=3.8342 pred=0.3734 kl=0.5448 recon=0.9780 spec=1.0024 | grad=32.58 | 10.9s
Epoch 10/200 | avg_loss=3.7727 pred=0.3652 kl=0.5560 recon=0.9683 spec=0.9797 | 40.8s
  Saved checkpoint: /home/ubuntu/lewm-tts/output_v3c/checkpoints/epoch_0010.pt
  New best model (loss=3.7727)
Epoch 11/200 | avg_loss=3.6309 pred=0.3330 kl=0.5902 recon=0.9486 spec=0.9328 | 40.5s
  New best model (loss=3.6309)
  [12/200] step 300 | loss=3.6239 pred=0.3101 kl=0.6098 recon=0.9584 spec=0.9303 | grad=26.33 | 4.9s
Epoch 12/200 | avg_loss=3.4695 pred=0.2884 kl=0.6412 recon=0.9204 spec=0.8922 | 40.6s
  New best model (loss=3.4695)
  [13/200] step 350 | loss=3.2157 pred=0.2090 kl=0.7828 recon=0.8850 spec=0.8227 | grad=14.26 | 39.2s
Epoch 13/200 | avg_loss=3.3178 pred=0.2341 kl=0.7302 recon=0.8977 spec=0.8574 | 40.8s
  New best model (loss=3.3178)
Epoch 14/200 | avg_loss=3.2315 pred=0.1837 kl=0.8377 recon=0.8898 spec=0.8436 | 40.9s
  New best model (loss=3.2315)
  [15/200] step 400 | loss=3.1591 pred=0.1422 kl=0.9733 recon=0.8933 spec=0.8179 | grad=22.70 | 33.5s
Epoch 15/200 | avg_loss=3.1433 pred=0.1514 kl=0.9374 recon=0.8741 spec=0.8269 | 40.9s
  New best model (loss=3.1433)
Epoch 16/200 | avg_loss=3.0754 pred=0.1303 kl=1.0223 recon=0.8569 spec=0.8182 | 40.5s
  New best model (loss=3.0754)
  [17/200] step 450 | loss=2.9534 pred=0.1163 kl=1.1075 recon=0.8214 spec=0.7931 | grad=13.75 | 27.6s
Epoch 17/200 | avg_loss=2.9850 pred=0.1171 kl=1.0986 recon=0.8310 spec=0.8009 | 40.9s
  New best model (loss=2.9850)
Epoch 18/200 | avg_loss=2.9984 pred=0.1021 kl=1.1966 recon=0.8366 spec=0.8119 | 40.5s
  [19/200] step 500 | loss=2.9102 pred=0.0872 kl=1.2991 recon=0.7968 spec=0.8156 | grad=12.00 | 21.1s
Epoch 19/200 | avg_loss=2.8936 pred=0.0886 kl=1.2971 recon=0.8056 spec=0.7920 | 40.6s
  New best model (loss=2.8936)
Epoch 20/200 | avg_loss=2.8444 pred=0.0758 kl=1.4101 recon=0.7931 spec=0.7837 | 40.7s
  Saved checkpoint: /home/ubuntu/lewm-tts/output_v3c/checkpoints/epoch_0020.pt
  New best model (loss=2.8444)
  [21/200] step 550 | loss=2.7382 pred=0.0664 kl=1.5054 recon=0.7477 spec=0.7792 | grad=7.86 | 15.4s
Epoch 21/200 | avg_loss=2.8097 pred=0.0648 kl=1.5170 recon=0.7812 spec=0.7832 | 40.7s
  New best model (loss=2.8097)
Epoch 22/200 | avg_loss=2.7211 pred=0.0561 kl=1.6157 recon=0.7555 spec=0.7635 | 40.8s
  New best model (loss=2.7211)
  [23/200] step 600 | loss=2.7167 pred=0.0525 kl=1.6758 recon=0.7557 spec=0.7624 | grad=10.03 | 9.4s
Epoch 23/200 | avg_loss=2.6877 pred=0.0490 kl=1.7129 recon=0.7431 spec=0.7620 | 40.4s
  New best model (loss=2.6877)
Epoch 24/200 | avg_loss=2.6451 pred=0.0435 kl=1.8101 recon=0.7303 spec=0.7536 | 40.8s
  New best model (loss=2.6451)
  [25/200] step 650 | loss=2.6094 pred=0.0415 kl=1.8704 recon=0.7259 spec=0.7366 | grad=9.57 | 3.6s
Epoch 25/200 | avg_loss=2.6333 pred=0.0396 kl=1.9033 recon=0.7274 spec=0.7515 | 40.7s
  New best model (loss=2.6333)
  [26/200] step 700 | loss=2.6726 pred=0.0363 kl=2.0206 recon=0.7374 spec=0.7656 | grad=9.21 | 37.5s
Epoch 26/200 | avg_loss=2.6300 pred=0.0374 kl=1.9832 recon=0.7267 spec=0.7511 | 40.5s
  New best model (loss=2.6300)
Epoch 27/200 | avg_loss=2.6164 pred=0.0357 kl=2.0697 recon=0.7212 spec=0.7497 | 40.7s
  New best model (loss=2.6164)
  [28/200] step 750 | loss=2.5593 pred=0.0320 kl=2.1418 recon=0.7061 spec=0.7335 | grad=13.84 | 31.9s
Epoch 28/200 | avg_loss=2.6100 pred=0.0332 kl=2.1325 recon=0.7218 spec=0.7457 | 40.8s
  New best model (loss=2.6100)
Epoch 29/200 | avg_loss=2.5911 pred=0.0311 kl=2.1768 recon=0.7150 spec=0.7429 | 40.5s
  New best model (loss=2.5911)
  [30/200] step 800 | loss=2.5686 pred=0.0305 kl=2.2163 recon=0.7134 spec=0.7299 | grad=10.11 | 25.6s
Epoch 30/200 | avg_loss=2.5796 pred=0.0301 kl=2.2186 recon=0.7101 spec=0.7420 | 40.7s
  Saved checkpoint: /home/ubuntu/lewm-tts/output_v3c/checkpoints/epoch_0030.pt
  New best model (loss=2.5796)
Epoch 31/200 | avg_loss=2.5567 pred=0.0290 kl=2.2608 recon=0.7027 spec=0.7367 | 40.5s
  New best model (loss=2.5567)
  [32/200] step 850 | loss=2.5136 pred=0.0283 kl=2.2881 recon=0.6939 spec=0.7197 | grad=6.13 | 20.0s
Epoch 32/200 | avg_loss=2.5509 pred=0.0285 kl=2.2890 recon=0.7021 spec=0.7334 | 40.8s
  New best model (loss=2.5509)
Epoch 33/200 | avg_loss=2.5990 pred=0.0287 kl=2.3147 recon=0.7185 spec=0.7430 | 40.6s
  [34/200] step 900 | loss=2.6683 pred=0.0284 kl=2.3074 recon=0.7330 spec=0.7698 | grad=14.52 | 13.8s
Epoch 34/200 | avg_loss=2.6028 pred=0.0286 kl=2.3078 recon=0.7215 spec=0.7412 | 40.8s
Epoch 35/200 | avg_loss=2.5449 pred=0.0293 kl=2.3040 recon=0.6980 spec=0.7332 | 40.6s
  New best model (loss=2.5449)
  [36/200] step 950 | loss=2.5300 pred=0.0277 kl=2.3304 recon=0.6851 spec=0.7410 | grad=5.92 | 8.0s
Epoch 36/200 | avg_loss=2.5258 pred=0.0275 kl=2.3351 recon=0.6917 spec=0.7295 | 40.6s
  New best model (loss=2.5258)
Epoch 37/200 | avg_loss=2.5108 pred=0.0268 kl=2.3397 recon=0.6855 spec=0.7278 | 40.4s
  New best model (loss=2.5108)
  [38/200] step 1000 | loss=2.5287 pred=0.0251 kl=2.3660 recon=0.6945 spec=0.7285 | grad=6.34 | 2.0s
Epoch 38/200 | avg_loss=2.5070 pred=0.0257 kl=2.3479 recon=0.6866 spec=0.7241 | 40.3s
  New best model (loss=2.5070)
  [39/200] step 1050 | loss=2.5116 pred=0.0256 kl=2.3389 recon=0.6930 spec=0.7182 | grad=8.98 | 36.3s
Epoch 39/200 | avg_loss=2.5086 pred=0.0256 kl=2.3421 recon=0.6875 spec=0.7236 | 40.8s
Epoch 40/200 | avg_loss=2.4941 pred=0.0256 kl=2.3290 recon=0.6813 spec=0.7219 | 40.8s
  Saved checkpoint: /home/ubuntu/lewm-tts/output_v3c/checkpoints/epoch_0040.pt
  New best model (loss=2.4941)
  [41/200] step 1100 | loss=2.5019 pred=0.0257 kl=2.3210 recon=0.6767 spec=0.7330 | grad=6.34 | 30.0s
Epoch 41/200 | avg_loss=2.4917 pred=0.0259 kl=2.3225 recon=0.6794 spec=0.7225 | 40.6s
  New best model (loss=2.4917)
Epoch 42/200 | avg_loss=2.4907 pred=0.0258 kl=2.3181 recon=0.6807 spec=0.7202 | 41.4s
  New best model (loss=2.4907)
  [43/200] step 1150 | loss=2.4448 pred=0.0256 kl=2.3266 recon=0.6714 spec=0.7022 | grad=3.72 | 24.4s
Epoch 43/200 | avg_loss=2.4890 pred=0.0260 kl=2.3112 recon=0.6805 spec=0.7192 | 40.9s
  New best model (loss=2.4890)
Epoch 44/200 | avg_loss=2.4703 pred=0.0259 kl=2.3046 recon=0.6737 spec=0.7160 | 41.1s
  New best model (loss=2.4703)
  [45/200] step 1200 | loss=2.4462 pred=0.0260 kl=2.3079 recon=0.6681 spec=0.7073 | grad=6.64 | 18.2s
Epoch 45/200 | avg_loss=2.4556 pred=0.0256 kl=2.3063 recon=0.6681 spec=0.7138 | 40.7s
  New best model (loss=2.4556)
Epoch 46/200 | avg_loss=2.4424 pred=0.0255 kl=2.3130 recon=0.6638 spec=0.7108 | 40.7s
  New best model (loss=2.4424)
  [47/200] step 1250 | loss=2.4447 pred=0.0262 kl=2.3129 recon=0.6634 spec=0.7124 | grad=4.21 | 12.8s
Epoch 47/200 | avg_loss=2.4271 pred=0.0265 kl=2.3131 recon=0.6583 spec=0.7072 | 41.5s
  New best model (loss=2.4271)
Epoch 48/200 | avg_loss=2.3963 pred=0.0274 kl=2.3100 recon=0.6461 spec=0.7023 | 40.9s
  New best model (loss=2.3963)
  [49/200] step 1300 | loss=2.3802 pred=0.0273 kl=2.3204 recon=0.6421 spec=0.6971 | grad=7.82 | 6.4s
Epoch 49/200 | avg_loss=2.3919 pred=0.0277 kl=2.3083 recon=0.6449 spec=0.7009 | 40.8s
  New best model (loss=2.3919)
  [50/200] step 1350 | loss=2.3420 pred=0.0264 kl=2.3338 recon=0.6256 spec=0.6940 | grad=4.91 | 40.8s
Epoch 50/200 | avg_loss=2.3553 pred=0.0277 kl=2.3177 recon=0.6307 spec=0.6954 | 40.9s
  Saved checkpoint: /home/ubuntu/lewm-tts/output_v3c/checkpoints/epoch_0050.pt
  New best model (loss=2.3553)
Epoch 51/200 | avg_loss=2.3503 pred=0.0266 kl=2.3342 recon=0.6285 spec=0.6956 | 40.8s
  New best model (loss=2.3503)
  [52/200] step 1400 | loss=2.3157 pred=0.0269 kl=2.3268 recon=0.6164 spec=0.6885 | grad=4.67 | 34.3s
Epoch 52/200 | avg_loss=2.3292 pred=0.0267 kl=2.3327 recon=0.6190 spec=0.6942 | 40.4s
  New best model (loss=2.3292)
Epoch 53/200 | avg_loss=2.3206 pred=0.0264 kl=2.3388 recon=0.6175 spec=0.6906 | 40.6s
  New best model (loss=2.3206)
  [54/200] step 1450 | loss=2.2774 pred=0.0256 kl=2.3475 recon=0.6093 spec=0.6731 | grad=6.15 | 28.8s
Epoch 54/200 | avg_loss=2.3003 pred=0.0258 kl=2.3416 recon=0.6098 spec=0.6877 | 40.7s
  New best model (loss=2.3003)
Epoch 55/200 | avg_loss=2.2905 pred=0.0255 kl=2.3496 recon=0.6073 spec=0.6845 | 40.6s
  New best model (loss=2.2905)
  [56/200] step 1500 | loss=2.3161 pred=0.0254 kl=2.3528 recon=0.6174 spec=0.6882 | grad=7.04 | 23.0s
Epoch 56/200 | avg_loss=2.3315 pred=0.0255 kl=2.3495 recon=0.6230 spec=0.6911 | 41.0s
Epoch 57/200 | avg_loss=2.2962 pred=0.0260 kl=2.3402 recon=0.6096 spec=0.6851 | 40.8s
  [58/200] step 1550 | loss=2.2628 pred=0.0259 kl=2.3417 recon=0.5985 spec=0.6777 | grad=10.75 | 16.8s
Epoch 58/200 | avg_loss=2.2690 pred=0.0254 kl=2.3435 recon=0.5994 spec=0.6809 | 40.7s
  New best model (loss=2.2690)
Epoch 59/200 | avg_loss=2.2864 pred=0.0249 kl=2.3518 recon=0.6068 spec=0.6829 | 41.0s
  [60/200] step 1600 | loss=2.2822 pred=0.0250 kl=2.3495 recon=0.6046 spec=0.6830 | grad=6.96 | 11.0s
Epoch 60/200 | avg_loss=2.2586 pred=0.0249 kl=2.3509 recon=0.5965 spec=0.6782 | 40.7s
  Saved checkpoint: /home/ubuntu/lewm-tts/output_v3c/checkpoints/epoch_0060.pt
  New best model (loss=2.2586)
Epoch 61/200 | avg_loss=2.2728 pred=0.0247 kl=2.3483 recon=0.6014 spec=0.6813 | 40.8s
  [62/200] step 1650 | loss=2.3741 pred=0.0253 kl=2.3433 recon=0.6319 spec=0.7076 | grad=14.34 | 4.8s
Epoch 62/200 | avg_loss=2.2717 pred=0.0245 kl=2.3454 recon=0.6001 spec=0.6823 | 40.5s
  [63/200] step 1700 | loss=2.2464 pred=0.0251 kl=2.3397 recon=0.5924 spec=0.6755 | grad=2.79 | 39.3s
Epoch 63/200 | avg_loss=2.2423 pred=0.0248 kl=2.3390 recon=0.5908 spec=0.6750 | 40.9s
  New best model (loss=2.2423)
Epoch 64/200 | avg_loss=2.2273 pred=0.0242 kl=2.3472 recon=0.5848 spec=0.6734 | 40.8s
  New best model (loss=2.2273)
  [65/200] step 1750 | loss=2.2269 pred=0.0237 kl=2.3583 recon=0.5934 spec=0.6619 | grad=5.92 | 33.1s
Epoch 65/200 | avg_loss=2.2275 pred=0.0233 kl=2.3628 recon=0.5839 spec=0.6752 | 40.5s
Epoch 66/200 | avg_loss=2.2466 pred=0.0232 kl=2.3605 recon=0.5930 spec=0.6759 | 40.7s
  [67/200] step 1800 | loss=2.2083 pred=0.0233 kl=2.3615 recon=0.5825 spec=0.6643 | grad=4.31 | 27.3s
Epoch 67/200 | avg_loss=2.2201 pred=0.0232 kl=2.3607 recon=0.5833 spec=0.6711 | 40.6s
  New best model (loss=2.2201)
Epoch 68/200 | avg_loss=2.2320 pred=0.0230 kl=2.3627 recon=0.5857 spec=0.6760 | 40.3s
  [69/200] step 1850 | loss=2.1959 pred=0.0225 kl=2.3655 recon=0.5727 spec=0.6696 | grad=4.18 | 21.2s
Epoch 69/200 | avg_loss=2.2099 pred=0.0226 kl=2.3653 recon=0.5805 spec=0.6685 | 41.0s
  New best model (loss=2.2099)
Epoch 70/200 | avg_loss=2.1966 pred=0.0221 kl=2.3680 recon=0.5759 spec=0.6661 | 40.7s
  Saved checkpoint: /home/ubuntu/lewm-tts/output_v3c/checkpoints/epoch_0070.pt
  New best model (loss=2.1966)
  [71/200] step 1900 | loss=2.2263 pred=0.0214 kl=2.3772 recon=0.5734 spec=0.6896 | grad=7.18 | 15.2s
Epoch 71/200 | avg_loss=2.2085 pred=0.0219 kl=2.3681 recon=0.5796 spec=0.6692 | 40.6s
Epoch 72/200 | avg_loss=2.1989 pred=0.0224 kl=2.3601 recon=0.5753 spec=0.6682 | 40.5s
  [73/200] step 1950 | loss=2.2031 pred=0.0222 kl=2.3601 recon=0.5749 spec=0.6716 | grad=4.75 | 9.5s
Epoch 73/200 | avg_loss=2.1931 pred=0.0220 kl=2.3632 recon=0.5743 spec=0.6659 | 40.8s
  New best model (loss=2.1931)
Epoch 74/200 | avg_loss=2.2029 pred=0.0218 kl=2.3658 recon=0.5777 spec=0.6681 | 41.1s
  [75/200] step 2000 | loss=2.1799 pred=0.0212 kl=2.3802 recon=0.5697 spec=0.6636 | grad=3.17 | 3.6s
Epoch 75/200 | avg_loss=2.1929 pred=0.0215 kl=2.3763 recon=0.5744 spec=0.6658 | 40.8s
  New best model (loss=2.1929)
  [76/200] step 2050 | loss=2.2045 pred=0.0210 kl=2.3783 recon=0.5762 spec=0.6716 | grad=5.58 | 38.4s
Epoch 76/200 | avg_loss=2.1819 pred=0.0214 kl=2.3794 recon=0.5706 spec=0.6637 | 41.4s
  New best model (loss=2.1819)
Epoch 77/200 | avg_loss=2.1824 pred=0.0212 kl=2.3782 recon=0.5712 spec=0.6633 | 41.1s
  [78/200] step 2100 | loss=2.1417 pred=0.0210 kl=2.3817 recon=0.5640 spec=0.6459 | grad=2.90 | 31.8s
Epoch 78/200 | avg_loss=2.1719 pred=0.0213 kl=2.3791 recon=0.5675 spec=0.6612 | 40.7s
  New best model (loss=2.1719)
Epoch 79/200 | avg_loss=2.1734 pred=0.0212 kl=2.3740 recon=0.5685 spec=0.6609 | 41.1s
  [80/200] step 2150 | loss=2.1445 pred=0.0206 kl=2.3876 recon=0.5638 spec=0.6483 | grad=4.33 | 25.4s
Epoch 80/200 | avg_loss=2.1680 pred=0.0208 kl=2.3801 recon=0.5645 spec=0.6629 | 40.4s
  Saved checkpoint: /home/ubuntu/lewm-tts/output_v3c/checkpoints/epoch_0080.pt
  New best model (loss=2.1680)
Epoch 81/200 | avg_loss=2.1702 pred=0.0204 kl=2.3827 recon=0.5673 spec=0.6609 | 40.8s
  [82/200] step 2200 | loss=2.1877 pred=0.0204 kl=2.3846 recon=0.5756 spec=0.6616 | grad=7.33 | 19.9s
Epoch 82/200 | avg_loss=2.1934 pred=0.0203 kl=2.3808 recon=0.5762 spec=0.6645 | 40.9s
Epoch 83/200 | avg_loss=2.1657 pred=0.0206 kl=2.3762 recon=0.5658 spec=0.6599 | 40.8s
  New best model (loss=2.1657)
  [84/200] step 2250 | loss=2.1685 pred=0.0209 kl=2.3816 recon=0.5681 spec=0.6584 | grad=5.59 | 14.2s
Epoch 84/200 | avg_loss=2.1576 pred=0.0207 kl=2.3773 recon=0.5625 spec=0.6588 | 40.7s
  New best model (loss=2.1576)
Epoch 85/200 | avg_loss=2.1639 pred=0.0205 kl=2.3841 recon=0.5651 spec=0.6597 | 40.5s
  [86/200] step 2300 | loss=2.1748 pred=0.0203 kl=2.3869 recon=0.5637 spec=0.6688 | grad=4.85 | 8.0s
Epoch 86/200 | avg_loss=2.1583 pred=0.0205 kl=2.3799 recon=0.5634 spec=0.6582 | 40.5s
Epoch 87/200 | avg_loss=2.1500 pred=0.0201 kl=2.3894 recon=0.5606 spec=0.6565 | 40.7s
  New best model (loss=2.1500)
  [88/200] step 2350 | loss=2.1683 pred=0.0197 kl=2.4012 recon=0.5673 spec=0.6600 | grad=6.14 | 2.0s
Epoch 88/200 | avg_loss=2.1607 pred=0.0198 kl=2.3925 recon=0.5634 spec=0.6601 | 40.6s
  [89/200] step 2400 | loss=2.1479 pred=0.0201 kl=2.3842 recon=0.5575 spec=0.6593 | grad=4.23 | 36.3s
Epoch 89/200 | avg_loss=2.1456 pred=0.0202 kl=2.3839 recon=0.5593 spec=0.6554 | 40.9s
  New best model (loss=2.1456)
Epoch 90/200 | avg_loss=2.1441 pred=0.0201 kl=2.3830 recon=0.5595 spec=0.6541 | 41.1s
  Saved checkpoint: /home/ubuntu/lewm-tts/output_v3c/checkpoints/epoch_0090.pt
  New best model (loss=2.1441)
  [91/200] step 2450 | loss=2.1513 pred=0.0195 kl=2.3962 recon=0.5631 spec=0.6545 | grad=3.39 | 30.2s
Epoch 91/200 | avg_loss=2.1518 pred=0.0199 kl=2.3881 recon=0.5618 spec=0.6562 | 40.7s
Epoch 92/200 | avg_loss=2.1548 pred=0.0200 kl=2.3849 recon=0.5619 spec=0.6581 | 40.6s
  [93/200] step 2500 | loss=2.1726 pred=0.0203 kl=2.3807 recon=0.5622 spec=0.6695 | grad=3.79 | 24.3s
Epoch 93/200 | avg_loss=2.1408 pred=0.0202 kl=2.3859 recon=0.5581 spec=0.6537 | 40.7s
  New best model (loss=2.1408)
Epoch 94/200 | avg_loss=2.1370 pred=0.0201 kl=2.3889 recon=0.5554 spec=0.6548 | 40.6s
  New best model (loss=2.1370)
  [95/200] step 2550 | loss=2.1540 pred=0.0201 kl=2.3872 recon=0.5557 spec=0.6657 | grad=3.94 | 18.3s
Epoch 95/200 | avg_loss=2.1514 pred=0.0201 kl=2.3882 recon=0.5616 spec=0.6561 | 40.8s
Epoch 96/200 | avg_loss=2.1444 pred=0.0199 kl=2.3963 recon=0.5598 spec=0.6540 | 40.8s
  [97/200] step 2600 | loss=2.1690 pred=0.0197 kl=2.3967 recon=0.5676 spec=0.6601 | grad=10.26 | 12.4s
Epoch 97/200 | avg_loss=2.1384 pred=0.0199 kl=2.3951 recon=0.5571 spec=0.6536 | 40.6s
Epoch 98/200 | avg_loss=2.1370 pred=0.0203 kl=2.3892 recon=0.5565 spec=0.6532 | 41.2s
  New best model (loss=2.1370)
  [99/200] step 2650 | loss=2.1515 pred=0.0200 kl=2.3915 recon=0.5504 spec=0.6712 | grad=3.93 | 6.6s
Epoch 99/200 | avg_loss=2.1327 pred=0.0198 kl=2.3979 recon=0.5542 spec=0.6536 | 41.1s
  New best model (loss=2.1327)
  [100/200] step 2700 | loss=2.0931 pred=0.0195 kl=2.4133 recon=0.5473 spec=0.6366 | grad=1.89 | 40.9s
Epoch 100/200 | avg_loss=2.1194 pred=0.0195 kl=2.4127 recon=0.5508 spec=0.6495 | 41.0s
  Saved checkpoint: /home/ubuntu/lewm-tts/output_v3c/checkpoints/epoch_0100.pt
  New best model (loss=2.1194)
Epoch 101/200 | avg_loss=2.1080 pred=0.0195 kl=2.4126 recon=0.5469 spec=0.6471 | 41.0s
  New best model (loss=2.1080)
  [102/200] step 2750 | loss=2.0846 pred=0.0197 kl=2.3963 recon=0.5433 spec=0.6363 | grad=4.95 | 35.3s
Epoch 102/200 | avg_loss=2.0944 pred=0.0197 kl=2.4024 recon=0.5440 spec=0.6418 | 41.2s
  New best model (loss=2.0944)
Epoch 103/200 | avg_loss=2.0998 pred=0.0196 kl=2.3997 recon=0.5427 spec=0.6472 | 40.5s
  [104/200] step 2800 | loss=2.0818 pred=0.0193 kl=2.4141 recon=0.5474 spec=0.6291 | grad=3.10 | 28.7s
Epoch 104/200 | avg_loss=2.1022 pred=0.0195 kl=2.4072 recon=0.5449 spec=0.6458 | 40.7s
Epoch 105/200 | avg_loss=2.0867 pred=0.0194 kl=2.4118 recon=0.5408 spec=0.6410 | 40.8s
  New best model (loss=2.0867)
  [106/200] step 2850 | loss=2.1014 pred=0.0189 kl=2.4173 recon=0.5482 spec=0.6413 | grad=4.48 | 22.7s
Epoch 106/200 | avg_loss=2.1020 pred=0.0192 kl=2.4132 recon=0.5458 spec=0.6447 | 40.8s
Epoch 107/200 | avg_loss=2.0944 pred=0.0192 kl=2.4109 recon=0.5430 spec=0.6433 | 40.8s
  [108/200] step 2900 | loss=2.0839 pred=0.0196 kl=2.3993 recon=0.5391 spec=0.6413 | grad=1.92 | 17.0s
Epoch 108/200 | avg_loss=2.0772 pred=0.0193 kl=2.4077 recon=0.5370 spec=0.6399 | 40.8s
  New best model (loss=2.0772)
Epoch 109/200 | avg_loss=2.0766 pred=0.0193 kl=2.4100 recon=0.5366 spec=0.6400 | 40.6s
  New best model (loss=2.0766)
  [110/200] step 2950 | loss=2.0721 pred=0.0192 kl=2.4132 recon=0.5361 spec=0.6377 | grad=3.52 | 11.0s
Epoch 110/200 | avg_loss=2.0798 pred=0.0191 kl=2.4096 recon=0.5381 spec=0.6403 | 40.7s
  Saved checkpoint: /home/ubuntu/lewm-tts/output_v3c/checkpoints/epoch_0110.pt
Epoch 111/200 | avg_loss=2.0729 pred=0.0191 kl=2.4085 recon=0.5355 spec=0.6392 | 40.6s
  New best model (loss=2.0729)
  [112/200] step 3000 | loss=2.1085 pred=0.0196 kl=2.4101 recon=0.5382 spec=0.6589 | grad=2.72 | 4.9s
Epoch 112/200 | avg_loss=2.0696 pred=0.0188 kl=2.4096 recon=0.5343 spec=0.6388 | 40.6s
  New best model (loss=2.0696)
  [113/200] step 3050 | loss=2.0915 pred=0.0185 kl=2.4134 recon=0.5299 spec=0.6594 | grad=2.52 | 39.2s
Epoch 113/200 | avg_loss=2.0668 pred=0.0186 kl=2.4093 recon=0.5323 spec=0.6396 | 40.6s
  New best model (loss=2.0668)
Epoch 114/200 | avg_loss=2.0673 pred=0.0185 kl=2.4170 recon=0.5319 spec=0.6405 | 40.5s
  [115/200] step 3100 | loss=2.0705 pred=0.0189 kl=2.4155 recon=0.5384 spec=0.6337 | grad=6.95 | 33.1s
Epoch 115/200 | avg_loss=2.0717 pred=0.0186 kl=2.4176 recon=0.5347 spec=0.6398 | 40.6s
Epoch 116/200 | avg_loss=2.0661 pred=0.0187 kl=2.4103 recon=0.5325 spec=0.6389 | 40.5s
  New best model (loss=2.0661)
  [117/200] step 3150 | loss=2.0836 pred=0.0187 kl=2.4056 recon=0.5354 spec=0.6467 | grad=1.59 | 27.4s
Epoch 117/200 | avg_loss=2.0594 pred=0.0186 kl=2.4083 recon=0.5308 spec=0.6368 | 40.6s
  New best model (loss=2.0594)
Epoch 118/200 | avg_loss=2.0511 pred=0.0183 kl=2.4134 recon=0.5284 spec=0.6346 | 40.7s
  New best model (loss=2.0511)
  [119/200] step 3200 | loss=2.0833 pred=0.0181 kl=2.4119 recon=0.5289 spec=0.6555 | grad=3.95 | 21.4s
Epoch 119/200 | avg_loss=2.0593 pred=0.0183 kl=2.4101 recon=0.5310 spec=0.6365 | 40.6s
Epoch 120/200 | avg_loss=2.0569 pred=0.0185 kl=2.4065 recon=0.5308 spec=0.6351 | 40.9s
  Saved checkpoint: /home/ubuntu/lewm-tts/output_v3c/checkpoints/epoch_0120.pt
  [121/200] step 3250 | loss=2.0750 pred=0.0185 kl=2.4085 recon=0.5315 spec=0.6463 | grad=2.07 | 15.3s
Epoch 121/200 | avg_loss=2.0463 pred=0.0184 kl=2.4109 recon=0.5264 spec=0.6341 | 40.8s
  New best model (loss=2.0463)
Epoch 122/200 | avg_loss=2.0506 pred=0.0182 kl=2.4147 recon=0.5275 spec=0.6355 | 40.7s
  [123/200] step 3300 | loss=2.0926 pred=0.0180 kl=2.4286 recon=0.5284 spec=0.6624 | grad=2.93 | 9.5s
Epoch 123/200 | avg_loss=2.0510 pred=0.0181 kl=2.4289 recon=0.5268 spec=0.6368 | 40.5s
Epoch 124/200 | avg_loss=2.0472 pred=0.0180 kl=2.4319 recon=0.5261 spec=0.6351 | 40.5s
  [125/200] step 3350 | loss=2.0537 pred=0.0182 kl=2.4239 recon=0.5326 spec=0.6307 | grad=2.51 | 3.6s
Epoch 125/200 | avg_loss=2.0495 pred=0.0179 kl=2.4288 recon=0.5270 spec=0.6354 | 40.5s
  [126/200] step 3400 | loss=2.0151 pred=0.0179 kl=2.4286 recon=0.5211 spec=0.6205 | grad=1.61 | 37.7s
Epoch 126/200 | avg_loss=2.0393 pred=0.0179 kl=2.4295 recon=0.5242 spec=0.6325 | 40.7s
  New best model (loss=2.0393)
Epoch 127/200 | avg_loss=2.0461 pred=0.0180 kl=2.4290 recon=0.5256 spec=0.6352 | 40.3s
  [128/200] step 3450 | loss=2.0332 pred=0.0179 kl=2.4242 recon=0.5251 spec=0.6273 | grad=1.94 | 31.6s
Epoch 128/200 | avg_loss=2.0379 pred=0.0179 kl=2.4275 recon=0.5237 spec=0.6322 | 40.6s
  New best model (loss=2.0379)
Epoch 129/200 | avg_loss=2.0379 pred=0.0180 kl=2.4192 recon=0.5232 spec=0.6329 | 40.8s
  [130/200] step 3500 | loss=2.0166 pred=0.0177 kl=2.4200 recon=0.5200 spec=0.6231 | grad=2.40 | 26.1s
Epoch 130/200 | avg_loss=2.0319 pred=0.0180 kl=2.4172 recon=0.5228 spec=0.6295 | 41.0s
  Saved checkpoint: /home/ubuntu/lewm-tts/output_v3c/checkpoints/epoch_0130.pt
  New best model (loss=2.0319)
Epoch 131/200 | avg_loss=2.0348 pred=0.0179 kl=2.4194 recon=0.5220 spec=0.6324 | 40.5s
  [132/200] step 3550 | loss=2.0231 pred=0.0180 kl=2.4095 recon=0.5227 spec=0.6238 | grad=5.60 | 19.7s
Epoch 132/200 | avg_loss=2.0360 pred=0.0181 kl=2.4107 recon=0.5225 spec=0.6326 | 40.7s
Epoch 133/200 | avg_loss=2.0322 pred=0.0179 kl=2.4092 recon=0.5228 spec=0.6297 | 40.9s
  [134/200] step 3600 | loss=2.0284 pred=0.0179 kl=2.4118 recon=0.5230 spec=0.6270 | grad=2.02 | 14.0s
Epoch 134/200 | avg_loss=2.0282 pred=0.0179 kl=2.4178 recon=0.5205 spec=0.6302 | 40.8s
  New best model (loss=2.0282)
Epoch 135/200 | avg_loss=2.0333 pred=0.0178 kl=2.4273 recon=0.5218 spec=0.6318 | 40.6s
  [136/200] step 3650 | loss=2.0110 pred=0.0177 kl=2.4286 recon=0.5181 spec=0.6219 | grad=3.39 | 8.0s
Epoch 136/200 | avg_loss=2.0266 pred=0.0177 kl=2.4307 recon=0.5195 spec=0.6304 | 40.8s
  New best model (loss=2.0266)
Epoch 137/200 | avg_loss=2.0270 pred=0.0177 kl=2.4306 recon=0.5194 spec=0.6308 | 40.7s
  [138/200] step 3700 | loss=2.0429 pred=0.0177 kl=2.4280 recon=0.5166 spec=0.6451 | grad=3.94 | 2.0s
Epoch 138/200 | avg_loss=2.0252 pred=0.0177 kl=2.4298 recon=0.5192 spec=0.6298 | 40.6s
  New best model (loss=2.0252)
  [139/200] step 3750 | loss=2.0221 pred=0.0178 kl=2.4218 recon=0.5182 spec=0.6292 | grad=1.43 | 36.2s
Epoch 139/200 | avg_loss=2.0215 pred=0.0178 kl=2.4248 recon=0.5175 spec=0.6296 | 40.6s
  New best model (loss=2.0215)
Epoch 140/200 | avg_loss=2.0212 pred=0.0179 kl=2.4269 recon=0.5178 spec=0.6290 | 40.8s
  Saved checkpoint: /home/ubuntu/lewm-tts/output_v3c/checkpoints/epoch_0140.pt
  New best model (loss=2.0212)
  [141/200] step 3800 | loss=2.0163 pred=0.0179 kl=2.4324 recon=0.5182 spec=0.6251 | grad=1.71 | 30.2s
Epoch 141/200 | avg_loss=2.0192 pred=0.0178 kl=2.4313 recon=0.5173 spec=0.6284 | 40.7s
  New best model (loss=2.0192)
Epoch 142/200 | avg_loss=2.0169 pred=0.0179 kl=2.4289 recon=0.5162 spec=0.6282 | 40.7s
  New best model (loss=2.0169)
  [143/200] step 3850 | loss=2.0133 pred=0.0178 kl=2.4366 recon=0.5199 spec=0.6209 | grad=1.36 | 24.1s
Epoch 143/200 | avg_loss=2.0144 pred=0.0178 kl=2.4356 recon=0.5156 spec=0.6273 | 40.6s
  New best model (loss=2.0144)
Epoch 144/200 | avg_loss=2.0149 pred=0.0177 kl=2.4383 recon=0.5152 spec=0.6284 | 40.4s
  [145/200] step 3900 | loss=2.0197 pred=0.0177 kl=2.4395 recon=0.5171 spec=0.6290 | grad=3.49 | 18.2s
Epoch 145/200 | avg_loss=2.0141 pred=0.0177 kl=2.4402 recon=0.5151 spec=0.6280 | 40.6s
  New best model (loss=2.0141)
Epoch 146/200 | avg_loss=2.0110 pred=0.0176 kl=2.4478 recon=0.5139 spec=0.6274 | 40.7s
  New best model (loss=2.0110)
  [147/200] step 3950 | loss=2.0148 pred=0.0180 kl=2.4460 recon=0.5153 spec=0.6278 | grad=1.23 | 12.5s
Epoch 147/200 | avg_loss=2.0125 pred=0.0176 kl=2.4460 recon=0.5143 spec=0.6279 | 40.7s
Epoch 148/200 | avg_loss=2.0090 pred=0.0176 kl=2.4441 recon=0.5137 spec=0.6263 | 40.7s
  New best model (loss=2.0090)
  [149/200] step 4000 | loss=1.9978 pred=0.0176 kl=2.4428 recon=0.5099 spec=0.6240 | grad=3.30 | 6.5s
Epoch 149/200 | avg_loss=2.0047 pred=0.0176 kl=2.4410 recon=0.5126 spec=0.6250 | 40.7s
  New best model (loss=2.0047)
  [150/200] step 4050 | loss=2.0125 pred=0.0178 kl=2.4379 recon=0.5127 spec=0.6299 | grad=1.22 | 40.8s
Epoch 150/200 | avg_loss=2.0024 pred=0.0177 kl=2.4377 recon=0.5120 spec=0.6242 | 40.9s
  Saved checkpoint: /home/ubuntu/lewm-tts/output_v3c/checkpoints/epoch_0150.pt
  New best model (loss=2.0024)
Epoch 151/200 | avg_loss=2.0051 pred=0.0177 kl=2.4353 recon=0.5112 spec=0.6272 | 40.6s
  [152/200] step 4100 | loss=2.0031 pred=0.0178 kl=2.4328 recon=0.5134 spec=0.6228 | grad=4.07 | 34.4s
Epoch 152/200 | avg_loss=2.0020 pred=0.0176 kl=2.4342 recon=0.5107 spec=0.6257 | 40.6s
  New best model (loss=2.0020)
Epoch 153/200 | avg_loss=2.0001 pred=0.0176 kl=2.4368 recon=0.5102 spec=0.6251 | 40.7s
  New best model (loss=2.0001)
  [154/200] step 4150 | loss=1.9898 pred=0.0173 kl=2.4485 recon=0.5088 spec=0.6202 | grad=1.91 | 28.8s
Epoch 154/200 | avg_loss=1.9991 pred=0.0176 kl=2.4422 recon=0.5099 spec=0.6248 | 40.7s
  New best model (loss=1.9991)
Epoch 155/200 | avg_loss=1.9954 pred=0.0176 kl=2.4446 recon=0.5092 spec=0.6234 | 40.8s
  New best model (loss=1.9954)
  [156/200] step 4200 | loss=2.0261 pred=0.0174 kl=2.4505 recon=0.5082 spec=0.6452 | grad=3.33 | 22.7s
Epoch 156/200 | avg_loss=1.9970 pred=0.0175 kl=2.4491 recon=0.5090 spec=0.6247 | 40.5s
Epoch 157/200 | avg_loss=1.9939 pred=0.0175 kl=2.4483 recon=0.5087 spec=0.6229 | 40.7s
  New best model (loss=1.9939)
  [158/200] step 4250 | loss=2.0006 pred=0.0174 kl=2.4530 recon=0.5037 spec=0.6342 | grad=1.74 | 16.7s
Epoch 158/200 | avg_loss=1.9965 pred=0.0175 kl=2.4532 recon=0.5082 spec=0.6254 | 40.5s
Epoch 159/200 | avg_loss=1.9919 pred=0.0174 kl=2.4545 recon=0.5080 spec=0.6226 | 40.6s
  New best model (loss=1.9919)
  [160/200] step 4300 | loss=2.0142 pred=0.0172 kl=2.4594 recon=0.5066 spec=0.6394 | grad=7.58 | 10.6s
Epoch 160/200 | avg_loss=1.9945 pred=0.0173 kl=2.4571 recon=0.5076 spec=0.6250 | 40.4s
  Saved checkpoint: /home/ubuntu/lewm-tts/output_v3c/checkpoints/epoch_0160.pt
Epoch 161/200 | avg_loss=1.9903 pred=0.0173 kl=2.4519 recon=0.5076 spec=0.6222 | 40.8s
  New best model (loss=1.9903)
  [162/200] step 4350 | loss=2.0074 pred=0.0172 kl=2.4526 recon=0.5113 spec=0.6286 | grad=2.33 | 5.0s
Epoch 162/200 | avg_loss=1.9896 pred=0.0173 kl=2.4500 recon=0.5070 spec=0.6224 | 40.8s
  New best model (loss=1.9896)
  [163/200] step 4400 | loss=1.9792 pred=0.0172 kl=2.4542 recon=0.5023 spec=0.6219 | grad=0.99 | 39.2s
Epoch 163/200 | avg_loss=1.9893 pred=0.0173 kl=2.4517 recon=0.5065 spec=0.6230 | 40.7s
  New best model (loss=1.9893)
Epoch 164/200 | avg_loss=1.9887 pred=0.0173 kl=2.4511 recon=0.5063 spec=0.6228 | 40.5s
  New best model (loss=1.9887)
  [165/200] step 4450 | loss=1.9915 pred=0.0175 kl=2.4437 recon=0.5096 spec=0.6202 | grad=1.65 | 33.2s
Epoch 165/200 | avg_loss=1.9914 pred=0.0174 kl=2.4465 recon=0.5065 spec=0.6244 | 40.6s
Epoch 166/200 | avg_loss=1.9860 pred=0.0174 kl=2.4440 recon=0.5061 spec=0.6213 | 40.7s
  New best model (loss=1.9860)
  [167/200] step 4500 | loss=2.0048 pred=0.0173 kl=2.4415 recon=0.5059 spec=0.6342 | grad=1.11 | 27.4s
Epoch 167/200 | avg_loss=1.9864 pred=0.0174 kl=2.4411 recon=0.5057 spec=0.6222 | 40.7s
Epoch 168/200 | avg_loss=1.9844 pred=0.0173 kl=2.4448 recon=0.5056 spec=0.6209 | 40.8s
  New best model (loss=1.9844)
  [169/200] step 4550 | loss=1.9810 pred=0.0174 kl=2.4455 recon=0.5035 spec=0.6214 | grad=1.37 | 21.4s
Epoch 169/200 | avg_loss=1.9870 pred=0.0174 kl=2.4474 recon=0.5053 spec=0.6230 | 40.5s
Epoch 170/200 | avg_loss=1.9853 pred=0.0173 kl=2.4464 recon=0.5050 spec=0.6223 | 40.5s
  Saved checkpoint: /home/ubuntu/lewm-tts/output_v3c/checkpoints/epoch_0170.pt
  [171/200] step 4600 | loss=1.9547 pred=0.0172 kl=2.4498 recon=0.5016 spec=0.6065 | grad=2.12 | 15.4s
Epoch 171/200 | avg_loss=1.9857 pred=0.0173 kl=2.4483 recon=0.5048 spec=0.6229 | 40.7s
Epoch 172/200 | avg_loss=1.9834 pred=0.0173 kl=2.4491 recon=0.5044 spec=0.6218 | 40.7s
  New best model (loss=1.9834)
  [173/200] step 4650 | loss=1.9600 pred=0.0172 kl=2.4502 recon=0.4995 spec=0.6129 | grad=1.70 | 9.3s
Epoch 173/200 | avg_loss=1.9846 pred=0.0173 kl=2.4477 recon=0.5045 spec=0.6225 | 40.6s
Epoch 174/200 | avg_loss=1.9808 pred=0.0172 kl=2.4492 recon=0.5043 spec=0.6203 | 40.9s
  New best model (loss=1.9808)
  [175/200] step 4700 | loss=1.9886 pred=0.0172 kl=2.4495 recon=0.4998 spec=0.6315 | grad=1.89 | 3.5s
Epoch 175/200 | avg_loss=1.9818 pred=0.0173 kl=2.4466 recon=0.5041 spec=0.6213 | 40.7s
  [176/200] step 4750 | loss=1.9816 pred=0.0171 kl=2.4488 recon=0.5050 spec=0.6199 | grad=1.24 | 37.4s
Epoch 176/200 | avg_loss=1.9836 pred=0.0173 kl=2.4470 recon=0.5038 spec=0.6228 | 40.4s
Epoch 177/200 | avg_loss=1.9806 pred=0.0172 kl=2.4478 recon=0.5038 spec=0.6209 | 40.7s
  New best model (loss=1.9806)
  [178/200] step 4800 | loss=1.9574 pred=0.0172 kl=2.4515 recon=0.5023 spec=0.6074 | grad=5.63 | 31.5s
Epoch 178/200 | avg_loss=1.9820 pred=0.0172 kl=2.4502 recon=0.5035 spec=0.6222 | 40.4s
Epoch 179/200 | avg_loss=1.9786 pred=0.0172 kl=2.4505 recon=0.5036 spec=0.6197 | 40.8s
  New best model (loss=1.9786)
  [180/200] step 4850 | loss=1.9658 pred=0.0171 kl=2.4504 recon=0.5016 spec=0.6140 | grad=1.18 | 25.9s
Epoch 180/200 | avg_loss=1.9799 pred=0.0172 kl=2.4502 recon=0.5032 spec=0.6212 | 40.7s
  Saved checkpoint: /home/ubuntu/lewm-tts/output_v3c/checkpoints/epoch_0180.pt
Epoch 181/200 | avg_loss=1.9833 pred=0.0172 kl=2.4504 recon=0.5033 spec=0.6234 | 40.6s
  [182/200] step 4900 | loss=1.9646 pred=0.0171 kl=2.4547 recon=0.5069 spec=0.6062 | grad=1.04 | 20.0s
Epoch 182/200 | avg_loss=1.9778 pred=0.0172 kl=2.4518 recon=0.5033 spec=0.6197 | 40.9s
  New best model (loss=1.9778)
Epoch 183/200 | avg_loss=1.9787 pred=0.0172 kl=2.4520 recon=0.5033 spec=0.6203 | 40.8s
  [184/200] step 4950 | loss=1.9813 pred=0.0172 kl=2.4491 recon=0.5088 spec=0.6146 | grad=7.07 | 14.0s
Epoch 184/200 | avg_loss=1.9772 pred=0.0172 kl=2.4519 recon=0.5030 spec=0.6197 | 40.9s
  New best model (loss=1.9772)
Epoch 185/200 | avg_loss=1.9803 pred=0.0172 kl=2.4521 recon=0.5030 spec=0.6218 | 40.5s
  [186/200] step 5000 | loss=1.9795 pred=0.0171 kl=2.4528 recon=0.5049 spec=0.6187 | grad=1.56 | 7.9s
Epoch 186/200 | avg_loss=1.9789 pred=0.0171 kl=2.4531 recon=0.5029 spec=0.6210 | 40.7s
Epoch 187/200 | avg_loss=1.9771 pred=0.0171 kl=2.4528 recon=0.5028 spec=0.6199 | 40.8s
  New best model (loss=1.9771)
  [188/200] step 5050 | loss=1.9892 pred=0.0170 kl=2.4546 recon=0.5050 spec=0.6252 | grad=0.89 | 2.0s
Epoch 188/200 | avg_loss=1.9775 pred=0.0172 kl=2.4517 recon=0.5027 spec=0.6203 | 40.7s
  [189/200] step 5100 | loss=1.9634 pred=0.0172 kl=2.4512 recon=0.5033 spec=0.6101 | grad=3.41 | 36.1s
Epoch 189/200 | avg_loss=1.9782 pred=0.0172 kl=2.4519 recon=0.5027 spec=0.6207 | 40.5s
Epoch 190/200 | avg_loss=1.9801 pred=0.0171 kl=2.4521 recon=0.5027 spec=0.6219 | 40.7s
  Saved checkpoint: /home/ubuntu/lewm-tts/output_v3c/checkpoints/epoch_0190.pt
  [191/200] step 5150 | loss=1.9591 pred=0.0170 kl=2.4533 recon=0.5012 spec=0.6101 | grad=1.52 | 30.2s
Epoch 191/200 | avg_loss=1.9790 pred=0.0171 kl=2.4524 recon=0.5026 spec=0.6214 | 40.6s
Epoch 192/200 | avg_loss=1.9759 pred=0.0171 kl=2.4522 recon=0.5026 spec=0.6193 | 40.9s
  New best model (loss=1.9759)
  [193/200] step 5200 | loss=1.9515 pred=0.0170 kl=2.4544 recon=0.5019 spec=0.6041 | grad=1.74 | 24.2s
Epoch 193/200 | avg_loss=1.9763 pred=0.0171 kl=2.4524 recon=0.5025 spec=0.6197 | 40.5s
Epoch 194/200 | avg_loss=1.9755 pred=0.0171 kl=2.4521 recon=0.5025 spec=0.6192 | 40.6s
  New best model (loss=1.9755)
  [195/200] step 5250 | loss=1.9719 pred=0.0171 kl=2.4506 recon=0.5019 spec=0.6177 | grad=1.07 | 18.4s
Epoch 195/200 | avg_loss=1.9780 pred=0.0171 kl=2.4521 recon=0.5026 spec=0.6208 | 40.6s
Epoch 196/200 | avg_loss=1.9765 pred=0.0171 kl=2.4522 recon=0.5025 spec=0.6199 | 40.7s
  [197/200] step 5300 | loss=1.9615 pred=0.0171 kl=2.4543 recon=0.5012 spec=0.6117 | grad=0.89 | 12.5s
Epoch 197/200 | avg_loss=1.9785 pred=0.0171 kl=2.4519 recon=0.5026 spec=0.6211 | 40.6s
Epoch 198/200 | avg_loss=1.9734 pred=0.0171 kl=2.4519 recon=0.5026 spec=0.6177 | 40.9s
  New best model (loss=1.9734)
  [199/200] step 5350 | loss=1.9805 pred=0.0172 kl=2.4502 recon=0.5032 spec=0.6216 | grad=1.39 | 6.6s
Epoch 199/200 | avg_loss=1.9756 pred=0.0171 kl=2.4519 recon=0.5025 spec=0.6193 | 40.9s
  [200/200] step 5400 | loss=1.9890 pred=0.0171 kl=2.4549 recon=0.5023 spec=0.6285 | grad=1.26 | 40.6s
Epoch 200/200 | avg_loss=1.9766 pred=0.0171 kl=2.4520 recon=0.5026 spec=0.6199 | 40.6s
  Saved checkpoint: /home/ubuntu/lewm-tts/output_v3c/checkpoints/epoch_0200.pt

Training complete. Best loss: 1.9734


Now let me look at the v3b log to see what happened later, the v3c samples, and the preprocess.py to understand the full picture. Let me also check what configs were used for each run.


Let me check the config files and the v2 directory for more context.


Bash

List v3c output directory


Show v3c config


cat /home/ubuntu/lewm-tts/output_v3b/config.json 2>/dev/null && echo "---" && cat /home/ubuntu/lewm-tts/output_v3/config.json 2>/dev/null

Show v3b and v3 configs


List v2 directory


     1→"""
     2→Data preprocessing for LeWM TTS.
     3→- Loads Modi dataset from HuggingFace Arrow format
     4→- Segments long clips into 3-15 second chunks using silence detection
     5→- Extracts mel spectrograms
     6→- Saves processed dataset
     7→"""
     8→
     9→import os
    10→import json
    11→import numpy as np
    12→import torch
    13→import torchaudio
    14→from datasets import load_from_disk
    15→from pathlib import Path
    16→import argparse
    17→
    18→
    19→# ─── Mel spectrogram config ───
    20→MEL_CONFIG = {
    21→    "sample_rate": 24000,
    22→    "n_fft": 1024,
    23→    "hop_length": 256,
    24→    "n_mels": 100,
    25→    "f_min": 0,
    26→    "f_max": None,      # full bandwidth (matches Vocos mel-24khz)
    27→    "power": 1.0,
    28→    "clip_val": 1e-7,   # log clamp value (matches Vocos safe_log)
    29→}
    30→
    31→# ─── Segmentation config ───
    32→SEG_CONFIG = {
    33→    "min_duration": 3.0,   # seconds
    34→    "max_duration": 12.0,  # seconds
    35→    "target_duration": 8.0,  # preferred chunk size
    36→    "silence_threshold_db": -40,
    37→    "min_silence_len": 0.3,  # seconds
    38→}
    39→
    40→
    41→def detect_silence_points(waveform: np.ndarray, sr: int,
    42→                          threshold_db: float = -40,
    43→                          min_silence_len: float = 0.3) -> list:
    44→    """Find silence points in audio for splitting."""
    45→    # Convert to dB
    46→    frame_length = int(0.025 * sr)  # 25ms frames
    47→    hop = int(0.010 * sr)           # 10ms hop
    48→
    49→    # RMS energy per frame
    50→    n_frames = 1 + (len(waveform) - frame_length) // hop
    51→    rms_db = []
    52→    for i in range(n_frames):
    53→        start = i * hop
    54→        frame = waveform[start:start + frame_length]
    55→        rms = np.sqrt(np.mean(frame ** 2) + 1e-10)
    56→        rms_db.append(20 * np.log10(rms + 1e-10))
    57→
    58→    rms_db = np.array(rms_db)
    59→    is_silence = rms_db < threshold_db
    60→
    61→    # Find contiguous silence regions
    62→    silence_points = []
    63→    min_frames = int(min_silence_len / 0.010)
    64→
    65→    in_silence = False
    66→    silence_start = 0
    67→    for i, s in enumerate(is_silence):
    68→        if s and not in_silence:
    69→            silence_start = i
    70→            in_silence = True
    71→        elif not s and in_silence:
    72→            if i - silence_start >= min_frames:
    73→                # Midpoint of silence region in samples
    74→                mid_frame = (silence_start + i) // 2
    75→                mid_sample = mid_frame * hop
    76→                silence_points.append(mid_sample)
    77→            in_silence = False
    78→
    79→    return silence_points
    80→
    81→
    82→def segment_audio(waveform: np.ndarray, sr: int, text: str, config: dict) -> list:
    83→    """Segment a long audio clip into shorter chunks at silence boundaries."""
    84→    total_duration = len(waveform) / sr
    85→
    86→    # If already short enough, return as-is
    87→    if total_duration <= config["max_duration"]:
    88→        if total_duration >= config["min_duration"]:
    89→            return [(waveform, text)]
    90→        else:
    91→            return []  # Too short
    92→
    93→    # Find silence points for natural splitting
    94→    silence_points = detect_silence_points(
    95→        waveform, sr,
    96→        threshold_db=config["silence_threshold_db"],
    97→        min_silence_len=config["min_silence_len"]
    98→    )
    99→
   100→    # Add start and end
   101→    split_points = [0] + silence_points + [len(waveform)]
   102→
   103→    # Greedily merge segments to hit target duration
   104→    segments = []
   105→    current_start = 0
   106→
   107→    for i, point in enumerate(split_points[1:], 1):
   108→        current_duration = (point - current_start) / sr
   109→
   110→        if current_duration >= config["target_duration"]:
   111→            # Take this segment
   112→            chunk = waveform[current_start:point]
   113→            chunk_duration = len(chunk) / sr
   114→            if config["min_duration"] <= chunk_duration <= config["max_duration"] * 1.2:
   115→                segments.append(chunk)
   116→            elif chunk_duration > config["max_duration"] * 1.2:
   117→                # Force split at max_duration
   118→                max_samples = int(config["max_duration"] * sr)
   119→                for j in range(0, len(chunk), max_samples):
   120→                    sub = chunk[j:j + max_samples]
   121→                    if len(sub) / sr >= config["min_duration"]:
   122→                        segments.append(sub)
   123→            current_start = point
   124→
   125→    # Handle remaining audio
   126→    if current_start < len(waveform):
   127→        remaining = waveform[current_start:]
   128→        if len(remaining) / sr >= config["min_duration"]:
   129→            segments.append(remaining)
   130→        elif segments:
   131→            # Append to last segment if it won't make it too long
   132→            combined = np.concatenate([segments[-1], remaining])
   133→            if len(combined) / sr <= config["max_duration"] * 1.2:
   134→                segments[-1] = combined
   135→
   136→    # For text: we can't perfectly align without forced alignment,
   137→    # so we use the full text for all segments (will be refined later with forced alignment)
   138→    # For now, distribute text roughly proportionally
   139→    results = []
   140→    total_samples = sum(len(s) for s in segments)
   141→    text_chars = list(text)
   142→    char_idx = 0
   143→
   144→    for seg in segments:
   145→        proportion = len(seg) / total_samples
   146→        n_chars = max(1, int(len(text) * proportion))
   147→        seg_text = text[char_idx:char_idx + n_chars]
   148→        char_idx += n_chars
   149→        results.append((seg, seg_text.strip()))
   150→
   151→    return results
   152→
   153→
   154→def extract_mel(waveform: torch.Tensor, config: dict) -> torch.Tensor:
   155→    """Extract log-mel spectrogram (matches Vocos mel-24khz config)."""
   156→    mel_kwargs = dict(
   157→        sample_rate=config["sample_rate"],
   158→        n_fft=config["n_fft"],
   159→        hop_length=config["hop_length"],
   160→        n_mels=config["n_mels"],
   161→        power=config["power"],
   162→    )
   163→    if config.get("f_min") is not None:
   164→        mel_kwargs["f_min"] = config["f_min"]
   165→    if config.get("f_max") is not None:
   166→        mel_kwargs["f_max"] = config["f_max"]
   167→
   168→    mel_transform = torchaudio.transforms.MelSpectrogram(**mel_kwargs)
   169→    mel = mel_transform(waveform)
   170→    clip_val = config.get("clip_val", 1e-7)
   171→    log_mel = torch.log(torch.clamp(mel, min=clip_val))
   172→    return log_mel
   173→
   174→
   175→def trim_silence(waveform: np.ndarray, sr: int, threshold_db: float = -45) -> np.ndarray:
   176→    """Trim leading and trailing silence."""
   177→    # Convert to amplitude threshold
   178→    threshold = 10 ** (threshold_db / 20)
   179→
   180→    # Find non-silent regions
   181→    abs_wav = np.abs(waveform)
   182→
   183→    # Use a sliding window
   184→    window = int(0.01 * sr)  # 10ms
   185→    for start in range(0, len(abs_wav) - window, window):
   186→        if np.max(abs_wav[start:start + window]) > threshold:
   187→            break
   188→
   189→    for end in range(len(abs_wav), window, -window):
   190→        if np.max(abs_wav[end - window:end]) > threshold:
   191→            break
   192→
   193→    # Add small padding
   194→    pad = int(0.05 * sr)  # 50ms padding
   195→    start = max(0, start - pad)
   196→    end = min(len(waveform), end + pad)
   197→
   198→    return waveform[start:end]
   199→
   200→
   201→def preprocess_dataset(input_path: str, output_path: str):
   202→    """Main preprocessing pipeline."""
   203→    print(f"Loading dataset from {input_path}...")
   204→    ds = load_from_disk(input_path)
   205→
   206→    output_dir = Path(output_path)
   207→    audio_dir = output_dir / "audio"
   208→    mel_dir = output_dir / "mels"
   209→    audio_dir.mkdir(parents=True, exist_ok=True)
   210→    mel_dir.mkdir(parents=True, exist_ok=True)
   211→
   212→    manifest = []
   213→    total_duration = 0.0
   214→    idx = 0
   215→
   216→    for i in range(len(ds)):
   217→        sample = ds[i]
   218→        audio_data = sample["audio"]
   219→        waveform = np.array(audio_data["array"], dtype=np.float32)
   220→        sr = audio_data["sampling_rate"]
   221→        text = sample["text"]
   222→
   223→        # Trim silence
   224→        waveform = trim_silence(waveform, sr)
   225→
   226→        # Segment into chunks
   227→        segments = segment_audio(waveform, sr, text, SEG_CONFIG)
   228→
   229→        for seg_wav, seg_text in segments:
   230→            if not seg_text:
   231→                continue
   232→
   233→            duration = len(seg_wav) / sr
   234→            total_duration += duration
   235→
   236→            # Save audio
   237→            audio_path = audio_dir / f"{idx:06d}.wav"
   238→            wav_tensor = torch.from_numpy(seg_wav).unsqueeze(0)
   239→            torchaudio.save(str(audio_path), wav_tensor, sr)
   240→
   241→            # Extract and save mel
   242→            mel = extract_mel(wav_tensor, MEL_CONFIG)
   243→            mel_path = mel_dir / f"{idx:06d}.pt"
   244→            torch.save(mel.squeeze(0), str(mel_path))  # [n_mels, T]
   245→
   246→            manifest.append({
   247→                "id": idx,
   248→                "audio_path": str(audio_path),
   249→                "mel_path": str(mel_path),
   250→                "text": seg_text,
   251→                "duration": round(duration, 3),
   252→                "mel_frames": mel.shape[-1],
   253→            })
   254→
   255→            idx += 1
   256→
   257→        if (i + 1) % 100 == 0:
   258→            print(f"  Processed {i+1}/{len(ds)} samples, {idx} segments, {total_duration/3600:.2f}h total")
   259→
   260→    # Save manifest
   261→    manifest_path = output_dir / "manifest.json"
   262→    with open(manifest_path, "w", encoding="utf-8") as f:
   263→        json.dump(manifest, f, ensure_ascii=False, indent=2)
   264→
   265→    # Save config
   266→    config = {
   267→        "mel": MEL_CONFIG,
   268→        "segmentation": SEG_CONFIG,
   269→        "total_segments": idx,
   270→        "total_duration_hours": round(total_duration / 3600, 2),
   271→        "original_samples": len(ds),
   272→    }
   273→    config_path = output_dir / "config.json"
   274→    with open(config_path, "w") as f:
   275→        json.dump(config, f, indent=2)
   276→
   277→    print(f"\nDone! {idx} segments, {total_duration/3600:.2f} hours")
   278→    print(f"Manifest: {manifest_path}")
   279→    print(f"Config: {config_path}")
   280→
   281→
   282→if __name__ == "__main__":
   283→    parser = argparse.ArgumentParser()
   284→    parser.add_argument("--input", default="/home/ubuntu/modi_dataset")
   285→    parser.add_argument("--output", default="/home/ubuntu/lewm-tts/processed_data")
   286→    args = parser.parse_args()
   287→    preprocess_dataset(args.input, args.output)
   288→

"""
Data preprocessing for LeWM TTS.
- Loads Modi dataset from HuggingFace Arrow format
- Segments long clips into 3-15 second chunks using silence detection
- Extracts mel spectrograms
- Saves processed dataset
"""

import os
import json
import numpy as np
import torch
import torchaudio
from datasets import load_from_disk
from pathlib import Path
import argparse

# ─── Mel spectrogram config ───
MEL_CONFIG = {
    "sample_rate": 24000,
    "n_fft": 1024,
    "hop_length": 256,
    "n_mels": 100,
    "f_min": 0,
    "f_max": None,      # full bandwidth (matches Vocos mel-24khz)
    "power": 1.0,
    "clip_val": 1e-7,   # log clamp value (matches Vocos safe_log)
}

# ─── Segmentation config ───
SEG_CONFIG = {
    "min_duration": 3.0,   # seconds
    "max_duration": 12.0,  # seconds
    "target_duration": 8.0,  # preferred chunk size
    "silence_threshold_db": -40,
    "min_silence_len": 0.3,  # seconds
}

def detect_silence_points(waveform: np.ndarray, sr: int,
                          threshold_db: float = -40,
                          min_silence_len: float = 0.3) -> list:
    """Find silence points in audio for splitting."""
    # Convert to dB
    frame_length = int(0.025 * sr)  # 25ms frames
    hop = int(0.010 * sr)           # 10ms hop

    # RMS energy per frame
    n_frames = 1 + (len(waveform) - frame_length) // hop
    rms_db = []
    for i in range(n_frames):
        start = i * hop
        frame = waveform[start:start + frame_length]
        rms = np.sqrt(np.mean(frame ** 2) + 1e-10)
        rms_db.append(20 * np.log10(rms + 1e-10))

    rms_db = np.array(rms_db)
    is_silence = rms_db < threshold_db

    # Find contiguous silence regions
    silence_points = []
    min_frames = int(min_silence_len / 0.010)

    in_silence = False
    silence_start = 0
    for i, s in enumerate(is_silence):
        if s and not in_silence:
            silence_start = i
            in_silence = True
        elif not s and in_silence:
            if i - silence_start >= min_frames:
                # Midpoint of silence region in samples
                mid_frame = (silence_start + i) // 2
                mid_sample = mid_frame * hop
                silence_points.append(mid_sample)
            in_silence = False

    return silence_points

def segment_audio(waveform: np.ndarray, sr: int, text: str, config: dict) -> list:
    """Segment a long audio clip into shorter chunks at silence boundaries."""
    total_duration = len(waveform) / sr

    # If already short enough, return as-is
    if total_duration <= config["max_duration"]:
        if total_duration >= config["min_duration"]:
            return [(waveform, text)]
        else:
            return []  # Too short

    # Find silence points for natural splitting
    silence_points = detect_silence_points(
        waveform, sr,
        threshold_db=config["silence_threshold_db"],
        min_silence_len=config["min_silence_len"]
    )

    # Add start and end
    split_points = [0] + silence_points + [len(waveform)]

    # Greedily merge segments to hit target duration
    segments = []
    current_start = 0

    for i, point in enumerate(split_points[1:], 1):
        current_duration = (point - current_start) / sr

        if current_duration >= config["target_duration"]:
            # Take this segment
            chunk = waveform[current_start:point]
            chunk_duration = len(chunk) / sr
            if config["min_duration"] <= chunk_duration <= config["max_duration"] * 1.2:
                segments.append(chunk)
            elif chunk_duration > config["max_duration"] * 1.2:
                # Force split at max_duration
                max_samples = int(config["max_duration"] * sr)
                for j in range(0, len(chunk), max_samples):
                    sub = chunk[j:j + max_samples]
                    if len(sub) / sr >= config["min_duration"]:
                        segments.append(sub)
            current_start = point

    # Handle remaining audio
    if current_start < len(waveform):
        remaining = waveform[current_start:]
        if len(remaining) / sr >= config["min_duration"]:
            segments.append(remaining)
        elif segments:
            # Append to last segment if it won't make it too long
            combined = np.concatenate([segments[-1], remaining])
            if len(combined) / sr <= config["max_duration"] * 1.2:
                segments[-1] = combined

    # For text: we can't perfectly align without forced alignment,
    # so we use the full text for all segments (will be refined later with forced alignment)
    # For now, distribute text roughly proportionally
    results = []
    total_samples = sum(len(s) for s in segments)
    text_chars = list(text)
    char_idx = 0

    for seg in segments:
        proportion = len(seg) / total_samples
        n_chars = max(1, int(len(text) * proportion))
        seg_text = text[char_idx:char_idx + n_chars]
        char_idx += n_chars
        results.append((seg, seg_text.strip()))

    return results

def extract_mel(waveform: torch.Tensor, config: dict) -> torch.Tensor:
    """Extract log-mel spectrogram (matches Vocos mel-24khz config)."""
    mel_kwargs = dict(
        sample_rate=config["sample_rate"],
        n_fft=config["n_fft"],
        hop_length=config["hop_length"],
        n_mels=config["n_mels"],
        power=config["power"],
    )
    if config.get("f_min") is not None:
        mel_kwargs["f_min"] = config["f_min"]
    if config.get("f_max") is not None:
        mel_kwargs["f_max"] = config["f_max"]

    mel_transform = torchaudio.transforms.MelSpectrogram(**mel_kwargs)
    mel = mel_transform(waveform)
    clip_val = config.get("clip_val", 1e-7)
    log_mel = torch.log(torch.clamp(mel, min=clip_val))
    return log_mel

def trim_silence(waveform: np.ndarray, sr: int, threshold_db: float = -45) -> np.ndarray:
    """Trim leading and trailing silence."""
    # Convert to amplitude threshold
    threshold = 10 ** (threshold_db / 20)

    # Find non-silent regions
    abs_wav = np.abs(waveform)

    # Use a sliding window
    window = int(0.01 * sr)  # 10ms
    for start in range(0, len(abs_wav) - window, window):
        if np.max(abs_wav[start:start + window]) > threshold:
            break

    for end in range(len(abs_wav), window, -window):
        if np.max(abs_wav[end - window:end]) > threshold:
            break

    # Add small padding
    pad = int(0.05 * sr)  # 50ms padding
    start = max(0, start - pad)
    end = min(len(waveform), end + pad)

    return waveform[start:end]

def preprocess_dataset(input_path: str, output_path: str):
    """Main preprocessing pipeline."""
    print(f"Loading dataset from {input_path}...")
    ds = load_from_disk(input_path)

    output_dir = Path(output_path)
    audio_dir = output_dir / "audio"
    mel_dir = output_dir / "mels"
    audio_dir.mkdir(parents=True, exist_ok=True)
    mel_dir.mkdir(parents=True, exist_ok=True)

    manifest = []
    total_duration = 0.0
    idx = 0

    for i in range(len(ds)):
        sample = ds[i]
        audio_data = sample["audio"]
        waveform = np.array(audio_data["array"], dtype=np.float32)
        sr = audio_data["sampling_rate"]
        text = sample["text"]

        # Trim silence
        waveform = trim_silence(waveform, sr)

        # Segment into chunks
        segments = segment_audio(waveform, sr, text, SEG_CONFIG)

        for seg_wav, seg_text in segments:
            if not seg_text:
                continue

            duration = len(seg_wav) / sr
            total_duration += duration

            # Save audio
            audio_path = audio_dir / f"{idx:06d}.wav"
            wav_tensor = torch.from_numpy(seg_wav).unsqueeze(0)
            torchaudio.save(str(audio_path), wav_tensor, sr)

            # Extract and save mel
            mel = extract_mel(wav_tensor, MEL_CONFIG)
            mel_path = mel_dir / f"{idx:06d}.pt"
            torch.save(mel.squeeze(0), str(mel_path))  # [n_mels, T]

            manifest.append({
                "id": idx,
                "audio_path": str(audio_path),
                "mel_path": str(mel_path),
                "text": seg_text,
                "duration": round(duration, 3),
                "mel_frames": mel.shape[-1],
            })

            idx += 1

        if (i + 1) % 100 == 0:
            print(f"  Processed {i+1}/{len(ds)} samples, {idx} segments, {total_duration/3600:.2f}h total")

    # Save manifest
    manifest_path = output_dir / "manifest.json"
    with open(manifest_path, "w", encoding="utf-8") as f:
        json.dump(manifest, f, ensure_ascii=False, indent=2)

    # Save config
    config = {
        "mel": MEL_CONFIG,
        "segmentation": SEG_CONFIG,
        "total_segments": idx,
        "total_duration_hours": round(total_duration / 3600, 2),
        "original_samples": len(ds),
    }
    config_path = output_dir / "config.json"
    with open(config_path, "w") as f:
        json.dump(config, f, indent=2)

    print(f"\nDone! {idx} segments, {total_duration/3600:.2f} hours")
    print(f"Manifest: {manifest_path}")
    print(f"Config: {config_path}")

if __name__ == "__main__":
    parser = argparse.ArgumentParser()
    parser.add_argument("--input", default="/home/ubuntu/modi_dataset")
    parser.add_argument("--output", default="/home/ubuntu/lewm-tts/processed_data")
    args = parser.parse_args()
    preprocess_dataset(args.input, args.output)


checkpoints
config.json
logs


dataset.py
inference.py
model.py
output
preprocess.py
processed_data
__pycache__
train.py


{
  "d_model": 256,
  "nhead": 4,
  "n_mels": 100,
  "text_vocab_size": 256,
  "text_encoder_layers": 4,
  "audio_encoder_layers": 4,
  "predictor_layers": 6,
  "dropout": 0.1,
  "kl_weight": 0.01,
  "recon_weight": 2.0,
  "spectral_weight": 1.5
}---
{
  "d_model": 256,
  "nhead": 4,
  "n_mels": 100,
  "text_vocab_size": 256,
  "text_encoder_layers": 4,
  "audio_encoder_layers": 4,
  "predictor_layers": 6,
  "dropout": 0.1,
  "kl_weight": 0.1,
  "recon_weight": 1.0,
  "spectral_weight": 0.5
}


Now I have a thorough understanding of the project. Let me analyze the training logs and identify the issues.

Key observations from training logs:

**v3 (original)**: kl_weight=0.1, recon_weight=1.0, spectral_weight=0.5, downsample_factor=4 (default)
- 24M params
- total_loss went from 6.4 → 1.21 by epoch 49 (much lower!)
- KL stabilized around 1.17-1.18

**v3b**: kl_weight=0.01, recon_weight=2.0, spectral_weight=1.5, downsample_factor=4 (default)

- 24M params
- total_loss went from 13.65 → 2.25 by epoch 95
- KL grew much higher: ~2.44

**v3c**: kl_weight=0.01, recon_weight=2.0, spectral_weight=1.5, downsample_factor=2
- 21.11M params (smaller due to downsample_factor=2)
- total_loss went from 11.98 → 1.97 by epoch 200
- KL grew even higher: ~2.45

The fundamental problems I can see: reducing the KL weight from 0.1 to 0.01 caused the KL divergence to explode, suggesting the model isn't being properly regularized toward the prior distribution. Meanwhile, increasing reconstruction and spectral weights didn't help stabilize training—it just made the loss landscape harder to navigate.

The prediction loss is suspiciously low at 0.017, which means the JEPA predictor is essentially learning a near-identity mapping instead of actually predicting future embeddings. This happens when the latent space is too rich and unconstrained—consecutive frames are similar enough that the predictor can just copy the input rather than learning meaningful temporal dynamics.

The core issue is architectural: unlike the original JEPA paper which uses an EMA target encoder to prevent collapse, this model has only one encoder. When the predictor learns to map z[t] to z[t+1] from the same encoder on similar mel frames, the MSE loss drives them nearly identical, so the predictor learns almost nothing useful. During autoregressive inference, this becomes catastrophic—the model feeds its own predictions back, but since it hasn't learned real transformations, it just drifts and generates incoherent audio.

The loss weighting makes this worse: reconstruction and spectral losses completely dominate (recon_weight=2.0, spectral_weight=1.5), while the prediction loss is negligible at 0.86% of total loss. The model optimizes almost entirely for reconstruction rather than learning meaningful predictive dynamics. v3c's smaller parameter count and less temporal compression only amplified the redundancy problem, making the prediction task even more trivial. I'm checking what v3 did differently with its lower loss weights to see if that's where things worked better.

Looking at v3's metrics, the KL divergence settled lower (~1.18 vs 2.45 in the current version) and prediction loss was higher (0.057 vs 0.017), suggesting a better-constrained latent space but still fundamentally broken. The core problem is that the JEPA predictor is learning near-identity mappings because consecutive embeddings are too similar—they come from overlapping mel frames. I need to either increase temporal stride to make consecutive embeddings represent different content, use a separate target encoder with EMA updates like proper JEPA, or rethink the approach entirely.

Beyond that, the KL weight at 0.01 is too weak to properly regularize the latent space, the prediction loss is underweighted relative to reconstruction and spectral losses, there's no exposure bias mitigation during training (the predictor never sees its own errors), and initializing with random noise for the start embedding creates instability that compounds through autoregressive generation.

Let me see if there are audio samples to evaluate.


Now let me listen to some of the samples to understand quality, and check the v3b log tail.


ls -la /home/ubuntu/lewm-tts/samples_v3c/ 2>/dev/null && echo "---" && ls -la /home/ubuntu/lewm-tts/sample_*.wav

List sample audio files


python3 -c "
import json
m = json.load(open('/home/ubuntu/lewm-tts/processed_data_100mel/manifest.json'))
print(f'Total samples: {len(m)}')
# Show duration distribution
durs = [e['duration'] for e in m]
import numpy as np
durs = np.array(durs)
print(f'Duration: min={durs.min():.1f}s, max={durs.max():.1f}s, mean={durs.mean():.1f}s, total={durs.sum()/3600:.2f}h')
# Show mel frames distribution
frames = [e['mel_frames'] for e in m]
frames = np.array(frames)
print(f'Mel frames: min={frames.min()}, max={frames.max()}, mean={frames.mean():.0f}')
# Text length distribution
texts = [len(e['text'].encode('utf-8')) for e in m]
texts = np.array(texts)
print(f'Text tokens: min={texts.min()}, max={texts.max()}, mean={texts.mean():.0f}')
# Show a few examples
for e in m[:3]:
    print(f'  id={e[\"id\"]}: {e[\"duration\"]}s, {e[\"mel_frames\"]} frames, text={e[\"text\"][:80]}...')
"

Analyze dataset statistics


   220→  [95/200] step 2550 | loss=2.2863 pred=0.0187 kl=2.4448 recon=0.5999 spec=0.6956 | grad=6.09 | 12.2s
   221→Epoch 95/200 | avg_loss=2.2549 pred=0.0190 kl=2.4427 recon=0.5989 spec=0.6758 | 26.6s
   222→

  [95/200] step 2550 | loss=2.2863 pred=0.0187 kl=2.4448 recon=0.5999 spec=0.6956 | grad=6.09 | 12.2s
Epoch 95/200 | avg_loss=2.2549 pred=0.0190 kl=2.4427 recon=0.5989 spec=0.6758 | 26.6s