APEX FusedRMSNorm not available, using native implementation
/home/ubuntu/vibevoice/vibevoice/processor/vibevoice_asr_processor.py:23: UserWarning: audio_utils not available, will fall back to soundfile for audio loading
  warnings.warn("audio_utils not available, will fall back to soundfile for audio loading")
The tokenizer class you load from this checkpoint is not the same type as the class this function is called from. It may result in unexpected tokenization. 
The tokenizer class you load from this checkpoint is 'Qwen2Tokenizer'. 
The class this function is called from is 'VibeVoiceTextTokenizerFast'.

Loading checkpoint shards:   0%|          | 0/3 [00:00<?, ?it/s]
Loading checkpoint shards:  33%|███▎      | 1/3 [00:00<00:01,  1.75it/s]
Loading checkpoint shards:  67%|██████▋   | 2/3 [00:01<00:00,  1.91it/s]
Loading checkpoint shards: 100%|██████████| 3/3 [00:01<00:00,  2.03it/s]
Loading checkpoint shards: 100%|██████████| 3/3 [00:01<00:00,  1.97it/s]
warmup                                        RTF=1.369x TTFB=669ms Audio=1.60s GPU=19%
[1] Baseline (10 steps)                       RTF=0.965x TTFB=144ms Audio=10.93s GPU=27%
[1] Baseline (20 steps)                       RTF=1.154x TTFB=167ms Audio=10.93s GPU=28%

Setting up CUDA graph for diffusion head...
CUDA graph ready.
[2] CUDA graph diffusion (10 steps)           RTF=0.805x TTFB=123ms Audio=10.67s GPU=31%
[2] CUDA graph diffusion (20 steps)           RTF=0.847x TTFB=129ms Audio=10.67s GPU=35%

Compiling LM backbone...
W0314 21:23:26.046000 1947406 torch/_dynamo/variables/tensor.py:776] [0/0] Graph break from `Tensor.item()`, consider setting:
W0314 21:23:26.046000 1947406 torch/_dynamo/variables/tensor.py:776] [0/0]     torch._dynamo.config.capture_scalar_outputs = True
W0314 21:23:26.046000 1947406 torch/_dynamo/variables/tensor.py:776] [0/0] or:
W0314 21:23:26.046000 1947406 torch/_dynamo/variables/tensor.py:776] [0/0]     env TORCHDYNAMO_CAPTURE_SCALAR_OUTPUTS=1
W0314 21:23:26.046000 1947406 torch/_dynamo/variables/tensor.py:776] [0/0] to include these operations in the captured graph.
W0314 21:23:26.046000 1947406 torch/_dynamo/variables/tensor.py:776] [0/0] 
W0314 21:23:26.046000 1947406 torch/_dynamo/variables/tensor.py:776] [0/0] Graph break: from user code at:
W0314 21:23:26.046000 1947406 torch/_dynamo/variables/tensor.py:776] [0/0]   File "/home/ubuntu/.local/lib/python3.10/site-packages/transformers/utils/generic.py", line 965, in wrapper
W0314 21:23:26.046000 1947406 torch/_dynamo/variables/tensor.py:776] [0/0]     output = func(self, *args, **kwargs)
W0314 21:23:26.046000 1947406 torch/_dynamo/variables/tensor.py:776] [0/0]   File "/home/ubuntu/.local/lib/python3.10/site-packages/transformers/models/qwen2/modeling_qwen2.py", line 519, in forward
W0314 21:23:26.046000 1947406 torch/_dynamo/variables/tensor.py:776] [0/0]     causal_mask = self._update_causal_mask(
W0314 21:23:26.046000 1947406 torch/_dynamo/variables/tensor.py:776] [0/0]   File "/home/ubuntu/.local/lib/python3.10/site-packages/transformers/models/qwen2/modeling_qwen2.py", line 589, in _update_causal_mask
W0314 21:23:26.046000 1947406 torch/_dynamo/variables/tensor.py:776] [0/0]     is_padding_right = attention_mask[:, -1].sum().item() != input_tensor.size()[0]
W0314 21:23:26.046000 1947406 torch/_dynamo/variables/tensor.py:776] [0/0] 
W0314 21:23:26.046000 1947406 torch/_dynamo/variables/tensor.py:776] [0/0] 
W0314 21:23:29.896000 1947406 torch/_dynamo/convert_frame.py:844] [7/8] torch._dynamo hit config.cache_size_limit (8)
W0314 21:23:29.896000 1947406 torch/_dynamo/convert_frame.py:844] [7/8]    function: 'forward' (/home/ubuntu/.local/lib/python3.10/site-packages/transformers/models/qwen2/modeling_qwen2.py:152)
W0314 21:23:29.896000 1947406 torch/_dynamo/convert_frame.py:844] [7/8]    last reason: 7/0: L['self'].layer_idx == 0                                    
W0314 21:23:29.896000 1947406 torch/_dynamo/convert_frame.py:844] [7/8] To log all recompilation reasons, use TORCH_LOGS="recompiles".
W0314 21:23:29.896000 1947406 torch/_dynamo/convert_frame.py:844] [7/8] To diagnose recompilation issues, see https://pytorch.org/docs/main/torch.compiler_troubleshooting.html.
W0314 21:23:32.730000 1947406 torch/_dynamo/convert_frame.py:844] [6/8] torch._dynamo hit config.cache_size_limit (8)
W0314 21:23:32.730000 1947406 torch/_dynamo/convert_frame.py:844] [6/8]    function: 'forward' (/home/ubuntu/.local/lib/python3.10/site-packages/transformers/models/qwen2/modeling_qwen2.py:245)
W0314 21:23:32.730000 1947406 torch/_dynamo/convert_frame.py:844] [6/8]    last reason: 6/0: len(L['kwargs']) == 5                                       
W0314 21:23:32.730000 1947406 torch/_dynamo/convert_frame.py:844] [6/8] To log all recompilation reasons, use TORCH_LOGS="recompiles".
W0314 21:23:32.730000 1947406 torch/_dynamo/convert_frame.py:844] [6/8] To diagnose recompilation issues, see https://pytorch.org/docs/main/torch.compiler_troubleshooting.html.
compile warmup 1                              RTF=3.056x TTFB=7067ms Audio=3.20s GPU=8%
compile warmup 2                              RTF=0.838x TTFB=122ms Audio=3.20s GPU=30%
[3] CUDA graph diff + compile LM (10 steps)   RTF=0.826x TTFB=546ms Audio=12.27s GPU=30%
[3] CUDA graph diff + compile LM (20 steps)   RTF=0.851x TTFB=128ms Audio=10.93s GPU=34%
[FINAL] Best config 20 steps                  RTF=0.851x TTFB=130ms Audio=10.93s GPU=34%
Saved: samples_cuda_graph_v2/modi_best.wav