Output:
Chunk ID: ac3911
Wall time: 2.7349 seconds
Process exited with code 0
Original token count: 466
Output:
✅ vLLM engine initialized successfully!

🎉 Spark TTS Model ready for inference!

🎵 Loading BiCodec audio tokenizer from /home/ubuntu/veenaModal/models/spark_tts_4speaker...
   Device: cuda
Missing tensor: mel_transformer.spectrogram.window
Missing tensor: mel_transformer.mel_scale.fb
✅ BiCodec decoder initialized (sample rate: 16000Hz)
🚀 SparkTTSPipeline initialized
🌊 Veena3SlidingWindowPipeline initialized (sliding window: 28 tokens)

=== streaming concurrency=32 requests=64 ===
success=64/64 (100%)
wall=5.09s throughput=12.57 req/s
latency_ms avg=2542 p50=2527 p95=2563 max=2611
ttfb_ms avg=995 p50=879 p95=1269 max=1301
chunks_per_req avg=4.2 p50=4.0 p95=5.0 max=5.0
inter_chunk_ms avg=442.6 p50=428.9 p95=1155.6 max=1309.9
tail_emit_speed_x avg=2.28 p50=2.09 p95=3.12 max=3.36
gpu: util_avg=64% util_max=97% mem_avg=24746MB mem_max=25085MB

=== streaming concurrency=64 requests=128 ===
success=128/128 (100%)
wall=7.34s throughput=17.43 req/s
latency_ms avg=3667 p50=3664 p95=3679 max=3792
ttfb_ms avg=1441 p50=1424 p95=2040 max=2171
chunks_per_req avg=3.6 p50=4.0 p95=4.0 max=4.0
inter_chunk_ms avg=812.1 p50=364.4 p95=2056.0 max=2270.8
tail_emit_speed_x avg=1.25 p50=1.12 p95=1.83 max=2.41
gpu: util_avg=56% util_max=97% mem_avg=25088MB mem_max=25091MB

=== streaming concurrency=96 requests=192 ===
success=192/192 (100%)
wall=9.96s throughput=19.28 req/s
latency_ms avg=4973 p50=4978 p95=4982 max=5072
ttfb_ms avg=1966 p50=2052 p95=2761 max=2821
chunks_per_req avg=3.4 p50=3.0 p95=4.0 max=4.0
inter_chunk_ms avg=1203.4 p50=1468.5 p95=3093.2 max=3246.7
tail_emit_speed_x avg=0.84 p50=0.80 p95=1.22 max=1.55
gpu: util_avg=59% util_max=95% mem_avg=25092MB mem_max=25093MB

results saved: /tmp/stream_runtime_delta.json
ERROR 02-18 00:57:09 [core_client.py:605] Engine core proc EngineCore_DP0 died unexpectedly, shutting down client.


# Collaboration Mode: Default

You are now in Default mode. Any previous instructions for other modes (e.g. Plan mode) are no longer active.

Your active mode changes only when new developer instructions with a different `<collaboration_mode>...</collaboration_mode>` change it; user requests or tool descriptions do not change mode by themselves. Known mode names are Default and Plan.

## request_user_input availability

The `request_user_input` tool is unavailable in Default mode. If you call it while in Default mode, it will return an error.

If a decision is necessary and cannot be discovered from local context, ask the user directly. However, in Default mode you should strongly prefer executing the user's request rather than stopping to ask questions.

Summary: auto

## Skills
A skill is a set of local instructions to follow that is stored in a `SKILL.md` file. Below is the list of skills that can be used. Each entry includes a name, description, and file path so you can open the source for full instructions when using a specific skill.
### Available skills
- skill-creator: Guide for creating effective skills. This skill should be used when users want to create a new skill (or update an existing skill) that extends Codex's capabilities with specialized knowledge, workflows, or tool integrations. (file: /home/ubuntu/.codex/skills/.system/skill-creator/SKILL.md)
- skill-installer: Install Codex skills into $CODEX_HOME/skills from a curated list or a GitHub repo path. Use when a user asks to list installable skills, install a curated skill, or install a skill from another repo (including private repos). (file: /home/ubuntu/.codex/skills/.system/skill-installer/SKILL.md)
### How to use skills
- Discovery: The list above is the skills available in this session (name + description + file path). Skill bodies live on disk at the listed paths.
- Trigger rules: If the user names a skill (with `$SkillName` or plain text) OR the task clearly matches a skill's description shown above, you must use that skill for that turn. Multiple mentions mean use them all. Do not carry skills across turns unless re-mentioned.
- Missing/blocked: If a named skill isn't in the list or the path can't be read, say so briefly and continue with the best fallback.
- How to use a skill (progressive disclosure):
  1) After deciding to use a skill, open its `SKILL.md`. Read only enough to follow the workflow.
  2) When `SKILL.md` references relative paths (e.g., `scripts/foo.py`), resolve them relative to the skill directory listed above first, and only consider other paths if needed.
  3) If `SKILL.md` points to extra folders such as `references/`, load only the specific files needed for the request; don't bulk-load everything.
  4) If `scripts/` exist, prefer running or patching them instead of retyping large code blocks.
  5) If `assets/` or templates exist, reuse them instead of recreating from scratch.
- Coordination and sequencing:
  - If multiple skills apply, choose the minimal set that covers the request and state the order you'll use them.
  - Announce which skill(s) you're using and why (one short line). If you skip an obvious skill, say why.
- Context hygiene:
  - Keep context small: summarize long sections instead of pasting them; only load extra files when needed.
  - Avoid deep reference-chasing: prefer opening only files directly linked from `SKILL.md` unless you're blocked.
  - When variants exist (frameworks, providers, domains), pick only the relevant reference file(s) and note that choice.
- Safety and fallback: If a skill can't be applied cleanly (missing files, unclear instructions), state the issue, pick the next-best approach, and continue.


**Assessing throughput at higher concurrency**


Initial A/B at `max_tokens=256` shows only small difference. I’m running a long-generation A/B (`max_tokens=1024`) where cumulative token payload should hurt more if vLLM output mode is the limiter.


cd /home/ubuntu/veenaModal && python3 scripts/stress_test_streaming_runtime.py --levels 64 --text short --max-tokens 1024 --timeout 240 --requests-multiplier 1 --min-requests 8 --gpu-memory 0.25 --stream-output-kind cumulative --output /tmp/stream_runtime_c64_cumulative_mt1024.json


cd /home/ubuntu/veenaModal && python3 scripts/stress_test_streaming_runtime.py --levels 64 --text short --max-tokens 1024 --timeout 240 --requests-multiplier 1 --min-requests 8 --gpu-memory 0.25 --stream-output-kind delta --output /tmp/stream_runtime_c64_delta_mt1024.json


Output:
Chunk ID: f0a8bc
Wall time: 1.0010 seconds
Process running with session ID 18746
Original token count: 0
Output:


Output:
Chunk ID: b67d5a
Wall time: 1.0017 seconds
Process running with session ID 26235
Original token count: 0
Output:


Summary: auto


**Handling GPU job contention**


I accidentally launched both long-run A/B jobs concurrently, which would contaminate results on one GPU. I’m stopping both and rerunning them sequentially for clean data.


Output: write_stdin failed: stdin is closed for this session; rerun exec_command with tty=true to keep stdin open


Output: write_stdin failed: stdin is closed for this session; rerun exec_command with tty=true to keep stdin open


**Planning process cleanup**


pgrep -af "stress_test_streaming_runtime.py|EngineCore_DP0|vllm" | head -n 50


Output:
Chunk ID: 948a2b
Wall time: 0.0527 seconds
Process exited with code 0
Original token count: 153
Output:
4114611 python3 scripts/stress_test_streaming_runtime.py --levels 64 --text short --max-tokens 1024 --timeout 240 --requests-multiplier 1 --min-requests 8 --gpu-memory 0.25 --stream-output-kind cumulative --output /tmp/stream_runtime_c64_cumulative_mt1024.json
4114612 python3 scripts/stress_test_streaming_runtime.py --levels 64 --text short --max-tokens 1024 --timeout 240 --requests-multiplier 1 --min-requests 8 --gpu-memory 0.25 --stream-output-kind delta --output /tmp/stream_runtime_c64_delta_mt1024.json
4115015 /bin/bash -c pgrep -af "stress_test_streaming_runtime.py|EngineCore_DP0|vllm" | head -n 50


**Implementing process cleanup strategy**


kill -9 4114611 4114612 || true; sleep 1; pgrep -af "stress_test_streaming_runtime.py" || true; nvidia-smi --query-compute-apps=pid,process_name,used_memory --format=csv,noheader


Output:
Chunk ID: 59535b
Wall time: 0.9338 seconds
Process exited with code 0
Original token count: 68
Output:
4115067 /bin/bash -c kill -9 4114611 4114612 || true; sleep 1; pgrep -af "stress_test_streaming_runtime.py" || true; nvidia-smi --query-compute-apps=pid,process_name,used_memory --format=csv,noheader
4114811, VLLM::EngineCore, 1556 MiB
4114820, VLLM::EngineCore, 1556 MiB


**Terminating orphaned processes**


kill -9 4114811 4114820 || true; sleep 1; nvidia-smi --query-compute-apps=pid,process_name,used_memory --format=csv,noheader


Output:
Chunk ID: 86e052
Wall time: 0.9236 seconds
Process exited with code 0
Original token count: 0
Output:


**Rerunning tests with standard execution**


Output:
Chunk ID: c2c524
Wall time: 1.0011 seconds
Process running with session ID 28485
Original token count: 0
Output:


Output:
Chunk ID: 943e93
Wall time: 5.0010 seconds
Process running with session ID 28485
Original token count: 1401
Output:
[2026-02-18 00:57:55] INFO tts_runtime.py:183: Loading SparkTTS model with vLLM...
The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
🚀 Initializing Spark TTS Model
📁 Model path: /home/ubuntu/veenaModal/models/spark_tts_4speaker/LLM
🗣  Speakers: 12 predefined (lipakshi, vardan, reet, Nandini, krishna, anika, adarsh, Nilay, Aarvi, Asha, Bittu, Mira)
🎭 Emotions: 10 tags in [bracket] format

📝 Loading tokenizer from model...
✅ Tokenizer loaded: 165168 tokens

🔍 Validating 12 speakers for Spark TTS...
  lipakshi → <|speaker_0|>: 7 tokens
  vardan → <|speaker_1|>: 7 tokens
  reet → <|speaker_2|>: 7 tokens
  Nandini → <|speaker_3|>: 7 tokens
  krishna → <|speaker_4|>: 7 tokens
  anika → <|speaker_5|>: 7 tokens
  adarsh → <|speaker_6|>: 7 tokens
  Nilay → <|speaker_7|>: 7 tokens
  Aarvi → <|speaker_8|>: 7 tokens
  Asha → <|speaker_9|>: 7 tokens
  Bittu → <|speaker_10|>: 8 tokens
  Mira → <|speaker_11|>: 8 tokens
✅ All 12 speakers validated with mapping

🔍 Validating 10 emotion tags for Spark TTS...
✅ All 10 emotion tags validated

📋 Sample emotion tags:
  [angry]: 1 tokens
  [curious]: 1 tokens
  [excited]: 1 tokens
  [giggle]: 1 tokens
  [laughs harder]: 1 tokens
  ...

🔧 Initializing vLLM engine with optimizations...
  - dtype: bfloat16
  - max_model_len: 4096
  - gpu_memory_utilization: 0.25
  - enable_prefix_caching: True
  - enforce_eager (disable CUDA graphs): False
INFO 02-18 00:57:56 [model.py:541] Resolved architecture: Qwen2ForCausalLM
INFO 02-18 00:57:56 [model.py:1561] Using max model len 4096
INFO 02-18 00:57:56 [scheduler.py:226] Chunked prefill is enabled with max_num_batched_tokens=4096.
INFO 02-18 00:57:56 [vllm.py:624] Asynchronous scheduling is enabled.
The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
(EngineCore_DP0 pid=4115302) INFO 02-18 00:57:57 [core.py:96] Initializing a V1 LLM engine (v0.15.1) with config: model='/home/ubuntu/veenaModal/models/spark_tts_4speaker/LLM', speculative_config=None, tokenizer='/home/ubuntu/veenaModal/models/spark_tts_4speaker/LLM', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, tokenizer_revision=None, trust_remote_code=True, dtype=torch.bfloat16, max_seq_len=4096, download_dir=None, load_format=auto, tensor_parallel_size=1, pipeline_parallel_size=1, data_parallel_size=1, disable_custom_all_reduce=False, quantization=None, enforce_eager=False, enable_return_routed_experts=False, kv_cache_dtype=auto, device_config=cuda, structured_outputs_config=StructuredOutputsConfig(backend='auto', disable_fallback=False, disable_any_whitespace=False, disable_additional_properties=False, reasoning_parser='', reasoning_parser_plugin='', enable_in_reasoning=False), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None, kv_cache_metrics=False, kv_cache_metrics_sample=0.01, cudagraph_metrics=False, enable_layerwise_nvtx_tracing=False, enable_mfu_metrics=False, enable_mm_processor_stats=False, enable_logging_iteration_details=False), seed=0, served_model_name=/home/ubuntu/veenaModal/models/spark_tts_4speaker/LLM, enable_prefix_caching=True, enable_chunked_prefill=True, pooler_config=None, compilation_config={'level': None, 'mode': <CompilationMode.VLLM_COMPILE: 3>, 'debug_dump_path': None, 'cache_dir': '', 'compile_cache_save_format': 'binary', 'backend': 'inductor', 'custom_ops': ['none'], 'splitting_ops': ['vllm::unified_attention', 'vllm::unified_attention_with_output', 'vllm::unified_mla_attention', 'vllm::unified_mla_attention_with_output', 'vllm::mamba_mixer2', 'vllm::mamba_mixer', 'vllm::short_conv', 'vllm::linear_attention', 'vllm::plamo2_mamba_mixer', 'vllm::gdn_attention_core', 'vllm::kda_attention', 'vllm::sparse_attn_indexer', 'vllm::rocm_aiter_sparse_attn_indexer', 'vllm::unified_kv_cache_update'], 'compile_mm_encoder': False, 'compile_sizes': [], 'compile_ranges_split_points': [4096], 'inductor_compile_config': {'enable_auto_functionalized_v2': False, 'combo_kernels': True, 'benchmark_combo_kernel': True}, 'inductor_passes': {}, 'cudagraph_mode': <CUDAGraphMode.FULL_AND_PIECEWISE: (2, 1)>, 'cudagraph_num_of_warmups': 1, 'cudagraph_capture_sizes': [1, 2, 4, 8, 16, 24, 32, 40, 48, 56, 64, 72, 80, 88, 96, 104, 112, 120, 128, 136, 144, 152, 160, 168, 176, 184, 192, 200, 208, 216, 224, 232, 240, 248, 256], 'cudagraph_copy_inputs': False, 'cudagraph_specialize_lora': True, 'use_inductor_graph_partition': False, 'pass_config': {'fuse_norm_quant': False, 'fuse_act_quant': False, 'fuse_attn_quant': False, 'eliminate_noops': True, 'enable_sp': False, 'fuse_gemm_comms': False, 'fuse_allreduce_rms': False}, 'max_cudagraph_capture_size': 256, 'dynamic_shapes_config': {'type': <DynamicShapesType.BACKED: 'backed'>, 'evaluate_guards': False, 'assume_32_bit_indexing': True}, 'local_cache_dir': None, 'static_all_moe_layers': []}
(EngineCore_DP0 pid=4115302) INFO 02-18 00:57:59 [parallel_state.py:1212] world_size=1 rank=0 local_rank=0 distributed_init_method=tcp://216.81.248.184:42663 backend=nccl
(EngineCore_DP0 pid=4115302) INFO 02-18 00:57:59 [parallel_state.py:1423] rank 0 in world size 1 is assigned as DP rank 0, PP rank 0, PCP rank 0, TP rank 0, EP rank N/A
(EngineCore_DP0 pid=4115302) INFO 02-18 00:58:00 [gpu_model_runner.py:4033] Starting to load model /home/ubuntu/veenaModal/models/spark_tts_4speaker/LLM...


Output:
Chunk ID: b37c31
Wall time: 5.0005 seconds
Process running with session ID 28485
Original token count: 387
Output:
(EngineCore_DP0 pid=4115302) INFO 02-18 00:58:01 [cuda.py:364] Using FLASH_ATTN attention backend out of potential backends: ('FLASH_ATTN', 'FLASHINFER', 'TRITON_ATTN', 'FLEX_ATTENTION')
(EngineCore_DP0 pid=4115302) <frozen importlib._bootstrap_external>:1184: FutureWarning: The cuda.cudart module is deprecated and will be removed in a future release, please switch to use the cuda.bindings.runtime module instead.
(EngineCore_DP0 pid=4115302) <frozen importlib._bootstrap_external>:1184: FutureWarning: The cuda.nvrtc module is deprecated and will be removed in a future release, please switch to use the cuda.bindings.nvrtc module instead.
(EngineCore_DP0 pid=4115302)
Loading safetensors checkpoint shards:   0% Completed | 0/1 [00:00<?, ?it/s]
(EngineCore_DP0 pid=4115302)
Loading safetensors checkpoint shards: 100% Completed | 1/1 [00:00<00:00,  2.07it/s]
(EngineCore_DP0 pid=4115302)
Loading safetensors checkpoint shards: 100% Completed | 1/1 [00:00<00:00,  2.07it/s]
(EngineCore_DP0 pid=4115302)
(EngineCore_DP0 pid=4115302) INFO 02-18 00:58:01 [default_loader.py:291] Loading weights took 0.56 seconds
(EngineCore_DP0 pid=4115302) INFO 02-18 00:58:02 [gpu_model_runner.py:4130] Model loading took 0.95 GiB memory and 1.121981 seconds
(EngineCore_DP0 pid=4115302) INFO 02-18 00:58:07 [backends.py:812] Using cache directory: /home/ubuntu/.cache/vllm/torch_compile_cache/f3ff13511c/rank_0_0/backbone for vLLM's torch.compile
(EngineCore_DP0 pid=4115302) INFO 02-18 00:58:07 [backends.py:872] Dynamo bytecode transform time: 4.29 s


Output:
Chunk ID: 678663
Wall time: 5.0009 seconds
Process running with session ID 28485
Original token count: 787
Output:
(EngineCore_DP0 pid=4115302) INFO 02-18 00:58:10 [backends.py:267] Directly load the compiled graph(s) for compile range (1, 4096) from the cache, took 0.687 s
(EngineCore_DP0 pid=4115302) INFO 02-18 00:58:10 [monitor.py:34] torch.compile takes 4.98 s in total
(EngineCore_DP0 pid=4115302) INFO 02-18 00:58:11 [gpu_worker.py:356] Available KV cache memory: 18.06 GiB
(EngineCore_DP0 pid=4115302) INFO 02-18 00:58:11 [kv_cache_utils.py:1307] GPU KV cache size: 1,577,712 tokens
(EngineCore_DP0 pid=4115302) INFO 02-18 00:58:11 [kv_cache_utils.py:1312] Maximum concurrency for 4,096 tokens per request: 385.18x
(EngineCore_DP0 pid=4115302)
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):   0%|          | 0/35 [00:00<?, ?it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  11%|█▏        | 4/35 [00:00<00:00, 31.85it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  23%|██▎       | 8/35 [00:00<00:00, 34.65it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  34%|███▍      | 12/35 [00:00<00:00, 35.61it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  46%|████▌     | 16/35 [00:00<00:00, 35.94it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  57%|█████▋    | 20/35 [00:00<00:00, 35.58it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  69%|██████▊   | 24/35 [00:00<00:00, 35.80it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  80%|████████  | 28/35 [00:00<00:00, 35.75it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  91%|█████████▏| 32/35 [00:00<00:00, 34.23it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 100%|██████████| 35/35 [00:01<00:00, 34.19it/s]
(EngineCore_DP0 pid=4115302)
Capturing CUDA graphs (decode, FULL):   0%|          | 0/19 [00:00<?, ?it/s]
Capturing CUDA graphs (decode, FULL):  16%|█▌        | 3/19 [00:00<00:00, 27.86it/s]
Capturing CUDA graphs (decode, FULL):  37%|███▋      | 7/19 [00:00<00:00, 32.04it/s]
Capturing CUDA graphs (decode, FULL):  58%|█████▊    | 11/19 [00:00<00:00, 33.50it/s]
Capturing CUDA graphs (decode, FULL):  79%|███████▉  | 15/19 [00:00<00:00, 34.08it/s]
Capturing CUDA graphs (decode, FULL): 100%|██████████| 19/19 [00:00<00:00, 33.66it/s]
Capturing CUDA graphs (decode, FULL): 100%|██████████| 19/19 [00:00<00:00, 33.20it/s]
(EngineCore_DP0 pid=4115302) INFO 02-18 00:58:13 [gpu_model_runner.py:5063] Graph capturing finished in 3 secs, took 0.28 GiB
(EngineCore_DP0 pid=4115302) INFO 02-18 00:58:13 [core.py:272] init engine (profile, create kv cache, warmup model) took 11.27 seconds
[2026-02-18 00:58:15] INFO tts_runtime.py:192: Initializing prompt builder...
[2026-02-18 00:58:15] INFO tts_runtime.py:199: Initializing BiCodec decoder...
/home/ubuntu/.local/lib/python3.10/site-packages/torch/nn/utils/weight_norm.py:144: FutureWarning: `torch.nn.utils.weight_norm` is deprecated in favor of `torch.nn.utils.parametrizations.weight_norm`.
  WeightNorm.apply(module, name, dim)


Output:
Chunk ID: b3d25f
Wall time: 5.0006 seconds
Process running with session ID 28485
Original token count: 127
Output:
[2026-02-18 00:58:20] INFO tts_runtime.py:206: Initializing TTS pipeline...
[2026-02-18 00:58:20] INFO tts_runtime.py:217: Initializing streaming pipeline...
[2026-02-18 00:58:21] INFO long_text_processor.py:112: LongTextProcessor initialized
[2026-02-18 00:58:21] INFO tts_runtime.py:274: ✅ TTS runtime initialized in 31178ms
[2026-02-18 00:58:21] INFO tts_runtime.py:275:    Model version: LLM
[2026-02-18 00:58:21] INFO tts_runtime.py:286: Speaker globals cache startup precompute skipped (disabled)


Output:
Chunk ID: ba628a
Wall time: 0.0513 seconds
Process exited with code 0
Original token count: 262
Output:
✅ vLLM engine initialized successfully!

🎉 Spark TTS Model ready for inference!

🎵 Loading BiCodec audio tokenizer from /home/ubuntu/veenaModal/models/spark_tts_4speaker...
   Device: cuda
Missing tensor: mel_transformer.spectrogram.window
Missing tensor: mel_transformer.mel_scale.fb
✅ BiCodec decoder initialized (sample rate: 16000Hz)
🚀 SparkTTSPipeline initialized
🌊 Veena3SlidingWindowPipeline initialized (sliding window: 28 tokens)

=== streaming concurrency=64 requests=64 ===
success=64/64 (100%)
wall=3.93s throughput=16.30 req/s
latency_ms avg=3924 p50=3924 p95=3925 max=3925
ttfb_ms avg=1548 p50=1525 p95=2004 max=2248
chunks_per_req avg=3.6 p50=4.0 p95=4.0 max=4.0
inter_chunk_ms avg=898.5 p50=1241.7 p95=2153.9 max=2329.6
tail_emit_speed_x avg=1.14 p50=1.07 p95=1.60 max=1.78
gpu: util_avg=60% util_max=93% mem_avg=24373MB mem_max=24867MB

results saved: /tmp/stream_runtime_c64_cumulative_mt1024.json
ERROR 02-18 00:58:26 [core_client.py:605] Engine core proc EngineCore_DP0 died unexpectedly, shutting down client.


The current TTS application here is an implementation in Modal labs. Which has autoscaling and other rules self scaling rules and everything suitable for my target audience. Now, i need you to create a local entry point or implementation point. then we will plan for optimising the pipeline, then we will discuss on stress testing, then we will discuss on scaling laws post optimisations locally. @veena3modal/app.py

the above part is performed by a different agent. and benchmarks are made. Now can you do the benchmarks on this newly optimized path to understand the concurrency and stress testing.

Now i need you to capture detailed time logs for each request happening in the GPU. like what happens with 1req, time per LLM token, time per batch decoding, and everything else. detailed time about where we are spending each minute in the GPU per generation.

and when you have the details for per request. you can obviously add more logging to get details of concurrency. so you understand what's the bottleneck for the peak concurrency. and then we will decide on effective methods in optimising this TTS engine.

yes. and also this time, before the benchmarks, can you also like put some timeline markers to spot the latency differences and the time spent, to observe. i feel like a 0.5B auto-regressive model and a decoder at some 500M params, in a 80GB A100 GPU should do better than these. So, i'd like to see where we are spending the max time for each stream. what do you think. we can do better ?

yes. onto next steps.

;et

let's go towards achieving the biggest optimisation in decoding/schediling/queing behaviours which increases per GPU throughput and reduces latency overheads and has best overall throughput also with higher concurrency

<permissions instructions>
Filesystem sandboxing defines which files can be read or written. `sandbox_mode` is `danger-full-access`: No filesystem sandboxing - all commands are permitted. Network access is enabled.
Approval policy is currently never. Do not provide the `sandbox_permissions` for any reason, commands will be rejected.
</permissions instructions>

<collaboration_mode># Collaboration Mode: Default

You are now in Default mode. Any previous instructions for other modes (e.g. Plan mode) are no longer active.

Your active mode changes only when new developer instructions with a different `<collaboration_mode>...</collaboration_mode>` change it; user requests or tool descriptions do not change mode by themselves. Known mode names are Default and Plan.

## request_user_input availability

The `request_user_input` tool is unavailable in Default mode. If you call it while in Default mode, it will return an error.

If a decision is necessary and cannot be discovered from local context, ask the user directly. However, in Default mode you should strongly prefer executing the user's request rather than stopping to ask questions.
</collaboration_mode>

# AGENTS.md instructions for /home/ubuntu/veenaModal

<INSTRUCTIONS>
## Skills
A skill is a set of local instructions to follow that is stored in a `SKILL.md` file. Below is the list of skills that can be used. Each entry includes a name, description, and file path so you can open the source for full instructions when using a specific skill.
### Available skills
- skill-creator: Guide for creating effective skills. This skill should be used when users want to create a new skill (or update an existing skill) that extends Codex's capabilities with specialized knowledge, workflows, or tool integrations. (file: /home/ubuntu/.codex/skills/.system/skill-creator/SKILL.md)
- skill-installer: Install Codex skills into $CODEX_HOME/skills from a curated list or a GitHub repo path. Use when a user asks to list installable skills, install a curated skill, or install a skill from another repo (including private repos). (file: /home/ubuntu/.codex/skills/.system/skill-installer/SKILL.md)
### How to use skills
- Discovery: The list above is the skills available in this session (name + description + file path). Skill bodies live on disk at the listed paths.
- Trigger rules: If the user names a skill (with `$SkillName` or plain text) OR the task clearly matches a skill's description shown above, you must use that skill for that turn. Multiple mentions mean use them all. Do not carry skills across turns unless re-mentioned.
- Missing/blocked: If a named skill isn't in the list or the path can't be read, say so briefly and continue with the best fallback.
- How to use a skill (progressive disclosure):
  1) After deciding to use a skill, open its `SKILL.md`. Read only enough to follow the workflow.
  2) When `SKILL.md` references relative paths (e.g., `scripts/foo.py`), resolve them relative to the skill directory listed above first, and only consider other paths if needed.
  3) If `SKILL.md` points to extra folders such as `references/`, load only the specific files needed for the request; don't bulk-load everything.
  4) If `scripts/` exist, prefer running or patching them instead of retyping large code blocks.
  5) If `assets/` or templates exist, reuse them instead of recreating from scratch.
- Coordination and sequencing:
  - If multiple skills apply, choose the minimal set that covers the request and state the order you'll use them.
  - Announce which skill(s) you're using and why (one short line). If you skip an obvious skill, say why.
- Context hygiene:
  - Keep context small: summarize long sections instead of pasting them; only load extra files when needed.
  - Avoid deep reference-chasing: prefer opening only files directly linked from `SKILL.md` unless you're blocked.
  - When variants exist (frameworks, providers, domains), pick only the relevant reference file(s) and note that choice.
- Safety and fallback: If a skill can't be applied cleanly (missing files, unclear instructions), state the issue, pick the next-best approach, and continue.
</INSTRUCTIONS>

<environment_context>
  <cwd>/home/ubuntu/veenaModal</cwd>
  <shell>bash</shell>
</environment_context>

we're doing a TTS system for voice agents. ain't no way is a non-streaming path useful in any manner. we just need organically true max tokens omitting at insane speed. i wanna know if vLLM can do better than we are doign currently.


**Acknowledging request and planning next steps**


You’re right to prioritize true streaming only. I’m going to run the missing long-generation DELTA benchmark now, compare it against the completed cumulative run, then identify whether the current bottleneck is vLLM decode/scheduling or our downstream chunk/decode path.


**Preparing to run delta benchmark**


Output:
Chunk ID: 840fba
Wall time: 1.0013 seconds
Process running with session ID 93514
Original token count: 0
Output:


**Planning repeated polling with keep-alive commentary**


I’ve kicked off the c64 long-token DELTA run. I’ll wait for completion, then compute a direct A/B against the cumulative baseline and isolate where latency is accumulating.


Output:
Chunk ID: 301aa8
Wall time: 5.0006 seconds
Process running with session ID 93514
Original token count: 1362
Output:
[2026-02-18 00:59:09] INFO tts_runtime.py:183: Loading SparkTTS model with vLLM...
The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
🚀 Initializing Spark TTS Model
📁 Model path: /home/ubuntu/veenaModal/models/spark_tts_4speaker/LLM
🗣  Speakers: 12 predefined (lipakshi, vardan, reet, Nandini, krishna, anika, adarsh, Nilay, Aarvi, Asha, Bittu, Mira)
🎭 Emotions: 10 tags in [bracket] format

📝 Loading tokenizer from model...
✅ Tokenizer loaded: 165168 tokens

🔍 Validating 12 speakers for Spark TTS...
  lipakshi → <|speaker_0|>: 7 tokens
  vardan → <|speaker_1|>: 7 tokens
  reet → <|speaker_2|>: 7 tokens
  Nandini → <|speaker_3|>: 7 tokens
  krishna → <|speaker_4|>: 7 tokens
  anika → <|speaker_5|>: 7 tokens
  adarsh → <|speaker_6|>: 7 tokens
  Nilay → <|speaker_7|>: 7 tokens
  Aarvi → <|speaker_8|>: 7 tokens
  Asha → <|speaker_9|>: 7 tokens
  Bittu → <|speaker_10|>: 8 tokens
  Mira → <|speaker_11|>: 8 tokens
✅ All 12 speakers validated with mapping

🔍 Validating 10 emotion tags for Spark TTS...
✅ All 10 emotion tags validated

📋 Sample emotion tags:
  [angry]: 1 tokens
  [curious]: 1 tokens
  [excited]: 1 tokens
  [giggle]: 1 tokens
  [laughs harder]: 1 tokens
  ...

🔧 Initializing vLLM engine with optimizations...
  - dtype: bfloat16
  - max_model_len: 4096
  - gpu_memory_utilization: 0.25
  - enable_prefix_caching: True
  - enforce_eager (disable CUDA graphs): False
INFO 02-18 00:59:10 [model.py:541] Resolved architecture: Qwen2ForCausalLM
INFO 02-18 00:59:10 [model.py:1561] Using max model len 4096
INFO 02-18 00:59:10 [scheduler.py:226] Chunked prefill is enabled with max_num_batched_tokens=4096.
INFO 02-18 00:59:10 [vllm.py:624] Asynchronous scheduling is enabled.
The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
(EngineCore_DP0 pid=4116119) INFO 02-18 00:59:11 [core.py:96] Initializing a V1 LLM engine (v0.15.1) with config: model='/home/ubuntu/veenaModal/models/spark_tts_4speaker/LLM', speculative_config=None, tokenizer='/home/ubuntu/veenaModal/models/spark_tts_4speaker/LLM', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, tokenizer_revision=None, trust_remote_code=True, dtype=torch.bfloat16, max_seq_len=4096, download_dir=None, load_format=auto, tensor_parallel_size=1, pipeline_parallel_size=1, data_parallel_size=1, disable_custom_all_reduce=False, quantization=None, enforce_eager=False, enable_return_routed_experts=False, kv_cache_dtype=auto, device_config=cuda, structured_outputs_config=StructuredOutputsConfig(backend='auto', disable_fallback=False, disable_any_whitespace=False, disable_additional_properties=False, reasoning_parser='', reasoning_parser_plugin='', enable_in_reasoning=False), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None, kv_cache_metrics=False, kv_cache_metrics_sample=0.01, cudagraph_metrics=False, enable_layerwise_nvtx_tracing=False, enable_mfu_metrics=False, enable_mm_processor_stats=False, enable_logging_iteration_details=False), seed=0, served_model_name=/home/ubuntu/veenaModal/models/spark_tts_4speaker/LLM, enable_prefix_caching=True, enable_chunked_prefill=True, pooler_config=None, compilation_config={'level': None, 'mode': <CompilationMode.VLLM_COMPILE: 3>, 'debug_dump_path': None, 'cache_dir': '', 'compile_cache_save_format': 'binary', 'backend': 'inductor', 'custom_ops': ['none'], 'splitting_ops': ['vllm::unified_attention', 'vllm::unified_attention_with_output', 'vllm::unified_mla_attention', 'vllm::unified_mla_attention_with_output', 'vllm::mamba_mixer2', 'vllm::mamba_mixer', 'vllm::short_conv', 'vllm::linear_attention', 'vllm::plamo2_mamba_mixer', 'vllm::gdn_attention_core', 'vllm::kda_attention', 'vllm::sparse_attn_indexer', 'vllm::rocm_aiter_sparse_attn_indexer', 'vllm::unified_kv_cache_update'], 'compile_mm_encoder': False, 'compile_sizes': [], 'compile_ranges_split_points': [4096], 'inductor_compile_config': {'enable_auto_functionalized_v2': False, 'combo_kernels': True, 'benchmark_combo_kernel': True}, 'inductor_passes': {}, 'cudagraph_mode': <CUDAGraphMode.FULL_AND_PIECEWISE: (2, 1)>, 'cudagraph_num_of_warmups': 1, 'cudagraph_capture_sizes': [1, 2, 4, 8, 16, 24, 32, 40, 48, 56, 64, 72, 80, 88, 96, 104, 112, 120, 128, 136, 144, 152, 160, 168, 176, 184, 192, 200, 208, 216, 224, 232, 240, 248, 256], 'cudagraph_copy_inputs': False, 'cudagraph_specialize_lora': True, 'use_inductor_graph_partition': False, 'pass_config': {'fuse_norm_quant': False, 'fuse_act_quant': False, 'fuse_attn_quant': False, 'eliminate_noops': True, 'enable_sp': False, 'fuse_gemm_comms': False, 'fuse_allreduce_rms': False}, 'max_cudagraph_capture_size': 256, 'dynamic_shapes_config': {'type': <DynamicShapesType.BACKED: 'backed'>, 'evaluate_guards': False, 'assume_32_bit_indexing': True}, 'local_cache_dir': None, 'static_all_moe_layers': []}
(EngineCore_DP0 pid=4116119) INFO 02-18 00:59:13 [parallel_state.py:1212] world_size=1 rank=0 local_rank=0 distributed_init_method=tcp://216.81.248.184:50161 backend=nccl
(EngineCore_DP0 pid=4116119) INFO 02-18 00:59:13 [parallel_state.py:1423] rank 0 in world size 1 is assigned as DP rank 0, PP rank 0, PCP rank 0, TP rank 0, EP rank N/A


Output:
Chunk ID: 6a661f
Wall time: 5.0011 seconds
Process running with session ID 93514
Original token count: 352
Output:
(EngineCore_DP0 pid=4116119) INFO 02-18 00:59:14 [gpu_model_runner.py:4033] Starting to load model /home/ubuntu/veenaModal/models/spark_tts_4speaker/LLM...
(EngineCore_DP0 pid=4116119) INFO 02-18 00:59:14 [cuda.py:364] Using FLASH_ATTN attention backend out of potential backends: ('FLASH_ATTN', 'FLASHINFER', 'TRITON_ATTN', 'FLEX_ATTENTION')
(EngineCore_DP0 pid=4116119) <frozen importlib._bootstrap_external>:1184: FutureWarning: The cuda.cudart module is deprecated and will be removed in a future release, please switch to use the cuda.bindings.runtime module instead.
(EngineCore_DP0 pid=4116119) <frozen importlib._bootstrap_external>:1184: FutureWarning: The cuda.nvrtc module is deprecated and will be removed in a future release, please switch to use the cuda.bindings.nvrtc module instead.
(EngineCore_DP0 pid=4116119)
Loading safetensors checkpoint shards:   0% Completed | 0/1 [00:00<?, ?it/s]
(EngineCore_DP0 pid=4116119)
Loading safetensors checkpoint shards: 100% Completed | 1/1 [00:01<00:00,  1.38s/it]
(EngineCore_DP0 pid=4116119)
Loading safetensors checkpoint shards: 100% Completed | 1/1 [00:01<00:00,  1.38s/it]
(EngineCore_DP0 pid=4116119)
(EngineCore_DP0 pid=4116119) INFO 02-18 00:59:16 [default_loader.py:291] Loading weights took 1.46 seconds
(EngineCore_DP0 pid=4116119) INFO 02-18 00:59:17 [gpu_model_runner.py:4130] Model loading took 0.95 GiB memory and 1.978113 seconds


**Noting potential long polling**


Model init is still in progress for this run; once it finishes I’ll extract per-level concurrency/latency and do a strict delta-vs-cumulative comparison on the same 64-stream workload.


Output:
Chunk ID: f23ed8
Wall time: 5.0006 seconds
Process running with session ID 93514
Original token count: 801
Output:
(EngineCore_DP0 pid=4116119) INFO 02-18 00:59:21 [backends.py:812] Using cache directory: /home/ubuntu/.cache/vllm/torch_compile_cache/f3ff13511c/rank_0_0/backbone for vLLM's torch.compile
(EngineCore_DP0 pid=4116119) INFO 02-18 00:59:21 [backends.py:872] Dynamo bytecode transform time: 4.22 s
(EngineCore_DP0 pid=4116119) INFO 02-18 00:59:24 [backends.py:267] Directly load the compiled graph(s) for compile range (1, 4096) from the cache, took 0.634 s
(EngineCore_DP0 pid=4116119) INFO 02-18 00:59:24 [monitor.py:34] torch.compile takes 4.86 s in total
(EngineCore_DP0 pid=4116119) INFO 02-18 00:59:25 [gpu_worker.py:356] Available KV cache memory: 18.06 GiB
(EngineCore_DP0 pid=4116119) INFO 02-18 00:59:25 [kv_cache_utils.py:1307] GPU KV cache size: 1,577,712 tokens
(EngineCore_DP0 pid=4116119) INFO 02-18 00:59:25 [kv_cache_utils.py:1312] Maximum concurrency for 4,096 tokens per request: 385.18x
(EngineCore_DP0 pid=4116119)
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):   0%|          | 0/35 [00:00<?, ?it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  11%|█▏        | 4/35 [00:00<00:00, 32.69it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  23%|██▎       | 8/35 [00:00<00:00, 34.75it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  34%|███▍      | 12/35 [00:00<00:00, 35.71it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  46%|████▌     | 16/35 [00:00<00:00, 36.20it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  57%|█████▋    | 20/35 [00:00<00:00, 36.19it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  69%|██████▊   | 24/35 [00:00<00:00, 36.10it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  80%|████████  | 28/35 [00:00<00:00, 35.96it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE):  91%|█████████▏| 32/35 [00:00<00:00, 34.60it/s]
Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 100%|██████████| 35/35 [00:01<00:00, 34.49it/s]
(EngineCore_DP0 pid=4116119)
Capturing CUDA graphs (decode, FULL):   0%|          | 0/19 [00:00<?, ?it/s]
Capturing CUDA graphs (decode, FULL):  16%|█▌        | 3/19 [00:00<00:00, 29.24it/s]
Capturing CUDA graphs (decode, FULL):  37%|███▋      | 7/19 [00:00<00:00, 32.73it/s]
Capturing CUDA graphs (decode, FULL):  58%|█████▊    | 11/19 [00:00<00:00, 34.09it/s]
Capturing CUDA graphs (decode, FULL):  79%|███████▉  | 15/19 [00:00<00:00, 34.62it/s]
Capturing CUDA graphs (decode, FULL): 100%|██████████| 19/19 [00:00<00:00, 35.26it/s]
Capturing CUDA graphs (decode, FULL): 100%|██████████| 19/19 [00:00<00:00, 34.40it/s]
(EngineCore_DP0 pid=4116119) INFO 02-18 00:59:27 [gpu_model_runner.py:5063] Graph capturing finished in 2 secs, took 0.28 GiB
(EngineCore_DP0 pid=4116119) INFO 02-18 00:59:27 [core.py:272] init engine (profile, create kv cache, warmup model) took 10.78 seconds
[2026-02-18 00:59:29] INFO tts_runtime.py:192: Initializing prompt builder...
[2026-02-18 00:59:29] INFO tts_runtime.py:199: Initializing BiCodec decoder...


Summary: auto


**Running benchmark process**


Output:
Chunk ID: 7627c8
Wall time: 5.0007 seconds
Process running with session ID 93514
Original token count: 100
Output:
/home/ubuntu/.local/lib/python3.10/site-packages/torch/nn/utils/weight_norm.py:144: FutureWarning: `torch.nn.utils.weight_norm` is deprecated in favor of `torch.nn.utils.parametrizations.weight_norm`.
  WeightNorm.apply(module, name, dim)
[2026-02-18 00:59:36] INFO tts_runtime.py:206: Initializing TTS pipeline...
[2026-02-18 00:59:36] INFO tts_runtime.py:217: Initializing streaming pipeline...


Output:
Chunk ID: 3df808
Wall time: 5.0030 seconds
Process running with session ID 93514
Original token count: 348
Output:
[2026-02-18 00:59:36] INFO long_text_processor.py:112: LongTextProcessor initialized
[2026-02-18 00:59:36] INFO tts_runtime.py:274: ✅ TTS runtime initialized in 32929ms
[2026-02-18 00:59:36] INFO tts_runtime.py:275:    Model version: LLM
[2026-02-18 00:59:36] INFO tts_runtime.py:286: Speaker globals cache startup precompute skipped (disabled)
✅ vLLM engine initialized successfully!

🎉 Spark TTS Model ready for inference!

🎵 Loading BiCodec audio tokenizer from /home/ubuntu/veenaModal/models/spark_tts_4speaker...
   Device: cuda
Missing tensor: mel_transformer.spectrogram.window
Missing tensor: mel_transformer.mel_scale.fb
✅ BiCodec decoder initialized (sample rate: 16000Hz)
🚀 SparkTTSPipeline initialized
🌊 Veena3SlidingWindowPipeline initialized (sliding window: 28 tokens)

=== streaming concurrency=64 requests=64 ===
success=64/64 (100%)
wall=3.92s throughput=16.33 req/s
latency_ms avg=3918 p50=3918 p95=3919 max=3919
ttfb_ms avg=1590 p50=1568 p95=2066 max=2297
chunks_per_req avg=3.5 p50=4.0 p95=4.0 max=4.0
inter_chunk_ms avg=886.5 p50=1214.5 p95=2160.7 max=2259.2
tail_emit_speed_x avg=1.18 p50=1.14 p95=1.65 max=1.78
gpu: util_avg=59% util_max=93% mem_avg=24362MB mem_max=24775MB

results saved: /tmp/stream_runtime_c64_delta_mt1024.json
ERROR 02-18 00:59:42 [core_client.py:605] Engine core proc EngineCore_DP0 died unexpectedly, shutting down client.


**Verifying session completion status**


Output:
Chunk ID: e67ea7
Wall time: 0.0518 seconds
Process exited with code 0
Original token count: 0
Output:


**Preparing file comparisons**


The DELTA long-run finished cleanly, and I now have both c64/1024 outputs for a direct A/B. I’m extracting detailed metrics now and then I’ll recommend concrete scheduler/decoder changes aimed at max streaming throughput.


cd /home/ubuntu/veenaModal && cat /tmp/stream_runtime_c64_cumulative_mt1024.json


cd /home/ubuntu/veenaModal && cat /tmp/stream_runtime_c64_delta_mt1024.json


Output:
Chunk ID: c77f47
Wall time: 0.0517 seconds
Process exited with code 0
Original token count: 11130
Output:
{
  "config": {
    "levels": "64",
    "text": "short",
    "max_tokens": 1024,
    "timeout_s": 240.0,
    "requests_multiplier": 1,
    "min_requests": 8,
    "gpu_memory": 0.25,
    "stream_output_kind": "cumulative",
    "chunking": false,
    "timestamp": 1771376306
  },
  "levels": [
    {
      "concurrency": 64,
      "total_requests": 64,
      "success_rate": 1.0,
      "throughput_rps": 16.30298563515798,
      "wall_time_s": 3.9256613133475184,
      "latency_ms": {
        "avg": 3923.9904452697374,
        "p50": 3923.8250013440847,
        "p95": 3924.751913174987,
        "max": 3924.830081872642
      },
      "ttfb_ms": {
        "avg": 1548.0432905897032,
        "p50": 1524.895941838622,
        "p95": 2003.7997914478183,
        "max": 2248.299007304013
      },
      "chunks_per_req": {
        "avg": 3.5625,
        "p50": 4.0,
        "p95": 4.0,
        "max": 4.0
      },
      "inter_chunk_ms": {
        "avg": 898.5251548824968,
        "p50": 1241.7313139885664,
        "p95": 2153.8842115551233,
        "max": 2329.5780699700117
      },
      "tail_emit_speed_x": {
        "avg": 1.141553799584772,
        "p50": 1.074118151548921,
        "p95": 1.5959087816987896,
        "max": 1.7773685862281259
      },
      "gpu": {
        "samples": 7,
        "memory_mb": {
          "min": 23681.0,
          "avg": 24372.714285714286,
          "max": 24867.0,
          "total": 81920.0
        },
        "gpu_util_pct": {
          "min": 28.0,
          "avg": 59.714285714285715,
          "max": 93.0
        }
      },
      "requests": [
        {
          "success": true,
          "latency_ms": 3924.7214756906033,
          "ttfb_ms": 952.4994995445013,
          "chunks_sent": 4,
          "total_bytes": 92524,
          "audio_seconds": 2.89,
          "speaker": "lipakshi",
          "error": null,
          "inter_chunk_ms": [
            1349.7192403301597,
            1619.8152238503098,
            0.017741695046424866
          ],
          "tail_emit_speed_x": 0.9723365290997834,
          "final_metrics": {
            "ttfb_ms": 951,
            "chunks_sent": 4,
            "total_bytes": 92524,
            "audio_duration_seconds": 2.89,
            "text_chunked": false
          }
        },
        {
          "success": true,
          "latency_ms": 3924.8283579945564,
          "ttfb_ms": 1054.7532020136714,
          "chunks_sent": 4,
          "total_bytes": 100204,
          "audio_seconds": 3.13,
          "speaker": "vardan",
          "error": null,
          "inter_chunk_ms": [
            1288.0721921101213,
            1580.5475916713476,
            0.008135102689266205
          ],
          "tail_emit_speed_x": 1.0905637761706217,
          "final_metrics": {
            "ttfb_ms": 1051,
            "chunks_sent": 4,
            "total_bytes": 100204,
            "audio_duration_seconds": 3.13,
            "text_chunked": false
          }
        },
        {
          "success": true,
          "latency_ms": 3924.751913174987,
          "ttfb_ms": 1062.8985380753875,
          "chunks_sent": 4,
          "total_bytes": 100204,
          "audio_seconds": 3.13,
          "speaker": "reet",
          "error": null,
          "inter_chunk_ms": [
            1250.7603904232383,
            1609.2263851314783,
            0.008034519851207733
          ],
          "tail_emit_speed_x": 1.0936968424844855,
          "final_metrics": {
            "ttfb_ms": 1059,
            "chunks_sent": 4,
            "total_bytes": 100204,
            "audio_duration_seconds": 3.13,
            "text_chunked": false
          }
        },
        {
          "success": true,
          "latency_ms": 3924.7919293120503,
          "ttfb_ms": 1065.8908979967237,
          "chunks_sent": 4,
          "total_bytes": 100204,
          "audio_seconds": 3.13,
          "speaker": "Nandini",
          "error": null,
          "inter_chunk_ms": [
            1279.6331411227584,
            1577.9159339144826,
            0.009075738489627838
          ],
          "tail_emit_speed_x": 1.0948262866448182,
          "final_metrics": {
            "ttfb_ms": 1062,
            "chunks_sent": 4,
            "total_bytes": 100204,
            "audio_duration_seconds": 3.13,
            "text_chunked": false
          }
        },
        {
          "success": true,
          "latency_ms": 3924.830081872642,
          "ttfb_ms": 1075.2320168539882,
          "chunks_sent": 4,
          "total_bytes": 101484,
          "audio_seconds": 3.17,
          "speaker": "krishna",
          "error": null,
          "inter_chunk_ms": [
            1508.8419960811734,
            1339.9488534778357,
            0.00713299959897995
          ],
          "tail_emit_speed_x": 1.1124375886250641,
          "final_metrics": {
            "ttfb_ms": 1071,
            "chunks_sent": 4,
            "total_bytes": 101484,
            "audio_duration_seconds": 3.17,
            "text_chunked": false
          }
        },
        {
          "success": true,
          "latency_ms": 3924.6310656890273,
          "ttfb_ms": 1025.1696379855275,
          "chunks_sent": 4,
          "total_bytes": 97004,
          "audio_seconds": 3.03,
          "speaker": "anika",
          "error": null,
          "inter_chunk_ms": [
            1265.8672844991088,
            1631.0888743028045,
            0.007806345820426941
          ],
          "tail_emit_speed_x": 1.0450216619711656,
          "final_metrics": {
            "ttfb_ms": 1020,
            "chunks_sent": 4,
            "total_bytes": 97004,
            "audio_duration_seconds": 3.03,
            "text_chunked": false
          }
        },
        {
          "success": true,
          "latency_ms": 3924.605446867645,
          "ttfb_ms": 1034.278266131878,
          "chunks_sent": 4,
          "total_bytes": 97644,
          "audio_seconds": 3.05,
          "speaker": "lipakshi",
          "error": null,
          "inter_chunk_ms": [
            1241.7313139885664,
            1646.0056183859706,
            0.010007992386817932
          ],
          "tail_emit_speed_x": 1.0552438562417652,
          "final_metrics": {
            "ttfb_ms": 1029,
            "chunks_sent": 4,
            "total_bytes": 97644,
            "audio_duration_seconds": 3.05,
            "text_chunked": false
          }
        },
        {
          "success": true,
          "latency_ms": 3924.68417622149,
          "ttfb_ms": 1017.3595864325762,
          "chunks_sent": 4,
          "total_bytes": 96364,
          "audio_seconds": 3.01,
          "speaker": "vardan",
          "error": null,
          "inter_chunk_ms": [
            1265.2807235717773,
            1640.2573976665735,
            0.014717690646648407
          ],
          "tail_emit_speed_x": 1.035316115225559,
          "final_metrics": {
            "ttfb_ms": 1012,
            "chunks_sent": 4,
            "total_bytes": 96364,
            "audio_duration_seconds": 3.01,
            "text_chunked": false
          }
        },
        {
          "success": true,
          "latency_ms": 3924.779294990003,
          "ttfb_ms": 1071.8784211203456,
          "chunks_sent": 4,
          "total_bytes": 100844,
          "audio_seconds": 3.15,
          "speaker": "reet",
          "error": null,
          "inter_chunk_ms": [
            1480.8762138709426,
            1371.2881561368704,
            0.007955357432365417
          ],
          "tail_emit_speed_x": 1.104139309168271,
          "final_metrics": {
            "ttfb_ms": 1066,
            "chunks_sent": 4,
            "total_bytes": 100844,
            "audio_duration_seconds": 3.15,
            "text_chunked": false
          }
        },
        {
          "success": true,
          "latency_ms": 3924.615074880421,
          "ttfb_ms": 1047.6508885622025,
          "chunks_sent": 4,
          "total_bytes": 98924,
          "audio_seconds": 3.09,
          "speaker": "Nandini",
          "error": null,
          "inter_chunk_ms": [
            1252.7827862650156,
            1622.0671208575368,
            0.009117648005485535
          ],
          "tail_emit_speed_x": 1.074048823650604,
          "final_metrics": {
            "ttfb_ms": 1041,
            "chunks_sent": 4,
            "total_bytes": 98924,
            "audio_duration_seconds": 3.09,
            "text_chunked": false
          }
        },
        {
          "success": true,
          "latency_ms": 3924.6949357911944,
          "ttfb_ms": 1071.6909682378173,
          "chunks_sent": 4,
          "total_bytes": 100844,
          "audio_seconds": 3.15,
          "speaker": "krishna",
          "error": null,
          "inter_chunk_ms": [
            1296.2136017158628,
            1555.521447211504,
            0.007082708179950714
          ],
          "tail_emit_speed_x": 1.1040994109451991,
          "final_metrics": {
            "ttfb_ms": 1065,
            "chunks_sent": 4,
            "total_bytes": 100844,
            "audio_duration_seconds": 3.15,
            "text_chunked": false
          }
        },
        {
          "success": true,
          "latency_ms": 3924.653857946396,
          "ttfb_ms": 1057.5583968311548,
          "chunks_sent": 4,
          "total_bytes": 99564,
          "audio_seconds": 3.11,
          "speaker": "anika",
          "error": null,
          "inter_chunk_ms": [
            1275.4294499754906,
            1590.1284823194146,
            0.008176080882549286
          ],
          "tail_emit_speed_x": 1.0847214688799633,
          "final_metrics": {
            "ttfb_ms": 1050,
            "chunks_sent": 4,
            "total_bytes": 99564,
            "audio_duration_seconds": 3.11,
            "text_chunked": false
          }
        },
        {
          "success": true,
          "latency_ms": 3924.6211955323815,
          "ttfb_ms": 1043.9817598089576,
          "chunks_sent": 4,
          "total_bytes": 98284,
          "audio_seconds": 3.07,
          "speaker": "lipakshi",
          "error": null,
          "inter_chunk_ms": [
            1281.588738784194,
            1597.3514737561345,
            0.00881589949131012
          ],
          "tail_emit_speed_x": 1.0657356008975907,
          "final_metrics": {
            "ttfb_ms": 1036,
            "chunks_sent": 4,
            "total_bytes": 98284,
            "audio_duration_seconds": 3.07,
            "text_chunked": false
          }
        },
        {
          "success": true,
          "latency_ms": 3924.661192111671,
          "ttfb_ms": 1065.9295115619898,
          "chunks_sent": 4,
          "total_bytes": 100204,
          "audio_seconds": 3.13,
          "speaker": "vardan",
          "error": null,
          "inter_chunk_ms": [
            1282.980932854116,
            1574.5639223605394,
            0.007244758307933807
          ],
          "tail_emit_speed_x": 1.0948911439628914,
          "final_metrics": {
            "ttfb_ms": 1058,
            "chunks_sent": 4,
            "total_bytes": 100204,
            "audio_duration_seconds": 3.13,
            "text_chunked": false
          }
        },
        {
          "success": true,
          "latency_ms": 3924.551154486835,
          "ttfb_ms": 997.4238388240337,
          "chunks_sent": 4,
          "total_bytes": 95084,
          "audio_seconds": 2.97,
          "speaker": "reet",
          "error": null,
          "inter_chunk_ms": [
            1267.3591058701277,
            1657.740124501288,
            0.009537674486637115
          ],
          "tail_emit_speed_x": 1.0146466756357986,
          "final_metrics": {
            "ttfb_ms": 989,
            "chunks_sent": 4,
            "total_bytes": 95084,
            "audio_duration_seconds": 2.97,
            "text_chunked": false
          }
        },
        {
          "success": true,
          "latency_ms": 3924.659258686006,
          "ttfb_ms": 1062.497990205884,
          "chunks_sent": 4,
          "total_bytes": 100204,
          "audio_seconds": 3.13,
          "speaker": "Nandini",
          "error": null,
          "inter_chunk_ms": [
            1307.5603926554322,
            1553.7099307402968,
            0.006532296538352966
          ],
          "tail_emit_speed_x": 1.09357918942915,
          "final_metrics": {
            "ttfb_ms": 1054,
            "chunks_sent": 4,
            "total_bytes": 100204,
            "audio_duration_seconds": 3.13,
            "text_chunked": false
          }
        },
        {
          "success": true,
          "latency_ms": 3924.6219880878925,
          "ttfb_ms": 1077.2596606984735,
          "chunks_sent": 4,
          "total_bytes": 101484,
          "audio_seconds": 3.17,
          "speaker": "krishna",
          "error": null,
          "inter_chunk_ms": [
            1529.8980819061399,
            1316.3524521514773,
            0.007074326276779175
          ],
          "tail_emit_speed_x": 1.1133110702164795,
          "final_metrics": {
            "ttfb_ms": 1068,
            "chunks_sent": 4,
            "total_bytes": 101484,
            "audio_duration_seconds": 3.17,
            "text_chunked": false
          }
        },
        {
          "success": true,
          "latency_ms": 3924.6195144951344,
          "ttfb_ms": 1098.061354830861,
          "chunks_sent": 4,
          "total_bytes": 103404,
          "audio_seconds": 3.23,
          "speaker": "anika",
          "error": null,
          "inter_chunk_ms": [
            1479.173512198031,
            1346.4281279593706,
            0.006843358278274536
          ],
          "tail_emit_speed_x": 1.1427325452180492,
          "final_metrics": {
            "ttfb_ms": 1088,
            "chunks_sent": 4,
            "total_bytes": 103404,
            "audio_duration_seconds": 3.23,
            "text_chunked": false
          }
        },
        {
          "success": true,
          "latency_ms": 3924.536275677383,
          "ttfb_ms": 1054.709899239242,
          "chunks_sent": 4,
          "total_bytes": 99564,
          "audio_seconds": 3.11,
          "speaker": "lipakshi",
          "error": null,
          "inter_chunk_ms": [
            1286.2677704542875,
            1581.9387324154377,
            0.0077141448855400085
          ],
          "tail_emit_speed_x": 1.0836892522606014,
          "final_metrics": {
            "ttfb_ms": 1045,
            "chunks_sent": 4,
            "total_bytes": 99564,
            "audio_duration_seconds": 3.11,
            "text_chunked": false
          }
        },
        {
          "success": true,
          "latency_ms": 3924.4792787358165,
          "ttfb_ms": 1047.700783237815,
          "chunks_sent": 4,
          "total_bytes": 98924,
          "audio_seconds": 3.09,
          "speaker": "vardan",
          "error": null,
          "inter_chunk_ms": [
            1269.653215073049,
            1605.181584134698,
            0.008686445653438568
          ],
          "tail_emit_speed_x": 1.074118151548921,
          "final_metrics": {
            "ttfb_ms": 1037,
            "chunks_sent": 4,
            "total_bytes": 98924,
            "audio_duration_seconds": 3.09,
            "text_chunked": false
          }
        },
        {
          "success": true,
          "latency_ms": 3923.9108823239803,
          "ttfb_ms": 1386.206834577024,
          "chunks_sent": 4,
          "total_bytes": 125484,
          "audio_seconds": 3.92,
          "speaker": "reet",
          "error": null,
          "inter_chunk_ms": [
            2149.666382931173,
            34.54186208546162,
            0.005019828677177429
          ],
          "tail_emit_speed_x": 1.5447033721210652,
          "final_metrics": {
            "ttfb_ms": 1375,
            "chunks_sent": 4,
            "total_bytes": 125484,
            "audio_duration_seconds": 3.92,
            "text_chunked": false
          }
        },
        {
          "success": true,
          "latency_ms": 3920.0300937518477,
          "ttfb_ms": 1420.1550036668777,
          "chunks_sent": 4,
          "total_bytes": 115884,
          "audio_seconds": 3.62,
          "speaker": "Nandini",
          "error": null,
          "inter_chunk_ms": [
            1204.963150434196,
            389.6792894229293,
            0.01074187457561493
          ],
          "tail_emit_speed_x": 1.4480723514377503,
          "final_metrics": {
            "ttfb_ms": 1409,
            "chunks_sent": 4,
            "total_bytes": 115884,
            "audio_duration_seconds": 3.62,
            "text_chunked": false
          }
        },
        {
          "success": true,
          "latency_ms": 3924.5330104604363,
          "ttfb_ms": 1444.2946407943964,
          "chunks_sent": 4,
          "total_bytes": 89004,
          "audio_seconds": 2.78,
          "speaker": "krishna",
          "error": null,
          "inter_chunk_ms": [
            1575.9364198893309,
            903.2688345760107,
            0.007214024662971497
          ],
          "tail_emit_speed_x": 1.1208600084573008,
          "final_metrics": {
            "ttfb_ms": 1433,
            "chunks_sent": 4,
            "total_bytes": 89004,
            "audio_duration_seconds": 2.78,
            "text_chunked": false
          }
        },
        {
          "success": true,
          "latency_ms": 3923.838405869901,
          "ttfb_ms": 1465.7411249354482,
          "chunks_sent": 3,
          "total_bytes": 59884,
          "audio_seconds": 1.87,
          "speaker": "anika",
          "error": null,
          "inter_chunk_ms": [
            2104.728627949953,
            0.011482276022434235
          ],
          "tail_emit_speed_x": 0.7607510144143336,
          "final_metrics": {
            "ttfb_ms": 1454,
            "chunks_sent": 3,
            "total_bytes": 59884,
            "audio_duration_seconds": 1.87,
            "text_chunked": false
          }
        },
        {
          "success": true,
          "latency_ms": 3923.8756764680147,
          "ttfb_ms": 1517.5228053703904,
          "chunks_sent": 4,
          "total_bytes": 112044,
          "audio_seconds": 3.5,
          "speaker": "lipakshi",
          "error": null,
          "inter_chunk_ms": [
            2153.8842115551233,
            33.37503597140312,
            0.003516674041748047
          ],
          "tail_emit_speed_x": 1.4544832730220167,
          "final_metrics": {
            "ttfb_ms": 1505,
            "chunks_sent": 4,
            "total_bytes": 112044,
            "audio_duration_seconds": 3.5,
            "text_chunked": false
          }
        },
        {
          "success": true,
          "latency_ms": 3923.827706836164,
          "ttfb_ms": 1482.017601840198,
          "chunks_sent": 4,
          "total_bytes": 122924,
          "audio_seconds": 3.84,
          "speaker": "vardan",
          "error": null,
          "inter_chunk_ms": [
            2122.2785618156195,
            33.41233637183905,
            0.004889443516731262
          ],
          "tail_emit_speed_x": 1.5726038614318634,
          "final_metrics": {
            "ttfb_ms": 1470,
            "chunks_sent": 4,
            "total_bytes": 122924,
            "audio_duration_seconds": 3.84,
            "text_chunked": false
          }
        },
        {
          "success": true,
          "latency_ms": 3923.9072473719716,
          "ttfb_ms": 1585.5333218351007,
          "chunks_sent": 4,
          "total_bytes": 118444,
          "audio_seconds": 3.7,
          "speaker": "reet",
          "error": null,
          "inter_chunk_ms": [
            2265.6223038211465,
            33.264586701989174,
            0.005410052835941315
          ],
          "tail_emit_speed_x": 1.5822961244962186,
          "final_metrics": {
            "ttfb_ms": 1573,
            "chunks_sent": 4,
            "total_bytes": 118444,
            "audio_duration_seconds": 3.7,
            "text_chunked": false
          }
        },
        {
          "success": true,
          "latency_ms": 3923.8656871020794,
          "ttfb_ms": 1524.6752128005028,
          "chunks_sent": 4,
          "total_bytes": 131884,
          "audio_seconds": 4.12,
          "speaker": "Nandini",
          "error": null,
          "inter_chunk_ms": [
            2247.094971127808,
            33.04060269147158,
            0.004889443516731262
          ],
          "tail_emit_speed_x": 1.7172458977853207,
          "final_metrics": {
            "ttfb_ms": 1511,
            "chunks_sent": 4,
            "total_bytes": 131884,
            "audio_duration_seconds": 4.12,
            "text_chunked": false
          }
        },
        {
          "success": true,
          "latency_ms": 3923.8323839381337,
          "ttfb_ms": 1538.628013804555,
          "chunks_sent": 4,
          "total_bytes": 128684,
          "audio_seconds": 4.02,
          "speaker": "krishna",
          "error": null,
          "inter_chunk_ms": [
            2166.228039190173,
            33.38706772774458,
            0.00436883419752121
          ],
          "tail_emit_speed_x": 1.6853901704761958,
          "final_metrics": {
            "ttfb_ms": 1525,
            "chunks_sent": 4,
            "total_bytes": 128684,
            "audio_duration_seconds": 4.02,
            "text_chunked": false
          }
        },
        {
          "success": true,
          "latency_ms": 3922.943202778697,
          "ttfb_ms": 1484.4077909365296,
          "chunks_sent": 4,
          "total_bytes": 123564,
          "audio_seconds": 3.86,
          "speaker": "anika",
          "error": null,
          "inter_chunk_ms": [
            1537.325320765376,
            373.88706021010876,
            0.009369105100631714
          ],
          "tail_emit_speed_x": 1.5829173450813254,
          "final_metrics": {
            "ttfb_ms": 1470,
            "chunks_sent": 4,
            "total_bytes": 123564,
            "audio_duration_seconds": 3.86,
            "text_chunked": false
          }
        },
        {
          "success": true,
          "latency_ms": 3923.751652240753,
          "ttfb_ms": 1509.0681156143546,
          "chunks_sent": 4,
          "total_bytes": 113964,
          "audio_seconds": 3.56,
          "speaker": "lipakshi",
          "error": null,
          "inter_chunk_ms": [
            2061.4295434206724,
            33.58868882060051,
            0.004007481038570404
          ],
          "tail_emit_speed_x": 1.4743132779104235,
          "final_metrics": {
            "ttfb_ms": 1495,
            "chunks_sent": 4,
            "total_bytes": 113964,
            "audio_duration_seconds": 3.56,
            "text_chunked": false
          }
        },
        {
          "success": true,
          "latency_ms": 3923.7018479034305,
          "ttfb_ms": 1492.485199123621,
          "chunks_sent": 4,
          "total_bytes": 124204,
          "audio_seconds": 3.88,
          "speaker": "vardan",
          "error": null,
          "inter_chunk_ms": [
            1904.3305749073625,
            138.7089053168893,
            0.0087665393948555
          ],
          "tail_emit_speed_x": 1.5959087816987896,
          "final_metrics": {
            "ttfb_ms": 1478,
            "chunks_sent": 4,
            "total_bytes": 124204,
            "audio_duration_seconds": 3.88,
            "text_chunked": false
          }
        },
        {
          "success": true,
          "latency_ms": 3923.758835531771,
          "ttfb_ms": 1524.895941838622,
          "chunks_sent": 4,
          "total_bytes": 119084,
          "audio_seconds": 3.72,
          "speaker": "reet",
          "error": null,
          "inter_chunk_ms": [
            2112.9688639193773,
            33.31288788467646,
            0.004348345100879669
          ],
          "tail_emit_speed_x": 1.5507347292670428,
          "final_metrics": {
            "ttfb_ms": 1510,
            "chunks_sent": 4,
            "total_bytes": 119084,
            "audio_duration_seconds": 3.72,
            "text_chunked": false
          }
        },
        {
          "success": true,
          "latency_ms": 3923.7764487043023,
          "ttfb_ms": 1538.227366283536,
          "chunks_sent": 4,
          "total_bytes": 135724,
          "audio_seconds": 4.24,
          "speaker": "Nandini",
          "error": null,
          "inter_chunk_ms": [
            2200.073883868754,
            33.22827909141779,
            0.006431713700294495
          ],
          "tail_emit_speed_x": 1.7773685862281259,
          "final_metrics": {
            "ttfb_ms": 1523,
            "chunks_sent": 4,
            "total_bytes": 135724,
            "audio_duration_seconds": 4.24,
            "text_chunked": false
          }
        },
        {
          "success": true,
          "latency_ms": 3923.819019459188,
          "ttfb_ms": 1597.5165255367756,
          "chunks_sent": 3,
          "total_bytes": 60524,
          "audio_seconds": 1.89,
          "speaker": "krishna",
          "error": null,
          "inter_chunk_ms": [
            2320.9434477612376,
            0.02079922705888748
          ],
          "tail_emit_speed_x": 0.8124480822841073,
          "final_metrics": {
            "ttfb_ms": 1582,
            "chunks_sent": 3,
            "total_bytes": 60524,
            "audio_duration_seconds": 1.89,
            "text_chunked": false
          }
        },
        {
          "success": true,
          "latency_ms": 3923.769216053188,
          "ttfb_ms": 1542.0685522258282,
          "chunks_sent": 4,
          "total_bytes": 122924,
          "audio_seconds": 3.84,
          "speaker": "anika",
          "error": null,
          "inter_chunk_ms": [
            2262.768043205142,
            12.804669328033924,
            0.003647059202194214
          ],
          "tail_emit_speed_x": 1.6122932903873712,
          "final_metrics": {
            "ttfb_ms": 1526,
            "chunks_sent": 4,
            "total_bytes": 122924,
            "audio_duration_seconds": 3.84,
            "text_chunked": false
          }
        },
        {
          "success": true,
          "latency_ms": 3923.694394528866,
          "ttfb_ms": 1512.8537556156516,
          "chunks_sent": 3,
          "total_bytes": 60524,
          "audio_seconds": 1.89,
          "speaker": "lipakshi",
          "error": null,
          "inter_chunk_ms": [
            2124.8426511883736,
            0.011119991540908813
          ],
          "tail_emit_speed_x": 0.783958910221455,
          "final_metrics": {
            "ttfb_ms": 1496,
            "chunks_sent": 3,
            "total_bytes": 60524,
            "audio_duration_seconds": 1.89,
            "text_chunked": false
          }
        },
        {
          "success": true,
          "latency_ms": 3923.752773553133,
          "ttfb_ms": 1551.2335561215878,
          "chunks_sent": 4,
          "total_bytes": 114604,
          "audio_seconds": 3.58,
          "speaker": "vardan",
          "error": null,
          "inter_chunk_ms": [
            2266.4989409968257,
            33.145842142403126,
            0.004777684807777405
          ],
          "tail_emit_speed_x": 1.5089445740615142,
          "final_metrics": {
            "ttfb_ms": 1534,
            "chunks_sent": 4,
            "total_bytes": 114604,
            "audio_duration_seconds": 3.58,
            "text_chunked": false
          }
        },
        {
          "success": true,
          "latency_ms": 3923.7419739365578,
          "ttfb_ms": 1554.816504009068,
          "chunks_sent": 4,
          "total_bytes": 116524,
          "audio_seconds": 3.64,
          "speaker": "reet",
          "error": null,
          "inter_chunk_ms": [
            2329.5780699700117,
            33.860452473163605,
            0.003857538104057312
          ],
          "tail_emit_speed_x": 1.536561637843092,
          "final_metrics": {
            "ttfb_ms": 1538,
            "chunks_sent": 4,
            "total_bytes": 116524,
            "audio_duration_seconds": 3.64,
            "text_chunked": false
          }
        },
        {
          "success": true,
          "latency_ms": 3923.803750425577,
          "ttfb_ms": 1614.2752133309841,
          "chunks_sent": 3,
          "total_bytes": 60524,
          "audio_seconds": 1.89,
          "speaker": "Nandini",
          "error": null,
          "inter_chunk_ms": [
            2304.8293190076947,
            0.006983056664466858
          ],
          "tail_emit_speed_x": 0.8183488403125065,
          "final_metrics": {
            "ttfb_ms": 1597,
            "chunks_sent": 3,
            "total_bytes": 60524,
            "audio_duration_seconds": 1.89,
            "text_chunked": false
          }
        },
        {
          "success": true,
          "latency_ms": 3923.7300315871835,
          "ttfb_ms": 1870.6102659925818,
          "chunks_sent": 3,
          "total_bytes": 60524,
          "audio_seconds": 1.89,
          "speaker": "krishna",
          "error": null,
          "inter_chunk_ms": [
            2047.8754863142967,
            0.009818002581596375
          ],
          "tail_emit_speed_x": 0.9205502921319543,
          "final_metrics": {
            "ttfb_ms": 1853,
            "chunks_sent": 3,
            "total_bytes": 60524,
            "audio_duration_seconds": 1.89,
            "text_chunked": false
          }
        },
        {
          "success": true,
          "latency_ms": 3923.793191090226,
          "ttfb_ms": 1909.6740605309606,
          "chunks_sent": 3,
          "total_bytes": 60524,
          "audio_seconds": 1.89,
          "speaker": "anika",
          "error": null,
          "inter_chunk_ms": [
            2009.7842803224921,
            0.0070855021476745605
          ],
          "tail_emit_speed_x": 0.9383754770628681,
          "final_metrics": {
            "ttfb_ms": 1891,
            "chunks_sent": 3,
            "total_bytes": 60524,
            "audio_duration_seconds": 1.89,
            "text_chunked": false
          }
        },
        {
          "success": true,
          "latency_ms": 3923.7535865977407,
          "ttfb_ms": 1902.2414833307266,
          "chunks_sent": 3,
          "total_bytes": 60524,
          "audio_seconds": 1.89,
          "speaker": "lipakshi",
          "error": null,
          "inter_chunk_ms": [
            2016.7261296883225,
            0.008386559784412384
          ],
          "tail_emit_speed_x": 0.9349436973172338,
          "final_metrics": {
            "ttfb_ms": 1884,
            "chunks_sent": 3,
            "total_bytes": 60524,
            "audio_duration_seconds": 1.89,
            "text_chunked": false
          }
        },
        {
          "success": true,
          "latency_ms": 3923.7805558368564,
          "ttfb_ms": 1932.8827988356352,
          "chunks_sent": 3,
          "total_bytes": 60524,
          "audio_seconds": 1.89,
          "speaker": "vardan",
          "error": null,
          "inter_chunk_ms": [
            1986.712159588933,
            0.006664544343948364
          ],
          "tail_emit_speed_x": 0.9493204728136326,
          "final_metrics": {
            "ttfb_ms": 1914,
            "chunks_sent": 3,
            "total_bytes": 60524,
            "audio_duration_seconds": 1.89,
            "text_chunked": false
          }
        },
        {
          "success": true,
          "latency_ms": 3923.683663830161,
          "ttfb_ms": 1883.6068585515022,
          "chunks_sent": 3,
          "total_bytes": 60524,
          "audio_seconds": 1.89,
          "speaker": "reet",
          "error": null,
          "inter_chunk_ms": [
            2034.9266743287444,
            0.008786097168922424
          ],
          "tail_emit_speed_x": 0.9264357082584646,
          "final_metrics": {
            "ttfb_ms": 1864,
            "chunks_sent": 3,
            "total_bytes": 60524,
            "audio_duration_seconds": 1.89,
            "text_chunked": false
          }
        },
        {
          "success": true,
          "latency_ms": 3923.659035935998,
          "ttfb_ms": 1931.4849441871047,
          "chunks_sent": 3,
          "total_bytes": 60524,
          "audio_seconds": 1.89,
          "speaker": "Nandini",
          "error": null,
          "inter_chunk_ms": [
            1987.2056460008025,
            0.007473863661289215
          ],
          "tail_emit_speed_x": 0.9487122675813957,
          "final_metrics": {
            "ttfb_ms": 1912,
            "chunks_sent": 3,
            "total_bytes": 60524,
            "audio_duration_seconds": 1.89,
            "text_chunked": false
          }
        },
        {
          "success": true,
          "latency_ms": 3923.880965448916,
          "ttfb_ms": 1965.7335421070457,
          "chunks_sent": 3,
          "total_bytes": 60524,
          "audio_seconds": 1.89,
          "speaker": "krishna",
          "error": null,
          "inter_chunk_ms": [
            1955.0478123128414,
            0.006311573088169098
          ],
          "tail_emit_speed_x": 0.9651980118914812,
          "final_metrics": {
            "ttfb_ms": 1946,
            "chunks_sent": 3,
            "total_bytes": 60524,
            "audio_duration_seconds": 1.89,
            "text_chunked": false
          }
        },
        {
          "success": true,
          "latency_ms": 3923.8517517223954,
          "ttfb_ms": 1964.737239293754,
          "chunks_sent": 3,
          "total_bytes": 60524,
          "audio_seconds": 1.89,
          "speaker": "anika",
          "error": null,
          "inter_chunk_ms": [
            1955.8603586629033,
            0.006921589374542236
          ],
          "tail_emit_speed_x": 0.9647215555853533,
          "final_metrics": {
            "ttfb_ms": 1944,
            "chunks_sent": 3,
            "total_bytes": 60524,
            "audio_duration_seconds": 1.89,
            "text_chunked": false
          }
        },
        {
          "success": true,
          "latency_ms": 3923.6315861344337,
          "ttfb_ms": 1881.0556828975677,
          "chunks_sent": 3,
          "total_bytes": 60524,
          "audio_seconds": 1.89,
          "speaker": "lipakshi",
          "error": null,
          "inter_chunk_ms": [
            2037.6863358542323,
            0.00855419784784317
          ],
          "tail_emit_speed_x": 0.9253022112935537,
          "final_metrics": {
            "ttfb_ms": 1860,
            "chunks_sent": 3,
            "total_bytes": 60524,
            "audio_duration_seconds": 1.89,
            "text_chunked": false
          }
        },
        {
          "success": true,
          "latency_ms": 3923.7888120114803,
          "ttfb_ms": 1952.4205913767219,
          "chunks_sent": 3,
          "total_bytes": 60524,
          "audio_seconds": 1.89,
          "speaker": "vardan",
          "error": null,
          "inter_chunk_ms": [
            1967.8985886275768,
            0.008073635399341583
          ],
          "tail_emit_speed_x": 0.958725001355374,
          "final_metrics": {
            "ttfb_ms": 1931,
            "chunks_sent": 3,
            "total_bytes": 60524,
            "audio_duration_seconds": 1.89,
            "text_chunked": false
          }
        },
        {
          "success": true,
          "latency_ms": 3923.8250013440847,
          "ttfb_ms": 1980.6726891547441,
          "chunks_sent": 3,
          "total_bytes": 60524,
          "audio_seconds": 1.89,
          "speaker": "reet",
          "error": null,
          "inter_chunk_ms": [
            1939.9735303595662,
            0.007463619112968445
          ],
          "tail_emit_speed_x": 0.972646347969782,
          "final_metrics": {
            "ttfb_ms": 1959,
            "chunks_sent": 3,
            "total_bytes": 60524,
            "audio_duration_seconds": 1.89,
            "text_chunked": false
          }
        },
        {
          "success": true,
          "latency_ms": 3923.8292379304767,
          "ttfb_ms": 1983.1554740667343,
          "chunks_sent": 3,
          "total_bytes": 60524,
          "audio_seconds": 1.89,
          "speaker": "Nandini",
          "error": null,
          "inter_chunk_ms": [
            1937.6561669632792,
            0.007432885468006134
          ],
          "tail_emit_speed_x": 0.9738885716871575,
          "final_metrics": {
            "ttfb_ms": 1961,
            "chunks_sent": 3,
            "total_bytes": 60524,
            "audio_duration_seconds": 1.89,
            "text_chunked": false
          }
        },
        {
          "success": true,
          "latency_ms": 3923.782851547003,
          "ttfb_ms": 1952.7635015547276,
          "chunks_sent": 3,
          "total_bytes": 60524,
          "audio_seconds": 1.89,
          "speaker": "krishna",
          "error": null,
          "inter_chunk_ms": [
            1967.6955426111817,
            0.006932765245437622
          ],
          "tail_emit_speed_x": 0.9588946957864249,
          "final_metrics": {
            "ttfb_ms": 1931,
            "chunks_sent": 3,
            "total_bytes": 60524,
            "audio_duration_seconds": 1.89,
            "text_chunked": false
          }
        },
        {
          "success": true,
          "latency_ms": 3923.7503400072455,
          "ttfb_ms": 1958.8462868705392,
          "chunks_sent": 3,
          "total_bytes": 60524,
          "audio_seconds": 1.89,
          "speaker": "anika",
          "error": null,
          "inter_chunk_ms": [
            1961.5153344348073,
            0.007623806595802307
          ],
          "tail_emit_speed_x": 0.9618790276211543,
          "final_metrics": {
            "ttfb_ms": 1936,
            "chunks_sent": 3,
            "total_bytes": 60524,
            "audio_duration_seconds": 1.89,
            "text_chunked": false
          }
        },
        {
          "success": true,
          "latency_ms": 3923.539189621806,
          "ttfb_ms": 1911.6989811882377,
          "chunks_sent": 3,
          "total_bytes": 60524,
          "audio_seconds": 1.89,
          "speaker": "lipakshi",
          "error": null,
          "inter_chunk_ms": [
            2006.786440499127,
            0.00872742384672165
          ],
          "tail_emit_speed_x": 0.9394384266092215,
          "final_metrics": {
            "ttfb_ms": 1889,
            "chunks_sent": 3,
            "total_bytes": 60524,
            "audio_duration_seconds": 1.89,
            "text_chunked": false
          }
        },
        {
          "success": true,
          "latency_ms": 3923.592190258205,
          "ttfb_ms": 1960.7274429872632,
          "chunks_sent": 3,
          "total_bytes": 60524,
          "audio_seconds": 1.89,
          "speaker": "vardan",
          "error": null,
          "inter_chunk_ms": [
            1958.6054617539048,
            0.007113441824913025
          ],
          "tail_emit_speed_x": 0.9628783657293509,
          "final_metrics": {
            "ttfb_ms": 1938,
            "chunks_sent": 3,
            "total_bytes": 60524,
            "audio_duration_seconds": 1.89,
            "text_chunked": false
          }
        },
        {
          "success": true,
          "latency_ms": 3923.5643688589334,
          "ttfb_ms": 1927.1126249805093,
          "chunks_sent": 3,
          "total_bytes": 60524,
          "audio_seconds": 1.89,
          "speaker": "reet",
          "error": null,
          "inter_chunk_ms": [
            1992.0363118872046,
            0.008266419172286987
          ],
          "tail_emit_speed_x": 0.9466795307200239,
          "final_metrics": {
            "ttfb_ms": 1904,
            "chunks_sent": 3,
            "total_bytes": 60524,
            "audio_duration_seconds": 1.89,
            "text_chunked": false
          }
        },
        {
          "success": true,
          "latency_ms": 3923.8435961306095,
          "ttfb_ms": 1992.1996798366308,
          "chunks_sent": 3,
          "total_bytes": 60524,
          "audio_seconds": 1.89,
          "speaker": "Nandini",
          "error": null,
          "inter_chunk_ms": [
            1929.296888411045,
            0.006961636245250702
          ],
          "tail_emit_speed_x": 0.978441204435921,
          "final_metrics": {
            "ttfb_ms": 1968,
            "chunks_sent": 3,
            "total_bytes": 60524,
            "audio_duration_seconds": 1.89,
            "text_chunked": false
          }
        },
        {
          "success": true,
          "latency_ms": 3923.7756486982107,
          "ttfb_ms": 2003.7997914478183,
          "chunks_sent": 3,
          "total_bytes": 60524,
          "audio_seconds": 1.89,
          "speaker": "krishna",
          "error": null,
          "inter_chunk_ms": [
            1917.1996153891087,
            0.007243826985359192
          ],
          "tail_emit_speed_x": 0.9843873780302003,
          "final_metrics": {
            "ttfb_ms": 1980,
            "chunks_sent": 3,
            "total_bytes": 60524,
            "audio_duration_seconds": 1.89,
            "text_chunked": false
          }
        },
        {
          "success": true,
          "latency_ms": 3923.7454487010837,
          "ttfb_ms": 1996.1626743897796,
          "chunks_sent": 3,
          "total_bytes": 60524,
          "audio_seconds": 1.89,
          "speaker": "anika",
          "error": null,
          "inter_chunk_ms": [
            1924.6436664834619,
            0.008835457265377045
          ],
          "tail_emit_speed_x": 0.9805026405028278,
          "final_metrics": {
            "ttfb_ms": 1972,
            "chunks_sent": 3,
            "total_bytes": 60524,
            "audio_duration_seconds": 1.89,
            "text_chunked": false
          }
        },
        {
          "success": true,
          "latency_ms": 3923.800826072693,
          "ttfb_ms": 2239.4884740933776,
          "chunks_sent": 3,
          "total_bytes": 60524,
          "audio_seconds": 1.89,
          "speaker": "lipakshi",
          "error": null,
          "inter_chunk_ms": [
            1681.8894380703568,
            0.009789131581783295
          ],
          "tail_emit_speed_x": 1.1221196577813914,
          "final_metrics": {
            "ttfb_ms": 2215,
            "chunks_sent": 3,
            "total_bytes": 60524,
            "audio_duration_seconds": 1.89,
            "text_chunked": false
          }
        },
        {
          "success": true,
          "latency_ms": 3923.8131679594517,
          "ttfb_ms": 2231.5897531807423,
          "chunks_sent": 3,
          "total_bytes": 60524,
          "audio_seconds": 1.89,
          "speaker": "vardan",
          "error": null,
          "inter_chunk_ms": [
            1690.0204988196492,
            0.006661750376224518
          ],
          "tail_emit_speed_x": 1.1168738025334282,
          "final_metrics": {
            "ttfb_ms": 2206,
            "chunks_sent": 3,
            "total_bytes": 60524,
            "audio_duration_seconds": 1.89,
            "text_chunked": false
          }
        },
        {
          "success": true,
          "latency_ms": 3923.7209148705006,
          "ttfb_ms": 2222.7585511282086,
          "chunks_sent": 3,
          "total_bytes": 60524,
          "audio_seconds": 1.89,
          "speaker": "reet",
          "error": null,
          "inter_chunk_ms": [
            1698.1047373265028,
            0.008667819201946259
          ],
          "tail_emit_speed_x": 1.111135696054912,
          "final_metrics": {
            "ttfb_ms": 2197,
            "chunks_sent": 3,
            "total_bytes": 60524,
            "audio_duration_seconds": 1.89,
            "text_chunked": false
          }
        },
        {
          "success": true,
          "latency_ms": 3923.7808575853705,
          "ttfb_ms": 2248.299007304013,
          "chunks_sent": 3,
          "total_bytes": 60524,
          "audio_seconds": 1.89,
          "speaker": "Nandini",
          "error": null,
          "inter_chunk_ms": [
            1673.2056951150298,
            0.008487142622470856
          ],
          "tail_emit_speed_x": 1.1280337054576979,
          "final_metrics": {
            "ttfb_ms": 2222,
            "chunks_sent": 3,
            "total_bytes": 60524,
            "audio_duration_seconds": 1.89,
            "text_chunked": false
          }
        }
      ]
    }
  ]
}