role: user

type: text

text:
<user_query>
Explore the /home/ubuntu/veenaModal directory thoroughly. I need to understand:
1. What this project does (it seems to be a TTS system on Modal)
2. How to run it - look for main entry points, Makefiles, scripts, README
3. What dependencies it needs
4. Any environment variables or config needed

Check these files specifically:
- README.md
- Makefile
- Any *.py files in the root
- scripts/ directory
- requirements.txt or pyproject.toml
- Any .env or config files

Return a comprehensive summary of: what the project is, its entry points, and the exact command(s) needed to run it.
</user_query>


role: assistant

text:
Let me explore the directory structure and key files in parallel.

Let me explore the /home/ubuntu/veenaModal directory thoroughly to understand the project structure, entry points, dependencies, and configuration.


text: Now let me read the main application files and scripts.


text: Now let me look at the remaining key source files to understand the internals.


text: Let me also quickly check the GitHub workflows directory and any Makefile.


text:
Here is a comprehensive summary of the `/home/ubuntu/veenaModal` project.

---

## What the Project Is

**Veena3 TTS** is a **GPU-accelerated Text-to-Speech service** built on [Modal](https://modal.com/) (serverless GPU cloud). It uses the **Spark TTS** model architecture (specifically a fine-tuned 0.5B parameter model `BayAreaBoys/spark_tts_4speaker`) with **vLLM** for high-throughput LLM inference and **BiCodec** for audio tokenization/decoding.

Key capabilities:
- **12 pre-trained speakers** (e.g., `lipakshi`, `vardan`, `Mira`, etc.) with friendly name aliases (e.g., `Mitra` -> `lipakshi`)
- **10 emotion tags** like `[angry]`, `[excited]`, `[whispers]`, etc.
- **True streaming TTS** via sliding-window pipeline (first audio bytes emitted before full generation)
- **Non-streaming mode** with multiple output formats (WAV, Opus, MP3, FLAC, mu-law)
- **Optional super-resolution** (16kHz -> 48kHz via AP-BWE)
- **Long text chunking** with crossfade stitching for voice consistency
- **Autoscaling** from 0 to 50 containers, with memory snapshots for fast cold starts
- FastAPI REST API + WebSocket support
- Prometheus metrics, structured logging, auth (API keys via Supabase), rate limiting

---

## Project Structure

```
veenaModal/
  veena3modal/           # Main Python package
    app.py               # Modal entrypoint (deploy/serve target)
    local_server.py      # Local dev server (no Modal required)
    __main__.py           # python -m veena3modal shortcut
    api/                 # FastAPI app, auth, rate limiter, WebSocket
    core/                # Inference: pipeline, streaming, BiCodec decoder, vLLM model loader
    processing/          # Text normalizer, prompt builder, long text chunker
    audio/               # WAV utils, crossfade, format encoders
    services/            # TTS runtime singleton, sentence storage
    shared/              # Logging, metrics
    tests/               # Unit, integration, edge case, performance, Modal live tests
  external/
    sparktts/            # Vendored SparkTTS BiCodec code
    AP-BWE/              # Vendored super-resolution model
  models/                # Local model weights (gitignored, downloaded on demand)
  scripts/               # Stress tests, validation scripts, setup
  .env.example           # Environment variable template
  requirements.txt       # Python dependencies
```

---

## Entry Points & How to Run

### 1. Deploy to Modal (Production)

```bash
# Serve interactively (hot-reload, for testing)
modal serve veena3modal/app.py

# Deploy as persistent service
modal deploy veena3modal/app.py

# Debug imports on a GPU container
modal run veena3modal/app.py::debug_imports

# Run local entrypoint (calls debug_imports remotely)
modal run veena3modal/app.py
```

This provisions L40S GPUs with autoscaling (1-50 containers, 20 concurrent requests per container). Model weights live on a Modal Volume named `veena3-models`. Secrets come from a Modal Secret named `veena3-secrets`.

### 2. Run Locally (GPU machine)

```bash
# One-time setup (creates venv, installs deps, downloads model)
bash scripts/setup_local.sh

# Activate venv
source venv/bin/activate

# Start local server (auto-downloads model from HuggingFace if missing)
python -m veena3modal.local_server

# Or equivalently:
python -m veena3modal

# With options:
python -m veena3modal.local_server --port 8080 --gpu-memory 0.5
python -m veena3modal.local_server --enable-sr --sr-path /path/to/ap_bwe
python -m veena3modal.local_server --device cpu  # CPU-only (very slow)
python -m veena3modal.local_server --num-engines 2  # Multi-engine mode
```

### 3. Test the API

```bash
# Health check
curl http://localhost:8000/v1/tts/health

# Generate speech (non-streaming)
curl -X POST http://localhost:8000/v1/tts/generate \
  -H "Content-Type: application/json" \
  -d '{"text": "[excited] Hello! This is a test.", "speaker": "lipakshi", "stream": false, "format": "wav"}' \
  -o output.wav

# Generate speech (streaming)
curl -X POST http://localhost:8000/v1/tts/generate \
  -H "Content-Type: application/json" \
  -d '{"text": "Hello, streaming test.", "speaker": "lipakshi", "stream": true, "format": "wav"}' \
  -o output.wav

# Metrics
curl http://localhost:8000/v1/tts/metrics
```

### 4. Run Tests

```bash
source venv/bin/activate
pytest -q veena3modal/tests/unit
pytest -q veena3modal/tests --ignore=veena3modal/tests/modal_live
```

---

## Dependencies

From `requirements.txt` and `app.py` image definition:

| Category | Packages |
|---|---|
| **ML/AI** | `torch>=2.0`, `torchaudio>=2.0`, `transformers>=4.30`, `vllm>=0.4`, `accelerate>=0.20` |
| **Audio** | `soundfile>=0.12`, `librosa>=0.10`, `scipy>=1.10`, `numpy>=1.24` |
| **Web** | `fastapi[standard]>=0.100`, `uvicorn[standard]>=0.23` |
| **Tokenizers** | `tiktoken>=0.5`, `sentencepiece>=0.1.99` |
| **SparkTTS deps** | `omegaconf>=2.3`, `safetensors>=0.5`, `soxr>=0.5`, `einops>=0.6`, `einx>=0.2` |
| **AP-BWE deps** | `matplotlib>=3.8`, `natsort>=8.0`, `joblib>=1.0` |
| **Monitoring/Auth** | `prometheus_client>=0.17`, `supabase>=2.0`, `pydantic>=2.0` |
| **Model download** | `huggingface-hub[cli]>=0.24` |
| **System** | `ffmpeg` (apt package, for opus/mp3/flac encoding) |

---

## Environment Variables & Config

From `.env.example`:

| Variable | Purpose | Default |
|---|---|---|
| `AUTH_BYPASS_MODE` | Skip API key validation | `true` (local dev) |
| `MODEL_PATH` | Path to Spark TTS model | `/models/spark_tts_4speaker` (Modal) or `./models/spark_tts_4speaker` (local) |
| `SPARK_TTS_MODEL_PATH` | Alias for MODEL_PATH | Same |
| `BICODEC_MODEL_PATH` | BiCodec model path | Same as MODEL_PATH |
| `AP_BWE_CHECKPOINT_DIR` | Super-resolution checkpoints | `/models/ap_bwe/16kto48k` |
| `GPU_MEMORY_UTILIZATION` | vLLM GPU memory fraction | `0.25` (local), `0.85` (Modal) |
| `HF_TOKEN` | HuggingFace token for private model download | (from `.env`) |
| `SUPABASE_URL` / `SUPABASE_KEY` | Sentence storage (optional) | |
| `VEENA3_STREAM_ADMISSION_MAX_INFLIGHT` | Max concurrent streams | `0` (disabled) |
| `VEENA3_BICODEC_BATCHING` | Enable BiCodec batch decoding | `true` |
| `PRECOMPUTE_SPEAKER_GLOBALS` | Warm speaker cache at startup | `false` |

For **Modal deployment**, secrets are stored in the Modal dashboard under the name `veena3-secrets`, and model weights go on a Modal Volume named `veena3-models`.

---

## No Makefile

There is **no Makefile** in the project. The primary entry points are:
- `scripts/setup_local.sh` for one-time local setup
- `python -m veena3modal.local_server` for local serving
- `modal serve/deploy veena3modal/app.py` for Modal deployment