Perfect! Now I have enough information. Let me create a comprehensive report:

## Summary: Hindi Language Handling in Pipecat Codebase

Based on my thorough analysis of the pipecat codebase, here's how Hindi language users are handled:

### 1. **Language-Based Routing & Pipeline Selection**

**File: `/home/ubuntu/pipecat/src/bot/core/main.py` (Lines 23, 38, 62-68, 74-77, 161, 167)**

The bot accepts `language` as a parameter and uses it for routing:
```python
async def run_bot_webrtc_global_agent(
    ...,
    language: str = None,  # Language passed from client
    ...
):
    bot_session.language = language

    # DB-based language override: Fetch device info early—use DB language instead of frontend-sent value
    if device_id != "common":
        device_info = await device_repo.get_device(device_id)
        if device_info and device_info.get("language"):
            language = device_info["language"]  # DB takes precedence
            bot_session.language = language
            logger.info(f"LANGUAGE_FROM_DB: device={device_id} language={language}")
```

**Key routing decision: DB-stored language overrides frontend language parameter.**

### 2. **Gemini Live Service & Language Mapping**

**File: `/home/ubuntu/pipecat/src/bot/core/pipeline.py` (Lines 533-606)**

Hindi language is explicitly mapped to Gemini Live service:
```python
def create_gemini_live_service(system_instruction: str, voice_id: str = None, ..., language: str = None):
    """Maps language codes to Gemini Live Language enum"""
    from pipecat.transcriptions.language import Language

    _LANG_MAP = {
        "te": Language.TE_IN,
        "hi": Language.HI_IN,  # Hindi (India)
        "en": Language.EN_US,
        "es": Language.ES_ES,
        "fr": Language.FR_FR,
        "pt": Language.PT_BR,
        "id": Language.ID_ID,
        "de": Language.DE_DE,
        "ja": Language.JA_JP,
        "ar": Language.AR_SA,
        "ko": Language.KO_KR,
        "vi": Language.VI_VN,
    }
    gemini_language = _LANG_MAP.get(language, Language.EN_US)

    llm = GeminiLiveVertexLLMService(
        settings=GeminiLiveVertexLLMService.Settings(
            voice=voice,
            language=gemini_language,  # Sets STT/TTS language
        ),
    )
```

**Hindi users get `Language.HI_IN` for both STT and TTS in Gemini Live pipeline.**

### 3. **TTS/STT Model Selection**

**File: `/home/ubuntu/pipecat/src/config/constants.py` (Lines 82-83)**

Default STT configuration:
```python
STT_MODEL = "gpt-4o-mini-transcribe-2025-12-15"
STT_LANGUAGE = "hi"  # Default to Hindi
```

**File: `/home/ubuntu/pipecat/src/bot/core/pipeline.py` (Lines 130-153, 391-427)**

For mobile (non-Gemini Live) pipelines:
```python
def _create_cf_nova3_stt_service():
    """CF Workers AI Deepgram Nova-3 STT service (Hindi-optimized)"""
    stt = DeepgramSTTService(
        settings=DeepgramSTTService.Settings(
            model="nova-3-general",
            language=Language.HI,  # Explicitly set to Hindi
            interim_results=True,
            punctuate=True,
            ...
        ),
    )
```

**TTS Service:**
```python
def create_gemini_tts_service(voice: str = None):
    """Uses Google Chirp3 HD for Telugu"""
    tts = GoogleTTSService(
        settings=GoogleTTSService.Settings(
            voice="te-IN-Chirp3-HD-Erinome",
            language="te-IN",
        ),
    )
```

### 4. **Config & DB-Based Language Routing**

**File: `/home/ubuntu/pipecat/src/repository/config_v2_repository.py` (Lines 12-27, 72-122, 125-161, 192-213)**

Languages are managed in DB and filtered by region:
```python
_DEFAULT_LANGUAGES_INDIA = [
    {"code": "te", "native_label": "తెలుగు", "script_char": "త", "english_label": "Telugu"},
    {"code": "hi", "native_label": "हिन्दी", "script_char": "ह", "english_label": "Hindi"}
]

async def get_languages(self, region: str = None) -> List[Dict[str, Any]]:
    """Get supported languages from config, optionally filtered by region."""
    # Returns Hindi for India region
```

Language-specific prompts stored in `configurations_v2`:
```python
async def get_language_prompt(self, lang_code: str) -> str:
    """Get language instruction for language code (hi, te)"""
    # Fetches cached language instructions like:
    _FALLBACK_PROMPTS = {
        "hi": "Speak in Hinglish — Hindi mixed with English words naturally. Use Roman script.",
        ...
    }
```

**Per-persona Hindi intros:**
```python
async def get_language_intro(self, lang_code: str, persona_id: str) -> str:
    """Get language-specific intro for a persona from configurations_v2['language_intros']"""
    # Looks up: language_intros[persona_id]["hi"] with fallback to "hi"
```

### 5. **System Prompt Assembly**

**File: `/home/ubuntu/pipecat/src/bot/core/config.py` (Lines 360-412)**

Full system prompt is assembled with language-specific instructions:
```python
async def load_system_instruction(
    language: Optional[str] = None,
    persona_prompt: Optional[str] = None,
    persona_id: Optional[str] = None,
) -> str:
    lang_code = language or "hi"  # Hindi is default!

    # 1. Global prompt from DB or file
    # 2. Language instruction: await config_v2_repo.get_language_prompt(lang_code)
    # 3. Language intro (per persona): await config_v2_repo.get_language_intro(lang_code, persona_id)
    # 4. Personality: persona's system_prompt

    # Assemble: global + language + intro + personality
    parts = [global_prompt]
    if language_instruction:
        parts.append(f"LANGUAGE:\n{language_instruction}")
    if language_intro:
        parts.append(f"INTRO (say this when user joins):\n{language_intro}")
```

### 6. **Language-Based Error Messages**

**File: `/home/ubuntu/pipecat/src/config/constants.py` (Lines 161-185)**

Hindi-specific error messages:
```python
ERROR_NO_IMAGE_FOR_EDIT = "कोई image नहीं मिली। पहले image बनाओ या upload करो, फिर edit करूंगी।"
ERROR_IMAGE_GENERATE_FAILED = "Image generate नहीं हो सकी।"
ERROR_SERVER_BUSY = "Server busy है। थोड़ी देर में try करो।"
```

**File: `/home/ubuntu/pipecat/src/bot/core/helpers.py` (Lines 11-56)**

Language-based error selection:
```python
def _is_hindi(language: str = None) -> bool:
    """Check if the session language is Hindi (default for India)."""
    return language in (None, "hi")

def get_error_messages(error, generation_type: str = "image", language: str = None):
    hi = _is_hindi(language)
    if isinstance(error, ModerationError):
        msg = constants.ERROR_IMAGE_EDIT_FAILED if hi else constants.ERROR_IMAGE_EDIT_FAILED_EN
```

### 7. **Device Language Storage & Region Detection**

**File: `/home/ubuntu/pipecat/src/repository/device_repository.py` (Lines 19-23)**

Region determination affects language selection:
```python
def determine_region(tz: str) -> str:
    """Map timezone to region. Asia/Kolkata = india (Hindi region)"""
    if tz in ("Asia/Kolkata", "Asia/Calcutta"):
        return "india"
    return "global"
```

**File: `/home/ubuntu/pipecat/src/service/app_v2_service.py` (Lines 12-22, 25-102)**

Language is set at onboarding and stored in device table:
```python
async def set_device_language(*, device_id: str, language: str) -> dict:
    """Called after user picks language (hi for Hindi)"""
    success = await device_repo.onboard_device(device_id, region, language, country)
    logger.info(f"DEVICE_ONBOARDED: device_id={device_id} region={region} language={language}")
```

### 8. **API Entry Points**

**File: `/home/ubuntu/pipecat/src/routes/bot.py` (Lines 178-219)**

Language is passed in session connection:
```python
@router.post("/start")
async def bot_connect(data: BotConnectRequest) -> Dict[Any, Any]:
    language = data.language

    params = {
        "language": language,  # Passed to pipeline
    }
```

**File: `/home/ubuntu/pipecat/src/routes/v2.py` (Lines 25-53)**

On app open, Hindi is returned as option for India region:
```python
@router.post("/set_language")
async def set_language(device_id: str, language: str):
    """Called after user selects 'hi' on onboarding screen"""
    result = await set_device_language(device_id=device_id, language=language)

@router.get("/on_open")
async def on_open(device_id: str = None, timezone: str = None):
    """Returns languages filtered by region"""
    languages = await config_v2_repo.get_languages(region=region)
    # India region → includes Hindi
```

---

## Key Findings

1. **Hindi is the DEFAULT language** (`language or "hi"` in load_system_instruction)
2. **DB-stored language takes precedence** over frontend-sent language
3. **Language determines:**
   - STT language (Deepgram nova-3 for mobile, Gemini Live for web)
   - TTS language (Veena3 TTS speaker, customizable per persona)
   - System prompt instructions (language-specific and persona-specific intros)
   - Error messages (bilingual Hindi/English)
4. **Region-based routing:** Asia/Kolkata timezone → India region → Hindi + Telugu options
5. **Gemini Live uses `Language.HI_IN` enum** for Hindi with full STT+TTS support
6. **Config stored in `configurations_v2` DB table** with caching (300s TTL)

Prompt:
In the /home/ubuntu/pipecat codebase, find how Hindi language users are handled. Look for:
1. Any language-based routing or pipeline selection (hindi, hi, hin)
2. How TTS/STT models are selected based on language
3. Any config or DB-based language routing
4. Check server.py, main.py, and any pipeline/bot configuration files

Search thoroughly across src/ directory. Report the relevant code snippets and file paths.


Here's how the pipeline works for **Hindi users**:

### Pipeline Flow

1. **Language Detection**: DB-stored language takes priority over frontend-sent value. Hindi (`"hi"`) is the **default** language.

2. **STT (Speech-to-Text)**:
   - Mobile: **Deepgram Nova-3** with `language=HI`
   - Gemini Live: Uses `Language.HI_IN` enum (handles STT+TTS together)

3. **LLM**: Gemini Live Vertex with Hindi language setting

4. **TTS (Text-to-Speech)**: Google Chirp3 HD voice

5. **System Prompt**: Assembled with:
   - Global prompt
   - Hindi language instruction (from DB): *"Speak in Hinglish — Hindi mixed with English words naturally"*
   - Per-persona Hindi intro
   - Persona personality prompt

6. **Error Messages**: Hindi-specific messages like *"कोई image नहीं मिली"*

### Key Config Points
- `src/config/constants.py` — `STT_LANGUAGE = "hi"` (default)
- `src/bot/core/pipeline.py` — Gemini Live language map: `"hi": Language.HI_IN`
- `src/bot/core/config.py` — `lang_code = language or "hi"` (Hindi is fallback)
- `configurations_v2` DB table — language prompts & intros cached 300s

### Region Routing
- `Asia/Kolkata` timezone → `india` region → Hindi + Telugu options
- Global users → different language set

What specifically did you want to check or change about the Hindi pipeline?


what the fuck .. we have openai + openai llm + veena3tts ryt ? for hindi can u check again


1	"""Pipeline creation for bot."""
2
3	from loguru import logger
4	from pipecat.pipeline.pipeline import Pipeline
5	from pipecat.pipeline.task import PipelineParams, PipelineTask
6	from pipecat.processors.aggregators.openai_llm_context import OpenAILLMContext
7	from pipecat.processors.audio.audio_buffer_processor import AudioBufferProcessor
8	from pipecat.processors.frameworks.rtvi import RTVIConfig, RTVIProcessor
9	from pipecat.processors.transcript_processor import TranscriptProcessor
10	from pipecat.services.openai.stt import OpenAISTTService
11	from pipecat.services.deepgram.stt import DeepgramSTTService
12	from pipecat.services.openai.llm import OpenAILLMService
13	from pipecat.services.google.llm_vertex import GoogleVertexLLMService
14	from pipecat.services.google.gemini_live.vertex.llm import GeminiLiveVertexLLMService
15	from pipecat.services.groq.llm import GroqLLMService
16	from pipecat.services.openrouter.llm import OpenRouterLLMService
17	from typing import Any, Dict
18	from pipecat.transports.smallwebrtc.transport import SmallWebRTCTransport
19	from pipecat.transports.base_transport import TransportParams
20	from pipecat.adapters.schemas.tools_schema import ToolsSchema
21	from pipecat.audio.vad.silero import SileroVADAnalyzer, VADParams
22	from pipecat.observers.loggers.user_bot_latency_log_observer import UserBotLatencyLogObserver
23
24	from pipecat.services.soniox.stt import SonioxSTTService
25	from pipecat.transcriptions.language import Language
26
27	from pipecat.services.google.tts import GoogleTTSService
28
29	from src.config import constants
30	from pipecat.processors.aggregators.llm_text_processor import LLMTextProcessor
31	from src.bot.services.custom_tts import CustomTTSService
32	from src.bot.services.double_sentence_aggregator import DoubleSentenceAggregator
33	from src.bot.core.observers import InfoMetricsObserver, SafeRTVIObserver
34	from src.bot.services.image_service import cf_image_function
35	from src.bot.services.video_service import generate_video_function, generate_video_from_image_function
36	from src.bot.services.ppt_service import generate_ppt_function
37	from src.bot.services.music_service import generate_music_function
38
39
40	class CustomOpenRouterLLMService(OpenRouterLLMService):
41	    """OpenRouter LLM service with provider routing support.
42
43	    Overrides build_chat_completion_params to pass OpenRouter-specific fields
44	    (like 'provider') via extra_body, since the OpenAI SDK rejects unknown
45	    top-level kwargs but allows extra_body for non-standard API fields.
46	    """
47
48	    def __init__(self, *, provider_order: list[str] = None, allow_fallbacks: bool = True, **kwargs):
49	        super().__init__(**kwargs)
50	        self._provider_routing = None
51	        if provider_order:
52	            self._provider_routing = {
53	                "order": provider_order,
54	                "allow_fallbacks": allow_fallbacks,
55	            }
56
57	    def build_chat_completion_params(self, params_from_context: Dict[str, Any]) -> Dict[str, Any]:
58	        params = super().build_chat_completion_params(params_from_context)
59	        if self._provider_routing:
60	            existing_extra = params.get("extra_body", {}) or {}
61	            existing_extra["provider"] = self._provider_routing
62	            params["extra_body"] = existing_extra
63	        return params
64
65
66	def create_vad_analyzer(session_id: str):
67	    """Create VAD analyzer with configured parameters."""
68	    vad = SileroVADAnalyzer(
69	        params=VADParams(
70	            confidence=constants.VAD_CONFIDENCE,
71	            stop_secs=constants.VAD_STOP_SECS,
72	            start_secs=constants.VAD_START_SECS,
73	            min_volume=constants.VAD_MIN_VOLUME,
74	        )
75	    )
76	    logger.info(f"VAD_CREATED: session_id={session_id}")
77	    return vad
78
79
80	def create_transport(webrtc_connection, vad_analyzer):
81	    """Create SmallWebRTC transport with configured parameters."""
82	    return SmallWebRTCTransport(
83	        webrtc_connection,
84	        TransportParams(
85	            audio_in_enabled=True,
86	            audio_out_enabled=True,
87	            video_in_enabled=False,
88	            audio_in_sample_rate=constants.SAMPLE_RATE_16K,
89	            audio_out_sample_rate=48000,
90	            vad_analyzer=vad_analyzer,
91	        ),
92	    )
93
94
95	def create_daily_transport(room_url: str, token: str, vad_analyzer):
96	    """Create Daily transport with configured parameters."""
97	    from pipecat.transports.daily.transport import DailyTransport, DailyParams
98
99	    return DailyTransport(
100	        room_url,
101	        token,
102	        "Maya Global Agent Bot",
103	        DailyParams(
104	            api_key=constants.DAILY_API_KEY,
105	            audio_in_enabled=True,
106	            audio_out_enabled=True,
107	            video_in_enabled=False,
108	            camera_out_enabled=False,
109	            audio_in_sample_rate=constants.SAMPLE_RATE_16K,
110	            audio_out_sample_rate=constants.SAMPLE_RATE_16K,
111	            vad_analyzer=vad_analyzer,
112	            transcription_enabled=False,
113	        ),
114	    )
115
116
117	def _create_openai_stt_service():
118	    """Create OpenAI STT service."""
119	    stt = OpenAISTTService(
120	        api_key=constants.OPENAI_API_KEY,
121	        settings=OpenAISTTService.Settings(
122	            model=constants.STT_MODEL,
123	            language=constants.STT_LANGUAGE,
124	        ),
125	    )
126	    logger.info(f"STT_SERVICE_CREATED: provider=openai model={constants.STT_MODEL}")
127	    return stt
128
129
130	def _create_cf_nova3_stt_service():
131	    """Create CF Workers AI Deepgram Nova-3 STT service.
132
133	    Connects to a CF Worker that proxies WebSocket to @cf/deepgram/nova-3.
134	    Uses Pipecat's DeepgramSTTService with base_url pointing to the CF Worker.
135	    """
136	    stt = DeepgramSTTService(
137	        api_key=constants.CF_STT_API_KEY,
138	        base_url=constants.CF_STT_WORKER_URL,
139	        encoding="linear16",
140	        sample_rate=constants.SAMPLE_RATE_16K,
141	        settings=DeepgramSTTService.Settings(
142	            model="nova-3-general",
143	            language=Language.HI,
144	            interim_results=True,
145	            punctuate=True,
146	            smart_format=False,
147	            endpointing=300,
148	            utterance_end_ms=1000,
149	            vad_events=False,
150	        ),
151	    )
152	    logger.info(f"STT_SERVICE_CREATED: provider=cf_nova3 base_url={constants.CF_STT_WORKER_URL}")
153	    return stt
154
155
156	def create_stt_service():
157	    """Create STT service — OpenAI Whisper (Deepgram disabled due to SDK incompatibility)."""
158	    return _create_openai_stt_service()
159
160
161	def create_soniox_stt_service():
162	    """Create Soniox STT service for Telugu (web sessions)."""
163	    stt = SonioxSTTService(
164	        api_key=constants.SONIOX_API_KEY,
165	        sample_rate=constants.SAMPLE_RATE_16K,
166	        settings=SonioxSTTService.Settings(
167	            model="stt-rt-v4",
168	            language_hints=[Language.TE],
169	        ),
170	    )
171	    logger.info("STT_SERVICE_CREATED: provider=soniox model=stt-rt-v4 language=te")
172	    return stt
173
174
175	def _select_llm_provider_weighted() -> str:
176	    """Select LLM provider using weighted traffic routing.
177
178	    Uses per-provider percentage weights from config. Each provider gets
179	    its configured percentage of traffic, remainder goes to gemini (default).
180
181	    Example config weights: {"openai": 30, "groq": 5, "openrouter": 10}
182	    -> 30% openai, 5% groq, 10% openrouter, 55% gemini
183
184	    Example config weights: {"groq": 5}
185	    -> 5% groq, 95% gemini
186	    """
187	    import random
188	    from src.repository.config_repository import config_repo
189
190	    weights = config_repo.get_llm_weights()
191
192	    # Sanitize: ensure all values are valid numbers, skip bad entries
193	    valid_providers = ["openai", "groq", "openrouter"]  # Known non-default providers
194	    sanitized = {}
195	    for provider, percentage in weights.items():
196	        if provider not in valid_providers:
197	            logger.warning(f"LLM_WEIGHTS_UNKNOWN_PROVIDER: provider={provider} skipped")
198	            continue
199	        try:
200	            pct = int(percentage)
201	            if pct > 0:
202	                sanitized[provider] = pct
203	        except (TypeError, ValueError):
204	            logger.warning(f"LLM_WEIGHTS_INVALID_VALUE: provider={provider} value={percentage} skipped")
205	            continue
206
207	    # Cap total at 100 to prevent gemini from being completely excluded by accident
208	    total = sum(sanitized.values())
209	    if total > 100:
210	        logger.warning(f"LLM_WEIGHTS_OVERFLOW: total={total} exceeds 100, capping proportionally")
211	        # Scale down proportionally so total = 100
212	        scale = 100.0 / total
213	        sanitized = {k: max(1, int(v * scale)) for k, v in sanitized.items()}
214
215	    # If no valid weights, fall back to gemini
216	    if not sanitized:
217	        logger.info("LLM_WEIGHTED_SELECTION: no valid weights, chosen=gemini (default)")
218	        return "gemini"
219
220	    roll = random.randint(1, 100)
221
222	    # Walk through providers in order, accumulating percentages
223	    cumulative = 0
224	    for provider, percentage in sanitized.items():
225	        cumulative += percentage
226	        if roll <= cumulative:
227	            logger.info(f"LLM_WEIGHTED_SELECTION: weights={sanitized} roll={roll} chosen={provider}")
228	            return provider
229
230	    # Remaining traffic goes to gemini (default)
231	    logger.info(f"LLM_WEIGHTED_SELECTION: weights={sanitized} roll={roll} chosen=gemini (default)")
232	    return "gemini"
233
234
235	def _create_groq_llm_service():
236	    """Create Groq LLM service."""
237	    if not constants.GROQ_API_KEY:
238	        logger.error("GROQ_API_KEY not set! Falling back to OpenAI")
239	        return _create_openai_llm_service()
240	    logger.info(f"LLM_PROVIDER: Groq {constants.GROQ_MODEL} active")
241	    return GroqLLMService(
242	        api_key=constants.GROQ_API_KEY,
243	        model=constants.GROQ_MODEL,
244	        params=GroqLLMService.InputParams(
245	            max_completion_tokens=constants.LLM_MAX_OUTPUT_TOKENS,
246	            temperature=constants.LLM_TEMPERATURE,
247	        ),
248	    )
249
250
251	def _create_openrouter_llm_service():
252	    """Create OpenRouter LLM service routed to Groq provider.
253
254	    Uses CustomOpenRouterLLMService to pass provider routing via extra_body
255	    (the only way to send non-standard fields through the OpenAI SDK).
256	    """
257	    if not constants.OPENROUTER_API_KEY:
258	        logger.error("OPENROUTER_API_KEY not set! Falling back to OpenAI")
259	        return _create_openai_llm_service()
260	    logger.info(f"LLM_PROVIDER: OpenRouter {constants.OPENROUTER_LLM_MODEL} active | provider=groq")
261	    return CustomOpenRouterLLMService(
262	        api_key=constants.OPENROUTER_API_KEY,
263	        model=constants.OPENROUTER_LLM_MODEL,
264	        provider_order=["groq"],
265	        allow_fallbacks=True,
266	        params=OpenRouterLLMService.InputParams(
267	            max_completion_tokens=constants.LLM_MAX_OUTPUT_TOKENS,
268	            temperature=constants.LLM_TEMPERATURE,
269	        ),
270	    )
271
272
273	def _create_openai_llm_service():
274	    """Create OpenAI LLM service."""
275	    logger.info("LLM_PROVIDER: OpenAI GPT-4o-mini active")
276	    return OpenAILLMService(
277	        api_key=constants.OPENAI_API_KEY,
278	        settings=OpenAILLMService.Settings(
279	            model="gpt-4o-mini",
280	            max_completion_tokens=constants.LLM_MAX_OUTPUT_TOKENS,
281	            temperature=constants.LLM_TEMPERATURE,
282	        ),
283	    )
284
285
286	def _create_gemini_llm_service(system_instruction: str):
287	    """Create Google Vertex AI Gemini LLM service with GCP account switching."""
288	    import random
289	    from src.repository.config_repository import config_repo
290
291	    # Get GCP mode from cache (no DB call - refreshed periodically by server)
292	    mode = config_repo.get_gcp_mode()
293
294	    # Determine which account to use
295	    if mode == "random":
296	        new_percentage = config_repo.get_gcp_new_account_percentage()
297	        use_new_account = random.randint(1, 100) <= new_percentage
298	        chosen = "new" if use_new_account else "old"
299	        logger.info(f"GCP_RANDOM_SELECTION: mode=random new_percentage={new_percentage} chosen={chosen}")
300	    else:
301	        use_new_account = (mode == "new")
302	        chosen = mode
303
304	    if use_new_account:
305	        project_id = constants.GCP_NEW_PROJECT_ID
306	        credentials_path = constants.GCP_NEW_CREDENTIALS_PATH
307	        account_tag = "NEW"
308	        logger.info(f"GCP_ACCOUNT: NEW account active | project={project_id} | credentials={credentials_path}")
309	    else:
310	        project_id = constants.GOOGLE_CLOUD_PROJECT
311	        credentials_path = constants.GCP_CREDENTIALS_PATH
312	        account_tag = "OLD"
313	        logger.info(f"GCP_ACCOUNT: OLD account active | project={project_id} | credentials={credentials_path}")
314
315	    # Use global endpoint for better capacity (Google recommends this for Standard PayGo)
316	    # Global endpoint routes to region with most availability, reducing 429 errors
317	    return GoogleVertexLLMService(
318	        name=f"GeminiLLM-{account_tag}",  # Shows in error logs which account caused the error
319	        project_id=project_id,
320	        location="global",  # Global endpoint for multi-region capacity pool
321	        model=constants.GEMINI_MODEL,
322	        credentials_path=credentials_path,
323	        system_instruction=system_instruction,
324	        params=GoogleVertexLLMService.InputParams(
325	            max_output_tokens=constants.LLM_MAX_OUTPUT_TOKENS,
326	            temperature=constants.LLM_TEMPERATURE,
327	        ),
328	    )
329
330
331	def create_llm_service(system_instruction: str):
332	    """Create LLM service with system instruction. Uses cached DB config to switch providers.
333
334	    LLM Provider Modes:
335	    - "gemini": Use Google Vertex AI Gemini (with GCP account switching)
336	    - "openai": Use OpenAI GPT-4o-mini
337	    - "groq": Use Groq (Llama 3.3 70B or configured model)
338	    - "openrouter": Use OpenRouter (access multiple providers via single API)
339	    - "random": Weighted traffic split between providers based on per-provider percentages
340
341	    Random mode traffic routing (via "weights" config):
342	      {"openai": 30, "groq": 5, "openrouter": 10} -> 30% OpenAI, 5% Groq, 10% OpenRouter, 55% Gemini
343	      {"groq": 5}               -> 5% Groq, 95% Gemini
344	      {"openai": 50}            -> 50% OpenAI, 50% Gemini (legacy behavior)
345
346	    GCP Account Modes (only for gemini):
347	    - "old": Always use mayaresearchgcp (old account)
348	    - "new": Always use custom-frame-484619-j2 (new account)
349	    - "random": Randomly choose between old and new based on percentage
350	    """
351	    from src.repository.config_repository import config_repo
352
353	    # Get LLM provider from cache (defaults to "gemini" if not set)
354	    llm_provider = config_repo.get_llm_provider()
355
356	    # Determine which LLM to use
357	    if llm_provider == "random":
358	        chosen_llm = _select_llm_provider_weighted()
359	    else:
360	        chosen_llm = llm_provider
361	    # Create the chosen LLM service
362	    if chosen_llm == "groq":
363	        return _create_groq_llm_service()
364
365	    if chosen_llm == "openai":
366	        return _create_openai_llm_service()
367
368	    if chosen_llm == "openrouter":
369	        return _create_openrouter_llm_service()
370
371	    # Default: Use Google Vertex AI Gemini
372	    return _create_gemini_llm_service(system_instruction)
373
374
375	def create_tts_service(speaker: str = None):
376	    """Create TTS service with optional custom speaker/voice.
377
378	    Args:
379	        speaker: TTS voice to use. If None, uses DEFAULT_TTS_SPEAKER.
380	    """
381	    voice = speaker or constants.DEFAULT_TTS_SPEAKER
382	    logger.info(f"TTS_SERVICE_CREATED: speaker={voice}")
383	    return CustomTTSService(
384	        api_url=constants.TTS_API_URL,
385	        api_key=constants.VEENA3_API_KEY,
386	        speaker=voice,
387	        sample_rate=constants.SAMPLE_RATE_16K,
388	    )
389
390
391	def create_gemini_tts_service(voice: str = None):
392	    """Create Gemini TTS service for web/Telugu sessions.
393
394	    Uses the same GCP account switching logic as Gemini LLM.
395	    Gemini TTS outputs at 24kHz natively.
396	    """
397	    import random
398	    from src.repository.config_repository import config_repo
399
400	    tts_voice = voice or constants.GEMINI_LIVE_VOICE  # "Kore" for Telugu
401
402	    mode = config_repo.get_gcp_mode()
403	    if mode == "random":
404	        new_percentage = config_repo.get_gcp_new_account_percentage()
405	        use_new_account = random.randint(1, 100) <= new_percentage
406	    else:
407	        use_new_account = (mode == "new")
408
409	    if use_new_account:
410	        credentials_path = constants.GCP_NEW_CREDENTIALS_PATH
411	        account_tag = "NEW"
412	    else:
413	        credentials_path = constants.GCP_CREDENTIALS_PATH
414	        account_tag = "OLD"
415
416	    tts = GoogleTTSService(
417	        credentials_path=credentials_path,
418	        location="asia-southeast1",
419	        settings=GoogleTTSService.Settings(
420	            voice="te-IN-Chirp3-HD-Erinome",
421	            language="te-IN",
422	        ),
423	        sample_rate=constants.SAMPLE_RATE,
424	    )
425	    logger.info(f"TTS_SERVICE_CREATED: provider=chirp3-hd voice=te-IN-Chirp3-HD-Erinome language=te-IN location=asia-southeast1 gcp={account_tag}")
426	    return tts
427
428
429	def create_context_and_tools(llm, system_instruction: str, include_video: bool = False, include_ppt: bool = False, include_music: bool = False):
430	    """Create LLM context with tools.
431
432	    All characters get image generation. Video, PPT, and music are controlled by persona config.
433	    """
434	    standard_tools = [cf_image_function]
435
436	    if include_video:
437	        standard_tools.append(generate_video_function)
438	        standard_tools.append(generate_video_from_image_function)
439	        logger.info("PIPELINE_TOOLS: Video generation functions ENABLED (t2v + i2v)")
440
441	    if include_ppt:
442	        standard_tools.append(generate_ppt_function)
443	        logger.info("PIPELINE_TOOLS: PPT generation function ENABLED")
444
445	    if include_music:
446	        standard_tools.append(generate_music_function)
447	        logger.info("PIPELINE_TOOLS: Music generation function ENABLED (Lyria)")
448
449	    logger.debug(f"TOOLS_REGISTERED: image=true video={include_video} ppt={include_ppt} music={include_music} total={len(standard_tools)}")
450
451	    tools = ToolsSchema(standard_tools=standard_tools) if standard_tools else None
452
453	    # Build messages with system instruction for OpenAI compatibility
454	    messages = [
455	        {"role": "system", "content": system_instruction},
456	        {"role": "user", "content": "Start the Conversation :"}
457	    ]
458
459	    context = OpenAILLMContext(
460	        messages=messages,
461	        tools=tools
462	    )
463	    context_aggregator = llm.create_context_aggregator(context)
464	    return context_aggregator
465
466
467	def create_llm_text_processor():
468	    """Create LLM text processor that combines 2 sentences before sending to TTS.
469
470	    This sits between the LLM and TTS in the pipeline. It aggregates streaming
471	    LLM tokens into 2-sentence chunks, reducing TTS API calls by ~half.
472	    """
473	    return LLMTextProcessor(
474	        text_aggregator=DoubleSentenceAggregator(sentences_per_chunk=2),
475	    )
476
477
478	def create_processors():
479	    """Create RTVI, transcript, and audio buffer processors."""
480	    rtvi = RTVIProcessor(config=RTVIConfig(config=[]))
481	    transcript = TranscriptProcessor()
482
483	    audio_buffer = AudioBufferProcessor(
484	        sample_rate=constants.SAMPLE_RATE,
485	        num_channels=1,
486	        buffer_size=constants.SAMPLE_RATE * 2 * constants.CHUNK_DURATION,
487	    )
488
489	    return rtvi, transcript, audio_buffer
490
491
492	def create_pipeline(transport, stt, context_aggregator, rtvi, llm, tts, audio_buffer, transcript, llm_text_processor=None):
493	    """Create the processing pipeline."""
494	    # Build core pipeline: llm -> [optional llm_text_processor] -> tts
495	    llm_to_tts = [llm, llm_text_processor, tts] if llm_text_processor else [llm, tts]
496
497	    pipeline = Pipeline([
498	        transport.input(),
499	        stt,
500	        transcript.user(),
501	        context_aggregator.user(),
502	        rtvi,
503	        *llm_to_tts,
504	        transport.output(),
505	        audio_buffer,
506	        transcript.assistant(),
507	        context_aggregator.assistant(),
508	    ])
509	    mode = "2-sentence aggregation" if llm_text_processor else "default (1 sentence)"
510	    logger.info(f"PIPELINE: Traditional STT mode (Audio → STT → Text → LLM → TTS) | text_aggregation={mode}")
511	    return pipeline
512
513
514	def create_task(pipeline, rtvi, bot_session):
515	    """Create pipeline task with observers."""
516	    return PipelineTask(
517	        pipeline,
518	        params=PipelineParams(
519	            enable_metrics=True,
520	            enable_usage_metrics=True,
521	            allow_interruptions=True
522	        ),
523	        observers=[
524	            SafeRTVIObserver(rtvi),  # Use SafeRTVIObserver to handle startup race condition
525	            InfoMetricsObserver(bot_session),
526	            UserBotLatencyLogObserver()
527	        ],
528	    )
529
530
531	# ---- Gemini Live pipeline (Telugu / multi-language) ----
532
533	def create_gemini_live_service(system_instruction: str, voice_id: str = None, include_video: bool = False, include_ppt: bool = False, include_music: bool = False, language: str = None):
534	    """Create Gemini Live Vertex AI service — handles STT + LLM + TTS in one service.
535
536	    Uses the same GCP account switching logic as the regular Gemini LLM service.
537	    """
538	    import random
539	    from src.repository.config_repository import config_repo
540
541	    voice = voice_id or constants.GEMINI_LIVE_VOICE
542
543	    standard_tools = [cf_image_function]
544	    if include_video:
545	        standard_tools.append(generate_video_function)
546	        standard_tools.append(generate_video_from_image_function)
547	        logger.info("GEMINI_LIVE_TOOLS: Video generation functions ENABLED (t2v + i2v)")
548
549	    if include_ppt:
550	        standard_tools.append(generate_ppt_function)
551	        logger.info("GEMINI_LIVE_TOOLS: PPT generation function ENABLED")
552
553	    if include_music:
554	        standard_tools.append(generate_music_function)
555	        logger.info("GEMINI_LIVE_TOOLS: Music generation function ENABLED (Lyria)")
556
557	    tools = ToolsSchema(standard_tools=standard_tools) if standard_tools else None
558
559	    # GCP account switching (same logic as _create_gemini_llm_service)
560	    mode = config_repo.get_gcp_mode()
561	    if mode == "random":
562	        new_percentage = config_repo.get_gcp_new_account_percentage()
563	        use_new_account = random.randint(1, 100) <= new_percentage
564	    else:
565	        use_new_account = (mode == "new")
566
567	    if use_new_account:
568	        project_id = constants.GCP_NEW_PROJECT_ID
569	        credentials_path = constants.GCP_NEW_CREDENTIALS_PATH
570	        account_tag = "NEW"
571	    else:
572	        project_id = constants.GOOGLE_CLOUD_PROJECT
573	        credentials_path = constants.GCP_CREDENTIALS_PATH
574	        account_tag = "OLD"
575
576	    # Map language code to Language enum for Gemini Live
577	    from pipecat.transcriptions.language import Language
578	    _LANG_MAP = {
579	        "te": Language.TE_IN,
580	        "hi": Language.HI_IN,
581	        "en": Language.EN_US,
582	        "es": Language.ES_ES,
583	        "fr": Language.FR_FR,
584	        "pt": Language.PT_BR,
585	        "id": Language.ID_ID,
586	        "de": Language.DE_DE,
587	        "ja": Language.JA_JP,
588	        "ar": Language.AR_SA,
589	        "ko": Language.KO_KR,
590	        "vi": Language.VI_VN,
591	    }
592	    gemini_language = _LANG_MAP.get(language, Language.EN_US)
593
594	    llm = GeminiLiveVertexLLMService(
595	        credentials_path=credentials_path,
596	        project_id=project_id,
597	        location="us-east4",
598	        system_instruction=system_instruction,
599	        tools=tools,
600	        settings=GeminiLiveVertexLLMService.Settings(
601	            voice=voice,
602	            language=gemini_language,
603	        ),
604	    )
605	    logger.info(f"GEMINI_LIVE_CREATED: voice={voice} language={gemini_language} tools={len(standard_tools)} gcp_account={account_tag} project={project_id}")
606	    return llm
607
608
609	def create_gemini_live_context():
610	    """Create context aggregator for Gemini Live.
611
612	    Uses LLMContext + LLMContextAggregatorPair directly (not deprecated OpenAILLMContext).
613	    System instruction is set on the LLM service via Settings, not in context messages.
614	    VAD is placed inside LLMUserAggregatorParams per official Gemini Live pattern.
615	    """
616	    from pipecat.processors.aggregators.llm_context import LLMContext
617	    from pipecat.processors.aggregators.llm_response_universal import (
618	        LLMContextAggregatorPair,
619	        LLMUserAggregatorParams,
620	    )
621
622	    messages = []
623
624	    context = LLMContext(messages)
625	    context_aggregator = LLMContextAggregatorPair(
626	        context,
627	        user_params=LLMUserAggregatorParams(
628	            vad_analyzer=SileroVADAnalyzer(params=VADParams(stop_secs=0.5)),
629	        ),
630	    )
631	    return context_aggregator
632
633
634	def create_gemini_live_pipeline(transport, gemini_live, context_aggregator, rtvi, audio_buffer, transcript):
635	    """Create pipeline for Gemini Live — audio goes directly to Gemini, no separate STT/TTS.
636
637	    Keeps RTVI (frontend events), audio_buffer (recording), transcript (conversation tracking).
638	    """
639	    pipeline = Pipeline([
640	        transport.input(),
641	        context_aggregator.user(),
642	        gemini_live,
643	        rtvi,
644	        transport.output(),
645	        audio_buffer,
646	        transcript.assistant(),
647	        context_aggregator.assistant(),
648	    ])
649	    logger.info("PIPELINE: Gemini Live mode (Audio → Context → Gemini Live → Audio)")
650	    return pipeline
651
652

"""Pipeline creation for bot."""

from loguru import logger
from pipecat.pipeline.pipeline import Pipeline
from pipecat.pipeline.task import PipelineParams, PipelineTask
from pipecat.processors.aggregators.openai_llm_context import OpenAILLMContext
from pipecat.processors.audio.audio_buffer_processor import AudioBufferProcessor
from pipecat.processors.frameworks.rtvi import RTVIConfig, RTVIProcessor
from pipecat.processors.transcript_processor import TranscriptProcessor
from pipecat.services.openai.stt import OpenAISTTService
from pipecat.services.deepgram.stt import DeepgramSTTService
from pipecat.services.openai.llm import OpenAILLMService
from pipecat.services.google.llm_vertex import GoogleVertexLLMService
from pipecat.services.google.gemini_live.vertex.llm import GeminiLiveVertexLLMService
from pipecat.services.groq.llm import GroqLLMService
from pipecat.services.openrouter.llm import OpenRouterLLMService
from typing import Any, Dict
from pipecat.transports.smallwebrtc.transport import SmallWebRTCTransport
from pipecat.transports.base_transport import TransportParams
from pipecat.adapters.schemas.tools_schema import ToolsSchema
from pipecat.audio.vad.silero import SileroVADAnalyzer, VADParams
from pipecat.observers.loggers.user_bot_latency_log_observer import UserBotLatencyLogObserver

from pipecat.services.soniox.stt import SonioxSTTService
from pipecat.transcriptions.language import Language

from pipecat.services.google.tts import GoogleTTSService

from src.config import constants
from pipecat.processors.aggregators.llm_text_processor import LLMTextProcessor
from src.bot.services.custom_tts import CustomTTSService
from src.bot.services.double_sentence_aggregator import DoubleSentenceAggregator
from src.bot.core.observers import InfoMetricsObserver, SafeRTVIObserver
from src.bot.services.image_service import cf_image_function
from src.bot.services.video_service import generate_video_function, generate_video_from_image_function
from src.bot.services.ppt_service import generate_ppt_function
from src.bot.services.music_service import generate_music_function

class CustomOpenRouterLLMService(OpenRouterLLMService):
    """OpenRouter LLM service with provider routing support.

    Overrides build_chat_completion_params to pass OpenRouter-specific fields
    (like 'provider') via extra_body, since the OpenAI SDK rejects unknown
    top-level kwargs but allows extra_body for non-standard API fields.
    """

    def __init__(self, *, provider_order: list[str] = None, allow_fallbacks: bool = True, **kwargs):
        super().__init__(**kwargs)
        self._provider_routing = None
        if provider_order:
            self._provider_routing = {
                "order": provider_order,
                "allow_fallbacks": allow_fallbacks,
            }

    def build_chat_completion_params(self, params_from_context: Dict[str, Any]) -> Dict[str, Any]:
        params = super().build_chat_completion_params(params_from_context)
        if self._provider_routing:
            existing_extra = params.get("extra_body", {}) or {}
            existing_extra["provider"] = self._provider_routing
            params["extra_body"] = existing_extra
        return params

def create_vad_analyzer(session_id: str):
    """Create VAD analyzer with configured parameters."""
    vad = SileroVADAnalyzer(
        params=VADParams(
            confidence=constants.VAD_CONFIDENCE,
            stop_secs=constants.VAD_STOP_SECS,
            start_secs=constants.VAD_START_SECS,
            min_volume=constants.VAD_MIN_VOLUME,
        )
    )
    logger.info(f"VAD_CREATED: session_id={session_id}")
    return vad

def create_transport(webrtc_connection, vad_analyzer):
    """Create SmallWebRTC transport with configured parameters."""
    return SmallWebRTCTransport(
        webrtc_connection,
        TransportParams(
            audio_in_enabled=True,
            audio_out_enabled=True,
            video_in_enabled=False,
            audio_in_sample_rate=constants.SAMPLE_RATE_16K,
            audio_out_sample_rate=48000,
            vad_analyzer=vad_analyzer,
        ),
    )

def create_daily_transport(room_url: str, token: str, vad_analyzer):
    """Create Daily transport with configured parameters."""
    from pipecat.transports.daily.transport import DailyTransport, DailyParams

    return DailyTransport(
        room_url,
        token,
        "Maya Global Agent Bot",
        DailyParams(
            api_key=constants.DAILY_API_KEY,
            audio_in_enabled=True,
            audio_out_enabled=True,
            video_in_enabled=False,
            camera_out_enabled=False,
            audio_in_sample_rate=constants.SAMPLE_RATE_16K,
            audio_out_sample_rate=constants.SAMPLE_RATE_16K,
            vad_analyzer=vad_analyzer,
            transcription_enabled=False,
        ),
    )

def _create_openai_stt_service():
    """Create OpenAI STT service."""
    stt = OpenAISTTService(
        api_key=constants.OPENAI_API_KEY,
        settings=OpenAISTTService.Settings(
            model=constants.STT_MODEL,
            language=constants.STT_LANGUAGE,
        ),
    )
    logger.info(f"STT_SERVICE_CREATED: provider=openai model={constants.STT_MODEL}")
    return stt

def _create_cf_nova3_stt_service():
    """Create CF Workers AI Deepgram Nova-3 STT service.

    Connects to a CF Worker that proxies WebSocket to @cf/deepgram/nova-3.
    Uses Pipecat's DeepgramSTTService with base_url pointing to the CF Worker.
    """
    stt = DeepgramSTTService(
        api_key=constants.CF_STT_API_KEY,
        base_url=constants.CF_STT_WORKER_URL,
        encoding="linear16",
        sample_rate=constants.SAMPLE_RATE_16K,
        settings=DeepgramSTTService.Settings(
            model="nova-3-general",
            language=Language.HI,
            interim_results=True,
            punctuate=True,
            smart_format=False,
            endpointing=300,
            utterance_end_ms=1000,
            vad_events=False,
        ),
    )
    logger.info(f"STT_SERVICE_CREATED: provider=cf_nova3 base_url={constants.CF_STT_WORKER_URL}")
    return stt

def create_stt_service():
    """Create STT service — OpenAI Whisper (Deepgram disabled due to SDK incompatibility)."""
    return _create_openai_stt_service()

def create_soniox_stt_service():
    """Create Soniox STT service for Telugu (web sessions)."""
    stt = SonioxSTTService(
        api_key=constants.SONIOX_API_KEY,
        sample_rate=constants.SAMPLE_RATE_16K,
        settings=SonioxSTTService.Settings(
            model="stt-rt-v4",
            language_hints=[Language.TE],
        ),
    )
    logger.info("STT_SERVICE_CREATED: provider=soniox model=stt-rt-v4 language=te")
    return stt

def _select_llm_provider_weighted() -> str:
    """Select LLM provider using weighted traffic routing.

    Uses per-provider percentage weights from config. Each provider gets
    its configured percentage of traffic, remainder goes to gemini (default).

    Example config weights: {"openai": 30, "groq": 5, "openrouter": 10}
    -> 30% openai, 5% groq, 10% openrouter, 55% gemini

    Example config weights: {"groq": 5}
    -> 5% groq, 95% gemini
    """
    import random
    from src.repository.config_repository import config_repo

    weights = config_repo.get_llm_weights()

    # Sanitize: ensure all values are valid numbers, skip bad entries
    valid_providers = ["openai", "groq", "openrouter"]  # Known non-default providers
    sanitized = {}
    for provider, percentage in weights.items():
        if provider not in valid_providers:
            logger.warning(f"LLM_WEIGHTS_UNKNOWN_PROVIDER: provider={provider} skipped")
            continue
        try:
            pct = int(percentage)
            if pct > 0:
                sanitized[provider] = pct
        except (TypeError, ValueError):
            logger.warning(f"LLM_WEIGHTS_INVALID_VALUE: provider={provider} value={percentage} skipped")
            continue

    # Cap total at 100 to prevent gemini from being completely excluded by accident
    total = sum(sanitized.values())
    if total > 100:
        logger.warning(f"LLM_WEIGHTS_OVERFLOW: total={total} exceeds 100, capping proportionally")
        # Scale down proportionally so total = 100
        scale = 100.0 / total
        sanitized = {k: max(1, int(v * scale)) for k, v in sanitized.items()}

    # If no valid weights, fall back to gemini
    if not sanitized:
        logger.info("LLM_WEIGHTED_SELECTION: no valid weights, chosen=gemini (default)")
        return "gemini"

    roll = random.randint(1, 100)

    # Walk through providers in order, accumulating percentages
    cumulative = 0
    for provider, percentage in sanitized.items():
        cumulative += percentage
        if roll <= cumulative:
            logger.info(f"LLM_WEIGHTED_SELECTION: weights={sanitized} roll={roll} chosen={provider}")
            return provider

    # Remaining traffic goes to gemini (default)
    logger.info(f"LLM_WEIGHTED_SELECTION: weights={sanitized} roll={roll} chosen=gemini (default)")
    return "gemini"

def _create_groq_llm_service():
    """Create Groq LLM service."""
    if not constants.GROQ_API_KEY:
        logger.error("GROQ_API_KEY not set! Falling back to OpenAI")
        return _create_openai_llm_service()
    logger.info(f"LLM_PROVIDER: Groq {constants.GROQ_MODEL} active")
    return GroqLLMService(
        api_key=constants.GROQ_API_KEY,
        model=constants.GROQ_MODEL,
        params=GroqLLMService.InputParams(
            max_completion_tokens=constants.LLM_MAX_OUTPUT_TOKENS,
            temperature=constants.LLM_TEMPERATURE,
        ),
    )

def _create_openrouter_llm_service():
    """Create OpenRouter LLM service routed to Groq provider.

    Uses CustomOpenRouterLLMService to pass provider routing via extra_body
    (the only way to send non-standard fields through the OpenAI SDK).
    """
    if not constants.OPENROUTER_API_KEY:
        logger.error("OPENROUTER_API_KEY not set! Falling back to OpenAI")
        return _create_openai_llm_service()
    logger.info(f"LLM_PROVIDER: OpenRouter {constants.OPENROUTER_LLM_MODEL} active | provider=groq")
    return CustomOpenRouterLLMService(
        api_key=constants.OPENROUTER_API_KEY,
        model=constants.OPENROUTER_LLM_MODEL,
        provider_order=["groq"],
        allow_fallbacks=True,
        params=OpenRouterLLMService.InputParams(
            max_completion_tokens=constants.LLM_MAX_OUTPUT_TOKENS,
            temperature=constants.LLM_TEMPERATURE,
        ),
    )

def _create_openai_llm_service():
    """Create OpenAI LLM service."""
    logger.info("LLM_PROVIDER: OpenAI GPT-4o-mini active")
    return OpenAILLMService(
        api_key=constants.OPENAI_API_KEY,
        settings=OpenAILLMService.Settings(
            model="gpt-4o-mini",
            max_completion_tokens=constants.LLM_MAX_OUTPUT_TOKENS,
            temperature=constants.LLM_TEMPERATURE,
        ),
    )

def _create_gemini_llm_service(system_instruction: str):
    """Create Google Vertex AI Gemini LLM service with GCP account switching."""
    import random
    from src.repository.config_repository import config_repo

    # Get GCP mode from cache (no DB call - refreshed periodically by server)
    mode = config_repo.get_gcp_mode()

    # Determine which account to use
    if mode == "random":
        new_percentage = config_repo.get_gcp_new_account_percentage()
        use_new_account = random.randint(1, 100) <= new_percentage
        chosen = "new" if use_new_account else "old"
        logger.info(f"GCP_RANDOM_SELECTION: mode=random new_percentage={new_percentage} chosen={chosen}")
    else:
        use_new_account = (mode == "new")
        chosen = mode

    if use_new_account:
        project_id = constants.GCP_NEW_PROJECT_ID
        credentials_path = constants.GCP_NEW_CREDENTIALS_PATH
        account_tag = "NEW"
        logger.info(f"GCP_ACCOUNT: NEW account active | project={project_id} | credentials={credentials_path}")
    else:
        project_id = constants.GOOGLE_CLOUD_PROJECT
        credentials_path = constants.GCP_CREDENTIALS_PATH
        account_tag = "OLD"
        logger.info(f"GCP_ACCOUNT: OLD account active | project={project_id} | credentials={credentials_path}")

    # Use global endpoint for better capacity (Google recommends this for Standard PayGo)
    # Global endpoint routes to region with most availability, reducing 429 errors
    return GoogleVertexLLMService(
        name=f"GeminiLLM-{account_tag}",  # Shows in error logs which account caused the error
        project_id=project_id,
        location="global",  # Global endpoint for multi-region capacity pool
        model=constants.GEMINI_MODEL,
        credentials_path=credentials_path,
        system_instruction=system_instruction,
        params=GoogleVertexLLMService.InputParams(
            max_output_tokens=constants.LLM_MAX_OUTPUT_TOKENS,
            temperature=constants.LLM_TEMPERATURE,
        ),
    )

def create_llm_service(system_instruction: str):
    """Create LLM service with system instruction. Uses cached DB config to switch providers.

    LLM Provider Modes:
    - "gemini": Use Google Vertex AI Gemini (with GCP account switching)
    - "openai": Use OpenAI GPT-4o-mini
    - "groq": Use Groq (Llama 3.3 70B or configured model)
    - "openrouter": Use OpenRouter (access multiple providers via single API)
    - "random": Weighted traffic split between providers based on per-provider percentages

    Random mode traffic routing (via "weights" config):
      {"openai": 30, "groq": 5, "openrouter": 10} -> 30% OpenAI, 5% Groq, 10% OpenRouter, 55% Gemini
      {"groq": 5}               -> 5% Groq, 95% Gemini
      {"openai": 50}            -> 50% OpenAI, 50% Gemini (legacy behavior)

    GCP Account Modes (only for gemini):
    - "old": Always use mayaresearchgcp (old account)
    - "new": Always use custom-frame-484619-j2 (new account)
    - "random": Randomly choose between old and new based on percentage
    """
    from src.repository.config_repository import config_repo

    # Get LLM provider from cache (defaults to "gemini" if not set)
    llm_provider = config_repo.get_llm_provider()

    # Determine which LLM to use
    if llm_provider == "random":
        chosen_llm = _select_llm_provider_weighted()
    else:
        chosen_llm = llm_provider
    # Create the chosen LLM service
    if chosen_llm == "groq":
        return _create_groq_llm_service()

    if chosen_llm == "openai":
        return _create_openai_llm_service()

    if chosen_llm == "openrouter":
        return _create_openrouter_llm_service()

    # Default: Use Google Vertex AI Gemini
    return _create_gemini_llm_service(system_instruction)

def create_tts_service(speaker: str = None):
    """Create TTS service with optional custom speaker/voice.

    Args:
        speaker: TTS voice to use. If None, uses DEFAULT_TTS_SPEAKER.
    """
    voice = speaker or constants.DEFAULT_TTS_SPEAKER
    logger.info(f"TTS_SERVICE_CREATED: speaker={voice}")
    return CustomTTSService(
        api_url=constants.TTS_API_URL,
        api_key=constants.VEENA3_API_KEY,
        speaker=voice,
        sample_rate=constants.SAMPLE_RATE_16K,
    )

def create_gemini_tts_service(voice: str = None):
    """Create Gemini TTS service for web/Telugu sessions.

    Uses the same GCP account switching logic as Gemini LLM.
    Gemini TTS outputs at 24kHz natively.
    """
    import random
    from src.repository.config_repository import config_repo

    tts_voice = voice or constants.GEMINI_LIVE_VOICE  # "Kore" for Telugu

    mode = config_repo.get_gcp_mode()
    if mode == "random":
        new_percentage = config_repo.get_gcp_new_account_percentage()
        use_new_account = random.randint(1, 100) <= new_percentage
    else:
        use_new_account = (mode == "new")

    if use_new_account:
        credentials_path = constants.GCP_NEW_CREDENTIALS_PATH
        account_tag = "NEW"
    else:
        credentials_path = constants.GCP_CREDENTIALS_PATH
        account_tag = "OLD"

    tts = GoogleTTSService(
        credentials_path=credentials_path,
        location="asia-southeast1",
        settings=GoogleTTSService.Settings(
            voice="te-IN-Chirp3-HD-Erinome",
            language="te-IN",
        ),
        sample_rate=constants.SAMPLE_RATE,
    )
    logger.info(f"TTS_SERVICE_CREATED: provider=chirp3-hd voice=te-IN-Chirp3-HD-Erinome language=te-IN location=asia-southeast1 gcp={account_tag}")
    return tts

def create_context_and_tools(llm, system_instruction: str, include_video: bool = False, include_ppt: bool = False, include_music: bool = False):
    """Create LLM context with tools.

    All characters get image generation. Video, PPT, and music are controlled by persona config.
    """
    standard_tools = [cf_image_function]

    if include_video:
        standard_tools.append(generate_video_function)
        standard_tools.append(generate_video_from_image_function)
        logger.info("PIPELINE_TOOLS: Video generation functions ENABLED (t2v + i2v)")

    if include_ppt:
        standard_tools.append(generate_ppt_function)
        logger.info("PIPELINE_TOOLS: PPT generation function ENABLED")

    if include_music:
        standard_tools.append(generate_music_function)
        logger.info("PIPELINE_TOOLS: Music generation function ENABLED (Lyria)")

    logger.debug(f"TOOLS_REGISTERED: image=true video={include_video} ppt={include_ppt} music={include_music} total={len(standard_tools)}")

    tools = ToolsSchema(standard_tools=standard_tools) if standard_tools else None

    # Build messages with system instruction for OpenAI compatibility
    messages = [
        {"role": "system", "content": system_instruction},
        {"role": "user", "content": "Start the Conversation :"}
    ]

    context = OpenAILLMContext(
        messages=messages,
        tools=tools
    )
    context_aggregator = llm.create_context_aggregator(context)
    return context_aggregator

def create_llm_text_processor():
    """Create LLM text processor that combines 2 sentences before sending to TTS.

    This sits between the LLM and TTS in the pipeline. It aggregates streaming
    LLM tokens into 2-sentence chunks, reducing TTS API calls by ~half.
    """
    return LLMTextProcessor(
        text_aggregator=DoubleSentenceAggregator(sentences_per_chunk=2),
    )

def create_processors():
    """Create RTVI, transcript, and audio buffer processors."""
    rtvi = RTVIProcessor(config=RTVIConfig(config=[]))
    transcript = TranscriptProcessor()

    audio_buffer = AudioBufferProcessor(
        sample_rate=constants.SAMPLE_RATE,
        num_channels=1,
        buffer_size=constants.SAMPLE_RATE * 2 * constants.CHUNK_DURATION,
    )

    return rtvi, transcript, audio_buffer

def create_pipeline(transport, stt, context_aggregator, rtvi, llm, tts, audio_buffer, transcript, llm_text_processor=None):
    """Create the processing pipeline."""
    # Build core pipeline: llm -> [optional llm_text_processor] -> tts
    llm_to_tts = [llm, llm_text_processor, tts] if llm_text_processor else [llm, tts]

    pipeline = Pipeline([
        transport.input(),
        stt,
        transcript.user(),
        context_aggregator.user(),
        rtvi,
        *llm_to_tts,
        transport.output(),
        audio_buffer,
        transcript.assistant(),
        context_aggregator.assistant(),
    ])
    mode = "2-sentence aggregation" if llm_text_processor else "default (1 sentence)"
    logger.info(f"PIPELINE: Traditional STT mode (Audio → STT → Text → LLM → TTS) | text_aggregation={mode}")
    return pipeline

def create_task(pipeline, rtvi, bot_session):
    """Create pipeline task with observers."""
    return PipelineTask(
        pipeline,
        params=PipelineParams(
            enable_metrics=True,
            enable_usage_metrics=True,
            allow_interruptions=True
        ),
        observers=[
            SafeRTVIObserver(rtvi),  # Use SafeRTVIObserver to handle startup race condition
            InfoMetricsObserver(bot_session),
            UserBotLatencyLogObserver()
        ],
    )

# ---- Gemini Live pipeline (Telugu / multi-language) ----

def create_gemini_live_service(system_instruction: str, voice_id: str = None, include_video: bool = False, include_ppt: bool = False, include_music: bool = False, language: str = None):
    """Create Gemini Live Vertex AI service — handles STT + LLM + TTS in one service.

    Uses the same GCP account switching logic as the regular Gemini LLM service.
    """
    import random
    from src.repository.config_repository import config_repo

    voice = voice_id or constants.GEMINI_LIVE_VOICE

    standard_tools = [cf_image_function]
    if include_video:
        standard_tools.append(generate_video_function)
        standard_tools.append(generate_video_from_image_function)
        logger.info("GEMINI_LIVE_TOOLS: Video generation functions ENABLED (t2v + i2v)")

    if include_ppt:
        standard_tools.append(generate_ppt_function)
        logger.info("GEMINI_LIVE_TOOLS: PPT generation function ENABLED")

    if include_music:
        standard_tools.append(generate_music_function)
        logger.info("GEMINI_LIVE_TOOLS: Music generation function ENABLED (Lyria)")

    tools = ToolsSchema(standard_tools=standard_tools) if standard_tools else None

    # GCP account switching (same logic as _create_gemini_llm_service)
    mode = config_repo.get_gcp_mode()
    if mode == "random":
        new_percentage = config_repo.get_gcp_new_account_percentage()
        use_new_account = random.randint(1, 100) <= new_percentage
    else:
        use_new_account = (mode == "new")

    if use_new_account:
        project_id = constants.GCP_NEW_PROJECT_ID
        credentials_path = constants.GCP_NEW_CREDENTIALS_PATH
        account_tag = "NEW"
    else:
        project_id = constants.GOOGLE_CLOUD_PROJECT
        credentials_path = constants.GCP_CREDENTIALS_PATH
        account_tag = "OLD"

    # Map language code to Language enum for Gemini Live
    from pipecat.transcriptions.language import Language
    _LANG_MAP = {
        "te": Language.TE_IN,
        "hi": Language.HI_IN,
        "en": Language.EN_US,
        "es": Language.ES_ES,
        "fr": Language.FR_FR,
        "pt": Language.PT_BR,
        "id": Language.ID_ID,
        "de": Language.DE_DE,
        "ja": Language.JA_JP,
        "ar": Language.AR_SA,
        "ko": Language.KO_KR,
        "vi": Language.VI_VN,
    }
    gemini_language = _LANG_MAP.get(language, Language.EN_US)

    llm = GeminiLiveVertexLLMService(
        credentials_path=credentials_path,
        project_id=project_id,
        location="us-east4",
        system_instruction=system_instruction,
        tools=tools,
        settings=GeminiLiveVertexLLMService.Settings(
            voice=voice,
            language=gemini_language,
        ),
    )
    logger.info(f"GEMINI_LIVE_CREATED: voice={voice} language={gemini_language} tools={len(standard_tools)} gcp_account={account_tag} project={project_id}")
    return llm

def create_gemini_live_context():
    """Create context aggregator for Gemini Live.

    Uses LLMContext + LLMContextAggregatorPair directly (not deprecated OpenAILLMContext).
    System instruction is set on the LLM service via Settings, not in context messages.
    VAD is placed inside LLMUserAggregatorParams per official Gemini Live pattern.
    """
    from pipecat.processors.aggregators.llm_context import LLMContext
    from pipecat.processors.aggregators.llm_response_universal import (
        LLMContextAggregatorPair,
        LLMUserAggregatorParams,
    )

    messages = []

    context = LLMContext(messages)
    context_aggregator = LLMContextAggregatorPair(
        context,
        user_params=LLMUserAggregatorParams(
            vad_analyzer=SileroVADAnalyzer(params=VADParams(stop_secs=0.5)),
        ),
    )
    return context_aggregator

def create_gemini_live_pipeline(transport, gemini_live, context_aggregator, rtvi, audio_buffer, transcript):
    """Create pipeline for Gemini Live — audio goes directly to Gemini, no separate STT/TTS.

    Keeps RTVI (frontend events), audio_buffer (recording), transcript (conversation tracking).
    """
    pipeline = Pipeline([
        transport.input(),
        context_aggregator.user(),
        gemini_live,
        rtvi,
        transport.output(),
        audio_buffer,
        transcript.assistant(),
        context_aggregator.assistant(),
    ])
    logger.info("PIPELINE: Gemini Live mode (Audio → Context → Gemini Live → Audio)")
    return pipeline


Now let me check how the pipeline is selected in main.py — which pipeline Hindi users actually get.


1	import asyncio
2	import threading
3	import time
4	from dotenv import load_dotenv
5	from loguru import logger
6
7	from pipecat.pipeline.runner import PipelineRunner
8
9	from src.config import constants
10	from src.bot.core.session_state import SessionState
11
12	from src.bot.core.config import load_system_instruction, cleanup_session, BotSessionState, ChunkState, TranscriptManager
13	from src.bot.core.pipeline import create_vad_analyzer, create_transport, create_daily_transport, create_stt_service, create_llm_service, create_tts_service, create_context_and_tools, create_processors, create_llm_text_processor, create_pipeline, create_task, create_gemini_live_service, create_gemini_live_context, create_gemini_live_pipeline
14	from src.bot.core.session_handlers import create_audio_data_handler, create_transcript_handler, create_client_message_handler, create_cleanup_if_no_user, create_idle_timeout_handler, create_app_review_trigger, create_first_client_connected_handler, create_client_disconnected_handler, create_client_ready_handler, create_daily_participant_joined_handler, create_daily_first_participant_joined_handler, create_daily_participant_left_handler
15	from src.bot.services.image_service import create_cf_image_handler
16	from src.bot.services.video_service import create_video_handler, create_video_from_image_handler
17	from src.bot.services.ppt_service import create_ppt_handler
18	from src.bot.services.music_service import create_music_handler
19
20	load_dotenv()
21
22
23	async def run_bot_webrtc_global_agent(webrtc_connection, device_id: str = None, session_id: str = None, version: str = None, character_id: str = None, app_key: str = None, language: str = None, source: str = None, persona_id: str = None, transport_type: str = "smallwebrtc", room_url: str = None, token: str = None, app_version: int = None):
24	    bot_start_time = time.time()
25	    device_id = device_id or "common"
26
27	    # Accept character_id as persona_id for backward compat
28	    if not persona_id and character_id:
29	        persona_id = character_id
30
31	    session_state = SessionState()
32	    await session_state.start_queue_processor()
33
34	    # Initialize bot session
35	    bot_session = BotSessionState(session_id, device_id, bot_start_time)
36	    bot_session.persona_id = persona_id
37	    bot_session.app_key = app_key
38	    bot_session.language = language
39	    bot_session.source = source
40	    bot_session.app_version = app_version
41
42	    # Load persona info
43	    persona_prompt = None
44	    persona_voice = None
45	    persona_tools = []
46	    if persona_id:
47	        from src.repository.persona_repository import persona_repo
48	        persona = await persona_repo.get_persona_by_id(persona_id)
49	        if persona:
50	            bot_session.persona_name = persona.get("name")
51	            bot_session.content_type = persona.get("content_type")
52	            persona_prompt = persona.get("system_prompt")
53	            persona_voice = persona.get("tts_voice")
54	            persona_tools = persona.get("tools", [])
55	            logger.info(f"SESSION_START: persona={persona_id} name={persona.get('name')} content_type={bot_session.content_type} tools={persona_tools} transport={transport_type}")
56	        else:
57	            logger.warning(f"PERSONA_NOT_FOUND: persona={persona_id}")
58	    else:
59	        logger.info(f"SESSION_START: no_persona_id device_id={device_id} transport={transport_type}")
60
61	    # Fetch device info early — use DB language instead of frontend-sent value
62	    if device_id != "common":
63	        from src.repository.device_repository import device_repo
64	        device_info = await device_repo.get_device(device_id)
65	        if device_info and device_info.get("language"):
66	            language = device_info["language"]
67	            bot_session.language = language
68	            logger.info(f"LANGUAGE_FROM_DB: device={device_id} language={language}")
69	    else:
70	        device_info = None
71
72	    # Load system instruction
73	    system_instruction = await load_system_instruction(
74	        language=language,
75	        persona_prompt=persona_prompt,
76	        persona_id=persona_id,
77	    )
78
79	    # Load usage state (persona-level free limits, content-level plan/topup)
80	    if persona_id and bot_session.content_type and device_id != "common":
81	        from src.service.usage_service import get_usage_state
82	        from src.repository.usage_repository import usage_repo
83	        from src.repository.plan_repository import plan_repo
84	        import hashlib
85
86	        device_region = (device_info.get("region") or "india") if device_info else "india"
87	        payment_provider = "google_play" if device_region == "global" else "razorpay"
88
89	        usage = await get_usage_state(device_id, bot_session.content_type, payment_provider)
90	        bot_session.plan_remaining = usage["plan_remaining"]
91	        bot_session.topup_remaining = usage["topup_remaining"]
92	        bot_session.subscription_id = usage["subscription_id"]
93	        bot_session.plan_content_type = usage.get("plan_content_type")
94
95	        if device_region == "india":
96	            # India: always use persona-level free limits (no change from old behavior)
97	            free_count = persona.get("free_count", 0) if persona else 0
98	            free_window = persona.get("free_window_hours", 24) if persona else 24
99	        else:
100	            # Global: use region_free_limits with A/B percentage
101	            # Group A (hash < ab_percentage): fully free, no limits
102	            # Group B (hash >= ab_percentage): gets free_count free, then paywall
103	            region_limits = await plan_repo.get_region_free_limits(device_region)
104	            region_limit = region_limits.get(bot_session.content_type)
105	            if region_limit:
106	                ab_pct = region_limit.get("ab_percentage", 100)
107	                device_h = int(hashlib.sha256(device_id.encode()).hexdigest(), 16) % 100
108	                if device_h < ab_pct:
109	                    # Group A: fully free — set unlimited
110	                    free_count = 999999
111	                    free_window = 24
112	                else:
113	                    # Group B: limited free
114	                    free_count = region_limit["free_count"]
115	                    free_window = region_limit["free_window_hours"]
116	            else:
117	                free_count = persona.get("free_count", 0) if persona else 0
118	                free_window = persona.get("free_window_hours", 24) if persona else 24
119
120	        if free_count > 0:
121	            if free_count == 999999:
122	                bot_session.free_remaining = 999999  # unlimited
123	            else:
124	                used_free = await usage_repo.count_persona_generations(device_id, persona_id, bot_session.content_type, free_window)
125	                bot_session.free_remaining = max(0, free_count - used_free)
126	        else:
127	            bot_session.free_remaining = 0
128
129	        logger.info(f"USAGE_STATE: session_id={session_id} device_id={device_id} persona={persona_id} content={bot_session.content_type} free={bot_session.free_remaining}/{free_count} plan={bot_session.plan_remaining} topup={bot_session.topup_remaining} transport={transport_type}")
130
131	    # Determine which tools to enable
132	    video_enabled = "generate_video" in persona_tools or "generate_video_from_image" in persona_tools
133
134	    ppt_enabled = "generate_ppt" in persona_tools
135	    music_enabled = "generate_music" in persona_tools
136	    bot_session.video_enabled = video_enabled
137
138	    use_gemini_live = True
139	    bot_session.use_gemini_live = use_gemini_live
140
141	    vad_analyzer = create_vad_analyzer(session_id)
142	    bot_session.vad_analyzer = vad_analyzer
143
144	    # Create services and store references IMMEDIATELY on bot_session after each creation.
145	    # This ensures cleanup_session can release them even if a later create_* call throws.
146	    if transport_type == "daily":
147	        transport = create_daily_transport(room_url, token, vad_analyzer)
148	        logger.info(f"TRANSPORT_CREATED: type=daily session_id={session_id}")
149	    else:
150	        transport = create_transport(webrtc_connection, vad_analyzer)
151	        logger.info(f"TRANSPORT_CREATED: type=smallwebrtc session_id={session_id}")
152	    bot_session.transport = transport
153	    bot_session.transport_type = transport_type
154
155	    rtvi, transcript, audio_buffer = create_processors()
156	    transcript_manager = TranscriptManager()
157	    chunk_state = ChunkState()
158
159	    if use_gemini_live:
160	        # Gemini Live: single service handles STT + LLM + TTS
161	        llm = create_gemini_live_service(system_instruction, include_video=video_enabled, include_ppt=ppt_enabled, include_music=music_enabled, language=language)
162	        bot_session.llm_service = llm
163
164	        context_aggregator = create_gemini_live_context()
165	        bot_session.context_aggregator = context_aggregator
166
167	        logger.info(f"GEMINI_LIVE_SESSION: session_id={session_id} language={language} persona={persona_id} transport={transport_type}")
168
169	        pipeline = create_gemini_live_pipeline(transport, llm, context_aggregator, rtvi, audio_buffer, transcript)
170	    else:
171	        # Traditional pipeline: separate STT → LLM → TTS (mobile app only, web always uses Gemini Live)
172	        stt = create_stt_service()
173	        bot_session.stt_service = stt
174
175	        llm = create_llm_service(system_instruction)
176	        bot_session.llm_service = llm
177
178	        tts = create_tts_service(speaker=persona_voice)
179	        session_state.tts_service = tts
180
181	        context_aggregator = create_context_and_tools(
182	            llm,
183	            system_instruction,
184	            include_video=video_enabled,
185	            include_ppt=ppt_enabled,
186	            include_music=music_enabled,
187	        )
188	        bot_session.context_aggregator = context_aggregator
189
190	        llm_text_processor = create_llm_text_processor()
191	        pipeline = create_pipeline(transport, stt, context_aggregator, rtvi, llm, tts, audio_buffer, transcript, llm_text_processor)
192
193	    bot_session.pipeline = pipeline
194	    task = create_task(pipeline, rtvi, bot_session)
195
196	    audio_buffer.event_handler("on_audio_data")(create_audio_data_handler(bot_session, chunk_state))
197
198	    trigger_app_review_if_eligible = create_app_review_trigger(bot_session, rtvi)
199
200	    # Register tools based on persona config or legacy defaults
201	    # timeout_secs=120: image/video generation can take 20-60s; default 10s causes premature timeout
202	    # cancel_on_interruption=False: user may speak while waiting; don't cancel generation
203	    _tool_handlers = {
204	        "cf_image": lambda: create_cf_image_handler(
205	            session_state, rtvi, bot_session, transcript_manager, task, llm, trigger_app_review_if_eligible
206	        ),
207	        "generate_video": lambda: create_video_handler(
208	            session_state, rtvi, bot_session, transcript_manager, task, llm
209	        ),
210	        "generate_video_from_image": lambda: create_video_from_image_handler(
211	            session_state, rtvi, bot_session, transcript_manager, task, llm
212	        ),
213	        "generate_ppt": lambda: create_ppt_handler(
214	            session_state, rtvi, bot_session, transcript_manager, task, llm
215	        ),
216	        "generate_music": lambda: create_music_handler(
217	            session_state, rtvi, bot_session, transcript_manager, task, llm
218	        ),
219	    }
220
221	    # Register tools from persona config (or default cf_image if no persona)
222	    tools_to_register = persona_tools if persona_tools else ["cf_image"]
223	    if video_enabled and not persona_tools:
224	        tools_to_register.extend(["generate_video", "generate_video_from_image"])
225
226	    for tool_name in tools_to_register:
227	        if tool_name in _tool_handlers:
228	            try:
229	                llm.register_function(tool_name, _tool_handlers[tool_name](), timeout_secs=120, cancel_on_interruption=False)
230	                logger.info(f"FUNCTION_REGISTERED: {tool_name}")
231	            except Exception as e:
232	                logger.error(f"Failed to register {tool_name} handler: {e}", exc_info=True)
233
234	    transcript.event_handler("on_transcript_update")(create_transcript_handler(bot_session, transcript_manager, rtvi))
235	    rtvi.event_handler("on_client_ready")(create_client_ready_handler())
236	    rtvi.event_handler("on_client_message")(create_client_message_handler(bot_session, rtvi, task, transcript_manager))
237
238	    cleanup_if_no_user = create_cleanup_if_no_user(bot_session, session_state, task)
239
240	    if transport_type == "daily":
241	        # Daily events: on_participant_joined, on_first_participant_joined, on_participant_left
242	        transport.event_handler("on_participant_joined")(create_daily_participant_joined_handler(bot_session, task))
243	        transport.event_handler("on_first_participant_joined")(create_daily_first_participant_joined_handler(bot_session, transport, rtvi, task, audio_buffer, chunk_state))
244	        transport.event_handler("on_participant_left")(create_daily_participant_left_handler(bot_session, session_state, transcript_manager, chunk_state, task))
245	    else:
246	        # SmallWebRTC events: on_client_connected, on_client_disconnected
247	        # NOTE: pipecat only supports one handler per event — the merged handler
248	        # includes both client_connected and first_client_connected logic.
249	        transport.event_handler("on_client_connected")(create_first_client_connected_handler(bot_session, rtvi, task, audio_buffer, chunk_state))
250	        transport.event_handler("on_client_disconnected")(create_client_disconnected_handler(bot_session, session_state, transcript_manager, chunk_state, task))
251	    task.event_handler("on_idle_timeout")(create_idle_timeout_handler(bot_session))
252
253	    runner = PipelineRunner(handle_sigint=False)
254
255	    bot_session.no_user_timeout_task = asyncio.create_task(cleanup_if_no_user())
256	    logger.info(f"NO_USER_TIMEOUT_STARTED: session_id={session_id} transport={transport_type} timeout={constants.NO_USER_TIMEOUT}s")
257
258	    try:
259	        await runner.run(task)
260	    except Exception as e:
261	        logger.error(f"PIPELINE_ERROR: session_id={session_id} transport={transport_type} error={e}", exc_info=True)
262	        await task.cancel()
263	    finally:
264	        threads_before = threading.active_count()
265	        logger.info(f"CLEANUP_TRIGGER: session_id={session_id} transport={transport_type} reason=runner_finally threads={threads_before}")
266	        logger.info(f"RUNNER_CLEANUP_START: session_id={session_id} transport={transport_type} threads={threads_before}")
267
268	        bot_session.cancel_timeout_task()
269	        if bot_session.no_user_timeout_task:
270	            try:
271	                await bot_session.no_user_timeout_task
272	            except asyncio.CancelledError:
273	                pass
274
275	        # Timeout cleanup to prevent stuck native threads from blocking forever.
276	        # If pipeline.cleanup() or transport.cleanup() hangs on C++ thread joins,
277	        # we force-release references after 30s so the task always completes.
278	        try:
279	            await asyncio.wait_for(
280	                cleanup_session(session_state, bot_session, session_id, transcript_manager, chunk_state),
281	                timeout=30.0
282	            )
283	            threads_after = threading.active_count()
284	            logger.info(f"CLEANUP_THREADS: session_id={session_id} transport={transport_type} threads_before={threads_before} threads_after={threads_after} delta={threads_after - threads_before}")
285	        except asyncio.TimeoutError:
286	            threads_after = threading.active_count()
287	            logger.error(
288	                f"CLEANUP_TIMEOUT: session_id={session_id} transport={transport_type} cleanup hung for 30s, forcing release threads_before={threads_before} threads_after={threads_after}"
289	            )
290	            # Force-null everything to drop references even if native cleanup hung
291	            bot_session.pipeline = None
292	            bot_session.transport = None
293	            bot_session.vad_analyzer = None
294	            bot_session.stt_service = None
295	            bot_session.llm_service = None
296	            bot_session.audio_buffer = None
297	            bot_session.context_aggregator = None
298
299	        logger.info(f"RUNNER_CLEANUP_COMPLETE: session_id={session_id} transport={transport_type} threads={threading.active_count()}")
300

import asyncio
import threading
import time
from dotenv import load_dotenv
from loguru import logger

from pipecat.pipeline.runner import PipelineRunner

from src.config import constants
from src.bot.core.session_state import SessionState

from src.bot.core.config import load_system_instruction, cleanup_session, BotSessionState, ChunkState, TranscriptManager
from src.bot.core.pipeline import create_vad_analyzer, create_transport, create_daily_transport, create_stt_service, create_llm_service, create_tts_service, create_context_and_tools, create_processors, create_llm_text_processor, create_pipeline, create_task, create_gemini_live_service, create_gemini_live_context, create_gemini_live_pipeline
from src.bot.core.session_handlers import create_audio_data_handler, create_transcript_handler, create_client_message_handler, create_cleanup_if_no_user, create_idle_timeout_handler, create_app_review_trigger, create_first_client_connected_handler, create_client_disconnected_handler, create_client_ready_handler, create_daily_participant_joined_handler, create_daily_first_participant_joined_handler, create_daily_participant_left_handler
from src.bot.services.image_service import create_cf_image_handler
from src.bot.services.video_service import create_video_handler, create_video_from_image_handler
from src.bot.services.ppt_service import create_ppt_handler
from src.bot.services.music_service import create_music_handler

load_dotenv()

async def run_bot_webrtc_global_agent(webrtc_connection, device_id: str = None, session_id: str = None, version: str = None, character_id: str = None, app_key: str = None, language: str = None, source: str = None, persona_id: str = None, transport_type: str = "smallwebrtc", room_url: str = None, token: str = None, app_version: int = None):
    bot_start_time = time.time()
    device_id = device_id or "common"

    # Accept character_id as persona_id for backward compat
    if not persona_id and character_id:
        persona_id = character_id

    session_state = SessionState()
    await session_state.start_queue_processor()

    # Initialize bot session
    bot_session = BotSessionState(session_id, device_id, bot_start_time)
    bot_session.persona_id = persona_id
    bot_session.app_key = app_key
    bot_session.language = language
    bot_session.source = source
    bot_session.app_version = app_version

    # Load persona info
    persona_prompt = None
    persona_voice = None
    persona_tools = []
    if persona_id:
        from src.repository.persona_repository import persona_repo
        persona = await persona_repo.get_persona_by_id(persona_id)
        if persona:
            bot_session.persona_name = persona.get("name")
            bot_session.content_type = persona.get("content_type")
            persona_prompt = persona.get("system_prompt")
            persona_voice = persona.get("tts_voice")
            persona_tools = persona.get("tools", [])
            logger.info(f"SESSION_START: persona={persona_id} name={persona.get('name')} content_type={bot_session.content_type} tools={persona_tools} transport={transport_type}")
        else:
            logger.warning(f"PERSONA_NOT_FOUND: persona={persona_id}")
    else:
        logger.info(f"SESSION_START: no_persona_id device_id={device_id} transport={transport_type}")

    # Fetch device info early — use DB language instead of frontend-sent value
    if device_id != "common":
        from src.repository.device_repository import device_repo
        device_info = await device_repo.get_device(device_id)
        if device_info and device_info.get("language"):
            language = device_info["language"]
            bot_session.language = language
            logger.info(f"LANGUAGE_FROM_DB: device={device_id} language={language}")
    else:
        device_info = None

    # Load system instruction
    system_instruction = await load_system_instruction(
        language=language,
        persona_prompt=persona_prompt,
        persona_id=persona_id,
    )

    # Load usage state (persona-level free limits, content-level plan/topup)
    if persona_id and bot_session.content_type and device_id != "common":
        from src.service.usage_service import get_usage_state
        from src.repository.usage_repository import usage_repo
        from src.repository.plan_repository import plan_repo
        import hashlib

        device_region = (device_info.get("region") or "india") if device_info else "india"
        payment_provider = "google_play" if device_region == "global" else "razorpay"

        usage = await get_usage_state(device_id, bot_session.content_type, payment_provider)
        bot_session.plan_remaining = usage["plan_remaining"]
        bot_session.topup_remaining = usage["topup_remaining"]
        bot_session.subscription_id = usage["subscription_id"]
        bot_session.plan_content_type = usage.get("plan_content_type")

        if device_region == "india":
            # India: always use persona-level free limits (no change from old behavior)
            free_count = persona.get("free_count", 0) if persona else 0
            free_window = persona.get("free_window_hours", 24) if persona else 24
        else:
            # Global: use region_free_limits with A/B percentage
            # Group A (hash < ab_percentage): fully free, no limits
            # Group B (hash >= ab_percentage): gets free_count free, then paywall
            region_limits = await plan_repo.get_region_free_limits(device_region)
            region_limit = region_limits.get(bot_session.content_type)
            if region_limit:
                ab_pct = region_limit.get("ab_percentage", 100)
                device_h = int(hashlib.sha256(device_id.encode()).hexdigest(), 16) % 100
                if device_h < ab_pct:
                    # Group A: fully free — set unlimited
                    free_count = 999999
                    free_window = 24
                else:
                    # Group B: limited free
                    free_count = region_limit["free_count"]
                    free_window = region_limit["free_window_hours"]
            else:
                free_count = persona.get("free_count", 0) if persona else 0
                free_window = persona.get("free_window_hours", 24) if persona else 24

        if free_count > 0:
            if free_count == 999999:
                bot_session.free_remaining = 999999  # unlimited
            else:
                used_free = await usage_repo.count_persona_generations(device_id, persona_id, bot_session.content_type, free_window)
                bot_session.free_remaining = max(0, free_count - used_free)
        else:
            bot_session.free_remaining = 0

        logger.info(f"USAGE_STATE: session_id={session_id} device_id={device_id} persona={persona_id} content={bot_session.content_type} free={bot_session.free_remaining}/{free_count} plan={bot_session.plan_remaining} topup={bot_session.topup_remaining} transport={transport_type}")

    # Determine which tools to enable
    video_enabled = "generate_video" in persona_tools or "generate_video_from_image" in persona_tools

    ppt_enabled = "generate_ppt" in persona_tools
    music_enabled = "generate_music" in persona_tools
    bot_session.video_enabled = video_enabled

    use_gemini_live = True
    bot_session.use_gemini_live = use_gemini_live

    vad_analyzer = create_vad_analyzer(session_id)
    bot_session.vad_analyzer = vad_analyzer

    # Create services and store references IMMEDIATELY on bot_session after each creation.
    # This ensures cleanup_session can release them even if a later create_* call throws.
    if transport_type == "daily":
        transport = create_daily_transport(room_url, token, vad_analyzer)
        logger.info(f"TRANSPORT_CREATED: type=daily session_id={session_id}")
    else:
        transport = create_transport(webrtc_connection, vad_analyzer)
        logger.info(f"TRANSPORT_CREATED: type=smallwebrtc session_id={session_id}")
    bot_session.transport = transport
    bot_session.transport_type = transport_type

    rtvi, transcript, audio_buffer = create_processors()
    transcript_manager = TranscriptManager()
    chunk_state = ChunkState()

    if use_gemini_live:
        # Gemini Live: single service handles STT + LLM + TTS
        llm = create_gemini_live_service(system_instruction, include_video=video_enabled, include_ppt=ppt_enabled, include_music=music_enabled, language=language)
        bot_session.llm_service = llm

        context_aggregator = create_gemini_live_context()
        bot_session.context_aggregator = context_aggregator

        logger.info(f"GEMINI_LIVE_SESSION: session_id={session_id} language={language} persona={persona_id} transport={transport_type}")

        pipeline = create_gemini_live_pipeline(transport, llm, context_aggregator, rtvi, audio_buffer, transcript)
    else:
        # Traditional pipeline: separate STT → LLM → TTS (mobile app only, web always uses Gemini Live)
        stt = create_stt_service()
        bot_session.stt_service = stt

        llm = create_llm_service(system_instruction)
        bot_session.llm_service = llm

        tts = create_tts_service(speaker=persona_voice)
        session_state.tts_service = tts

        context_aggregator = create_context_and_tools(
            llm,
            system_instruction,
            include_video=video_enabled,
            include_ppt=ppt_enabled,
            include_music=music_enabled,
        )
        bot_session.context_aggregator = context_aggregator

        llm_text_processor = create_llm_text_processor()
        pipeline = create_pipeline(transport, stt, context_aggregator, rtvi, llm, tts, audio_buffer, transcript, llm_text_processor)

    bot_session.pipeline = pipeline
    task = create_task(pipeline, rtvi, bot_session)

    audio_buffer.event_handler("on_audio_data")(create_audio_data_handler(bot_session, chunk_state))

    trigger_app_review_if_eligible = create_app_review_trigger(bot_session, rtvi)

    # Register tools based on persona config or legacy defaults
    # timeout_secs=120: image/video generation can take 20-60s; default 10s causes premature timeout
    # cancel_on_interruption=False: user may speak while waiting; don't cancel generation
    _tool_handlers = {
        "cf_image": lambda: create_cf_image_handler(
            session_state, rtvi, bot_session, transcript_manager, task, llm, trigger_app_review_if_eligible
        ),
        "generate_video": lambda: create_video_handler(
            session_state, rtvi, bot_session, transcript_manager, task, llm
        ),
        "generate_video_from_image": lambda: create_video_from_image_handler(
            session_state, rtvi, bot_session, transcript_manager, task, llm
        ),
        "generate_ppt": lambda: create_ppt_handler(
            session_state, rtvi, bot_session, transcript_manager, task, llm
        ),
        "generate_music": lambda: create_music_handler(
            session_state, rtvi, bot_session, transcript_manager, task, llm
        ),
    }

    # Register tools from persona config (or default cf_image if no persona)
    tools_to_register = persona_tools if persona_tools else ["cf_image"]
    if video_enabled and not persona_tools:
        tools_to_register.extend(["generate_video", "generate_video_from_image"])

    for tool_name in tools_to_register:
        if tool_name in _tool_handlers:
            try:
                llm.register_function(tool_name, _tool_handlers[tool_name](), timeout_secs=120, cancel_on_interruption=False)
                logger.info(f"FUNCTION_REGISTERED: {tool_name}")
            except Exception as e:
                logger.error(f"Failed to register {tool_name} handler: {e}", exc_info=True)

    transcript.event_handler("on_transcript_update")(create_transcript_handler(bot_session, transcript_manager, rtvi))
    rtvi.event_handler("on_client_ready")(create_client_ready_handler())
    rtvi.event_handler("on_client_message")(create_client_message_handler(bot_session, rtvi, task, transcript_manager))

    cleanup_if_no_user = create_cleanup_if_no_user(bot_session, session_state, task)

    if transport_type == "daily":
        # Daily events: on_participant_joined, on_first_participant_joined, on_participant_left
        transport.event_handler("on_participant_joined")(create_daily_participant_joined_handler(bot_session, task))
        transport.event_handler("on_first_participant_joined")(create_daily_first_participant_joined_handler(bot_session, transport, rtvi, task, audio_buffer, chunk_state))
        transport.event_handler("on_participant_left")(create_daily_participant_left_handler(bot_session, session_state, transcript_manager, chunk_state, task))
    else:
        # SmallWebRTC events: on_client_connected, on_client_disconnected
        # NOTE: pipecat only supports one handler per event — the merged handler
        # includes both client_connected and first_client_connected logic.
        transport.event_handler("on_client_connected")(create_first_client_connected_handler(bot_session, rtvi, task, audio_buffer, chunk_state))
        transport.event_handler("on_client_disconnected")(create_client_disconnected_handler(bot_session, session_state, transcript_manager, chunk_state, task))
    task.event_handler("on_idle_timeout")(create_idle_timeout_handler(bot_session))

    runner = PipelineRunner(handle_sigint=False)

    bot_session.no_user_timeout_task = asyncio.create_task(cleanup_if_no_user())
    logger.info(f"NO_USER_TIMEOUT_STARTED: session_id={session_id} transport={transport_type} timeout={constants.NO_USER_TIMEOUT}s")

    try:
        await runner.run(task)
    except Exception as e:
        logger.error(f"PIPELINE_ERROR: session_id={session_id} transport={transport_type} error={e}", exc_info=True)
        await task.cancel()
    finally:
        threads_before = threading.active_count()
        logger.info(f"CLEANUP_TRIGGER: session_id={session_id} transport={transport_type} reason=runner_finally threads={threads_before}")
        logger.info(f"RUNNER_CLEANUP_START: session_id={session_id} transport={transport_type} threads={threads_before}")

        bot_session.cancel_timeout_task()
        if bot_session.no_user_timeout_task:
            try:
                await bot_session.no_user_timeout_task
            except asyncio.CancelledError:
                pass

        # Timeout cleanup to prevent stuck native threads from blocking forever.
        # If pipeline.cleanup() or transport.cleanup() hangs on C++ thread joins,
        # we force-release references after 30s so the task always completes.
        try:
            await asyncio.wait_for(
                cleanup_session(session_state, bot_session, session_id, transcript_manager, chunk_state),
                timeout=30.0
            )
            threads_after = threading.active_count()
            logger.info(f"CLEANUP_THREADS: session_id={session_id} transport={transport_type} threads_before={threads_before} threads_after={threads_after} delta={threads_after - threads_before}")
        except asyncio.TimeoutError:
            threads_after = threading.active_count()
            logger.error(
                f"CLEANUP_TIMEOUT: session_id={session_id} transport={transport_type} cleanup hung for 30s, forcing release threads_before={threads_before} threads_after={threads_after}"
            )
            # Force-null everything to drop references even if native cleanup hung
            bot_session.pipeline = None
            bot_session.transport = None
            bot_session.vad_analyzer = None
            bot_session.stt_service = None
            bot_session.llm_service = None
            bot_session.audio_buffer = None
            bot_session.context_aggregator = None

        logger.info(f"RUNNER_CLEANUP_COMPLETE: session_id={session_id} transport={transport_type} threads={threading.active_count()}")


You're right, my bad. Look at line 138:

```python
use_gemini_live = True
```

It's **hardcoded to `True`** — so **ALL users (including Hindi) are going through Gemini Live pipeline**, not the traditional OpenAI STT + OpenAI LLM + Veena3 TTS pipeline.

The traditional pipeline (lines 170-191) with OpenAI STT → OpenAI/Gemini LLM → Veena3 TTS is **dead code** right now — it never executes because `use_gemini_live` is always `True`.

So currently for Hindi users:
- **STT**: Gemini Live (built-in)
- **LLM**: Gemini Live (built-in)
- **TTS**: Gemini Live (built-in, NOT Veena3)

The Veena3 TTS (`CustomTTSService`) and OpenAI STT are not being used at all. Want to switch Hindi users back to the traditional pipeline?


change that y the fuck .. it should go normally


change that y the fuck .. it should go normally ..for hindi users it should go normally


Edit

    use_gemini_live = True
    bot_session.use_gemini_live = use_gemini_live

    use_gemini_live = language not in (None, "hi")
    bot_session.use_gemini_live = use_gemini_live
    logger.info(f"PIPELINE_MODE: language={language} use_gemini_live={use_gemini_live}")