Compare the v3 and v4 bot main/runner implementations. Look for:
1. Check /home/ubuntu/ for any v3 project directories (maya-pipecat-v3, pipecat-v3, etc.)
2. Check git branches with `git branch -a` in /home/ubuntu/pipecat
3. Look for the v3 main bot file (equivalent of src/bot/core/main.py) and compare with the v4 version
4. Specifically look for differences in how Gemini Live sessions are set up, how function calls are handled, and how the pipeline is constructed

The goal is to find what changed between v3 and v4 that could cause the agent to stop speaking.


I'll investigate the v3 and v4 implementations systematically.


Bash

ls -d /home/ubuntu/maya-pipecat-v3 /home/ubuntu/pipecat-v3 /home/ubuntu/*v3* /home/ubuntu/*pipecat* 2>/dev/null

Check for v3 project directories


git branch -a 2>/dev/null || echo "Not a git repo"

Check git branches


List pipecat directory


Error: Exit code 2
/home/ubuntu/pipecat
/home/ubuntu/rtf_v3_log.txt


total 100
drwxrwxr-x 13 ubuntu ubuntu  4096 Mar 28 01:19 .
drwxr-x--x 81 ubuntu ubuntu 12288 Mar 28 01:29 ..
drwxrwxr-x  4 ubuntu ubuntu  4096 Mar 27 20:28 cf-stt-worker
drwxrwxr-x  2 ubuntu ubuntu  4096 Feb 12 12:36 .claude
drwxrwxr-x  2 ubuntu ubuntu  4096 Mar 21 21:51 .cursorrules
drwxrwxr-x  6 ubuntu ubuntu  4096 Feb 19 07:16 deployments
-rw-rw-r--  1 ubuntu ubuntu   402 Feb 27 07:05 .dockerignore
drwxrwxr-x  2 ubuntu ubuntu  4096 Mar 27 10:20 docs
-rw-rw-r--  1 ubuntu ubuntu  2968 Mar 26 22:28 .env
drwxrwxr-x  4 ubuntu ubuntu  4096 Mar 21 21:51 fnHandlers
-rw-rw-r--  1 ubuntu ubuntu  2369 Feb 19 16:35 gcp-new-service-account.json
-rw-rw-r--  1 ubuntu ubuntu  2354 Feb 27 06:56 gcp-service-account.json
drwxrwxr-x  7 ubuntu ubuntu  4096 Mar 28 01:20 .git
-rw-rw-r--  1 ubuntu ubuntu   558 Mar 21 21:51 .gitignore
drwxrwxr-x  2 ubuntu ubuntu  4096 Mar 21 22:21 logs
-rw-rw-r--  1 ubuntu ubuntu     6 Mar 28 01:19 pipecat-server.pid
drwxrwxr-x  2 ubuntu ubuntu  4096 Mar 27 20:28 prompts
-rw-rw-r--  1 ubuntu ubuntu  5490 Mar 27 23:07 README.md
-rw-rw-r--  1 ubuntu ubuntu  1148 Mar 27 20:28 requirements.txt
drwxrwxr-x  2 ubuntu ubuntu  4096 Mar 27 21:59 scripts
-rw-rw-r--  1 ubuntu ubuntu  3092 Mar 27 20:28 server.py
drwxrwxr-x 10 ubuntu ubuntu  4096 Mar 21 22:21 src


  main
  pipecat_modal_new_characters
  v3_version
* v4_version
  remotes/origin/HEAD -> origin/main
  remotes/origin/app_pipcat
  remotes/origin/app_pipecat_bup
  remotes/origin/app_pipecat_wo_mem
  remotes/origin/main
  remotes/origin/only_filters
  remotes/origin/only_filters_2
  remotes/origin/piepcat_maya1
  remotes/origin/pipecat_characters
  remotes/origin/pipecat_gemini_Live_2.0
  remotes/origin/pipecat_kotline
  remotes/origin/pipecat_livecat_local
  remotes/origin/pipecat_livekit
  remotes/origin/pipecat_maya1_te_hi
  remotes/origin/pipecat_mmodal_new_app
  remotes/origin/pipecat_modal
  remotes/origin/pipecat_modal_new_characters
  remotes/origin/pipecat_modal_v1
  remotes/origin/pipecat_veena
  remotes/origin/testing_webrtc
  remotes/origin/v3_version
  remotes/origin/v4_version
  remotes/origin/veena3
  remotes/origin/veena3_azure_nov1


Good, there's a `v3_version` branch and we're on `v4_version`. Let me look at the current v4 structure and get the v3 equivalent.


find /home/ubuntu/pipecat/src -name "*.py" | sort

List all Python files in src


git show v3_version:src/bot/core/main.py 2>/dev/null || git show v3_version:src/main.py 2>/dev/null || (echo "--- Listing v3 tree ---" && git ls-tree -r --name-only v3_version -- src/ | head -50)

Get v3 main bot file


/home/ubuntu/pipecat/src/bot/core/config.py
/home/ubuntu/pipecat/src/bot/core/helpers.py
/home/ubuntu/pipecat/src/bot/core/__init__.py
/home/ubuntu/pipecat/src/bot/core/main.py
/home/ubuntu/pipecat/src/bot/core/observers.py
/home/ubuntu/pipecat/src/bot/core/pipeline.py
/home/ubuntu/pipecat/src/bot/core/rtvi_events.py
/home/ubuntu/pipecat/src/bot/core/session_handlers.py
/home/ubuntu/pipecat/src/bot/core/session_state.py
/home/ubuntu/pipecat/src/bot/__init__.py
/home/ubuntu/pipecat/src/bot/services/custom_tts.py
/home/ubuntu/pipecat/src/bot/services/double_sentence_aggregator.py
/home/ubuntu/pipecat/src/bot/services/image_service.py
/home/ubuntu/pipecat/src/bot/services/image_upload.py
/home/ubuntu/pipecat/src/bot/services/__init__.py
/home/ubuntu/pipecat/src/bot/services/limit_checker.py
/home/ubuntu/pipecat/src/bot/services/music_service.py
/home/ubuntu/pipecat/src/bot/services/ppt_service.py
/home/ubuntu/pipecat/src/bot/services/video_service.py
/home/ubuntu/pipecat/src/config/constants.py
/home/ubuntu/pipecat/src/config/__init__.py
/home/ubuntu/pipecat/src/core/cache_layer.py
/home/ubuntu/pipecat/src/core/daily_client.py
/home/ubuntu/pipecat/src/core/database.py
/home/ubuntu/pipecat/src/core/exceptions.py
/home/ubuntu/pipecat/src/core/__init__.py
/home/ubuntu/pipecat/src/core/logging.py
/home/ubuntu/pipecat/src/core/pod_lifecycle.py
/home/ubuntu/pipecat/src/core/redis.py
/home/ubuntu/pipecat/src/core/storage.py
/home/ubuntu/pipecat/src/dto/common.py
/home/ubuntu/pipecat/src/dto/__init__.py
/home/ubuntu/pipecat/src/__init__.py
/home/ubuntu/pipecat/src/repository/base.py
/home/ubuntu/pipecat/src/repository/config_repository.py
/home/ubuntu/pipecat/src/repository/config_v2_repository.py
/home/ubuntu/pipecat/src/repository/device_repository.py
/home/ubuntu/pipecat/src/repository/__init__.py
/home/ubuntu/pipecat/src/repository/persona_repository.py
/home/ubuntu/pipecat/src/repository/plan_repository.py
/home/ubuntu/pipecat/src/repository/session_repository.py
/home/ubuntu/pipecat/src/repository/subscription_repository.py
/home/ubuntu/pipecat/src/repository/support_repository.py
/home/ubuntu/pipecat/src/repository/topup_repository.py
/home/ubuntu/pipecat/src/repository/usage_repository.py
/home/ubuntu/pipecat/src/routes/app.py
/home/ubuntu/pipecat/src/routes/bot.py
/home/ubuntu/pipecat/src/routes/device.py
/home/ubuntu/pipecat/src/routes/health.py
/home/ubuntu/pipecat/src/routes/__init__.py
/home/ubuntu/pipecat/src/routes/payment.py
/home/ubuntu/pipecat/src/routes/personas.py
/home/ubuntu/pipecat/src/routes/support.py
/home/ubuntu/pipecat/src/routes/usage.py
/home/ubuntu/pipecat/src/routes/v2.py
/home/ubuntu/pipecat/src/routes/video.py
/home/ubuntu/pipecat/src/service/app_service.py
/home/ubuntu/pipecat/src/service/app_v2_service.py
/home/ubuntu/pipecat/src/service/cf_flux_client.py
/home/ubuntu/pipecat/src/service/config_sync_service.py
/home/ubuntu/pipecat/src/service/device_service.py
/home/ubuntu/pipecat/src/service/gemini_image_client.py
/home/ubuntu/pipecat/src/service/image_describer.py
/home/ubuntu/pipecat/src/service/__init__.py
/home/ubuntu/pipecat/src/service/openai_image_client.py
/home/ubuntu/pipecat/src/service/payment_service.py
/home/ubuntu/pipecat/src/service/persona_service.py
/home/ubuntu/pipecat/src/service/prompt_writer.py
/home/ubuntu/pipecat/src/service/replicate_flux_client.py
/home/ubuntu/pipecat/src/service/support_service.py
/home/ubuntu/pipecat/src/service/usage_service.py
/home/ubuntu/pipecat/src/service/video_service.py


import asyncio
import threading
import time
from dotenv import load_dotenv
from loguru import logger

from pipecat.pipeline.runner import PipelineRunner

from src.config import constants
from src.bot.core.session_state import SessionState

from src.bot.core.config import load_system_instruction, cleanup_session, BotSessionState, ChunkState, TranscriptManager
from src.bot.core.pipeline import create_vad_analyzer, create_transport, create_stt_service, create_llm_service, create_tts_service, create_context_and_tools, create_processors, create_llm_text_processor, create_pipeline, create_task, create_gemini_live_service, create_gemini_live_context, create_gemini_live_pipeline
from src.bot.core.session_handlers import create_audio_data_handler, create_transcript_handler, create_client_message_handler, create_cleanup_if_no_user, create_participant_joined_handler, create_idle_timeout_handler, create_app_review_trigger, create_first_participant_joined_handler, create_participant_left_handler, create_client_disconnected_handler, create_client_ready_handler
from src.bot.services.image_service import create_cf_image_handler
from src.bot.services.video_service import create_video_handler, create_video_from_image_handler
from src.bot.services.ppt_service import create_ppt_handler
from src.bot.services.music_service import create_music_handler

load_dotenv()

async def run_bot_webrtc_global_agent(room_url: str, token: str, device_id: str = None, room_name: str = None, version: str = None, character_id: str = None, app_key: str = None, language: str = None, source: str = None, persona_id: str = None):
    bot_start_time = time.time()
    device_id = device_id or "common"
    session_id = room_name
    is_web = (source == "web")

    # Accept character_id as persona_id for backward compat
    if not persona_id and character_id:
        persona_id = character_id

    # Web beta: force Telugu language
    if is_web:
        language = "te"
        logger.info(f"WEB_SESSION: device_id={device_id} forcing language=te, no limits")

    session_state = SessionState()
    await session_state.start_queue_processor()

    # Initialize bot session
    bot_session = BotSessionState(session_id, device_id, bot_start_time)
    bot_session.persona_id = persona_id
    bot_session.app_key = app_key
    bot_session.language = language
    bot_session.source = source

    # Load persona info
    persona_prompt = None
    persona_voice = None
    persona_tools = []
    if persona_id:
        from src.repository.persona_repository import persona_repo
        persona = await persona_repo.get_persona_by_id(persona_id)
        if persona:
            bot_session.persona_name = persona.get("name")
            bot_session.content_type = persona.get("content_type")
            persona_prompt = persona.get("system_prompt")
            persona_voice = persona.get("tts_voice")
            persona_tools = persona.get("tools", [])
            logger.info(f"SESSION_START: persona={persona_id} name={persona.get('name')} content_type={bot_session.content_type} tools={persona_tools}")
        else:
            logger.warning(f"PERSONA_NOT_FOUND: persona={persona_id}")
    else:
        logger.info(f"SESSION_START: no_persona_id device_id={device_id}")

    # Load system instruction
    system_instruction = await load_system_instruction(
        language=language,
        persona_prompt=persona_prompt,
        persona_id=persona_id,
    )

    # Load usage state (persona-level free limits, content-level plan/topup)
    if persona_id and bot_session.content_type and device_id != "common":
        from src.service.usage_service import get_usage_state
        from src.repository.usage_repository import usage_repo

        usage = await get_usage_state(device_id, bot_session.content_type)
        bot_session.plan_remaining = usage["plan_remaining"]
        bot_session.topup_remaining = usage["topup_remaining"]
        bot_session.subscription_id = usage["subscription_id"]

        # Persona-level free limits
        free_count = persona.get("free_count", 0) if persona else 0
        free_window = persona.get("free_window_hours", 24) if persona else 24
        used_free = await usage_repo.count_persona_generations(device_id, persona_id, bot_session.content_type, free_window)
        bot_session.free_remaining = max(0, free_count - used_free)

        logger.info(f"USAGE_STATE: session_id={session_id} device_id={device_id} persona={persona_id} content={bot_session.content_type} free={bot_session.free_remaining}/{free_count} plan={bot_session.plan_remaining} topup={bot_session.topup_remaining}")

    # Determine which tools to enable
    if is_web:
        video_enabled = True
        logger.info(f"WEB_NO_LIMITS: session_id={session_id} device_id={device_id}")
    else:
        video_enabled = "generate_video" in persona_tools or "generate_video_from_image" in persona_tools

    ppt_enabled = "generate_ppt" in persona_tools
    music_enabled = "generate_music" in persona_tools
    bot_session.video_enabled = video_enabled

    use_gemini_live = (language == "te")
    bot_session.use_gemini_live = use_gemini_live

    vad_analyzer = create_vad_analyzer(session_id)
    bot_session.vad_analyzer = vad_analyzer

    # Create services and store references IMMEDIATELY on bot_session after each creation.
    # This ensures cleanup_session can release them even if a later create_* call throws.
    transport = create_transport(room_url, token, vad_analyzer)
    bot_session.transport = transport

    rtvi, transcript, audio_buffer = create_processors()
    transcript_manager = TranscriptManager()
    chunk_state = ChunkState()

    if use_gemini_live:
        # Gemini Live: single service handles STT + LLM + TTS
        llm = create_gemini_live_service(system_instruction, include_video=video_enabled, include_ppt=ppt_enabled, include_music=music_enabled, language=language)
        bot_session.llm_service = llm

        context_aggregator = create_gemini_live_context()
        bot_session.context_aggregator = context_aggregator

        logger.info(f"GEMINI_LIVE_SESSION: session_id={session_id} language={language} persona={persona_id}")

        pipeline = create_gemini_live_pipeline(transport, llm, context_aggregator, rtvi, audio_buffer, transcript)
    else:
        # Traditional pipeline: separate STT → LLM → TTS (mobile app only, web always uses Gemini Live)
        stt = create_stt_service()
        bot_session.stt_service = stt

        llm = create_llm_service(system_instruction)
        bot_session.llm_service = llm

        tts = create_tts_service(speaker=persona_voice)
        session_state.tts_service = tts

        context_aggregator = create_context_and_tools(
            llm,
            system_instruction,
            include_video=video_enabled,
            include_ppt=ppt_enabled,
            include_music=music_enabled,
        )
        bot_session.context_aggregator = context_aggregator

        llm_text_processor = create_llm_text_processor()
        pipeline = create_pipeline(transport, stt, context_aggregator, rtvi, llm, tts, audio_buffer, transcript, llm_text_processor)

    bot_session.pipeline = pipeline
    task = create_task(pipeline, rtvi, bot_session)

    audio_buffer.event_handler("on_audio_data")(create_audio_data_handler(bot_session, chunk_state))

    trigger_app_review_if_eligible = create_app_review_trigger(bot_session, rtvi)

    # Register tools based on persona config or legacy defaults
    # timeout_secs=120: image/video generation can take 20-60s; default 10s causes premature timeout
    # cancel_on_interruption=False: user may speak while waiting; don't cancel generation
    _tool_handlers = {
        "cf_image": lambda: create_cf_image_handler(
            session_state, rtvi, bot_session, transcript_manager, task, llm, trigger_app_review_if_eligible
        ),
        "generate_video": lambda: create_video_handler(
            session_state, rtvi, bot_session, transcript_manager, task, llm
        ),
        "generate_video_from_image": lambda: create_video_from_image_handler(
            session_state, rtvi, bot_session, transcript_manager, task, llm
        ),
        "generate_ppt": lambda: create_ppt_handler(
            session_state, rtvi, bot_session, transcript_manager, task, llm
        ),
        "generate_music": lambda: create_music_handler(
            session_state, rtvi, bot_session, transcript_manager, task, llm
        ),
    }

    # Register tools from persona config (or default cf_image if no persona)
    tools_to_register = persona_tools if persona_tools else ["cf_image"]
    if video_enabled and not persona_tools:
        tools_to_register.extend(["generate_video", "generate_video_from_image"])

    for tool_name in tools_to_register:
        if tool_name in _tool_handlers:
            try:
                llm.register_function(tool_name, _tool_handlers[tool_name](), timeout_secs=120, cancel_on_interruption=False)
                logger.info(f"FUNCTION_REGISTERED: {tool_name}")
            except Exception as e:
                logger.error(f"Failed to register {tool_name} handler: {e}", exc_info=True)

    transcript.event_handler("on_transcript_update")(create_transcript_handler(bot_session, transcript_manager, rtvi))
    rtvi.event_handler("on_client_ready")(create_client_ready_handler())
    rtvi.event_handler("on_client_message")(create_client_message_handler(bot_session, rtvi, task, transcript_manager))

    cleanup_if_no_user = create_cleanup_if_no_user(bot_session, session_state, task)

    transport.event_handler("on_participant_joined")(create_participant_joined_handler(bot_session, task))
    transport.event_handler("on_first_participant_joined")(create_first_participant_joined_handler(bot_session, rtvi, task, audio_buffer, chunk_state))
    transport.event_handler("on_participant_left")(create_participant_left_handler(bot_session, session_state, transcript_manager, chunk_state, task))
    transport.event_handler("on_client_disconnected")(create_client_disconnected_handler(bot_session, task))
    task.event_handler("on_idle_timeout")(create_idle_timeout_handler(bot_session))

    runner = PipelineRunner(handle_sigint=False)

    bot_session.no_user_timeout_task = asyncio.create_task(cleanup_if_no_user())
    logger.info(f"NO_USER_TIMEOUT_STARTED: session_id={session_id} timeout={constants.NO_USER_TIMEOUT}s")

    try:
        await runner.run(task)
    except Exception as e:
        logger.error(f"PIPELINE_ERROR: session_id={session_id} error={e}", exc_info=True)
        await task.cancel()
    finally:
        threads_before = threading.active_count()
        logger.info(f"CLEANUP_TRIGGER: session_id={session_id} reason=runner_finally threads={threads_before}")
        logger.info(f"RUNNER_CLEANUP_START: session_id={session_id} threads={threads_before}")

        bot_session.cancel_timeout_task()
        if bot_session.no_user_timeout_task:
            try:
                await bot_session.no_user_timeout_task
            except asyncio.CancelledError:
                pass

        # Timeout cleanup to prevent stuck native threads from blocking forever.
        # If pipeline.cleanup() or transport.cleanup() hangs on C++ thread joins,
        # we force-release references after 30s so the task always completes.
        try:
            await asyncio.wait_for(
                cleanup_session(session_state, bot_session, session_id, transcript_manager, chunk_state),
                timeout=30.0
            )
            threads_after = threading.active_count()
            logger.info(f"CLEANUP_THREADS: session_id={session_id} threads_before={threads_before} threads_after={threads_after} delta={threads_after - threads_before}")
        except asyncio.TimeoutError:
            threads_after = threading.active_count()
            logger.error(
                f"CLEANUP_TIMEOUT: session_id={session_id} cleanup hung for 30s, forcing release threads_before={threads_before} threads_after={threads_after}"
            )
            # Force-null everything to drop references even if native cleanup hung
            bot_session.pipeline = None
            bot_session.transport = None
            bot_session.vad_analyzer = None
            bot_session.stt_service = None
            bot_session.llm_service = None
            bot_session.audio_buffer = None
            bot_session.context_aggregator = None

        logger.info(f"RUNNER_CLEANUP_COMPLETE: session_id={session_id} threads={threading.active_count()}")


Now let me get the v4 version of main.py and the pipeline.py files for both versions.


git -C /home/ubuntu/pipecat show v3_version:src/bot/core/pipeline.py 2>/dev/null

Get v3 pipeline.py


     1→import asyncio
     2→import threading
     3→import time
     4→from dotenv import load_dotenv
     5→from loguru import logger
     6→
     7→from pipecat.pipeline.runner import PipelineRunner
     8→
     9→from src.config import constants
    10→from src.bot.core.session_state import SessionState
    11→
    12→from src.bot.core.config import load_system_instruction, cleanup_session, BotSessionState, ChunkState, TranscriptManager
    13→from src.bot.core.pipeline import create_vad_analyzer, create_transport, create_daily_transport, create_stt_service, create_llm_service, create_tts_service, create_context_and_tools, create_processors, create_llm_text_processor, create_pipeline, create_task, create_gemini_live_service, create_gemini_live_context, create_gemini_live_pipeline
    14→from src.bot.core.session_handlers import create_audio_data_handler, create_transcript_handler, create_client_message_handler, create_cleanup_if_no_user, create_client_connected_handler, create_idle_timeout_handler, create_app_review_trigger, create_first_client_connected_handler, create_client_disconnected_handler, create_client_ready_handler, create_daily_participant_joined_handler, create_daily_first_participant_joined_handler, create_daily_participant_left_handler
    15→from src.bot.services.image_service import create_cf_image_handler
    16→from src.bot.services.video_service import create_video_handler, create_video_from_image_handler
    17→from src.bot.services.ppt_service import create_ppt_handler
    18→from src.bot.services.music_service import create_music_handler
    19→
    20→load_dotenv()
    21→
    22→
    23→async def run_bot_webrtc_global_agent(webrtc_connection, device_id: str = None, session_id: str = None, version: str = None, character_id: str = None, app_key: str = None, language: str = None, source: str = None, persona_id: str = None, transport_type: str = "smallwebrtc", room_url: str = None, token: str = None):
    24→    bot_start_time = time.time()
    25→    device_id = device_id or "common"
    26→    is_web = (source == "web")
    27→
    28→    # Accept character_id as persona_id for backward compat
    29→    if not persona_id and character_id:
    30→        persona_id = character_id
    31→
    32→    # Web beta: force Telugu language
    33→    if is_web:
    34→        language = "te"
    35→        logger.info(f"WEB_SESSION: device_id={device_id} forcing language=te, no limits")
    36→
    37→    session_state = SessionState()
    38→    await session_state.start_queue_processor()
    39→
    40→    # Initialize bot session
    41→    bot_session = BotSessionState(session_id, device_id, bot_start_time)
    42→    bot_session.persona_id = persona_id
    43→    bot_session.app_key = app_key
    44→    bot_session.language = language
    45→    bot_session.source = source
    46→
    47→    # Load persona info
    48→    persona_prompt = None
    49→    persona_voice = None
    50→    persona_tools = []
    51→    if persona_id:
    52→        from src.repository.persona_repository import persona_repo
    53→        persona = await persona_repo.get_persona_by_id(persona_id)
    54→        if persona:
    55→            bot_session.persona_name = persona.get("name")
    56→            bot_session.content_type = persona.get("content_type")
    57→            persona_prompt = persona.get("system_prompt")
    58→            persona_voice = persona.get("tts_voice")
    59→            persona_tools = persona.get("tools", [])
    60→            logger.info(f"SESSION_START: persona={persona_id} name={persona.get('name')} content_type={bot_session.content_type} tools={persona_tools} transport={transport_type}")
    61→        else:
    62→            logger.warning(f"PERSONA_NOT_FOUND: persona={persona_id}")
    63→    else:
    64→        logger.info(f"SESSION_START: no_persona_id device_id={device_id} transport={transport_type}")
    65→
    66→    # Load system instruction
    67→    system_instruction = await load_system_instruction(
    68→        language=language,
    69→        persona_prompt=persona_prompt,
    70→        persona_id=persona_id,
    71→    )
    72→
    73→    # Load usage state (persona-level free limits, content-level plan/topup)
    74→    if persona_id and bot_session.content_type and device_id != "common":
    75→        from src.service.usage_service import get_usage_state
    76→        from src.repository.usage_repository import usage_repo
    77→
    78→        usage = await get_usage_state(device_id, bot_session.content_type)
    79→        bot_session.plan_remaining = usage["plan_remaining"]
    80→        bot_session.topup_remaining = usage["topup_remaining"]
    81→        bot_session.subscription_id = usage["subscription_id"]
    82→
    83→        # Persona-level free limits
    84→        free_count = persona.get("free_count", 0) if persona else 0
    85→        free_window = persona.get("free_window_hours", 24) if persona else 24
    86→        used_free = await usage_repo.count_persona_generations(device_id, persona_id, bot_session.content_type, free_window)
    87→        bot_session.free_remaining = max(0, free_count - used_free)
    88→
    89→        logger.info(f"USAGE_STATE: session_id={session_id} device_id={device_id} persona={persona_id} content={bot_session.content_type} free={bot_session.free_remaining}/{free_count} plan={bot_session.plan_remaining} topup={bot_session.topup_remaining} transport={transport_type}")
    90→
    91→    # Determine which tools to enable
    92→    if is_web:
    93→        video_enabled = True
    94→        logger.info(f"WEB_NO_LIMITS: session_id={session_id} device_id={device_id}")
    95→    else:
    96→        video_enabled = "generate_video" in persona_tools or "generate_video_from_image" in persona_tools
    97→
    98→    ppt_enabled = "generate_ppt" in persona_tools
    99→    music_enabled = "generate_music" in persona_tools
   100→    bot_session.video_enabled = video_enabled
   101→
   102→    use_gemini_live = (language == "te")
   103→    bot_session.use_gemini_live = use_gemini_live
   104→
   105→    vad_analyzer = create_vad_analyzer(session_id)
   106→    bot_session.vad_analyzer = vad_analyzer
   107→
   108→    # Create services and store references IMMEDIATELY on bot_session after each creation.
   109→    # This ensures cleanup_session can release them even if a later create_* call throws.
   110→    if transport_type == "daily":
   111→        transport = create_daily_transport(room_url, token, vad_analyzer)
   112→        logger.info(f"TRANSPORT_CREATED: type=daily session_id={session_id}")
   113→    else:
   114→        transport = create_transport(webrtc_connection, vad_analyzer)
   115→        logger.info(f"TRANSPORT_CREATED: type=smallwebrtc session_id={session_id}")
   116→    bot_session.transport = transport
   117→    bot_session.transport_type = transport_type
   118→
   119→    rtvi, transcript, audio_buffer = create_processors()
   120→    transcript_manager = TranscriptManager()
   121→    chunk_state = ChunkState()
   122→
   123→    if use_gemini_live:
   124→        # Gemini Live: single service handles STT + LLM + TTS
   125→        llm = create_gemini_live_service(system_instruction, include_video=video_enabled, include_ppt=ppt_enabled, include_music=music_enabled, language=language)
   126→        bot_session.llm_service = llm
   127→
   128→        context_aggregator = create_gemini_live_context()
   129→        bot_session.context_aggregator = context_aggregator
   130→
   131→        logger.info(f"GEMINI_LIVE_SESSION: session_id={session_id} language={language} persona={persona_id} transport={transport_type}")
   132→
   133→        pipeline = create_gemini_live_pipeline(transport, llm, context_aggregator, rtvi, audio_buffer, transcript)
   134→    else:
   135→        # Traditional pipeline: separate STT → LLM → TTS (mobile app only, web always uses Gemini Live)
   136→        stt = create_stt_service()
   137→        bot_session.stt_service = stt
   138→
   139→        llm = create_llm_service(system_instruction)
   140→        bot_session.llm_service = llm
   141→
   142→        tts = create_tts_service(speaker=persona_voice)
   143→        session_state.tts_service = tts
   144→
   145→        context_aggregator = create_context_and_tools(
   146→            llm,
   147→            system_instruction,
   148→            include_video=video_enabled,
   149→            include_ppt=ppt_enabled,
   150→            include_music=music_enabled,
   151→        )
   152→        bot_session.context_aggregator = context_aggregator
   153→
   154→        llm_text_processor = create_llm_text_processor()
   155→        pipeline = create_pipeline(transport, stt, context_aggregator, rtvi, llm, tts, audio_buffer, transcript, llm_text_processor)
   156→
   157→    bot_session.pipeline = pipeline
   158→    task = create_task(pipeline, rtvi, bot_session)
   159→
   160→    audio_buffer.event_handler("on_audio_data")(create_audio_data_handler(bot_session, chunk_state))
   161→
   162→    trigger_app_review_if_eligible = create_app_review_trigger(bot_session, rtvi)
   163→
   164→    # Register tools based on persona config or legacy defaults
   165→    # timeout_secs=120: image/video generation can take 20-60s; default 10s causes premature timeout
   166→    # cancel_on_interruption=False: user may speak while waiting; don't cancel generation
   167→    _tool_handlers = {
   168→        "cf_image": lambda: create_cf_image_handler(
   169→            session_state, rtvi, bot_session, transcript_manager, task, llm, trigger_app_review_if_eligible
   170→        ),
   171→        "generate_video": lambda: create_video_handler(
   172→            session_state, rtvi, bot_session, transcript_manager, task, llm
   173→        ),
   174→        "generate_video_from_image": lambda: create_video_from_image_handler(
   175→            session_state, rtvi, bot_session, transcript_manager, task, llm
   176→        ),
   177→        "generate_ppt": lambda: create_ppt_handler(
   178→            session_state, rtvi, bot_session, transcript_manager, task, llm
   179→        ),
   180→        "generate_music": lambda: create_music_handler(
   181→            session_state, rtvi, bot_session, transcript_manager, task, llm
   182→        ),
   183→    }
   184→
   185→    # Register tools from persona config (or default cf_image if no persona)
   186→    tools_to_register = persona_tools if persona_tools else ["cf_image"]
   187→    if video_enabled and not persona_tools:
   188→        tools_to_register.extend(["generate_video", "generate_video_from_image"])
   189→
   190→    for tool_name in tools_to_register:
   191→        if tool_name in _tool_handlers:
   192→            try:
   193→                llm.register_function(tool_name, _tool_handlers[tool_name](), timeout_secs=120, cancel_on_interruption=False)
   194→                logger.info(f"FUNCTION_REGISTERED: {tool_name}")
   195→            except Exception as e:
   196→                logger.error(f"Failed to register {tool_name} handler: {e}", exc_info=True)
   197→
   198→    transcript.event_handler("on_transcript_update")(create_transcript_handler(bot_session, transcript_manager, rtvi))
   199→    rtvi.event_handler("on_client_ready")(create_client_ready_handler())
   200→    rtvi.event_handler("on_client_message")(create_client_message_handler(bot_session, rtvi, task, transcript_manager))
   201→
   202→    cleanup_if_no_user = create_cleanup_if_no_user(bot_session, session_state, task)
   203→
   204→    if transport_type == "daily":
   205→        # Daily events: on_participant_joined, on_first_participant_joined, on_participant_left
   206→        transport.event_handler("on_participant_joined")(create_daily_participant_joined_handler(bot_session, task))
   207→        transport.event_handler("on_first_participant_joined")(create_daily_first_participant_joined_handler(bot_session, transport, rtvi, task, audio_buffer, chunk_state))
   208→        transport.event_handler("on_participant_left")(create_daily_participant_left_handler(bot_session, session_state, transcript_manager, chunk_state, task))
   209→    else:
   210→        # SmallWebRTC events: on_client_connected, on_client_disconnected
   211→        transport.event_handler("on_client_connected")(create_client_connected_handler(bot_session, task))
   212→        transport.event_handler("on_client_connected")(create_first_client_connected_handler(bot_session, rtvi, task, audio_buffer, chunk_state))
   213→        transport.event_handler("on_client_disconnected")(create_client_disconnected_handler(bot_session, session_state, transcript_manager, chunk_state, task))
   214→    task.event_handler("on_idle_timeout")(create_idle_timeout_handler(bot_session))
   215→
   216→    runner = PipelineRunner(handle_sigint=False)
   217→
   218→    bot_session.no_user_timeout_task = asyncio.create_task(cleanup_if_no_user())
   219→    logger.info(f"NO_USER_TIMEOUT_STARTED: session_id={session_id} transport={transport_type} timeout={constants.NO_USER_TIMEOUT}s")
   220→
   221→    try:
   222→        await runner.run(task)
   223→    except Exception as e:
   224→        logger.error(f"PIPELINE_ERROR: session_id={session_id} transport={transport_type} error={e}", exc_info=True)
   225→        await task.cancel()
   226→    finally:
   227→        threads_before = threading.active_count()
   228→        logger.info(f"CLEANUP_TRIGGER: session_id={session_id} transport={transport_type} reason=runner_finally threads={threads_before}")
   229→        logger.info(f"RUNNER_CLEANUP_START: session_id={session_id} transport={transport_type} threads={threads_before}")
   230→
   231→        bot_session.cancel_timeout_task()
   232→        if bot_session.no_user_timeout_task:
   233→            try:
   234→                await bot_session.no_user_timeout_task
   235→            except asyncio.CancelledError:
   236→                pass
   237→
   238→        # Timeout cleanup to prevent stuck native threads from blocking forever.
   239→        # If pipeline.cleanup() or transport.cleanup() hangs on C++ thread joins,
   240→        # we force-release references after 30s so the task always completes.
   241→        try:
   242→            await asyncio.wait_for(
   243→                cleanup_session(session_state, bot_session, session_id, transcript_manager, chunk_state),
   244→                timeout=30.0
   245→            )
   246→            threads_after = threading.active_count()
   247→            logger.info(f"CLEANUP_THREADS: session_id={session_id} transport={transport_type} threads_before={threads_before} threads_after={threads_after} delta={threads_after - threads_before}")
   248→        except asyncio.TimeoutError:
   249→            threads_after = threading.active_count()
   250→            logger.error(
   251→                f"CLEANUP_TIMEOUT: session_id={session_id} transport={transport_type} cleanup hung for 30s, forcing release threads_before={threads_before} threads_after={threads_after}"
   252→            )
   253→            # Force-null everything to drop references even if native cleanup hung
   254→            bot_session.pipeline = None
   255→            bot_session.transport = None
   256→            bot_session.vad_analyzer = None
   257→            bot_session.stt_service = None
   258→            bot_session.llm_service = None
   259→            bot_session.audio_buffer = None
   260→            bot_session.context_aggregator = None
   261→
   262→        logger.info(f"RUNNER_CLEANUP_COMPLETE: session_id={session_id} transport={transport_type} threads={threading.active_count()}")
   263→


"""Pipeline creation for bot."""

from loguru import logger
from pipecat.pipeline.pipeline import Pipeline
from pipecat.pipeline.task import PipelineParams, PipelineTask
from pipecat.processors.aggregators.openai_llm_context import OpenAILLMContext
from pipecat.processors.audio.audio_buffer_processor import AudioBufferProcessor
from pipecat.processors.frameworks.rtvi import RTVIConfig, RTVIProcessor
from pipecat.processors.transcript_processor import TranscriptProcessor
from pipecat.services.openai.stt import OpenAISTTService
from pipecat.services.openai.llm import OpenAILLMService
from pipecat.services.google.llm_vertex import GoogleVertexLLMService
from pipecat.services.google.gemini_live.vertex.llm import GeminiLiveVertexLLMService
from pipecat.services.groq.llm import GroqLLMService
from pipecat.services.openrouter.llm import OpenRouterLLMService
from typing import Any, Dict
from pipecat.transports.daily.transport import DailyParams, DailyTransport
from pipecat.adapters.schemas.tools_schema import ToolsSchema
from pipecat.audio.vad.silero import SileroVADAnalyzer, VADParams
from pipecat.observers.loggers.user_bot_latency_log_observer import UserBotLatencyLogObserver

from pipecat.services.soniox.stt import SonioxSTTService
from pipecat.transcriptions.language import Language

from pipecat.services.google.tts import GoogleTTSService

from src.config import constants
from pipecat.processors.aggregators.llm_text_processor import LLMTextProcessor
from src.bot.services.custom_tts import CustomTTSService
from src.bot.services.double_sentence_aggregator import DoubleSentenceAggregator
from src.bot.core.observers import InfoMetricsObserver, SafeRTVIObserver
from src.bot.services.image_service import cf_image_function
from src.bot.services.video_service import generate_video_function, generate_video_from_image_function
from src.bot.services.ppt_service import generate_ppt_function
from src.bot.services.music_service import generate_music_function

class CustomOpenRouterLLMService(OpenRouterLLMService):
    """OpenRouter LLM service with provider routing support.

    Overrides build_chat_completion_params to pass OpenRouter-specific fields
    (like 'provider') via extra_body, since the OpenAI SDK rejects unknown
    top-level kwargs but allows extra_body for non-standard API fields.
    """

    def __init__(self, *, provider_order: list[str] = None, allow_fallbacks: bool = True, **kwargs):
        super().__init__(**kwargs)
        self._provider_routing = None
        if provider_order:
            self._provider_routing = {
                "order": provider_order,
                "allow_fallbacks": allow_fallbacks,
            }

    def build_chat_completion_params(self, params_from_context: Dict[str, Any]) -> Dict[str, Any]:
        params = super().build_chat_completion_params(params_from_context)
        if self._provider_routing:
            existing_extra = params.get("extra_body", {}) or {}
            existing_extra["provider"] = self._provider_routing
            params["extra_body"] = existing_extra
        return params

def create_vad_analyzer(session_id: str):
    """Create VAD analyzer with configured parameters."""
    vad = SileroVADAnalyzer(
        params=VADParams(
            confidence=constants.VAD_CONFIDENCE,
            stop_secs=constants.VAD_STOP_SECS,
            start_secs=constants.VAD_START_SECS,
            min_volume=constants.VAD_MIN_VOLUME,
        )
    )
    logger.info(f"VAD_CREATED: session_id={session_id}")
    return vad

def create_transport(room_url: str, token: str, vad_analyzer):
    """Create Daily transport with configured parameters."""
    return DailyTransport(
        room_url,
        token,
        "Maya Global Agent Bot",
        DailyParams(
            audio_in_enabled=True,
            audio_out_enabled=True,
            video_in_enabled=False,
            transcription_enabled=False,
            audio_in_sample_rate=constants.SAMPLE_RATE_16K,
            audio_out_sample_rate=constants.SAMPLE_RATE_16K,
            # audio_in_filter=RNNoiseFilter(resampler_quality="MQ"),
            vad_analyzer=vad_analyzer,
            cam_enabled=True,
        ),
    )

def create_stt_service():
    """Create OpenAI STT service."""
    stt = OpenAISTTService(
        api_key=constants.OPENAI_API_KEY,
        settings=OpenAISTTService.Settings(
            model=constants.STT_MODEL,
            language=constants.STT_LANGUAGE,
        ),
    )
    logger.info(f"STT_SERVICE_CREATED: provider=openai model={constants.STT_MODEL}")
    return stt

def create_soniox_stt_service():
    """Create Soniox STT service for Telugu (web sessions)."""
    stt = SonioxSTTService(
        api_key=constants.SONIOX_API_KEY,
        sample_rate=constants.SAMPLE_RATE_16K,
        settings=SonioxSTTService.Settings(
            model="stt-rt-v4",
            language_hints=[Language.TE],
        ),
    )
    logger.info("STT_SERVICE_CREATED: provider=soniox model=stt-rt-v4 language=te")
    return stt

def _select_llm_provider_weighted() -> str:
    """Select LLM provider using weighted traffic routing.

    Uses per-provider percentage weights from config. Each provider gets
    its configured percentage of traffic, remainder goes to gemini (default).

    Example config weights: {"openai": 30, "groq": 5, "openrouter": 10}
    -> 30% openai, 5% groq, 10% openrouter, 55% gemini

    Example config weights: {"groq": 5}
    -> 5% groq, 95% gemini
    """
    import random
    from src.repository.config_repository import config_repo

    weights = config_repo.get_llm_weights()

    # Sanitize: ensure all values are valid numbers, skip bad entries
    valid_providers = ["openai", "groq", "openrouter"]  # Known non-default providers
    sanitized = {}
    for provider, percentage in weights.items():
        if provider not in valid_providers:
            logger.warning(f"LLM_WEIGHTS_UNKNOWN_PROVIDER: provider={provider} skipped")
            continue
        try:
            pct = int(percentage)
            if pct > 0:
                sanitized[provider] = pct
        except (TypeError, ValueError):
            logger.warning(f"LLM_WEIGHTS_INVALID_VALUE: provider={provider} value={percentage} skipped")
            continue

    # Cap total at 100 to prevent gemini from being completely excluded by accident
    total = sum(sanitized.values())
    if total > 100:
        logger.warning(f"LLM_WEIGHTS_OVERFLOW: total={total} exceeds 100, capping proportionally")
        # Scale down proportionally so total = 100
        scale = 100.0 / total
        sanitized = {k: max(1, int(v * scale)) for k, v in sanitized.items()}

    # If no valid weights, fall back to gemini
    if not sanitized:
        logger.info("LLM_WEIGHTED_SELECTION: no valid weights, chosen=gemini (default)")
        return "gemini"

    roll = random.randint(1, 100)

    # Walk through providers in order, accumulating percentages
    cumulative = 0
    for provider, percentage in sanitized.items():
        cumulative += percentage
        if roll <= cumulative:
            logger.info(f"LLM_WEIGHTED_SELECTION: weights={sanitized} roll={roll} chosen={provider}")
            return provider

    # Remaining traffic goes to gemini (default)
    logger.info(f"LLM_WEIGHTED_SELECTION: weights={sanitized} roll={roll} chosen=gemini (default)")
    return "gemini"

def _create_groq_llm_service():
    """Create Groq LLM service."""
    if not constants.GROQ_API_KEY:
        logger.error("GROQ_API_KEY not set! Falling back to OpenAI")
        return _create_openai_llm_service()
    logger.info(f"LLM_PROVIDER: Groq {constants.GROQ_MODEL} active")
    return GroqLLMService(
        api_key=constants.GROQ_API_KEY,
        model=constants.GROQ_MODEL,
        params=GroqLLMService.InputParams(
            max_completion_tokens=constants.LLM_MAX_OUTPUT_TOKENS,
            temperature=constants.LLM_TEMPERATURE,
        ),
    )

def _create_openrouter_llm_service():
    """Create OpenRouter LLM service routed to Groq provider.

    Uses CustomOpenRouterLLMService to pass provider routing via extra_body
    (the only way to send non-standard fields through the OpenAI SDK).
    """
    if not constants.OPENROUTER_API_KEY:
        logger.error("OPENROUTER_API_KEY not set! Falling back to OpenAI")
        return _create_openai_llm_service()
    logger.info(f"LLM_PROVIDER: OpenRouter {constants.OPENROUTER_LLM_MODEL} active | provider=groq")
    return CustomOpenRouterLLMService(
        api_key=constants.OPENROUTER_API_KEY,
        model=constants.OPENROUTER_LLM_MODEL,
        provider_order=["groq"],
        allow_fallbacks=True,
        params=OpenRouterLLMService.InputParams(
            max_completion_tokens=constants.LLM_MAX_OUTPUT_TOKENS,
            temperature=constants.LLM_TEMPERATURE,
        ),
    )

def _create_openai_llm_service():
    """Create OpenAI LLM service."""
    logger.info("LLM_PROVIDER: OpenAI GPT-4o-mini active")
    return OpenAILLMService(
        api_key=constants.OPENAI_API_KEY,
        settings=OpenAILLMService.Settings(
            model="gpt-4o-mini",
            max_completion_tokens=constants.LLM_MAX_OUTPUT_TOKENS,
            temperature=constants.LLM_TEMPERATURE,
        ),
    )

def _create_gemini_llm_service(system_instruction: str):
    """Create Google Vertex AI Gemini LLM service with GCP account switching."""
    import random
    from src.repository.config_repository import config_repo

    # Get GCP mode from cache (no DB call - refreshed periodically by server)
    mode = config_repo.get_gcp_mode()

    # Determine which account to use
    if mode == "random":
        new_percentage = config_repo.get_gcp_new_account_percentage()
        use_new_account = random.randint(1, 100) <= new_percentage
        chosen = "new" if use_new_account else "old"
        logger.info(f"GCP_RANDOM_SELECTION: mode=random new_percentage={new_percentage} chosen={chosen}")
    else:
        use_new_account = (mode == "new")
        chosen = mode

    if use_new_account:
        project_id = constants.GCP_NEW_PROJECT_ID
        credentials_path = constants.GCP_NEW_CREDENTIALS_PATH
        account_tag = "NEW"
        logger.info(f"GCP_ACCOUNT: NEW account active | project={project_id} | credentials={credentials_path}")
    else:
        project_id = constants.GOOGLE_CLOUD_PROJECT
        credentials_path = constants.GCP_CREDENTIALS_PATH
        account_tag = "OLD"
        logger.info(f"GCP_ACCOUNT: OLD account active | project={project_id} | credentials={credentials_path}")

    # Use global endpoint for better capacity (Google recommends this for Standard PayGo)
    # Global endpoint routes to region with most availability, reducing 429 errors
    return GoogleVertexLLMService(
        name=f"GeminiLLM-{account_tag}",  # Shows in error logs which account caused the error
        project_id=project_id,
        location="global",  # Global endpoint for multi-region capacity pool
        model=constants.GEMINI_MODEL,
        credentials_path=credentials_path,
        system_instruction=system_instruction,
        params=GoogleVertexLLMService.InputParams(
            max_output_tokens=constants.LLM_MAX_OUTPUT_TOKENS,
            temperature=constants.LLM_TEMPERATURE,
        ),
    )

def create_llm_service(system_instruction: str):
    """Create LLM service with system instruction. Uses cached DB config to switch providers.

    LLM Provider Modes:
    - "gemini": Use Google Vertex AI Gemini (with GCP account switching)
    - "openai": Use OpenAI GPT-4o-mini
    - "groq": Use Groq (Llama 3.3 70B or configured model)
    - "openrouter": Use OpenRouter (access multiple providers via single API)
    - "random": Weighted traffic split between providers based on per-provider percentages

    Random mode traffic routing (via "weights" config):
      {"openai": 30, "groq": 5, "openrouter": 10} -> 30% OpenAI, 5% Groq, 10% OpenRouter, 55% Gemini
      {"groq": 5}               -> 5% Groq, 95% Gemini
      {"openai": 50}            -> 50% OpenAI, 50% Gemini (legacy behavior)

    GCP Account Modes (only for gemini):
    - "old": Always use mayaresearchgcp (old account)
    - "new": Always use custom-frame-484619-j2 (new account)
    - "random": Randomly choose between old and new based on percentage
    """
    from src.repository.config_repository import config_repo

    # Get LLM provider from cache (defaults to "gemini" if not set)
    llm_provider = config_repo.get_llm_provider()

    # Determine which LLM to use
    if llm_provider == "random":
        chosen_llm = _select_llm_provider_weighted()
    else:
        chosen_llm = llm_provider
    # Create the chosen LLM service
    if chosen_llm == "groq":
        return _create_groq_llm_service()

    if chosen_llm == "openai":
        return _create_openai_llm_service()

    if chosen_llm == "openrouter":
        return _create_openrouter_llm_service()

    # Default: Use Google Vertex AI Gemini
    return _create_gemini_llm_service(system_instruction)

def create_tts_service(speaker: str = None):
    """Create TTS service with optional custom speaker/voice.

    Args:
        speaker: TTS voice to use. If None, uses DEFAULT_TTS_SPEAKER.
    """
    voice = speaker or constants.DEFAULT_TTS_SPEAKER
    logger.info(f"TTS_SERVICE_CREATED: speaker={voice}")
    return CustomTTSService(
        api_url=constants.TTS_API_URL,
        api_key=constants.VEENA3_API_KEY,
        speaker=voice,
        sample_rate=constants.SAMPLE_RATE_16K,
    )

def create_gemini_tts_service(voice: str = None):
    """Create Gemini TTS service for web/Telugu sessions.

    Uses the same GCP account switching logic as Gemini LLM.
    Gemini TTS outputs at 24kHz natively.
    """
    import random
    from src.repository.config_repository import config_repo

    tts_voice = voice or constants.GEMINI_LIVE_VOICE  # "Kore" for Telugu

    mode = config_repo.get_gcp_mode()
    if mode == "random":
        new_percentage = config_repo.get_gcp_new_account_percentage()
        use_new_account = random.randint(1, 100) <= new_percentage
    else:
        use_new_account = (mode == "new")

    if use_new_account:
        credentials_path = constants.GCP_NEW_CREDENTIALS_PATH
        account_tag = "NEW"
    else:
        credentials_path = constants.GCP_CREDENTIALS_PATH
        account_tag = "OLD"

    tts = GoogleTTSService(
        credentials_path=credentials_path,
        location="asia-southeast1",
        settings=GoogleTTSService.Settings(
            voice="te-IN-Chirp3-HD-Erinome",
            language="te-IN",
        ),
        sample_rate=constants.SAMPLE_RATE,
    )
    logger.info(f"TTS_SERVICE_CREATED: provider=chirp3-hd voice=te-IN-Chirp3-HD-Erinome language=te-IN location=asia-southeast1 gcp={account_tag}")
    return tts

def create_context_and_tools(llm, system_instruction: str, include_video: bool = False, include_ppt: bool = False, include_music: bool = False):
    """Create LLM context with tools.

    All characters get image generation. Video, PPT, and music are controlled by persona config.
    """
    standard_tools = [cf_image_function]

    if include_video:
        standard_tools.append(generate_video_function)
        standard_tools.append(generate_video_from_image_function)
        logger.info("PIPELINE_TOOLS: Video generation functions ENABLED (t2v + i2v)")

    if include_ppt:
        standard_tools.append(generate_ppt_function)
        logger.info("PIPELINE_TOOLS: PPT generation function ENABLED")

    if include_music:
        standard_tools.append(generate_music_function)
        logger.info("PIPELINE_TOOLS: Music generation function ENABLED (Lyria)")

    logger.debug(f"TOOLS_REGISTERED: image=true video={include_video} ppt={include_ppt} music={include_music} total={len(standard_tools)}")

    tools = ToolsSchema(standard_tools=standard_tools) if standard_tools else None

    # Build messages with system instruction for OpenAI compatibility
    messages = [
        {"role": "system", "content": system_instruction},
        {"role": "user", "content": "Start the Conversation :"}
    ]

    context = OpenAILLMContext(
        messages=messages,
        tools=tools
    )
    context_aggregator = llm.create_context_aggregator(context)
    return context_aggregator

def create_llm_text_processor():
    """Create LLM text processor that combines 2 sentences before sending to TTS.

    This sits between the LLM and TTS in the pipeline. It aggregates streaming
    LLM tokens into 2-sentence chunks, reducing TTS API calls by ~half.
    """
    return LLMTextProcessor(
        text_aggregator=DoubleSentenceAggregator(sentences_per_chunk=2),
    )

def create_processors():
    """Create RTVI, transcript, and audio buffer processors."""
    rtvi = RTVIProcessor(config=RTVIConfig(config=[]))
    transcript = TranscriptProcessor()

    audio_buffer = AudioBufferProcessor(
        sample_rate=constants.SAMPLE_RATE,
        num_channels=1,
        buffer_size=constants.SAMPLE_RATE * 2 * constants.CHUNK_DURATION,
    )

    return rtvi, transcript, audio_buffer

def create_pipeline(transport, stt, context_aggregator, rtvi, llm, tts, audio_buffer, transcript, llm_text_processor=None):
    """Create the processing pipeline."""
    # Build core pipeline: llm -> [optional llm_text_processor] -> tts
    llm_to_tts = [llm, llm_text_processor, tts] if llm_text_processor else [llm, tts]

    pipeline = Pipeline([
        transport.input(),
        stt,
        transcript.user(),
        context_aggregator.user(),
        rtvi,
        *llm_to_tts,
        transport.output(),
        audio_buffer,
        transcript.assistant(),
        context_aggregator.assistant(),
    ])
    mode = "2-sentence aggregation" if llm_text_processor else "default (1 sentence)"
    logger.info(f"PIPELINE: Traditional STT mode (Audio → STT → Text → LLM → TTS) | text_aggregation={mode}")
    return pipeline

def create_task(pipeline, rtvi, bot_session):
    """Create pipeline task with observers."""
    return PipelineTask(
        pipeline,
        params=PipelineParams(
            enable_metrics=True,
            enable_usage_metrics=True,
            allow_interruptions=True
        ),
        observers=[
            SafeRTVIObserver(rtvi),  # Use SafeRTVIObserver to handle startup race condition
            InfoMetricsObserver(bot_session),
            UserBotLatencyLogObserver()
        ],
    )

# ---- Gemini Live pipeline (Telugu / multi-language) ----

def create_gemini_live_service(system_instruction: str, voice_id: str = None, include_video: bool = False, include_ppt: bool = False, include_music: bool = False, language: str = None):
    """Create Gemini Live Vertex AI service — handles STT + LLM + TTS in one service.

    Uses the same GCP account switching logic as the regular Gemini LLM service.
    """
    import random
    from src.repository.config_repository import config_repo

    voice = voice_id or constants.GEMINI_LIVE_VOICE

    standard_tools = [cf_image_function]
    if include_video:
        standard_tools.append(generate_video_function)
        standard_tools.append(generate_video_from_image_function)
        logger.info("GEMINI_LIVE_TOOLS: Video generation functions ENABLED (t2v + i2v)")

    if include_ppt:
        standard_tools.append(generate_ppt_function)
        logger.info("GEMINI_LIVE_TOOLS: PPT generation function ENABLED")

    if include_music:
        standard_tools.append(generate_music_function)
        logger.info("GEMINI_LIVE_TOOLS: Music generation function ENABLED (Lyria)")

    tools = ToolsSchema(standard_tools=standard_tools) if standard_tools else None

    # GCP account switching (same logic as _create_gemini_llm_service)
    mode = config_repo.get_gcp_mode()
    if mode == "random":
        new_percentage = config_repo.get_gcp_new_account_percentage()
        use_new_account = random.randint(1, 100) <= new_percentage
    else:
        use_new_account = (mode == "new")

    if use_new_account:
        project_id = constants.GCP_NEW_PROJECT_ID
        credentials_path = constants.GCP_NEW_CREDENTIALS_PATH
        account_tag = "NEW"
    else:
        project_id = constants.GOOGLE_CLOUD_PROJECT
        credentials_path = constants.GCP_CREDENTIALS_PATH
        account_tag = "OLD"

    # Map language code to Language enum for Gemini Live
    from pipecat.transcriptions.language import Language
    _LANG_MAP = {"te": Language.TE_IN, "hi": Language.HI_IN}
    gemini_language = _LANG_MAP.get(language, Language.HI_IN)

    llm = GeminiLiveVertexLLMService(
        credentials_path=credentials_path,
        project_id=project_id,
        location="us-east4",
        system_instruction=system_instruction,
        tools=tools,
        settings=GeminiLiveVertexLLMService.Settings(
            voice=voice,
            language=gemini_language,
        ),
    )
    logger.info(f"GEMINI_LIVE_CREATED: voice={voice} language={gemini_language} tools={len(standard_tools)} gcp_account={account_tag} project={project_id}")
    return llm

def create_gemini_live_context():
    """Create context aggregator for Gemini Live.

    Uses LLMContext + LLMContextAggregatorPair directly (not deprecated OpenAILLMContext).
    System instruction is set on the LLM service via Settings, not in context messages.
    VAD is placed inside LLMUserAggregatorParams per official Gemini Live pattern.
    """
    from pipecat.processors.aggregators.llm_context import LLMContext
    from pipecat.processors.aggregators.llm_response_universal import (
        LLMContextAggregatorPair,
        LLMUserAggregatorParams,
    )

    messages = []

    context = LLMContext(messages)
    context_aggregator = LLMContextAggregatorPair(
        context,
        user_params=LLMUserAggregatorParams(
            vad_analyzer=SileroVADAnalyzer(params=VADParams(stop_secs=0.5)),
        ),
    )
    return context_aggregator

def create_gemini_live_pipeline(transport, gemini_live, context_aggregator, rtvi, audio_buffer, transcript):
    """Create pipeline for Gemini Live — audio goes directly to Gemini, no separate STT/TTS.

    Keeps RTVI (frontend events), audio_buffer (recording), transcript (conversation tracking).
    """
    pipeline = Pipeline([
        transport.input(),
        context_aggregator.user(),
        gemini_live,
        rtvi,
        transport.output(),
        audio_buffer,
        transcript.assistant(),
        context_aggregator.assistant(),
    ])
    logger.info("PIPELINE: Gemini Live mode (Audio → Context → Gemini Live → Audio)")
    return pipeline


     1→"""Pipeline creation for bot."""
     2→
     3→from loguru import logger
     4→from pipecat.pipeline.pipeline import Pipeline
     5→from pipecat.pipeline.task import PipelineParams, PipelineTask
     6→from pipecat.processors.aggregators.openai_llm_context import OpenAILLMContext
     7→from pipecat.processors.audio.audio_buffer_processor import AudioBufferProcessor
     8→from pipecat.processors.frameworks.rtvi import RTVIConfig, RTVIProcessor
     9→from pipecat.processors.transcript_processor import TranscriptProcessor
    10→from pipecat.services.openai.stt import OpenAISTTService
    11→from pipecat.services.deepgram.stt import DeepgramSTTService
    12→from pipecat.services.openai.llm import OpenAILLMService
    13→from pipecat.services.google.llm_vertex import GoogleVertexLLMService
    14→from pipecat.services.google.gemini_live.vertex.llm import GeminiLiveVertexLLMService
    15→from pipecat.services.groq.llm import GroqLLMService
    16→from pipecat.services.openrouter.llm import OpenRouterLLMService
    17→from typing import Any, Dict
    18→from pipecat.transports.smallwebrtc.transport import SmallWebRTCTransport
    19→from pipecat.transports.base_transport import TransportParams
    20→from pipecat.adapters.schemas.tools_schema import ToolsSchema
    21→from pipecat.audio.vad.silero import SileroVADAnalyzer, VADParams
    22→from pipecat.observers.loggers.user_bot_latency_log_observer import UserBotLatencyLogObserver
    23→
    24→from pipecat.services.soniox.stt import SonioxSTTService
    25→from pipecat.transcriptions.language import Language
    26→
    27→from pipecat.services.google.tts import GoogleTTSService
    28→
    29→from src.config import constants
    30→from pipecat.processors.aggregators.llm_text_processor import LLMTextProcessor
    31→from src.bot.services.custom_tts import CustomTTSService
    32→from src.bot.services.double_sentence_aggregator import DoubleSentenceAggregator
    33→from src.bot.core.observers import InfoMetricsObserver, SafeRTVIObserver
    34→from src.bot.services.image_service import cf_image_function
    35→from src.bot.services.video_service import generate_video_function, generate_video_from_image_function
    36→from src.bot.services.ppt_service import generate_ppt_function
    37→from src.bot.services.music_service import generate_music_function
    38→
    39→
    40→class CustomOpenRouterLLMService(OpenRouterLLMService):
    41→    """OpenRouter LLM service with provider routing support.
    42→
    43→    Overrides build_chat_completion_params to pass OpenRouter-specific fields
    44→    (like 'provider') via extra_body, since the OpenAI SDK rejects unknown
    45→    top-level kwargs but allows extra_body for non-standard API fields.
    46→    """
    47→
    48→    def __init__(self, *, provider_order: list[str] = None, allow_fallbacks: bool = True, **kwargs):
    49→        super().__init__(**kwargs)
    50→        self._provider_routing = None
    51→        if provider_order:
    52→            self._provider_routing = {
    53→                "order": provider_order,
    54→                "allow_fallbacks": allow_fallbacks,
    55→            }
    56→
    57→    def build_chat_completion_params(self, params_from_context: Dict[str, Any]) -> Dict[str, Any]:
    58→        params = super().build_chat_completion_params(params_from_context)
    59→        if self._provider_routing:
    60→            existing_extra = params.get("extra_body", {}) or {}
    61→            existing_extra["provider"] = self._provider_routing
    62→            params["extra_body"] = existing_extra
    63→        return params
    64→
    65→
    66→def create_vad_analyzer(session_id: str):
    67→    """Create VAD analyzer with configured parameters."""
    68→    vad = SileroVADAnalyzer(
    69→        params=VADParams(
    70→            confidence=constants.VAD_CONFIDENCE,
    71→            stop_secs=constants.VAD_STOP_SECS,
    72→            start_secs=constants.VAD_START_SECS,
    73→            min_volume=constants.VAD_MIN_VOLUME,
    74→        )
    75→    )
    76→    logger.info(f"VAD_CREATED: session_id={session_id}")
    77→    return vad
    78→
    79→
    80→def create_transport(webrtc_connection, vad_analyzer):
    81→    """Create SmallWebRTC transport with configured parameters."""
    82→    return SmallWebRTCTransport(
    83→        webrtc_connection,
    84→        TransportParams(
    85→            audio_in_enabled=True,
    86→            audio_out_enabled=True,
    87→            video_in_enabled=False,
    88→            audio_in_sample_rate=constants.SAMPLE_RATE_16K,
    89→            audio_out_sample_rate=48000,
    90→            vad_analyzer=vad_analyzer,
    91→        ),
    92→    )
    93→
    94→
    95→def create_daily_transport(room_url: str, token: str, vad_analyzer):
    96→    """Create Daily transport with configured parameters."""
    97→    from pipecat.transports.daily.transport import DailyTransport, DailyParams
    98→
    99→    return DailyTransport(
   100→        room_url,
   101→        token,
   102→        "Maya Global Agent Bot",
   103→        DailyParams(
   104→            api_key=constants.DAILY_API_KEY,
   105→            audio_in_enabled=True,
   106→            audio_out_enabled=True,
   107→            video_in_enabled=False,
   108→            camera_out_enabled=False,
   109→            audio_in_sample_rate=constants.SAMPLE_RATE_16K,
   110→            audio_out_sample_rate=constants.SAMPLE_RATE_16K,
   111→            vad_analyzer=vad_analyzer,
   112→            transcription_enabled=False,
   113→        ),
   114→    )
   115→
   116→
   117→def _create_openai_stt_service():
   118→    """Create OpenAI STT service."""
   119→    stt = OpenAISTTService(
   120→        api_key=constants.OPENAI_API_KEY,
   121→        settings=OpenAISTTService.Settings(
   122→            model=constants.STT_MODEL,
   123→            language=constants.STT_LANGUAGE,
   124→        ),
   125→    )
   126→    logger.info(f"STT_SERVICE_CREATED: provider=openai model={constants.STT_MODEL}")
   127→    return stt
   128→
   129→
   130→def _create_cf_nova3_stt_service():
   131→    """Create CF Workers AI Deepgram Nova-3 STT service.
   132→
   133→    Connects to a CF Worker that proxies WebSocket to @cf/deepgram/nova-3.
   134→    Uses Pipecat's DeepgramSTTService with base_url pointing to the CF Worker.
   135→    """
   136→    stt = DeepgramSTTService(
   137→        api_key=constants.CF_STT_API_KEY,
   138→        base_url=constants.CF_STT_WORKER_URL,
   139→        encoding="linear16",
   140→        sample_rate=constants.SAMPLE_RATE_16K,
   141→        settings=DeepgramSTTService.Settings(
   142→            model="nova-3-general",
   143→            language=Language.HI,
   144→            interim_results=True,
   145→            punctuate=True,
   146→            smart_format=False,
   147→            endpointing=300,
   148→            utterance_end_ms=1000,
   149→            vad_events=False,
   150→        ),
   151→    )
   152→    logger.info(f"STT_SERVICE_CREATED: provider=cf_nova3 base_url={constants.CF_STT_WORKER_URL}")
   153→    return stt
   154→
   155→
   156→def create_stt_service():
   157→    """Create STT service — hardcoded to CF Deepgram Nova-3 for testing.
   158→
   159→    TODO: revert to weighted selection after testing
   160→    # STT config (from configurations table, key: stt_config):
   161→    #   {"openai": 100}                  -> 100% OpenAI
   162→    #   {"cf_deepgram": 100}             -> 100% CF Deepgram Nova-3
   163→    #   {"openai": 70, "cf_deepgram": 30} -> 70% OpenAI, 30% CF Deepgram
   164→    """
   165→    try:
   166→        return _create_cf_nova3_stt_service()
   167→    except Exception as e:
   168→        logger.error(f"STT_CF_DEEPGRAM_FAILED: {e}, falling back to openai")
   169→        return _create_openai_stt_service()
   170→
   171→
   172→def create_soniox_stt_service():
   173→    """Create Soniox STT service for Telugu (web sessions)."""
   174→    stt = SonioxSTTService(
   175→        api_key=constants.SONIOX_API_KEY,
   176→        sample_rate=constants.SAMPLE_RATE_16K,
   177→        settings=SonioxSTTService.Settings(
   178→            model="stt-rt-v4",
   179→            language_hints=[Language.TE],
   180→        ),
   181→    )
   182→    logger.info("STT_SERVICE_CREATED: provider=soniox model=stt-rt-v4 language=te")
   183→    return stt
   184→
   185→
   186→def _select_llm_provider_weighted() -> str:
   187→    """Select LLM provider using weighted traffic routing.
   188→
   189→    Uses per-provider percentage weights from config. Each provider gets
   190→    its configured percentage of traffic, remainder goes to gemini (default).
   191→
   192→    Example config weights: {"openai": 30, "groq": 5, "openrouter": 10}
   193→    -> 30% openai, 5% groq, 10% openrouter, 55% gemini
   194→
   195→    Example config weights: {"groq": 5}
   196→    -> 5% groq, 95% gemini
   197→    """
   198→    import random
   199→    from src.repository.config_repository import config_repo
   200→
   201→    weights = config_repo.get_llm_weights()
   202→
   203→    # Sanitize: ensure all values are valid numbers, skip bad entries
   204→    valid_providers = ["openai", "groq", "openrouter"]  # Known non-default providers
   205→    sanitized = {}
   206→    for provider, percentage in weights.items():
   207→        if provider not in valid_providers:
   208→            logger.warning(f"LLM_WEIGHTS_UNKNOWN_PROVIDER: provider={provider} skipped")
   209→            continue
   210→        try:
   211→            pct = int(percentage)
   212→            if pct > 0:
   213→                sanitized[provider] = pct
   214→        except (TypeError, ValueError):
   215→            logger.warning(f"LLM_WEIGHTS_INVALID_VALUE: provider={provider} value={percentage} skipped")
   216→            continue
   217→
   218→    # Cap total at 100 to prevent gemini from being completely excluded by accident
   219→    total = sum(sanitized.values())
   220→    if total > 100:
   221→        logger.warning(f"LLM_WEIGHTS_OVERFLOW: total={total} exceeds 100, capping proportionally")
   222→        # Scale down proportionally so total = 100
   223→        scale = 100.0 / total
   224→        sanitized = {k: max(1, int(v * scale)) for k, v in sanitized.items()}
   225→
   226→    # If no valid weights, fall back to gemini
   227→    if not sanitized:
   228→        logger.info("LLM_WEIGHTED_SELECTION: no valid weights, chosen=gemini (default)")
   229→        return "gemini"
   230→
   231→    roll = random.randint(1, 100)
   232→
   233→    # Walk through providers in order, accumulating percentages
   234→    cumulative = 0
   235→    for provider, percentage in sanitized.items():
   236→        cumulative += percentage
   237→        if roll <= cumulative:
   238→            logger.info(f"LLM_WEIGHTED_SELECTION: weights={sanitized} roll={roll} chosen={provider}")
   239→            return provider
   240→
   241→    # Remaining traffic goes to gemini (default)
   242→    logger.info(f"LLM_WEIGHTED_SELECTION: weights={sanitized} roll={roll} chosen=gemini (default)")
   243→    return "gemini"
   244→
   245→
   246→def _create_groq_llm_service():
   247→    """Create Groq LLM service."""
   248→    if not constants.GROQ_API_KEY:
   249→        logger.error("GROQ_API_KEY not set! Falling back to OpenAI")
   250→        return _create_openai_llm_service()
   251→    logger.info(f"LLM_PROVIDER: Groq {constants.GROQ_MODEL} active")
   252→    return GroqLLMService(
   253→        api_key=constants.GROQ_API_KEY,
   254→        model=constants.GROQ_MODEL,
   255→        params=GroqLLMService.InputParams(
   256→            max_completion_tokens=constants.LLM_MAX_OUTPUT_TOKENS,
   257→            temperature=constants.LLM_TEMPERATURE,
   258→        ),
   259→    )
   260→
   261→
   262→def _create_openrouter_llm_service():
   263→    """Create OpenRouter LLM service routed to Groq provider.
   264→
   265→    Uses CustomOpenRouterLLMService to pass provider routing via extra_body
   266→    (the only way to send non-standard fields through the OpenAI SDK).
   267→    """
   268→    if not constants.OPENROUTER_API_KEY:
   269→        logger.error("OPENROUTER_API_KEY not set! Falling back to OpenAI")
   270→        return _create_openai_llm_service()
   271→    logger.info(f"LLM_PROVIDER: OpenRouter {constants.OPENROUTER_LLM_MODEL} active | provider=groq")
   272→    return CustomOpenRouterLLMService(
   273→        api_key=constants.OPENROUTER_API_KEY,
   274→        model=constants.OPENROUTER_LLM_MODEL,
   275→        provider_order=["groq"],
   276→        allow_fallbacks=True,
   277→        params=OpenRouterLLMService.InputParams(
   278→            max_completion_tokens=constants.LLM_MAX_OUTPUT_TOKENS,
   279→            temperature=constants.LLM_TEMPERATURE,
   280→        ),
   281→    )
   282→
   283→
   284→def _create_openai_llm_service():
   285→    """Create OpenAI LLM service."""
   286→    logger.info("LLM_PROVIDER: OpenAI GPT-4o-mini active")
   287→    return OpenAILLMService(
   288→        api_key=constants.OPENAI_API_KEY,
   289→        settings=OpenAILLMService.Settings(
   290→            model="gpt-4o-mini",
   291→            max_completion_tokens=constants.LLM_MAX_OUTPUT_TOKENS,
   292→            temperature=constants.LLM_TEMPERATURE,
   293→        ),
   294→    )
   295→
   296→
   297→def _create_gemini_llm_service(system_instruction: str):
   298→    """Create Google Vertex AI Gemini LLM service with GCP account switching."""
   299→    import random
   300→    from src.repository.config_repository import config_repo
   301→
   302→    # Get GCP mode from cache (no DB call - refreshed periodically by server)
   303→    mode = config_repo.get_gcp_mode()
   304→
   305→    # Determine which account to use
   306→    if mode == "random":
   307→        new_percentage = config_repo.get_gcp_new_account_percentage()
   308→        use_new_account = random.randint(1, 100) <= new_percentage
   309→        chosen = "new" if use_new_account else "old"
   310→        logger.info(f"GCP_RANDOM_SELECTION: mode=random new_percentage={new_percentage} chosen={chosen}")
   311→    else:
   312→        use_new_account = (mode == "new")
   313→        chosen = mode
   314→
   315→    if use_new_account:
   316→        project_id = constants.GCP_NEW_PROJECT_ID
   317→        credentials_path = constants.GCP_NEW_CREDENTIALS_PATH
   318→        account_tag = "NEW"
   319→        logger.info(f"GCP_ACCOUNT: NEW account active | project={project_id} | credentials={credentials_path}")
   320→    else:
   321→        project_id = constants.GOOGLE_CLOUD_PROJECT
   322→        credentials_path = constants.GCP_CREDENTIALS_PATH
   323→        account_tag = "OLD"
   324→        logger.info(f"GCP_ACCOUNT: OLD account active | project={project_id} | credentials={credentials_path}")
   325→
   326→    # Use global endpoint for better capacity (Google recommends this for Standard PayGo)
   327→    # Global endpoint routes to region with most availability, reducing 429 errors
   328→    return GoogleVertexLLMService(
   329→        name=f"GeminiLLM-{account_tag}",  # Shows in error logs which account caused the error
   330→        project_id=project_id,
   331→        location="global",  # Global endpoint for multi-region capacity pool
   332→        model=constants.GEMINI_MODEL,
   333→        credentials_path=credentials_path,
   334→        system_instruction=system_instruction,
   335→        params=GoogleVertexLLMService.InputParams(
   336→            max_output_tokens=constants.LLM_MAX_OUTPUT_TOKENS,
   337→            temperature=constants.LLM_TEMPERATURE,
   338→        ),
   339→    )
   340→
   341→
   342→def create_llm_service(system_instruction: str):
   343→    """Create LLM service with system instruction. Uses cached DB config to switch providers.
   344→
   345→    LLM Provider Modes:
   346→    - "gemini": Use Google Vertex AI Gemini (with GCP account switching)
   347→    - "openai": Use OpenAI GPT-4o-mini
   348→    - "groq": Use Groq (Llama 3.3 70B or configured model)
   349→    - "openrouter": Use OpenRouter (access multiple providers via single API)
   350→    - "random": Weighted traffic split between providers based on per-provider percentages
   351→
   352→    Random mode traffic routing (via "weights" config):
   353→      {"openai": 30, "groq": 5, "openrouter": 10} -> 30% OpenAI, 5% Groq, 10% OpenRouter, 55% Gemini
   354→      {"groq": 5}               -> 5% Groq, 95% Gemini
   355→      {"openai": 50}            -> 50% OpenAI, 50% Gemini (legacy behavior)
   356→
   357→    GCP Account Modes (only for gemini):
   358→    - "old": Always use mayaresearchgcp (old account)
   359→    - "new": Always use custom-frame-484619-j2 (new account)
   360→    - "random": Randomly choose between old and new based on percentage
   361→    """
   362→    from src.repository.config_repository import config_repo
   363→
   364→    # Get LLM provider from cache (defaults to "gemini" if not set)
   365→    llm_provider = config_repo.get_llm_provider()
   366→
   367→    # Determine which LLM to use
   368→    if llm_provider == "random":
   369→        chosen_llm = _select_llm_provider_weighted()
   370→    else:
   371→        chosen_llm = llm_provider
   372→    # Create the chosen LLM service
   373→    if chosen_llm == "groq":
   374→        return _create_groq_llm_service()
   375→
   376→    if chosen_llm == "openai":
   377→        return _create_openai_llm_service()
   378→
   379→    if chosen_llm == "openrouter":
   380→        return _create_openrouter_llm_service()
   381→
   382→    # Default: Use Google Vertex AI Gemini
   383→    return _create_gemini_llm_service(system_instruction)
   384→
   385→
   386→def create_tts_service(speaker: str = None):
   387→    """Create TTS service with optional custom speaker/voice.
   388→
   389→    Args:
   390→        speaker: TTS voice to use. If None, uses DEFAULT_TTS_SPEAKER.
   391→    """
   392→    voice = speaker or constants.DEFAULT_TTS_SPEAKER
   393→    logger.info(f"TTS_SERVICE_CREATED: speaker={voice}")
   394→    return CustomTTSService(
   395→        api_url=constants.TTS_API_URL,
   396→        api_key=constants.VEENA3_API_KEY,
   397→        speaker=voice,
   398→        sample_rate=constants.SAMPLE_RATE_16K,
   399→    )
   400→
   401→
   402→def create_gemini_tts_service(voice: str = None):
   403→    """Create Gemini TTS service for web/Telugu sessions.
   404→
   405→    Uses the same GCP account switching logic as Gemini LLM.
   406→    Gemini TTS outputs at 24kHz natively.
   407→    """
   408→    import random
   409→    from src.repository.config_repository import config_repo
   410→
   411→    tts_voice = voice or constants.GEMINI_LIVE_VOICE  # "Kore" for Telugu
   412→
   413→    mode = config_repo.get_gcp_mode()
   414→    if mode == "random":
   415→        new_percentage = config_repo.get_gcp_new_account_percentage()
   416→        use_new_account = random.randint(1, 100) <= new_percentage
   417→    else:
   418→        use_new_account = (mode == "new")
   419→
   420→    if use_new_account:
   421→        credentials_path = constants.GCP_NEW_CREDENTIALS_PATH
   422→        account_tag = "NEW"
   423→    else:
   424→        credentials_path = constants.GCP_CREDENTIALS_PATH
   425→        account_tag = "OLD"
   426→
   427→    tts = GoogleTTSService(
   428→        credentials_path=credentials_path,
   429→        location="asia-southeast1",
   430→        settings=GoogleTTSService.Settings(
   431→            voice="te-IN-Chirp3-HD-Erinome",
   432→            language="te-IN",
   433→        ),
   434→        sample_rate=constants.SAMPLE_RATE,
   435→    )
   436→    logger.info(f"TTS_SERVICE_CREATED: provider=chirp3-hd voice=te-IN-Chirp3-HD-Erinome language=te-IN location=asia-southeast1 gcp={account_tag}")
   437→    return tts
   438→
   439→
   440→def create_context_and_tools(llm, system_instruction: str, include_video: bool = False, include_ppt: bool = False, include_music: bool = False):
   441→    """Create LLM context with tools.
   442→
   443→    All characters get image generation. Video, PPT, and music are controlled by persona config.
   444→    """
   445→    standard_tools = [cf_image_function]
   446→
   447→    if include_video:
   448→        standard_tools.append(generate_video_function)
   449→        standard_tools.append(generate_video_from_image_function)
   450→        logger.info("PIPELINE_TOOLS: Video generation functions ENABLED (t2v + i2v)")
   451→
   452→    if include_ppt:
   453→        standard_tools.append(generate_ppt_function)
   454→        logger.info("PIPELINE_TOOLS: PPT generation function ENABLED")
   455→
   456→    if include_music:
   457→        standard_tools.append(generate_music_function)
   458→        logger.info("PIPELINE_TOOLS: Music generation function ENABLED (Lyria)")
   459→
   460→    logger.debug(f"TOOLS_REGISTERED: image=true video={include_video} ppt={include_ppt} music={include_music} total={len(standard_tools)}")
   461→
   462→    tools = ToolsSchema(standard_tools=standard_tools) if standard_tools else None
   463→
   464→    # Build messages with system instruction for OpenAI compatibility
   465→    messages = [
   466→        {"role": "system", "content": system_instruction},
   467→        {"role": "user", "content": "Start the Conversation :"}
   468→    ]
   469→
   470→    context = OpenAILLMContext(
   471→        messages=messages,
   472→        tools=tools
   473→    )
   474→    context_aggregator = llm.create_context_aggregator(context)
   475→    return context_aggregator
   476→
   477→
   478→def create_llm_text_processor():
   479→    """Create LLM text processor that combines 2 sentences before sending to TTS.
   480→
   481→    This sits between the LLM and TTS in the pipeline. It aggregates streaming
   482→    LLM tokens into 2-sentence chunks, reducing TTS API calls by ~half.
   483→    """
   484→    return LLMTextProcessor(
   485→        text_aggregator=DoubleSentenceAggregator(sentences_per_chunk=2),
   486→    )
   487→
   488→
   489→def create_processors():
   490→    """Create RTVI, transcript, and audio buffer processors."""
   491→    rtvi = RTVIProcessor(config=RTVIConfig(config=[]))
   492→    transcript = TranscriptProcessor()
   493→
   494→    audio_buffer = AudioBufferProcessor(
   495→        sample_rate=constants.SAMPLE_RATE,
   496→        num_channels=1,
   497→        buffer_size=constants.SAMPLE_RATE * 2 * constants.CHUNK_DURATION,
   498→    )
   499→
   500→    return rtvi, transcript, audio_buffer
   501→
   502→
   503→def create_pipeline(transport, stt, context_aggregator, rtvi, llm, tts, audio_buffer, transcript, llm_text_processor=None):
   504→    """Create the processing pipeline."""
   505→    # Build core pipeline: llm -> [optional llm_text_processor] -> tts
   506→    llm_to_tts = [llm, llm_text_processor, tts] if llm_text_processor else [llm, tts]
   507→
   508→    pipeline = Pipeline([
   509→        transport.input(),
   510→        stt,
   511→        transcript.user(),
   512→        context_aggregator.user(),
   513→        rtvi,
   514→        *llm_to_tts,
   515→        transport.output(),
   516→        audio_buffer,
   517→        transcript.assistant(),
   518→        context_aggregator.assistant(),
   519→    ])
   520→    mode = "2-sentence aggregation" if llm_text_processor else "default (1 sentence)"
   521→    logger.info(f"PIPELINE: Traditional STT mode (Audio → STT → Text → LLM → TTS) | text_aggregation={mode}")
   522→    return pipeline
   523→
   524→
   525→def create_task(pipeline, rtvi, bot_session):
   526→    """Create pipeline task with observers."""
   527→    return PipelineTask(
   528→        pipeline,
   529→        params=PipelineParams(
   530→            enable_metrics=True,
   531→            enable_usage_metrics=True,
   532→            allow_interruptions=True
   533→        ),
   534→        observers=[
   535→            SafeRTVIObserver(rtvi),  # Use SafeRTVIObserver to handle startup race condition
   536→            InfoMetricsObserver(bot_session),
   537→            UserBotLatencyLogObserver()
   538→        ],
   539→    )
   540→
   541→
   542→# ---- Gemini Live pipeline (Telugu / multi-language) ----
   543→
   544→def create_gemini_live_service(system_instruction: str, voice_id: str = None, include_video: bool = False, include_ppt: bool = False, include_music: bool = False, language: str = None):
   545→    """Create Gemini Live Vertex AI service — handles STT + LLM + TTS in one service.
   546→
   547→    Uses the same GCP account switching logic as the regular Gemini LLM service.
   548→    """
   549→    import random
   550→    from src.repository.config_repository import config_repo
   551→
   552→    voice = voice_id or constants.GEMINI_LIVE_VOICE
   553→
   554→    standard_tools = [cf_image_function]
   555→    if include_video:
   556→        standard_tools.append(generate_video_function)
   557→        standard_tools.append(generate_video_from_image_function)
   558→        logger.info("GEMINI_LIVE_TOOLS: Video generation functions ENABLED (t2v + i2v)")
   559→
   560→    if include_ppt:
   561→        standard_tools.append(generate_ppt_function)
   562→        logger.info("GEMINI_LIVE_TOOLS: PPT generation function ENABLED")
   563→
   564→    if include_music:
   565→        standard_tools.append(generate_music_function)
   566→        logger.info("GEMINI_LIVE_TOOLS: Music generation function ENABLED (Lyria)")
   567→
   568→    tools = ToolsSchema(standard_tools=standard_tools) if standard_tools else None
   569→
   570→    # GCP account switching (same logic as _create_gemini_llm_service)
   571→    mode = config_repo.get_gcp_mode()
   572→    if mode == "random":
   573→        new_percentage = config_repo.get_gcp_new_account_percentage()
   574→        use_new_account = random.randint(1, 100) <= new_percentage
   575→    else:
   576→        use_new_account = (mode == "new")
   577→
   578→    if use_new_account:
   579→        project_id = constants.GCP_NEW_PROJECT_ID
   580→        credentials_path = constants.GCP_NEW_CREDENTIALS_PATH
   581→        account_tag = "NEW"
   582→    else:
   583→        project_id = constants.GOOGLE_CLOUD_PROJECT
   584→        credentials_path = constants.GCP_CREDENTIALS_PATH
   585→        account_tag = "OLD"
   586→
   587→    # Map language code to Language enum for Gemini Live
   588→    from pipecat.transcriptions.language import Language
   589→    _LANG_MAP = {"te": Language.TE_IN, "hi": Language.HI_IN}
   590→    gemini_language = _LANG_MAP.get(language, Language.HI_IN)
   591→
   592→    llm = GeminiLiveVertexLLMService(
   593→        credentials_path=credentials_path,
   594→        project_id=project_id,
   595→        location="us-east4",
   596→        system_instruction=system_instruction,
   597→        tools=tools,
   598→        settings=GeminiLiveVertexLLMService.Settings(
   599→            voice=voice,
   600→            language=gemini_language,
   601→        ),
   602→    )
   603→    logger.info(f"GEMINI_LIVE_CREATED: voice={voice} language={gemini_language} tools={len(standard_tools)} gcp_account={account_tag} project={project_id}")
   604→    return llm
   605→
   606→
   607→def create_gemini_live_context():
   608→    """Create context aggregator for Gemini Live.
   609→
   610→    Uses LLMContext + LLMContextAggregatorPair directly (not deprecated OpenAILLMContext).
   611→    System instruction is set on the LLM service via Settings, not in context messages.
   612→    VAD is placed inside LLMUserAggregatorParams per official Gemini Live pattern.
   613→    """
   614→    from pipecat.processors.aggregators.llm_context import LLMContext
   615→    from pipecat.processors.aggregators.llm_response_universal import (
   616→        LLMContextAggregatorPair,
   617→        LLMUserAggregatorParams,
   618→    )
   619→
   620→    messages = []
   621→
   622→    context = LLMContext(messages)
   623→    context_aggregator = LLMContextAggregatorPair(
   624→        context,
   625→        user_params=LLMUserAggregatorParams(
   626→            vad_analyzer=SileroVADAnalyzer(params=VADParams(stop_secs=0.5)),
   627→        ),
   628→    )
   629→    return context_aggregator
   630→
   631→
   632→def create_gemini_live_pipeline(transport, gemini_live, context_aggregator, rtvi, audio_buffer, transcript):
   633→    """Create pipeline for Gemini Live — audio goes directly to Gemini, no separate STT/TTS.
   634→
   635→    Keeps RTVI (frontend events), audio_buffer (recording), transcript (conversation tracking).
   636→    """
   637→    pipeline = Pipeline([
   638→        transport.input(),
   639→        context_aggregator.user(),
   640→        gemini_live,
   641→        rtvi,
   642→        transport.output(),
   643→        audio_buffer,
   644→        transcript.assistant(),
   645→        context_aggregator.assistant(),
   646→    ])
   647→    logger.info("PIPELINE: Gemini Live mode (Audio → Context → Gemini Live → Audio)")
   648→    return pipeline
   649→
   650→


Now let me compare the session_handlers between v3 and v4, particularly the event handlers that deal with participant/client connections and the first-joined handler that likely triggers initial speech.


git -C /home/ubuntu/pipecat show v3_version:src/bot/core/session_handlers.py 2>/dev/null

Get v3 session_handlers.py