o i`@sldZddlmZmZmZmZddlmZddlm Z m Z m Z ddl m Z ddlmZGddde e Zd S) zbLLM switcher for switching between different LLMs at runtime, with different switching strategies.)AnyListOptionalType)DirectFunction)ServiceSwitcherServiceSwitcherStrategyManual StrategyType) LLMContext) LLMServicec seZdZdZefdeedeeffdd Z e deefddZ e defd d Z d e deefd d Z dddddeedededeefddZddddededeefddZZS) LLMSwitcherzA pipeline that switches between different LLMs at runtime. Example:: llm_switcher = LLMSwitcher(llms=[openai_llm, anthropic_llm]) llms strategy_typecst||dS)a/Initialize the service switcher with a list of LLMs and a switching strategy. Args: llms: List of LLM services to switch between. strategy_type: The strategy class to use for switching between LLMs. Defaults to ``ServiceSwitcherStrategyManual``. N)super__init__)selfr r __class__Q/home/ubuntu/.local/lib/python3.10/site-packages/pipecat/pipeline/llm_switcher.pyrs zLLMSwitcher.__init__returncCs|jS)zGet the list of LLMs managed by this switcher. Returns: List of LLM services managed by this switcher. )servicesrrrrr +szLLMSwitcher.llmscCs|jjS)zGet the currently active LLM. Returns: The currently active LLM service, or None if no LLM is active. )strategyactive_servicerrrr active_llm4szLLMSwitcher.active_llmcontextcs(|jr|jjdd|i|IdHSdS)aRun a one-shot, out-of-band (i.e. out-of-pipeline) inference with the given LLM context, using the currently active LLM. Args: context: The LLM context containing conversation history. **kwargs: Additional arguments forwarded to the active LLM's run_inference (e.g. max_tokens, system_instruction). Returns: The LLM's response as a string, or None if no response is generated. rNr)r run_inference)rrkwargsrrrr=s zLLMSwitcher.run_inferenceNT)cancel_on_interruption timeout_secs function_namehandlerrr cCs$|jD] }|j|||||dqdS)a]Register a function handler for LLM function calls, on all LLMs, active or not. Args: function_name: The name of the function to handle. Use None to handle all function calls with a catch-all handler. handler: The function handler. Should accept a single FunctionCallParams parameter. start_callback: Legacy callback function (deprecated). Put initialization code at the top of your handler instead. .. deprecated:: 0.0.59 The `start_callback` parameter is deprecated and will be removed in a future version. cancel_on_interruption: Whether to cancel this function call when an interruption occurs. Defaults to True. timeout_secs: Optional timeout in seconds for the function call. )r!r"start_callbackrr N)r register_function)rr!r"r#rr llmrrrr$Ls zLLMSwitcher.register_functioncCs |jD] }|j|||dqdS)aRegister a direct function handler for LLM function calls, on all LLMs, active or not. Args: handler: The direct function to register. Must follow DirectFunction protocol. cancel_on_interruption: Whether to cancel this function call when an interruption occurs. Defaults to True. timeout_secs: Optional timeout in seconds for the function call. )r"rr N)r register_direct_function)rr"rr r%rrrr&os z$LLMSwitcher.register_direct_function)N)__name__ __module__ __qualname____doc__rrr rr rpropertyr rr rstrrrboolfloatr$rr& __classcell__rrrrr sF  'r N)r*typingrrrr(pipecat.adapters.schemas.direct_functionr!pipecat.pipeline.service_switcherrrr *pipecat.processors.aggregators.llm_contextr pipecat.services.llm_servicer r rrrrs