o i@7@sdZddlZeededWdn1swYddlZddlZddlmZddl m Z m Z m Z m Z mZmZmZddlmZddlmZdd lmZmZd d lmZd d lmZmZGd ddeZGdddeZGdddeZdS)a)OpenAI Realtime LLM context and aggregator implementations. .. deprecated:: 0.0.92 OpenAI Realtime no longer uses types from this module under the hood. It now uses ``LLMContext`` and ``LLMContextAggregatorPair``. Using the new patterns should allow you to not need types from this module. BEFORE:: # Setup context = OpenAILLMContext(messages, tools) context_aggregator = llm.create_context_aggregator(context) # Context aggregator type context_aggregator: OpenAIContextAggregatorPair # Context frame type frame: OpenAILLMContextFrame # Context type context: OpenAIRealtimeLLMContext # or context: OpenAILLMContext AFTER:: # Setup context = LLMContext(messages, tools) context_aggregator = LLMContextAggregatorPair(context) # Context aggregator type context_aggregator: LLMContextAggregatorPair # Context frame type frame: LLMContextFrame # Context type context: LLMContext NalwaysaTypes in pipecat.services.openai.realtime.llm (or pipecat.services.openai_realtime.llm) are deprecated. OpenAI Realtime no longer uses types from this module under the hood. It now uses `LLMContext` and `LLMContextAggregatorPair`. Using the new patterns should allow you to not need types from this module. BEFORE: ``` # Setup context = OpenAILLMContext(messages, tools) context_aggregator = llm.create_context_aggregator(context) # Context aggregator type context_aggregator: OpenAIContextAggregatorPair # Context frame type frame: OpenAILLMContextFrame # Context type context: OpenAIRealtimeLLMContext # or context: OpenAILLMContext ``` AFTER: ``` # Setup context = LLMContext(messages, tools) context_aggregator = LLMContextAggregatorPair(context) # Context aggregator type context_aggregator: LLMContextAggregatorPair # Context frame type frame: LLMContextFrame # Context type context: LLMContext ``` )logger)FrameFunctionCallResultFrameInterimTranscriptionFrameLLMMessagesUpdateFrameLLMSetToolsFrame LLMTextFrameTranscriptionFrame)OpenAILLMContext)FrameDirection) OpenAIAssistantContextAggregatorOpenAIUserContextAggregator)events)RealtimeFunctionCallResultFrameRealtimeMessagesUpdateFramecsXeZdZdZdfdd ZddZededdfd d Zd d Z d dZ ddZ Z S)OpenAIRealtimeLLMContextaOpenAI Realtime LLM context with session management and message conversion. Extends the standard OpenAI LLM context to support real-time session properties, instruction management, and conversion between standard message formats and realtime conversation items. .. deprecated:: 0.0.99 `OpenAIRealtimeLLMContext` is deprecated and will be removed in a future version. Use the universal `LLMContext` and `LLMContextAggregatorPair` instead. See `OpenAILLMContext` docstring for migration guide. Nc s$tjd||d||dS)aInitialize the OpenAIRealtimeLLMContext. Args: messages: Initial conversation messages. Defaults to None. tools: Available function tools. Defaults to None. **kwargs: Additional arguments passed to parent OpenAILLMContext. )messagestoolsN)super__init__&_OpenAIRealtimeLLMContext__setup_local)selfrrkwargs __class__r\/home/ubuntu/.local/lib/python3.10/site-packages/pipecat/services/openai/realtime/context.pyr{s z!OpenAIRealtimeLLMContext.__init__cCsd|_d|_d|_dS)NT)llm_needs_settings_updatellm_needs_initial_messages_session_instructionsrrrr __setup_localsz&OpenAIRealtimeLLMContext.__setup_localobjreturncCs&t|trt|tst|_||S)zUpgrade a standard OpenAI LLM context to a realtime context. Args: obj: The OpenAILLMContext instance to upgrade. Returns: The upgraded OpenAIRealtimeLLMContext instance. ) isinstancer rrr)r%rrrupgrade_to_realtimes z,OpenAIRealtimeLLMContext.upgrade_to_realtimecCs|ddkrK|d}t|dtr=d}|dD]!}|ddkr.|d|d7}qtd|dd |qtjdd tjd |d gd S|ddkrq|drq|dd}tjd|d|dd|dddStd|dS)zConvert a standard message format to a realtime conversation item. Args: message: The standard message dictionary to convert. Returns: A ConversationItem instance for the realtime API. roleusercontentrtypetext z+Unhandled content type in context message: z - message input_textr,r-r)r,r+ assistant tool_callsr function_callidfunctionname arguments)r,call_idr8r9z1Unhandled message type in from_standard_message: N)getr'listrerrorrConversationItem ItemContent)rr/r+ctcrrrfrom_standard_messages0   z.OpenAIRealtimeLLMContext.from_standard_messagec Cs|jsgSt|j}|dddkr;d|_|d}|d}t|tr*||_n t|t r7|dd|_|s;gSt |dkrR|dddkrR| |dgSd }d }dd d d |t j|dd|gdgdgS)a@Get conversation items for initializing the realtime session history. Converts the context's messages to a format suitable for the realtime API, handling system instructions and conversation history packaging. Returns: List of conversation items for session initialization. rr)systemTr+r-rr*z This is a previously saved conversation. Please treat this conversation history as a starting point for the current conversation.aS This is the end of the previously saved conversation. Please continue the conversation from here. If the last message is a user instruction or question, act on that instruction or answer the question. If the last message is an assistant response, simple say that you are ready to continue the conversation.r/r0z )indentr1r2)rcopydeepcopyr;r popr'strr"r<lenrBjoinjsondumps)rrrCr+ intro_text trailing_textrrr%get_messages_for_initializing_historys6      z>OpenAIRealtimeLLMContext.get_messages_for_initializing_historycCs(dd|jdjdgd}||dS)zAdd a user content item as a standard message to the context. Args: item: The conversation item to add as a user message. r*r-rr1)r)r+N)r+ transcript add_message)ritemr/rrr add_user_content_item_as_messagesz9OpenAIRealtimeLLMContext.add_user_content_item_as_message)NN) __name__ __module__ __qualname____doc__rr staticmethodr r(rBrPrT __classcell__rrrrrns  #>rcs8eZdZdZejfdedeffdd ZddZZ S)#OpenAIRealtimeUserContextAggregatora]User context aggregator for OpenAI Realtime API. Handles user input frames and generates appropriate context updates for the realtime conversation, including message updates and tool settings. .. deprecated:: 0.0.99 `OpenAIRealtimeUserContextAggregator` is deprecated and will be removed in a future version. Use the universal `LLMContext` and `LLMContextAggregatorPair` instead. See `OpenAILLMContext` docstring for migration guide. Args: context: The OpenAI realtime LLM context. **kwargs: Additional arguments passed to parent aggregator. frame directioncs\t||IdHt|tr|t|jdIdHt|tr,|||IdHdSdS)zProcess incoming frames and handle realtime-specific frame types. Args: frame: The frame to process. direction: The direction of frame flow in the pipeline. N)context)r process_framer'r push_framer_contextrrr\r]rrrr_ s  z1OpenAIRealtimeUserContextAggregator.process_framecsdS)zPush user input aggregation. Currently ignores all user input coming into the pipeline as realtime audio input is handled directly by the service. Nrr#rrrpush_aggregation5s z4OpenAIRealtimeUserContextAggregator.push_aggregation) rUrVrWrXr DOWNSTREAMrr_rcrZrrrrr[sr[cs<eZdZdZdedeffdd Zdeffdd ZZ S)(OpenAIRealtimeAssistantContextAggregatoraZAssistant context aggregator for OpenAI Realtime API. Handles assistant output frames from the realtime service, filtering out duplicate text frames and managing function call results. .. deprecated:: 0.0.99 `OpenAIRealtimeAssistantContextAggregator` is deprecated and will be removed in a future version. Use the universal `LLMContext` and `LLMContextAggregatorPair` instead. See `OpenAILLMContext` docstring for migration guide. Args: context: The OpenAI realtime LLM context. **kwargs: Additional arguments passed to parent aggregator. r\r]cs.t|tttfst||IdHdSdS)zProcess assistant frames, filtering out duplicate text content. Args: frame: The frame to process. direction: The direction of frame flow in the pipeline. N)r'r r rrr_rbrrrr_Ysz6OpenAIRealtimeAssistantContextAggregator.process_framecs2t|IdH|t|dtjIdHdS)zHandle function call result and notify the realtime service. Args: frame: The function call result frame to handle. N) result_frame)rhandle_function_call_resultr`rr UPSTREAM)rr\rrrrgcs  zDOpenAIRealtimeAssistantContextAggregator.handle_function_call_result) rUrVrWrXrr r_rrgrZrrrrreAs re) rXwarningscatch_warnings simplefilterwarnrFrLlogururpipecat.frames.framesrrrrrr r 1pipecat.processors.aggregators.openai_llm_contextr "pipecat.processors.frame_processorr pipecat.services.openai.llmr rrrframesrrrr[rerrrrs*(  $ $  !3