o i-*@sdZddlZddlZddlmZddlmZmZmZm Z m Z m Z m Z ddl mZddlmZddlmZmZdd lmZdd lmZmZGd d d eZGd ddeZGdddeZdS)z;OpenAI Realtime LLM context and aggregator implementations.N)logger)FrameFunctionCallResultFrameInterimTranscriptionFrameLLMMessagesUpdateFrameLLMSetToolsFrame LLMTextFrameTranscriptionFrame)OpenAILLMContext)FrameDirection) OpenAIAssistantContextAggregatorOpenAIUserContextAggregator)events)RealtimeFunctionCallResultFrameRealtimeMessagesUpdateFramecsXeZdZdZdfdd ZddZededdfd d Zd d Z d dZ ddZ Z S)OpenAIRealtimeLLMContextaOpenAI Realtime LLM context with session management and message conversion. Extends the standard OpenAI LLM context to support real-time session properties, instruction management, and conversion between standard message formats and realtime conversation items. Nc s$tjd||d||dS)aInitialize the OpenAIRealtimeLLMContext. Args: messages: Initial conversation messages. Defaults to None. tools: Available function tools. Defaults to None. **kwargs: Additional arguments passed to parent OpenAILLMContext. )messagestoolsN)super__init__&_OpenAIRealtimeLLMContext__setup_local)selfrrkwargs __class__ra/home/ubuntu/.local/lib/python3.10/site-packages/pipecat/services/openai_realtime_beta/context.pyr*s z!OpenAIRealtimeLLMContext.__init__cCsd|_d|_d|_dS)NT)llm_needs_settings_updatellm_needs_initial_messages_session_instructionsrrrr __setup_local5sz&OpenAIRealtimeLLMContext.__setup_localobjreturncCs&t|trt|tst|_||S)zUpgrade a standard OpenAI LLM context to a realtime context. Args: obj: The OpenAILLMContext instance to upgrade. Returns: The upgraded OpenAIRealtimeLLMContext instance. ) isinstancer rrr)r$rrrupgrade_to_realtime<s z,OpenAIRealtimeLLMContext.upgrade_to_realtimecCs|ddkrK|d}t|dtr=d}|dD]!}|ddkr.|d|d7}qtd|dd |qtjdd tjd |d gd S|ddkrq|drq|dd}tjd|d|dd|dddStd|dS)zConvert a standard message format to a realtime conversation item. Args: message: The standard message dictionary to convert. Returns: A ConversationItem instance for the realtime API. roleusercontentrtypetext z+Unhandled content type in context message: z - message input_textr+r,r(r+r* assistant tool_callsr function_callidfunctionname arguments)r+call_idr7r8z1Unhandled message type in from_standard_message: N)getr&listrerrorrConversationItem ItemContent)rr.r*ctcrrrfrom_standard_messageNs0   z.OpenAIRealtimeLLMContext.from_standard_messagec Cs|jsgSt|j}|dddkr;d|_|d}|d}t|tr*||_n t|t r7|dd|_|s;gSt |dkrR|dddkrR| |dgSd }d }dd d d |t j|dd|gdgdgS)a@Get conversation items for initializing the realtime session history. Converts the context's messages to a format suitable for the realtime API, handling system instructions and conversation history packaging. Returns: List of conversation items for session initialization. rr(systemTr*r,rr)z This is a previously saved conversation. Please treat this conversation history as a starting point for the current conversation.aS This is the end of the previously saved conversation. Please continue the conversation from here. If the last message is a user instruction or question, act on that instruction or answer the question. If the last message is an assistant response, simple say that you are ready to continue the conversation.r.r/z )indentr0r1)rcopydeepcopyr:rpopr&strr!r;lenrAjoinjsondumps)rrrBr* intro_text trailing_textrrr%get_messages_for_initializing_historyqs6      z>OpenAIRealtimeLLMContext.get_messages_for_initializing_historycCs(dd|jdjdgd}||dS)zAdd a user content item as a standard message to the context. Args: item: The conversation item to add as a user message. r)r,rr0)r(r*N)r* transcript add_message)ritemr.rrr add_user_content_item_as_messagesz9OpenAIRealtimeLLMContext.add_user_content_item_as_message)NN) __name__ __module__ __qualname____doc__rr staticmethodr r'rArOrS __classcell__rrrrr"s #>rcs8eZdZdZejfdedeffdd ZddZZ S)#OpenAIRealtimeUserContextAggregatoraOUser context aggregator for OpenAI Realtime API. Handles user input frames and generates appropriate context updates for the realtime conversation, including message updates and tool settings. Args: context: The OpenAI realtime LLM context. **kwargs: Additional arguments passed to parent aggregator. frame directioncs\t||IdHt|tr|t|jdIdHt|tr,|||IdHdSdS)zProcess incoming frames and handle realtime-specific frame types. Args: frame: The frame to process. direction: The direction of frame flow in the pipeline. N)context)r process_framer&r push_framer_contextrrr[r\rrrr^s  z1OpenAIRealtimeUserContextAggregator.process_framecsdS)zPush user input aggregation. Currently ignores all user input coming into the pipeline as realtime audio input is handled directly by the service. Nrr"rrrpush_aggregations z4OpenAIRealtimeUserContextAggregator.push_aggregation) rTrUrVrWr DOWNSTREAMrr^rbrYrrrrrZs rZcs<eZdZdZdedeffdd Zdeffdd ZZ S)(OpenAIRealtimeAssistantContextAggregatoraGAssistant context aggregator for OpenAI Realtime API. Handles assistant output frames from the realtime service, filtering out duplicate text frames and managing function call results. Args: context: The OpenAI realtime LLM context. **kwargs: Additional arguments passed to parent aggregator. r[r\cs.t|tttfst||IdHdSdS)zProcess assistant frames, filtering out duplicate text content. Args: frame: The frame to process. direction: The direction of frame flow in the pipeline. N)r&rr rrr^rarrrr^sz6OpenAIRealtimeAssistantContextAggregator.process_framecs2t|IdH|t|dtjIdHdS)zHandle function call result and notify the realtime service. Args: frame: The function call result frame to handle. N) result_frame)rhandle_function_call_resultr_rr UPSTREAM)rr[rrrrfs  zDOpenAIRealtimeAssistantContextAggregator.handle_function_call_result) rTrUrVrWrr r^rrfrYrrrrrds rd)rWrErKlogururpipecat.frames.framesrrrrrrr 1pipecat.processors.aggregators.openai_llm_contextr "pipecat.processors.frame_processorr pipecat.services.openai.llmr r rrframesrrrrZrdrrrrs $  ,