o i&@sdZddlZddlZddlmZddlmZmZmZm Z m Z ddl m Z ddl mZddlmZddlmZmZdd lmZmZdd lmZGd d d e ZGd ddeZdS)z(OpenAI Realtime LLM adapter for Pipecat.N) dataclass)AnyDictListOptional TypedDict)logger)BaseLLMAdapter)FunctionSchema) AdapterType ToolsSchema) LLMContextLLMContextMessage)eventsc@s@eZdZUdZeeed<eej ed<ee ee fed<dS)!OpenAIRealtimeLLMInvocationParamszContext-based parameters for invoking OpenAI Realtime API. This is a placeholder until support for universal LLMContext machinery is added for OpenAI Realtime. system_instructionmessagestoolsN) __name__ __module__ __qualname____doc__rstr__annotations__rrConversationItemrrrrf/home/ubuntu/.local/lib/python3.10/site-packages/pipecat/adapters/services/open_ai_realtime_adapter.pyrs  rc@seZdZdZedefddZdedefddZ de e ee ffdd Z eGd d d Zd e edefd dZdedejfddZedede ee ffddZdede e ee ffddZdS)OpenAIRealtimeLLMAdapterzLLM adapter for OpenAI Realtime API function calling. Converts Pipecat's tool schemas into the specific format required by OpenAI's Realtime API for function calling capabilities. returncCsdS)zLGet the identifier used in LLMSpecificMessage instances for OpenAI Realtime.zopenai-realtimerselfrrrid_for_llm_specific_messages)sz5OpenAIRealtimeLLMAdapter.id_for_llm_specific_messagescontextcCs,|||}|j|j||jpgdS)a{Get OpenAI Realtime-specific LLM invocation parameters from a universal LLM context. This is a placeholder until support for universal LLMContext machinery is added for OpenAI Realtime. Args: context: The LLM context containing messages, tools, etc. Returns: Dictionary of parameters for invoking OpenAI Realtime's API. )rrr) _from_universal_context_messages get_messagesrrfrom_standard_toolsr)r r"rrrrget_llm_invocation_params.s  z2OpenAIRealtimeLLMAdapter.get_llm_invocation_paramscCsg}||D]N}t|}d|vrAt|dtrA|dD]#}|ddkr4|dddr4d|dd<|ddkr@d|dd <qd |vrP|d d rPd|d <||q|S) aGet messages from a universal LLM context in a format ready for logging about OpenAI Realtime. Removes or truncates sensitive data like image content for safe logging. This is a placeholder until support for universal LLMContext machinery is added for OpenAI Realtime. Args: context: The LLM context containing messages. Returns: List of messages in a format ready for logging about OpenAI Realtime. contenttype image_urlurlz data:image/zdata:image/... input_audioz...data mime_typezimage/)r$copydeepcopy isinstancelist startswithappend)r r"msgsmessagemsgitemrrrget_messages_for_loggingAs        z1OpenAIRealtimeLLMAdapter.get_messages_for_loggingc@s0eZdZUdZeejed<dZe e ed<dS)z*OpenAIRealtimeLLMAdapter.ConvertedMessageszIContainer for OpenAI-formatted messages converted from universal context.rNr) rrrrrrrrrrrrrrrConvertedMessagesbs r9universal_context_messagesc Cs|s|jgdSt|}d}|dddkr?|d}|d}t|tr*|}n t|tr6|dd}|s?|jg|dSt|dkr[|ddd kr[|j| |dg|dSd }d }|jd d d d |t j |dd|gdgdg|dS)N)rrrolesystemr'text)rruserz This is a previously saved conversation. Please treat this conversation history as a starting point for the current conversation.aS This is the end of the previously saved conversation. Please continue the conversation from here. If the last message is a user instruction or question, act on that instruction or answer the question. If the last message is an assistant response, simple say that you are ready to continue the conversation.r5 input_textz )indentr(r=r;r(r') r9r.r/getpopr0rr1len_from_universal_context_messagejoinjsondumps)r r:rrr<r' intro_text trailing_textrrrr#isB      z9OpenAIRealtimeLLMAdapter._from_universal_context_messagesr5cCs|ddkrK|d}t|dtr=d}|dD]!}|ddkr.|d|d7}qtd|dd |qtjdd tjd |d gd S|ddkrq|drq|dd}tjd|d|dd|dddStd|dS)Nr;r?r'r(r= z+Unhandled content type in context message: z - r5r@rCrD assistant tool_callsr function_callidfunctionname arguments)r(call_idrUrVz;Unhandled message type in _from_universal_context_message: )rEr0r1rerrorrr ItemContent)r r5r'ctcrrrrHs0   z8OpenAIRealtimeLLMAdapter._from_universal_context_messagerTcCsd|j|jd|j|jddS)zConvert a function schema to OpenAI Realtime format. Args: function: The function schema to convert. Returns: Dictionary in OpenAI Realtime function format. rTobject)r( propertiesrequired)r(rU description parameters)rUr_r]r^)rTrrr#_to_openai_realtime_function_formats zs zEOpenAIRealtimeLLMAdapter.to_provider_tools_format..)standard_tools custom_toolsrEr SHIM)r rbfunctions_schemarf shimmed_toolsrrrto_provider_tools_formats  z1OpenAIRealtimeLLMAdapter.to_provider_tools_formatN)rrrrpropertyrr!r rr&rrrr8rr9rr#rrrH staticmethodr rar rkrrrrr"s*! ? "r)rr.rJ dataclassesrtypingrrrrrlogurur!pipecat.adapters.base_llm_adapterr (pipecat.adapters.schemas.function_schemar %pipecat.adapters.schemas.tools_schemar r *pipecat.processors.aggregators.llm_contextr r pipecat.services.openai.realtimerrrrrrrs