o iv@sNdZddlZddlZddlmZmZmZmZmZm Z ddl m Z m Z m Z Gddde ZGddde ZGd d d e ZGd d d e ZGd dde ZGddde ZGddde ZGddde ZGddde ZGddde ZGddde ZGdddeZGdddeZGdd d eZGd!d"d"eZGd#d$d$eZGd%d&d&eZGd'd(d(eZGd)d*d*eZ Gd+d,d,eZ!Gd-d.d.eZ"Gd/d0d0e Z#Gd1d2d2e#Z$Gd3d4d4e#Z%Gd5d6d6e#Z&Gd7d8d8e#Z'Gd9d:d:e#Z(Gd;d<dd>e#Z*Gd?d@d@e#Z+GdAdBdBe#Z,GdCdDdDe#Z-GdEdFdFe#Z.GdGdHdHe#Z/GdIdJdJe#Z0GdKdLdLe#Z1GdMdNdNe#Z2GdOdPdPe#Z3GdQdRdRe#Z4GdSdTdTe#Z5GdUdVdVe#Z6GdWdXdXe#Z7GdYdZdZe#Z8Gd[d\d\e#Z9Gd]d^d^e#Z:Gd_d`d`e#Z;Gdadbdbe#ZGdgdhdhe#Z?Gdidjdje#Z@Gdkdldle#ZAGdmdndne ZBGdodpdpe ZCGdqdrdre ZDidse@dte$due%dve&dwe>dxe?dye<dze=d{e'd|e(d}e)d~e*de+de,de-de.de/e0e1e2e3e4e5e6e7e8e9e:e;eAd ZEddZFdS)zGEvent models and data structures for OpenAI Realtime API communication.N)AnyDictListLiteralOptionalUnion) BaseModel ConfigDictFieldcsheZdZUdZdZeed<eeed<eeed<   d deedeedeeffdd ZZ S) InputAudioTranscriptionz/Configuration for audio transcription settings.gpt-4o-transcribemodellanguagepromptNcstj|||ddS)a Initialize InputAudioTranscription. Args: model: Transcription model to use (e.g., "gpt-4o-transcribe", "whisper-1"). language: Optional language code for transcription. prompt: Optional transcription hint text. )r rrN)super__init__)selfr rr __class__`/home/ubuntu/.local/lib/python3.10/site-packages/pipecat/services/openai_realtime_beta/events.pyrs z InputAudioTranscription.__init__)r NN) __name__ __module__ __qualname____doc__r str__annotations__rr __classcell__rrrrr s    r c@sVeZdZUdZdZeeded<dZee ed<dZ ee ed<dZ ee ed <d S) TurnDetectionaServer-side voice activity detection configuration. Parameters: type: Detection type, must be "server_vad". threshold: Voice activity detection threshold (0.0-1.0). Defaults to 0.5. prefix_padding_ms: Padding before speech starts in milliseconds. Defaults to 300. silence_duration_ms: Silence duration to detect speech end in milliseconds. Defaults to 800. server_vadtypeg? thresholdi,prefix_padding_msi silence_duration_msN) rrrrr rrrr!floatr"intr#rrrrr+s  rc@sZeZdZUdZdZeeded<dZeeded<dZ ee ed<dZ ee ed<dS) SemanticTurnDetectionaSemantic-based turn detection configuration. Parameters: type: Detection type, must be "semantic_vad". eagerness: Turn detection eagerness level. Can be "low", "medium", "high", or "auto". create_response: Whether to automatically create responses on turn detection. interrupt_response: Whether to interrupt ongoing responses on turn detection. semantic_vadr N)lowmediumhighauto eagernesscreate_responseinterrupt_response) rrrrr rrrr,r-boolr.rrrrr&;s  r&c@s"eZdZUdZeeded<dS)InputAudioNoiseReductionzInput audio noise reduction configuration. Parameters: type: Noise reduction type for different microphone scenarios. ) near_field far_fieldr N)rrrrrrrrrrrr0K r0c@seZdZUdZdZeeeded<dZ ee ed<dZ ee ed<dZ eeded<dZ eeded <dZeeed <dZeeed <edd Zeeeeefed <dZeeeed<dZeeded<dZeeed<dZeeeedfed<dS)SessionPropertiesaConfiguration properties for an OpenAI Realtime session. Parameters: modalities: Communication modalities to enable (text, audio, or both). instructions: System instructions for the assistant. voice: Voice ID for text-to-speech output. input_audio_format: Format for input audio data. output_audio_format: Format for output audio data. input_audio_transcription: Configuration for input audio transcription. input_audio_noise_reduction: Configuration for input audio noise reduction. turn_detection: Turn detection configuration or False to disable. tools: Available function tools for the assistant. tool_choice: Tool usage strategy ("auto", "none", or "required"). temperature: Sampling temperature for response generation. max_response_output_tokens: Maximum tokens in response or "inf" for unlimited. Ntextaudio modalities instructionsvoicepcm16 g711_ulaw g711_alawinput_audio_formatoutput_audio_formatinput_audio_transcriptioninput_audio_noise_reduction)defaultturn_detectiontoolsr+nonerequired tool_choice temperatureinfmax_response_output_tokens)rrrrr8rrrrr9rr:r?r@rAr rBr0r rDrrr&r/rErrIrJr$rLr%rrrrr4Us  r4c@sNeZdZUdZeded<dZeeed<dZ eeed<dZ eeed<dS) ItemContenta Content within a conversation item. Parameters: type: Content type (text, audio, input_text, or input_audio). text: Text content for text-based items. audio: Base64-encoded audio data for audio items. transcript: Transcribed text for audio items. )r6r7 input_text input_audior Nr6r7 transcript) rrrrrrr6rrr7rPrrrrrM}s  rMc@seZdZUdZedddZeed<dZe e ded<e d ed <dZ e e d ed <dZ e e d ed<dZ e eeed<dZe eed<dZe eed<dZe eed<dZe eed<dS)ConversationItemaA conversation item in the realtime session. Parameters: id: Unique identifier for the item, auto-generated if not provided. object: Object type identifier for the realtime API. type: Item type (message, function_call, or function_call_output). status: Current status of the item. role: Speaker role for message items (user, assistant, or system). content: Content list for message items. call_id: Function call identifier for function_call items. name: Function name for function_call items. arguments: Function arguments as JSON string for function_call items. output: Function output as JSON string for function_call_output items. cCsttjSN)ruuiduuid4hexrrrrszConversationItem.default_factoryidNz realtime.itemobject)message function_callfunction_call_outputr ) completed in_progress incompletestatus)user assistantsystemrolecontentcall_idname argumentsoutput)rrrrr rYrrrZrrrarerfrrMrgrhrirjrrrrrQs  rQc@s&eZdZUdZeed<eded<dS)RealtimeConversationzA realtime conversation session. Parameters: id: Unique identifier for the conversation. object: Object type identifier, always "realtime.conversation". rYzrealtime.conversationrZN)rrrrrrrrrrrrks rkc@seZdZUdZddgZeeeded<dZ ee ed<dZ ee ed<dZ eed ed <e ed Zeeeed <dZeed ed<dZeeed<dZeeeedfed<dS)ResponsePropertiesaFProperties for configuring assistant responses. Parameters: modalities: Output modalities for the response. Defaults to ["audio", "text"]. instructions: Specific instructions for this response. voice: Voice ID for text-to-speech in this response. output_audio_format: Audio format for this response. tools: Available tools for this response. tool_choice: Tool usage strategy for this response. temperature: Sampling temperature for this response. max_response_output_tokens: Maximum tokens for this response. r7r6r5r8Nr9r:r;r@rWrErFrIrJrKrL)rrrrr8rrrrr9rr:r@r listrErrIrJr$rLrr%rrrrrls   rlc@sReZdZUdZeed<dZeeed<eed<dZeeed<dZ eeed<dS) RealtimeErrora>Error information from the realtime API. Parameters: type: Error type identifier. code: Specific error code. message: Human-readable error message. param: Parameter name that caused the error, if applicable. event_id: Event ID associated with the error, if applicable. r coder[Nparamevent_id) rrrrrrrprrqrrrrrrrns  rnc@s(eZdZUdZedddZeed<dS) ClientEventzBase class for client events sent to the realtime API. Parameters: event_id: Unique identifier for the event, auto-generated if not provided. cCs ttSrR)rrSrTrrrrrVs zClientEvent.rWrrN)rrrrr rrrrrrrrrss rscsHeZdZUdZdZeded<eed<dee e fffdd Z Z S)SessionUpdateEventzEvent to update session properties. Parameters: type: Event type, always "session.update". session: Updated session properties. zsession.updater sessionreturncs>tj|i|}d|dvr|dddurd|dd<|S)a`Serialize the event to a dictionary. Handles special serialization for turn_detection where False becomes null. Args: *args: Positional arguments passed to parent model_dump. **kwargs: Keyword arguments passed to parent model_dump. Returns: Dictionary representation of the event. rDruFN)r model_dump)rargskwargsdumprrrrws   zSessionUpdateEvent.model_dump) rrrrr rrr4rrrrwrrrrrrts "rtc@*eZdZUdZdZeded<eed<dS)InputAudioBufferAppendEventzEvent to append audio data to the input buffer. Parameters: type: Event type, always "input_audio_buffer.append". audio: Base64-encoded audio data to append. zinput_audio_buffer.appendr r7Nrrrrr rrrrrrrr|  r|c@"eZdZUdZdZeded<dS)InputAudioBufferCommitEventzEvent to commit the current input audio buffer. Parameters: type: Event type, always "input_audio_buffer.commit". zinput_audio_buffer.commitr Nrrrrr rrrrrrr r3rc@r)InputAudioBufferClearEventzyEvent to clear the input audio buffer. Parameters: type: Event type, always "input_audio_buffer.clear". zinput_audio_buffer.clearr Nrrrrrr*r3rc@s:eZdZUdZdZeded<dZee ed<e ed<dS)ConversationItemCreateEventzEvent to create a new conversation item. Parameters: type: Event type, always "conversation.item.create". previous_item_id: ID of the item to insert after, if any. item: The conversation item to create. zconversation.item.creater Nprevious_item_iditem) rrrrr rrrrrrQrrrrr4s  rc@s:eZdZUdZdZeded<eed<eed<eed<dS)ConversationItemTruncateEventaIEvent to truncate a conversation item's audio content. Parameters: type: Event type, always "conversation.item.truncate". item_id: ID of the item to truncate. content_index: Index of the content to truncate within the item. audio_end_ms: End time in milliseconds for the truncated audio. zconversation.item.truncater item_id content_index audio_end_msN) rrrrr rrrr%rrrrrBs   rc@r{)ConversationItemDeleteEventzEvent to delete a conversation item. Parameters: type: Event type, always "conversation.item.delete". item_id: ID of the item to delete. zconversation.item.deleter rNr}rrrrrRr~rc@r{)ConversationItemRetrieveEventzEvent to retrieve a conversation item by ID. Parameters: type: Event type, always "conversation.item.retrieve". item_id: ID of the item to retrieve. zconversation.item.retriever rNr}rrrrr^r~rc@s2eZdZUdZdZeded<dZee ed<dS)ResponseCreateEventzEvent to create a new assistant response. Parameters: type: Event type, always "response.create". response: Optional response configuration properties. zresponse.creater Nresponse) rrrrr rrrrrlrrrrrjs rc@r)ResponseCancelEventzyEvent to cancel the current assistant response. Parameters: type: Event type, always "response.cancel". zresponse.cancelr Nrrrrrrvr3rc@s,eZdZUdZeddZeed<eed<dS) ServerEventzBase class for server events received from the realtime API. Parameters: event_id: Unique identifier for the event. type: Type of the server event. T)arbitrary_types_allowedrrr N)rrrrr model_configrrrrrrrs   rc@&eZdZUdZeded<eed<dS)SessionCreatedEventzEvent indicating a session has been created. Parameters: type: Event type, always "session.created". session: The created session properties. session.createdr ruNrrrrrrr4rrrrr   rc@r)SessionUpdatedEventzEvent indicating a session has been updated. Parameters: type: Event type, always "session.updated". session: The updated session properties. session.updatedr ruNrrrrrrrrc@r)ConversationCreatedzEvent indicating a conversation has been created. Parameters: type: Event type, always "conversation.created". conversation: The created conversation. conversation.createdr conversationN)rrrrrrrkrrrrrrrc@s6eZdZUdZeded<dZeeed<e ed<dS)ConversationItemCreatedzEvent indicating a conversation item has been created. Parameters: type: Event type, always "conversation.item.created". previous_item_id: ID of the previous item, if any. item: The created conversation item. conversation.item.createdr Nrr) rrrrrrrrrrQrrrrr   rc@6eZdZUdZeded<eed<eed<eed<dS),ConversationItemInputAudioTranscriptionDeltaaOEvent containing incremental input audio transcription. Parameters: type: Event type, always "conversation.item.input_audio_transcription.delta". item_id: ID of the conversation item being transcribed. content_index: Index of the content within the item. delta: Incremental transcription text. 1conversation.item.input_audio_transcription.deltar rrdeltaNrrrrrrrr%rrrrr   rc@r)0ConversationItemInputAudioTranscriptionCompletedaXEvent indicating input audio transcription is complete. Parameters: type: Event type, always "conversation.item.input_audio_transcription.completed". item_id: ID of the conversation item that was transcribed. content_index: Index of the content within the item. transcript: Complete transcription text. 5conversation.item.input_audio_transcription.completedr rrrPNrrrrrrrrc@6eZdZUdZeded<eed<eed<eed<dS)-ConversationItemInputAudioTranscriptionFaileda`Event indicating input audio transcription failed. Parameters: type: Event type, always "conversation.item.input_audio_transcription.failed". item_id: ID of the conversation item that failed transcription. content_index: Index of the content within the item. error: Error details for the transcription failure. 2conversation.item.input_audio_transcription.failedr rrerrorN) rrrrrrrr%rnrrrrrrrc@s6eZdZUdZeded<eed<eed<eed<dS)ConversationItemTruncatedaKEvent indicating a conversation item has been truncated. Parameters: type: Event type, always "conversation.item.truncated". item_id: ID of the truncated conversation item. content_index: Index of the content within the item. audio_end_ms: End time in milliseconds for the truncated audio. conversation.item.truncatedr rrrNrrrrrrrrc@r)ConversationItemDeletedzEvent indicating a conversation item has been deleted. Parameters: type: Event type, always "conversation.item.deleted". item_id: ID of the deleted conversation item. conversation.item.deletedr rN)rrrrrrrrrrrrrrc@r)ConversationItemRetrievedzEvent containing a retrieved conversation item. Parameters: type: Event type, always "conversation.item.retrieved". item: The retrieved conversation item. conversation.item.retrievedr rN)rrrrrrrQrrrrrrrc@&eZdZUdZeded<ded<dS)ResponseCreatedzEvent indicating an assistant response has been created. Parameters: type: Event type, always "response.created". response: The created response object. response.createdr ResponserNrrrrrrrrrrrrrc@r) ResponseDonezEvent indicating an assistant response is complete. Parameters: type: Event type, always "response.done". response: The completed response object. response.doner rrNrrrrrr)rrc@r)ResponseOutputItemAddedaEvent indicating an output item has been added to a response. Parameters: type: Event type, always "response.output_item.added". response_id: ID of the response. output_index: Index of the output item. item: The added conversation item. response.output_item.addedr response_id output_indexrN rrrrrrrr%rQrrrrr5rrc@r)ResponseOutputItemDoneaEvent indicating an output item is complete. Parameters: type: Event type, always "response.output_item.done". response_id: ID of the response. output_index: Index of the output item. item: The completed conversation item. response.output_item.doner rrrNrrrrrrErrc@FeZdZUdZeded<eed<eed<eed<eed<eed<d S) ResponseContentPartAddedarEvent indicating a content part has been added to a response. Parameters: type: Event type, always "response.content_part.added". response_id: ID of the response. item_id: ID of the conversation item. output_index: Index of the output item. content_index: Index of the content part. part: The added content part. response.content_part.addedr rrrrpartN rrrrrrrr%rMrrrrrU   rc@r) ResponseContentPartDoneadEvent indicating a content part is complete. Parameters: type: Event type, always "response.content_part.done". response_id: ID of the response. item_id: ID of the conversation item. output_index: Index of the output item. content_index: Index of the content part. part: The completed content part. response.content_part.doner rrrrrNrrrrrrirrc@FeZdZUdZeded<eed<eed<eed<eed<eed<d S) ResponseTextDeltaabEvent containing incremental text from a response. Parameters: type: Event type, always "response.text.delta". response_id: ID of the response. item_id: ID of the conversation item. output_index: Index of the output item. content_index: Index of the content part. delta: Incremental text content. response.text.deltar rrrrrNrrrrrr}rrc@r) ResponseTextDoneaUEvent indicating text content is complete. Parameters: type: Event type, always "response.text.done". response_id: ID of the response. item_id: ID of the conversation item. output_index: Index of the output item. content_index: Index of the content part. text: Complete text content. response.text.doner rrrrr6Nrrrrrrrrc@r) ResponseAudioTranscriptDeltaa}Event containing incremental audio transcript from a response. Parameters: type: Event type, always "response.audio_transcript.delta". response_id: ID of the response. item_id: ID of the conversation item. output_index: Index of the output item. content_index: Index of the content part. delta: Incremental transcript text. response.audio_transcript.deltar rrrrrNrrrrrrrrc@r) ResponseAudioTranscriptDoneanEvent indicating audio transcript is complete. Parameters: type: Event type, always "response.audio_transcript.done". response_id: ID of the response. item_id: ID of the conversation item. output_index: Index of the output item. content_index: Index of the content part. transcript: Complete transcript text. response.audio_transcript.doner rrrrrPNrrrrrrrrc@r) ResponseAudioDeltaavEvent containing incremental audio data from a response. Parameters: type: Event type, always "response.audio.delta". response_id: ID of the response. item_id: ID of the conversation item. output_index: Index of the output item. content_index: Index of the content part. delta: Base64-encoded incremental audio data. response.audio.deltar rrrrrNrrrrrrrrc@s>eZdZUdZeded<eed<eed<eed<eed<dS) ResponseAudioDonea2Event indicating audio content is complete. Parameters: type: Event type, always "response.audio.done". response_id: ID of the response. item_id: ID of the conversation item. output_index: Index of the output item. content_index: Index of the content part. response.audio.doner rrrrNrrrrrrs   rc@FeZdZUdZeded<eed<eed<eed<eed<eed<d S) "ResponseFunctionCallArgumentsDeltaa~Event containing incremental function call arguments. Parameters: type: Event type, always "response.function_call_arguments.delta". response_id: ID of the response. item_id: ID of the conversation item. output_index: Index of the output item. call_id: ID of the function call. delta: Incremental function arguments as JSON. &response.function_call_arguments.deltar rrrrgrNrrrrrrrrc@r) !ResponseFunctionCallArgumentsDoneaEvent indicating function call arguments are complete. Parameters: type: Event type, always "response.function_call_arguments.done". response_id: ID of the response. item_id: ID of the conversation item. output_index: Index of the output item. call_id: ID of the function call. arguments: Complete function arguments as JSON string. %response.function_call_arguments.doner rrrrgriNrrrrrrrrc@.eZdZUdZeded<eed<eed<dS)InputAudioBufferSpeechStartedaEvent indicating speech has started in the input audio buffer. Parameters: type: Event type, always "input_audio_buffer.speech_started". audio_start_ms: Start time of speech in milliseconds. item_id: ID of the associated conversation item. !input_audio_buffer.speech_startedr audio_start_msrNrrrrrrr%rrrrrr   rc@r)InputAudioBufferSpeechStoppeda Event indicating speech has stopped in the input audio buffer. Parameters: type: Event type, always "input_audio_buffer.speech_stopped". audio_end_ms: End time of speech in milliseconds. item_id: ID of the associated conversation item. !input_audio_buffer.speech_stoppedr rrNrrrrrr)rrc@s6eZdZUdZeded<dZeeed<eed<dS)InputAudioBufferCommittedaEvent indicating the input audio buffer has been committed. Parameters: type: Event type, always "input_audio_buffer.committed". previous_item_id: ID of the previous item, if any. item_id: ID of the committed conversation item. input_audio_buffer.committedr Nrr) rrrrrrrrrrrrrr7rrc@seZdZUdZeded<dS)InputAudioBufferClearedzEvent indicating the input audio buffer has been cleared. Parameters: type: Event type, always "input_audio_buffer.cleared". input_audio_buffer.clearedr NrrrrrrEs rc@s&eZdZUdZeded<eed<dS) ErrorEventzEvent indicating an error occurred. Parameters: type: Event type, always "error". error: Error details. rr N)rrrrrrrnrrrrrOrrc@s2eZdZUdZeded<eeee fed<dS)RateLimitsUpdatedzEvent indicating rate limits have been updated. Parameters: type: Event type, always "rate_limits.updated". rate_limits: List of rate limit information. rate_limits.updatedr rate_limitsN) rrrrrrrrrrrrrrr[s  rc@sLeZdZUdZeddZdZeee d<dZ eee d<dZ eee d<dS) TokenDetailszDetailed token usage information. Parameters: cached_tokens: Number of cached tokens used. Defaults to 0. text_tokens: Number of text tokens used. Defaults to 0. audio_tokens: Number of audio tokens used. Defaults to 0. allow)extrar cached_tokens text_tokens audio_tokensN) rrrrr rrrr%rrrrrrrrgs  rc@s:eZdZUdZeed<eed<eed<eed<eed<dS)UsageaeToken usage statistics for a response. Parameters: total_tokens: Total number of tokens used. input_tokens: Number of input tokens used. output_tokens: Number of output tokens used. input_token_details: Detailed breakdown of input token usage. output_token_details: Detailed breakdown of output token usage. total_tokens input_tokens output_tokensinput_token_detailsoutput_token_detailsN)rrrrr%rrrrrrrws   rc@sVeZdZUdZeed<eded<eded<eed<ee ed<d Z e e ed <d S) raxA complete assistant response. Parameters: id: Unique identifier for the response. object: Object type, always "realtime.response". status: Current status of the response. status_details: Additional status information. output: List of conversation items in the response. usage: Token usage statistics for the response. rYzrealtime.responserZ)r^r_r` cancelledfailedrastatus_detailsrjNusage) rrrrrrrrrrQrrrrrrrrs    rrrrrrrrrrrrrrrrrr) rrrrrrrrrrrrrc Csdzt|}|d}|tvrtd|t||WSty1}z t|d|d}~ww)a Parse a server event from JSON string. Args: str: JSON string containing the server event. Returns: Parsed server event object of the appropriate type. Raises: Exception: If the event type is unimplemented or parsing fails. r z!Unimplemented server event type: z N)jsonloads_server_event_types Exceptionmodel_validate)revent event_typeerrrparse_server_events r)GrrrStypingrrrrrrpydanticrr r r rr&r0r4rMrQrkrlrnrsrtr|rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrs  (  !                      "