o i+q @sdZddlZddlZddlZddlZddlmZmZddlm Z m Z m Z m Z ddl Z ddlmZddlmZddlmZmZmZmZmZmZmZmZmZddlmZmZm Z m!Z!m"Z"dd l#m$Z$m%Z%m&Z&dd l'm(Z(dd l)m*Z*dd l+m,Z,m-Z-dd l.m/Z/ddl0m1Z1zddl2Z2ddl3m4Z5ddl6m7Z7Wn e8yZ9ze:de9e:de;de9dZ9[9wwde,de e<fddZ=GdddZ>eGddde%Z?Gddde*Z@dS)zGladia Speech-to-Text (STT) service implementation. This module provides a Speech-to-Text service using Gladia's real-time WebSocket API, supporting multiple languages, custom vocabulary, and various audio processing options. N) dataclassfield)AnyAsyncGeneratorLiteralOptional)logger)version) CancelFrameEndFrameFrameInterimTranscriptionFrame StartFrameTranscriptionFrameTranslationFrameUserStartedSpeakingFrameUserStoppedSpeakingFrame)GladiaInputParamsLanguageConfigMessagesConfigPreProcessingConfigRealtimeProcessingConfig) NOT_GIVEN STTSettings _NotGiven)GLADIA_TTFS_P99)WebsocketSTTService)Languageresolve_language)time_now_iso8601) traced_stt)connect)Statez Exception: zEIn order to use Gladia, you need to `pip install pipecat-ai[gladia]`.zMissing module: languagereturncCs"itjdtjdtjdtjdtjdtjdtjdtjdtj d tj d tj d tj d tj d tjdtjdtjdtjditjdtjdtjdtjdtjdtjdtjdtjdtjdtjdtjdtjdtjdtjdtj d tj!d!tj"d"itj#d#tj$d$tj%d%tj&d&tj'd'tj(d(tj)d)tj*d*tj+d+tj,d,tj-d-tj.d.tj/d/tj0d0tj1d1tj2d2tj3d3itj4d4tj5d5tj6d6tj7d7tj8d8tj9d9tj:d:tj;d;tjd>tj?d?tj@d@tjAdAtjBdBtjCdCtjDdDitjEdEtjFdFtjGdGtjHdHtjIdItjJdJtjKdKtjLdLtjMdMtjNdNtjOdOtjPdPtjQdQtjRdRtjSdStjTdTtjUdUtjVdVtjWdWtjXdXtjYdYtjZdZtj[d[tj\d\tj]d]tj^d^tj_d_tj`d`tjadatjbdbtjcdci}td||dddeS)fzConvert a Language enum to Gladia's language code format. Args: language: The Language enum value to convert. Returns: The Gladia language code string or None if not supported. afamarasazbabebgbnbobrbscacscydadeeleneseteufafifofrglguhahawhehihrhthuhyidisitjajvkakkkmknkolalblnloltlvmgmimkmlmnmrmsmtmymrnenlnnnoocpaplpsptrorusasdsiskslsnsosqsrsusvswtatetgthtktltrttukuruzviyiyozhT) use_base_code)erAFAMARASAZBABEBGBNBOBRBSCACSCYDADEELENESETEUFAFIFOFRGLGUHAHAWHEHIHRHTHUHYIDISITJAJVKAKKKMKNKOLALBLNLOLTLVMGMIMKMLMNMRMSMTMY_MRNENLNNNOOCPAPLPSPTRORUSASDSISKSLSNSOSQSRSUSVSWTATETGTHTKTLTRTTUKURUZVIYIYOZHr)r# LANGUAGE_MAPrO/home/ubuntu/.local/lib/python3.10/site-packages/pipecat/services/gladia/stt.pylanguage_to_gladia_language;sv       !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVfrc@seZdZdZdddZdS)_InputParamsDescriptorz?Descriptor for backward compatibility with deprecation warning.NcCsHttdtjdtddWdtS1swYtS)NalwayszGladiaSTTService.InputParams is deprecated and will be removed in a future version. Import and use GladiaInputParams directly instead. stacklevel)warningscatch_warnings simplefilterwarnDeprecationWarningr)selfobjobjtyperrr__get__s   z_InputParamsDescriptor.__get__N)__name__ __module__ __qualname____doc__rrrrrrsrc@s eZdZUdZedddZedBeBed<edddZ e e e fdBeBed<ed ddZ edBeBed <ed ddZedBeBed <ed ddZedBeBed<edddZedBeBed<edddZedBeBed<edddZedBeBed<dS)GladiaSTTSettingsa[Settings for GladiaSTTService. Parameters: language_config: Language detection and handling configuration. custom_metadata: Additional metadata to include with requests. endpointing: Silence duration in seconds to mark end of speech. maximum_duration_without_endpointing: Maximum utterance duration without silence. pre_processing: Audio pre-processing options. realtime_processing: Real-time processing features. messages_config: WebSocket message filtering options. enable_vad: Enable VAD to trigger end of utterance detection. cCtSrrrrrrzGladiaSTTSettings.)default_factoryNlanguage_configcCrrrrrrrrrcustom_metadatacCrrrrrrrrr endpointingcCrrrrrrrrr$maximum_duration_without_endpointingcCrrrrrrrrrpre_processingcCrrrrrrrrrrealtime_processingcCrrrrrrrrrmessages_configcCrrrrrrrrr enable_vad)rrrrrr rr__annotations__r dictstrrr floatr intr rrrrrrboolrrrrrs  &"rcseZdZUdZeZeed<eZddddddddddd de d d e d e d dBde de de de de ede e de e de ede dede ede effddZddZdefdd Zd!ede e fd"d#Zdee effd$d%Zd&effd'd( Zd)edee efffd*d+ Zd&effd,d- Zd&effd.d/ Zd0e de!e"dffd1d2Z#fd3d4Z$fd5d6Z%d7d8Z&d9d:Z'dee effd;d<Z(e) dRd=e d>ed!e e fd?d@Z*dAdBZ+dCdDZ,d0e fdEdFZ-dGdHZ.dIdJZ/dKdLZ0dMdNZ1dOe fdPdQZ2Z3S)SGladiaSTTServiceaSpeech-to-Text service using Gladia's API. This service connects to Gladia's WebSocket API for real-time transcription with support for multiple languages, custom vocabulary, and various processing options. Provides automatic reconnection, audio buffering, and comprehensive error handling. For complete API documentation, see: https://docs.gladia.io/api-reference/v2/live/init .. deprecated:: 0.0.62 Use :class:`~pipecat.services.gladia.config.GladiaInputParams` directly instead. _settingsNzhttps://api.gladia.io/v2/livewav/pcmi@T) regionurlencoding bit_depthchannels confidence sample_ratemodelparamsmax_buffer_sizeshould_interruptsettingsttfs_p99_latencyapi_keyr)zus-westzeu-westrrrr r!r"r#r$r%r&r'r(c s|r#ttdtjdtddWdn1swY|jddddddddddd }| dur?|d d | |_| dur|d | jdurnttdtjd tddWdn1siwY| s| j durx| j }| j dur| j }| j dur| j }| j |_ | j |_ | j|_| j|_| j|_| j|_| j|_| jr| j|_n| jr|| j}|rt|gdd |_| dur|| tjd||dd|d|||_||_||_d|_||_||_||_d|_ d|_!d|_"t#|_$d|_%| |_&t'(|_)d|_*| |_+dS)aInitialize the Gladia STT service. Args: api_key: Gladia API key for authentication. region: Region used to process audio. eu-west or us-west. Defaults to eu-west. url: Gladia API URL. Defaults to "https://api.gladia.io/v2/live". encoding: Audio encoding format. Defaults to ``"wav/pcm"``. bit_depth: Audio bit depth. Defaults to 16. channels: Number of audio channels. Defaults to 1. confidence: Minimum confidence threshold for transcriptions (0.0-1.0). .. deprecated:: 0.0.86 The 'confidence' parameter is deprecated and will be removed in a future version. No confidence threshold is applied. sample_rate: Audio sample rate in Hz. If None, uses service default. model: Model to use for transcription. .. deprecated:: 0.0.105 Use ``settings=GladiaSTTService.Settings(model=...)`` instead. params: Additional configuration parameters for Gladia service. .. deprecated:: 0.0.105 Use ``settings=GladiaSTTService.Settings(...)`` for runtime-updatable fields and direct init parameters for encoding/bit_depth/channels. max_buffer_size: Maximum size of audio buffer in bytes. Defaults to 20MB. should_interrupt: Determine whether the bot should be interrupted when Gladia VAD detects user speech. Defaults to True. settings: Runtime-updatable settings. When provided alongside deprecated parameters, ``settings`` values take precedence. ttfs_p99_latency: P99 latency from speech end to final transcript in seconds. Override for your deployment. See https://github.com/pipecat-ai/stt-benchmark **kwargs: Additional arguments passed to the STTService parent class. rzuThe 'confidence' parameter is deprecated and will be removed in a future version. No confidence threshold is applied.rrNz solaria-1F) r#r#r r r r r rrrr#r$znThe 'language' parameter is deprecated and will be removed in a future version. Use 'language_config' instead.) languagescode_switching)r"r(keepalive_timeoutkeepalive_intervalr'rr),rrrrrSettings"_warn_init_param_moved_to_settingsr#r#rrr r r r r rrrr language_to_service_languager apply_updatesuper__init___api_key_region_url _receive_task _encoding _bit_depth _channels _session_url _session_id_connection_active bytearray _audio_buffer _bytes_sent_max_buffer_sizeasyncioLock _buffer_lock _is_speaking_should_interrupt)rr)rrrrr r!r"r#r$r%r&r'r(kwargsdefault_settings language_code __class__rrr5s7                 zGladiaSTTService.__init__cCs|jd|jdS)Nz [])namer>rrrr__str__szGladiaSTTService.__str__r$cCsdS)zCheck if the service can generate performance metrics. Returns: True, indicating this service supports metrics generation. TrrPrrrcan_generate_metricssz%GladiaSTTService.can_generate_metricsr#cCst|S)zConvert pipecat Language enum to Gladia's language code. Args: language: The Language enum value to convert. Returns: The Gladia language code string or None if not supported. )r)rr#rrrr2s z-GladiaSTTService.language_to_service_languagecCs|j}|jpd|jp d|j|jpd|jd}t|jpi|d<t|dd<|j dur0|j |d<|j dur:|j |d<|j rF|j j d d |d <|j rR|j j d d |d <|jr^|jj d d |d <|jrj|jj d d |d<|S)Nrrr)rrr"r r#r pipecatr r T) exclude_noner r rr)rr:r;r"r<r#rr pipecat_versionr r r  model_dumpr rr)rsr'rrr_prepare_settingss.    z"GladiaSTTService._prepare_settingsframec&t|IdH|IdHdS)zStart the Gladia STT websocket connection. Args: frame: The start frame triggering service startup. N)r4start_connectrrYrLrrr[zGladiaSTTService.startdeltacs*t|IdH}|s|S|||S)zApply settings delta. Settings are stored but not applied to the active session. Args: delta: A settings delta. Returns: Dict mapping changed field names to their previous values. N)r4_update_settings _warn_unhandled_updated_settings)rr_changedrLrrr`s   z!GladiaSTTService._update_settingscs4t|IdH|IdH|IdHdS)zStop the Gladia STT websocket connection. Args: frame: The end frame triggering service shutdown. N)r4stop_send_stop_recording _disconnectr]rLrrrcszGladiaSTTService.stopcrZ)zCancel the Gladia STT websocket connection. Args: frame: The cancel frame triggering service cancellation. N)r4cancelrer]rLrrrfr^zGladiaSTTService.cancelaudioc Cs*|IdH|j4IdH=|j|t|j|jkrBt|j|j}|j|d|_td|j||_t |d|dWdIdHn 1IdHsRwY|j r|j r|j j t jurz ||IdHWn!tjjy}zt |d|d|_ WYd}~nd}~wwdVdS)zRun speech-to-text on audio data. Args: audio: Raw audio bytes to transcribe. Yields: None (processing is handled asynchronously via WebSocket). Nrz) Audio buffer exceeded max size, trimmed z bytesz- Websocket closed while sending audio chunk: F)start_processing_metricsrFrAextendlenrCmaxrBrwarningr? _websocketstater"OPEN _send_audio websockets exceptionsConnectionClosed)rrg trim_sizeerrrrun_stts(  (  zGladiaSTTService.run_sttcs|js%|}||IdH}|d|_|d|_t|d|j|IdHtIdH|j rF|j sH| | |j |_ dSdSdS)zxConnect to the Gladia service. Initializes the session if needed and establishes websocket connection. NrrIz Session URL: )r=rX _setup_gladiar>rinfo_connect_websocketr4r\rmr9 create_task_receive_task_handler _report_error)rr'responserLrrr\%s   zGladiaSTTService._connectcsHtIdHd|_|jr||jIdHd|_|IdHdS)zfDisconnect from the Gladia service. Cleans up tasks and closes websocket connection. NF)r4rer?r9 cancel_task_disconnect_websocketrPrLrrre9szGladiaSTTService._disconnectc sz[|jr|jjtjurWdSt|dt|jIdH|_d|_|j 4IdHd|_ WdIdHn 1IdHs>wY| dIdH| IdHt|dWdSt yv}z|jd||dIdHd}~ww) z-Establish the websocket connection to Gladia.NzConnecting to Gladia WebSocketTr on_connectedz Connected to Gladia WebSocketzUnable to connect to Gladia:  error_msg exception)rmrnr"rordebugwebsocket_connectr=r?rFrB_call_event_handler_send_buffered_audio Exception push_errorrrurrrryHs$(z#GladiaSTTService._connect_websocketc szXz|jr|jjtjurt|d|jIdHWnty=}z|jd||dIdHWYd}~nd}~wwWd|_| dIdHdSWd|_| dIdHdSd|_| dIdHw)z)Close the websocket connection to Gladia.z$ Disconnecting from Gladia WebSocketNzError closing websocket: ron_disconnected) rmrnr"rorrcloserrrrrrrras$&z&GladiaSTTService._disconnect_websocketc st4IdHv}i}|jr|j|d<|j|jd|ji||d4IdHD}|jrD|IdHWdIdHWdIdHS|IdH}t |d|j d|pX|j t |d|j d|1IdHsowYWdIdHdS1IdHswYdS)Nrz X-Gladia-Key)headersjsonr$z Gladia error: z: z& Failed to initialize Gladia session: z - )aiohttp ClientSessionr7postr8r6okrtextrerrorstatusreasonr)rr'sessionr$r} error_textrrrrwms2   .zGladiaSTTService._setup_gladia transcriptis_finalcs|IdHdSr)stop_processing_metrics)rrrr#rrr_handle_transcriptionsz&GladiaSTTService._handle_transcriptioncsV|jjr|jr dSt|dd|_|tIdH|jr)|IdHdSdS)zHandle speech start event from Gladia. Broadcasts UserStartedSpeakingFrame and optionally triggers interruption when VAD is enabled. Nz User started speakingT) rrrGrrbroadcast_framerrHbroadcast_interruptionrPrrr_on_speech_startedsz#GladiaSTTService._on_speech_startedcs>|jjr|js dSd|_|tIdHt|ddS)zoHandle speech end event from Gladia. Broadcasts UserStoppedSpeakingFrame when VAD is enabled. NFz User stopped speaking)rrrGrrrrrPrrr_on_speech_endeds z!GladiaSTTService._on_speech_endedcsX|jr(|jjtjur*t|d}dd|id}|jt |IdHdSdSdS)z,Send audio chunk with proper message format.zutf-8 audio_chunkchunk)typedataN) rmrnr"robase64 b64encodedecodesendrdumps)rrgrmessagerrrrps zGladiaSTTService._send_audioc s|j4IdH3|jr0t|dt|jd|t|jIdHWdIdHdSWdIdHdS1IdHsAwYdS)z+Send any buffered audio after reconnection.Nz Sending z bytes of buffered audio)rFrArrrjrpbytesrPrrrrs.z%GladiaSTTService._send_buffered_audiocs>|jr|jjtjur|jtddiIdHdSdSdS)Nrstop_recording)rmrnr"rorrrrPrrrrds z%GladiaSTTService._send_stop_recordingcCs|jr|jStd)zGet the current WebSocket connection. Returns: The WebSocket connection. Raises: Exception: If WebSocket is not connected. zWebsocket not connected)rmrrPrrr_get_websockets zGladiaSTTService._get_websocketc s |2z3dHW}zt|}|ddkrY|drY|dd}|j4IdH$|d}||jkrC||j}|j|d|_||_WdIdHn 1IdHsSwYn|ddkr|dd }|d }|d }|dd } | r|t||j t ||d IdH|j || |dIdHnZ|t ||j t ||d IdHnH|ddkr|dd} |dd} | d } | d } | | kr|t | dt | IdHn|ddkr|IdHn |ddkr|IdHWqtjyt|d|Yqw6dS)zxReceive and process websocket messages. Continuously processes messages from the websocket connection. Nrr acknowledgedr byte_rangerr utterancer#rr)result)rrr# translationtranslated_utteranceoriginal_language speech_start speech_endz Received non-JSON message: )rrloadsgetrFrBrA push_framer_user_idrrr rrrJSONDecodeErrorrrl)rrcontentrend_bytertrr#rrrrtranslated_languagerrrr_receive_messagess    (           z"GladiaSTTService._receive_messagessilencecs|dIdHdS)zSend an empty audio chunk to keep the Gladia connection alive. Args: silence: Silent PCM audio bytes (unused, Gladia accepts empty chunks). N)rp)rrrrr_send_keepalivesz GladiaSTTService._send_keepaliver)4rrrrrr0rr InputParamsrrrrrrrrr5rQrRrr2rrrXrr[r`r rcr rfrrr rvr\reryrrwr rrrrprrdrrr __classcell__rrrLrrs        " )        Dr)ArrDrrr dataclassesrrtypingrrrrrlogururrSr rUpipecat.frames.framesr r r r rrrrrpipecat.services.gladia.configrrrrrpipecat.services.settingsrrrpipecat.services.stt_latencyrpipecat.services.stt_servicerpipecat.transcriptions.languagerrpipecat.utils.timer(pipecat.utils.tracing.service_decoratorsr rqwebsockets.asyncio.clientr!rwebsockets.protocolr"ModuleNotFoundErrorrurrrrrrrrrrrsB  ,       s