o i#J @sdZddlZddlZddlZddlmZmZddlmZm Z m Z ddl m Z ddl mZddlmZmZmZmZmZmZmZmZddlmZdd lmZmZmZdd lmZdd l m!Z!dd l"m#Z#m$Z$dd l%m&Z&ddl'm(Z(zddl)m*Z+ddl,m-Z-Wn e.yZ/ze 0de/e 0de1de/dZ/[/wwdZ2de3de4de3fddZ5de#de e3fddZ6eGdddeZ7Gdd d e!Z8dS)!zGradium's speech-to-text service implementation. This module provides integration with Gradium's real-time speech-to-text WebSocket API for streaming audio transcription. N) dataclassfield)AnyAsyncGeneratorOptional)logger) BaseModel) CancelFrameEndFrameFrameInterimTranscriptionFrame StartFrameTranscriptionFrameVADUserStartedSpeakingFrameVADUserStoppedSpeakingFrame)FrameDirection) NOT_GIVEN STTSettings _NotGiven)GRADIUM_TTFS_P99)WebsocketSTTService)Languageresolve_language)time_now_iso8601) traced_stt)connect)Statez Exception: zIIn order to use Gradium, you need to `pip install "pipecat-ai[gradium]"`.zMissing module: g?encoding sample_ratereturncCsJ|dkr#|dkr dSdkrdSdkrdStd|d dS|S) aBuild Gradium input_format from encoding type and sample rate. For PCM encoding, appends the sample rate (e.g., "pcm_16000"). For other encodings (wav, opus), returns the encoding as-is. Args: encoding: Base encoding type ("pcm", "wav", or "opus"). sample_rate: Audio sample rate in Hz. Returns: The full input_format string for the Gradium API. pcmi@pcm_8000i> pcm_16000i] pcm_24000z+GradiumSTTService: unsupported sample rate z" for PCM encoding, using pcm_16000)rwarning)rrr%P/home/ubuntu/.local/lib/python3.10/site-packages/pipecat/services/gradium/stt.py_input_format_from_encoding5s    r'languagec Cs0tjdtjdtjdtjdtjdi}t||ddS)zConvert a Language enum to Gradium's language code format. Args: language: The Language enum value to convert. Returns: The Gradium language code string or None if not supported. deenesfrptT) use_base_code)rDEENESFRPTr)r( LANGUAGE_MAPr%r%r&language_to_gradium_languageQs r5c@s0eZdZUdZedddZeeeBe d<dS)GradiumSTTSettingsa^Settings for GradiumSTTService. Parameters: delay_in_frames: Delay in audio frames (80ms each) before text is generated. Higher delays allow more context but increase latency. Allowed values: 7, 8, 10, 12, 14, 16, 20, 24, 36, 48. Default is 10 (800ms). Lower values like 7-8 give faster response. cCstSN)rr%r%r%r&pszGradiumSTTSettings.)default_factorydelay_in_framesN) __name__ __module__ __qualname____doc__rr:rintr__annotations__r%r%r%r&r6es " r6cseZdZUdZeZeed<GdddeZdddddde dd e d e d e d e e d e ede e de ede e ffddZdefddZdedee efffdd Zdeffdd Zdeffdd Zdeffdd Zd d!Zded"effd#d$ Zd%d&Zd'e de!edffd(d)Z"e#d*e d+ed,e$fd-d.Z%fd/d0Z&d1d2Z'fd3d4Z(d5d6Z)d7d8Z*d9d:Z+d;e fdd?Z-d@dAZ.dBdCZ/Z0S)DGradiumSTTServicezGradium real-time speech-to-text service. Provides real-time speech transcription using Gradium's WebSocket API. Supports both interim and final transcriptions with configurable parameters for audio processing and connection management. _settingsc@s2eZdZUdZdZeeed<dZee ed<dS)zGradiumSTTService.InputParamsaConfiguration parameters for Gradium STT API. .. deprecated:: 0.0.105 Use ``settings=GradiumSTTService.Settings(...)`` instead. Parameters: language: Expected language of the audio (e.g., "en", "es", "fr"). This helps ground the model to a specific language and improve transcription quality. delay_in_frames: Delay in audio frames (80ms each) before text is generated. Higher delays allow more context but increase latency. Allowed values: 7, 8, 10, 12, 14, 16, 20, 24, 36, 48. Default is 10 (800ms). Lower values like 7-8 give faster response. Nr(r:) r;r<r=r>r(rrr@r:r?r%r%r%r& InputParams~s rCz&wss://eu.api.gradium.ai/api/speech/asrr N)api_endpoint_base_urlrrparams json_configsettingsttfs_p99_latencyapi_keyrDrrrErFrGrHc  s|durddl} | jdtdd|jdddd} |dur0|d|s0|j| _|jdur0|j| _|dur9| |tj d ||| d | ||_ ||_ ||_ d|_ ||_d|_d |_t|_d |_d|_g|_d|_d|_dS) aInitialize the Gradium STT service. Args: api_key: Gradium API key for authentication. api_endpoint_base_url: WebSocket endpoint URL. Defaults to Gradium's streaming endpoint. encoding: Base audio encoding type. One of "pcm", "wav", or "opus". For PCM, the sample rate is appended automatically from the pipeline's audio_in_sample_rate (e.g., "pcm" becomes "pcm_16000"). Defaults to "pcm". sample_rate: Audio sample rate in Hz. If None, uses the pipeline sample rate. params: Configuration parameters for language and delay settings. .. deprecated:: 0.0.105 Use ``settings=GradiumSTTService.Settings(...)`` instead. json_config: Optional JSON configuration string for additional model settings. .. deprecated:: 0.0.101 Use `params` instead for type-safe configuration. settings: Runtime-updatable settings. When provided alongside deprecated parameters, ``settings`` values take precedence. ttfs_p99_latency: P99 latency from speech end to final transcript in seconds. Override for your deployment. See https://github.com/pipecat-ai/stt-benchmark **kwargs: Additional arguments passed to parent STTService class. NrzdParameter 'json_config' is deprecated and will be removed in a future version, use 'params' instead.) stackleveldefault)modelr(r:rE)rrHrGPr%)warningswarnDeprecationWarningSettings"_warn_init_param_moved_to_settingsr(r: apply_updatesuper__init___api_key_api_endpoint_base_url _encoding _websocket _json_config _receive_task _input_format bytearray _audio_buffer_chunk_size_ms_chunk_size_bytes_accumulated_text_flush_counter_transcript_aggregation_task) selfrIrDrrrErFrGrHkwargsrPdefault_settings __class__r%r&rWsP(    zGradiumSTTService.__init__rcCsdS)zzCheck if the service can generate metrics. Returns: True if metrics generation is supported. Tr%rfr%r%r&can_generate_metricssz&GradiumSTTService.can_generate_metricsdeltacsBt|IdH}|s|S|jr|IdH|IdH|S)zApply a settings delta, sync params, and reconnect. Args: delta: A :class:`STTSettings` (or ``GradiumSTTService.Settings``) delta. Returns: Dict mapping changed field names to their previous values. N)rV_update_settingsr[ _disconnect_connect)rfrmchangedrir%r&rns z"GradiumSTTService._update_settingsframecsPt|IdHt|j|j|_t|j|jdd|_| IdHdS)zmStart the speech-to-text service. Args: frame: Start frame to begin processing. NrJi) rVstartr'rZrr^r?rarbrprfrrrir%r&rss zGradiumSTTService.startc&t|IdH|IdHdS)ziStop the speech-to-text service. Args: frame: End frame to stop processing. N)rVstoprortrir%r&rvzGradiumSTTService.stopcru)zoCancel the speech-to-text service. Args: frame: Cancel frame to abort processing. N)rVcancelrortrir%r&rx"rwzGradiumSTTService.cancelcs|IdHdS)zBStart performance metrics collection for transcription processing.N)start_processing_metricsrkr%r%r&_start_metrics+sz GradiumSTTService._start_metrics directioncsRt||IdHt|tr|IdHdSt|tr'|IdHdSdS)zProcess incoming frames and handle speech events. Args: frame: The frame to process. direction: Direction of frame flow in the pipeline. N)rV process_frame isinstancerrzr _send_flush)rfrrr{rir%r&r|/s  zGradiumSTTService.process_framec s|jr |jjtjur dS|jd7_t|j}d|d}z|jt|IdHWdSt yH}zt d|WYd}~dSd}~ww)aSend a flush request to process any buffered audio immediately. Sends a flush message to tell the server to process buffered audio. The server responds with text fragments followed by a "flushed" acknowledgment, which triggers finalization. Nflush)typeflush_idzFailed to send flush: ) r[staterOPENrdstrsendjsondumps Exceptionrr$)rfrmsger%r%r&r~=s  zGradiumSTTService._send_flushaudiocCs|j|t|j|jkrMt|jd|j}|j|jd|_t|d}d|d}|jrE|jj t j urE|j t |IdHt|j|jksdVdS)zProcess audio data for speech-to-text conversion. Args: audio: Raw audio bytes to process. Yields: None (processing handled via WebSocket messages). Nzutf-8r)rr)r`extendlenrbbytesbase64 b64encodedecoder[rrrrrr)rfrchunkrr%r%r&run_sttOs   zGradiumSTTService.run_stt transcriptis_finalr(csdS)z'Record transcription event for tracing.Nr%)rfrrr(r%r%r&_trace_transcriptiondsz&GradiumSTTService._trace_transcriptioncsLtIdH|IdH|jr"|js$|||j|_dSdSdSr7)rVrp_connect_websocketr[r] create_task_receive_task_handler _report_errorrkrir%r&rpis  zGradiumSTTService._connectc sxz|jr|jjtjurWdStd|j}|jdd}t||dIdH|_| dIdHd|j j |j d}i}|j rCt|j }|j jrSt|j j}|rS||d<|j jr]|j j|d <|rc||d <|jt|IdH|jIdH}t|}|d d krtd |d|d dkrtd|d tdWdSty}z|jd||dIdHd}~ww)NzConnecting to Gradium STTpipecat)z x-api-keyz x-api-source)additional_headers on_connectedsetup)r model_name input_formatr(r:rFrerrorzreceived error messagereadyzunexpected first message type zConnected to Gradium STTUnknown error occurred:  error_msg exception)r[rrrrdebugrYrXwebsocket_connect_call_event_handlerrBrMr^r\rloadsr(r5r:rrrecvr push_error)rfws_urlheaders setup_msgrFgradium_language ready_msgrr%r%r&rqsT       z$GradiumSTTService._connect_websocketcsptIdH|jr||jIdHd|_|jd|_|jr/||jIdHd|_|IdHdS)Nr) rVrore cancel_taskrcclearrdr]_disconnect_websocketrkrir%r&ros zGradiumSTTService._disconnectc szUz|jr|jjtjurtd|jIdHWnty:}z|jd||dIdHWYd}~nd}~wwWd|_| dIdHdSWd|_| dIdHdSd|_| dIdHw)NzDisconnecting from Gradium STTrron_disconnected) r[rrrrrcloserrr)rfrr%r%r&rs$ &z'GradiumSTTService._disconnect_websocketcCs|jr|jStd)NzWebsocket not connected)r[rrkr%r%r&_get_websocketsz GradiumSTTService._get_websocketc s|2z[3dHW}zt|}Wntjy$td|Yqw|dd}|dkr:||dIdHq|dkrF|IdHq|dkrPt dq|dkr`|j d |d IdHq6dS) NzReceived non-JSON message: rrNtextflushed end_of_streamz*Received end_of_stream message from serverrzError: )r) rrrJSONDecodeErrorrr$get _handle_text_handle_flushedrr)rfrrtype_r%r%r&_receive_messagess&  z#GradiumSTTService._receive_messagesrcsP|j|d|j}|t||jt|jjdIdH| IdHdS)zHandle streaming transcription fragment. Accumulates text and pushes an InterimTranscriptionFrame with the full accumulated text so far.  )ruser_id timestampr(N) rcappendjoin push_framer _user_idrrBr(stop_processing_metrics)rfr accumulatedr%r%r&rs   zGradiumSTTService._handle_textcs0|jr ||jIdH||d|_dS)a7Handle flush completion by starting a transcript aggregation timer. The "flushed" message confirms that buffered audio has been processed, but text tokens may still arrive after this point. A short timer allows trailing tokens to accumulate before finalizing the transcription. Ntranscript_aggregation)rerr_transcript_aggregation_handlerrkr%r%r&rs  z!GradiumSTTService._handle_flushedcs$ttIdH|IdHdS)zEWait for trailing tokens then finalize the accumulated transcription.N)asynciosleepTRANSCRIPT_AGGREGATION_DELAY_finalize_accumulated_textrkr%r%r&rsz1GradiumSTTService._transcript_aggregation_handlercsz|jsdSd|_d|j}|jtd|d|t||jt |j j IdH|j |d|j j dIdHdS)z@Join accumulated text, push TranscriptionFrame, and clear state.NrzFinal transcription: []T)rr() rcrerrrrrrrrrBr(r)rfrr%r%r&rs    z,GradiumSTTService._finalize_accumulated_text)1r;r<r=r>r6rSr@rrCrrrr?floatrWboolrlrdictrrnr rsr rvr rxrzr rr|r~rrrrrrrprrorrrrrrr __classcell__r%r%rir&rAssd   c     0  rA)9r>rrr dataclassesrrtypingrrrlogururpydanticrpipecat.frames.framesr r r r r rrr"pipecat.processors.frame_processorrpipecat.services.settingsrrrpipecat.services.stt_latencyrpipecat.services.stt_servicerpipecat.transcriptions.languagerrpipecat.utils.timer(pipecat.utils.tracing.service_decoratorsrwebsockets.asyncio.clientrrwebsockets.protocolrModuleNotFoundErrorrrrrrr?r'r5r6rAr%r%r%r&s>  (