o iF @sdZddlZddlZddlmZddlmZmZddlm Z ddl m Z m Z m Z mZmZmZmZmZddlmZddlmZdd lmZzdd lmZdd lmZWn eyrZze d ee d e!dedZ[wweGdddeZ"GdddeZ#dS)z3Resemble AI text-to-speech service implementations.N) dataclass)AsyncGeneratorOptional)logger) CancelFrameEndFrame ErrorFrameFrame StartFrameTTSAudioRawFrameTTSStartedFrameTTSStoppedFrame) TTSSettings)WebsocketTTSService) traced_tts)connect)Statez Exception: zNIn order to use Resemble AI, you need to `pip install pipecat-ai[resembleai]`.zMissing module: c@seZdZdZdS)ResembleAITTSSettingsz"Settings for ResembleAITTSService.N)__name__ __module__ __qualname____doc__rrS/home/ubuntu/.local/lib/python3.10/site-packages/pipecat/services/resembleai/tts.pyr'srcsReZdZUdZeZeed<dddddddd ed eed ed eed eedee deeffddZ de fddZ d7dedefddZ deffdd Zdeffdd Zdeffdd Zd d!Zd"d#Zd$d%Zd&d'Zd(d)Zd*efd+d,Zd*efd-d.Zd8d*eefd/d0Zd1d2Zd3d4Zeded*edee dffd5d6Z!Z"S)9ResembleAITTSServicea0Resemble AI TTS service with WebSocket streaming and word timestamps. Provides text-to-speech using Resemble AI's streaming WebSocket API. Supports word-level timestamps and audio context management for handling multiple simultaneous synthesis requests with proper interruption support. _settingsNz*wss://websocket.cluster.resemble.ai/streamPCM_16wavi"V)voice_idurl precision output_format sample_ratesettingsapi_keyrrr r!r"r#c  s|jdddd} |dur|dd|| _|dur| |tjd |d| d|||_||_|p4d|_|p9d|_ d |_ d|_ d |_ d|_ i|_i|_d |_d |_i|_dS) a/Initialize the Resemble AI TTS service. Args: api_key: Resemble AI API key for authentication. voice_id: Voice UUID to use for synthesis. .. deprecated:: 0.0.105 Use ``settings=ResembleAITTSService.Settings(voice=...)`` instead. url: WebSocket URL for Resemble AI TTS API. precision: PCM bit depth (PCM_32, PCM_24, PCM_16, or MULAW). output_format: Audio format (wav or mp3). sample_rate: Audio sample rate (8000, 16000, 22050, 32000, or 44100). Defaults to 22050. settings: Runtime-updatable settings. When provided alongside deprecated parameters, ``settings`` values take precedence. **kwargs: Additional arguments passed to the parent service. N)modelvoicelanguagerr&F)r"reuse_context_id_within_turnr#rrriiDr)Settings"_warn_init_param_moved_to_settingsr& apply_updatesuper__init___api_key_url _precision_output_format_resemble_sample_rate _websocket_request_id_counter _receive_task_request_id_to_context_audio_buffers_buffer_threshold_bytes_jitter_buffer_bytes_playback_started) selfr$rrr r!r"r#kwargsdefault_settings __class__rrr-9s<     zResembleAITTSService.__init__returncCsdS)zCheck if this service can generate processing metrics. Returns: True, as Resemble AI service supports metrics generation. Trr;rrrcan_generate_metricssz)ResembleAITTSService.can_generate_metricstextc Cs:|jj|d|j|j|j|jdd}|jd7_t|S)zBuild a JSON message for the Resemble AI WebSocket API. Args: text: The text or SSML to synthesize. Returns: JSON string containing the request payload. FT) voice_uuiddatabinary_response request_idr!r"r no_audio_header)rr&r4r1r2r0jsondumps)r;rDmsgrrr _build_msgs  zResembleAITTSService._build_msgframecs.t|IdH|j|_|IdHdS)zStart the Resemble AI TTS service. Args: frame: The start frame containing initialization parameters. N)r,startr"r2_connectr;rOr>rrrPszResembleAITTSService.startc&t|IdH|IdHdS)z[Stop the Resemble AI TTS service. Args: frame: The end frame. N)r,stop _disconnectrRr>rrrTzResembleAITTSService.stopcrS)z`Cancel the Resemble AI TTS service. Args: frame: The cancel frame. N)r,cancelrUrRr>rrrWrVzResembleAITTSService.cancelcs<|IdH|jr|js|||j|_dSdSdS)z%Connect to the Resemble AI WebSocket.N)_connect_websocketr3r5 create_task_receive_task_handler _report_errorrArrrrQs  zResembleAITTSService._connectcs2|jr||jIdHd|_|IdHdS)z*Disconnect from the Resemble AI WebSocket.N)r5 cancel_task_disconnect_websocketrArrrrUs z ResembleAITTSService._disconnectc sz0|jr|jjtjurWdStddd|ji}t|j|dIdH|_| dIdHWdSt y^}z!|j d||dIdHd|_| d |IdHWYd}~dSd}~ww) z.Establish WebSocket connection to Resemble AI.NzConnecting to Resemble AI TTS AuthorizationzBearer )additional_headers on_connectedUnknown error occurred:  error_msg exceptionon_connection_error) r3staterOPENrdebugr.websocket_connectr/_call_event_handler Exception push_error)r;headerserrrrXs "z'ResembleAITTSService._connect_websocketc s(zwz|IdH|jrtdd|j_|jIdHWnty>}z|jd||dIdHWYd}~n"d}~wwWd|_|j |j |j | dIdHdSWd|_|j |j |j | dIdHdSd|_|j |j |j | dIdHw)z*Close WebSocket connection to Resemble AI.NzDisconnecting from Resemble AIrrarbon_disconnected) stop_all_metricsr3rrh close_timeoutcloserkrlr7clearr:r6rjr;rnrrrr]s: &         z*ResembleAITTSService._disconnect_websocketcCs|jr|jStd)zGet the current WebSocket connection. Returns: The active WebSocket connection. Raises: Exception: If websocket is not connected. zWebsocket not connected)r3rkrArrr_get_websockets z#ResembleAITTSService._get_websocket context_idcs|IdHdS)z)Stop metrics when the bot is interrupted.N)rpr;rvrrron_audio_context_interruptedsz1ResembleAITTSService.on_audio_context_interruptedcsdS)aStop metrics after the Resemble AI context finishes playing. No close message is needed: Resemble AI signals completion with an ``audio_end`` message (handled in ``_process_messages``), after which the server-side context is already closed. Nrrwrrron_audio_context_completedsz/ResembleAITTSService.on_audio_context_completedcst|ddS)z9Flush any pending audio and finalize the current context.z: flushing audioN)rtracerwrrr flush_audiosz ResembleAITTSService.flush_audioc s |2z3dHW}zt|}Wntjy)|jd|dIdHYqw|s-q|d}|d}|j|t|}||sFq|dkr|dd}|sTqt |}t |d kr`q||j vrpt |j |<d |j|<|j |}|||j|d st ||jkrqd |j|<t ||jkr|j} | d d kr| d 8} t|d| } || d|j |<|j |}t | d krqt| |jd |d} ||| IdHt ||jks|di} | r| dg} | dg}g}t| |D]\}}|r t |d kr |d }|||fq|r|||IdHq|dkr|IdH|j |t }|rZt|}t |d d krE|dd}|rZt||jd |d} ||| IdH||j vrd|j |=||jvrx|j|=||jvrx|j|=|ddg|IdH||IdHq|dkr|dd}|dd}|dd }|jd|d|d|dIdH||j vr|j |=||jvr|j|=|t|dIdH|IdH|t|d |d!|d"IdH|d#vr|IdH| IdHqt!"|d$|q6dS)%z5Process incoming WebSocket messages from Resemble AI.NzReceived invalid JSON: )rctyperHaudio audio_contentrCrFTrJ)r}r" num_channelsrvaudio_timestamps graph_chars graph_times audio_end)r r)Resetrerror error_nameUnknownmessagez Unknown error status_codezError: z (status z): rvz error: z - r)iiz unknown message type: )#rurKloadsJSONDecodeErrorrlgetr6straudio_context_availablebase64 b64decodelenr7 bytearrayr:extendr9r8bytesr r"append_to_audio_contextzipappendadd_word_timestampsstop_ttfb_metricsremove_audio_context push_framer rprr]rXrwarning)r;rrMmsg_typerHrvr~ audio_bytesbuffer chunk_size chunk_to_sendrO timestampsrr word_timeschartimes start_time remainingrrcrrrr_process_messagess                              & z&ResembleAITTSService._process_messagesc sv z |IdHWn.ty9}z"|jd||dIdHt|d|IdHWYd}~nd}~wwq)z2Main loop for receiving messages from Resemble AI.TNzError in receive loop: rbz* Resemble AI connection lost, reconnecting)rrkrlrrhrXrtrrr_receive_messagessz&ResembleAITTSService._receive_messagesc CsXt|d|dz|jr|jjtjur|IdH||s8||IdH| IdHt |dV||j |j <|j |d}z||IdH||IdHWn1ty}z%td|dVt|dV|IdH|IdHWYd}~WdSd}~wwdVWdSty}ztd|dVWYd}~dSd}~ww)aGenerate speech from text using Resemble AI's streaming API. Args: text: The text to synthesize into speech. context_id: Unique identifier for this TTS context. Yields: Frame: Audio frames containing the synthesized speech. z: Generating TTS []Nr)rDrar)rrhr3rfrCLOSEDrQrcreate_audio_contextstart_ttfb_metricsr r6r4rNrusendstart_tts_usage_metricsrkrr rU)r;rDrvrMrnrrrrun_ttss6        zResembleAITTSService.run_tts)rC)N)#rrrrrr)__annotations__rrintr-boolrBrNr rPrrTrrWrQrUrXr]rurxryr{rrrrr r __classcell__rrr>rr.sV  Q      (r)$rrrK dataclassesrtypingrrlogururpipecat.frames.framesrrrr r r r r pipecat.services.settingsrpipecat.services.tts_servicer(pipecat.utils.tracing.service_decoratorsrwebsockets.asyncio.clientrriwebsockets.protocolrModuleNotFoundErrorrnrrkrrrrrrs,  (