o iB @sNdZddlmZmZddlmZmZmZmZm Z m Z ddl m Z ddl mZddlmZmZmZmZmZmZmZmZddlmZddlmZmZmZdd lmZdd l m!Z!dd l"m#Z#zdd l$Z$dd l%m&Z'ddl(m)Z)Wn e*yZ+ze ,de+e ,de-de+d Z+[+wwedZ.eGdddeZ/GdddeZ0d S)zFish Audio text-to-speech service implementation. This module provides integration with Fish Audio's real-time TTS WebSocket API for streaming text-to-speech synthesis with customizable voice parameters. ) dataclassfield)AnyAsyncGeneratorLiteralMappingOptionalSelf)logger) BaseModel) CancelFrameEndFrame ErrorFrameFrameInterruptionFrame StartFrameTTSAudioRawFrameTTSStoppedFrame)FrameDirection) NOT_GIVEN TTSSettings _NotGiven)InterruptibleTTSService)Language) traced_ttsN)connect)Statez Exception: zGIn order to use Fish Audio, you need to `pip install pipecat-ai[fish]`.zMissing module: )opusmp3pcmwavcseZdZUdZedddZedBeBed<edddZ e dBeBed<ed ddZ e dBeBed <ed ddZ e dBeBed <ed ddZe dBeBed<edddZedBeBed<edeeefdeffdd ZZS)FishAudioTTSSettingsaSettings for FishAudioTTSService. Parameters: latency: Latency mode ("normal" or "balanced"). Defaults to "balanced". normalize: Whether to normalize audio output. Defaults to True. temperature: Controls randomness in speech generation (0.0-1.0). top_p: Controls diversity via nucleus sampling (0.0-1.0). prosody_speed: Speech speed multiplier (0.5-2.0). Defaults to 1.0. prosody_volume: Volume adjustment in dB (-20 to 20). Defaults to 0. cCtSNrr%r%M/home/ubuntu/.local/lib/python3.10/site-packages/pipecat/services/fish/tts.py=zFishAudioTTSSettings.)default_factoryNlatencycCr"r#r$r%r%r%r&r'>r( normalizecCr"r#r$r%r%r%r&r'?r( temperaturecCr"r#r$r%r%r%r&r'@r(top_pcCr"r#r$r%r%r%r&r'Ar( prosody_speedcCr"r#r$r%r%r%r&r'Br(prosody_volumesettingsreturncsNt|}|dd}t|tr!|d|d|d|dt|S)zNConstruct settings from a plain dict, destructuring legacy nested ``prosody``.prosodyNr.speedr/volume)dictpop isinstance setdefaultgetsuper from_mapping)clsr0flatnested __class__r%r&r;Ds    z!FishAudioTTSSettings.from_mapping)__name__ __module__ __qualname____doc__rr*strr__annotations__r+boolr,floatr-r.r/int classmethodrrr r; __classcell__r%r%r?r&r!0s  (r!cspeZdZUdZeZeed<GdddeZddddddddde d e e d e e d e e d e d e e de ede effddZ defddZdedee efffdd Zdeffdd Zdeffdd Zdeffdd Zfdd Zfd!d"Zd#d$Zd%d&Zd4d'e e fd(d)Zd*d+Zde d,e!ffd-d. Z"d/d0Z#e$d1e d'e de%e&dffd2d3Z'Z(S)5FishAudioTTSServiceaFish Audio text-to-speech service with WebSocket streaming. Provides real-time text-to-speech synthesis using Fish Audio's WebSocket API. Supports various audio formats, customizable prosody controls, and streaming audio generation with interruption handling. _settingsc@sdeZdZUdZejZeeed<dZ ee ed<dZ ee ed<dZ eeed<d Zeeed <d S) zFishAudioTTSService.InputParamsa-Input parameters for Fish Audio TTS configuration. .. deprecated:: 0.0.105 Use ``settings=FishAudioTTSService.Settings(...)`` instead. Parameters: language: Language for synthesis. Defaults to English. latency: Latency mode ("normal" or "balanced"). Defaults to "normal". normalize: Whether to normalize audio output. Defaults to True. prosody_speed: Speech speed multiplier (0.5-2.0). Defaults to 1.0. prosody_volume: Volume adjustment in dB. Defaults to 0. languagenormalr*Tr+?r.rr/N)rArBrCrDrENrNrrFr*rEr+rGr.rHr/rIr%r%r%r& InputParamsZs  rRNr) reference_idmodelmodel_id output_format sample_rateparamsr0api_keyrSrTrUrVrWrXr0c sh|r|rtd|r1ddl} | | d| jdtddWdn1s*wY|}|jdddd d ddd dd } |durL|d d|| _|durY|dd|| _ |dur|d|s|j durm|j | _ |j durv|j | _ |j dur|j | _ |j dur|j | _ |dur| |tjdd d d || d| ||_d|_d|_d|_d|_||_dS)ajInitialize the Fish Audio TTS service. Args: api_key: Fish Audio API key for authentication. reference_id: Reference ID of the voice model to use for synthesis. .. deprecated:: 0.0.105 Use ``settings=FishAudioTTSService.Settings(voice=...)`` instead. model: Deprecated. Reference ID of the voice model to use for synthesis. .. deprecated:: 0.0.74 The ``model`` parameter is deprecated and will be removed in version 0.1.0. Use ``reference_id`` instead to specify the voice model. model_id: Specify which Fish Audio TTS model to use (e.g. "s1"). .. deprecated:: 0.0.105 Use ``settings=FishAudioTTSService.Settings(model=...)`` instead. output_format: Audio output format. Defaults to "pcm". sample_rate: Audio sample rate. If None, uses default. params: Additional input parameters for voice customization. .. deprecated:: 0.0.105 Use ``settings=FishAudioTTSService.Settings(...)`` instead. settings: Runtime-updatable settings. When provided alongside deprecated parameters, ``settings`` values take precedence. **kwargs: Additional arguments passed to the parent service. zHCannot specify both 'model' and 'reference_id'. Use 'reference_id' only.rNalwayszdParameter 'model' is deprecated and will be removed in a future version. Use 'reference_id' instead.) stacklevelzs2-probalancedTrP) rTvoicerNr*r+r,r-r.r/rSr^rUrTrX)push_stop_framespush_start_framepause_frame_processingrWr0z wss://api.fish.audio/v1/tts/liver%) ValueErrorwarningscatch_warnings simplefilterwarnDeprecationWarningSettings"_warn_init_param_moved_to_settingsr^rTr*r+r.r/ apply_updater:__init___api_key _base_url _websocket _receive_task_fish_sample_rate_output_format) selfrYrSrTrUrVrWrXr0kwargsrcdefault_settingsr?r%r&rknsx-            zFishAudioTTSService.__init__r1cCsdS)zCheck if this service can generate processing metrics. Returns: True, as Fish Audio service supports metrics generation. Tr%rrr%r%r&can_generate_metricssz(FishAudioTTSService.can_generate_metricsdeltacs8t|IdH}|r|IdH|IdH|S)a=Apply a settings delta and reconnect if needed. Any change to voice or model triggers a WebSocket reconnect. Args: delta: A :class:`TTSSettings` (or ``FishAudioTTSService.Settings``) delta. Returns: Dict mapping changed field names to their previous values. N)r:_update_settings _disconnect_connect)rrrwchangedr?r%r&rxs  z$FishAudioTTSService._update_settingsframecs.t|IdH|j|_|IdHdS)zStart the Fish Audio TTS service. Args: frame: The start frame containing initialization parameters. N)r:startrWrprzrrr|r?r%r&r}szFishAudioTTSService.startc&t|IdH|IdHdS)zZStop the Fish Audio TTS service. Args: frame: The end frame. N)r:stopryr~r?r%r&r zFishAudioTTSService.stopcr)z_Cancel the Fish Audio TTS service. Args: frame: The cancel frame. N)r:cancelryr~r?r%r&rrzFishAudioTTSService.cancelcsLtIdH|IdH|jr"|js$|||j|_dSdSdSr#)r:rz_connect_websocketrnro create_task_receive_task_handler _report_errorrur?r%r&rzs  zFishAudioTTSService._connectcsBtIdH|jr||jIdHd|_|IdHdSr#)r:ryro cancel_task_disconnect_websocketrur?r%r&ry$s zFishAudioTTSService._disconnectc s^z|jr|jjtjurWdStddd|ji}|jj|d<t |j |dIdH|_|j |jj |j |jj|jj|jjd|jjd}|jjdurQ|jj|d<|jjdur]|jj|d <d d d i|d }|jt|IdHtd|dIdHWdSty}z!|jd||dIdHd|_|d|IdHWYd}~dSd}~ww)NzConnecting to Fish Audio AuthorizationzBearer rT)additional_headers)r3r4)rWr*formatr+r2rSr,r-r}text)eventrequestz Sent start message to Fish Audio on_connectedUnknown error occurred:  error_msg exceptionon_connection_error)rnstaterOPENr debugrlrMrTwebsocket_connectrmrpr*rqr+r.r/r^r,r-send ormsgpackpackb_call_event_handler Exception push_error)rrheadersrequest_settings start_messageer%r%r&r-s>      "z&FishAudioTTSService._connect_websocketc szez)|IdH|jr*tdddi}|jt|IdH|jIdHWntyJ}z|j d||dIdHWYd}~nd}~wwWd|_| dIdHdSWd|_| dIdHdSd|_| dIdHw)NzDisconnecting from Fish Audiorrrron_disconnected) stop_all_metricsrnr rrrrcloserrr)rr stop_messagerr%r%r&rQs* &z)FishAudioTTSService._disconnect_websocket context_idcsPt|d|jr|jjtjurdSddi}|t |IdHdS)z@Flush any buffered audio by sending a flush event to Fish Audio.z: Flushing audio buffersNrflush) r tracernrrCLOSED_get_websocketrrr)rrr flush_messager%r%r& flush_audio`s zFishAudioTTSService.flush_audiocCs|jr|jStd)NzWebsocket not connected)rnrrur%r%r&rhsz"FishAudioTTSService._get_websocket directioncs(t||IdH|IdHdSr#)r:_handle_interruptionr)rrr|rr?r%r&rmsz(FishAudioTTSService._handle_interruptionc s|2z3dHW}z^t|trht|}t|trh|d}|dkrH|d}|rGt|dkrGt||j d}| |IdH| IdHn |dkrh|dd}|dkr`|j d d IdHnt d |Wqty}z|j d ||d IdHWYd}~qd}~ww6dS)Nraudioifinishreasonunknownerrorz(Fish Audio server error during synthesis)rzFish Audio session finished: rr)rr7bytesrunpackbr5r9lenrrW push_framestop_ttfb_metricsrr rr)rrmessagemsgr audio_datar|rrr%r%r&_receive_messagesqs6      &z%FishAudioTTSService._receive_messagesrc Cs:t|d|dzt|jr|jjtjur|IdHd|d}z(|t |IdH| |IdHddi}|t |IdHWn/t yz}z#t d|d Vt|d V|IdH|IdHWYd}~nd}~wwdVWdSt y}zt d|d VWYd}~dSd}~ww) a+Generate speech from text using Fish Audio's streaming API. Args: text: The text to synthesize into speech. context_id: The context ID for tracking audio frames. Yields: Frame: Audio frames and control frames for the synthesized speech. z: Generating Fish TTS: []Nr)rrrrr)r)r)r rrnrrrrzrrrrstart_tts_usage_metricsrrrry)rrrr text_messagerrr%r%r&run_ttss2    zFishAudioTTSService.run_ttsr#))rArBrCrDr!rhrFr rRrErFishAudioOutputFormatrIrkrGrvrr5rrxrr}r rr rrzryrrrrrrrrrrrrrKr%r%r?r&rLOsX   w   $(rL)1rD dataclassesrrtypingrrrrrr logurur pydanticr pipecat.frames.framesr r rrrrrr"pipecat.processors.frame_processorrpipecat.services.settingsrrrpipecat.services.tts_servicerpipecat.transcriptions.languager(pipecat.utils.tracing.service_decoratorsrrwebsockets.asyncio.clientrrwebsockets.protocolrModuleNotFoundErrorrrrrr!rLr%r%r%r&s2   (