o is @sddlZddlmZddlmZddlmZmZddlZ ddl m Z ddl m Z ddlmZddlmZdd lmZmZdd lmZdd lmZeeZGd d d eZdS)N)AsyncGenerator)cached_property)Literalcast) EngineClient) RequestLogger) OpenAIServing)OpenAIServingModels) PromptTypeStreamingInput) init_logger)SupportsRealtimec seZdZdZdddedededBdeffd d Ze d e e fd d Z de ejdfdejeed e edffddZZS)OpenAIServingRealtimezRealtime audio transcription service via WebSocket streaming. Provides streaming audio-to-text transcription by transforming audio chunks into StreamingInput objects that can be consumed by the engine. F)log_error_stack engine_clientmodelsrequest_loggerNrcs,tj||||dd|_td|jdS)N)rrrrrealtimez.OpenAIServingRealtime initialized for task: %s)super__init__ task_typeloggerinfo)selfrrrr __class__^/home/ubuntu/vllm_env/lib/python3.10/site-packages/vllm/entrypoints/openai/realtime/serving.pyrszOpenAIServingRealtime.__init__returncCs$ddlm}||j}ttt|S)z0Get the model class that supports transcription.r) get_model_cls) vllm.model_executor.model_loaderr model_configrtyper )rr model_clsrrrr#0s  zOpenAIServingRealtime.model_cls audio_stream input_streamcCsFtttdf|j|||j}|2z 3dHW}t|dVq6dS)a Transform audio stream into StreamingInput for engine.generate(). Args: audio_stream: Async generator yielding float32 numpy audio arrays input_stream: Queue containing context token IDs from previous generation outputs. Used for autoregressive multi-turn processing where each generation's output becomes the context for the next iteration. Yields: StreamingInput objects containing audio prompts for the engine N)prompt)rrr r#buffer_realtime_audior!r )rr$r%stream_input_iterr&rrrtranscribe_realtime8s z)OpenAIServingRealtime.transcribe_realtime)__name__ __module__ __qualname____doc__rr rboolrrr"r r#rnpndarrayasyncioQueuelistintr r) __classcell__rrrrrs*    r)r1collections.abcr functoolsrtypingrrnumpyr/vllm.engine.protocolrvllm.entrypoints.loggerr&vllm.entrypoints.openai.engine.servingr&vllm.entrypoints.openai.models.servingr vllm.inputs.datar r vllm.loggerr %vllm.model_executor.models.interfacesr r*rrrrrrs