o iL@s~dZddlmZddlZddlZddlZddlZddlZej ej ej e Z ej e dZej edZej edZeee fD]ZeejvrRejdeqDejddZed ed e Zerqed eejd<neejd<ed Zej e d d Zd ZdZeedfd9ddZd:ddZ             d;d^s$z!download_model..zModel already present at r)snapshot_downloadzyhuggingface_hub not installed. Install with: pip install huggingface-hub Or download model manually and pass --model-pathT)exist_okzAttempting download from z...)z*.binztraining_args.binz optimizer.pt)repo_idr tokenignore_patternszModel downloaded to zPrivate repo download failed: NzTrying public base model: )r"r r#zBase model downloaded to zpUsing public base SparkTTS model (no custom speakers). For full speaker support, download your fine-tuned model.z#Public model download also failed: z Model download failed. Options: 1. Download from Modal volume: modal volume get veena3-models spark_tts_4speaker models/spark_tts_4speaker 2. Download from HuggingFace (if you have access): huggingface-cli download z --local-dir z> 3. Use base SparkTTS model: huggingface-cli download za 4. Specify a custom path: python -m veena3modal.local_server --model-path /path/to/model)anyloggerinfohuggingface_hubr ImportErrorerrorsysexitrmakedirs ExceptionwarningPUBLIC_FALLBACK_REPOPRIVATE_HF_REPO)r r r marker_filesrdownloaded_pathee2rrrdownload_modelMsp  r6 model_pathtuple[str, str]cCs~tj|d}tj|d}tj|r"tjtj|dr"|}n|}|}tdtd|td|||fS)a Resolve LLM and BiCodec paths from a Spark TTS model directory. Spark TTS model structure (from SparkAudio/Spark-TTS-0.5B): model_path/ LLM/ <- Language model (vLLM loads this) BiCodec/ <- Audio tokenizer/decoder config.yaml <- Top-level config Fine-tuned models may have LLM files at root (flat structure). This function auto-detects both layouts. Returns: (llm_path, bicodec_path) resolved paths LLMBiCodecrzResolved model paths:z LLM (vLLM): z BiCodec: )rrrisdirrr&r')r7 llm_subdirbicodec_subdirllm_path bicodec_pathrrrresolve_model_pathss" r@cuda333333?Fr devicegpu_memory_utilizationfloat enable_srboolsr_checkpoint_dir num_enginesintmax_num_batched_tokens int | None max_num_seqsenable_chunked_prefill bool | Noneenable_prefix_cachingdisable_log_stats enforce_eagerprecompute_speaker_globalscCs&ddlm}t|\}}t}tdtdtd|td|td|td|td |d d td ||d urXtd||d urdtd|| d urttd| rodnd| d urtd| rdnd| d urtd| rdnd| d urtd| rdndtd| rdndtd|rdndtd||||r|nd |||||||| | | | | d}t|}td|ddtd|jtd|jrdndtd |jr |jj r d!nd|S)"z Initialize TTS runtime for local serving. This replaces Modal's @modal.enter lifecycle hook. Loads: vLLM engine, BiCodec decoder, prompt builder, pipelines. r)initialize_runtime<============================================================z'Initializing Veena3 TTS Runtime (local)z Model: z LLM path: z BiCodec: z Device: z GPU Memory: z.0%z (total)z Engines: Nz Max batched tokens: z Max seqs: z Chunked prefill: enableddisabledz Prefix caching: z Engine stats logs: z Enforce eager: z Precompute globals: z SR: )r7r?rHrCr rDrFrIrKrMrNrPrQrRrSzRuntime initialized in z.1fsz Model version: z Streaming: available unavailablez Super-resolution: loaded) veena3modal.services.tts_runtimerTr@timer&r' model_versionstreaming_pipeline sr_service is_loaded)r7rCrDrFrHr rIrKrMrNrPrQrRrSrTr>r?startruntimeelapsedrrrinitialize_local_runtimesb       &recCsddlm}|S)z} Create the FastAPI app for local serving. Same app factory as Modal deployment, but called from local context. r create_app)veena3modal.api.fastapi_apprgrfrrrcreate_local_apps ricCsRtjdtjdd}|d}|jdttjdt dt dd |jd tt d t dd |jd ttjd dd |jdddd|d}|jdtdddgdd|jdt t tjdddd |jdt t tjddd d |jd!t d"d#d |jd$t d"d%d |jd&dd"d'd(|jd)dd*d+d(|jd,dd*d-d(|jd.dd*d/d(|jd0dd*d1d(|jd2dd*d3d(|d4}|jd5dd*d6d(|jd7ttjd8d9d |d:}|jd;tdt t tjd?d@dAd |jdBt dCdDd |jdEdd*dFd(|dG}|jdHdd*dId(|dJ}|jdKtdLgdMdNd| S)Oz3Parse CLI arguments for local server configuration.z8Veena3 TTS Local Server - GPU-accelerated text-to-speecha Examples: python -m veena3modal.local_server # Auto-download model, serve on :8000 python -m veena3modal.local_server --port 8080 # Custom port python -m veena3modal.local_server --model-path ./my_model # Use local model python -m veena3modal.local_server --gpu-memory 0.5 # Limit GPU memory python -m veena3modal.local_server --workers 2 # Multiple workers (careful: each loads model) ) descriptionformatter_classepilogModelz --model-path MODEL_PATHz"Path to Spark TTS model (default: ))typedefaulthelpz --hf-repoz,HuggingFace repo to download from (default: z --hf-tokenHF_TOKENz2HuggingFace token (default: from HF_TOKEN env var)z--skip-download store_truez-Don't auto-download model (fail if not found))actionrrGPUz--devicerAcpuz$Device for inference (default: cuda))rprqchoicesrrz --gpu-memoryGPU_MEMORY_UTILIZATIONz0.25z4vLLM GPU memory utilization fraction (default: 0.25)z --num-engines NUM_ENGINES1zKNumber of vLLM engine instances (default: 1, use 2-3 for multi-engine mode)z--max-num-batched-tokensNzLOverride vLLM max_num_batched_tokens scheduler limit (default: model config)z--max-num-seqsz?Override vLLM max_num_seqs scheduler limit (default: vLLM auto)z--enable-chunked-prefillz!Force-enable vLLM chunked prefill)rurqrrz--disable-chunked-prefillFz"Force-disable vLLM chunked prefillz--disable-prefix-cachingzDisable vLLM prefix cachingz--disable-engine-stats-logsz#Disable vLLM periodic stats loggingz--enforce-eagerz(Enable eager mode (disables CUDA graphs)z--precompute-globalszWPrecompute speaker global tokens at startup (streaming-only optimization, experimental)zSuper-Resolutionz --enable-srz/Enable AP-BWE super-resolution (16kHz -> 48kHz)z --sr-pathAP_BWE_CHECKPOINT_DIRz#Path to AP-BWE checkpoint directoryServerz--hostz0.0.0.0zBind address (default: 0.0.0.0)z--portPORT8000z Port to serve on (default: 8000)z --workersr z=Number of uvicorn workers (default: 1, each loads full model)z--reloadzDEnable auto-reload for development (incompatible with --workers > 1)Authz--authz;Enable API key authentication (default: disabled for local)Loggingz --log-levelINFO)DEBUGrWARNINGERRORzLog level (default: INFO))argparseArgumentParserRawDescriptionHelpFormatteradd_argument_group add_argumentr renvirongetDEFAULT_LOCAL_MODEL_DIRr1rErJ parse_args)parser model_group gpu_groupsr_group server_group auth_group log_grouprrrrs.           rcst}tjtt|jddd|jsdtjd<t dntj dd|j t fdd d D}|sN|j rEtd d td t|j|jd|j}|jrVd}t|j|j|j|j|j|j|j|j||jrndnd|jrtdnd|j rzdnd|j!dddl"}t dt dt dt d|j#d|j$t d|j#d|j$dt d|j#d|j$dt d|j#d|j$dt d|j#d|j$d t d|j%d!d|j#|j$|j&|j'|j(d"d#dS)$z Main entry point for local TTS server. Lifecycle: 1. Parse args + configure logging 2. Download model if needed 3. Initialize TTS runtime (vLLM engine, decoders, pipelines) 4. Create FastAPI app 5. Run uvicorn z6%(asctime)s | %(levelname)-8s | %(name)s | %(message)sz%Y-%m-%d %H:%M:%S)levelformatdatefmttrueAUTH_BYPASS_MODEzAAuth bypass enabled (local mode). Use --auth to require API keys.falsec3rrr)rmarkerr7rrrs  zmain..rzModel not found at z and --skip-download is setr )r r r FNT)r7rCrDrFrHr rIrKrMrNrPrQrRrSrrrUz Veena3 TTS Local Server startingz URL: http://rz Health: http://z/v1/tts/healthz Generate: POST http://z/v1/tts/generatez Metrics: http://z/v1/tts/metricsz Docs: http://z/docsz)veena3modal.local_server:create_local_app)factoryhostportworkersreload log_leveltimeout_keep_alive))rlogging basicConfiggetattrrauthrrr&r' setdefaultr7r% skip_downloadr*r+r,r6r r rNdisable_chunked_prefillrerC gpu_memoryrFsr_pathrIrKrMdisable_prefix_cachingdisable_engine_stats_logsrRprecompute_globalsuvicornrrrunrrlower)args model_existsrNrrrrmains|              r__main__)r r r r r rrr )r7r rr8) rArBFNNr NNNNNNF)r7r rCr rDrErFrGrHrr rrIrJrKrLrMrLrNrOrPrOrQrOrRrOrSrG)"__doc__ __future__rrrrr+r]rdirnameabspath__file__ REPO_ROOTr EXTERNAL_DIR SPARKTTS_PATH AP_BWE_PATHinsertrrexisting_pythonpath new_paths getLoggerr&rr1r0r6r@rerirr__name__rrrrsd !    O& O 0]