o oia' @sdZddlZddlmZddlmmZddlZddl m Z m Z ddl m Z mZGdddejZGddde ZGd d d Z dd e ejejefdedede edejf ddZdS)aV Speaker Embedder Module for KaniTTS-2 ==================================== Lightweight module for generating speaker embeddings from audio using WavLM model. Model: Orange/Speaker-wavLM-tbr (16kHz input, 128-dim L2-normalized output) Based on spk_embeddings.py from Orange SA (CC-BY-SA-3.0) https://huggingface.co/Orange/Speaker-wavLM-tbr N)UnionOptional)WavLMPreTrainedModel WavLMModelcs4eZdZdZd dedeffdd Zdd ZZS) TopLayersu Projection layers on top of WavLM for speaker embedding extraction. Architecture: - Conv1d: 2048 → 512 - BatchNorm + ReLU - Conv1d: 512 → embd_size (default 128) - BatchNorm + ReLU - L2 normalization  embd_sizetop_interm_sizecshtt|tjd|dd|_tj|ddd|_tj||dd|_tj|ddd|_ tj dd|_ dS) Ni) in_channels out_channels kernel_sizeFgMbP?) num_featuresaffineepsT)inplace) superr__init__nnConv1daffine1 BatchNorm1d batchnorm1affine2 batchnorm2ReLU activation)selfr r  __class__M/home/ubuntu/.local/lib/python3.10/site-packages/kani_tts/speaker_embedder.pyr s zTopLayers.__init__cCsH||||}||||}t|dddddfS)z Args: x: Stats pooling output [batch, 2048, 1] Returns: L2-normalized embeddings [batch, embd_size] Nr)rrrrrF normalize)rxoutr!r!r"forward(szTopLayers.forward)rr)__name__ __module__ __qualname____doc__intrr' __classcell__r!r!rr"rs rcs(eZdZdZfddZddZZS)EmbeddingsModelz Complete WavLM-based speaker embedding model. Architecture: 1. MVN normalization on input audio 2. WavLM encoder 3. Stats pooling (mean + std) 4. Top projection layers 5. L2 normalization cs*t|t||_t|j|j|_dSN)rrrwavlmrr r top_layers)rconfigrr!r"rAs  zEmbeddingsModel.__init__cCs||jddd|jdddd}|j|ddj}|jdd}|jddjdd}|d }tj||fddj d d}| |S) z Args: input_values: Audio waveform [batch, time_samples] Returns: Speaker embeddings [batch, embd_size] r T)dimkeepdimg|=F) input_valuesoutput_hidden_statesr3)ming?) meanstdr0last_hidden_statevarclamppowtorchcat unsqueezer1)rr5x_normbase_outr:r=r;x_statsr!r!r"r'Fs    zEmbeddingsModel.forward)r(r)r*r+rr'r-r!r!rr"r.5s r.c@seZdZdZ   ddedeedefdd Zd ej d e d ej fd dZ dd e e jej fd ee d ej fddZded ej fddZdS)SpeakerEmbedderu Simple speaker embedder for single audio → embedding generation. Features: - Loads WavLM model once - Generates 128-dim L2-normalized speaker embeddings - Expects 16kHz audio input - Handles variable-length audio (max 20 seconds recommended) - Returns PyTorch tensors ready for TTS model Usage: embedder = SpeakerEmbedder() # From numpy array (16kHz) audio = np.random.randn(16000 * 5) # 5 seconds embedding = embedder.embed_audio(audio) # [1, 128] # From torch tensor audio_tensor = torch.randn(1, 16000 * 5) embedding = embedder.embed_audio(audio_tensor) nineninesix/speaker-emb-tbrN>@ model_namedevicemax_duration_seccCs||_d|_||_t||j|_|dur#ttjrdnd|_nt||_t d|dt ||_ |j |j|j t d|jdS)a Initialize speaker embedder. Args: model_name: HuggingFace model ID device: Target device ('cuda', 'cpu', or None for auto-detect) max_duration_sec: Maximum audio duration in seconds (longer audio will be truncated) >Ncudacpuu)🔊 Loading WavLM speaker embedder from z...u✅ Speaker embedder ready on )rI target_srrKr, max_samplesr@rJrM is_availableprintr.from_pretrainedmodeltoeval)rrIrJrKr!r!r"rws   zSpeakerEmbedder.__init__audio sample_ratereturncCs|dkr|jd|jdkr|jdd}n|d}|dkr)td|j||jkrNzddlm}Wn tyAtdw|j||jd}||}|jddkrYtd |jd|j krzt d |jd|d d |j d |d|j }|S)a( Prepare audio for embedding extraction: resample if needed, convert to mono, truncate. Args: audio: Audio tensor (1D or 2D) sample_rate: Current sample rate of audio Returns: Prepared audio tensor [time_samples] at target_sr r9rr r7z#Expected 1D or 2D audio, got shape NzKtorchaudio is required for resampling. Install with: pip install torchaudio) orig_freqnew_freqzAudio is emptyu⚠️ Audio is z.2fzs, truncating to s) r3shaper: ValueErrorrOtorchaudio.transforms transforms ImportErrorResamplerPrRrK)rrWrXT resamplerr!r!r"_prepare_audios,    $zSpeakerEmbedder._prepare_audiocCst|tjrt|}n|}|dur|j}|||}|d |j }t | |}Wd|S1s=wY|S)a Generate speaker embedding from audio. Args: audio: Audio waveform as numpy array or torch tensor - If 1D: shape [time_samples] - If 2D: shape [batch, time_samples] or [channels, time_samples] sample_rate: Sample rate of input audio (if None, assumes 16kHz) Returns: Speaker embedding tensor [1, 128] (L2-normalized) Raises: ValueError: If audio is empty Nr) isinstancenpndarrayr@ from_numpyfloatrOrerBrUrJno_gradrT)rrWrX audio_batch embeddingr!r!r" embed_audios     zSpeakerEmbedder.embed_audio audio_pathcCsdzddl}Wn tytdw||\}}|jddkr'|jdd}n|d}|j||dS)ae Generate speaker embedding from audio file. Args: audio_path: Path to audio file (supports any sample rate, will be resampled) Returns: Speaker embedding tensor [1, 128] Raises: ImportError: If torchaudio is not installed ValueError: If audio file cannot be loaded rNzTtorchaudio is required for loading audio files. Install with: pip install torchaudior r7rX) torchaudioraloadr]r:rn)rrorqrWsrr!r!r"embed_audio_files  z SpeakerEmbedder.embed_audio_file)rGNrHr/)r(r)r*r+strrrjrr@Tensorr,rerrgrhrnrtr!r!r!r"rF`s8   6 *rFrLrGrWrXrIrJrYcCs.t||d}t|tr||S|j||dS)a+ Convenience function to generate speaker embedding in one line. Args: audio: Audio as numpy array, torch tensor, or file path sample_rate: Sample rate of audio (ignored if audio is file path) model_name: HuggingFace model ID device: Target device Returns: Speaker embedding [1, 128] Example: # From numpy array audio_np = np.random.randn(16000 * 5) emb = compute_speaker_embedding(audio_np) # From file emb = compute_speaker_embedding("speaker.wav") )rIrJrp)rFrfrurtrn)rWrXrIrJembedderr!r!r"compute_speaker_embeddings   rx)rLrGN)r+r@torch.nnrtorch.nn.functional functionalr#numpyrgtypingrr(transformers.models.wavlm.modeling_wavlmrrModulerr.rFrhrvrur,rxr!r!r!r"s2 !+7