o oi@sJddlmZddlmZddlZddlmZddlmZGdddZ dS)) defaultdict)ListN)Dataset)tqdmc@sBeZdZdZdefddZdedefddZdeefd d Z d S) SpeakerEmbeddingGroupera Computes per-speaker averaged embeddings and adds them as a new column. Groups all embedding vectors by a speaker column, averages them, then maps the averaged vector back to every sample of that speaker. Can be used: - Inside SpeakerEmbeddingProcessor (after embedding + clustering) - Independently when embeddings already exist in the dataset (add_speaker_emb: false, group_sp_emb.do_this: true) embedding_columncCs||_||_dS)z Args: settings: GroupEmbeddingSettings instance. embedding_column: Name of the source embedding column to average (e.g. 'wavlm_embedding'). N)settingsr)selfrrr M/home/ubuntu/kanitts-2-dataset-pipeline/utils/speaker_emb/group_embeddings.py__init__s z SpeakerEmbeddingGrouper.__init__datasetreturnc sZ|jj|j}|jjtdtddtd|dtddtdt|dtt}t|dd D]'}|}||}|d urJq;t j |t j d }t t |r[q;|||q;t|}td |it|d d D]\}} t jt j| dddd} | |<qxfdd} |j| dddd}tdd|S)a Add a grouped (per-speaker averaged) embedding column to the dataset. Steps: 1. Collect all embeddings per unique speaker value, skipping None/NaN. 2. Average embeddings per speaker. 3. Map the averaged vector back to every sample via dataset.map(). Samples whose speaker has no valid embeddings at all receive None in the output column (they are not filtered out). Args: dataset: HuggingFace Dataset containing both the embedding column and the group-by column. Returns: Dataset with the new grouped embedding column added. z( [Group Embeddings] Starting grouping...z![Group Embeddings] Group by: ''z![Group Embeddings] Source emb: 'z![Group Embeddings] Output col: 'z![Group Embeddings] Dataset size: z samplesz)[Group Embeddings] Collecting per speaker)descN)dtypez:[Group Embeddings] Unique speakers with valid embeddings: z[Group Embeddings] Averagingr)axiscs|}|diS)N)get)samplespeakeraveraged_embeddings group_colout_colr r _add_grouped_embZsz7SpeakerEmbeddingGrouper.group.._add_grouped_emb zroup Embeddings] Adding 'z' column)num_procru[Group Embeddings] Done — 'z' column added. )rgroup_by_column_namergrouped_embedding_columnprintlenrlistrnparrayfloat32anyisnanappenditemsmeanstacktolistmap) r r emb_colspeaker_embeddingsrr embeddingemb_arr n_speakers embeddingsavg_embrr rr group sD zSpeakerEmbeddingGrouper.groupcCs |jjgS)z6Return the list of columns produced by this processor.)rr)r r r r get_preserved_columnsgs z-SpeakerEmbeddingGrouper.get_preserved_columnsN) __name__ __module__ __qualname____doc__strr rr4rr5r r r r r s   Gr) collectionsrtypingrnumpyr"datasetsrrrr r r r s