o 9wiS@sdZddlZddlmZddlmZmZddlZddl m Z ddl m Z m Z ddlmZddlmZmZmZdd lmZmZdd lmZdd lmZdd lmZdd lmZddlm Z GdddeZ!Gddde!Z"Gddde!Z#GdddeZ$dS)zClustering pipelinesN)Enum)OptionalTuple) rearrange) SlidingWindowSlidingWindowFeature)Pipeline) CategoricalIntegerUniform)fclusterlinkage)linear_sum_assignment)cdist) AudioFile)oracle_segmentation) permutatec seZdZ   d dededeffdd Z d!d ed eed eed eefddZ d"de j dee de e j e j e j ffddZ de j de j fddZ d#de j de j de j de j def ddZ d$de j dee d eed eed eede j f ddZZS)%BaseClusteringcosineFmetricmax_num_embeddingsconstrained_assignmentcs t||_||_||_dSN)super__init__rrrselfrrr __class__`/home/ubuntu/sommelier/.venv/lib/python3.10/site-packages/pyannote/audio/pipelines/clustering.pyr-s  zBaseClustering.__init__Nnum_embeddings num_clusters min_clusters max_clusterscCsp|p|pd}tdt||}|p|p|}tdt||}||kr-td|dd|dd||kr3|}|||fS)NzQmin_clusters must be smaller than (or equal to) max_clusters (here: min_clusters=gz and max_clusters=z).)maxmin ValueError)rr"r#r$r%r r r!set_num_clusters8s   zBaseClustering.set_num_clusters embeddings segmentationsreturnc Cstj|jdddk}tjt|dd}t||\}}t|}||jkrCtt |}t |t |d|j}||}||}|||f||fS)aFilter NaN embeddings and downsample embeddings Parameters ---------- embeddings : (num_chunks, num_speakers, dimension) array Sequence of embeddings. segmentations : (num_chunks, num_frames, num_speakers) array Binary segmentations. Returns ------- filtered_embeddings : (num_embeddings, dimension) array chunk_idx : (num_embeddings, ) array speaker_idx : (num_embeddings, ) array r&axisrN) npsumdataanyisnanwherelenrlistrangerandomshufflesorted) rr,r-activevalid chunk_idx speaker_idxr"indicesr r r!filter_embeddingsOs   z BaseClustering.filter_embeddings soft_clustersc Cs~tj|t|d}|j\}}}dtj||ftjd}t|D]\}}t|dd\}} t|| D] \} } | ||| f<q1q |S)N)nandtypeT)maximize) r2 nan_to_numnanminshapeonesint8 enumeraterzip) rrD num_chunks num_speakersr# hard_clustersccostspeakersclustersskr r r!constrained_argmaxws z!BaseClustering.constrained_argmaxtrain_chunk_idxtrain_speaker_idxtrain_clusters constrainedcstd}|j\}}} |||ftfddt|D} ttt|d| |jdd||d} d| } |r?|| } ntj | dd } | | | fS) aAssign embeddings to the closest centroid Cluster centroids are computed as the average of the train embeddings previously assigned to them. Parameters ---------- embeddings : (num_chunks, num_speakers, dimension)-shaped array Complete set of embeddings. train_chunk_idx : (num_embeddings,)-shaped array train_speaker_idx : (num_embeddings,)-shaped array Indices of subset of embeddings used for "training". train_clusters : (num_embedding,)-shaped array Clusters of the above subset constrained : bool, optional Use constrained_argmax, instead of (default) argmax. Returns ------- soft_clusters : (num_chunks, num_speakers, num_clusters)-shaped array hard_clusters : (num_chunks, num_speakers)-shaped array centroids : (num_clusters, dimension)-shaped array Clusters centroids r&c"g|] }tj|kddqSrr/r2mean.0rYr]train_embeddingsr r! z4BaseClustering.assign_embeddings..zc s d -> (c s) drz(c s) k -> c s k)rTrXr1r/) r2r(rLvstackr:rrrrZargmax)rr,r[r\r]r^r#rQrR dimension centroids e2k_distancerDrSr rer!assign_embeddingss.#      z BaseClustering.assign_embeddingscKs|j||d\}}} |j\} } |j| |||d\}}}|dkrD|j\} } } tj| | ftjd}t| | df}tj|ddd}|||fS|j||||d }|j ||| ||j d \}}}|||fS) aApply clustering Parameters ---------- embeddings : (num_chunks, num_speakers, dimension) array Sequence of embeddings. segmentations : (num_chunks, num_frames, num_speakers) array Binary segmentations. num_clusters : int, optional Number of clusters, when known. Default behavior is to use internal threshold hyper-parameter to decide on the number of clusters. min_clusters : int, optional Minimum number of clusters. Has no effect when `num_clusters` is provided. max_clusters : int, optional Maximum number of clusters. Has no effect when `num_clusters` is provided. Returns ------- hard_clusters : (num_chunks, num_speakers) array Hard cluster assignment (hard_clusters[c, s] = k means that sth speaker of cth chunk is assigned to kth cluster) soft_clusters : (num_chunks, num_speakers, num_clusters) array Soft cluster assignment (the higher soft_clusters[c, s, k], the most likely the sth speaker of cth chunk belongs to kth cluster) centroids : (num_clusters, dimension) array Centroid vectors of each cluster r-)r#r$r%r1rGr&rTr0keepdims)r#)r^) rCrLr+r2zerosrNrMrbclusterror)rr,r-r#r$r%kwargsrfr[r\r"_rQrRrSrDrmr]r r r!__call__s>&     zBaseClustering.__call__)rrF)NNNr)FNNNN)__name__ __module__ __qualname__strintboolrrr+r2ndarrayrrrCrZrorw __classcell__r r rr!r,sz  ( Krc s\eZdZdZdejdfdededeffdd Z dd ej d ed ed e efddZ Z S)AgglomerativeClusteringaAgglomerative clustering Parameters ---------- metric : {"cosine", "euclidean", ...}, optional Distance metric to use. Defaults to "cosine". Hyper-parameters ---------------- method : {"average", "centroid", "complete", "median", "single", "ward"} Linkage method. threshold : float in range [0.0, 2.0] Clustering threshold. min_cluster_size : int in range [1, 20] Minimum cluster size rFrrrcs<tj|||dtdd|_tgd|_tdd|_dS)N)rrrgg@)averagecentroidcompletemediansinglewardweightedr&)rrr thresholdr methodr min_cluster_sizerrr r!r-s z AgglomerativeClustering.__init__Nr,r$r%r#csj\}}t|jtdtd|}|dkrtjdtjdS|jdkrS|j dvrStj dddtj j d d d Wd n1sEwYt |j d d}n t |j |jd}t||jdddtjd d\} } | | |k} t| } | |kr|}n| |kr|}|d ur%| |kr%t|} t|d| d d df<|d}d}tt|d d df|jD]?}| |df}||krqt| |dddtjd d\} } | | |k} t| } t| |t||kr|}| }| |krnq||kr%t| |dddtjd d\} } | | |k} t| } td| d|d| dkr2dd d <S| | |k}t|dkrAStfdd| D}tfdd|D}t|||jd}ttj|ddD]\}}| |||k<qltjd d\}S)a@ Parameters ---------- embeddings : (num_embeddings, dimension) array Embeddings min_clusters : int Minimum number of clusters max_clusters : int Maximum number of clusters num_clusters : int, optional Actual number of clusters. Default behavior is to estimate it based on values provided for `min_clusters`, `max_clusters`, and `threshold`. Returns ------- clusters : (num_embeddings, ) array 0-indexed cluster indices. r&g?)r&rGr)rrrignore)divideinvalidTrqN euclidean)rrdistance) criterion) return_countsr1z Found only z& clusters. Using a smaller value than z# for `min_cluster_size` might help.rcr_r`ra)rdlarge_krWr,r r!rgrhz3AgglomerativeClustering.cluster..cr_r`ra)rdsmall_krr r!rgrhrir/)return_inverse)rLr)rr(roundr2rsuint8rrerrstatelinalgnormr r runiquer8copyarangeargsortabsprintrjrrOargmin)rr,r$r%r#r"rvr dendrogramcluster_uniquecluster_countslarge_clustersnum_large_clusters _dendrogrambest_iterationbest_num_large_clusters iterationnew_cluster_sizesmall_clusterslarge_centroidssmall_centroidscentroids_cdistrrr rr!rtAs    &          zAgglomerativeClustering.clusterr)ryrzr{__doc__r2infr|r}r~rrrrtrr r rr!rs,rc @sLeZdZdZ    d deejdeedeedee dejf dd Z dS) OracleClusteringzOracle clusteringNr,r-fileframesr.c s\|jj\}}}|j} t|| |d} | |d<| jj\} } } |jdddt|| f}| jdddt|| f} dtj||ftjd}t||| f}t t || D]-\}\}}t |tj |\} ^}} t |D]\}}|durrqi||||f<d||||f<qiqS|dur||dfS|j ||d\}}|||ftfdd t| D}|||fS) aApply oracle clustering Parameters ---------- embeddings : (num_chunks, num_speakers, dimension) array, optional Sequence of embeddings. When provided, compute speaker centroids based on these embeddings. segmentations : (num_chunks, num_frames, num_speakers) array Binary segmentations. file : AudioFile frames : SlidingWindow Returns ------- hard_clusters : (num_chunks, num_speakers) array Hard cluster assignment (hard_clusters[c, s] = k means that sth speaker of cth chunk is assigned to kth cluster) soft_clusters : (num_chunks, num_speakers, num_clusters) array Soft cluster assignment (the higher soft_clusters[c, s, k], the most likely the sth speaker of cth chunk belongs to kth cluster) centroids : (num_clusters, dimension), optional Clusters centroids if `embeddings` is provided, None otherwise. )roracle_segmentationsNrFrGg?rpcr_r`rarcrer r!rg&rhz-OracleClustering.__call__..)r4rLsliding_windowrr)r2rMrNrsrOrPrnewaxisrCrjr:)rr,r-rrrurQ num_framesrRwindowrrvoracle_num_framesr#rSrDrT segmentationoracle permutationjir[r\rmr rer!rwsL      zOracleClustering.__call__rx) ryrzr{rrr2rrrrrwr r r r!rs"rc@seZdZeZeZdS) ClusteringN)ryrzr{rrr r r r!r/sr)%rr;enumrtypingrrnumpyr2einopsr pyannote.corerrpyannote.pipelinerpyannote.pipeline.parameterr r r scipy.cluster.hierarchyr r scipy.optimizerscipy.spatial.distancerpyannote.audio.core.iorpyannote.audio.pipelines.utilsr pyannote.audio.utils.permutationrrrrrr r r r!s*        p@U