o pim@s(dZddlmZddlZddlmZddlmZddl m Z ddl m Z ddl mZdd lmZdd lmZmZdd lmZdd lmZmZmZdd lmZmZddlmZddlm Z ddl!m"Z"GdddeZ#Gddde#Z$Gddde#Z%Gddde#Z&Gddde#Z'GdddeZ(dS)zClustering pipelines)EnumN) rearrange) AudioFile)PLDA)oracle_segmentation) permutate) cluster_vbx) SlidingWindowSlidingWindowFeature)Pipeline) CategoricalIntegerUniform)fclusterlinkage)linear_sum_assignment)cdist)KMeansc seZdZ  d dedeffdd Z   d!ded edBd edBd edBfd d Z  d"dej de dBde de ej ej ej ffddZ dej dej fddZ d#dej dej dej dej def ddZ    d$dej de dBd edBd edBd edBdej f ddZZS)%BaseClusteringcosineFmetricconstrained_assignmentcst||_||_dS)N)super__init__rrselfrr __class__W/home/ubuntu/.local/lib/python3.10/site-packages/pyannote/audio/pipelines/clustering.pyr-s  zBaseClustering.__init__Nnum_embeddings num_clusters min_clusters max_clusterscCsp|p|pd}tdt||}|p|p|}tdt||}||kr-td|dd|dd||kr3|}|||fS)NzQmin_clusters must be smaller than (or equal to) max_clusters (here: min_clusters=gz and max_clusters=z).)maxmin ValueError)rr r!r"r#rrrset_num_clusters6s   zBaseClustering.set_num_clusters皙? embeddings segmentationsmin_active_ratioreturnc Cs~|jj\}}}tj|jddddk}tj|j|dd}|||k}tjt|dd} t|| \} } || | f| | fS)aFilter embeddings before clustering Embeddings that are removed: * NaN embeddings * embeddings speaking less than `min_active_ratio` times the chunk duration Parameters ---------- embeddings : (num_chunks, num_speakers, dimension) array Sequence of embeddings. segmentations : (num_chunks, num_frames, num_speakers) array Binary segmentations. min_active_ratio : float, optional Minimum active ratio for a speaker to be considered active during clustering. Returns ------- filtered_embeddings : (num_embeddings, dimension) array chunk_idx : (num_embeddings, ) array speaker_idx : (num_embeddings, ) array Taxiskeepdimsr$r1)datashapenpsumanyisnanwhere) rr+r,r-_ num_framessingle_active_masknum_clean_framesactivevalid chunk_idx speaker_idxrrrfilter_embeddingsMs z BaseClustering.filter_embeddings soft_clustersc Cs~tj|t|d}|j\}}}dtj||ftjd}t|D]\}}t|dd\}} t|| D] \} } | ||| f<q1q |S)N)nandtypeT)maximize) r6 nan_to_numnanminr5onesint8 enumeraterzip) rrD num_chunks num_speakersr! hard_clustersccostspeakersclustersskrrrconstrained_argmaxs z!BaseClustering.constrained_argmaxtrain_chunk_idxtrain_speaker_idxtrain_clusters constrainedcstd}|j\}}} |||ftfddt|D} ttt|d| |jdd||d} d| } |r?|| } ntj | dd } | | | fS) aAssign embeddings to the closest centroid Cluster centroids are computed as the average of the train embeddings previously assigned to them. Parameters ---------- embeddings : (num_chunks, num_speakers, dimension)-shaped array Complete set of embeddings. train_chunk_idx : (num_embeddings,)-shaped array train_speaker_idx : (num_embeddings,)-shaped array Indices of subset of embeddings used for "training". train_clusters : (num_embedding,)-shaped array Clusters of the above subset constrained : bool, optional Use constrained_argmax, instead of (default) argmax. Returns ------- soft_clusters : (num_chunks, num_speakers, num_clusters)-shaped array hard_clusters : (num_chunks, num_speakers)-shaped array centroids : (num_clusters, dimension)-shaped array Clusters centroids r$c"g|] }tj|kddqSrr3r6mean.0rXr\train_embeddingsrr z4BaseClustering.assign_embeddings..c s d -> (c s) dr(c s) k -> c s krSrWr/r3) r6r&r5vstackrangerrrrYargmax)rr+rZr[r\r]r!rPrQ dimension centroids e2k_distancerDrRrrdrassign_embeddingss.#      z BaseClustering.assign_embeddingscKs|j||d\}}} |j\} } |j| |||d\}}}|dkrD|j\} } } tj| | ftjd}t| | df}tj|ddd}|||fS|j||||d }|j ||| ||j d \}}}|||fS) aApply clustering Parameters ---------- embeddings : (num_chunks, num_speakers, dimension) array Sequence of embeddings. segmentations : (num_chunks, num_frames, num_speakers) array Binary segmentations. num_clusters : int, optional Number of clusters, when known. Default behavior is to use internal threshold hyper-parameter to decide on the number of clusters. min_clusters : int, optional Minimum number of clusters. Has no effect when `num_clusters` is provided. max_clusters : int, optional Maximum number of clusters. Has no effect when `num_clusters` is provided. Returns ------- hard_clusters : (num_chunks, num_speakers) array Hard cluster assignment (hard_clusters[c, s] = k means that sth speaker of cth chunk is assigned to kth cluster) soft_clusters : (num_chunks, num_speakers, num_clusters) array Soft cluster assignment (the higher soft_clusters[c, s, k], the most likely the sth speaker of cth chunk belongs to kth cluster) centroids : (num_clusters, dimension) array Centroid vectors of each cluster r,)r!r"r#r/rGr$rTr0)r"r#r!)r]) rCr5r)r6zerosrMrLraclusterrrr)rr+r,r!r"r#kwargsrerZr[r r;rPrQrRrDrpr\rrr__call__s>&     zBaseClustering.__call__rFNNN)Nr*)FNNNN)__name__ __module__ __qualname__strboolrintr)r6ndarrayr floattuplerCrYrrrw __classcell__rrrrr,sz   2 Krc sneZdZUdZdZeed<  ddedeffdd Z dd e j d e d Bd e d Bd e d BfddZ Z S)AgglomerativeClusteringaAgglomerative clustering Parameters ---------- metric : {"cosine", "euclidean", ...}, optional Distance metric to use. Defaults to "cosine". Hyper-parameters ---------------- method : {"average", "centroid", "complete", "median", "single", "ward"} Linkage method. threshold : float in range [0.0, 2.0] Clustering threshold. min_cluster_size : int in range [1, 20] Minimum cluster size Fexpects_num_clustersrrrcs:tj||dtdd|_tgd|_tdd|_dS)Nrrgg@)averagecentroidcompletemediansinglewardweightedr$)rrr thresholdr methodr min_cluster_sizerrrrr8s z AgglomerativeClustering.__init__Nr+r"r#r!csj\}}t|jtdtd|}|dkrtjdtjdS|jdkrS|j dvrStj dddtj j d d d Wd n1sEwYt |j d d}n t |j |jd}t||jdddtjd d\} } | | |k} t| } | |kr|}n| |kr|}|d ur%| |kr%t|} t|d| d d df<|d}d}tt|d d df|jD]?}| |df}||krqt| |dddtjd d\} } | | |k} t| } t| |t||kr|}| }| |krnq||kr%t| |dddtjd d\} } | | |k} t| } td| d|d| dkr2dd d <S| | |k}t|dkrAStfdd| D}tfdd|D}t|||jd}ttj|ddD]\}}| |||k<qltjd d\}S)a@ Parameters ---------- embeddings : (num_embeddings, dimension) array Embeddings min_clusters : int Minimum number of clusters max_clusters : int Maximum number of clusters num_clusters : int, optional Actual number of clusters. Default behavior is to estimate it based on values provided for `min_clusters`, `max_clusters`, and `threshold`. Returns ------- clusters : (num_embeddings, ) array 0-indexed cluster indices. r$g?)r$rGr)rrrignoredivideinvalidTr0N euclideanrrdistance criterion) return_countsr/z Found only z& clusters. Using a smaller value than z# for `min_cluster_size` might help.rcr^r_r`)rclarge_krVr+rrrfrgz3AgglomerativeClustering.cluster..cr^r_r`)rcsmall_krrrrfrgrir3return_inverse)r5r'rr&roundr6rtuint8rrerrstatelinalgnormrrruniquelencopyarangeargsortabsprintrlrrNargmin)rr+r"r#r!r r;r dendrogramcluster_uniquecluster_countslarge_clustersnum_large_clusters _dendrogrambest_iterationbest_num_large_clusters iterationnew_cluster_sizesmall_clusterslarge_centroidssmall_centroidscentroids_cdistrrrrrruJs    &          zAgglomerativeClustering.clusterrxryr{r|r}__doc__rr__annotations__r~rr6rrrurrrrrr$s,  rc sheZdZUdZdZeed< ddeffdd Z   dd e j d e dBd e dBd e dBfd dZ Z S)KMeansClusteringzKMeans clustering Parameters ---------- metric : {"cosine", "euclidean"}, optional Distance metric to use. Defaults to "cosine". Hyper-parameters ---------------- None Trrrcs*|dvr td|dtj|ddS)N)rrzUnsupported metric: z". Must be 'cosine' or 'euclidean'.ri)r(rr)rrrrrrs  zKMeansClustering.__init__Nr+r"r#r!cCs|durtd|j\}}||krtj|tjdS|jdkr@tjddd|tjj|ddd }Wdn1s;wYt |d d d d  |S)aYPerform KMeans clustering Parameters ---------- embeddings : (num_embeddings, dimension) array Embeddings num_clusters : int, optional Expected number of clusters. Returns ------- clusters : (num_embeddings, ) array 0-indexed cluster indices. Nz `num_clusters` must be provided.rGrrrrTr0r*F n_clustersn_init random_statecopy_x) r(r5r6rint32rrrrr fit_predict)rr+r"r#r!r r;rrrrus  zKMeansClustering.cluster)rryrrrrrrs&  rc s~eZdZUdZeed<  ddededeffdd Z dd e j d e d Bd e d Bde d Bde d Bde j f ddZ ZS) VBxClusteringFrrTpldarrcs>tj||d||_tdd|_tdd|_tdd|_dS)Nrg?g?g{Gz?g.@)rrrrrFaFb)rrrrrrrr+s  zVBxClustering.__init__Nr+r,r!r"r#r.c s|j}|j||d\}}jddkr9|j\} } }tj| | ftjd} t| | df} tjddd} | | | fStjj ddd}t |dd d }t ||j d d d}tj |dd \}}|}t|||jj|j|jdd\}}|j\} } }|dd|dkf}|jd||jdddj} | j\}}||kr|}n||kr|}|r||krd}t|dddd|tfddt|D} ttt|d| |jdd| | d}d|} |r| d}|| |jddk<|| } ntj | dd} | | | } | | | fS)Nrsrr/rGr$Tr0rrrrrrr)rrmaxItersgHz>r)r2Frrrcr^r_r`rbkmeans_clustersrerrrfrgz*VBxClustering.__call__..rhrirjrk?r3)!rrCr5r6rtrMrLrarrrrrrrrphirrTreshaper7rrrlrmrrrr'r4rYrn)rr+r,r!r"r#rvrr;rPrQrRrDrptrain_embeddings_normedr ahc_clustersfeaqsproWauto_num_clustersrqconstrrrrw<s      "       zVBxClustering.__call__)rTrz)r{r|r}rrrrr~rr6rr rrwrrrrrr&s8 rc @sZeZdZUdZdZeed<    d dejdBde dBde dBde dBd ejf d d Z dS) OracleClusteringzOracle clusteringTrNr+r,fileframesr.c s\|jj\}}}|j} t|| |d} | |d<| jj\} } } |jdddt|| f}| jdddt|| f} dtj||ftjd}t||| f}t t || D]-\}\}}t |tj |\} ^}} t |D]\}}|durrqi||||f<d||||f<qiqS|dur||dfS|j ||d\}}|||ftfdd t| D}|||fS) aApply oracle clustering Parameters ---------- embeddings : (num_chunks, num_speakers, dimension) array, optional Sequence of embeddings. When provided, compute speaker centroids based on these embeddings. segmentations : (num_chunks, num_frames, num_speakers) array Binary segmentations. file : AudioFile frames : SlidingWindow Returns ------- hard_clusters : (num_chunks, num_speakers) array Hard cluster assignment (hard_clusters[c, s] = k means that sth speaker of cth chunk is assigned to kth cluster) soft_clusters : (num_chunks, num_speakers, num_clusters) array Soft cluster assignment (the higher soft_clusters[c, s, k], the most likely the sth speaker of cth chunk belongs to kth cluster) centroids : (num_clusters, dimension), optional Clusters centroids if `embeddings` is provided, None otherwise. )roracle_segmentationsNrFrGrrscr^r_r`rbrdrrrfrgz-OracleClustering.__call__..)r4r5sliding_windowrr'r6rLrMrtrNrOrnewaxisrCrlrm)rr+r,rrrvrPr<rQwindowrr;oracle_num_framesr!rRrDrS segmentationoracle permutationjirZr[rprrdrrwsL      zOracleClustering.__call__rz) r{r|r}rrrrr6rr rr rwrrrrrs$  rc@seZdZeZeZeZeZdS) ClusteringN)r{r|r}rrrrrrrrrs r))renumrnumpyr6einopsrpyannote.audio.core.iorpyannote.audio.core.pldarpyannote.audio.pipelines.utilsr pyannote.audio.utils.permutationrpyannote.audio.utils.vbxr pyannote.corer r pyannote.pipeliner pyannote.pipeline.parameterr r rscipy.cluster.hierarchyrrscipy.optimizerscipy.spatial.distancersklearn.clusterrrrrrrrrrrrs0           y@CzW