o ߥiL @s.dZddlZddlmZmZmZmZddlZddl Z ddl Z ddl mZddlmZedZdZd Zedddedddd Zd Zd Zd ZdZdeedeefddZdejdedejfddZdede jfddZde jdee e jffddZ!dee jdeee"fddZ#d eedee"ejffd!d"Z$d#ee"ejfdejfd$d%Z%d&d'd(ejd)e&dejfd*d+Z'd,ed-eed.e(defd/d0Z)deed1e"deefd2d3Z*deed.e(defd4d5Z+deedeefd6d7Z,d8edefd9d:Z-d-eed.e(d1e"defd;d<Z.d=eedeefd>d?Z/dS)@z Pairing logic for multimer data N)DictIterableListSequence) NumpyDict)restypes_with_x_and_gap-g?g?) msa_all_seqmsa_mask_all_seqdeletion_matrix_all_seqdeletion_matrix_int_all_seqmsamsa_maskdeletion_matrixdeletion_matrix_int)rrrr) residue_indexaatypeall_atom_positions all_atom_maskseq_maskbetween_segment_residueshas_alt_locations has_hetatomsasym_id entity_idsym_id entity_mask deletion_meanprediction_atom_maskliterature_positionsatom_indices_to_group_indicesrigid_group_default_framenum_sym)template_aatypetemplate_all_atom_positionstemplate_all_atom_mask)num_alignments seq_lengthchainsreturnc Cst|}|d}t|dkr|Sg}t|}t|}t|D]>\}}dd|D}|D]}|drIt|||} | |dd|f||<q/t t|dd|f|d<| |q |S)aReturns the original chains with paired NUM_SEQ features. Args: chains: A list of feature dictionaries for each chain. Returns: A list of feature dictionaries with sequence features including only rows to be paired. rcSsi|] \}}d|vr||qS)_all_seq).0kvr-r-f/home/ubuntu/.local/lib/python3.10/site-packages/modelscope/models/science/unifold/data/msa_pairing.py \z*create_paired_features..r,Nnum_alignments_all_seq) listkeyslenpair_sequencesreorder_paired_rows enumerateitemsendswith pad_featuresnpasarrayappend) r) chain_keysupdated_chains#paired_chains_to_paired_row_indices paired_rows chain_numchain new_chain feature_name feats_paddedr-r-r1create_paired_featuresGs.    rJfeaturerHcCsn|jttjks J|dvr"|jd}t|td|g|j}n |dkr*dg}n|Stj||gdd}|S)a~Add a 'padding' row at the end of the features list. The padding row will be selected as a 'paired' row in the case of partial alignment - for the chain that doesn't have paired alignment. Args: feature: The feature to be padded. feature_name: The name of the feature to be padded. Returns: The feature with an additional padding row. )r r r r rmsa_species_identifiers_all_seqraxis)dtyper>str_shapeMSA_PAD_VALUESones concatenate)rKrHnum_respaddingrIr-r-r1r=is  r=chain_featurescCsz|d}|d}tj|d|kddtt|}tj|dkddtt|}t|dtt|d||d}|S) z9Makes dataframe with msa features needed for msa pairing.r rNrNrL)msa_species_identifiersmsa_rowmsa_similaritygap)r>sumfloatr7pd DataFramearange)rX chain_msa query_seqper_seq_similarity per_seq_gapmsa_dfr-r-r1 _make_msa_dfs   rirhcCs$i}|dD]\}}|||<q|S)z>Creates mapping from species to msa dataframe of that species.r[)groupby)rhspecies_lookupspecies species_dfr-r-r1_create_species_dicts rnthis_species_msa_dfscCs|g}dd|D}t|}dd}|D]}|dur(||}|jjd|j}ndg|}||qtt|}|S)a7Finds MSA sequence pairings across chains based on sequence similarity. Each chain's MSA sequences are first sorted by their sequence similarity to their respective target sequence. The sequences are then paired, starting from the sequences most similar to their target sequence. Args: this_species_msa_dfs: a list of dataframes containing MSA features for sequences for a specific species. Returns: A list of lists, each containing M indices corresponding to paired MSA rows, where M is the number of chains. cSsg|] }|durt|qSN)r7r.rmr-r-r1 s z6_match_rows_by_sequence_similarity..cSs|jddddS)Nr]rF)rO ascending) sort_values)xr-r-r1sort_by_similaritysz>_match_rows_by_sequence_similarity..sort_by_similarityNrY) r>minr\ilocvaluesr@r5array transpose)roall_paired_msa_rowsnum_seqs take_num_seqsrvrmspecies_df_sortedmsa_rowsr-r-r1"_match_rows_by_sequence_similaritys   rexamplesc Cs<t|}g}t}|D]}t|}t|}|||t|q t|}|dt t|t g}ddt |D}t t|t g||<|D]H} | sOqJg} d} |D]}| |vrg| || | d7} qU| dqU| dkrrqJt t dd| Dd krqJt| } || || | qJd d|D}|S) z7Returns indices for paired MSA sequences across chains.rMcSsi|]}|gqSr-r-)r.r/r-r-r1r2sz"pair_sequences..rrNcSs g|] }t|tjrt|qSr-) isinstancerarbr7rqr-r-r1rrs  z"pair_sequences..iXcSsi|] \}}|t|qSr-)r>rz)r. num_examplespaired_msa_rowsr-r-r1r2s )r7setrirnr@updatesortedremover>zerosintrangeanyrzrextendr;) rrall_chain_species_dictcommon_speciesrXrh species_dictr|all_paired_msa_rows_dictrlrospecies_dfs_presentrr-r-r1r8sT      r8rcCsZg}t|ddD]}||}ttdd|D}t|}|||qt|S)aCreates a list of indices of paired MSA rows across chains. Args: all_paired_msa_rows_dict: a mapping from the number of paired chains to the paired indices. Returns: a list of lists, each containing indices of paired MSA rows across chains. The paired-index lists are ordered by: 1) the number of chains in the paired alignment, i.e, all-chain pairings will come first. 2) e-values T)reversecSsg|] }t|tjqSr-)r>prodastypefloat64)r.rowsr-r-r1rrr3z'reorder_paired_rows..)rr>absrzargsortr)rr| num_pairingsrDpaired_rows_productpaired_rows_sort_indexr-r-r1r9s   r9g) pad_valuearrsrcGsBdd|D}dtjj|}tjj|}||||j7}|S)z@Like scipy.linalg.block_diag but with an optional padding value.cSsg|]}t|qSr-)r> ones_liker.rur-r-r1rr!szblock_diag..r)scipylinalg block_diagrrP)rr ones_arrs off_diag_maskdiagr-r-r1rs  r np_examplenp_chains_listpair_msa_sequencesc Cstj|djdtjd|d<tj|djdtjd|d<|sSg}|D]}t|djd}d|d<||q$t||d<d d |D}t|d di|d <|St|djd|d<d|dd<d d |D}dd |D}t|d di}tj|dd} tj| |gdd|d <|S)z?Adds features that need to be computed/recomputed post merging.rrrPr(rr'rcluster_bias_maskcS"g|] }tj|djtjdqSrrr>rTrRint8rr-r-r1rr?z._correct_post_merged_feats..r bert_maskcSrrrrr-r-r1rrIrcSr)r rrrr-r-r1rrLsrN)r>r?rRint32rr@rUr) rrrcluster_bias_masksrFmask msa_masksmsa_masks_all_seqmsa_mask_block_diagr r-r-r1_correct_post_merged_feats(s@     r max_templatescCsh|D]/}|D](\}}|tvr0t|j}||jd|d<dd|D}tj||dd||<qq|S)a0For each chain pad the number of templates to a fixed size. Args: chains: A list of protein chains. max_templates: Each chain will be padded to have this many templates. Returns: The list of chains, updated to have template features padded to max_templates. rcSsg|]}d|fqS)rr-)r.pr-r-r1rriz"_pad_templates..constant)mode)r;TEMPLATE_FEATURESr> zeros_likerRpad)r)rrFr/r0rWr-r-r1_pad_templatesXs  rc s\i}|dD]fdd|D}dd}|tvry|s"dvr@tj|dd|<|dkr?t|jddd|d <qt|d ti|<|dkrxg}t |D]\}}|jd}t||d } | | qUt|dd|d <q|t vrtj|dd|<q|t vrtj|dd|<q|t vrt|tj|<q|d|<q|S) aYMerge features from multiple chains. Args: chains: A list of feature dictionaries that we want to merge. pair_msa_sequences: Whether to concatenate MSA features along the num_res dimension (if True), or to block diagonalize them (if False). Returns: A feature dictionary for the merged example. rcsg|]}|qSr-r-rrHr-r1rr|rz8_merge_features_from_multiple_chains..r,rrNrrYmsa_chains_all_seqrr+ msa_chains)split MSA_FEATURESr>rUrTrRreshaperrSr:r@ SEQ_FEATURESrCHAIN_FEATURESr_rr) r)rmerged_examplefeatsfeature_name_splitrifeat cur_shapevalsr-rr1$_merge_features_from_multiple_chainsnsN      rcCsdtt}|D]}|dd}|||qg}t|D] }||}||qdd|D}|S)aUMerge all identical chains, making the resulting MSA dense. Args: chains: An iterable of features for each chain. Returns: A list of feature dictionaries. All features with the same entity_id will be merged - MSA features will be concatenated along the num_res dimension - making them dense. rrcSsg|]}t|ddqS)Tr)r)r.r)r-r-r1rrs z-_merge_homomers_dense_msa..) collections defaultdictr5r@r)r) entity_chainsrFrgrouped_chainsr-r-r1_merge_homomers_dense_msas    rexamplec Cstd}|D]8}||vr>||}||d}z tj||gdd}Wnty9}z td||j|j|j|d}~ww|||<qtj|djdtjd|d <|S) z.Merges paired and block-diagonalised features.)rr,rrNzconcat failed.Nrrr')rr>rU ExceptionrR __class__rzr)rfeaturesrHr feat_all_seq merged_featexr-r-r1)_concatenate_paired_and_unpaired_featuress0   rcCs>t||d}t|}t|dd}|rt|}t|||d}|S)aAMerges features for multiple chains to single FeatureDict. Args: np_chains_list: List of FeatureDicts for each chain. pair_msa_sequences: Whether to merge paired MSAs. max_templates: The maximum number of templates to include. Returns: Single FeatureDict for entire complex. )rFr)rrr)rrrrr)rrrrr-r-r1merge_chain_featuress r np_chainsc Cs|d}t}i}|D]}t|dd}||vrntdd|dD}g}t|dD]\}} | |vr<||q-i} |D](} | |vri|rR|| || | <qAt|| j} d| d<t j | || j d| | <qA| ||<||D] } ||| || <qrt j |djdt j d|d<q |S) z=Removes unpaired sequences which duplicate a paired sequence.rrcss|]}|VqdSrp)tobytes)r.sr-r-r1 sz1deduplicate_unpaired_sequences..r rrr')r6rrrr:rr@r5rRr>rrPrzr) r feature_names msa_featurescache_msa_featuresrFr sequence_set keep_rowsrow_numseqnew_msa_featuresrH new_shaper-r-r1deduplicate_unpaired_sequencess@        r)0__doc__rtypingrrrrnumpyr>pandasra scipy.linalgrdata_opsrresidue_constantsrindex MSA_GAP_IDXSEQUENCE_GAP_CUTOFFSEQUENCE_SIMILARITY_CUTOFFrSrrrrrJndarraystrr=rbribytesrnrrr8r9r`rboolrrrrrrrr-r-r-r1s     "  )6    0  ,