o ߥi9 @stdZddlZddlmZmZmZddlZddlm Z m Z ddl m Z ee ejfZehdZdZd Zd eed efd d Zdee efd efddZdeedededed eef ddZdedededed ef ddZded efddZddZdd Zded efd!d"Zdee effd#d$Zd%d&Zd'ed efd(d)Z d*ed e fd+d,Z!d-d.Z"d/d0Z#d1d2Z$d3d4Z%dS)5z+Feature processing logic for multimer data N)IterableListMutableMapping) msa_pairingresidue_constants)correct_template_restypes>!msaaatypesym_idasym_idnum_symasym_lenmem_peakmsa_maskseq_mask bert_mask entity_id msa_chains queue_size resolution seq_length entity_mask all_atom_mask deletion_mean num_templates residue_indexnum_alignmentsdeletion_matrixtemplate_aatypecluster_bias_maskall_atom_positionstemplate_sum_probsassembly_num_chainsall_chains_entity_idstemplate_all_atom_maskall_crops_all_chains_masktemplate_all_atom_positionsall_crops_all_chains_positions all_crops_all_chains_residue_idsichainsreturncCs&tttdd|D}|dkS)z@Checks if a list of chains represents a homomer/monomer example.cSs&g|]}t|d|ddkqS)rr)npunique).0chainr1k/home/ubuntu/.local/lib/python3.10/site-packages/modelscope/models/science/unifold/data/process_multimer.py Isz*_is_homomer_or_monomer..r)lenr-r. concatenate)r+num_unique_chainsr1r1r2_is_homomer_or_monomerDs r7all_chain_featurescCs\t||}t| }|rtj|d}t|}t|t|td}tj||td}t |}|S)zRuns processing on features to augment, pair and merge. Args: all_chain_features: A MutableMap of dictionaries of features for each chain. Returns: A dictionary of features. )r+ msa_crop_sizepair_msa_sequences max_templates)np_chains_listr;r<) process_unmerged_featuresr7rcreate_paired_featuresdeduplicate_unpaired_sequences crop_chains MSA_CROP_SIZE MAX_TEMPLATESmerge_chain_features process_final)r8r=r; np_exampler1r1r2pair_and_mergePs.  rG chains_listr:r;r<cCs,g}|D]}t||||d}||q|S)akCrops the MSAs for a set of chains. Args: chains_list: A list of chains to be cropped. msa_crop_size: The total number of sequences to crop from the MSA. pair_msa_sequences: Whether we are operating in sequence-pairing mode. max_templates: The maximum templates to use per chain. Returns: The chains cropped. r9)_crop_single_chainappend)rHr:r;r<cropped_chainsr0 cropped_chainr1r1r2rAus rAr0cCs|d}|r@|d}t||d}|dd|ddf}ttj|tjkdd}t||}t||d} t|| }nt||}d |voK|} | r[|d jd} t| |} |D]E} | d d}|tj vrz|| d| ddf|| <q]|tj vrd | vr|r|| d|ddf|| <q]|| d|ddf|| <q]tj |tj d |d<| rtj | tj d |d <|rtj |tj d |d<|S) z'Crops msa sequences to `msa_crop_size`.rnum_alignments_all_seq msa_all_seqNraxisrr_all_seqdtyper) r-minimumsumanyr MSA_GAP_IDXmaximumshapesplitTEMPLATE_FEATURES MSA_FEATURESasarrayint32)r0r:r;r<msa_sizemsa_size_all_seqmsa_crop_size_all_seqrOnum_non_gapped_pairsmax_msa_crop_sizeinclude_templatesrtemplates_crop_sizekk_splitr1r1r2rIsN        rIrFcCst|}t|}t|}|S)zCFinal processing steps in data pipeline, after merging and pairing.)_make_seq_mask_make_msa_mask_filter_featuresrFr1r1r2rEsrEcCs|ddktj|d<|S)Nrrr)astyper-float32rlr1r1r2risricCsDtj|dtjd|d<|ddktj}|d|d9<|S)z:Mask features are all ones, but will later be zero-padded.r rSrrrN)r- ones_likeint8rm)rFrr1r1r2rjsrjcCsdd|DS)z4Filters features of example to only those requested.cSsi|] \}}|tvr||qSr1)REQUIRED_FEATURES)r/rgvr1r1r2 sz$_filter_features..)itemsrlr1r1r2rksrkcCst|}|D]R}d|vrtj|dtjd|d<d|vr*tj|dtjd|d<tj|ddd|d<d |vrQtj|d }||d <tt |j d g|d <t||d <q|D]}|ddk tj |d<q[dS)z;Postprocessing stage for per-chain features before merging.deletion_matrix_intrSrdeletion_matrix_int_all_seqdeletion_matrix_all_seqrrPrr!r rr#rrN) r4r-r^poprnmeanrSTANDARD_ATOM_MASKzeroslistrZrmr_)r8 num_chainschain_featuresrr1r1r2r>s<    r>cCsXtd|ftjtd|ddftjtdtjtd|dftjdS)Nr%rx)rr)rr'r"r%)r-r|rmint64rn)n_resr1r1r2empty_template_feats s rmonomer_featurescCs|djddkr|t|djdi}hd}|D]S\}}||vr2tj|d|jd}n1|dkrBtj|ddtj }n!|dkrR|jddkrQt |}n|dkrYd }n |d krc|tj }| d rn|tj }|||<qd |vr|d tj |d <|d|S)z;Reshapes and modifies monomer features for multimer models.rrr >sequencer domain_namerrSrPtemplate_all_atom_masksr%r _maskrurr")rZupdaterrtr-r^rTargmaxrmr_ruint8endswithrnry)r convertedunnecessary_leading_dim_feats feature_namefeaturer1r1r2convert_monomer_featuress@    rnumcCsd|dkr td|d|d}g}|dkr-|t|dtd|dd}|dksd|S)a`Encodes a number as a string, using reverse spreadsheet style naming. Args: num: A positive integer. Returns: A string that encodes the positive integer using reverse spreadsheet style, naming e.g. 1 = A, 2 = B, ..., 27 = AA, 28 = BA, 29 = CA, ... This is the usual way to encode chain IDs in mmCIF files. rz$Only positive integers allowed, got .rA) ValueErrorrJchrordjoin)routputr1r1r2int_id_to_str_id@s   rc Csi}tt}|D]#}d|vsJt|d}||vr#t|d||<||||q g}d}|D]D\}}t|} t|ddD]5\} }|d} |t | |d<| t | |d<|t | |d<| t | |d<|d7}||qCq5|S) aAdd features to distinguish between chains. Args: all_chain_features: A dictionary which maps chain_id to a dictionary of features for each chain. Returns: all_chain_features: A dictionary which maps strings of the form `_` to the corresponding chain features. E.g. two chains from a homodimer would have keys A_1 and A_2. Two chains from a heterodimer would have keys A_1 and B_1. rr)startrr r rr ) collections defaultdictr}strr4rJrt enumerater-ones) r8seq_to_entity_idgrouped_chainsrseqnew_all_chain_featureschain_idrgroup_chain_featuresr r rr1r1r2add_assembly_featuresVs,     rcCsjt|}|djd}||kr3dD]}t||d||fdf||<qt|dd||ff|d<|S)Nr r)r rrrr)rrr )dictrZr-pad)rF min_num_seqnum_seqfeatr1r1r2pad_msa}s  rcCs:t|d}gd}|D]}||vr||d||<q |S)Ni)rr#rrrr)rreshape)rF no_dim_keysrgr1r1r2 post_processs rc Csrtdd|D}g}t|D]\}}t||vr||qtj|||gdd}tj|||gdd} || fS)NcSsg|]}t|qSr1)tuple)r/mr1r1r2r3szmerge_msas..rrP)setrrrJr-r5) r del_matnew_msa new_del_mat cur_msa_setnew_rowsisret_msa ret_del_matr1r1r2 merge_msass  r)&__doc__rtypingrrrnumpyr-&modelscope.models.science.unifold.datarrutilsrrndarray FeatureDict frozensetrqrCrBboolr7rGintrArIrErirjrkr>rrrrrrrr1r1r1r2s`   %  %  4 # *'