o ॵiT @sddlZddlZddlZddlZddlZddlZddlZddlZddlZddl m Z ddl m Z m Z mZmZmZmZmZddlmZddlZddlZddlZddlZddlmZddlmZddlmZmZddl m!Z!dd l"m#Z#dd l$m%Z%m&Z&m'Z'dd l(m)Z)dd l*m+Z+dd l,m-Z-ddl.m/Z/ddl0m1Z1dgZ2dZ3dZ4    d%deee5ee5ffddZ6 d&deee5e5fde7de e5e ffddZ8de5de5de5de e5e ffdd Z9e/j:e1j;ejd#kr e=Z?d$Z@e?Ae@\ZBZCddlDZDeDEdSdS)'N)Path)AnyDictListOptionalSequenceTupleUnion)result)tqdm) Preprocessors)proteinresidue_constants) PDB_CHAIN_IDS)compress_features)parserspipeline templates)hhsearch)divide_multi_chains) Preprocessor) PREPROCESSORS)FieldsUniFoldPreprocessorzM{l_bar}{bar}| {n_fmt}/{total_fmt} [elapsed: {elapsed} remaining: {remaining}]https://api.colabfold.comTFreturnc* sn|rdnddAfdd }fdd}fdd }t|tr"|gn|} d } |r.d } d }d }|} tj| s}|}|D]/}|}|d|d+|d,|d-f\}}} } t|}||vrg||<|||qWdn 1swYi}!|D]I\}"}#|d.|"}$tj|$st|$d/|#dd0}%td1d2|%d3|$d4td5|$d6|$d7td8|$d9|$|!|"<qi|D]b}&d:\}'}t|&d*d;d<K}|}|D]<}t|dkrqd=|vrL|d=d }d>}'|d?rj|'rjt|d+d}d }'|vrjg|<||q6Wdn 1s~wYq"fd@d|D|rg}(|D]})|)|!vr|(dq|(|!|)q|(}!|r|!fSS)BNz ticket/pairz ticket/msaecs||d}}|D]}|d|d|d7}|d7}qtjd||dd}z|}W|Sty=dd i}Y|Sw) N> /)qmode)datastatusERROR)requestspostjson ValueError)seqsr#Nnqueryseqresout)host_urlsubmission_endpoint]/home/ubuntu/.local/lib/python3.10/site-packages/modelscope/preprocessors/science/uni_fold.pysubmit3s"     zrun_mmseqs2..submitcs@td|}z|}W|Styddi}Y|Sw)Nz/ticket/r%r&)r'getr)r*)IDr0r1r2r4r5r%Es  zrun_mmseqs2..statuscsPtd|}t|d}||jWddS1s!wYdS)Nz/result/download/wb)r'r7openwritecontent)r8pathr0r1r9r4r5downloadMs "zrun_mmseqs2..downloadenvrFz/out_z.tar.gz)rTcg|] }|vr|qSr4append.0x) seqs_uniquer4r5 gszrun_mmseqs2..csg|] }|qSr4)indexrEr/)r,rGr4r5rHh)total bar_formatSUBMITr%)UNKNOWN RATELIMITrr&zTMMseqs2 API is giving errors. Please confirm your input is a valid protein sequence.z2If error persists, please try again an hour later. MAINTENANCEzIMMseqs2 API is undergoing maintenance. Please try again in a few minutes.id)rPRUNNINGPENDINGrU)r-COMPLETEz /pair.a3mz /uniref.a3mz"/bfd.mgnify30.metaeuk30.smag30.a3mcss|] }tj| VqdSN)osr>isfile)rEa3m_filer4r4r5 szrun_mmseqs2..z /pdb70.m8rr  z /templates_,z curl -s -L z /template/z | tar xzf - -C r!zcp z/pdb70_a3m.ffindex z/pdb70_cs219.ffindexztouch z/pdb70_cs219.ffdata)TNzutf-8)encodingTrcsg|] }d|qS)r)join)rEr-) a3m_linesr4r5rHrK)r) isinstancestrrYr>isdirmkdirrZlenr TQDM_BAR_FORMATset_descriptionrandomrandinttimesleep ExceptionupdaterCanytarfiler; extractall readlinesrstripsplitintitemsrdsystemreplace startswith)*rFprefixuse_env use_templates use_pairingr2r6r%r?r+r#r> tar_gz_fileREDOMs TIME_ESTIMATEpbarr1 sleep_timeerrorr8TIMEt a3m_filestar_gzrflineslinepMpdb_template_pathskTMPL TMPL_PATH TMPL_LINEr[update_Mtemplate_paths_r-r4)r,rer2rGr3r5 run_mmseqs2*s                    /3    $        rr query_sequencenum_tempc Cst|tr t|ntdd|D}d|}t|tjjdf}t|tjjf}tj |tjj }t |d|dddgt |d|ddgd g|t t |d|ddgd g|tj|gtjdd}|S) Ncss|]}t|VqdSrXrj)rEsr4r4r5r\s z$get_null_template..Ar none)dtype)template_all_atom_positionstemplate_all_atom_maskstemplate_sequencetemplate_aatypetemplate_domain_namestemplate_sum_probs)rfrgrjsumnpzerosrr atom_type_numsequence_to_onehotHHBLITS_AA_TO_IDtileencodearrayfloat32)rrlnoutput_templates_sequencetemplates_all_atom_positionstemplates_all_atom_maskstemplates_aatypetemplate_featuresr4r4r5get_null_templates.     rre template_pathcCsZtj|dddddd}tjd|dgd}||}tj|}|j||d}t |j S) Nz 2100-01-01rakalign) mmcif_dirmax_template_datemax_hitskalign_binary_pathrelease_dates_pathobsolete_pdbs_pathrz/pdb70) binary_path databases)rhits) rHhsearchHitFeaturizerrHHSearchr.rr parse_hhr get_templatesdictfeatures)rerrtemplate_featurizerhhsearch_pdb70_runnerhhsearch_result hhsearch_hitstemplates_resultr4r4r5 get_template s"    r) module_namec@seZdZddZdedededefddZd eed ededed edeeee ff d d Z ddZ de fdede eeefdedede dededeeeeeeeeeeeeeeefffddZde eeffddZdS)rcKsH|d|_|js d|_d|_d|_d|_d|_d|_tj|jdddS)Nsymmetry_groupiunifoldz./unifold-predictionsT)exist_ok)rMIN_SINGLE_SEQUENCE_LENGTHMAX_SINGLE_SEQUENCE_LENGTHMAX_MULTIMER_LENGTHjobnameoutput_dir_baserYmakedirs)selfcfgr4r4r5__init__"s zUniFoldPreprocessor.__init__input_sequence min_length max_lengthrcCs|tddd}ttj}t||s$tdt||dt ||kr6tdt |d|t ||krItdt |d|d |S) Nrz z0Input sequence contains non-amino acid letters: z<. AlphaFold only supports 20 standard amino acids as inputs.zInput sequence is too short: z# amino acids, while the minimum is zInput sequence is too long: z# amino acids, while the maximum is zr. You may be able to run it with the full Uni-Fold system depending on your resources (system memory, GPU memory).) translaterg maketransuppersetrrestypesissubsetr*rj)rrrrclean_sequenceaatypesr4r4r5clean_and_validate_sequence-s2       z/UniFoldPreprocessor.clean_and_validate_sequenceinput_sequencesrmax_multimer_lengthc Csg}|D]}|r|j|||d}||q|durH|dkrH|dr@|ddr@td|dt|dk}|||fStd|d t|dkrWtd |d dfSt|dkrtd d |D} | |krutd| d|dtdt|d|ddfStd)zGValidates and cleans input sequences and determines which model to use.)rrrNC1Cr zUsing UF-Symmetry with group zp. If you do not want to use UF-Symmetry, please use `C1` and copy the AU sequences to the count in the assembly.z,UF-Symmetry does not support symmetry group z2 currently. Cyclic groups (Cx) are supported only.zUsing the single-chain model.FcSsg|]}t|qSr4rrJr4r4r5rHesz6UniFoldPreprocessor.validate_input..z4The total length of multimer sequences is too long: z, while the maximum is z:. Please use the full AlphaFold system for long multimers.zUsing the multimer model with z sequences.TzLNo input amino acid sequence provided, please provide at least one sequence.) striprrCr} isnumericprintrjr*r) rrrrrr sequencesr is_multimertotal_multimer_lengthr4r4r5validate_inputCsX        z"UniFoldPreprocessor.validate_inputcCs"|dt|ddS)NrrR)hashlibsha1r hexdigest)rrFyr4r4r5add_hashts"zUniFoldPreprocessor.add_hashr rquery_seqs_unique result_dirmsa_moderhomooligomers_numr2cCs|dk}g} |rit|t|||d|d\} } | dur2tdt|D] } t|| } | | q#nLtdt|D].} | | dur\t| | | | || } t| ddkr[t|| } nt|| } | | q9ntdt|D] } t|| } | | qp|dkrg}d}t|D]\}}|dt||d |qnt|t|||d |d }t|d krt|t|||d|d }nd}g}td|D]}|dt||d |dd q||| fS) NMMseqs2T)rr2rrsingle_sequencerrrF)rr2r ) rrgjoinpathrangerjrrCr enumerate)rrrrrrrr2rra3m_lines_mmseqs2rrItemplate_featurerenumir/paired_a3m_linesr4r4r5get_msa_and_templatesws            z)UniFoldPreprocessor.get_msa_and_templatesr$c!st|tr|}t|dkr|dgdt|}d|}tdd|}||j ||j ||j |j |j |jd\}}}fddtt|D}|rWt|j||g}t||D] \}} ||| g7}q^gfdd|Dtdkrt|} nd} t|j d d } | d |Wdn1swYt|j} tj|j} d }d }|j| ||| d\}}}g}g}tD]\}} t|}tj| d|j d|t| d}t||}t|g}||}i|||}t |}tj| d!|}t"j#|t$%|ddd|&||rIt||}t|g}t |}tj| d!|} t"j#|t$%| ddd|&|q|||dS)Nrz\W+)rrrrrcs g|] }ddt|qS)> z seq)rg)rEii) target_idr4r5rHsz0UniFoldPreprocessor.__call__..crAr4rBrD)unique_sequencesr4r5rHs r z.fastawrrT)rrrrr z seq )sequence descriptionnum_resz{}.feature.pkl.gzr:)protocolz{}.uniprot.pkl.gz)r pair_featuresrr)'rfrgrrxrjrdresubrrrrrrrrrrzipr;r<rrYr>r rrrmake_sequence_featuresr parse_a3mmake_msa_featuresrformatpickledumpgzipGzipFilerC)!rr$ basejobnamerrr descriptionsrdesr/rrr output_dirrr unpaired_msa paired_msatemplate_resultsrpair_features_listidxchain_idsequence_features monomer_msa msa_featuresr feature_dictfeatures_output_path multimer_msarpair_feature_dictuniprot_output_pathr4)rrr5__call__s                    zUniFoldPreprocessor.__call__N)__name__ __module__ __qualname__rrgryrrrboolrrDEFAULT_API_SERVERr rrrrrr r3r4r4r4r5rsR   1 "  T__main__ڜLILNLRGGAFVSNTQITMADKQKKFINEIQEGDLVRSYSITDETFQQNAVTSIVKHEADQLCQINFGKQHVVCTVNHRFYDPESKLWKSVCPHPGSGISFLKKYDYLLSEEGEKLQITEIKTFTTKQPVFIYHIQVENNHNFFANGVLAHAMQVSI)TFFr)r )FrrloggingrYrrmrrtropathlibrtypingrrrrrrr unittestr r)numpyrr'torchr modelscope.metainfor &modelscope.models.science.unifold.datar r.modelscope.models.science.unifold.data.proteinr,modelscope.models.science.unifold.data.utilsr%modelscope.models.science.unifold.msarrr+modelscope.models.science.unifold.msa.toolsr+modelscope.models.science.unifold.msa.utilsrmodelscope.preprocessors.baser modelscope.preprocessors.builderrmodelscope.utils.constantr__all__rkr8rgrryrrregister_modulescienceunifold_preprocessorrr4procprotein_exampler3rripdb set_tracer4r4r4r5s~ $           C