o ߥi!@sdZddlmZmZddlmZddlmZddlm Z ddl m Z ddl m Z ddlmZdd lmZdd lmZeZd d Ze jejejd Gddde eZdS)zPyTorch BERT model. ) AutoConfig AutoModel)PreTrainedModel)Models) TorchModel)MODELS)AttentionBackboneModelOutput)Tasks) get_logger)parse_labels_in_ordercCs|t|}t|ttfs|Sdd|D}t|dg}|D],}||vr*||Sd||vr:|d|Sd||vrJ|d|Sq|dS)NcSsi|]}|j|qS__name__).0modelr r b/home/ubuntu/.local/lib/python3.10/site-packages/modelscope/models/nlp/hf_transformers/backbone.py %sz$_get_model_class.. architecturesTFFlaxr)type isinstancelisttuplegetattr)config model_mappingsupported_models name_to_modelrarchr r r_get_model_class s   r ) group_key module_namec@seZdZdZedddZdS)TransformersModelaThe Bert Model transformer outputting raw hidden-states without any specific head on top. This model inherits from [`PreTrainedModel`]. Check the superclass documentation for the generic methods the library implements for all its model (such as downloading or saving, resizing the input embeddings, pruning heads etc.) This model is also a PyTorch [torch.nn.Module](https://pytorch.org/docs/stable/nn.html#torch.nn.Module) subclass. Use it as a regular PyTorch Module and refer to the PyTorch documentation for all matter related to general usage and behavior. Parameters: config ([`BertConfig`]): Model configuration class with all the parameters of the model. Initializing with a config file does not load the weights associated with the model, only the configuration. Check out the [`~PreTrainedModel.from_pretrained`] method to load the model weights. The model can behave as an encoder (with only self-attention) as well as a decoder, in which case a layer of cross-attention is added between the self-attention layers, following the architecture described in [Attention is all you need](https://arxiv.org/abs/1706.03762) by Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser and Illia Polosukhin. To behave as an decoder the model needs to be initialized with the `is_decoder` argument of the configuration set to `True`. To be used in a Seq2Seq model, the model needs to initialized with both `is_decoder` argument and `add_cross_attention` set to `True`; an `encoder_hidden_states` is then expected as an input to the forward pass. Nc Ks|dd}|rt|}|Stj|fddd|\}}tj}t||vr6t||}||}||_|St d|j dd dd |Dd ) N init_backboneFT)return_unused_kwargstrust_remote_codez!Unrecognized configuration class z. for the AutoModelModel type should be one of z, css|]}|jVqdSNr )rcr r r usz1TransformersModel._instantiate...) poprfrom_pretrainedr_model_mappingrkeysr model_dir ValueError __class__join)clsr/rr$rkwargsr model_classr r r _instantiate\s.     zTransformersModel._instantiater')r __module__ __qualname____doc__ classmethodr6r r r rr#4s%r#N)r9 transformersrrtransformers.modeling_utilsrmodelscope.metainformodelscope.modelsrmodelscope.models.builderrmodelscope.outputsrmodelscope.utils.constantr modelscope.utils.loggerr modelscope.utils.nlp.utilsr loggerr register_modulebackboner#r r r rs