o ߥi @sddlmZddlmZddlmZmZddlmZddl m Z ddl m Z ddl mZdd lmZeje jejd eje jejd eje jejd eje jejd Gd d d eeZd S)) RobertaForSequenceClassification)Models)Model TorchModel)MODELS)&AttentionTextClassificationModelOutput)Tasks)parse_labels_in_order) VecoConfig) module_namecs@eZdZdZeZfddZfddZefddZ Z S)VecoForSequenceClassificationa Veco Model transformer with a sequence classification/regression head on top (a linear layer on top of the pooled output) e.g. for GLUE tasks. This model inherits from [`PreTrainedModel`]. Check the superclass documentation for the generic methods the library implements for all its model (such as downloading or saving, resizing the input embeddings, pruning heads etc.) This model is also a PyTorch [torch.nn.Module](https://pytorch.org/docs/stable/nn.html#torch.nn.Module) subclass. Use it as a regular PyTorch Module and refer to the PyTorch documentation for all matter related to general usage and behavior. Preprocessor: This is the text classification model of Veco, the preprocessor of this model is `modelscope.preprocessors.TextClassificationTransformersPreprocessor`. Trainer: This model should be trained by dataset which has mixed languages, and evaluated by datasets of languages one by one. For example, if the training dataset is xnli (which has sub datasets of multiple languages), then you should mix the sub-datasets with the languages you want to train to one training dataset, and evaluate the model one sub-dataset by one sub-dataset of different languages. This procedure can be done by custom code. If you are using trainer of ModelScope, the `VecoTrainer` is suggested to use to train this model. This trainer overrides the basic evaluation loop, and will call the evaluation dataset one by one. Besides, this trainer will use the `VecoTaskDataset` to mix the input datasets to one, you can check the API Doc for the details. To check the complete example please view the unittest `test_veco_xnli` in `tests.trainers.test_finetune_sequence_classification.py` Parameters: config ([`VecoConfig`]): Model configuration class with all the parameters of the model. Initializing with a config file does not load the weights associated with the model, only the configuration. Check out the [`~PreTrainedModel.from_pretrained`] method to load the model weights. This class overrides [`RobertaForSequenceClassification`]. Please check the superclass for the appropriate documentation alongside usage examples. c s*tj|jfi|tt||dS)N)super__init__ name_or_pathr)selfconfigkwargs __class__b/home/ubuntu/.local/lib/python3.10/site-packages/modelscope/models/nlp/veco/text_classification.pyrMsz&VecoForSequenceClassification.__init__cs6d|d<tt|j|i|}t|j|j|j|jdS)u* Returns: Returns `modelscope.outputs.AttentionTextClassificationModelOutput` Examples: >>> from modelscope.models import Model >>> from modelscope.preprocessors import Preprocessor >>> model = Model.from_pretrained('damo/nlp_veco_fill-mask-large', >>> task='text-classification', num_labels=2) >>> preprocessor = Preprocessor.from_pretrained('damo/nlp_veco_fill-mask-large', >>> label2id={'0': 0, '1': 1}) >>> # Call the model, return some tensors >>> print(model(**preprocessor('这是个测试'))) >>> # Call the pipeline, the result may be incorrect >>> from modelscope.pipelines import pipeline >>> pipeline_ins = pipeline('text-classification', pipeline_name='text-classification', >>> model=model, preprocessor=preprocessor) >>> print(pipeline_ins('这是个测试')) T return_dict)losslogits hidden_states attentions)rrforwardrrrrr)rargsroutputsrrrrQsz%VecoForSequenceClassification.forwardc sj|dd}|dd}t||fi|}|dur&tdi|}||}|Stt|jdd|i|}|S)alInstantiate the model. Args: kwargs: Input args. model_dir: The model dir used to load the checkpoint and the label information. num_labels: An optional arg to tell the model how many classes to initialize. Method will call utils.parse_label_mapping if num_labels is not input. label2id: An optional label2id mapping, which will cover the label2id in configuration (if exists). Returns: The loaded model, which is initialized by transformers.PreTrainedModel.from_pretrained model_dirNcfgpretrained_model_name_or_pathr)popr r rrfrom_pretrained)clsrr r! model_argsrmodelrrr _instantiateos  z*VecoForSequenceClassification._instantiate) __name__ __module__ __qualname____doc__r config_classrr classmethodr( __classcell__rrrrr s'  r N) transformersrmodelscope.metainformodelscope.modelsrrmodelscope.models.builderrmodelscope.outputsrmodelscope.utils.constantrmodelscope.utils.nlp.utilsr configurationr register_modulenlivecosentiment_classificationsentence_similaritytext_classificationr rrrrs"