o %ݫi@sLdZddlmZddlZddlmZddlmZeeZ GdddeZ dS)zThis lobe enables the integration of generic huggingface pretrained text encoders (e.g. BERT). Transformer from HuggingFace needs to be installed: https://huggingface.co/transformers/installation.html Authors * Sylvain de Langen 2024 )OptionalN)HFTransformersInterface) get_loggercsNeZdZdZ  ddeeffdd Zdefdd Zdd efd d Z Z S) TextEncoderaThis lobe enables the integration of a generic HuggingFace text encoder (e.g. BERT). Requires the `AutoModel` found from the `source` to have a `last_hidden_state` key in the output dict. Arguments --------- source : str HuggingFace hub name: e.g "google-bert/bert-base" save_path : str Path (dir) of the downloaded model. freeze : bool (default: True) If True, the model is frozen. If False, the model will be trained alongside with the rest of the pipeline. num_layers : int, optional When specified, and assuming the passed LM can be truncated that way, the encoder for the passed model will be truncated to the specified layer (mutating it). This means that the embeddings will be those of the Nth layer rather than the last layer. The last layer is not necessarily the best for certain tasks. **kwargs Extra keyword arguments passed to the `from_pretrained` function. Example ------- >>> inputs = ["La vie est belle"] >>> model_hub = "google-bert/bert-base-multilingual-cased" >>> save_path = "savedir" >>> model = TextEncoder(model_hub, save_path) >>> outputs = model(inputs) TN num_layersc s@tjd|||d||j|d|dur||dSdS)N)source save_pathfreeze)r)super__init__load_tokenizertruncate)selfrrr rkwargs __class__r q/home/ubuntu/.local/lib/python3.10/site-packages/speechbrain/lobes/models/huggingface_transformers/textencoder.pyr 6s zTextEncoder.__init__ keep_layerscCsF|dksJd|t|jjjksJd|jjjd||jj_dS)a Truncates the encoder to a specific layer so that output embeddings are the hidden state of the n-th layer. Arguments --------- keep_layers : int Number of layers to keep, e.g. 4 would keep layers `[0, 1, 2, 3]`. rz`Invalid requested layer count: Must keep at least one LM layer (negative values are not allowed)z