o ̳iw@sTddlmZddlmZddlmZddedefddZdd eded e fd d Z dS) t5_encoder) T5Encoder T5Tokenizer max_seq_lenreturnc Cstddddddddd|d S) a Builder for the T5 v1.1 XXL (11B parameters) encoder. T5 paper: https://arxiv.org/abs/1910.10683 1.1 release: https://github.com/google-research/text-to-text-transfer-transformer/blob/main/released_checkpoints.md#t511 Args: max_seq_len (int): The maximum sequence length (context length) of the model. Default: 512 Returns: T5Encoder: Instantiation of the T5 encoder ii(@ i}gư>) embed_dimmlp_dim num_headshead_dim num_layersrel_pos_num_bucketsrel_pos_max_dist vocab_sizenorm_epsrr)rrW/home/ubuntu/.local/lib/python3.10/site-packages/torchtune/models/t5/_model_builders.pyt5_v1_1_xxl_encoder srTpathtruncatecCst|||dS)aF Builder for the T5 tokenizer. Args: path (str): the path to the T5 sentencepiece tokenizer file max_seq_len (int): the context length truncate (bool): whether to truncate the token sequence when longer than max_seq_len Returns: T5Tokenizer: Instantiation of the T5 tokenizer )rrr)rrrrrr t5_tokenizer*s rN)r)rT) 'torchtune.models.t5._component_buildersrtorchtune.models.t5._encoderrtorchtune.models.t5._tokenizerrintrstrboolrrrrrs