o }o™iË ã@s€ddlmZddlmZddlmZmZmZmZm Z m Z ddl m Z ddl mZer6ddlmZddlmZGdd „d eƒZd S) é)Ú lru_cache)ÚPath)Ú TYPE_CHECKINGÚAnyÚDictÚListÚOptionalÚUnion)Úcreate_sft_dataset)ÚFineTuningDataModule)Ú TokenizerSpec)ÚPackedSequenceSpecscs¨eZdZdZ            dd eeefd ed ed dededee edededede de deddee ee fde f‡fdd„ Z eddd„ƒZ‡ZS) ÚChatDataModulezÀ Base class for fine-tuning an LLM on chat datasets. This class calls `GPTSFTChatDataset` for chat template processing See base class `FineTuningDataModule` for more details. éNéééÒéTFÚ dataset_rootÚ seq_lengthÚ tokenizerr Úmicro_batch_sizeÚglobal_batch_sizeÚrampup_batch_sizeÚseedÚmemmap_workersÚ num_workersÚ pin_memoryÚpersistent_workersÚpacked_sequence_specsr Údataset_kwargsÚuse_hf_tokenizer_chat_templatecs.tƒ ||||||||| | | | | ¡ ||_dS)a©Data module for finetuning on chat datasets. See base class `FineTuningDataModule` for more details of the arguments. Args: use_hf_tokenizer_chat_template: Whether to use the chat template from the HuggingFace tokenizer. If True, uses the tokenizer's built-in chat template. If False, uses default chat template from GPTSFTChatDataset. Defaults to False. N)ÚsuperÚ__init__r!)Úselfrrrrrrrrrrrrr r!©Ú __class__©úV/home/ubuntu/.local/lib/python3.10/site-packages/nemo/collections/llm/gpt/data/chat.pyr##s ó zChatDataModule.__init__c KsBt|f|j|s |jdkr|jn|j|j|jd|dd|jdœ |¤ŽS)NrTF) rrrrÚchatÚis_testÚpack_metadata_file_pathÚpad_cu_seqlensr!)r rÚpacked_sequence_sizerrrr!)r$ÚpathÚpack_metadata_pathr*Úkwargsr'r'r(Ú_create_datasetMsÿö õzChatDataModule._create_dataset) rNrrNrrrTFNNF)NF)Ú__name__Ú __module__Ú __qualname__Ú__doc__r ÚstrrÚintrrÚboolrrr#rr1Ú __classcell__r'r'r%r(rs\ ñ þýüûú ùø ÷ ö õ ô óòñ*rN)Ú functoolsrÚpathlibrÚtypingrrrrrr Ú"nemo.collections.llm.gpt.data.corer Ú)nemo.collections.llm.gpt.data.fine_tuningr Ú"nemo.collections.common.tokenizersr Ú-nemo.collections.llm.gpt.data.packed_sequencer rr'r'r'r(Ús