o
}oi @ s d dl Z d dlmZmZ d dlmZmZ d dlmZ d dl m
Z
m Z m Z m
Z
G dd de
Zeeeded ed
eee jf fd d Zeeed
ed efddZeeed
ed efddZdZdZdZdZdZG dd de
Zeeeded ed
eee jf fddZdS ) N)CutMixedCut)NeMoSFTExampleSourceTargetTextExample)registered_prompt_format_fn)BOS_SLOTEOS_SLOTModalityPromptFormatterc @ sb e Zd ZdZdZdZde dejejddde d d
ejided e d
ejidiZ
d S )
Llama2PromptFormatterz
This template has been validated to provide identical tokenized results to the official code
in https://github.com/meta-llama/llama/blob/main/llama/generation.py
llama2 assistantsystem_and_userz3[INST] <>
|system|
<>
|message| [/INST]systemmessagetemplateslotsuserz[INST] |message| [/INST]r z
|message| N) __name__
__module__ __qualname____doc__NAME OUTPUT_ROLEr r Textr TEMPLATE r r Y/home/ubuntu/.local/lib/python3.10/site-packages/nemo/collections/common/prompts/llama.pyr s* r cutpromptreturnc C s t | tr| j} | dr| j}n | dr| j}n| j}g }| dr1|d| j|dd n
|dd|id | j d j
}d urO|d
d|id | |S ) Ncontextquestion
system_promptr r roler r r r r
isinstancer first_non_padding_cut
has_customr# r$ default_contextappendr% supervisionstext
encode_dialogr r! r# turnsanswerr r r r 7 s
r examplec C sT | j d urd| j j| jjdd}n dd| jjid}|||jd| jjidgS )Nr r r& r r )r$ r/ sourcer0 r target)r4 r! user_turnr r r llama2_src_tgt_text_exampleL s
r8 c sx | j d d d }d| j v r | j d r d| j d |dd}ndd |id} |g fd
d | j d d d D S )
N
conversationsr valuer r r r& r r c s0 g | ]}|d dkr dn j d|d idqS )fromUserr r r: r& )r ).0turnr! r r
o s "z+llama2_sft_text_example.. )datar0 )r4 r! first_user_turn
first_turnr r? r llama2_sft_text_example` s
rE z<|begin_of_text|>z<|start_header_id|>z<|end_header_id|>z
<|eot_id|>z
c @ s e Zd ZdZdZdZe de e Z dde
ide de e de de j
id d
e d
e e de de j
id ee de de j
id iZd S ) Llama3PromptFormatterz|
Implemented following the code at:
https://github.com/meta-llama/llama3/blob/main/llama/test_tokenizer.py#L56
llama3r
preambler r z |message|r r r N)r r r r r r LLAMA3_HEADER_BEGINLLAMA3_HEADER_END LLAMA3_NLINFERENCE_PREFIX
LLAMA3_BOSLLAMA3_END_OF_TURNr r r r r r r rF } s0 rF c C s t | tr| j} | dr| j}n | dr| j}n| j}g }| dr/|dd| jid |dd|id | j d j
}d urM|d d|id | |S )
Nr# r$ r% r r r& r r r
r( r1 r r r rG s
rG ) torch
lhotse.cutr r 1nemo.collections.common.data.lhotse.text_adaptersr r &nemo.collections.common.data.prompt_fnr )nemo.collections.common.prompts.formatterr r r r
r dictstrTensorr r8 rE rM rI rJ rN rK rF rG r r r r s( ""&