o
}oi @ s d dl mZmZ d dlZd dlZd dlmZ d dlmZ ej deeeef fddZ
ej dg d d
d Z
d d
Zdd ZdS ) )Any GeneratorN)create_server_process)
AdapterConfigreturnc c s. t dddd}t|\}}|V | d S )Nz)http://localhost:3300/v1/chat/completionsTz)api_url
use_reasoningend_reasoning_token)r r terminate)fake_openai_endpointadapter_configp r \/home/ubuntu/.local/lib/python3.10/site-packages/tests/evaluation/adapters/test_reasoning.pyadapter_server s r z input_content,expected_content))zuLet me think about this...
This is my reasoning process that should be removed
Here's my final answer.zHere's my final answer.)%No reasoning tokens in this response.r )zaFirst I'll analyze the problem
Then I'll solve it step by stepHere's the solution.zHere's the solution.c
C sv d| j }ddd|digi}ddd|d }tj||d
}|jd ks%J | }|d d d d
} | |ks9J d S )Nhttp://localhost:choicesmessage assistantrolecontentThis is a test promptd ?prompt
max_tokens temperature
fake_responsejson r r
local_portrequestspost status_coder" )
r r
input_contentexpected_contenturl
response_datadataresponse cleaned_datacleaned_contentr r r test_reasoning_responses* s& r1 c C s d| j }dddddiddddigi}dd d
|d }tj||d }|jd
ks+J | }|d d d d dks=J |d d d d dksKJ d S )Nr r r r z"Reasoning 1Answer 1r z"Reasoning 2Answer 2r r r r r! r# r r zAnswer 1 zAnswer 2r$ r r r+ r, r- r. r/ r r r test_multiple_choices\ s0 r4 c C sd d| j }dddddigi}ddd |d
}tj||d }| }|d d d d
dks0J d S )Nr r r systemz9This should not be processedSystem messager r r r r r! r r )r% r& r' r" r3 r r r test_non_assistant_role s" r6 )typingr r pytestr& /nemo.collections.llm.evaluation.adapters.serverr #nemo.collections.llm.evaluation.apir fixturer mark parametrizer1 r4 r6 r r r r s
! %