o پi@sRddlZddlmZmZddlZdZGdddZGdddZGdd d ZdS) N)ThreadPoolExecutor as_completed<c@s>eZdZUdZdZeed<dZeed<dZe ed<dd Z d S) AbortAllMixinzbTest /abort_request with abort_all=True. Server needs sufficient --max-running-requests. abort_all_num_requestsi>abort_all_max_new_tokensabort_all_sleepcsj}fddt|Ifddt|D}tjtjjdddidd  t |D]}| } |d d d d q3 jWddS1sYwYdS)Nc.tjjdddjdddtd}|S)N /generatezThe capital of France isrT temperaturemax_new_tokens ignore_eostextsampling_paramsjsontimeout)requestspostbase_urlr_REQUEST_TIMEOUTrresponseselfV/home/ubuntu/.local/lib/python3.10/site-packages/sglang/test/kits/abort_timeout_kit.py run_decode z0AbortAllMixin.test_abort_all..run_decodecg|]}qSrsubmit.0_executorr!rr 'z0AbortAllMixin.test_abort_all..z/abort_request abort_allT r meta_info finish_reasontypeabort)rrrangetimesleepr rrrraise_for_statusrresult assertEqual assertIsNoneprocesspoll)r num_requestsfuturesfuturer7rr*r!rr test_abort_alls     "zAbortAllMixin.test_abort_allN) __name__ __module__ __qualname____doc__rint__annotations__rr floatr@rrrr r s     rc@s2eZdZUdZdZeed<dZeed<ddZdS) WaitingTimeoutMixinzkTest waiting queue timeout. Server needs SGLANG_REQ_WAITING_TIMEOUT and --max-running-requests=1. r waiting_timeout_num_requestsiwaiting_timeout_max_new_tokenscsj}fddt|Cfddt|D}d}t|D]}|}|ddkr9|d7}|d d q |dj WddS1sSwYdS) Ncr )Nr z Today is rTr rr)rrrrJrrrrrr r!Dr"z.run_decodecr#rr$r&r)rr r+Tr,z.robjecterrorcode) rIrr3rr7getr8r9r:r;)rr<r= error_countr>r7rr?r test_waiting_timeoutAs    "z(WaitingTimeoutMixin.test_waiting_timeoutN) rArBrCrDrIrErFrJrRrrrr rH8s    rHc@sJeZdZUdZdZeed<dZeed<dZe ed<dZ eed<d d Z d S) RunningTimeoutTwoWaveMixinaUTest running timeout with a two-wave pattern. Sends two waves with different forward_entry_time so that timeouts are triggered in separate batches. Regression test for https://github.com/sgl-project/sglang/pull/18760 Server needs SGLANG_REQ_RUNNING_TIMEOUT and sufficient --max-running-requests to hold both waves. running_timeout_num_wave1running_timeout_num_wave2running_timeout_sleepirunning_timeout_max_new_tokenscsj}j}fddt||Bfddt|D}tjfddt|D}t||D]}|}| ddkrL |dd q7Wdn1sWwY j dS) Ncr )Nr z+Write a long story about a magical kingdom.rTr rr)rrrrYrrrrrr r!ur"zLRunningTimeoutTwoWaveMixin.test_running_timeout_no_crash..run_decodecr#rr$r&r)rr r+r,zLRunningTimeoutTwoWaveMixin.test_running_timeout_no_crash..cr#rr$r&r)rr r+r,rKrLrNrO)rUrVrr3r4r5rXrr7rPr8r9r:r;)r num_wave1 num_wave2futures1futures2r>r7rr?r test_running_timeout_no_crashqs   z8RunningTimeoutTwoWaveMixin.test_running_timeout_no_crashN) rArBrCrDrUrErFrVrXrGrYr^rrrr rSas     rS) r4concurrent.futuresrrrrrrHrSrrrr s.)