o پi@s$ddlmZmZddlmZmZmZddlmZddl m Z ddl m Z m Z eGdddZeGdd d Z  d!de de deedeedfdeedfdededeedefddZ d"dee dededeedeedeedfdeedfdededefdd Zd S)#) dataclassfield)ListOptionalTuple)BenchmarkResult)NightlyBenchmarkRunner)DEFAULT_URL_FOR_TESTModelLaunchSettingsc@seZdZUdZedddZeeed<dZ e edfed<d Z e edfed <d Z e eed <d Zeed<d Ze eed<d S)PerformanceTestParamsz#Parameters for performance testing.cCsgdS)N@rrrW/home/ubuntu/.local/lib/python3.10/site-packages/sglang/test/performance_test_runner.py szPerformanceTestParams.)default_factory batch_sizesi. input_lensi output_lensN profile_dirmmmu dataset_namespec_accept_length_threshold)__name__ __module__ __qualname____doc__rrrint__annotations__rrrrrstrrrfloatrrrrr s  r c@seZdZUdZeed<eed<eeed<dZee ed<dZ ee ed<dZ ee ed<dZ ee ed <dZ eeeed <dZee ed <dS) PerformanceTestResultzlResult of a performance test. Aggregates metrics across all batch sizes tested for a single model. modelpassederrorNlatencyinput_throughputoutput_throughputoverall_throughputbenchmark_resultsavg_spec_accept_length)rrr r!r$r#boolrr*r%r+r,r-r.rrr/rrrrr&s  r&NrrFrr' perf_runnerrr.ris_vlmrrreturnc Cs,|durgd}tddtd|jtd|jtd|td|td||dur?td |tddd g}|rS|d |z|j|j||||j|jpad ||jd \} } } | r| r|j| |jdtd|jd} d} |dur| durd} d} td| n"| |krd| dd|} d} td| n td| dd|t| ddd}t |j| | |j |j |j |j | | d WSd|j} td| t |jd| dWSty}z d|jd t|} td| t |jd| dWYd}~Sd}~ww)!Nr  <============================================================zRunning PERFORMANCE test for z Variant: z Batch sizes: z Input lens: z Output lens: z Spec accept length threshold: z--trust-remote-codez--dataset-name=) model_pathrrr other_argsvariantextra_bench_argsenv)r9u#✓ Performance test succeeded for Tz>Spec accept length threshold set but no accept length reportedFu✗ zSpec accept length z.2fz < threshold u✓ Spec accept length z >= threshold cSs|jS)N) batch_size)rrrrrosz&run_performance_test..)key) r'r(r)r*r+r,r-r.r/zPerformance test failed for )r'r(r)zPerformance test exception for : )printr7r9appendrun_benchmark_for_model extra_argsr; add_reportmaxr&r*r+r,r- Exceptionr$)r'r1rrrr2rrr:resultssuccessr/ error_msgr(largest_batch_resulterrrrun_performance_test+s      rLmodelsr test_namebase_urlc  CsR|pt}t|||d} | g} d} |D]7} tdtd| jtd| jtd| jtdt| | |||||d} | | | j sLd } q| td td |td | D].} | j rid nd}| j rvd| j ddnd}td| j d||| j rtd| j qbtd td| rdndtd| | dS)aRun performance tests for multiple models. Args: models: List of ModelLaunchSettings to test profile_dir: Directory for performance profiles test_name: Name for the test (used in reports) base_url: Server base URL (default: DEFAULT_URL_FOR_TEST) batch_sizes: Batch sizes for perf test input_lens: Input lengths output_lens: Output lengths is_vlm: Whether these are VLM models dataset_name: Dataset name for VLM benchmarks Returns: dict with results: { "all_passed": bool, "results": [PerformanceTestResult, ...] } )rrNrOTzQ ================================================================================zPERFORMANCE TEST: z TP Size: z Extra Args: zP================================================================================)r'r1rrrr2rFz= ============================================================zPerformance Test Summary: r5PASSFAILz , output: z.1fz tok/sr6z r?z Error: z OVERALL: z ALL PASSEDz SOME FAILEDz============================================================= ) all_passedrG)r rsetup_profile_directoryr@r7tp_sizerCrLrAr(write_final_reportr,r'r))rMrrNrOrrrr2rr1 all_resultsrRr'resultstatusthroughput_strrrrrun_performance_for_modelss` rZ)NrrFrN)NNrrFr) dataclassesrrtypingrrrsglang.test.nightly_bench_utilsrsglang.test.nightly_utilsrsglang.test.test_utilsr r r r&r"r0r$r%rLdictrZrrrrsz      g