o پiH3@sdZddlZddlZddlZddlZddlmZmZmZddl Z ddl m Z ddl m Z mZddlmZmZmZmZGdddZdS) zDUtilities for running nightly performance benchmarks with profiling.N)ListOptionalTuple)kill_process_tree)BenchmarkResultgenerate_markdown_report)!DEFAULT_TIMEOUT_FOR_SERVER_LAUNCHis_in_cipopen_launch_serverwrite_github_step_summaryc@seZdZdZ d2dedededefddZd3d d Z d4d eded eeeffddZ   d5d ede e dee dfdee dfdedede e ede e ede d e efddZ d4de eded eeje ffdd Z d4deded ee ee ffd!d"Z     d6d ede e dee dfdee dfd#e e eded$e e ede d%e e d&e ed ee ee e effd'd(Zd e efd)d*Z d2d+e ede ed dfd,d-Zd3d.d/Zd efd0d1ZdS)7NightlyBenchmarkRunneraHelper class for running nightly performance benchmarks with profiling. This class encapsulates common patterns used across nightly performance tests, including profile directory management, benchmark command construction, result parsing, and report generation. N profile_dir test_namebase_url gpu_configcCsb||_||_||_|ptjdd|_d|}|jr$|d|jd7}|d7}|t|_ dS)aAInitialize the benchmark runner. Args: profile_dir: Directory to store performance profiles test_name: Name of the test (used for reporting) base_url: Base URL for the server gpu_config: Optional GPU configuration string (e.g., "2-gpu-h100", "8-gpu-b200") GPU_CONFIGz##  () N) r rrosenvirongetrrhelp_str full_report)selfr rrrheaderrM/home/ubuntu/.local/lib/python3.10/site-packages/sglang/test/nightly_utils.py__init__s zNightlyBenchmarkRunner.__init__returncCstj|jdddS)z1Create the profile directory if it doesn't exist.T)exist_okN)rmakedirsr rrrrsetup_profile_directory8sz.NightlyBenchmarkRunner.setup_profile_directoryr model_pathvariantcCstt}|dd}|r%|d|d|}d|d|d|d}n|d|}d|d|d}tj|j|}||fS)a8Generate unique profile filename and path for the model. Args: model_path: Path to the model (e.g., "deepseek-ai/DeepSeek-V3.1") variant: Optional variant suffix (e.g., "basic", "mtp", "nsa") Returns: Tuple of (profile_path_prefix, json_output_file) /_results_z.json)inttimereplacerpathjoinr )rr%r& timestampmodel_safe_nameprofile_filename json_filenameprofile_path_prefixrrrgenerate_profile_filename<s z0NightlyBenchmarkRunner.generate_profile_filenameT batch_sizes input_lens. output_lensr3json_output_file extra_args server_argsenable_profilec Csdddd|d|jdgdd|Dd d d|Dd d d|Dd d|dd} | r<|r<| ddd|g|rC| ||rO| d| || S)aBuild the benchmark command with all required arguments. Args: model_path: Path to the model batch_sizes: List of batch sizes to test input_lens: Tuple of input lengths to test output_lens: Tuple of output lengths to test profile_path_prefix: Prefix for profile output files json_output_file: Path to JSON output file extra_args: Optional extra arguments to append to command server_args: Optional server launch arguments to record in metrics enable_profile: Whether to enable profiling (default True for NVIDIA) Returns: List of command arguments ready for subprocess.run() python3z-mzsglang.bench_one_batch_serverz--modelz --base-urlz --batch-sizecSg|]}t|qSrstr.0xrrr |zBNightlyBenchmarkRunner.build_benchmark_command..z --input-lencSr=rr>r@rrrrC~rDz --output-lencSr=rr>r@rrrrCrDz --show-reportz--pydantic-result-filename=z--no-append-to-github-summaryz--trust-remote-codez --profilez--profile-by-stagez--profile-output-dirz--server-args-for-metrics)rextendappend) rr%r5r6r7r3r8r9r:r;commandrrrbuild_benchmark_commandWsR        z.NightlyBenchmarkRunner.build_benchmark_commandrGmodel_descriptioncCs`tdd|tj|ddd}|jdkr,|pd}td|dt|j|d fS|dfS) aExecute the benchmark command and return the result. Args: command: Command to execute model_description: Description for logging (e.g., "model_name (variant)") Returns: Tuple of (CompletedProcess, success_bool) zRunning command:  T)capture_outputtextr benchmarkzError running benchmark for :F)printr. subprocessrun returncodestderr)rrGrIresultdescrrrrun_benchmark_commands   z,NightlyBenchmarkRunner.run_benchmark_commandc Csg}tj|s|p d}td|d||dfSzs