o پi@sdZddlZddlZddlZddlZddlmZddlmZddlm Z m Z ddl Z e ddZ    d5d e ed ed e ed e ed edede edefddZedkreddZejdedddejdedddejdedddejdejedd d!ejd"ed#d$ejd%ejed&d'd!ejd(ejed&d)d!ejd*ejedd+d!ejd,ejedd-d!ejd.ejedd/d!eZgZejred0ejred1ejred2ejred3eejejeej ej!ej"ej#d4dSdS)6z8 Run live profiling. Usage: python3 -m sglang.profiler N)ArgumentParser)Path)ListOptionalSGLANG_TORCH_PROFILER_DIRz/tmpFurl num_steps activities output_dirprofile_by_stagemerge_profilesprofile_prefixreturnc Cs|durt}ttjtj|tt}|jdddt d|t d|d|dt|d}| sgt |d}| |} t|d } | t| Wdn1sbwYt|t|||||d } t j|d | d }| t|} | S) NT)exist_okparentszDump profiling traces to z Waiting for z9 steps and the trace to be flushed.... (profile_by_stage=)zserver_args.jsonz/get_server_infow)r rr r r r z/start_profile)rjson) PROFILER_DIRrospathabspathnormpathstrtimemkdirprintexistsrequestsgetraise_for_statusropenwritedumpspost) rrr r r r r file_pathresponseserver_args_datafile json_data trace_linkr+C/home/ubuntu/.local/lib/python3.10/site-packages/sglang/profiler.py run_profiles6 $   r-__main__z(Benchmark the online serving throughput.) descriptionz--urlzhttp://localhost:30000z7Server or API base url if not using http host and port.)typedefaulthelpz --output-dirz)Profile directory to dump profile traces.z --num-stepsz'The number of forward steps to profile.z--profile-by-stagez0Whether to profile prefill and decode separately)actionr0r1r2z--profile-prefixz!The prefix of this profiler file.)r0r2z--cpuTzWhether to profile CPU activityz--gpuzWhether to profile GPU activityz--memz@Whether to profile memory usage (https://pytorch.org/memory_viz)z--rpdzJWhether to use ROCM rpd profiler (https://github.com/ROCm/rocmProfileData)z--merge-profileszAWhether to merge profiles from all ranks into a single trace fileCPUGPUMEMRPD)rrr r r r r )NFFN)$__doc__argparserrrrpathlibrtypingrrrgetenvrrintboolr-__name__parser add_argumentBooleanOptionalAction parse_argsargsr cpuappendgpumemrpdrrr r r r r+r+r+r,s    /