o پi< @sddlZddlZddlZddlZddlmZddlmZddlm Z m Z m Z m Z ddl mZdejdejddfddZdejdejddfd d Zdejdejddfd d Ze d'dejdejdeddfddZe ddeddDdgdZe gdddgdZgdZgdZgdZeeeeZej ejj!ddgedeeedd id! de"de"de#fd"d#Z$e%d$kre$j&d%d&dSdS)(Nrmsnorm)DEFAULT_DEVICE DEFAULT_DTYPEget_benchmark_range run_benchmarkinputweightreturncCt|||ddSN)outrrr r]/home/ubuntu/.local/lib/python3.10/site-packages/sglang/jit_kernel/benchmark/bench_rmsnorm.pysglang_aot_rmsnormrcCr )N)output) jit_rmsnormrrrrsglang_jit_rmsnormrrcCr r ) fi_rmsnormrrrrflashinfer_rmsnorm rrư>epscCsB|djddd}||}||||dS)NT)dimkeepdim)floatpowmeanrsqrtcopy_)rr rr normrrrtorch_impl_rmsnorm's r$cCsg|]}d|qS)rr).0nrrr 3sr') full_rangeci_range)ii iii iiaotjitfitorch)zSGL AOT KernelzSGL JIT Kernel FlashInferPyTorch))orange-)bluez--)greenz-.)red: hidden_size batch_sizeprovideruszrmsnorm-performance) x_namesx_valsline_arg line_vals line_namesstylesylabel plot_nameargscsLtj||fttdtj|ttdttttdfdd}t|S)N)dtypedevicer,csS)N)clonerFN_MAPrr;r rrZszbenchmark..) r0randnrrrrrr$r)r9r:r;fnrrIr benchmarkBs rN__main__T) print_data)r)' itertoolsr0tritontriton.testing flashinferrr sgl_kernel!sglang.jit_kernel.benchmark.utilsrrrrsglang.jit_kernel.normrTensorrrrcompilerr$rangeBS_LISTHIDDEN_SIZE_LIST LINE_VALS LINE_NAMESSTYLESlistproductconfigstesting perf_report BenchmarkintstrrN__name__runrrrrs