o ۾i@sdZddlZddlmZddlmZddlmZmZee Z de de e e efdee e fdBfd d Zde e e effd d Zde dee e fdBfd dZdS)zK Utility functions that create ORCA endpoint load report response headers. N)Mapping) init_logger)Gaugeget_metrics_snapshotmetrics_format named_metricsreturncCs|dvrtdtdSi}ddd|Di}|dkr1dd d |D}d ||d <n|d krAdt||d <td||S)a Creates ORCA headers named 'endpoint-load-metrics' in the specified format and adds custom metrics to named_metrics. ORCA headers format description: https://docs.google.com/document/d/1C1ybMmDKJIVlrbOLbywhu9iRYo4rilR-cT50OTtOFTs/edit?tab=t.0 ORCA proto https://github.com/cncf/xds/blob/main/xds/data/orca/v3/orca_load_report.proto Parameters: - metrics_format (str): The format of the header ('TEXT', 'JSON'). - named_metrics (List[Tuple[str, float]]): List of tuples with metric names and their corresponding double values. Returns: - Optional[Mapping[str,str]]: A dictionary with header key as 'endpoint-load-metrics' and values as the ORCA header strings with format prefix and data in with named_metrics in. )textjsonzAWarning: `%s` format is not supported in the ORCA response headerNrcSs*i|]\}}t|trt|tr||qS isinstancestrfloat.0 metric_namevaluer r X/home/ubuntu/.local/lib/python3.10/site-packages/vllm/entrypoints/openai/orca_metrics.py -sz&create_orca_header..r z, cSs4g|]\}}t|trt|trd|d|qS)znamed_metrics.=r rr r r 7sz&create_orca_header..zTEXT zendpoint-load-metricsr zJSON zCreated ORCA header %s)lowerloggerwarningformatjoinr dumpsinfo)rrheader orca_reportnative_http_headerr r rcreate_orca_headers,    r"cCsXg}ddd}t}|D]}||j}|dur)t|tr)|t|t|jfq |S)a  Collects current metrics from Prometheus and returns some of them in the form of the `named_metrics` list for `create_orca_header()`. Parameters: - None Returns: - list[tuple[str, float]]: List of tuples of metric names and their values. kv_cache_usage_percnum_requests_waiting)zvllm:kv_cache_usage_perczvllm:num_requests_waitingN) rgetnamer rappendrrr)rprometheus_to_orca_metricsmetricsmetric orca_namer r r!get_named_metrics_from_prometheusIs  r,cCs|sdSt}t||S)a Creates ORCA headers named 'endpoint-load-metrics' in the specified format. Metrics are collected from Prometheus using `get_named_metrics_from_prometheus()`. ORCA headers format description: https://docs.google.com/document/d/1C1ybMmDKJIVlrbOLbywhu9iRYo4rilR-cT50OTtOFTs/edit?tab=t.0 ORCA proto https://github.com/cncf/xds/blob/main/xds/data/orca/v3/orca_load_report.proto Parameters: - metrics_format (str): The format of the header ('TEXT', 'JSON'). Returns: - Optional[Mapping[str,str]]: A dictionary with header key as 'endpoint-load-metrics' and values as the ORCA header strings with format prefix and data in with named_metrics in. N)r,r")rrr r rmetrics_headerds r-)__doc__r collections.abcr vllm.loggerrvllm.v1.metrics.readerrr__name__rrlisttuplerr"r,r-r r r rs   9"