o `۷i(@sddlZddlmZmZmZmZmZmZddlm Z ddl m Z ddl m Z ddlmZddlmZmZmZddlmZdd lmZmZdd lmZeeZd Zd ZGd ddZdS)N)AnyCallableDictListOptionalTuple) Starlette)Request)Route)Scope)ApplicationName DeploymentID EndpointInfo)SERVE_LOGGER_NAME) RoutePatternget_asgi_route_name)DeploymentHandlez!Route table is not populated yet.zNo replicas are available yet.c@seZdZdZdeeegeffddZdede eeffddZ d e e e ffd d Zd edee eeeffd dZdedee eeeffddZdededefddZdS) ProxyRouterz&Router interface for the proxy to use. get_handlecCsH||_t|_d|_t|_t|_t|_t|_t|_ t|_ dS)NF) _get_handledicthandles_route_table_populatedlist sorted_routes route_infoapp_to_is_cross_language endpointsroute_patterns_route_pattern_apps)selfrr!U/home/ubuntu/vllm_env/lib/python3.10/site-packages/ray/serve/_private/proxy_router.py__init__s zProxyRouter.__init__is_headreturncCs<|jsdtfS|r dS|jD] }|rdSqdtfS)aWhether the proxy router is ready to serve traffic. The first return value will be false if any of the following hold: - The route table has not been populated yet with a non-empty set of routes - The route table has been populated, but none of the handles have received running replicas yet AND it lives on a worker node. Otherwise, the first return value will be true. F)T)rNO_ROUTES_MESSAGErvaluesrunning_replicas_populatedNO_REPLICAS_MESSAGE)r r$handler!r!r"ready_for_traffic:s zProxyRouter.ready_for_trafficrc Cs&tjd|dddid|rd|_||_t|j}g}i}i}i}|D]2\}}||j |||j <|j ||j <|j rF|j ||j <||jvrQ| |q(||||j|<q(t|dkrptjdt|ddd id|D]}|j|=qrt|d d dd |_||_||_||_ |jdS) NzGot updated endpoints: . log_to_stderrT)extrarz Deleting z unused handles.FcSst|SN)len)xr!r!r"{sz+ProxyRouter.update_routes..)keyreverse)loggerinforrsetrkeysitemsappendrouteapp_is_cross_languageapp_namerremoverr1sortedrrrrclear) r rexisting_handlesroutesrrrendpointr7r!r!r" update_routesWs@        zProxyRouter.update_routes target_routecCs||jD]8}||r;d}|drd}nt|t|ks$|t|dkr&d}|r;|j|}||j||j|jfSqdS)zReturn the longest prefix match among existing routes for the route. Args: target_route: route to match against. Returns: (route, handle, is_cross_language) if found, else None. F/TN)r startswithendswithr1rrrr>)r rFr<matchedrDr!r!r" match_routes     zProxyRouter.match_routetarget_app_namecCsL|jD]\}}||jkst|jdkr#|j|}|j||jfSqdS)a2Return the handle that matches with endpoint. Args: target_app_name: app_name to match against. Returns: (route, handle, is_cross_language) for the single app if there is only one, else find the app and handle for exact match. Else return None. N)rr:r>r1rr<r=)r rL endpoint_tagr+ endpoint_infor!r!r"get_handle_for_endpoints  z#ProxyRouter.get_handle_for_endpoint route_prefix asgi_scopecs||jvr|S|j|}|s|S|j|}|durLzdtfddfdd|D}t|d}||j|<WntyKtjd|d d |YSwz t||}|rW|WSW|Stymtjd |d d Y|Sw) aMatch an incoming request to a specific route pattern. This attempts to match the request path to a route pattern (e.g., /api/{user_id}) rather than just the route prefix. This provides more granular metrics. The mock Starlette app is cached per route_prefix for performance, avoiding the overhead of recreating the app and routes on every request. Args: route_prefix: The matched route prefix from match_route() asgi_scope: The ASGI scope containing the request path and method Returns: The matched route pattern if available, otherwise the route_prefix NrequestcsdSr0r!)rSr!r!r"dummy_endpointsz7ProxyRouter.match_route_pattern..dummy_endpointcsg|] }t|j|jdqS))methods)r pathrU).0patternrTr!r" sz3ProxyRouter.match_route_pattern..)rCz6Failed to create mock app for route pattern matching: T)exc_infoz"Failed to match route pattern for ) rrgetr r Exceptionr6debugr)r rQrRpatternsmock_apprCrJr!rYr"match_route_patternsD        zProxyRouter.match_route_patternN)__name__ __module__ __qualname____doc__rstrrr#boolrr,rr rrErrKrPr rar!r!r!r"rs$  + $ r) loggingtypingrrrrrrstarlette.applicationsrstarlette.requestsr starlette.routingr starlette.typesr ray.serve._private.commonr r rray.serve._private.constantsr1ray.serve._private.thirdparty.get_asgi_route_namerrray.serve.handler getLoggerr6r'r*rr!r!r!r"s