o `۷i( @sddlZddlZddlmZddlmZmZmZddlZddl m Z ddl m Z ddl mZddlmZddlmZmZmZdd lmZdd lmZmZdd lmZmZdd lmZdd lm Z ddl!m"Z"e#eZ$dedee%efddfddZ&   d dede%efdede%efdede%e"fdefddZ'   d dede%efdede%efdede%e"fdefddZ(   d dede%efdede%efdede%e"fdefddZ)deeefdee*efdefddZ+dS)!N) FunctionType)AnyDictUnion)is_subclass_of_base_model) usage_lib) ActorHandle)ServeControllerClient)HTTP_PROXY_TIMEOUTSERVE_LOGGER_NAMESERVE_NAMESPACE)get_controller_impl) HTTPOptions gRPCOptions)_get_global_client_set_global_client) Application)RayServeException) LoggingConfigclient http_optionsreturncCsz|r9|j}t|tr |nt|}g}|j}|D]}t||t||kr)||qt|r;t d|ddSdSdS)NzRThe new client HTTP config differs from the existing one in the following fields: z!. The new HTTP config is ignored.) http_config isinstancer parse_obj__dict__getattrappendlenloggerwarning)rrclient_http_optionsnew_http_optionsdifferent_fieldsall_http_option_fieldsfieldr&L/home/ubuntu/vllm_env/lib/python3.10/site-packages/ray/serve/_private/api.py_check_http_optionss(  r( grpc_optionsglobal_logging_configc Ks6dtjjj_tstjtdgd}|D]}||vr+t|d|d||dqt |t r6t |}|dur=t }t |t rIt di|}|durQt}n t |t r]tdi|}t}|j|||d}t|j}t|d krztjd d |Dtd W|Stjjytd tdw|S)zStart Ray Serve controller. The function makes sure controller is ready to start deploying apps after it returns. Parameters are same as ray.serve._private.api.serve_start(). Returns: controller actor handle. F) namespace) http_host http_porthttp_middlewaresz6 is deprecated, please use serve.start(http_options={"z": z }) instead.N)rr)r*rcSsg|]}|jqSr&)readyremote).0handler&r&r' hsz%_start_controller..)timeoutz!HTTP proxies not available after zs.r&)ray_privateworker global_worker_filter_logs_by_jobis_initializedinitr ValueErrorrdictrrrrr r0get get_proxiesrvaluesr exceptionsGetTimeoutError TimeoutError) rr)r*kwargshttp_deprecated_argskeycontroller_impl controller proxy_handlesr&r&r'_start_controller2sX        rJcstdztdd}tdtd|rt|||WSty'Ynwt t j ddj |||fi|IdH}t |}t |td td |S) a]Initialize a serve instance asynchronously. This function is not thread-safe. The caller should maintain the async lock in order to start the serve instance asynchronously. This function has the same functionality as ray.serve._private.api.serve_start(). Parameters & Returns are same as ray.serve._private.api.serve_start(). serveT_health_check_controller/Connecting to existing Serve app in namespace "(". New http options will not be applied.r)num_cpusNStarted Serve in namespace "".)rrecord_library_usagerrinfor r(rr5r0rJoptionsr rrr)r*rDrrHr&r&r'serve_start_asyncrs6      rWcKstdztdd}tdtd|rt|||WSty&Ynwt|||fi|}t |}t |tdtd|S)a2Initialize a serve instance. By default, the instance will be scoped to the lifetime of the returned Client object (or when the script exits). This is only relevant if connecting to a long-running Ray cluster (e.g., with ray.init(address="auto") or ray.init("ray://")). Args: http_options (Optional[Dict, serve.HTTPOptions]): Configuration options for HTTP proxy. You can pass in a dictionary or HTTPOptions object with fields: - host(str, None): Host for HTTP servers to listen on. Defaults to "127.0.0.1". To expose Serve publicly, you probably want to set this to "0.0.0.0". - port(int): Port for HTTP server. Defaults to 8000. - root_path(str): Root path to mount the serve application (for example, "/serve"). All deployment routes will be prefixed with this path. Defaults to "". - middlewares(list): A list of Starlette middlewares that will be applied to the HTTP servers in the cluster. Defaults to []. - location(str, serve.config.DeploymentMode): The deployment location of HTTP servers: - "HeadOnly": start one HTTP server on the head node. Serve assumes the head node is the node you executed serve.start on. This is the default. - "EveryNode": start one HTTP server per node. - "NoServer" or None: disable HTTP server. - num_cpus (int): The number of CPU cores to reserve for each internal Serve HTTP proxy actor. Defaults to 0. grpc_options: [Experimental] Configuration options for gRPC proxy. You can pass in a gRPCOptions object with fields: - port(int): Port for gRPC server. Defaults to 9000. - grpc_servicer_functions(list): List of import paths for gRPC `add_servicer_to_server` functions to add to Serve's gRPC proxy. Default empty list, meaning not to start the gRPC server. rKTrLrNrOrQrR) rrSrrrTr r(rrJr rrVr&r&r' serve_starts. .    rXbuilderargscCst|trt|dkrtd|St|ts tdt|dt|}t|j dkr0td|j t |j d}t |j rMt|j rM|j |}||}t|ts`tdt|d|S)abCalls a user-provided function that returns Serve application. If an Application object is passed, this is a no-op. Else, we validate the signature of the function, convert the args dictionary to the user-annotated Pydantic model if provided, and call the function. The output of the function is returned (must be an Application). rzbArguments can only be passed to an application builder function, not an already built application.zOExpected a built Serve application or an application builder function but got: .znApplication builder functions should take exactly one parameter, a dictionary containing the passed arguments.zhApplication builder functions must return an `Application` returned `from `Deployment.bind()`, but got: )rrrr<r TypeErrortypeinspect signature parameterslistkeysisclass annotationrr)rYrZr`paramappr&r&r',call_user_app_builder_with_args_if_necessarys<     rh)NNN),r_loggingtypesrtypingrrrr5ray._common.pydantic_compatrray._common.usager ray.actorrray.serve._private.clientr ray.serve._private.constantsr r r ray.serve._private.default_implr ray.serve.configrrray.serve.contextrrray.serve.deploymentrray.serve.exceptionsrray.serve.schemar getLoggerrr=r(rJrWrXstrrhr&r&r&r's                A    -    H