o `۷iM@sddlZddlZddlmZddlmZmZmZmZm Z m Z m Z ddl m Z mZmZmZddlmZddlmZddlmZmZddlmZdd lmZmZmZdd lmZe eZ!ed d Gd ddZ"ed d GdddZ#de#defddZ$dede#fddZ%dS)N)deepcopy)AnyCallableDictListOptionalTupleUnion)DeploymentConfig ReplicaConfigRequestRouterConfighandle_num_replicas_auto)SERVE_LOGGER_NAME) ServeUsageTag)DEFAULTDefault)AutoscalingConfig)DeploymentSchema LoggingConfigRayActorOptionsSchema) PublicAPIstable) stabilityc@seZdZdZdddZdS) ApplicationaOne or more deployments bound with arguments that can be deployed together. Can be passed into another `Deployment.bind()` to compose multiple deployments in a single application, passed to `serve.run`, or deployed via a Serve config file. For example, to define an Application and run it in Python: .. code-block:: python from ray import serve from ray.serve import Application @serve.deployment class MyDeployment: pass app: Application = MyDeployment.bind(OtherDeployment.bind()) serve.run(app) To run the same app using the command line interface (CLI): .. code-block:: bash serve run python_file:app To deploy the same app via a config file: .. code-block:: yaml applications: my_app: import_path: python_file:app bound_deployment DeploymentcCs ||_dSN)_bound_deployment)selfrrJ/home/ubuntu/vllm_env/lib/python3.10/site-packages/ray/serve/deployment.py__init__;s zApplication.__init__N)rr)__name__ __module__ __qualname____doc__r!rrrr rs#rc3@seZdZdZ  dKdedededeeddf d d Zdefd d Z e defd dZ e deefddZ e de eeffddZe defddZe defddZe defddZe defddZe ddZe deefddZe deefdd Ze deefd!d"Ze deefd#d$Ze defd%d&Zd'efd(d)Zd*d+Zde fd,d-Z!de"j#e"j#e"j#e"j#e"j#e"j#e"j#e"j#e"j#e"j#e"j#e"j#e"j#e"j#e"j#e"j#e"j#e"j#e"j#e"j#e"j#de"j#fd.eede$ede$ed/e$ee eefd0e$e edfd1e$eed2e$e%eee&fd3e$ed4e$e%eeefd5e$ed6e$eed7e$ed8e$ed9e$e ee'dfd:e$e&d;e$e&de$e ee)dfd?e$eed@e$eeefdAe*dBe$eddf2dCdDZ+dEdFZ,dGdHZ-dIdJZ.dS)LraClass (or function) decorated with the `@serve.deployment` decorator. This is run on a number of replica actors. Requests to those replicas call this class. One or more deployments can be composed together into an `Application` which is then run via `serve.run` or a config file. Example: .. code-block:: python @serve.deployment class MyDeployment: def __init__(self, name: str): self._name = name def __call__(self, request): return "Hello world!" app = MyDeployment.bind() # Run via `serve.run` or the `serve run` CLI command. serve.run(app) NFnamedeployment_configreplica_configversionreturncCsL|std|||dust|tstd||_||_||_||_dS)NzZThe Deployment constructor should not be called directly. Use `@serve.deployment` instead.zversion must be a string.) RuntimeError_validate_name isinstancestr TypeError_name_version_deployment_config_replica_config)rr&r'r(r) _internalrrr r!\s  zDeployment.__init__cCs4t|ts tdd|vrtd|ddSdS)Nzname must be a string.#zvDeployment names should not contain the '#' character, this will raise an error starting in Ray 2.46.0. Current name: .)r-r.r/warningswarn)rr&rrr r,rs zDeployment._validate_namecC|jS)zUnique name of this deployment.r0rrrr r&}szDeployment.namecCr9r)r1r;rrr r)szDeployment.versioncC|jjS)z8Underlying class or function that this deployment wraps.)r3deployment_defr;rrr func_or_classzDeployment.func_or_classcCr<)zTarget number of replicas.)r2 num_replicasr;rrr r@r?zDeployment.num_replicascCr<)z%Dynamic user-provided config options.)r2 user_configr;rrr rAr?zDeployment.user_configcCr<)z4Max number of requests a replica can handle at once.)r2max_ongoing_requestsr;rrr rBr?zDeployment.max_ongoing_requestscCr<)zDMax number of requests that can be queued in each deployment handle.)r2max_queued_requestsr;rrr rCr?zDeployment.max_queued_requestscCtd)N`route_prefix` can no longer be specified at the deployment level. Pass it to `serve.run` or in the application config instead.) ValueErrorr;rrr route_prefixszDeployment.route_prefixcCr<)z:Actor options such as resources required for each replica.)r3ray_actor_optionsr;rrr rHr?zDeployment.ray_actor_optionscCr<r)r3 init_argsr;rrr rIzDeployment.init_argscCr<r)r3 init_kwargsr;rrr rKrJzDeployment.init_kwargscCstddS)NzUDeprecationWarning: `Deployment.url` is deprecated and will be removed in the future.)loggerwarningr;rrr urlszDeployment.urlcCr<rr2logging_configr;rrr rPrJzDeployment.logging_configrPcCs ||j_dSrrO)rrPrrr set_logging_configs zDeployment.set_logging_configcCrD)NzNDeployments cannot be constructed directly. Use `deployment.deploy() instead.`)r+r;rrr __call__szDeployment.__call__cOst|j||dS)zBind the arguments to the deployment and return an Application. The returned Application can be deployed using `serve.run` (or via config file) or bound to another deployment for composition. ) _init_args _init_kwargs)roptions)rargskwargsrrr bindszDeployment.bindr>r@rGrHplacement_group_bundlesplacement_group_strategy%placement_group_bundle_label_selectormax_replicas_per_noderArBrCautoscaling_configgraceful_shutdown_wait_loop_sgraceful_shutdown_timeout_shealth_check_period_shealth_check_timeout_srequest_router_configrSrTr4max_constructor_retry_countc Cs|tjur td| durtd|dkr$d}t| |\} }tjdddtD}t |j }|s;|j ||tjddfvrN|tjdfvrNtd|d krVtd |sc|tjurct d n |tjdfvrm||_| tjuru| |_| tjur}| |_| tjur| |_|tjur||_|dur|jj}|tjur|j}|tjur|j}|tjur|jj}|tjur|jj}|tjur|jj}|tjur|jj}|tjur|jj}| tjur|jj} |jj}| tjur|jj} |tjur||_ |tjur||_!|tjur||_"|tjur ||_#|tjur||_$|tjur||_%|tjur/t&|t'r,|(}||_)t*j+||||||| || d }t,||||d dS)aReturn a copy of this deployment with updated options. Only those options passed in will be updated, all others will remain unchanged from the existing deployment. Refer to the `@serve.deployment` decorator docs for available arguments. rENz2`max_ongoing_requests` must be non-null, got None.auto1cSs&g|]\}}|dvr|tjur|qS)>rr4r>)rVALUE).0optionvaluerrr s  z&Deployment.options..zQManually setting num_replicas is not allowed when autoscaling_config is provided.rz)num_replicas is expected to larger than 0zDeprecationWarning: `version` in `Deployment.options()` has been deprecated. Explicitly specifying version will raise an error in the future!)rIrKrHrYrZr[!placement_group_fallback_strategyr\T)r)r4)-rrfrFr rAUTO_NUM_REPLICAS_USEDrecordlocalsitemsrr2user_configured_option_namesupdaterLrMr@rArBrCrcr3r=r0r1rIrKrHrYrZr[rkr\r]rbr^r_r`rar-rdictrPr creater)rr>r&r)r@rGrHrYrZr[r\rArBrCr]r^r_r`rarPrbrSrTr4rcrpnew_deployment_configrknew_replica_configrrr rUs (                        zDeployment.optionscCsPt|j|jk|j|jk|j|jk|jj|jjk|jj|jjk|jj|jjkgSr)allr0r1r2r3rIrKrH)rotherrrr __eq__s   zDeployment.__eq__cCsd|jdS)NzDeployment(name=)r:r;rrr __str__szDeployment.__str__cCst|Sr)r.r;rrr __repr__szDeployment.__repr__)NF)/r"r#r$r%r.r r rr!r,propertyr&r)r rr>intr@rrArBrCrGrrHrrIrKrNrPrQrRrrXrrfrrfloatrrr boolrUrxrzr{rrrr r@s             I rdr*cCs|jdur t|j}nd}id|jd|jjrdn|jd|jd|jd|j d|jjd|jj d |jj d |jj d |jj d |d |jjd|jjd|jjd|jjd|jj}t|D]}|dkrt||jjvrt||=qetdi|S)zConverts a live deployment object to a corresponding structured schema. Args: d: Deployment object to convert Nr&r@rBrCrAr]r^r_r`rarHrZrYr\rPrbr)rHr parse_objr&r2r]r@rBrCrAr^r_r`rar3rZrYr\rPrblistkeysrpr)rray_actor_options_schemadeployment_optionsrhrrr deployment_to_schemasZ       rsc Cs|jtjur d}n|jjdd}|jtjurd}n|j}|jtjur%d}n|j}|jtjur1d}n|j}tj|j |j |j |j |j |j|j|j|j|j|jd }||_tjddi||||d}t|j||ddS) aMCreates a deployment with parameters specified in schema. The returned deployment CANNOT be deployed immediately. It's func_or_class value is an empty string (""), which is not a valid import path. The func_or_class value must be overwritten with a valid function or class before the deployment can be deployed. NT) exclude_unset) r@rArBrCr]r^r_r`rarPrbr)r=rIrKrHrYrZr\)r&r'r(r4)rHrrfrrrYrZr\r from_defaultr@rArBrCr]r^r_r`rarPrb!_get_user_configured_option_namesrpr rsrr&)rrHrYrZr\r'r(rrr schema_to_deploymentsT     r)&loggingr7copyrtypingrrrrrrr ray.serve._private.configr r r r ray.serve._private.constantsrray.serve._private.usagerray.serve._private.utilsrrray.serve.configrray.serve.schemarrrray.util.annotationsr getLoggerrLrrrrrrrr s( $     )m3