o ci^@sddlZddlZddlZddlZddlZddlZddlmZddlm Z m Z m Z ddl m Z ddlmZmZmZmZddlZddlmZddlmZddlmZdd lmZmZdd lmZmZmZm Z m!Z!m"Z"dd l#m$Z$dd l%m&Z&m'Z'dd l(m)Z)m*Z*m+Z+ddl,m-Z-ddl.m/Z/ddl0m1Z1ddl2m3Z3ddl4m5Z5ddl6m7Z7ddl8m9Z9m:Z:m;Z;mZ>ddl?m@Z@ddlAmBZBddlCmDZEmFZGmHZHmIZJmKZLddlMmNZNmDZDmOZOmPZPmQZQddlRmSZSeTe'ZUdZVGddde ZWe GdddZXe d d!Gd"d#d#ZFe d d!Gd$d%d%ZKe Gd&d'd'ZYGd(d)d)ZZGd*d+d+Z[ej\dd,d-d.e]d/e]d0e]d1ed2ePd3eeeeee]ff d4d5Z^d6ee]e-fd7eeQd3ee]e-ffd8d9Z_dS):N)deepcopy)asdict dataclassfield)Enum)DictListOptionalTuple) cloudpickle) import_attr)RuntimeEnvSetupError)BuiltApplication build_app) DeploymentIDDeploymentStatusDeploymentStatusInfoDeploymentStatusTrigger EndpointInfoTargetCapacityDirection)DeploymentConfig)RAY_SERVE_ENABLE_TASK_EVENTSSERVE_LOGGER_NAME)deploy_args_to_deployment_infoget_app_code_versionget_deploy_args)DeploymentInfo)DeploymentStateManager) EndpointState)configure_component_logger) KVStoreBase) ServeUsageTag)DEFAULTcheck_obj_ref_ready_nowait%override_runtime_envs_except_env_varsvalidate_route_prefix)ASGIAppReplicaWrapper)AutoscalingConfig)RayServeException)ApplicationStatusApplicationStatusInfoDeploymentLanguageDeploymentStatusInfoListStatusOverview)APITyper)DeploymentDetails LoggingConfigServeApplicationSchema) ObjectRefz"serve-application-state-checkpointc@s eZdZdZdZdZdZdZdS)BuildAppStatusz%Status of the build application task.N)__name__ __module__ __qualname____doc__NO_TASK_IN_PROGRESS IN_PROGRESS SUCCEEDEDFAILEDr@r@X/home/ubuntu/.local/lib/python3.10/site-packages/ray/serve/_private/application_state.pyr3Ds r3c@sJeZdZUdZeed<eed<eed<ee ed<ee ed<e ed<dS) BuildAppTaskInfoa8Stores info on the current in-progress build app task. We use a class instead of only storing the task object ref because when a new config is deployed, there can be an outdated in-progress build app task. We attach the code version to the task info to distinguish outdated build app tasks. obj_ref code_versionconfigtarget_capacitytarget_capacity_directionfinishedN) r8r9r:r;r2__annotations__strr1r floatrboolr@r@r@rArBMs    rBT)eqc@sPeZdZUeed<dZeed<dZeed<ddZ dd Z e d e fd d Z d S)r*statusmessagerdeployment_timestampcCtjt|ddSNr7)indentjsondumpsrselfr@r@rA debug_stringez"ApplicationStatusInfo.debug_stringcCstd|jj|j|jdSNAPPLICATION_STATUS_)rNrPrQ)ApplicationStatusInfoProtorNnamerPrQrXr@r@rAto_protohs  zApplicationStatusInfo.to_protoprotocCs.t|jtdd}|t||j|jdSr\)ApplicationStatusProtoNamerNlenr)rPrQ)clsrarNr@r@rA from_protoos z ApplicationStatusInfo.from_protoN)r8r9r:r)rIrPrJrQrKrZr` classmethodr^rfr@r@r@rAr*_s   r*c@steZdZUeed<dZeed<eedZ e e ed<ddZ dede e fd d Zd d Zed eddfddZdS)r- app_statusrOr_)default_factorydeployment_statusescCrRrSrUrXr@r@rArZr[zStatusOverview.debug_stringreturncCs"|jD] }||jkr|SqdS)zGet a deployment's status by name. Args: name: Deployment's name. Returns: Optional[DeploymentStatusInfo]: The status of the deployment if it exists, otherwise None. N)rjr_)rYr_deployment_statusr@r@rAget_deployment_statuss z$StatusOverview.get_deployment_statuscCs<|j}tdd|j}t}|j|t|j||dS)NcSs|SN)r`)rNr@r@rAsz)StatusOverview.to_proto..)r_rhrj) rhr`maprjDeploymentStatusInfoListProtodeployment_status_infosextendStatusOverviewProtor_)rYapp_status_protodeployment_status_protosdeployment_status_proto_listr@r@rAr`s  zStatusOverview.to_protoracCs>t|j}g}|jjD] }|t|q ||||jdS)N)rhrjr_)r*rfrhrjrrappendrr_)rerarhrj info_protor@r@rArfs  zStatusOverview.from_protoN)r8r9r:r*rIr_rJrlistrjrrrZr rmr`rgrtrfr@r@r@rAr-ys  r-c@sfeZdZUdZeeeefed<eeed<ee ed<ee ed<ee ed<e ed<e ed<d S) ApplicationTargetStateaDefines target state of application. Target state can become inconsistent if the code version doesn't match that of the config. When that happens, a new build app task should be kicked off to reconcile the inconsistency. deployment_infos: map of deployment name to deployment info. This is - None if a config was deployed but the app hasn't finished building yet, - An empty dict if the app is deleting. code_version: Code version of all deployments in target state. None if application was deployed through serve.run. config: application config deployed by user. None if application was deployed through serve.run. target_capacity: the target_capacity to use when adjusting num_replicas. target_capacity_direction: the scale direction to use when running the Serve autoscaler. deleting: whether the application is being deleted. deployment_infosrDrErFrGdeletingapi_typeN)r8r9r:r;r rrJrrIr1rKrrLr.r@r@r@rAr{s      r{c@seZdZdZdedededefddZe de efd d Z e de efd d Z e de fd dZe defddZe deefddZe de efddZe defddZdefddZddddde eeefdede ed e ed!e ed"e ed#efd$d%Zd&d'Zd e efd(d)Zd*d+Z d,d-Z!defd.d/Z"d0ed1eddfd2d3Z#deeeffd4d5Z$d6ed!e ed"e ed7eddf d8d9Z%deefd:d;Z&de'e effdd?Z*deeefddfd@dAZ+deeefde'eeffdBdCZ,dUdDdEZ-defdFdGZ.defdHdIZ/dee0fdJdKZ1de2fdLdMZ3deee4ffdNdOZ5dVdQe dReddfdSdTZ6dS)WApplicationStatez4Manage single application states with all operationsr_deployment_state_managerendpoint_statelogging_configc Csf||_d|_||_||_d|_d|_d|_tj|_ t |_ d|_ t ddddddtjd|_||_dS)z Args: name: Application name. deployment_state_manager: State manager for all deployments in the cluster. endpoint_state: State manager for endpoints in the system. rONF)r|rDrErFrGr}r~)_name _status_msg_deployment_state_manager_endpoint_state _route_prefix _docs_path_ingress_deployment_namer) DEPLOYING_statustime_deployment_timestamp_build_app_task_infor{r.UNKNOWN _target_state_logging_config)rYr_rrrr@r@rA__init__s(  zApplicationState.__init__rkcC|jSrn)rrXr@r@rA route_prefixzApplicationState.route_prefixcCs*|jdur|jSt|j|j}|j|Srn)rrrrrget_deployment_docs_path)rYingress_deploymentr@r@rA docs_path s zApplicationState.docs_pathcCr)aStatus of the application. DEPLOYING: The build task is still running, or the deployments have started deploying but aren't healthy yet. RUNNING: All deployments are healthy. DEPLOY_FAILED: The build task failed or one or more deployments became unhealthy in the process of deploying UNHEALTHY: While the application was running, one or more deployments transition from healthy to unhealthy. DELETING: Application and its deployments are being deleted. )rrXr@r@rArNs zApplicationState.statuscCrrn)rrXr@r@rArQ)rz%ApplicationState.deployment_timestampcCs |jjdurgSt|jjS)z/List of target deployment names in application.N)rr|rzkeysrXr@r@rAtarget_deployments-s z#ApplicationState.target_deploymentscCrrn)rrXr@r@rAr4rz#ApplicationState.ingress_deploymentcCs|jjSrn)rr~rXr@r@rAr~8szApplicationState.api_typecheckpoint_datac Cs`td|jd|j|j|j|j|j|j|j |j d|jdur.| |j\|_ |_ dSdS)Nz)Recovering target state for application 'z' from checkpoint.)r~rD target_configrFrGr})loggerinfor_set_target_stater|r~rDrErFrGr} _check_routesrr)rYrr@r@rA$recover_target_state_from_checkpoint<s"  z5ApplicationState.recover_target_state_from_checkpointNF)rFrGr}r|r~rDrrFrGr}c Csl|r |tjn|tj|durd|_n|D] \}} | jr%||_qt|||||||d} | |_dS)acSet application target state. While waiting for build task to finish, this should be (None, False) When build task has finished and during normal operation, this should be (target_deployments, False) When a request to delete the application has been received, this should be ({}, True) N)r~) _update_statusr)DELETINGrritemsingressr{r) rYr|r~rDrrFrGr}r_r target_stater@r@rArSs&  z"ApplicationState._set_target_statecCs|ji|jjdddddS)zlSet target state to deleting. Wipes the target deployment infos, code version, and config. NTr|r~rDrr})rrr~rXr@r@rA_set_target_state_deletings z+ApplicationState._set_target_state_deletingcCs|jdtjd|dddS)zClears the target state and stores the config. NOTE: this currently assumes that this method is *only* called when managing apps deployed with the declarative API. NFr)rr. DECLARATIVE)rYrr@r@rA$_clear_target_state_and_store_configs  z5ApplicationState._clear_target_state_and_store_configcCs*t||jd}|j||j|dS)Nr_app_name)rrrdelete_endpointrdelete_deployment)rYr_idr@r@rA_delete_deployments z#ApplicationState._delete_deploymentcCs4|jtjkrtjd|jdddid|dS)zDelete the applicationzDeleting app ''. log_to_stderrF)extraN)rr)rrrrrrXr@r@rAdeletes   zApplicationState.deletecCs|jjo t|dkS)zCheck whether the application is already deleted. For an application to be considered deleted, the target state has to be set to deleting and all deployments have to be deleted. r)rr}rd_get_live_deploymentsrXr@r@rA is_deletedszApplicationState.is_deleteddeployment_namedeployment_infocCs|j}|dur|dstd|dt||jd}|j|||jdur<|j}|j |t |j|j t j kddS|j|dS)z(Deploys a deployment in the application.N/zInvalid route prefix "z", it must start with "/"r)routeapp_is_cross_language)r startswithr(rrrdeploydeployment_configrupdate_endpointrdeployment_languager+PYTHONr)rYrrr deployment_idrEr@r@rAapply_deployment_infos&  z&ApplicationState.apply_deployment_infocCs8||||\|_|_|j|tjddddddS)a%(Re-)deploy the application from list of deployment infos. This function should only be called to deploy an app from an imperative API (i.e., `serve.run` or Java API). Raises: RayServeException if there is more than one route prefix or docs path. N)r|r~rDrrFrG)_check_ingress_deploymentsrrrrr. IMPERATIVE)rYr|r@r@rA deploy_apps  zApplicationState.deploy_apprEdeployment_timec Cs||_t|}||jjkrhz!t|jj|}||\|_|_|j |jjt j ||||dWdSt t tfyI|||tjtYdStyg|||tjd|jdtYdSw|jr|jjstd|jdt|jj|||jjj ds|jjj drt!j"#dtd |jd t$j%|jt&d '|j(||j)|j*|j+}t,|||||d d |_dS)aApply the config to the application. If the code version matches that of the current live deployments then it only applies the updated config to the deployment state manager. If the code version doesn't match, this will re-build the application. This function should only be called to (re-)deploy an app from the declarative API (i.e., through the REST API). )rDr~r|rrFrGAUnexpected error occurred while applying config for application '': z%Received new config for application 'z'. Cancelling previous request. container image_uri1zImporting and building app 'r) runtime_envenable_task_eventsF)rCrDrErFrGrHN)-rrrrDoverride_deployment_infor|rrrrr.r TypeError ValueErrorr(rrr) DEPLOY_FAILED traceback format_exc ExceptionrrrHrrraycancelrCrErgetr!APP_CONTAINER_RUNTIME_ENV_USEDrecordbuild_serve_applicationoptionsrremote import_pathr_argsrrB)rYrErFrGrconfig_versionoverrided_infosbuild_app_obj_refr@r@rAapply_app_configs               z!ApplicationState.apply_app_configcCs|j|jSrn)rget_deployments_in_applicationrrXr@r@rArKsz&ApplicationState._get_live_deploymentscCs|jjr tjdfS|}t|ddd}|jtjkr+dd|D}tjd|dfS|jtj krAd d|D}tj d |d fS|jtj krLtj dfS|jtj tj fvr`|jtjkr`tj dfStjdfS) aCheck deployment statuses and target state, and determine the corresponding application status. Returns: Status (ApplicationStatus): RUNNING: all deployments are healthy or autoscaling. DEPLOYING: there is one or more updating deployments, and there are no unhealthy deployments. DEPLOY_FAILED: one or more deployments became unhealthy while the application was deploying. UNHEALTHY: one or more deployments became unhealthy while the application was running. DELETING: the application is being deleted. Error message (str): Non-empty string if status is DEPLOY_FAILED or UNHEALTHY rOcSrrn)rank)rr@r@rArogsz8ApplicationState._determine_app_status..)keycSg|] }|jtjkr|jqSr@)rNrrr_.0sr@r@rA i  z:ApplicationState._determine_app_status..z!Failed to update the deployments .cSrr@)rNr UNHEALTHYr_rr@r@rArsrzThe deployments z are UNHEALTHY.)rr}r)rget_deployments_statusesminrNrrrUPDATINGr UPSCALING DOWNSCALINGstatus_triggerrCONFIG_UPDATE_STARTEDRUNNING)rYrjlowest_rank_statusfailed_deploymentsunhealthy_deployment_namesr@r@rA_determine_app_statusNs8          z&ApplicationState._determine_app_statusc sjdus jjrdtjdfStjjsdtjdfSdj_z(tjj\}}|dur8t dj dndtj dj d|fWSWn3t yadj d t}dtj |fYStyzd j d t}dtj |fYSwz fd d |D}t|jj}|\__|tjdfWStttfydtj tfYStydj d t}dtj |fYSw)aIf necessary, reconcile the in-progress build task. Returns: Deploy arguments (Dict[str, DeploymentInfo]): The deploy arguments returned from the build app task and their code version. Status (BuildAppStatus): NO_TASK_IN_PROGRESS: There is no build task to reconcile. SUCCEEDED: Task finished successfully. FAILED: An error occurred during execution of build app task IN_PROGRESS: Task hasn't finished yet. Error message (str): Non-empty string if status is DEPLOY_FAILED or UNHEALTHY NrOTzImported and built app 'z' successfully.zDeploying app 'z' failed with exception: zRuntime env setup for app 'z ' failed: z7Unexpected error occurred while deploying application 'rcs*i|]}|dtdi|djiqSrrr@)rrrparamsrXr@rA s z>ApplicationState._reconcile_build_app_task..r)rrHr3r<r#rCr=rrrrrr?r rrrrrErrrr>rrr()rYrerr error_msgr|rr@rXrA_reconcile_build_app_tasksb        z*ApplicationState._reconcile_build_app_taskcCsTd}|D]}t|jjrt|jjtr|d7}q|dkr(td|jddS)zCheck @serve.ingress of deployments in app. Raises: RayServeException if more than one @serve.ingress is found among deployments. rr43Found multiple FastAPI deployments in application "z`".Please only include one deployment with @serve.ingressin your application to avoid this issue.N) valuesinspectisclassreplica_configdeployment_def issubclassr&r(r)rYr|num_ingress_deploymentsrr@r@rArs  z+ApplicationState._check_ingress_deploymentscCsd}d}d}d}|D]}|jdur|j}|d7}|jdur&|j}|d7}q |dkr4td|jd|dkrAtd|jd||fS)aCheck route prefixes and docs paths of deployments in app. There should only be one non-null route prefix. If there is one, set it as the application route prefix. This function must be run every control loop iteration because the target config could be updated without kicking off a new task. Returns: tuple of route prefix, docs path. Raises: RayServeException if more than one route prefix or docs path is found among deployments. rNr4z0Found multiple route prefixes from application "zP", Please specify only one route prefix for the application to avoid this issue.z+Found multiple deployments in application "z" that have a docs path. This may be due to using multiple FastAPI deployments in your application. Please only include one deployment with a docs path in your application to avoid this issue.)rrrr(r)rYr|num_route_prefixesnum_docs_pathsrrrr@r@rArs*     zApplicationState._check_routescCs|jjD]/\}}t|}|j|jj|jjd|jjr/|jjjr/|j jdur/|jjj|j _| ||q| D] }||j vrF| |q:dS)zReconcile target deployments in application target state. Ensure each deployment is running on up-to-date info, and remove outdated deployments from the application. )new_target_capacitynew_target_capacity_directionN)rr|rrset_target_capacityrFrGrErrrrrr)rYrr deploy_infor@r@rA_reconcile_target_deploymentss&     z.ApplicationState._reconcile_target_deploymentscCs|\}}}|tjkr"|j||jj|jj|jj|jj |jj dn |tj kr.| t j||jjdurD||\}}| |||jjrL|SdS)a&Attempts to reconcile this application to match its target state. Updates the application status and status message based on the current state of the system. Returns: A boolean indicating whether the application is ready to be deleted. )r|rDr~rrFrGNF)rr3r>rrrDrr~rErFrGr?rr)rr|rrr}r)rYinfos task_statusmsgrN status_msgr@r@rAupdate8s&     zApplicationState.updatecCrrn)rrXr@r@rAget_checkpoint_data_sz$ApplicationState.get_checkpoint_datacs fddjD}j|S)z(Return all deployment status informationcsg|] }t|jdqSr)rr)r deploymentrXr@rArds z=ApplicationState.get_deployments_statuses..)rrget_deployment_statuses)rY deploymentsr@rXrArbs  z)ApplicationState.get_deployments_statusescCst|j|j|jdS)z)Return the application status informationrPrQ)r*rrrrXr@r@rAget_application_status_infojs z,ApplicationState.get_application_status_infocs&fddjD}dd|DS)aGets detailed info on all live deployments in this application. (Does not include deleted deployments.) Returns: A dictionary of deployment infos. The set of deployment info returned may not be the full list of deployments that are part of the application. This can happen when the application is still deploying and bringing up deployments, or when the application is deleting and some deployments have been deleted. c s$i|]}|jt|jdqSr)rget_deployment_detailsrr)rrrXr@rAr}s  z.cSsi|] \}}|dur||qSrnr@)rkvr@r@rArs)rr)rYdetailsr@rXrAlist_deployment_detailsrs z(ApplicationState.list_deployment_detailsrOrNrcCs8|r|tjtjfvr||jkrt|||_||_dSrn)r)rrrrerrorr)rYrNrr@r@rArs   zApplicationState._update_statusrkN)rO)7r8r9r:r;rJrrr0rpropertyr rrr)rNrKrQrrrr.r~r{rrrr1rrLrrrrrrrrrrr rr3rrrrrrrrr*rr/r!rr@r@r@rArs (  ,    # ^:E     1$'rc @seZdZdedededefddZddZd e d d fd d Z de e e e fd d fddZ d e de e d d fddZdd d dde ededeedeefddZde d e e fddZde d e efdd Zd e d efd!d"Zd e d efd#d$Zde d ee fd%d&Zd e d ee fd'd(Zd e d ee fd)d*Zd e d efd+d,Zd e e effd-d.Z d e d e e e!ffd/d0Z"d1d2Z#d9d3d4Z$d e%fd5d6Z&d9d7d8Z'd S):ApplicationStateManagerrrkv_storercCs0||_||_||_||_d|_i|_|dS)NF)rr _kv_storer_shutting_down_application_states_recover_from_checkpoint)rYrrr&rr@r@rArs z ApplicationStateManager.__init__cCs`|jt}|dur,t|}|D]\}}t||j|j|j }| |||j |<qdSdSrn) r'rCHECKPOINT_KEYr loadsrrrrrrr))rY checkpointapplication_state_inforr app_stater@r@rAr*s    z0ApplicationStateManager._recover_from_checkpointr_rkNcCs ||jvrdS|j|dS)zDelete application by nameN)r)rrYr_r@r@rA delete_apps z"ApplicationStateManager.delete_appname_to_deployment_argsc sdd|jD}|D]`\}|D]+}|d}|dur q||}|dur;|kr;td|d|dd||<q|jvrRt|j|j|j|j<tj t t |jfdd|D}|j |qdS) NcSs.i|]\}}|jdur|jtjks|j|qSrn)rrNr)rrrr/r@r@rArs   z7ApplicationStateManager.deploy_apps..rzPrefix z is being used by application "z!". Failed to deploy application "z".cs(i|]}|dtdi|diqSr)rrr_r@rArs )r)rrr(rrrrr!NUM_APPSrrJrdr)rYr2live_route_prefixesdeployment_args deploy_paramdeploy_app_prefixexisting_app_namer|r@r4rA deploy_appss>     z#ApplicationStateManager.deploy_appsr7cCs|||idS)aDeploy the specified app to the list of deployment arguments. This function should only be called if the app is being deployed through serve.run instead of from a config. Args: name: application name deployment_args_list: arguments for deploying a list of deployments. Raises: RayServeException: If the list of deployments is trying to use a route prefix that is already used by another application N)r;)rYr_r7r@r@rArsz"ApplicationStateManager.deploy_appr)rrFrG app_configsrrFrGc Cs|D]/}|j|jvr$td|jdt|j|j|j|jd|j|j<|j|jj||||dqdd|j D}dd|D}||D]}| |qGt j tt|jdS) a Declaratively apply the list of application configs. The applications will be reconciled to match the target state of the config. Any applications previously deployed declaratively that are *not* present in the list will be deleted. zDeploying new app 'r)rr)rcSs h|] \}}|jtjkr|qSr@)r~r.r)rr_r/r@r@rA s  z.cSsh|]}|jqSr@r4)r app_configr@r@rAr="sN)r_r)rrrrrrrrr1r!r5rrJrd) rYr<rrFrGr> existing_appsapps_in_config app_to_deleter@r@rAapply_app_configss,     z)ApplicationStateManager.apply_app_configsrcCs||jvrgS|j|jS)z'Return all deployment names by app name)r)rrYrr@r@rAget_deployments(s  z'ApplicationStateManager.get_deploymentscCs||jvrgS|j|S)z*Return all deployment statuses by app name)r)rrCr@r@rAr. z0ApplicationStateManager.get_deployments_statusescCs||jvrtjS|j|jSrn)r)r) NOT_STARTEDrNr0r@r@rAget_app_status4s  z&ApplicationStateManager.get_app_statuscCs0||jvrttjd|dddS|j|S)Nz Application z doesn't existrr)r)r*r)rFrr0r@r@rAget_app_status_info:s  z+ApplicationStateManager.get_app_status_infocC |j|jSrn)r)rrCr@r@rA get_docs_pathC z%ApplicationStateManager.get_docs_pathcCrIrn)r)rr0r@r@rAget_route_prefixFrKz(ApplicationStateManager.get_route_prefixcCs||jvrdS|j|jSrn)r)rr0r@r@rAget_ingress_deployment_nameIs  z3ApplicationStateManager.get_ingress_deployment_namecCrIrn)r)r~r0r@r@rAget_app_sourceOrKz&ApplicationStateManager.get_app_sourcecsfddjDS)z5Return a dictionary with {app name: application info}csi|] }|j|qSr@)r)r)rr_rXr@rArTsz=ApplicationStateManager.list_app_statuses..)r)rXr@rXrAlist_app_statusesRs z)ApplicationStateManager.list_app_statusescCs||jvriS|j|S)z?Gets detailed info on all deployments in specified application.)r)r!r0r@r@rAr!YrEz/ApplicationStateManager.list_deployment_detailscCs|g}|jD]\}}|}|r||td|dqt|dkr<|D]}|j|=q(tj t t|jdSdS)zUpdate each application statez Application 'z' deleted successfully.rN) r)rrrxrdebugrdr!r5rrJ)rYapps_to_be_deletedr_appready_to_be_deletedrr@r@rAr_s   zApplicationStateManager.updatecCs.d|_|jD]}|q|jtdS)NT)r(r)rrr'r+)rYr/r@r@rAshutdownms z ApplicationStateManager.shutdowncCs|jotdd|jDS)zReturn whether all applications have shut down. Iterate through all application states and check if all their applications are deleted. css|]}|VqdSrn)r)rr/r@r@rA {s z@ApplicationStateManager.is_ready_for_shutdown..)r(allr)rrXr@r@rAis_ready_for_shutdownusz-ApplicationStateManager.is_ready_for_shutdowncCs6|jrdSdd|jD}|jtt|dS)z-Write a checkpoint of all application states.NcSsi|] \}}||qSr@)rr3r@r@rArsz;ApplicationStateManager.save_checkpoint..)r(r)rr'putr+r rW)rYr.r@r@rAsave_checkpointsz'ApplicationStateManager.save_checkpointr#)(r8r9r:rrr r0rr*rJr1rrr;rr1rKr rrBrDrrr)rGr*rHrJrLrMr.rNrOr/r!rrTrLrWrYr@r@r@rAr%sP / -   r%r4)num_cpus max_callsrrDr_rrrkc Cs^tdd|dt|dzuddlm}|rd|nd}td |d |d |t||}g}t||t j d } d} | j D].} t | jrUt| jtrU| d 7} | j| jk} |t| j| j| | j|| rjdnd| jdqC| d krdd| jdfWS|dfWStytdYdStytd|dtdtfYSw)aImport and build a Serve application. Args: import_path: import path to top-level bound deployment. code_version: code version inferred from app config. All deployment versions are set to this code version. name: application name. If specified, application will be deployed without removing existing applications. args: Arguments to be passed to the application builder. logging_config: the logging config for the build app task. Returns: Deploy arguments: a list of deployment arguments if application was built successfully, otherwise None. Error message: a string if an error was raised, otherwise None. controllerbuild__)component_name component_idrr),call_user_app_builder_with_args_if_necessaryz with arguments rOzImporting application ''r)r_default_runtime_envr4rN)r_rrrversionrrrzb". Please only include one deployment with @serve.ingress in your application to avoid this issue.zLExisting config deployment request terminated because of keyboard interrupt.)NNz!Exception importing application 'z'. )rosgetpidray.serve._private.apirarrr rrget_runtime_contextrrrr func_or_classrr&r_ingress_deployment_namerxrr_replica_config_deployment_configrKeyboardInterruptrr"rr) rrDr_rrra args_info_strrRdeploy_args_list built_appr r is_ingressr@r@rArsh         rr|override_configcCs|t|}|dur |S|jdd}|dg}|D]}d|vr%|d|d<|d}||vr;td|dt|||}|j}|d t||d d kr}d|d <t }|d pf|jj } | rn|| t di||d <t j d i} |j} |j} d|vr|di} n| jpi} |d| j}|d| j}|d| j}| dr| dds| ddrt jd t| | di}| d|i| j| |||d| | d<|dd||tdi|| d<|jdi| ||<||j}|j dur|j|j krtdq|dtj}t|t| D]}|tjur:|j!dur:||_!q)|S)aOverride deployment infos with options from app config. Args: app_name: application name deployment_infos: deployment info loaded from code override_config: application config deployed by user with options to override those loaded from code. Returns: the updated deployment infos. Raises: ValueError: If config options have invalid values. TypeError: If config options have invalid types. NT) exclude_unsetrmax_ongoing_requestsr_z Deployment 'z' does not exist. Available: user_configured_option_names num_replicasautoautoscaling_configrray_actor_optionsplacement_group_bundlesplacement_group_strategymax_replicas_per_noderrr)ryrzr{r|rrziAutoscaling will never happen, because 'max_ongoing_requests' is less than 'target_ongoing_requests' now.rr@)"rdictrrrzrrrsetr'defaultrxr!AUTO_NUM_REPLICAS_USEDrrrpopryrzr{r|%DEPLOYMENT_CONTAINER_RUNTIME_ENV_USEDr$rrtget_target_ongoing_requestsrwarningr"VALUEr%rr)r|rr config_dictdeployment_override_optionsrrroriginal_options new_configrxoverride_optionsrapp_runtime_envoverride_actor_options override_placement_group_bundles!override_placement_group_strategyoverride_max_replicas_per_node merged_envrapp_route_prefixrr@r@rArs                    r)`rrVloggingrerrcopyr dataclassesrrrenumrtypingrrr r rr ray._common.utilsr ray.exceptionsr ray.serve._private.build_apprrray.serve._private.commonrrrrrrray.serve._private.configrray.serve._private.constantsrrray.serve._private.deploy_utilsrrr"ray.serve._private.deployment_infor#ray.serve._private.deployment_stater!ray.serve._private.endpoint_stater ray.serve._private.logging_utilsr#ray.serve._private.storage.kv_storer ray.serve._private.usager!ray.serve._private.utilsr"r#r$r% ray.serve.apir&ray.serve.configr'ray.serve.exceptionsr(ray.serve.generated.serve_pb2r)rbr*r^r+r,rqr-rtray.serve.schemar.r/r0r1 ray.typesr2 getLoggerrr+r3rBr{rr%rrJrrr@r@r@rAs                   A@ } U