o $i @sddlZddlZddlmZmZmZmZddlmZm Z m Z ddl m Z m Z ddlmZee Z  dde deded eed eed ef d d Zeddde d eeeeefffddZeZdS)N)AnyDictOptionalTuple)CONTROL_LOOP_INTERVAL_S'SERVE_AUTOSCALING_DECISION_COUNTERS_KEYSERVE_LOGGER_NAME)AutoscalingConfigAutoscalingContext) PublicAPIautoscaling_configtotal_num_requestsnum_running_replicasoverride_min_replicasoverride_max_replicasreturnc Cs|dkrtd||}||}|dkr|}n|}d|d|}t||} t|||kr?| |kr?| d8} |j} |j} |durK|} |durQ|} t| t | | } | S)a8Returns the number of replicas to scale to based on the given metrics. Args: autoscaling_config: The autoscaling parameters to use for this calculation. current_num_ongoing_requests (List[float]): A list of the number of ongoing requests for each replica. Assumes each entry has already been time-averaged over the desired lookback window. override_min_replicas: Overrides min_replicas from the config when calculating the final number of replicas. override_max_replicas: Overrides max_replicas from the config when calculating the final number of replicas. Returns: desired_num_replicas: The desired number of replicas to scale to, based on the input metrics and the current number of replicas. rz!Number of replicas cannot be zeroN) ValueErrorget_target_ongoing_requestsget_upscaling_factorget_downscaling_factormathceil min_replicas max_replicasmaxmin) r r rrrtarget_num_requests error_ratioscaling_factorsmoothed_error_ratiodesired_num_replicasrrr"Y/home/ubuntu/veenaModal/venv/lib/python3.10/site-packages/ray/serve/autoscaling_policy.py_calculate_desired_num_replicass*  r$alpha) stabilityctxc Cs:|j}|j}|j}|j}|j}|j}|j}|td}|dkr5|dkr1t t d| ||fS||fS|} t |||||d} | |kr\|dkrJd}|d7}|t|jtkr[d}| } n9| |kr|dkrfd}|d8}|dk} | r}|jdury|j} n |j} n|j} t d| } |t| t krd}| } nd}||t<| |fS)aThe default autoscaling policy based on basic thresholds for scaling. There is a minimum threshold for the average queue length in the cluster to scale up and a maximum threshold to scale down. Each period, a 'scale up' or 'scale down' decision is made. This decision must be made for a specified number of periods in a row before the number of replicas is actually scaled. See config options for more details. Assumes `get_decision_num_replicas` is called once every CONTROL_LOOP_PERIOD_S seconds. rr)rrrN)target_num_replicasr current_num_replicasconfigcapacity_adjusted_min_replicascapacity_adjusted_max_replicas policy_stategetrrrrrr$intupscale_delay_srdownscale_to_zero_delay_sdownscale_delay_s) r'curr_target_num_replicasr rr*r+r,r-decision_counterdecision_num_replicasr!is_scaling_to_zerodelay_sr"r"r#'replica_queue_length_autoscaling_policyXsf   r8)NN)loggingrtypingrrrrray.serve._private.constantsrrrray.serve.configr r ray.util.annotationsr getLoggerloggerr/floatr$strr8default_autoscaling_policyr"r"r"r#s:   H [