o bi*@sddlZddlmZddlmZmZmZddlmZddl m Z ddl m Z ddl mZddlmZmZmZdd lmZmZeeZGd d d ZdS) N) defaultdict)DictListOptional)EventLoggerAdapter)NodeTypeConfig)NodeType)ResourceRequestUtil)ClusterResourceConstraintGangResourceRequestResourceRequest) LaunchRequestTerminationRequestc@seZdZdZdefddZ     ddeeefdeee fde e e d e e e d e e ed e e ed e e ed dfddZdS)AutoscalerEventLoggerz Logs events related to the autoscaler. # TODO: - Add more logging for other events. - Rate limit the events if too spammy. loggercCs ||_dS)N)_logger)selfrrR/home/ubuntu/.local/lib/python3.10/site-packages/ray/autoscaler/v2/event_logger.py__init__s zAutoscalerEventLogger.__init__Nnode_type_configs cluster_shapelaunch_requeststerminate_requestsinfeasible_requestsinfeasible_gang_requests'infeasible_cluster_resource_constraintsreturncCsv|r8tt}|D] } || j| j7<qt|D]\} \} } d| d| d} |j| t| q|rtt}|D]} || j | jfd7<q@t j j dt j j dt j jdt j jdi}t|D]#\} \\}} } d | d | d ||d } |j| t| qg|s|rtt}|D]\}} ||}|jD]\}}|||| 7<qq|d d}dt|d} d|vr| dt|dd7} d|vr| dt|dd7} |j| d|jdt|d|r2t|}d} t|D]"\} }t|j}| |d|j7} | t|dkr&| d7} q| d7} |j| |ry|D]A}dj|jd} t|j}t|D]"\} }t|j}| |d|j7} | t|dkrk| d7} qJ| d7} |j| q7|r|D]:}d} t|jD]#\}}t|j}| |d|j7} |t|jdkr| d7} q| d7} |j| q~dSdS)zA Log any update of the cluster scheduling state. zAdding z node(s) of type .outdatedz"max number of worker nodes reachedz+max number of worker nodes per type reachedidlez Removing z nodes of type z (z).CPUrz Resized to z CPUsGPUz, z GPUsTPUz TPUszCurrent cluster shape: z6No available node types can fulfill resource requests *z@. Add suitable node types to this cluster to resolve this issue.zQNo available node types can fulfill placement group requests (detail={details}): )detailsz8No available node types can fulfill cluster constraint: N)rint instance_typecount enumerateitemsrinforcauserCauseOUTDATED MAX_NUM_NODESMAX_NUM_NODE_PER_TYPEIDLEfloat resourcesgetdebugdictr group_by_countto_resource_maprequestlenwarningformatr&requestsresource_requests)rrrrrrrrlaunch_type_countreqidxr(r)log_strtermination_by_causes_and_typecause_reason_mapr-total_resources node_type node_config resource_nameresource_quantitynum_cpusrequests_by_count req_count resource_map gang_requestinfeasible_constraintirrrlog_cluster_scheduling_updates     z3AutoscalerEventLogger.log_cluster_scheduling_update)NNNNN)__name__ __module__ __qualname____doc__rrrrrr'rrr rr r r rRrrrrrs6       r)logging collectionsrtypingrrrray._private.event.event_loggerr)ray.autoscaler.v2.instance_manager.configrray.autoscaler.v2.schemarray.autoscaler.v2.utilsr !ray.core.generated.autoscaler_pb2r r r 'ray.core.generated.instance_manager_pb2r r getLoggerrSrrrrrrs