o ۾ij@sddlZddlZddlZddlmZddlmZddlmZddl m Z ddl m Z e e ZGdddZGdd d Zdd d Zdd dZded efddZdeeded efddZdS)N)Counter)suppress)Any) init_loggerc@s6eZdZdZd dedBddfddZdefddZdS) GCDebugConfigz Config for GC Debugger. - 0: disable GC debugger - 1: enable GC debugger with gc.collect elapsed times - '{"top_objects":5}': enable GC debugger with top 5 collected objects N gc_debug_confreturncCsd|_d|_|r |dkr n-|dkrd|_n%zt|}d|_|dd|_Wnty9d|_tdtj Ynwt dt |dS) NF01T top_objectsz!Failed to parse VLLM_GC_DEBUG(%s)zGC Debug Config. %s) enabledr jsonloadsget Exceptionloggererrorenvs VLLM_GC_DEBUGdebugstr)selfr json_confrG/home/ubuntu/.local/lib/python3.10/site-packages/vllm/utils/gc_utils.py__init__s   zGCDebugConfig.__init__cCsd|jd|jS)Nzenabled:z ,top_objects:)r r )rrrr__repr__*szGCDebugConfig.__repr__N)__name__ __module__ __qualname____doc__rrrrrrrrsrc@s@eZdZdZdeddfddZdedeeefddfd d Z dS) GCDebuggerz Debugger for GC which logs helpful information for GC understanding. To enable, you should call maybe_attach_gc_debug_callback in the process. configrNcCs ||_t|_d|_d|_dS)Nr)r$time monotonic_ns start_time_ns num_objectsgc_top_collected_objects)rr$rrrr4s  zGCDebugger.__init__phaseinfoc Cs|d}|dur dS|dkr(t|_t|}t||_t||j j |_ dS|dkrQt|jd}t d|t|dd|j||j rLd |j nd dSdS) zA Handles a GC event (e.g. GC start or GC finish) generationNstartstopg.AzSGC took %.3fms to complete. Collected %s objects (out of %d) in GC generation %d.%s collected?z Top collected objects: r%)rr&r'r(gc get_objectslenr)!_compute_top_gc_collected_objectsr$r r*rr,r)rr+r,r-objects elpased_msrrrhandle=s.     zGCDebugger.handle) rr r!r"rrrdictintr8rrrrr#.s" r#rcCs*tdtdtdtdS)z Freeze all objects tracked by the garbage collector. It should be invoked after server init / warmup, to reduce GC overhead from static objects during serving time. rN)r2collectfreezerrrrfreeze_gc_heap`s    r?csNttj}|jr%t|dtdtttfddffdd }tj |dSdS)zI Attached a callback for GC debug when VLLM_GC_DEBUG is enabled. r+r,rNcs||dSr)r8)r+r,debuggerrr gc_callbackwsz3maybe_attach_gc_debug_callback..gc_callback) rrrr r#rr9r:r2 callbacksappend)r$rBrr@rmaybe_attach_gc_debug_callbackos "rEocCsPd}ttdt|d}Wdn1swYtt||S)z Detailed object type. TODO(Jialin): Further enhance the detailed type with element types for easier debugging. We tried but occasionally it would run into signals which kills the engine. r%z(size:)N)rrr4rtype)rFsize_strrrr_compute_detailed_type}s  rJr6topcCs8|dkrdSdd|D}dddt||DS)z+ Group collected objects by types. rr%cSsg|]}t|qSr)rJ).0rFrrr sz5_compute_top_gc_collected_objects.. css$|] \}}|dd|VqdS)z>5:Nr)rL object_typecountrrr s  z4_compute_top_gc_collected_objects..)joinr most_common)r6rK object_typesrrrr5s   r5)rN)r2rr& collectionsr contextlibrtypingr vllm.envsr vllm.loggerrrrrr#r?rErrJlistr:r5rrrrs      2