o پi@sdZddlZddlZddlmZmZddlmZddlm Z m Z e e Z GdddeZGdd d eZGd d d eZd ed efddZdS)z7 Eviction policies for LoRA adapter memory management. N)ABCabstractmethod) OrderedDict)OptionalSetc@sfeZdZdZedeeddfddZedeeedeefdd Z edeeddfd d Z dS) EvictionPolicyz7Abstract base class for LoRA adapter eviction policies.uidreturnNcCdS)zMarks an adapter as used.Nselfrr r S/home/ubuntu/.local/lib/python3.10/site-packages/sglang/srt/lora/eviction_policy.py mark_usedzEvictionPolicy.mark_used candidatescCr )z,Selects an adapter to evict from candidates.Nr )r rr r r select_victim$rzEvictionPolicy.select_victimcCr )z.Removes an adapter from the policy's tracking.Nr r r r rremove)rzEvictionPolicy.remove) __name__ __module__ __qualname____doc__rrstrrrrrr r r rrs rc@beZdZdZddZdeeddfddZd eeedeefd d Z deeddfd d Z dS)LRUEvictionPolicyz=LRU eviction policy - evicts the least recently used adapter.cCst|_d|_d|_dSNr)r access_ordertotal_accesseseviction_countr r r r__init__2s zLRUEvictionPolicy.__init__rr NcCsT|dur(t}|j|d||j|<|jd7_td|d|dSdS)NzLoRA z marked as used at )time monotonicrpoprloggerdebug)r r current_timer r rr7s zLRUEvictionPolicy.mark_usedrcCvt|jD]}||vr!td|d|jd7_|Sqd|vr4td|jd7_dSJd|)z7Select the least recently used adapter from candidates.Selected LoRA z for eviction (LRU)r!N'Selected None (base model) for evictionFz-Failed to select LRU victim from candidates: )listrkeysr%r&rr rrr r rr@ zLRUEvictionPolicy.select_victimcCs0|dur|j|dtd|ddSdS)Nz Removed LoRA z from LRU tracking)rr$r%r&r r r rrTszLRUEvictionPolicy.remove rrrrr rrrrrrr r r rr/s  rc@r)FIFOEvictionPolicyz2FIFO eviction policy - for backward compatibility.cCst|_d|_dSr)rinsertion_orderrrr r rr ]s zFIFOEvictionPolicy.__init__rr NcCs(|dur||jvrd|j|<dSdSdS)z:For FIFO, we only track insertion order (not access time).NT)r1r r r rrcs zFIFOEvictionPolicy.mark_usedrcCr()z2Select the first inserted adapter from candidates.r)z for eviction (FIFO)r!Nr*Fz.Failed to select FIFO victim from candidates: )r+r1r,r%r&rr-r r rrjr.z FIFOEvictionPolicy.select_victimcCs|dur |j|ddSdS)N)r1r$r r r rr~szFIFOEvictionPolicy.remover/r r r rr0Zs r0 policy_namer cCs*ttd}||vrtd|||S)z5Factory function to create eviction policy instances.)fifolruzUnknown eviction policy: )r0r ValueError)r2policiesr r rget_eviction_policys  r7)rloggingr"abcrr collectionsrtypingrr getLoggerrr%rrr0rr7r r r rs  +)