o پi&@sddlZddlmZddlmZmZmZddlmZm Z m Z m Z ddl m Z ddlmZddlmZedd Gd d d ZGd d d ZdS)N) OrderedDict) dataclassfieldfields)DictListOptionalUnion)uuid4)ConcurrentCounter)RWLockT)frozenc@sneZdZUdZedddZeed<dZe eed<dZ e eed<dZ e e ed <d d Z d efd dZdS)LoRARefa) Reference record for a LoRA model. This object guarantees a unique ``lora_id`` and may include ``lora_name``, ``lora_path``, and ``pinned``. The ID eliminates conflicts from reused LoRA names or paths and can be used to generate deterministic cache keys (e.g., radix cache). cCstjSN)r hexrrQ/home/ubuntu/.local/lib/python3.10/site-packages/sglang/srt/lora/lora_registry.py$szLoRARef.)default_factorylora_idN lora_name lora_pathpinnedcCs|jdur tddS)Nzlora_id cannot be None)r ValueErrorselfrrr __post_init__)s zLoRARef.__post_init__returncs2fddtD}jjdd|dS)Ncs0g|]}t|jdur|jdqS)N=)getattrname).0frvaluerr .s z#LoRARef.__str__..(z, ))r __class____name__join)rpartsrr#r__str__-s zLoRARef.__str__)r) __module__ __qualname____doc__rrstr__annotations__rrrrboolrr,rrrrrs rc@seZdZdZddeeefddZdefddZd e d e fd d Z d e e ee fd e e ee ffd dZ de e ee ffddZ de fddZd ee fddZd ddZdefddZed efddZd ee effddZdS)! LoRARegistrya The central registry to keep track of available LoRA adapters and ongoing LoRA requests. The `LoRARegistry` resides in the tokenizer manager process and acts as the single source of truth for all available LoRA adapters. It supports concurrent inference and dynamic adapter updates through a two-phase update / eventual consistency model between the tokenizer manager process and the scheduler processes. N lora_pathscCsX|dustdd|DsJdt|_t|_i|_|r(|D] }||q dSdS)Ncss|]}t|tVqdSr) isinstancer)r!lorarrr @s  z(LoRARegistry.__init__..zserver_args.lora_paths should have been normalized to LoRARef objects during server initialization. Please file an issue if you see this error.)allr _registry_lockr _registry _counters_register_adapter)rr4lora_refrrr__init__?s   zLoRARegistry.__init__r=c sN|jj4IdH||WdIdHdS1IdHs wYdS)z Register a new LoRARef object in the registry. Args: lora_ref (LoRARef): The LoRARef object to register. N)r9 writer_lockr<rr=rrrregisterXs .zLoRARegistry.registerrrc s|jj4IdH)|j|d}|dur"td|d|j|j|=WdIdH|jS1IdHs8wY|jS)z Unregister a LoRARef object from the registry and returns the removed LoRA ID. Args: lora_name (str): The name of the LoRA model to unregister. NLoRA with name z does not exist. Loaded LoRAs: )r9r?r:getrkeysr)rrr=rrr unregisterbs zLoRARegistry.unregisterc sdtdtffdd t|trAjj4IdH|}WdIdHn 1IdHs.wYj|jddIdH|St|tr~jj4IdHfdd |D}WdIdHn 1IdHshwYtjfd d |DIdH|St d ) z Queries registry for LoRA IDs based on LoRA names and start tracking the usage of the corresponding LoRA adapters by incrementing its counter. r rcsP|durdSj|d}|durtd|djdj||jS)Nz6The following requested LoRA adapters are not loaded: z Loaded adapters: .)r:rCrrD move_to_endr)r r=rrr_lookupys z%LoRARegistry.acquire.._lookupNF notify_allcsg|]}|qSrr)r!r )rHrrr%sz(LoRARegistry.acquire..cs&g|]}|durj|jddqS)NFrI)r; incrementr!idrrrr%s z7lora_name must be either a string or a list of strings.) r0r5r9r?r;rKlistasynciogather TypeError)rrrlora_idsr)rHrracquiress&  ( (  zLoRARegistry.acquirerc sjj4IdH5t|trj|IdHnt|tr/tjfdd|DIdHnt dWdIdHdS1IdHsDwYdS)zj Decrements the usage counter for a LoRA adapter, indicating that it is no longer in use. Ncs"g|] }|durj|qSr)r; decrementrLrrrr%s  z(LoRARegistry.release..z5lora_id must be either a string or a list of strings.) r9 reader_lockr5r0r;rTrNrOrPrQrrrrrreleases    .zLoRARegistry.releasecsF||jvs Jd||jvsJd|j|IdH|j|=dS)at Waits until the usage counter for a LoRA adapter reaches zero, indicating that it is no longer in use. This is useful for ensuring that a LoRA adapter can be safely unloaded. This method itself is not synchronized, which is safe because it should only be called during LoRA unloading, which itself is guaranteed to be sequential. zTwait_for_unload should only be called after the LoRA adapter has been unregistered. zIThe LoRA ID should still have a counter if it has been registered before.N)r:r; wait_for_zerorVrrrwait_for_unloads   zLoRARegistry.wait_for_unloadc sv|jj4IdH%g}|D]}||jvr|j|q||q|WdIdHS1IdHs4wYdS)z^ Returns all LoRA adapters in lora_name that are not found in self._registry. N)r9r?r:rGappend)rrunregistered_lorasr rrrget_unregistered_lorass  0z#LoRARegistry.get_unregistered_lorasFc s|jj4IdH<|stt|jdWdIdHS|jD]\}}|js8|WdIdHSq# WdIdHdS1IdHsKwYdS)z Returns the least recently used LoRA adapter. If exclude_pinned is True, then return the LRU LoRA adapter that isn't pinned. N)r9rUnextiterr:itemsr)rexclude_pinnedrr=rrr lru_lora_names0zLoRARegistry.lru_lora_namecCsF|j|jvrtd|jd|j||j|j<t|j|j<|S)zD Internal helper method to register a LoRA adapter. rBz already exists. Loaded LoRAs: )rr:rrDr r;rr@rrrr<s  zLoRARegistry._register_adaptercC t|jS)zQ Returns the total number of LoRA adapters currently registered. )lenr:rrrrnum_registered_lorass z!LoRARegistry.num_registered_lorascCrb)z Returns a dictionary of all registered LoRA adapters. Returns: Dict[str, LoRARef]: A dictionary mapping LoRA names to LoRARef objects. )dictr:rrrrget_all_adapterss zLoRARegistry.get_all_adaptersr)F)r)r-r.r/rrrr>rAr0rEr rSrWrYsetr\rar<propertyintrdrrfrrrrr36s *)  r3)rO collectionsr dataclassesrrrtypingrrrr uuidr sglang.srt.utilsr sglang.srt.utils.aio_rwlockr rr3rrrrs