o پi@sHddlmZddlmZmZddlZeGdddZGdddZdS)) dataclass)ListTupleNc@sFeZdZUdZeed<ejed<ejed<e ed<e ed<e ed<dS) FlattenedTensorMetadataz+Metadata for a tensor in a flattened bucketnameshapedtype start_idxend_idxnumelN) __name__ __module__ __qualname____doc__str__annotations__torchSizerintrrX/home/ubuntu/.local/lib/python3.10/site-packages/sglang/srt/weight_sync/tensor_bucket.pyrs    rc@seZdZdZdZ   ddeeeej fdej dee fddZ d ej fd d Z d ee fd d Z d eeeej ffddZdS)FlattenedTensorBucketz A bucket that flattens multiple tensors into a single tensor for efficient processing while preserving all metadata needed for reconstruction. TN named_tensorsflattened_tensormetadatac Cs|durZdgt||_d|_|stdd}dgt|}t|D]-\}\}}|tj} | ||<| } t ||j |j ||| | d} | |j|<|| 7}q"tj |dd|_dS|dusb|durftd||_||_dS)aO Initialize a tensor bucket from a list of named tensors OR from pre-flattened data. Args: named_tensors: List of (name, tensor) tuples (for creating new bucket) flattened_tensor: Pre-flattened tensor (for reconstruction) metadata: Pre-computed metadata (for reconstruction) Nz!Cannot create empty tensor bucketr)rrrr r r )dimzGMust provide either named_tensors or both flattened_tensor and metadata)lenrr ValueError enumerateflattenviewruint8r rrrcat) selfrrr current_idxflattened_tensorsirtensor flattenedr metadata_objrrr__init__s8    zFlattenedTensorBucket.__init__returncC|jS)z6Get the flattened tensor containing all bucket tensors)rr#rrrget_flattened_tensorRz*FlattenedTensorBucket.get_flattened_tensorcCr,)z*Get metadata for all tensors in the bucket)rr-rrr get_metadataVr/z"FlattenedTensorBucket.get_metadatacCsXdgt|j}t|jD]\}}|j|j|j|j|j }|j |f||<q |S)z Reconstruct original tensors from flattened tensor with optimized performance. Uses memory-efficient operations to minimize allocations and copies. N) rrrrr r r rreshaperr)r# reconstructedr&metar'rrrreconstruct_tensorsZsz)FlattenedTensorBucket.reconstruct_tensors)NNN)r r rrsupports_multi_dtypesrrrrTensorrr*r.r0r4rrrrrs  6 r) dataclassesrtypingrrrrrrrrrs