o ci @sjddlZddlZddlZddlmZddlmZdZdZe e Z GdddZ e add Zd d ZdS) N) nccl_util)ENV c@s(eZdZdZddZddZddZdS) StreamPoolaThe class that represents a stream pool associated with a GPU. When multistream is enabled, we will allocate a pool of streams for each GPU, and get available stream from this pool when a collective kernel is initialized. This enables overlapping computation/communication kernels using multiple CUDA streams, given that the streams a appropriately synchronized. The class is thread-safe. Args: device_idx: the absolute index of the device for this pool. cCs6||_d|_t|_dgt|_d|_t|_dS)NFr) device_idx _initialized threadingLock_initialized_lockNCCL_STREAM_POOL_SIZE_pool_counter _pool_lock)selfrrd/home/ubuntu/.local/lib/python3.10/site-packages/ray/util/collective/collective_group/cuda_stream.py__init__s   zStreamPool.__init__cCsV|j|js ||j|j|j|j}|jdt|_|j|S)zGet an available stream from the pool. The function locks the stream pool and releases the lock before returning. Returns: stream (cupy.cuda.Stream): the returned stream from pool. ) r acquirer _init_oncereleaserr rr )rstreamrrr get_stream&s     zStreamPool.get_streamcCst|j1ttD]$}tjjr"t dt j j ddd|j |<q t dt j j j|j |<q Wdn1s:wYd|_dS)z)Initialize the stream pool only for once.zNCCL multistream enabled.F)null non_blockingzNCCL multistream disabled.NT)rDevicerranger rNCCL_USE_MULTISTREAMvalloggerdebugcupycudaStreamr r _init_flag)rirrrr=s    zStreamPool._init_onceN)__name__ __module__ __qualname____doc__rrrrrrrrs   rcCsttD]}t|t|<qdS)N)rMAX_GPU_PER_ACTORr_device_stream_pool_map)r&rrr_init_stream_poolPs r-cCs*t}|ts t|t|S)z)Get the CUDA stream pool of a GPU device.)r r rr,r-r)rlockrrrget_stream_poolVs r/)loggingr r"$ray.util.collective.collective_grouprray.util.collective.constrr r+ getLoggerr'r rdictr,r-r/rrrrs   ?