o zij @sddlmZddlmZddlZddlmZddlmZddl m Z ddl m Z ddl mZdd lmZmZdd lmZGd d d eZdS) ) annotations)AnyN)Module)override) Accelerator) CheckpointIO) Precision)Strategy TBroadcast)_DEVICEcseZdZdZ    d1d2fd d Zeed3ddZeed4ddZed5ddZ ed6dd Z ed7d8d&d'Z ed9d(d)Z ed:d;d/d0Z ZS)<SingleDeviceStrategyz7Strategy that handles communication on a single device.cpuNdevicer acceleratorAccelerator | None checkpoint_ioCheckpointIO | None precisionPrecision | NonecsDtj|||dt|tjst|}||_d|_d|_d|_dS)N)rrrr) super__init__ isinstancetorchr _root_device global_rank local_rank world_size)selfrrrr __class__]/home/ubuntu/.local/lib/python3.10/site-packages/lightning_fabric/strategies/single_device.pyr s   zSingleDeviceStrategy.__init__return torch.devicecCs|jSN)rrr!r!r" root_device/sz SingleDeviceStrategy.root_deviceboolcCsdS)NTr!r&r!r!r"is_global_zero4z#SingleDeviceStrategy.is_global_zeromodulerNonecCs||jdSr%)tor')rr+r!r!r"module_to_device9sz%SingleDeviceStrategy.module_to_devicetensorAny | torch.TensorargsrkwargscO|S)aReduces a tensor from several distributed processes to one aggregated tensor. As this plugin only operates with a single device, the reduction is simply the identity. Args: tensor: the tensor to sync and reduce *args: ignored **kwargs: ignored Return: the unmodified input as reduction is not needed for single process operation r!)rr/r1r2r!r!r" all_reduce=szSingleDeviceStrategy.all_reduceF torch.Tensorgroup Any | None sync_gradscCr3)z*Perform a ``all_gather`` on all processes.r!)rr/r6r8r!r!r" all_gatherMr*zSingleDeviceStrategy.all_gathercOsdSr%r!)rr1r2r!r!r"barrierRzSingleDeviceStrategy.barrierrobjr srcintcCr3r%r!)rr<r=r!r!r" broadcastVr;zSingleDeviceStrategy.broadcast)r NNN)rr rrrrrr)r#r$)r#r()r+rr#r,)r/r0r1rr2rr#r0)NF)r/r5r6r7r8r(r#r5)r1rr2rr#r,)r)r<r r=r>r#r )__name__ __module__ __qualname____doc__rpropertyrr'r)r.r4r9r:r? __classcell__r!r!rr"r s.   r ) __future__rtypingrrtorch.nnrtyping_extensionsrlightning_fabric.acceleratorsr)lightning_fabric.plugins.io.checkpoint_ior"lightning_fabric.plugins.precisionr$lightning_fabric.strategies.strategyr r lightning_fabric.utilities.typesr r r!r!r!r"s