o zi5@sfddlZddlZddlmZddlmZddlmZmZm Z ddl m Z e e ZGddde ZdS)N)Any)override)_XLA_AVAILABLE_XLA_GREATER_EQUAL_2_1XLAAccelerator)ClusterEnvironmentcs*eZdZdZdededdffdd Zeedefdd Z eede fd d Z eede fd d Z eedefddZeejddde fddZede ddfddZeejddde fddZede ddfddZeejddde fddZeejddde fddZZS) XLAEnvironmentaCluster environment for training on a TPU Pod with the `PyTorch/XLA `_ library. A list of environment variables set by XLA can be found `here `_. argskwargsreturnNcs&tsttttj|i|dSN)rModuleNotFoundErrorstrsuper__init__)selfr r  __class__]/home/ubuntu/.local/lib/python3.10/site-packages/lightning_fabric/plugins/environments/xla.pyr"s zXLAEnvironment.__init__cCsdS)NFrrrrrcreates_processes_externally'sz+XLAEnvironment.creates_processes_externallycCtr NotImplementedErrorrrrr main_address,zXLAEnvironment.main_addresscCrr rrrrr main_port2rzXLAEnvironment.main_portcCstSr )r is_availablerrrrdetect8szXLAEnvironment.detect)maxsizecCddlmm}|S)znThe number of processes across all devices and hosts. The output is cached for performance. rN)torch_xla.core.xla_modelcore xla_modelxrt_world_sizerxmrrr world_size=zXLAEnvironment.world_sizesizecCtddS)NzYXLAEnvironment.set_world_size was called, but setting world size is not allowed. Ignored.logdebug)rr+rrrset_world_sizeIzXLAEnvironment.set_world_sizecCr")zThe rank (index) of the currently running process across all host and devices. The output is cached for performance. rN)r#r$r% get_ordinalr'rrr global_rankMr*zXLAEnvironment.global_rankrankcCr,)Nz[XLAEnvironment.set_global_rank was called, but setting global rank is not allowed. Ignored.r-)rr4rrrset_global_rankYr1zXLAEnvironment.set_global_rankcCr")zThe rank (index) of the currently running process inside of the current host. The output is cached for performance. rN)r#r$r%get_local_ordinalr'rrr local_rank]r*zXLAEnvironment.local_rankcCsDtr ddlm}|Sddlmm}ddlm}||j t dS)zxThe rank (index) of the host on which the current process runs. The output is cached for performance. r)runtimeN) getenv_as) r torch_xlar8 host_indextorch_xla.core.xla_env_varsr$ xla_env_varstorch_xla.utils.utilsr9 HOST_ORDINALint)rxrxenvr9rrr node_rankis   zXLAEnvironment.node_rank)__name__ __module__ __qualname____doc__rrpropertyrboolrrrr@r staticmethodr functools lru_cacher)r0r3r5r7rC __classcell__rrrrrs>      r)rKloggingtypingrtyping_extensionsr!lightning_fabric.accelerators.xlarrr9lightning_fabric.plugins.environments.cluster_environmentr getLoggerrDr.rrrrrs