o `۷i3@sddlZddlZddlZddlmZmZmZddlZddl Z ddl Z ddl m Z ddl mZmZddlmZmZmZerBddlmZeeZe d>deded eefd d Zed?d ddefddZed@defddZed@defddZGdddZ GdddZ!GdddZ"eddZ#ed@defd d!Z$Gd"d#d#Z%Gd$d%d%Z&Gd&d'd'Z'ed@defd(d)Z(d*d+Z)e ,  -    dAd.eded/ed0ed1ed eed2eed3eed4efd5d6Z*ee d7d8d9d:dBd;eedefd/s z#convert_to_tensor..z framework=z( not supported in `convert_to_tensor()`!)ray.rllib.utils.torch_utilsr try_import_tftree map_structureNotImplementedError)r r rr_rrrrs    rconfigr num_gpus_requestedcCs|jdkrHt\}}|dkrCddlm}|}t|dkr,tjjtjjj kr,|dS|j |j ks=Jd|j d| |j S| dStd|jd ) aReturns a single device (CPU or some GPU) depending on a config. Args: config: An AlgorithmConfig to extract information from about the device to use. num_gpus_requested: The number of GPUs actually requested. This may be the value of `config.num_gpus_per_env_runner` when for example calling this function from an EnvRunner. Returns: A single device (or name) given `config` and `num_gpus_requested`. rr) get_devicesr zlocal_gpu_idx z+ is not a valid GPU ID or is not available.cpuz`framework_str` z not supported!) framework_strtry_import_torchray.air._internal.torch_utilsr#lenray_privateworker_mode WORKER_MODE local_gpu_idxcuda device_countrr)r!r"rrr#devicesrrr get_device5s       r2FerrorcCsTdtjvr tddSz ddl}ddl}W||fSty)|r&tdYdSw)a*Tries importing JAX and FLAX and returns both modules (or Nones). Args: error: Whether to raise an error if JAX/FLAX cannot be imported. Returns: Tuple containing the jax- and the flax modules. Raises: ImportError: If error=True and JAX is not installed. RLLIB_TEST_NO_JAX_IMPORTz$Not importing JAX for test purposes.)NNrNzCould not import JAX! RLlib requires you to install at least one deep-learning framework: `pip install [torch|tensorflow|jax]`.)osenvironloggerwarningflaxjax ImportError)r3r9r:rrrtry_import_jaxis   r<cCst}dtjvrtdd|dfSdtjvrdtjd<d}dtjvr+tjd}d}nzd dl}WntyE|r>td d|dfYSwz |j j }|j |j j |s]|||j |j jWn tyq|}Ynwt|d szd }n d |jddvrdnd }|||fS)aTries importing tf and returns the module (or None). Args: error: Whether to raise an error if tf cannot be imported. Returns: Tuple containing 1) tf1.x module (either from tf2.x.compat.v1 OR as tf1.x). 2) tf module (resulting from `import tensorflow`). Either tf1.x or 2.x. 3) The actually installed tf version as int: 1 or 2. Raises: ImportError: If error=True and tf is not installed. RLLIB_TEST_NO_TF_IMPORTz*Not importing TensorFlow for test purposesNTF_CPP_MIN_LOG_LEVEL3F tensorflowTrzCould not import TensorFlow! RLlib requires you to install at least one deep-learning framework: `pip install [torch|tensorflow|jax]`. __version__r z2.)_TFStubr5r6r7r8sysmodulesr@r;compatv1logging set_verbosityERRORdisable_v2_behaviorenable_resource_variablesWARNAttributeErrorhasattrrA)r3tf_stub was_imported tf_module tf1_moduleversionrrrrsB             rc@seZdZdddZddZdS)rCreturnNcCs t|_dSr) _KerasStubkerasselfrrr__init__s z_TFStub.__init__cCsdS)NFrrXrrr__bool__sz_TFStub.__bool__rUN)__name__ __module__ __qualname__rZr[rrrrrCs  rCc@seZdZdddZdS)rVrUNcCs t|_dSr)_FakeTfClassStubModelrXrrrrZs z_KerasStub.__init__r\r]r^r_rZrrrrrVsrVc@eZdZddZdS)r`cOtd)Nz:Could not import `tensorflow`. Try pip install tensorflow.r;rYakwrrrrZz_FakeTfClassStub.__init__Nrbrrrrr` r`csfdd}|S)zqConditional decorator for @tf.function. Use @tf_function(tf) instead to avoid errors if tf is not installed.csdusr |S|Sr)executing_eagerlyfunction)funcrRrr decorators ztf_function..decoratorr)rRrorrnr tf_functions rpc CsVdtjvr tddSzddl}|WSty*}z |r|WYd}~dSd}~ww)zTries importing tfp and returns the module (or None). Args: error: Whether to raise an error if tfp cannot be imported. Returns: The tfp module. Raises: ImportError: If error=True and tfp is not installed. r=z7Not importing TensorFlow Probability for test purposes.Nr)r5r6r7r8tensorflow_probabilityr;)r3tfperrrtry_import_tfps rtc@rc)_NNStubcOsd|_t|_t|_dSr) functional_FakeTorchClassStubModule _ParallelStubparallelrfrrrrZs z_NNStub.__init__Nrbrrrrrurjruc@rc)rwcOrd)Nz0Could not import `torch`. Try pip install torch.rerfrrrrZriz_FakeTorchClassStub.__init__Nrbrrrrrwrjrwc@rc)rycOst|_t|_dSr)rw DataParallelDistributedDataParallelrfrrrrZs z_ParallelStub.__init__NrbrrrrryrjrycCs^dtjvr tdtSzddl}ddlm}||fWSty.|r)tdtYSw)a&Tries importing torch and returns the module (or None). Args: error: Whether to raise an error if torch cannot be imported. Returns: Tuple consisting of the torch- AND torch.nn modules. Raises: ImportError: If error=True and PyTorch is not installed. RLLIB_TEST_NO_TORCH_IMPORTz(Not importing PyTorch for test purposes.rNzCould not import PyTorch! RLlib requires you to install at least one deep-learning framework: `pip install [torch|tensorflow|jax]`.) r5r6r7r8 _torch_stubsrtorch.nnnnr;)r3rrrrrr&"s     r&cCst}d|fSr)ru)rrrrr~Bsr~runnamed-variablevalue trainabletf_name torch_tensorshapedtyperUc Cs|dvr9ddl}|pt|dt|tr|jn t|tr|jnd}|jjj |f|||d|dur3iSd|iS|dkr|durt \} } t|t j sQt |}| |} || jt jfvrc| } n|| jt jfvrp| } n || jt jfvr|| } |r| |} || _| S|S) aCreates a tf variable, a torch tensor, or a python primitive. Args: value: The initial value to use. In the non-tf case, this will be returned as is. In the tf case, this could be a tf-Initializer object. framework: One of "tf", "torch", or None. trainable: Whether the generated variable should be trainable (tf)/require_grad (torch) or not (default: False). tf_name: For framework="tf": An optional name for the tf.Variable. torch_tensor: For framework="torch": Whether to actually create a torch.tensor, or just a python value (default). device: An optional torch device to use for the created torch tensor. shape: An optional shape to use iff `value` does not have any (e.g. if it's an initializer w/o explicit value). dtype: An optional dtype to use iff `value` does not have any (e.g. if it's an initializer w/o explicit value). This should always be a numpy dtype (e.g. np.float32, np.int64). Returns: A framework-specific variable (tf.Variable, torch.tensor, or python primitive). )rrrNr) initializerrrrrT)r@getattr isinstancefloatfloat32intint32rFrG get_variabler&npndarrayarray from_numpyfloat64doubleto requires_grad) rr rrrrrrrrrvar_rrrrGsL$         rz+rllib/utils/framework.py::get_activation_fnz(rllib/models/utils.py::get_activation_fnT)oldnewr3namecCsdSrr)rr rrrget_activation_fnsrr)r )F)rFrFNNN)Nr),rHr5rDtypingrrrnumpyrrr)ray._common.deprecationrray.rllib.utils.annotationsrrray.rllib.utils.typingrr r %ray.rllib.algorithms.algorithm_configr getLoggerr]r7strrrr2boolr<rrCrVr`rprtrurwryr&r~rrrrrrs    3B    J