o $ia @sVddlZddlZddlmZddlmZddlm Z m Z m Z m Z eGdddZ dS)N)ModelV2) OldAPIStack)ListModelConfigDict TensorTypeUnionc @seZdZdZdeedefddZdefddZdefd d Z defd d Z d edefddZ dddefddZ defddZ dddefddZdefddZeedejdedeeejffddZdS)ActionDistributionzThe policy action distribution of an agent. Attributes: inputs: input vector to compute samples from. model (ModelV2): reference to model producing the inputs. inputsmodelcCs||_||_dS)aInitializes an ActionDist object. Args: inputs: input vector to compute samples from. model (ModelV2): reference to model producing the inputs. This is mainly useful if you want to use model variables to compute action outputs (i.e., for autoregressive action distributions, see examples/autoregressive_action_dist.py). N)r r )selfr r r Y/home/ubuntu/veenaModal/venv/lib/python3.10/site-packages/ray/rllib/models/action_dist.py__init__s zActionDistribution.__init__returncCt)z+Draw a sample from the action distribution.NotImplementedErrorr r r r samplezActionDistribution.samplecCr)z Get the deterministic "sampling" output from the distribution. This is usually the max likelihood output, i.e. mean for Normal, argmax for Categorical, etc.. rrr r r deterministic_sample#sz'ActionDistribution.deterministic_samplecCr)z7Returns the log probability of the last sampled action.rrr r r sampled_action_logp+rz&ActionDistribution.sampled_action_logpxcCr)z.The log-likelihood of the action distribution.r)r rr r r logp/rzActionDistribution.logpothercCr)z3The KL-divergence between two action distributions.rr rr r r kl3rzActionDistribution.klcCr)z'The entropy of the action distribution.rrr r r entropy7rzActionDistribution.entropycCs ||S)zThe KL-divergence between two action distributions. This differs from kl() in that it can return an array for MultiDiscrete. TODO(ekl) consider removing this. )rrr r r multi_kl;s zActionDistribution.multi_klcCs|S)zThe entropy of the action distribution. This differs from entropy() in that it can return an array for MultiDiscrete. TODO(ekl) consider removing this. )rrr r r multi_entropyCsz ActionDistribution.multi_entropy action_space model_configcCr)a|Returns the required shape of an input parameter tensor for a particular action space and an optional dict of distribution-specific options. Args: action_space (gym.Space): The action space this distribution will be used for, whose shape attributes will be used to determine the required shape of the input parameter tensor. model_config: Model's config dict (as defined in catalog.py) Returns: model_output_shape (int or np.ndarray of ints): size of the required input vector (minus leading batch dimension). r)r r!r r r required_model_output_shapeKsz.ActionDistribution.required_model_output_shapeN)__name__ __module__ __qualname____doc__rrrrrrrrrrrr staticmethodrgymSpacerrintnpndarrayr"r r r r r s(  r) gymnasiumr(numpyr+ray.rllib.models.modelv2rray.rllib.utils.annotationsrray.rllib.utils.typingrrrrrr r r r s