o `Û·ia ã@sVddlZddlZddlmZddlmZddlm Z m Z m Z m Z eGdd„dƒƒZ dS)éN)ÚModelV2)Ú OldAPIStack)ÚListÚModelConfigDictÚ TensorTypeÚUnionc @sÌeZdZdZdeedefdd„Zdefdd„Zdefd d „Z defd d „Z d edefdd„Z dddefdd„Z defdd„Z dddefdd„Zdefdd„Zeedejdedeeejffdd„ƒƒZdS)ÚActionDistributionz¹The policy action distribution of an agent. Attributes: inputs: input vector to compute samples from. model (ModelV2): reference to model producing the inputs. ÚinputsÚmodelcCs||_||_dS)a™Initializes an ActionDist object. Args: inputs: input vector to compute samples from. model (ModelV2): reference to model producing the inputs. This is mainly useful if you want to use model variables to compute action outputs (i.e., for autoregressive action distributions, see examples/autoregressive_action_dist.py). N)r r )Úselfr r ©r úR/home/ubuntu/vllm_env/lib/python3.10/site-packages/ray/rllib/models/action_dist.pyÚ__init__s zActionDistribution.__init__ÚreturncCót‚)z+Draw a sample from the action distribution.©ÚNotImplementedError©r r r r ÚsampleózActionDistribution.samplecCr)z¿ Get the deterministic "sampling" output from the distribution. This is usually the max likelihood output, i.e. mean for Normal, argmax for Categorical, etc.. rrr r r Údeterministic_sample#sz'ActionDistribution.deterministic_samplecCr)z7Returns the log probability of the last sampled action.rrr r r Úsampled_action_logp+rz&ActionDistribution.sampled_action_logpÚxcCr)z.The log-likelihood of the action distribution.r)r rr r r Úlogp/rzActionDistribution.logpÚothercCr)z3The KL-divergence between two action distributions.r©r rr r r Úkl3rzActionDistribution.klcCr)z'The entropy of the action distribution.rrr r r Úentropy7rzActionDistribution.entropycCs | |¡S)z¸The KL-divergence between two action distributions. This differs from kl() in that it can return an array for MultiDiscrete. TODO(ekl) consider removing this. )rrr r r Úmulti_kl;s zActionDistribution.multi_klcCs| ¡S)z±The entropy of the action distribution. This differs from entropy() in that it can return an array for MultiDiscrete. TODO(ekl) consider removing this. )rrr r r Ú multi_entropyCsz ActionDistribution.multi_entropyÚ action_spaceÚ model_configcCr)a|Returns the required shape of an input parameter tensor for a particular action space and an optional dict of distribution-specific options. Args: action_space (gym.Space): The action space this distribution will be used for, whose shape attributes will be used to determine the required shape of the input parameter tensor. model_config: Model's config dict (as defined in catalog.py) Returns: model_output_shape (int or np.ndarray of ints): size of the required input vector (minus leading batch dimension). r)r r!r r r Úrequired_model_output_shapeKsz.ActionDistribution.required_model_output_shapeN)Ú__name__Ú __module__Ú __qualname__Ú__doc__rrrrrrrrrrrrÚ staticmethodrÚgymÚSpacerrÚintÚnpÚndarrayr"r r r r r s( ÿÿ þr)Ú gymnasiumr(Únumpyr+Úray.rllib.models.modelv2rÚray.rllib.utils.annotationsrÚray.rllib.utils.typingrrrrrr r r r Ús