o ci@sddlmZddlmZddlmZmZeGdddZdedefdd Z ded edefd d Z ded ededefddZ dS))List) OldAPIStack) TensorTypeTensorStructTypec@sbeZdZdZdedeedefddZdeeefdd Zdeefd d Z d d Z ddZ dS)RepeatedValuesaRepresents a variable-length list of items from spaces.Repeated. RepeatedValues are created when you use spaces.Repeated, and are accessible as part of input_dict["obs"] in ModelV2 forward functions. Example: Suppose the gym space definition was: Repeated(Repeated(Box(K), N), M) Then in the model forward function, input_dict["obs"] is of type: RepeatedValues(RepeatedValues()) The tensor is accessible via: input_dict["obs"].values.values And the actual data lengths via: # outer repetition, shape [B], range [0, M] input_dict["obs"].lengths -and- # inner repetition, shape [B, M], range [0, N] input_dict["obs"].values.lengths Attributes: values: The padded data tensor of shape [B, max_len, ..., sz], where B is the batch dimension, max_len is the max length of this list, followed by any number of sub list max lens, followed by the actual data size. lengths (List[int]): Tensor of shape [B, ...] that represents the number of valid items in each list. When the list is nested within other lists, there will be extra dimensions for the parent list max lens. max_len: The max number of items allowed in each list. TODO(ekl): support conversion to tf.RaggedTensor. valueslengthsmax_lencCs||_||_||_d|_dSN)rrr _unbatched_repr)selfrrr r T/home/ubuntu/.local/lib/python3.10/site-packages/ray/rllib/models/repeated_values.py__init__-s zRepeatedValues.__init__returncCs|jdurXt|j}|durtdt|}|}g}t|D]4}t|j|dr4t|j| }n t|j| }g}t|D] }| t |||qC| |q ||_|jS)aUnbatch both the repeat and batch dimensions into Python lists. This is only supported in PyTorch / TF eager mode. This lets you view the data unbatched in its original form, but is not efficient for processing. .. testcode:: :skipif: True batch = RepeatedValues() items = batch.unbatch_all() print(len(items) == B) .. testoutput:: True .. testcode:: :skipif: True print(max(len(x) for x in items) <= N) .. testoutput:: True .. testcode:: :skipif: True print(items) .. testoutput:: [[, ..., ], ... [, ], ... [], ... [, ..., ]] NzjCannot call unbatch_all() when batch_dim is unknown. This is probably because you are using TF graph mode.item) r _get_batch_dim_helperr ValueErrorintunbatch_repeat_dimrangehasattrrrnumpyappend_batch_index_helper)r Bslicesresulti dynamic_len dynamic_slicejr r r unbatch_all3s& ,    zRepeatedValues.unbatch_allcCst|j|jS)akUnbatches the repeat dimension (the one `max_len` in size). This removes the repeat dimension. The result will be a Python list of with length `self.max_len`. Note that the data is still padded. .. testcode:: :skipif: True batch = RepeatedValues() items = batch.unbatch() len(items) == batch.max_len .. testoutput:: True .. testcode:: :skipif: True print(items) .. testoutput:: [, ..., ] )_unbatch_helperrr r r r rrwsz!RepeatedValues.unbatch_repeat_dimcCsdt|jt|j|jS)Nz0RepeatedValues(value={}, lengths={}, max_len={}))formatreprrrr r$r r r__repr__szRepeatedValues.__repr__cCst|Sr )r&r$r r r__str__szRepeatedValues.__str__N) __name__ __module__ __qualname____doc__rrrrr"rr'r(r r r rrs$D rvrcCsnt|tr|D]}t|SdSt|trt|dSt|tr(t|jS|jd}t|dr5|j}|S)z5Tries to find the batch dimension size of v, or None.rvalueN) isinstancedictrrtuplershaperr.)r-urr r rrs         rr csttrfddDSttr tfddDSttr6tj}fddt|DSfddtDS)z3Recursively unpacks the repeat dimension (max_len).csi|] \}}|t|qSr r#.0kr3r r r sz#_unbatch_helper..c3s|]}t|VqdSr r4r6r3r8r r sz"_unbatch_helper..cs.g|]\}}t|jdd|dfjqSN.)rrr )r6rr3r-r r sz#_unbatch_helper..csg|] }dd|dfqSr<r )r6rr=r rr>) r/r0itemsr1rr#r enumerater)r-r unbatchedr )r r-rr#s     r#rr!cst|trfdd|DSt|tr"tfdd|DSt|tr/t|St|tr<|}|S|dfS)z;Selects the item at the ith batch index and jth repetition.csi|] \}}|t|qSr rr5rr!r rr9r?z'_batch_index_helper..c3s|] }t|VqdSr rCr:rDr rr;sz&_batch_index_helper...)r/r0r@r1listrrr")r-rr!rBr rDrrs     rN) typingrray.rllib.utils.annotationsrray.rllib.utils.typingrrrrrr#rr r r rs