o ci@@srddlmZddlZddlZddlZddlmZm Z ddl m Z m Z ddl mZddlmZddlmZddlmZd Zd Zd ZeeZe Gd d d Ze GdddeZe GdddeZe GdddeZe GdddeZe GdddeZ e GdddeZ!e GdddeZ"e GdddeZ#e d%dej$d e%fd!d"Z&dej$d e e'fd#d$Z(dS)&) OrderedDictN)AnyList) OldAPIStackoverride)Repeated) TensorType)resize)convert_element_to_space_type)dc@seZdZdZddejdefddZdejdedee fdd Z d e de j fd d Zd e d e j de ddfddZd eddfddZede fddZedejfddZdS) PreprocessorzDefines an abstract observation preprocessor function. Attributes: shape (List[int]): Shape of the preprocessed output. N obs_spaceoptionscCsht|||_|sddlm}||_n||_|||j|_tt |j|_ d|_ |j |_dS)Nr)MODEL_DEFAULTS)_legacy_patch_shapes _obs_spaceray.rllib.models.catalogrcopy_options _init_shapeshapeintnpprod_size_isample_obs_for_type_matching)selfrrrr$R/home/ubuntu/.local/lib/python3.10/site-packages/ray/rllib/models/preprocessors.py__init__s  zPreprocessor.__init__returncCt)z&Returns the shape after preprocessing.NotImplementedErrorr#rrr$r$r%r-zPreprocessor._init_shape observationcCr()z%Returns the preprocessed observation.r)r#r-r$r$r% transform1r,zPreprocessor.transformarrayoffsetcCs||||||j<dS)z7Alternative to transform for more efficient flattening.N)r/rr#r-r0r1r$r$r%write5zPreprocessor.writec Cs|jtdkrZt|turt|jtjjrt | t j }|j |s*t||j}z|j |sFtd|t|jtjjr@|jnd|jWntyY}ztd||d}~ww|jd7_dS)z*Checks the shape of the given observation.rz3Observation ({} dtype={}) outside given space ({})!Nz_Observation for a Box/MultiBinary/MultiDiscrete space should be an np.array, not a Python list.)r OBS_VALIDATION_INTERVALtypelist isinstancergymspacesBoxrr0astypefloat32containsr r" ValueErrorformatdtypeAttributeError)r#r-er$r$r% check_shape9s>    zPreprocessor.check_shapecC|jSN)rr#r$r$r%sizeXszPreprocessor.sizecCs>tjjdd|jtjd}tttt t t f}t ||r|j |_|S)Ng?rB)r:r;r<rrr>DictFlatteningPreprocessorOneHotPreprocessorRepeatedValuesPreprocessorTupleFlatteningPreprocessorAtariRamPreprocessorGenericPixelPreprocessorr9roriginal_space)r#rclassesr$r$r%observation_space\s zPreprocessor.observation_spacerG)__name__ __module__ __qualname____doc__r:Spacedictr&rrrrrndarrayr/r3rrEpropertyrIrTr$r$r$r%rsrc@sPeZdZdZeedejdede e fddZ eede de jfdd Zd S) rQzGeneric image preprocessor. Note: for Atari games, use config {"preprocessor_pref": "deepmind"} instead for deepmind-style Atari preprocessing. rrr'cCsN|d|_|d|_|d|_|jr|j|jdf}|S|j|jdf}|S)N grayscale zero_meandimr5r )get _grayscale _zero_mean_dim)r#rrrr$r$r%rvs   z$GenericPixelPreprocessor._init_shaper-cCs|||ddddddf}|jdkrt|ddd}t||j|jd}|jr@|d}|tj}t||j|jdg}|j rK|dd}|S|d 9}|S) z?Downsamples images from (210, 160, 3) by the configured factor.iNT)heightwidthr5rgp?) rErcr rameanr=rr>reshaperb)r#r-scaledr$r$r%r/s     z"GenericPixelPreprocessor.transformN)rUrVrWrXrrr:rYrZrrrrrr[r/r$r$r$r%rQns  rQc@sLeZdZeedejdedee fddZ eede de j fddZd S) rPrrr'cCsdS)Nrr$r+r$r$r%rr,z AtariRamPreprocessor._init_shaper-cCs|||dddS)Nr>r)rEr=r.r$r$r%r/s zAtariRamPreprocessor.transformN)rUrVrWrrr:rYrZrrrrrr[r/r$r$r$r%rPs rPc @teZdZdZeedejdede e fddZ eede de jfdd Zeede d e jd e dd fd dZd S)rMarOne-hot preprocessor for Discrete and MultiDiscrete spaces. .. testcode:: :skipif: True self.transform(Discrete(3).sample()) .. testoutput:: np.array([0.0, 1.0, 0.0]) .. testcode:: :skipif: True self.transform(MultiDiscrete([2, 3]).sample()) .. testoutput:: np.array([0.0, 1.0, 0.0, 0.0, 1.0]) rrr'cCs(t|tjjr |jjfSt|jjfSrG) r9r:r;Discreternrsumnvecr+r$r$r%rs zOneHotPreprocessor._init_shaper-cCs$||tjj|j|tjSrG) rEr:r;utilsflattenrr=rr>r.r$r$r%r/s zOneHotPreprocessor.transformr0r1NcCs||||||j<dSrG)r/rIr2r$r$r%r3r4zOneHotPreprocessor.writerUrVrWrXrrr:rYrZrrrrrr[r/r3r$r$r$r%rMs"rMc @seZdZeedejdedee fddZ eede de j fddZeede d e j d e dd fd d ZeeedejfddZd S)NoPreprocessorrrr'cC|jjSrGrrr+r$r$r%rzNoPreprocessor._init_shaper-cCs|||SrG)rEr.r$r$r%r/s zNoPreprocessor.transformr0r1NcC$tj|dd||||j<dSNF)rrr0ravelrr2r$r$r%r3$zNoPreprocessor.writecCrFrG)rrHr$r$r%rTsz NoPreprocessor.observation_space)rUrVrWrrr:rYrZrrrrrr[r/r3r\rTr$r$r$r%rtsrtc @seZdZdZeedejdede e fddZ eede de jfdd Zeede d e jd e dd fd dZeeedejfddZd S)MultiBinaryPreprocessoraPreprocessor that turns a MultiBinary space into a Box. Note: Before RLModules were introduced, RLlib's ModelCatalogV2 would produce ComplexInputNetworks that treat MultiBinary spaces as Boxes. This preprocessor is needed to get rid of the ComplexInputNetworks and use RLModules instead because RLModules lack the logic to handle MultiBinary or other non-Box spaces. rrr'cCrurGrvr+r$r$r%rrwz#MultiBinaryPreprocessor._init_shaper-cCs|||tjSrG)rEr=rr>r.r$r$r%r/s  z!MultiBinaryPreprocessor.transformr0r1NcCrxryrzr2r$r$r%r3r|zMultiBinaryPreprocessor.writecCs$tjjdd|jtjd}|j|_|S)NgrJrK)r:r;r<rrr>rrR)r#rr$r$r%rTsz)MultiBinaryPreprocessor.observation_space)rUrVrWrXrrr:rYrZrrrrrr[r/r3r\rTr$r$r$r%r}sr}c @rl)rOzPreprocesses each tuple element, then flattens it all into a vector. RLlib models will unpack the flattened output before _build_layers_v2(). rrr'cCst|jtjjs Jd}g|_tt|jjD]6}|jj|}t d |t |}|dur;|||j }||j 7}n d}|tt|j7}|j|q|fSNrz Creating sub-preprocessor for {})r9rr:r;Tuple preprocessorsrangelenloggerdebugrAget_preprocessorrrIrrrrappend)r#rrrIispacepreprocessor_class preprocessorr$r$r%r s   z'TupleFlatteningPreprocessor._init_shaper-cC.||tj|jtjd}|||d|SNrKrrErzerosrr>r3r#r-r0r$r$r%r/ z%TupleFlatteningPreprocessor.transformr0r1NcCsLt|t|jks J|t||jD]\}}||||||j7}qdSrG)rrzipr3rIr#r-r0r1opr$r$r%r3"s  z!TupleFlatteningPreprocessor.writersr$r$r$r%rOs"rOc @rl)rLzPreprocesses each dict value, then flattens it all into a vector. RLlib models will unpack the flattened output before _build_layers_v2(). rrr'cCst|jtjjs Jd}g|_|jjD]0}td |t |}|dur3|||j }||j 7}n d}|t t|j7}|j|q|fSr~)r9rr:r;DictrvaluesrrrArrrIrrrrr)r#rrrIrrrr$r$r%r1s  z&DictFlatteningPreprocessor._init_shaper-cCrrrrr$r$r%r/Brz$DictFlatteningPreprocessor.transformr0r1NcCsxt|ts tt|}t|t|jks!Jt|t|jft||jD]\}}||||||j 7}q)dSrG) r9rsorteditemsrrrrr3rIrr$r$r%r3Is  z DictFlatteningPreprocessor.writersr$r$r$r%rL*s"rLc @rl)rNz0Pads and batches the variable-length list value.rrr'cCs@t|jtsJ|j}t|||j|_d|jj|j}|fS)Nr5) r9rr child_spacerrchild_preprocessorrImax_len)r#rrrrIr$r$r%rZsz&RepeatedValuesPreprocessor._init_shaper-cCsBt|j}t|tr|D]}|j|q n |||d|S)Nr)rrrr9r8rrEr3)r#r-r0elemr$r$r%r/es  z$RepeatedValuesPreprocessor.transformr0r1NcCst|ttjfstd||t||jjkr"td||jjt|||<t |D]\}}|d||j j }|j |||q,dS)Nz&Input for {} must be list type, got {}z$Input {} exceeds max len of space {}r5) r9r8rr[r@rArrr enumeraterrIr3)r#r-r0r1rroffset_ir$r$r%r3ps  z RepeatedValuesPreprocessor.writersr$r$r$r%rNVs  "rNFrr'cCst||j}t|tjjtjjfrt}|S|tkr&t d tt }|S|t kr6t d tt}|St|tjjrAt}|St|tjjrLt}|St|trUt}|St|tjjrb|rbt}|St}|S)z>Returns an appropriate preprocessor class for the given space.aDefaulting to RLlib's GenericPixelPreprocessor because input space has the atari-typical shape {}. Turn this behaviour off by setting `preprocessor_pref=None` or `preprocessor_pref='deepmind'` or disabling the preprocessing API altogether with `_disable_preprocessor_api=True`.aDefaulting to RLlib's AtariRamPreprocessor because input space has the atari-typical shape {}. Turn this behaviour off by setting `preprocessor_pref=None` or `preprocessor_pref='deepmind' or disabling the preprocessing API altogether with `_disable_preprocessor_api=True`.`.)rrr9r:r;rm MultiDiscreterMATARI_OBS_SHAPErrrArQATARI_RAM_OBS_SHAPErPrrOrrLrrN MultiBinaryr}rt)rinclude_multi_binary obs_shaperr$r$r%rsB    rcCsdt|ds/t|tjjrd|_|jSt|tjjr/g}|jD] }t|}||qt ||_|jS)zAssigns shapes to spaces that don't have shapes. This is only needed for older gym versions that don't set shapes properly for Tuple and Discrete spaces. rr$) hasattrr9r:r;rmrrrrtuple)rshapessrr$r$r%rs    r)F)) collectionsrloggingnumpyr gymnasiumr:typingrrray.rllib.utils.annotationsrrray.rllib.utils.spaces.repeatedrray.rllib.utils.typingrray.rllib.utils.imagesr "ray.rllib.utils.spaces.space_utilsr rrr6 getLoggerrUrrrQrPrMrtr}rOrLrNrYr7rrrr$r$r$r%sF      V* ''+,*