o $i-3@sddlZddlmZddlmZddlZddlmZddlm Z ddl m Z ddl m Z ddlmZdd lmZmZmZdd lmZdd lmZdd lmZdd lmZeeZeddGddde ZdS)N)Path)List)AlgorithmConfig)Columns) EnvRunner)SingleAgentEnvRunner)SingleAgentEpisode)OverrideToImplementCustomLogic5OverrideToImplementCustomLogic_CallToSuperRecommendedoverride)pack_if_needed) EpisodeType) PublicAPI)log_oncealpha) stabilitycseZdZdZeeedeffdd Zeee ddddddde d e d e d e d e d e e f fddZeee dddZe de ed dfddZZS)OfflineSingleAgentEnvRunnerz7The environment runner to record the single agent case.configc stjdd|i|d|_tjj}|jjj |j d|j_|j j |_ |j j |_ |j j|_|j j|_d|_|j j|_|j j|_d|jd|_|jdkr`ddl}|jdi|j|_n8|jdkrvdd lm}|jdi|j|_n"|jd krddl }|j!di|j|_n|jdurt"d |jd |j #d |ji|j j$|_$|j j%|_%|j j&|_&|j&rd|_'nd|_'|j j(|_)d|_*g|_+dS)NrF)cpuzrun-gcsrs3)fsabszUnknown filesystem: z9. Filesystems can be 'gcs' for GCS, 's3' for S3, or 'abs' filesystemT),super__init__episodes_to_numpyraydata DataContext get_currentexecution_optionsresource_limitscopynum_cpus_per_env_runnerroutput_write_methodoutput_write_method_kwargsoutput_filesystemroutput_filesystem_kwargsfilesystem_kwargsfilesystem_objectoutput output_pathenvlower subdir_path worker_indexzfill worker_pathgcsfs GCSFileSystempyarrowr S3FileSystemadlfsAzureBlobFileSystem ValueErrorupdateoutput_write_episodesoutput_compress_columnsoutput_max_rows_per_filewrite_data_this_iteroutput_write_remaining_datawrite_remaining_data_sample_counter_samples)selfrkwargs data_contextr5rr9 __class__ra/home/ubuntu/veenaModal/venv/lib/python3.10/site-packages/ray/rllib/offline/offline_env_runner.pyrsR                 z$OfflineSingleAgentEnvRunner.__init__NF num_timesteps num_episodesexplorerandom_actions force_resetrLrMrNrOrPreturnc stj|||||d}|jd7_|jr@ddlddltdr&tdt dd|Ds1J|j fd d |Dn| ||j rSt|j |j krSd |_|jr|j rd |_t|j |j kr|j d|j }|j |j d|_ tj|}t|j |j ksdntj|j }z0t|j|j|jd |jd} t||j| fi|jtd| dWnty} z t| WYd} ~ nd} ~ ww|jj dt|j d|S)z2Samples from environments and writes data to disk.rKrNmsgpackzPacking episodes with `msgpack` and encode array with `msgpack_numpy` for serialization. This is needed for recording episodes.css|]}|jduVqdS)FN)is_numpy.0epsrrrJ sz5OfflineSingleAgentEnvRunner.sample..cs g|] }j|jdqS))default)packb get_stateencoderUmnprSrrJ s z6OfflineSingleAgentEnvRunner.sample..TF-rzWrote samples to storage at .recording_buffer_size)keyvalue)!rsamplerCr=rS msgpack_numpyrloggerinfoallrDextend_map_episodes_to_datar?lenr@rr from_itemsrr.joinpathr1r4r3getattrr'as_posixr( Exceptionerrormetrics log_value) rErLrMrNrOrPsamplessamples_to_write samples_dspatherHr]rJre{sj    z"OfflineSingleAgentEnvRunner.samplec Cs|jr\|jr\tj|j}|jd7_z0t|j|j |j d|j d}t ||j |fi|jtd|dWnty[}z t|WYd}~nd}~wwtdt|jdS)zWrites the reamining samples to disk Note, if the user defined `max_rows_per_file` the number of rows for the remaining samples could be less than the defined maximum row number by the user. rRr`rz"Wrote final samples to storage at zY. Note Note, final samples could be smaller in size than `max_rows_per_file`, if defined.NzExperience buffer length: )rDrBrr rmrCrr.rnr1r4r3ror'rpr(rgrhrqrrdebugrl)rErwrxryrrrJstops*   z OfflineSingleAgentEnvRunner.stoprucs"|D]ttD]tjjtjjtjjtj tj j vr't  n tj tj j vr:t ntjtjtj j vrUt  dn dtjtdkrgdnjtjtdkrudnji fddjD}j|q qdS)zConverts list of episodes to list of single dict experiences. Note, this method also appends all sampled experiences to the buffer. Args: samples: List of episodes to be converted. rRFcs4i|]}||jvrt|n|qSr)r>r get_extra_model_outputs)rVkirerErrJ (s  zEOfflineSingleAgentEnvRunner._map_episodes_to_data..N)rangerlrEPS_IDid_AGENT_IDagent_id MODULE_ID module_idOBSr>r get_observationsACTIONS get_actionsREWARDS get_rewardsNEXT_OBS TERMINATEDS is_terminated TRUNCATEDS is_truncatedextra_model_outputskeysrDappend)rEru sample_datarr~rJrksD       z1OfflineSingleAgentEnvRunner._map_episodes_to_data)rQN)__name__ __module__ __qualname____doc__r rr rrr intboolrrrerr{r rk __classcell__rrrHrJrs<Z_% r) loggingpathlibrtypingrr%ray.rllib.algorithms.algorithm_configrray.rllib.core.columnsrray.rllib.env.env_runnerr%ray.rllib.env.single_agent_env_runnerr"ray.rllib.env.single_agent_episoderray.rllib.utils.annotationsr r r ray.rllib.utils.compressionr ray.rllib.utils.typingr ray.util.annotationsrray.util.debugrLogger__file__rgrrrrrJs"