o ciz<@sddlmZmZmZddlZddlZddlZddl Z ddl m Z m Z mZddl mZddlmZddlmZddlmZmZmZeeZeGdd d eZdS) )Box MultiDiscreteTupleN)CallableOptionalr) MultiAgentEnv) PolicySpec) PublicAPI)MultiAgentDictPolicyIDAgentIDc seZdZdZdZdZdZ      dd ed ee d e d e d e de f fdd Z de de e e e e e ffddZdddde e e ffddZddZedede eeegefffddZZS) Unity3DEnvapA MultiAgentEnv representing a single Unity3D game instance. For an example on how to use this Env with a running Unity3D editor or with a compiled game, see: `rllib/examples/unity3d_env_local.py` For an example on how to use it inside a Unity game client, which connects to an RLlib Policy server, see: `rllib/examples/envs/external_envs/unity3d_[client|server].py` Supports all Unity3D (MLAgents) examples, multi- or single-agent and gets converted automatically into an ExternalMultiAgentEnv, when used inside an RLlib PolicyClient for cloud/distributed training of Unity games. iirNF, file_nameportseed no_graphics timeout_waitepisode_horizonc st|dur tdddl}ddlm}d} | dur'tt dd|p0|r.|j n|j } |r6t j nd} t j d7_ z||| | |||d|_td | | Wn |jjy`Ynwnq|jjd |_d d |jD|_||_d|_dS) a Initializes a Unity3DEnv object. Args: file_name (Optional[str]): Name of the Unity game binary. If None, will assume a locally running Unity3D editor to be used, instead. port (Optional[int]): Port number to connect to Unity environment. seed: A random seed value to use for the Unity3D game. no_graphics: Whether to run the Unity3D simulator in no-graphics mode. Default: False. timeout_wait: Time (in seconds) to wait for connection from the Unity3D instance. episode_horizon: A hard horizon to abide to. After at most this many steps (per-agent episode `step()` calls), the Unity3D game is reset and will start again (finishing the multi-agent episode that the game represents). Note: The game itself may contain its own episode length limits, which are always obeyed (on top of this value here). NzNo game binary provided, will use a running Unity editor instead. Make sure you are pressing the Play (|>) button in your editor to start.r)UnityEnvironmentT )r worker_id base_portrrrz$Created UnityEnvironment for port {}.cSsg|]}t|qS)int).0srrV/home/ubuntu/.local/lib/python3.10/site-packages/ray/rllib/env/wrappers/unity3d_env.py osz'Unity3DEnv.__init__..)super__init__print mlagents_envsmlagents_envs.environmentrtimesleeprandomrandint_BASE_PORT_ENVIRONMENT_BASE_PORT_EDITORr _WORKER_ID unity_envformat exceptionUnityWorkerInUseException API_VERSIONsplit api_versionrepisode_timesteps) selfrrrrrrr%rport_ worker_id_ __class__rr r#'sF   zUnity3DEnv.__init__ action_dictreturncCsddlm}g}|jjD]}|jddks#|jddkrg|jddkrgg}|j|djD]}|d|}|||||q.|rf|dj t j krW|t |d}n|t |d}|j ||q |j|djD]}|d|}|||j||||qrq |j|\} } } } } |jd7_|j|jkr| | | tdd ifid d |D| fS| | | | | fS) a Performs one multi-agent step through the game. Args: action_dict: Multi-agent action dict with: keys=agent identifier consisting of [MLagents behavior name, e.g. "Goalie?team=1"] + "_" + [Agent index, a unique MLAgent-assigned index per single agent] Returns: tuple: - obs: Multi-agent observation dict. Only those observations for which to get new actions are returned. - rewards: Rewards dict matching `obs`. - dones: Done dict with only an __all__ multi-agent entry in it. __all__=True, if episode is done for all agents. - infos: An (empty) info dict. r) ActionTupler_{}) continuous)discrete__all__TcSsi|]}|dqS)Tr)ragent_idrrr sz#Unity3DEnv.step..)mlagents_envs.base_envr=r.behavior_specsr4 get_stepsrCr/appenddtypenpfloat32array set_actionsagent_id_to_indexkeysset_action_for_agentstep_get_step_resultsr5rdict)r6r;r= all_agents behavior_nameactionsrCkey action_tupleobsrewards terminateds truncatedsinfosrrr rQvsL        zUnity3DEnv.step)roptionscCs*d|_|j|\}}}}}||fS)z?Resets the entire Unity3D scene (a single multi-agent episode).r)r5r.resetrR)r6rr^rY_r]rrr r_s zUnity3DEnv.resetc s,i}i}i}|jjD]}|j|\}}|jD]3\}|d|}tfdd|jD} t| dkr:| dn| } | ||<|j |j ||<q|jD]7\}|d|}||vr}tfdd|jD} t| dkrw| dn| ||<} |j |j ||<qRq ||ddiddi|fS) aCollects those agents' obs/rewards that have to act in next `step`. Returns: Tuple: obs: Multi-agent observation dict. Only those observations for which to get new actions are returned. rewards: Rewards dict matching `obs`. dones: Done dict with only an __all__ multi-agent entry in it. __all__=True, if episode is done for all agents. infos: An (empty) info dict. r?c3|]}|VqdSNrroidxrr z/Unity3DEnv._get_step_results..rrc3rarbrrcrerr rgrhrBF) r.rFrGrNitemsr/tuplerYlenreward group_reward) r6rYrZr]rUdecision_stepsterminal_stepsrCrWosrrer rRs*    zUnity3DEnv._get_step_results game_namecsttdtddttdtddttdtddtttdtddttdtddttdtddttdtddgttdd d tdd d gttdtdd tttdtdd ttdtddgtttdtddttdtddttdtddgttdtddttdtddttdtddtttdtddttdtddgd }tdd dtjdtdd dtjdtgdtdgtgdtgdtgdtgdtdd dtdgtdd dtgdd }dkr#t|d|dd t|d!|d!d d"}d#d$}||fSd%krEt|d&|d&d t|d&|d&d d'}d(d$}||fSt||d i}fd)d$}||fS)*Nz-infinf))-)(ru)8)r>gg?)i)H)i))?))r))Tr~))1) 3DBall 3DBallHardGridFoodCollectorPyramids SoccerPlayerGoalieStrikerSorterTennis VisualHallwayWalker FoodCollector))rI)rrrr)rrr)r)') rrrrrrrrrrrrSoccerStrikersVsGoalier)observation_space action_spacer)rrc[sd|vrdSdS)NrrrrCepisodeworkerkwargsrrr policy_mapping_fn`zAUnity3DEnv.get_policy_configs_for_game..policy_mapping_fn SoccerTwosr) PurplePlayer BluePlayerc[sd|vrdSdS)N1_rrrrrrr rorcsSrbrrrqrr rzs)rfloat TupleSpacerJrKrr)rq obs_spaces action_spacespoliciesrrrr get_policy_configs_for_games   >             z&Unity3DEnv.get_policy_configs_for_game)NNrFrr)__name__ __module__ __qualname____doc__r,r+r-strrrboolr#r rrQr_rR staticmethodrSrr r r __classcell__rrr9r r sTO  K   -r )gymnasium.spacesrrrrloggingnumpyrJr)r'typingrrray.rllib.env.multi_agent_envrray.rllib.policy.policyrray.rllib.utils.annotationsr ray.rllib.utils.typingr r r getLoggerrloggerr rrrr s