o bi$ @sddlZddlZddlZddlmZddlmZmZmZm Z m Z ddl m Z ddl mZmZmZmZmZmZddlmZddlmZddlmZeeZeGd d d eZGd d d ZGd ddZefdddedededej f ddZ!dS)N)ABC)AnyDictListOptionalType) import_attr) RAY_RUNTIME_ENV_CLASS_FIELD_NAME'RAY_RUNTIME_ENV_PLUGIN_DEFAULT_PRIORITY#RAY_RUNTIME_ENV_PLUGIN_MAX_PRIORITY#RAY_RUNTIME_ENV_PLUGIN_MIN_PRIORITYRAY_RUNTIME_ENV_PLUGINS_ENV_VAR#RAY_RUNTIME_ENV_PRIORITY_FIELD_NAME)RuntimeEnvContext)URICache) DeveloperAPIc @seZdZUdZdZeed<eZe ed<e de ddfddZ d d de efd d Zd eededejdefddZde ed d dedejddf ddZd edejdefddZdS)RuntimeEnvPluginz4Abstract base class for runtime environment plugins.Nnamepriorityruntime_env_dictreturncCdS)a Validate user entry for this plugin. The method is invoked upon installation of runtime env. Args: runtime_env_dict: The user-supplied runtime environment dict. Raises: ValueError: If the validation fails. N)rrrS/home/ubuntu/.local/lib/python3.10/site-packages/ray/_private/runtime_env/plugin.pyvalidates zRuntimeEnvPlugin.validate runtime_env RuntimeEnvcCsgSNr)selfrrrrget_uris,szRuntimeEnvPlugin.get_urisuricontextloggercsdS)a{Create and install the runtime environment. Gets called in the runtime env agent at install time. The URI can be used as a caching mechanism. Args: uri: A URI uniquely describing this resource. runtime_env: The RuntimeEnv object. context: Auxiliary information supplied by Ray. logger: A logger to log messages during the context modification. Returns: float: The disk space taken up by this plugin installation for this environment. e.g. for working_dir, this downloads the files to the local node. rr)rr rr!r"rrrcreate/szRuntimeEnvPlugin.createuriscCr)aModify context to change worker startup behavior. For example, you can use this to prepend "cd " command to worker startup, or add new environment variables. Args: uris: The URIs used by this resource. runtime_env: The RuntimeEnv object. context: Auxiliary information supplied by Ray. logger: A logger to log messages during the context modification. Nr)rr$rr!r"rrrmodify_contextHszRuntimeEnvPlugin.modify_contextcCr)aDelete the runtime environment given uri. Args: uri: A URI uniquely describing this resource. logger: The logger used to log messages during the deletion. Returns: float: The amount of space reclaimed by the deletion. rr)rr r"rrr delete_uri\s zRuntimeEnvPlugin.delete_uri)__name__ __module__ __qualname____doc__rstr__annotations__r rint staticmethoddictrrrrrloggingLoggerfloatr#r%r&rrrrrs8      rc@s&eZdZdedededefddZdS)PluginSetupContextrclass_instancer uri_cachecCs||_||_||_||_dSr)rr4rr5)rrr4rr5rrr__init__js zPluginSetupContext.__init__N)r'r(r)r+rr-rr6rrrrr3isr3c@seZdZdZddZdeeddfddZd eddfd d Z d e e ddfd dZ deddfddZ dedefddZde efddZdS)RuntimeEnvPluginManagerz:This manager is used to load plugins in runtime env agent.cCs2i|_tjt}|rt|}||dSdSr)pluginsosenvirongetr jsonloads load_plugins)rplugin_config_strplugin_configsrrrr6zs   z RuntimeEnvPluginManager.__init__ plugin_classrNcCs^t|ts td|d|jstd|d|j|jvr-td|d|j|jddS)Nz!Invalid runtime env plugin class zQ. The plugin class must inherit ray._private.runtime_env.plugin.RuntimeEnvPlugin.z$No valid name in runtime env plugin .zThe name of runtime env plugin z conflicts with ) issubclassr RuntimeErrorrr8)rrArrrvalidate_plugin_classs    z-RuntimeEnvPluginManager.validate_plugin_classrcCs:t|tr |tks |tkrtd|dtdtddS)NzInvalid runtime env priority z", it should be an integer between z and rB) isinstancer-r r rD)rrrrrvalidate_prioritysz)RuntimeEnvPluginManager.validate_priorityr@cCs|D]D}t|tr t|vrtd|dtdt|t}||t|vr,|t}n|j}|||}t |j ||| ||j |j <qdS)z8Load runtime env plugins and create URI caches for them.z"Invalid runtime env plugin config z+, it should be a object which contains the z field.N) rFr/r rDrrErrrGr3rcreate_uri_cache_for_pluginr8)rr@ plugin_configrArr4rrrr>s.    z$RuntimeEnvPluginManager.load_pluginsplugincCsBt|}||||jt|j||j|||j|j<dS)zAdd a plugin to the manager and create a URI cache for it. Args: plugin: The class instance of the plugin. N)typerErGrr3rrHr8)rrJrArrr add_plugins  z"RuntimeEnvPluginManager.add_plugincCs8d|jd}tdttj|d}t|j|S)zCreate a URI cache for a plugin. Args: plugin_name: The name of the plugin. Returns: The created URI cache for the plugin. RAY_RUNTIME_ENV__CACHE_SIZE_GBi@ ) rupperr-r2r9r:r;rr&)rrJcache_size_env_varcache_size_bytesrrrrHs   z3RuntimeEnvPluginManager.create_uri_cache_for_plugincCst|jdddS)zGet the sorted plugin setup contexts, sorted by increasing priority. Returns: The sorted plugin setup contexts. cSs|jSr)r)xrrrszFRuntimeEnvPluginManager.sorted_plugin_setup_contexts..)key)sortedr8values)rrrrsorted_plugin_setup_contextssz4RuntimeEnvPluginManager.sorted_plugin_setup_contexts)r'r(r)r*r6rrrErrGrrr>rLrrHr3rXrrrrr7ws  r7rrrJr5r!r"cs|j|vs ||jdurdS||||}|s1|d|jd|jd|||dIdH|D]8}||vrW|d|d|j||||dIdH}|j|||dq3|d|jd|d |j||dq3|||||dS) zISet up the environment using the plugin if not already set up and cached.NzNo URIs for runtime env plugin z+; create always without checking the cache.)r"zCache miss for URI rBz Runtime env  zq is already installed and will be reused. Search all runtime_env_setup-*.log to find the corresponding setup log.) rrrdebugr#addinfo mark_usedr%)rrJr5r!r"r$r size_bytesrrrcreate_for_plugin_if_neededs(   r_)"r<r0r9abcrtypingrrrrrray._common.utilsr"ray._private.runtime_env.constantsr r r r r r ray._private.runtime_env.contextr"ray._private.runtime_env.uri_cacherray.util.annotationsr getLoggerr'default_loggerrr3r7r1r_rrrrs6       Qt