o `۷i@spddlmZddlmZmZddlmZddlmZm Z ddl m Z e\Z Z Z e\ZZ e GdddZdS) )Optional) try_import_tftry_import_torch)PiecewiseSchedule)LearningRateOrSchedule TensorType) DeveloperAPIc @seZdZdZddddededeefdd Zeded ed ed dfd dZ d e fddZ de d e fddZde d e fddZdS) SchedulerzClass to manage a scheduled (framework-dependent) tensor variable. Uses the PiecewiseSchedule (for maximum configuration flexibility) torchN) frameworkdevicefixed_value_or_scheduler r cCs^||_||_t|ttf|_|jr*t||dddd|_|j|ddd|_ dS||_ dS)aInitializes a Scheduler instance. Args: fixed_value_or_schedule: A fixed, constant value (in case no schedule should be used) or a schedule configuration in the format of [[timestep, value], [timestep, value], ...] Intermediary timesteps will be assigned to linerarly interpolated values. A schedule config's first entry must start with timestep 0, i.e.: [[0, initial_value], [...]]. framework: The framework string, for which to create the tensor variable that hold the current value. This is the variable that can be used in the graph, e.g. in a loss function. device: Optional device (for torch) to place the tensor variable on. N) outside_valuer r) initial_value) r r isinstancelisttuple use_scheduler _schedule_create_tensor_variable _curr_value)selfr r r rY/home/ubuntu/vllm_env/lib/python3.10/site-packages/ray/rllib/utils/schedules/scheduler.py__init__s   zScheduler.__init__ setting_name descriptionreturnc Cst|ttfs |dur dSt|ttfrt|dkr(td|d|d|d|dddkrLtd|d |d |ddd |d |dd d tdd|Dr`td|d|ddS)aPerforms checking of a certain schedule configuration. The first entry in `value_or_schedule` (if it's not a fixed value) must have a timestep of 0. Args: fixed_value_or_schedule: A fixed, constant value (in case no schedule should be used) or a schedule configuration in the format of [[timestep, value], [timestep, value], ...] Intermediary timesteps will be assigned to linerarly interpolated values. A schedule config's first entry must start with timestep 0, i.e.: [[0, initial_value], [...]]. setting_name: The property name of the schedule setting (within a config), e.g. `lr` or `entropy_coeff`. description: A full text description of the property that's being scheduled, e.g. `learning rate`. Raises: ValueError: In case, errors are found in the schedule's format. Nz Invalid `z` (zP) specified! Must be a list of 2 or more tuples, each of the form (`timestep`, `zG to reach`), for example `[(0, 0.001), (1e6, 0.0001), (2e6, 0.00005)]`.rzWhen providing a `zW` schedule, the first timestep must be 0 and the corresponding lr value is the initial z! You provided ts= =r.css|] }t|dkVqdS)r N)len).0pairrrr osz%Scheduler.validate..z_` schedule, each tuple in the schedule list must have exctly 2 items of the form (`timestep`, `)rintfloatrrr$ ValueErrorany)r rrrrrvalidate=s<    zScheduler.validatecCs|jS)aReturns the current value (as a tensor variable). This method should be used in loss functions of other (in-graph) places where the current value is needed. Returns: The tensor variable (holding the current value to be used). )r)rrrrget_current_valuews zScheduler.get_current_valuetimestepcCsJ|jr |jj|d}|jdkrt||j_|S|j||S|j}|S)afUpdates the underlying (framework specific) tensor variable. In case of a fixed value, this method does nothing and only returns the fixed value as-is. Args: timestep: The current timestep that the update might depend on. Returns: The current value of the tensor variable as a python float. )tr ) rrvaluer r tensorrdataassign)rr. python_valuerrrupdates   zScheduler.updatercCs2|jdkrtj|dtj|jdStj|dtjdS)zCreates a framework-specific tensor variable to be scheduled. Args: initial_value: The initial (float) value for the variable to hold. Returns: The created framework-specific tensor variable. r F) requires_graddtyper ) trainabler7)r r r1float32r tfVariable)rrrrrrs z!Scheduler._create_tensor_variable)__name__ __module__ __qualname____doc__rstrrr staticmethodr,rr-r(r)r5rrrrrr s0  * 9 r N)typingrray.rllib.utils.frameworkrr,ray.rllib.utils.schedules.piecewise_schedulerray.rllib.utils.typingrrray.util.annotationsr_r:r r rrrrs