o `۷i&@sUddlZddlZddlmZmZmZmZmZmZddl Z ddl Z ddl mZddlmZmZmZddlmZddlmZmZmZz ddlZejZWn eyUdZYnwer^ddlmZe jej e!ej e j"ej#iZ$eeeefe%d<ed d Gd d d e&Z'eGd ddej(Z)dS)N) TYPE_CHECKINGCallableDictOptionalTypeUnion) DataBatchType) BatchFormat_convert_batch_type_to_numpy_convert_batch_type_to_pandas) Checkpoint) Deprecated DeveloperAPI PublicAPI) Preprocessor TYPE_TO_ENUMbeta) stabilityc@seZdZdZdS)!PredictorNotSerializableExceptionz;Error raised when trying to serialize a Predictor instance.N)__name__ __module__ __qualname____doc__rrI/home/ubuntu/vllm_env/lib/python3.10/site-packages/ray/train/predictor.pyr#src @s"eZdZdZd"dedfddZeejde ddfd d Z ed e e j ge j fddfd d ZdedfddZdedddfddZeedefddZedefddZddZdedefddZed#ddZedeejeeejffdeejeeejfffddZd d!ZdS)$ PredictoraVPredictors load models from checkpoints to perform inference. .. note:: The base ``Predictor`` class cannot be instantiated directly. Only one of its subclasses can be used. **How does a Predictor work?** Predictors expose a ``predict`` method that accepts an input batch of type ``DataBatchType`` and outputs predictions of the same type as the input batch. When the ``predict`` method is called the following occurs: - The input batch is converted into a pandas DataFrame. Tensor input (like a ``np.ndarray``) will be converted into a single column Pandas Dataframe. - If there is a :ref:`Preprocessor ` saved in the provided :class:`Checkpoint `, the preprocessor will be used to transform the DataFrame. - The transformed DataFrame will be passed to the model for inference (via the ``predictor._predict_pandas`` method). - The predictions will be outputted by ``predict`` in the same type as the original input. **How do I create a new Predictor?** To implement a new Predictor for your particular framework, you should subclass the base ``Predictor`` and implement the following two methods: 1. ``_predict_pandas``: Given a pandas.DataFrame input, return a pandas.DataFrame containing predictions. 2. ``from_checkpoint``: Logic for creating a Predictor from a :class:`Checkpoint `. 3. Optionally ``_predict_numpy`` for better performance when working with tensor data to avoid extra copies from Pandas conversions. N preprocessorrcCs*tj|jjdtdd||_d|_dS)zBSubclasseses must call Predictor.__init__() to set a preprocessor.z4 is deprecated and will be removed after April 2026.) stacklevelFN)warningswarn __class__rDeprecationWarning _preprocessor_cast_tensor_columnsselfrrrr__init__Ps  zPredictor.__init__ checkpointreturncKt)aCreate a specific predictor from a checkpoint. Args: checkpoint: Checkpoint to load predictor data from. kwargs: Arguments specific to predictor implementations. Returns: Predictor: Predictor object. NotImplementedErrorclsr(kwargsrrrfrom_checkpoint\ zPredictor.from_checkpoint pandas_udfcsGfdddtS)zCreate a Predictor from a Pandas UDF. Args: pandas_udf: A function that takes a pandas.DataFrame and other optional kwargs and returns a pandas.DataFrame. cs4eZdZededdffdd Zd fdd Zd S) z5Predictor.from_pandas_udf..PandasUDFPredictorr(r)rcsSNrr-)PandasUDFPredictorrrr0vzEPredictor.from_pandas_udf..PandasUDFPredictor.from_checkpoint pd.DataFramecs|fi|Sr3r)r&dfr/)r2rr_predict_pandaszszEPredictor.from_pandas_udf..PandasUDFPredictor._predict_pandasN)r)r6)rrr classmethodr r0r8rr4r2rrr4usr4)r)r.r2rr:rfrom_pandas_udfjs zPredictor.from_pandas_udfcCs|jS)z;Get the preprocessor to use prior to executing predictions.r#r&rrrget_preprocessorr5zPredictor.get_preprocessorcCs ||_dS)z;Set the preprocessor to use prior to executing predictions.Nr<r%rrrset_preprocessors zPredictor.set_preprocessorcCstjS)aBatch format hint for upstream producers to try yielding best block format. The preferred batch format to use if both `_predict_pandas` and `_predict_numpy` are implemented. Defaults to Pandas. Can be overridden by predictor classes depending on the framework type, e.g. TorchPredictor prefers Numpy and XGBoostPredictor prefers Pandas as native batch format. )r PANDAS)r.rrrpreferred_batch_formats z Predictor.preferred_batch_formatcCsN|jtjk}|jtjk}|r|r|S|rtjS|rtjStd|jd)z4Determine the batch format to use for the predictor.z Predictor zG must implement at least one of `_predict_pandas` and `_predict_numpy`.) r8r_predict_numpyrAr r@NUMPYr,r)r.has_pandas_implementedhas_numpy_implementedrrr_batch_format_to_uses   zPredictor._batch_format_to_usecCs d|_dS)aEnable automatic tensor column casting. If this is called on a predictor, the predictor will cast tensor columns to NumPy ndarrays in the input to the preprocessors and cast tensor columns back to the tensor extension type in the prediction outputs. TN)r$r=rrr_set_cast_tensor_columnss z"Predictor._set_cast_tensor_columnsdatac Kst|ds tdztt|}Wnty(tdt|dttw|jr2|j |}| }|t j kr_|t j krK|j t|fi|S|t jkr]t|jt|fi|SdS|t jkr|t j krvt|j t|fi|S|t jkr|jt|fi|SdSdS)aPerform inference on a batch of data. Args: data: A batch of input data of type ``DataBatchType``. kwargs: Arguments specific to predictor implementations. These are passed directly to ``_predict_numpy`` or ``_predict_pandas``. Returns: DataBatchType: Prediction result. The return type will be the same as the input type. r#zCSubclasses of Predictor must call Predictor.__init__(preprocessor).zInvalid input data type of z, supported types: N)hasattrr,rtypeKeyError RuntimeErrorlistkeysr#transform_batchrFr r@r8r rCrBr )r&rHr/ batch_formatbatch_format_to_userrrpredictsJ           zPredictor.predictr6cKr*)aPerform inference on a Pandas DataFrame. Args: data: A pandas DataFrame to perform predictions on. kwargs: Arguments specific to the predictor implementation. Returns: A pandas DataFrame containing the prediction result. r+r&rHr/rrrr8r1zPredictor._predict_pandascKr*)aPerform inference on a Numpy data. All Predictors working with tensor data (like deep learning predictors) should implement this method. Args: data: A Numpy ndarray or dictionary of ndarrays to perform predictions on. kwargs: Arguments specific to the predictor implementation. Returns: A Numpy ndarray or dictionary of ndarray containing the prediction result. r+rSrrrrBszPredictor._predict_numpycCstd)NzPredictor instances are not serializable. Instead, you may want to serialize a checkpoint and initialize the Predictor with Predictor.from_checkpoint.)rr=rrr __reduce__szPredictor.__reduce__r3)rHr6r)r6) rrrrrr'r9abcabstractmethodr r0rpd DataFramer;r>r?rr rArFrGrrRr8rnpndarrayrstrrBrTrrrrr*s<$     0   r)*rUrtypingrrrrrrnumpyrYpandasrWray.air.data_batch_typer"ray.air.util.data_batch_conversionr r r ray.trainr ray.util.annotationsr rrpyarrowTablepa_table ImportErrorray.datarrZrCdictrXr@r__annotations__rLrABCrrrrrs2