o bi@sJddlZddlmZmZmZmZmZmZddlZ ddl m Z ddl m Z ddlmZer2ddlmZeddGd d d eZeddd ed efd dZd ed efddZdeeee jefded efddZddZded efddZde jd efddZdee jdeefd e jfdd Ze d!ddeed e jfd"d#ZdS)$N) TYPE_CHECKINGAnySequenceUnionListProtocol)TENSOR_COLUMN_NAME) PublicAPI) DeveloperAPI) ABCSeriesbeta) stabilityc@s eZdZdZddZddZdS) ArrayLikez:Protocol matching ndarray-like objects (like torch.Tensor)cCdSNselfrrX/home/ubuntu/.local/lib/python3.10/site-packages/ray/air/util/tensor_extensions/utils.py __array__zArrayLike.__array__cCrrrrrrr__len__rzArrayLike.__len__N)__name__ __module__ __qualname____doc__rrrrrrrs rvaluereturncCs"t|dot|dot|tj S)z]Checks whether objects are ndarray-like (for ex, torch.Tensor) but NOT and ndarray itselfrr)hasattr isinstancenpndarray)rrrris_ndarray_likes  r"cCsddl}t||j|jfSNr)pyarrowrArray ChunkedArray)rparrr_is_arrow_array$sr( column_values column_namecCsDt|dksJ|tkp!t|p!t|p!t|dtjp!t|dSr#)lenr_is_ndarray_tensor"_is_ndarray_like_not_pyarrow_arrayrr r!)r)r*rrr_should_convert_to_tensor*s  r.cCst|ot| Sr)r"r()r)rrrr-Dsr-tcCsPt|tjsdS|jdkrdS|jjtjur&t|dkr&t|dtjr&dSdS)zReturn whether provided ndarray is a tensor (ie ndim > 1). NOTE: Tensor is defined as a NumPy array such that `len(arr.shape) > 1` FTr)rr r!ndimdtypetypeobject_r+)r/rrrr,Hs  *r,arrcCsx|jjtjur dSt|dkrdSt|dtjsdS|dj}|ddD]}t|tjs1dS|j|kr9dSq&dS)zReturn whether the provided NumPy ndarray is comprised of variable-shaped tensors. NOTE: This is an O(rows) check. Frr0NT)r2r3r r4r+rr!shape)r5r6arrr"_is_ndarray_variable_shaped_tensor\s    r8valuesr c Csz=t.tjtjdkrd}tjj}nd}tj}tjd|dtj ||d}|WdWS1s6wYWdSt yb}zt |}d|vsQd|vr[t |WYd}~S|dd}~ww) a; Create a possibly ragged ndarray. Using the np.array() constructor will fail to construct a ragged ndarray that has a uniform first dimension (e.g. uniform channel dimension in imagery). This function catches this failure and tries a create-and-fill method to construct the ragged ndarray. z2.0.0NFignore)category)copyz*could not broadcast input array from shapez.The requested array has an inhomogeneous shape) warningscatch_warningsr lib NumpyVersion __version__ exceptionsVisibleDeprecationWarning simplefilterarray ValueErrorstrcreate_ragged_ndarray)r9copy_if_needed warning_typer5e error_strrrr_create_possibly_ragged_ndarrayqs&  (rMalphacCs&tjt|td}t||dd<|S)aCreate an array that contains arrays of different length If you're working with variable-length arrays like images, use this function to create ragged arrays instead of ``np.array``. .. note:: ``np.array`` fails to construct ragged arrays if the input arrays have a uniform first dimension: .. testsetup:: import numpy as np from ray.air.util.tensor_extensions.utils import create_ragged_ndarray .. doctest:: >>> values = [np.zeros((3, 1)), np.zeros((3, 2))] >>> np.array(values, dtype=object) Traceback (most recent call last): ... ValueError: could not broadcast input array from shape (3,1) into shape (3,) >>> create_ragged_ndarray(values) array([array([[0.], [0.], [0.]]), array([[0., 0.], [0., 0.], [0., 0.]])], dtype=object) Or if you're creating a ragged array from a single array: .. doctest:: >>> values = [np.zeros((3, 1))] >>> np.array(values, dtype=object)[0].dtype dtype('O') >>> create_ragged_ndarray(values)[0].dtype dtype('float64') ``create_ragged_ndarray`` avoids the limitations of ``np.array`` by creating an empty array and filling it with pointers to the variable-length arrays. )r2N)r emptyr+objectlist)r9r5rrrrHs,rH)r=typingrrrrrrnumpyr ray.air.constantsrray.utilr ray.util.annotationsr pandas.core.dtypes.genericr rboolr"r(r!rGr.r-r,r8rMrHrrrrs<         .