o `۷i@shddlmZddlZddlmZmZddlZddlZ ddl m Z dddZ dddd dd dddZdS)) annotationsN)ListUnioncountspa.ArrayreturncCs&t|}ttjdg|jd|gS)z:Convert per-row counts to list offsets via cumulative sum.rtype)pccumulative_sumpa concat_arraysarrayr )rcumsumrT/home/ubuntu/vllm_env/lib/python3.10/site-packages/ray/data/_internal/arrow_utils.py_counts_to_offsets s rF)offsetsvaluesis_large null_mask column_values-List[Union[pa.Array, pa.ChunkedArray]] | Nonerpa.Array | Nonerrboolrc Cs|dur|dus |durtdn.dd|D}|rtnt}tjtdgt|g|d}tt j dd|D}|rEtnt}t ||}|rTtj ntj}|r_t|jnt|j}|j||||dS) zBCombine list arrays or build a list array from offsets and values.NzAEither column_values or both offsets and values must be provided.cSsg|]}t|qSr)len.0vrrr sz*_combine_as_list_array..rrcSs$g|]}t|tjr |jn|gqSr) isinstancer ChunkedArraychunksrrrrr%s)mask) ValueErrorr int64int32rnp concatenaterr itertoolschainr castLargeListArray ListArray large_listr list_ from_arrays) rrrrrlens offsets_type array_cls list_typerrr_combine_as_list_arrays*   r5)rrrr)N) rrrrrrrrrrrr) __future__rr)typingrrnumpyr'pyarrowr pyarrow.computecomputer rr5rrrrs