o X۷i:I@sjddlmZddlZddlZddlmZddl m Z ddZ ddZ d d Zd d Zd dZdddZdS)) annotationsN)cublas)devicec Cs0ddlm}|jdkrtd|j|jdvr td|j|jd|jdkr.td|jd|jdkrBtd |j|j|j|jkrRtd |j|j|j}|d kr\d }n|d krcd }n|dkrjd}n|dkrqd}ntd|jt||d}t||d}t||d}|jd}|jdkr|jdnd} |j rt j } n |j rt j } ntd|j stdt} tj|tjd} tjdtjd} || |||jj|}tj||jd}|| |||jj||jj| jj| jjtjj|| || | || |jj|| jj|jj|| jj tjj|| dS)aSolve a linear matrix equation using cusolverDngetr[fs](). Computes the solution to a system of linear equation ``ax = b``. Args: a (cupy.ndarray): The matrix with dimension ``(M, M)``. b (cupy.ndarray): The matrix with dimension ``(M)`` or ``(M, K)``. Returns: cupy.ndarray: The matrix with dimension ``(M)`` or ``(M, K)``. Note: ``a`` and ``b`` will be overwritten. rcusolvera.ndim must be 2 (actual: {}))r"b.ndim must be 1 or 2 (actual: {})r za must be a square matrix.shape mismatch (a: {}, b: {}).zdtype mismatch (a: {}, b: {})fsdFcDzzunsupported dtype (actual:{})getrf_bufferSizegetrfgetrsz'a must be F-contiguous or C-contiguous.zb must be F-contiguous.dtypeN)cupy_backends.cuda.libsrndim ValueErrorformatshaper TypeErrorgetattr _f_contiguous_cublas CUBLAS_OP_N _c_contiguous CUBLAS_OP_T_deviceget_cusolver_handle_cupyempty_numpyint32dataptrlinalg_util3_check_cusolver_dev_info_if_synchronization_allowed)ab _cusolverrthelperrrnnrhstranshandledipivdinfolworkdworkr<B/home/ubuntu/vllm_env/lib/python3.10/site-packages/cupyx/lapack.pygesv sl       r>cCs\ddlm}|jdkrtd|j|jdkrd}n|jdkr&|jd}ntd|j|jd|jdkrBtd|j|j|j|jkrRtd|j|j|j}|d kr\d }n|d krcd }n|d krjd }n|dkrqd}ntd|t||d}t||d}tt|d}|dvrt||d} t||d} nt||d} t||d} tj } |j dvrtj } ntj } |j\} }t | |}tjdtjd}tj||d}t}t}tjd|d}| |kr|jd d}|jd d}||| ||jj| }tj||d}||| ||jj| |jj|jj||jj tjj||| |tj| | |||jj| |jj|jj| }tj||d}| |tj| | |||jj| |jj|jj| |jj||jjtjj| |||tjtj| tj|||jj|jj| |jj| |d|S| j!jd d}|}|jdkr|fn||f}tj"||d d}||d| <|||| |jj|}tj||d}|||| |jj||jj|jj||jj tjj||||tjtj| tj| ||jj|jj||jj| | |tj| ||||jj||jj|jj| }tj||d}| |tj| ||||jj||jj|jj||jj||jjtjj| ||S)aSolves over/well/under-determined linear systems. Computes least-square solution to equation ``ax = b` by QR factorization using cusolverDngeqrf(). Args: a (cupy.ndarray): The matrix with dimension ``(M, N)``. b (cupy.ndarray): The matrix with dimension ``(M)`` or ``(M, K)``. Returns: cupy.ndarray: The matrix with dimension ``(N)`` or ``(N, K)``. rrrrr r r zdtype mismatch (a: {}, b: {}).r r rrrrrzunsupported dtype (actual: {})geqrf_bufferSizegeqrftrsmsdormqr_bufferSizeormqrunmqr_bufferSizeunmqrfdrg?)orderNrrH)#rrrrrrrrr r!charr# CUBLAS_OP_Cminr&r'r(r)r$r%get_cublas_handlearraycopyr*r+r,r-r.CUBLAS_SIDE_LEFTCUBLAS_FILL_MODE_UPPERCUBLAS_DIAG_NON_UNITctypesconjTzeros)r/r0r1r5rr2 geqrf_helperr@rA ormqr_helperrDno_transr6mr4mn_mindev_infotaucusolver_handle cublas_handleonews_size workspacebb out_shaper<r<r=gelsRs                 rec Csddlm}ddl}|jdstdt|j|j}t|d}|dkr-|j}|j }n+|dkr8|j }|j }n |dkrC|j }|j }n|dkrN|j}|j}n d |j}t||j|d d d }tj|}|jd d\} } tt|jdd } t} tj} tj| tjd}|| | | |jj | |jj | tj!j"#|||j}|$%| | dj|d d d }tj|}|jd d\}}tjdtjd}|| | | ||jj | |jj ||jj | tj!j"#|||$%|S)Nrr potrsBatchedpotrsBatched is not availabler rrrDdtype must be float32, float64, complex64 or complex128 (actual: {})CTrHrOrr )&rrcupyx.cusolvercheck_availability RuntimeErrorr( promote_typesr spotrfBatched spotrsBatched dpotrfBatched dpotrsBatched cpotrfBatched cpotrsBatched zpotrfBatched zpotrsBatchedrrastyper&_core _mat_ptrsrintprodr$r%r CUBLAS_FILL_MODE_LOWERr'r)r*r+r,r-r.rTreshape)r/r0r1cupyxr potrfBatchedrfmsgapldar4 batch_sizer7uplor\b_shapebpldbr5r<r<r= _batched_posvs^     rc Csddlm}tjj}||||||||jdkr$t ||St |j |j }t |d}|dkr@|j }|j}|j}n4|dkrN|j}|j}|j}n&|dkr\|j}|j}|j}n|dkrj|j}|j}|j}n d|j }t||j|dd d }|j\} } t} tj } tj!d t j"d } || | | |j#j$| }tj!||d }|| | | |j#j$| |j#j$|| j#j$tjj%|| |j}|&| d j|dd d }|j\}}|| | | ||j#j$| |j#j$|| j#j$ tjj%|| t'|&|S)aPSolve the linear equations A x = b via Cholesky factorization of A, where A is a real symmetric or complex Hermitian positive-definite matrix. If matrix ``A`` is not positive definite, Cholesky factorization fails and it raises an error. Note: For batch input, NRHS > 1 is not currently supported. Args: a (cupy.ndarray): Array of real symmetric or complex hermitian matrices with dimension (..., N, N). b (cupy.ndarray): right-hand side (..., N) or (..., N, NRHS). Returns: x (cupy.ndarray): The solution (shape matches b). rrrr rrrrhTrjr rrl)(rrr&r,r-_assert_cupy_array_assert_stacked_2d_assert_stacked_squarerrr(rprspotrfspotrf_bufferSizespotrsdpotrfdpotrf_bufferSizedpotrscpotrfcpotrf_bufferSizecpotrszpotrfzpotrf_bufferSizezpotrsrrryrr$r%r r~r'r)r*r+r.rascontiguousarray)r/r0r1r-rpotrfpotrf_bufferSizepotrsrrr4r7rr\worksizerbrrr5r<r<r=posvsh         rc Cs&ddlm}tjj}||||||||jdkr%t |||S|j d}|j }|jdkr:|dddf}|jdksCJd|j d|ksNJdt |j |j }t |d }|j|d d |j|d d }}|jjsx|jjsxt|}|jjr| }|j|jkr|}n|}|j |ks|jjst|}|d kr|j}n"|d kr|j}n|d kr|j}n|dkr|j}n d|j } t| t} tjdtjd} || |rtj ntj!|j d|j d|j"j#|j d|j"j#|j d| j"j# tjj$|| |jjr|j|jkr|}|%|S)ak Implements lapack XPOTRS through cusolver.potrs. Solves linear system A x = b given the cholesky decomposition of A, namely L. Supports also batches of linear systems and more than one right-hand side (NRHS > 1). Args: L (cupy.ndarray): Array of Cholesky decomposition of real symmetric or complex hermitian matrices with dimension (..., N, N). b (cupy.ndarray): right-hand side (..., N) or (..., N, NRHS). Note that this array may be modified in place, as usually done in LAPACK. lower (bool): If True, L is lower triangular. If False, L is upper triangular. Returns: cupy.ndarray: The solution to the linear system. Note this may point to the same memory as b, since b may be modified in place. .. warning:: This function calls one or more cuSOLVER routine(s) which may yield invalid results if input conditions are not met. To detect these invalid results, you can set the `linalg` configuration to a value that is not `ignore` in :func:`cupyx.errstate` or :func:`cupyx.seterr`. rrrrlr Nzb is not a vector or a matrixz.length of arrays in b does not match size of Lr F)rOrrrrhr)&rrr&r,r-rrrr_batched_potrsrr(rprryflags f_contiguous c_contiguousasfortranarraysizerTrrrrrrr$r%r'r)r r~rQr*r+r.r) Lr0lowerr1r-r4rrrrr7r\r<r<r=rasr                rrboolc Csfddlm}ddl}|jdstd|j|jdks Jd|j}|j|jkr-|d}|jdd|jddks?Jd t|j |j }t|d }|j |d d d }|j |d d d }|j j re|j j siJd| }|d krt|j }n"|dkr||j}n|dkr|j}n|dkr|j}n d|j }t|t} tjdtjd} t|jdd} |jd} |}tj|} |jd}|dkr|d}ntj|jdd|j d d}tj|d}t|jdD]G}|dkr|d|f|d<|| |rtjntj| d| j j!|jd|j j!|jd| j j!| tj"j#$|| |dkr)||d|f<q|}q|%|S)Nrrrfrgr z8Batch dimension of b is different than that of L).Nrkz'Batch dimension of L and b do not matchr riFrjz"Unexpected non C-contiguous arraysrrrrhrrl).rrI.)&rrrmrnrorrr(rprryrrrrrtrvrxrrr$r%r&r'r)r}rTrzr{ranger r~rQr*r+r,r-r.r)rr0rr1rrrrfrr7r\rr4L_pr5b_tmpb_tmp_pir<r<r=rs             r)rr) __future__rnumpyr(cupyr&rrr cupy.cudarr$r>rerrrrr<r<r<r=s   H;Ob