o X۷i5 @sddlmZddlZddlZddlmZddlmZddlmZddl mZ ddl m Z ddl mZddd dddd d d Zddd dddddd ddZddd dddd ddZddZddZddZdS)) annotationsN)cublas)_dtype)device)_csr) _interfacegh㈵>)rtolatolmaxiterMcallbackcCs^t||||\}}}}|j} |j} |jd} |dur| d}| dkr)t|dfStj|} | dkr7|dfStt||t| }|| |} d}d}||kr| | }|}t | |}|dkrc|}n ||}|||}| |}|t ||}|||}| ||} |d7}|dur||t | }||krn||ksPd}||kr||ks|}||fS)aUses Conjugate Gradient iteration to solve ``Ax = b``. Args: A (ndarray, spmatrix or LinearOperator): The real or complex matrix of the linear system with shape ``(n, n)``. ``A`` must be a hermitian, positive definitive matrix with type of :class:`cupy.ndarray`, :class:`cupyx.scipy.sparse.spmatrix` or :class:`cupyx.scipy.sparse.linalg.LinearOperator`. b (cupy.ndarray): Right hand side of the linear system with shape ``(n,)`` or ``(n, 1)``. x0 (cupy.ndarray): Starting guess for the solution. rtol, atol (float): Tolerance for convergence. maxiter (int): Maximum number of iterations. M (ndarray, spmatrix or LinearOperator): Preconditioner for ``A``. The preconditioner should approximate the inverse of ``A``. ``M`` must be :class:`cupy.ndarray`, :class:`cupyx.scipy.sparse.spmatrix` or :class:`cupyx.scipy.sparse.linalg.LinearOperator`. callback (function): User-specified function to call after each iteration. It is called as ``callback(xk)``, where ``xk`` is the current solution vector. Returns: tuple: It returns ``x`` (cupy.ndarray) and ``info`` (int) where ``x`` is the converged solution and ``info`` provides convergence information. .. seealso:: :func:`scipy.sparse.linalg.cg` rN ) _make_systemmatvecshapecupy empty_likelinalgnormmaxfloatrdotcnrm2)Abx0r r r r r xrpsolvenb_normritersrhozrho1pbetaqalpharesidinfor-Z/home/ubuntu/vllm_env/lib/python3.10/site-packages/cupyx/scipy/sparse/linalg/_iterative.pycgsL         r/)r r restartr r r callback_typecCst||||\}}} }|j} |j} |jd} | dkr!t|dfStj|}|dkr/|dfStt||t|}|durB| d}|durHd}t || }| durSd} | dvr^t d | |durdd} tj | |f|j dd }tj|d |f|j dd }tj|d f|j d }t|}d} | | }|| |}t|}| d kr||n| dkr|dkr|||||ks||krnr||}||dddf<||d<t|D]A}| |}| |}|||\|d|d |f<}tj|||d |fd|d |kr|||d |f}||dd|d f<qtjt||}t|d}| ||7} ||7}qd}||kr=||ks=|}||fS)aUses Generalized Minimal RESidual iteration to solve ``Ax = b``. Args: A (ndarray, spmatrix or LinearOperator): The real or complex matrix of the linear system with shape ``(n, n)``. ``A`` must be :class:`cupy.ndarray`, :class:`cupyx.scipy.sparse.spmatrix` or :class:`cupyx.scipy.sparse.linalg.LinearOperator`. b (cupy.ndarray): Right hand side of the linear system with shape ``(n,)`` or ``(n, 1)``. x0 (cupy.ndarray): Starting guess for the solution. rtol, atol (float): Tolerance for convergence. restart (int): Number of iterations between restarts. Larger values increase iteration cost, but may be necessary for convergence. maxiter (int): Maximum number of iterations. M (ndarray, spmatrix or LinearOperator): Preconditioner for ``A``. The preconditioner should approximate the inverse of ``A``. ``M`` must be :class:`cupy.ndarray`, :class:`cupyx.scipy.sparse.spmatrix` or :class:`cupyx.scipy.sparse.linalg.LinearOperator`. callback (function): User-specified function to call on every restart. It is called as ``callback(arg)``, where ``arg`` is selected by ``callback_type``. callback_type (str): 'x' or 'pr_norm'. If 'x', the current solution vector is used as an argument of callback function. if 'pr_norm', relative (preconditioned) residual norm is used as an argument. Returns: tuple: It returns ``x`` (cupy.ndarray) and ``info`` (int) where ``x`` is the converged solution and ``info`` provides convergence information. Reference: M. Wang, H. Klie, M. Parashar and H. Sudan, "Solving Sparse Linear Systems on NVIDIA Tesla GPUs", ICCS 2009 (2009). .. seealso:: :func:`scipy.sparse.linalg.gmres` rNrpr_norm)rr3zUnknown callback_type: {}F)dtypeorderrr5Tr)out)rrrrrrrrrmin ValueErrorformatemptyr5zerosnumpy_make_compute_hurrrangelstsqasnumpyarray)rrrr r r0r r r r1rrrr r!VHe compute_hur#mxr"r_normvjr%uretyr,r-r-r.gmresZsr(         rOcCst||||\}}}}|j} |j} |jd} | dkr!t|dfStj|} | dkr/|dfStt||t| }|durB| d}|| |} t | | }| }| }| }d} | |}| |}t | |}||}|||}| ||}|||7}| |}|||8}tj|}|d7}|dur||||ks||krn#t | |}||}|}|||}||9}||7}||9}||7}q[d}||kr||ks|}||fS)aUse Conjugate Gradient Squared iteration to solve ``Ax = b``. Args: A (ndarray, spmatrix or LinearOperator): The real or complex matrix of the linear system with shape ``(n, n)``. b (cupy.ndarray): Right hand side of the linear system with shape ``(n,)`` or ``(n, 1)``. x0 (cupy.ndarray): Starting guess for the solution. rtol, atol (float): Tolerance for convergence. maxiter (int): Maximum number of iterations. M (ndarray, spmatrix or LinearOperator): Preconditioner for ``A``. The preconditioner should approximate the inverse of ``A``. ``M`` must be :class:`cupy.ndarray`, :class:`cupyx.scipy.sparse.spmatrix` or :class:`cupyx.scipy.sparse.linalg.LinearOperator`. callback (function): User-specified function to call after each iteration. It is called as ``callback(xk)``, where ``xk`` is the current solution vector. Returns: tuple: It returns ``x`` (cupy.ndarray) and ``info`` (int) where ``x`` is the converged solution and ``info`` provides convergence information. .. seealso:: :func:`scipy.sparse.linalg.cgs` rNTr) rrrrrrrrrdotcopy)rrrr r r r r rrrr r!r0r$r"rLr'r#rNrJsigmar*r)r%AzrIrho_newr(r,r-r-r.cgss^            rWcCsjt|}t|}|durtj|j||j|jd}|jd|jdkr+td|j|jj dvr9t d|j|jd}|j|fksO|j|dfksOtd| |j }|duret j|f|jd }n|j|fksv|j|dfksvtd | |j }|durtj|j|jd }n#t|}t|}|durtj|j||j|jd}|j|jkrtd ||||fS) aMake a linear system Ax = b Args: A (cupy.ndarray or cupyx.scipy.sparse.spmatrix or cupyx.scipy.sparse.LinearOperator): sparse or dense matrix. M (cupy.ndarray or cupyx.scipy.sparse.spmatrix or cupyx.scipy.sparse.LinearOperator): preconditioner. x0 (cupy.ndarray): initial guess to iterative method. b (cupy.ndarray): right hand side. Returns: tuple: It returns (A, M, x, b). A (LinaerOperator): matrix of linear system M (LinearOperator): preconditioner x (cupy.ndarray): initial guess b (cupy.ndarray): right hand side. N)rrmatvecr5rrz"expected square matrix (shape: {})fdFDzunsupprted dtype (actual: {})zb has incompatible dimensionsr7zx0 has incompatible dimensions)rr5z/matrix and preconditioner have different shapes)_make_fast_matvecraslinearoperatorLinearOperatorrrXr5r:r;char TypeErrorastyperavelrr=IdentityOperator)rr rr fast_matvecr rr-r-r.r!s<        rc s ddlmddlmt|rdrtj t d|j t d|j t |j jtj|jdf|j d}tj|jdf|j d}j|j|}j|} jjj|jjj|j }t|tj~~~~ f dd}|SdS) Nr)cusparsespmv?rr7c sTt|}j|}j|} jjj|jjj|jjj |SN) rrDnVecDescriptorcreatespMVctypesdatadescptr)rrNdesc_xdesc_y _cusparsealgr*r(buff cuda_dtypercdesc_Ahandleop_ar-r.rjs   z!_make_fast_matvec..matvec)cupy_backends.cuda.libsrccupyxrisspmatrix_csrcheck_availabilityrget_cusparse_handle CUSPARSE_OPERATION_NON_TRANSPOSEr>rCr5r to_cuda_dtypeCUSPARSE_MV_ALG_DEFAULTrr<rSpMatDescriptorrhrgspMV_bufferSizerjrkrlint8)rrrNrnro buff_sizerr-rpr.rZTs.       rZcstjjdkrtjnjjdkrtjnjjdkr"tjn jjdkr+tjj dt djt djt djfd d }|S) Nfdr4Drrergc stj|dfjd}tj|djjjj|jjdjj|jjd tj|djjjj|jjdjj|jjd ||fS)Nrr7) rr<r5_cublas CUBLAS_OP_Crjrkrm CUBLAS_OP_N)rLrKhrDgemvrvmoner onezeror-r.rGsz$_make_compute_hu..compute_hu) rget_cublas_handler5r]rsgemvdgemvcgemvzgemvrr>rC)rDrGr-rr.r?ws      r?rf) __future__rr>rr cupy._corer cupy.cudarrxrcupyx.scipy.sparsercupyx.scipy.sparse.linalgrr/rOrWrrZr?r-r-r-r.s*         L k \3 #