o װiZ@sddlZddlZddlZddlZddlmZddlmZej Z ej j j ZejddZddZdPd d Zd d Z dQd dZdRddZ  dSddZdTddZddZ dUddZddZ dQddZ  dVd!d"ZdWd#d$ZdXd&d'ZdXd(d)Z dYd+d,Z!dYd-d.Z"dQd/d0Z#dQd1d2Z$dXd3d4Z%dXd5d6Z&dYd7d8Z'dYd9d:Z(dQd;d<Z)d=d>Z*dQd?d@Z+dAdBZ,dXdCdDZ-dXdEdFZ.dZdHdIZ/dZdJdKZ0dRdLdMZ1dRdNdOZ2dS)[N)config)get_plan_cachecCs|dkr|tjtjfvrtjS|tjtjfvrtjS|S|tjtjfvr,t|jS|tjkr4tjS|tjtjfvr?tjS|S)NR2C) npfloat16float32 complex64 complex128dtypecharlowerfloat64)r value_typerA/home/ubuntu/.local/lib/python3.10/site-packages/cupy/fft/_fft.py _output_dtypes rcCs$t|j|}||jkr||}|SN)rr astype)ar out_dtyperrr_convert_dtype s   rCc Cs|dus ||jkr |S|dkr#|ddur#t|}|ddd|d<t||D]W\}}|dur||j|krt|j}|||krXtdg|j}td|||<|t|}q(tdg|j}td||||<|||<tj||jj |d} || t|<| }q(|S)NC2Rrorder) shapelistzipslicendimtuplecupyzerosr r ) rsaxesrrszaxisrindexzrrr _cook_shape's(   r,cCsddlm}|dkr|tjkr|jS|dkr|tjkr|jS|dkr*|tjkr*|jS|dkr6|tjkr6|j S|dkrB|tj krB|j S|dkrN|tjkrN|j St )NrcufftC2Crr) cupy.cudar.rr CUFFT_C2Cr CUFFT_R2C CUFFT_C2Rr CUFFT_Z2Zr CUFFT_D2Z CUFFT_Z2D ValueError)r rr.rrr_convert_fft_type>s r8c Csddlm} t|j|} ||j|jdkr||d}|jdus%|jjs*| }nt j j j s:|dkr:|s:| }n t j j j rG|dkrG| }|jd} | dkrVtd| t j j j r}|dkr}d|d_|durnd|d _n|d dkr}d|d |d f_|dur| }|j| } | } | dur|dur| }ntd |durtjsdntj}|| | |f}t}|dur|j}|j}||j|j|durdn|jj|durdn|jjf7}t}||}|dur|}nf|dur| j || | |d }|||<nT|rt!d|"d|ddf}|#||||<n9t$|| j std| |j%kr'td||j&kr4td||j&| |j'kr>tdtj|j(dukrKtd|rV|dkrV|}n|durb|)||n|*|}| dkrs|+||||jd}| | j,ks| | j-kr| }|dkr|| j.kr||}n|dkr|t/0|}n|dkr|| j1kr||}||j|jdkr||d}|S)Nrr-rrrr/1Invalid number of FFT data points (%d) specified.).r).rr.AUse the cuFFT plan either as a context manager or as an argument.)devicesz/multi-GPU cuFFT callbacks are not yet supportedPlan1dz'expected plan to have type cufft.Plan1dcuFFT plan dtype mismatch.z*Target array size does not match the plan.z#Batch size does not match the plan.z/Unclear if multiple GPUs are to be used or not.backwardorthoforward)2r0r.r8r r"swapaxesbaseflags c_contiguouscopyr$cudaruntimeis_hiprr7imagsizeget_current_plan RuntimeErrorruse_multi_gpus_devicesget_current_callback_managercb_load_aux_arrcb_store_aux_arrcb_loadcb_storedataptrrgetr<NotImplementedError create_plan set_callbacks isinstancefft_typenxbatchgpuscheck_output_arrayget_output_arrayfftr2r5 CUFFT_INVERSEmathsqrt CUFFT_FORWARD)r directionrnormr) overwrite_xout_sizeoutplanr.r\nr^ curr_planr;keysmgrload_aux store_auxcache cached_planr(rrr _exec_fftQs                            ruc Cs$|D] }t||d||||d}q|S)Nr/rl)ru)rrgrhr'rirlr)rrr_fft_c2csrwr/Fc Csnt|tjs td|dur|durt|t|krtd|dur:|dur*|j}nt|}ddt| dD}nt|}|sJ|dkrF|St d|durPd}|d vrZtd |t ||}t ||||}|dkrvt ||||||d }|S|d krt |||||d |}t ||||dd |}|St ||||dd |}t|j||d |} t |||||d || }|S)N(The input array a must be a cupy.ndarray&Shape and axes have different lengths.cSsg|]}|qSrr.0irrr sz_fft..rr/list index out of ranger?r?r@rACInvalid norm value %s, should be "backward", "ortho", or "forward".rvrr)r[r$ndarray TypeErrorlenr7r"ranger# IndexErrorrr,rwru_get_fftn_out_sizer) rr&r'rhrgrrirldimrjrrr_fftsF    rcs|dur|durt|t|krtd|dur8|dur}nt|}tfddt| dD}|}||fSt|}|s@dStt| ksQtt|dkrUtd|d krjttfd d|D}||fStfd d|dd D}||d t|}||fS) zwConfigure axes argument for an n-dimensional FFT. The axes to be transformed are returned in ascending order. Nrycsg|]}|qSrrrzr"rrr}z#_prep_fftn_axes..r)rrrz/The specified axes exceed the array dimensions.r/cg|]}|qSrrr{axrrrr}rcrrrrrrrr} rr) rr7r#r_reduceminmaxsortedappend)r"r&r'rr axes_sortedrrr_prep_fftn_axess* " rcsRdtko dkno(dvp|dvo(tfddttdDS)Nrrc3s(|]}|d|dkVqdS)rNr)r{rmrrr -s z'_nd_plan_is_possible..)rallr)rr"rrr_nd_plan_is_possible(s rTc s"ddlm}t}||j|jfvrd}n ||j|jfvr d}nd}|dur-tt|} n t |d||d\} } t | |s@t d|d vrHt d  fd d | D} |d kr]| ddd} t| } ||j|jfvrn| } | } nt | } |durz|| d<t| } ||j |jfvr| } n| } | }| }| tt|krd}d}}d}}nhd| vrtt| }td|}|dkrt| }t| }d}d}nF|d krd}d}|}|}n9|d| vr|t| }t| d}|dkrd}d}|}|}n|d krt| }t| }d}d}nt d| D] }|dkrt dq | |||||||||| d|f }t}|durT|j}|j}||j|j|durDdn|jj|durNdn|jjf7}t}||}|dure|}|S|durx|j|}|rv|||<|S|d|ddf}|||r|||<|S)aFGenerate a CUDA FFT plan for transforming up to three axes. Args: shape (tuple of int): The shape of the array to transform fft_type (int): The FFT type to perform. Supported values are: `cufft.CUFFT_C2C`, `cufft.CUFFT_C2R`, `cufft.CUFFT_R2C`, `cufft.CUFFT_Z2Z`, `cufft.CUFFT_Z2D`, and `cufft.CUFFT_D2Z`. axes (None or int or tuple of int): The axes of the array to transform. Currently, these must be a set of up to three adjacent axes and must include either the first or the last axis of the array. If `None`, it is assumed that all axes are transformed. order ({'C', 'F'}): Specify whether the data to be transformed has C or Fortran ordered data layout. out_size (int): The output length along the last axis for R2C/C2R FFTs. For C2C FFT, this is ignored (and set to `None`). to_cache (bool): Whether to cache the generated plan. Default is ``True``. Returns: plan (cufft.PlanNd): A cuFFT Plan for the chosen `fft_type`. rr-r/rrN)r&r'rzAn n-dimensional cuFFT plan could not be created. The axes must be contiguous and non-repeating. Between one and three axes can be transformed and either the first or last axis must be included in axes.rFzorder must be 'C' or 'F'csg|]}|qSrr)r{drrrr}wrz&_get_cufft_plan_nd..rrrrzzGeneral subsets of FFT axes not currently supported for GPU case (Can only batch FFT over the first or last spatial axes).z,Invalid number of FFT data points specified.PlanNd)r0r.rr1r4r3r6r#rrrr7rr2r5rr_prodrrPrQrRrSrTrUrVrrWrrYrZ)rr\r'rrjto_cacher.r"rfft_axes_ in_dimensionsout_dimensionsplan_dimensionsinembedonembednbatchidistodististrideostride min_axis_fftnum_axes_batchrmrorprqrrrsrtrlrrr_get_cufft_plan_nd1s             rcCs`|dkr|dus|ddurd||d}|S|d}|S|dkr,||dd}|Sd}|S)Nrrrrrr)in_shaper& last_axisrrjrrrrsrc s\ddlm} tj|} jjrd} n jjrd} ntd|dkr)|s)n t j j j r6|dkr6| } | dur@| }|durOtj| || |d}nqt|| jsYtd | |jkrgtd |j| jjr}fd d |D} |dkr||| d <nfdd |ddd D} t| } | |jkrtd|j| j| |jkrtd|dkr|d |jkrtd||jkrtd|r|dkrɈ}n|dur|j| d}n|||jdkr|||| | j| jfvrn|tfdd |D}|dkr|| jkr||}|S|dkr|t !|}|S|dkr,|| j"kr,||}|S)Nrr-rrza must be contiguousrr/)r'rrjz'expected plan to have type cufft.PlanNdz+array orders mismatch (plan: {}, input: {})cg|]}j|qSrrrrrrr}z_exec_fftn..rcrrrrrrrr}rzYThe cuFFT plan and a.shape do not match: plan.shape = {}, expected_shape={}, a.shape = {}r>z"The last axis for R2C/C2R mismatchz-The size along the last R2C/C2R axis mismatchrcrrrr)arrrrr}4rr?r@rA)#r0r.r8r rDrE f_contiguousr7rFr$rGrHrIrLrrr[rrformatr#r\r last_sizerar`rKrbr2r5rrcrdrerf)rrgrrhr'rirlrkrjr.r\rrnexpected_shaper(r)rrr _exec_fftns~                rAc CsNt|tjs td|durd}|dvrtd|t|j|||\}} | s0|dkr,|Stdt||}|dkrO|j j r@d}n|j j rGd }nt |}d }n |d vrZtd |t|||||d }|jD] } | d krrtd| qf|d kr|j j st |}n |dkr|j j st|}t|j|| d|} t||||| ||| | d }|S)Nrxr?rrr/r~rrrrzUnsupported order: {}rrr9r)rhr'rirlrkrj)r[r$rrr7rr"rrrDrrEascontiguousarrayrr,rasfortranarrayrr) rr&r'rhrgrrrlrirkrrmrjrrr_fftn?sP       rc Csddlm}|}|dur|dur|}ntdt||jr!tSt||js/|jdks/t j s1t S|j j r;|dkr;t St|j|||\}}t|dkrqt||jrqtjjjrod|dkrit||jkri|j jrit S|dkrot StSt S)Nrr-r:rr/r)r0r.rLrMr[rrr<r"renable_nd_planningrrDrrrrr$rGrHrIrE) rr&r'rlrr.rnrrrrr_default_fft_funcrs0    rrcC"ddlm}t||f|f||jS)aCompute the one-dimensional FFT. Args: a (cupy.ndarray): Array to be transform. n (None or int): Length of the transformed axis of the output. If ``n`` is not given, the length of the input along the axis specified by ``axis`` is used. axis (int): Axis over which to compute the FFT. norm (``"backward"``, ``"ortho"``, or ``"forward"``): Optional keyword to specify the normalization mode. Default is ``None``, which is an alias of ``"backward"``. Returns: cupy.ndarray: The transformed array which shape is specified by ``n`` and type will convert to complex if the input is other. .. seealso:: :func:`numpy.fft.fft` rr-r0r.rrfrrmr)rhr.rrrrb rbcCr)aCompute the one-dimensional inverse FFT. Args: a (cupy.ndarray): Array to be transform. n (None or int): Length of the transformed axis of the output. If ``n`` is not given, the length of the input along the axis specified by ``axis`` is used. axis (int): Axis over which to compute the FFT. norm (``"backward"``, ``"ortho"``, or ``"forward"``): Optional keyword to specify the normalization mode. Default is ``None``, which is an alias of ``"backward"``. Returns: cupy.ndarray: The transformed array which shape is specified by ``n`` and type will convert to complex if the input is other. .. seealso:: :func:`numpy.fft.ifft` rr-r0r.rrcrrrrifftrrrcC*ddlm}t|||}||||||jS)aCompute the two-dimensional FFT. Args: a (cupy.ndarray): Array to be transform. s (None or tuple of ints): Shape of the transformed axes of the output. If ``s`` is not given, the lengths of the input along the axes specified by ``axes`` are used. axes (tuple of ints): Axes over which to compute the FFT. norm (``"backward"``, ``"ortho"``, or ``"forward"``): Optional keyword to specify the normalization mode. Default is ``None``, which is an alias of ``"backward"``. Returns: cupy.ndarray: The transformed array which shape is specified by ``s`` and type will convert to complex if the input is other. .. seealso:: :func:`numpy.fft.fft2` rr-r0r.rrfrr&r'rhr.funcrrrfft2  rcCr)a Compute the two-dimensional inverse FFT. Args: a (cupy.ndarray): Array to be transform. s (None or tuple of ints): Shape of the transformed axes of the output. If ``s`` is not given, the lengths of the input along the axes specified by ``axes`` are used. axes (tuple of ints): Axes over which to compute the FFT. norm (``"backward"``, ``"ortho"``, or ``"forward"``): Optional keyword to specify the normalization mode. Default is ``None``, which is an alias of ``"backward"``. Returns: cupy.ndarray: The transformed array which shape is specified by ``s`` and type will convert to complex if the input is other. .. seealso:: :func:`numpy.fft.ifft2` rr-r0r.rrcrrrrifft2rrcCr)aCompute the N-dimensional FFT. Args: a (cupy.ndarray): Array to be transform. s (None or tuple of ints): Shape of the transformed axes of the output. If ``s`` is not given, the lengths of the input along the axes specified by ``axes`` are used. axes (tuple of ints): Axes over which to compute the FFT. norm (``"backward"``, ``"ortho"``, or ``"forward"``): Optional keyword to specify the normalization mode. Default is ``None``, which is an alias of ``"backward"``. Returns: cupy.ndarray: The transformed array which shape is specified by ``s`` and type will convert to complex if the input is other. .. seealso:: :func:`numpy.fft.fftn` rr-rrrrrfftnrrcCr)a Compute the N-dimensional inverse FFT. Args: a (cupy.ndarray): Array to be transform. s (None or tuple of ints): Shape of the transformed axes of the output. If ``s`` is not given, the lengths of the input along the axes specified by ``axes`` are used. axes (tuple of ints): Axes over which to compute the FFT. norm (``"backward"``, ``"ortho"``, or ``"forward"``): Optional keyword to specify the normalization mode. Default is ``None``, which is an alias of ``"backward"``. Returns: cupy.ndarray: The transformed array which shape is specified by ``s`` and type will convert to complex if the input is other. .. seealso:: :func:`numpy.fft.ifftn` rr-rrrrrifftnrrcC$ddlm}t||f|f||jdS)aMCompute the one-dimensional FFT for real input. Args: a (cupy.ndarray): Array to be transform. n (None or int): Number of points along transformation axis in the input to use. If ``n`` is not given, the length of the input along the axis specified by ``axis`` is used. axis (int): Axis over which to compute the FFT. norm (``"backward"``, ``"ortho"``, or ``"forward"``): Optional keyword to specify the normalization mode. Default is ``None``, which is an alias of ``"backward"``. Returns: cupy.ndarray: The transformed array which shape is specified by ``n`` and type will convert to complex if the input is other. The length of the transformed axis is ``n//2+1``. .. seealso:: :func:`numpy.fft.rfft` rr-rrrrrrrfft1s rcCr)aCompute the one-dimensional inverse FFT for real input. Args: a (cupy.ndarray): Array to be transform. n (None or int): Length of the transformed axis of the output. For ``n`` output points, ``n//2+1`` input points are necessary. If ``n`` is not given, it is determined from the length of the input along the axis specified by ``axis``. axis (int): Axis over which to compute the FFT. norm (``"backward"``, ``"ortho"``, or ``"forward"``): Optional keyword to specify the normalization mode. Default is ``None``, which is an alias of ``"backward"``. Returns: cupy.ndarray: The transformed array which shape is specified by ``n`` and type will convert to complex if the input is other. If ``n`` is not given, the length of the transformed axis is`2*(m-1)` where `m` is the length of the transformed axis of the input. .. seealso:: :func:`numpy.fft.irfft` rr-rrrrrrirfftKs rcC0ddlm}t|||dd}||||||jdS)aOCompute the two-dimensional FFT for real input. Args: a (cupy.ndarray): Array to be transform. s (None or tuple of ints): Shape to use from the input. If ``s`` is not given, the lengths of the input along the axes specified by ``axes`` are used. axes (tuple of ints): Axes over which to compute the FFT. norm (``"backward"``, ``"ortho"``, or ``"forward"``): Optional keyword to specify the normalization mode. Default is ``None``, which is an alias of ``"backward"``. Returns: cupy.ndarray: The transformed array which shape is specified by ``s`` and type will convert to complex if the input is other. The length of the last axis transformed will be ``s[-1]//2+1``. .. seealso:: :func:`numpy.fft.rfft2` rr-rrrrrrrrfft2g rcCr)aCompute the two-dimensional inverse FFT for real input. Args: a (cupy.ndarray): Array to be transform. s (None or tuple of ints): Shape of the output. If ``s`` is not given, they are determined from the lengths of the input along the axes specified by ``axes``. axes (tuple of ints): Axes over which to compute the FFT. norm (``"backward"``, ``"ortho"``, or ``"forward"``): Optional keyword to specify the normalization mode. Default is ``None``, which is an alias of ``"backward"``. Returns: cupy.ndarray: The transformed array which shape is specified by ``s`` and type will convert to complex if the input is other. If ``s`` is not given, the length of final transformed axis of output will be `2*(m-1)` where `m` is the length of the final transformed axis of the input. .. seealso:: :func:`numpy.fft.irfft2` rr-rrrrrrrirfft2 rcCr)aMCompute the N-dimensional FFT for real input. Args: a (cupy.ndarray): Array to be transform. s (None or tuple of ints): Shape to use from the input. If ``s`` is not given, the lengths of the input along the axes specified by ``axes`` are used. axes (tuple of ints): Axes over which to compute the FFT. norm (``"backward"``, ``"ortho"``, or ``"forward"``): Optional keyword to specify the normalization mode. Default is ``None``, which is an alias of ``"backward"``. Returns: cupy.ndarray: The transformed array which shape is specified by ``s`` and type will convert to complex if the input is other. The length of the last axis transformed will be ``s[-1]//2+1``. .. seealso:: :func:`numpy.fft.rfftn` rr-rrrrrrrrfftnrrcCs@|dur|ddur|dS|dS|dur||dS|dS)Nrr)rr&r'rrr_size_last_transform_axiss  rcCr)aCompute the N-dimensional inverse FFT for real input. Args: a (cupy.ndarray): Array to be transform. s (None or tuple of ints): Shape of the output. If ``s`` is not given, they are determined from the lengths of the input along the axes specified by ``axes``. axes (tuple of ints): Axes over which to compute the FFT. norm (``"backward"``, ``"ortho"``, or ``"forward"``): Optional keyword to specify the normalization mode. Default is ``None``, which is an alias of ``"backward"``. Returns: cupy.ndarray: The transformed array which shape is specified by ``s`` and type will convert to complex if the input is other. If ``s`` is not given, the length of final transformed axis of output will be ``2*(m-1)`` where `m` is the length of the final transformed axis of the input. .. seealso:: :func:`numpy.fft.irfftn` rr-rrrrrrrirfftnrrcCs8|dvrd}|S|dkrd}|S|dkrtd||S)N)Nr?rAr?r@zCInvalid norm value %s; should be "backward", "ortho", or "forward".)r7)rhrrr_swap_directionsrcCst|||t|S)aCompute the FFT of a signal that has Hermitian symmetry. Args: a (cupy.ndarray): Array to be transform. n (None or int): Length of the transformed axis of the output. For ``n`` output points, ``n//2+1`` input points are necessary. If ``n`` is not given, it is determined from the length of the input along the axis specified by ``axis``. axis (int): Axis over which to compute the FFT. norm (``"backward"``, ``"ortho"``, or ``"forward"``): Optional keyword to specify the normalization mode. Default is ``None``, which is an alias of ``"backward"``. Returns: cupy.ndarray: The transformed array which shape is specified by ``n`` and type will convert to complex if the input is other. If ``n`` is not given, the length of the transformed axis is ``2*(m-1)`` where `m` is the length of the transformed axis of the input. .. seealso:: :func:`numpy.fft.hfft` )rconjrrrmr)rhrrrhfftsrcCst|||t|S)aWCompute the FFT of a signal that has Hermitian symmetry. Args: a (cupy.ndarray): Array to be transform. n (None or int): Number of points along transformation axis in the input to use. If ``n`` is not given, the length of the input along the axis specified by ``axis`` is used. axis (int): Axis over which to compute the FFT. norm (``"backward"``, ``"ortho"``, or ``"forward"``): Optional keyword to specify the normalization mode. Default is ``None``, which is an alias of ``"backward"``. Returns: cupy.ndarray: The transformed array which shape is specified by ``n`` and type will convert to complex if the input is other. The length of the transformed axis is ``n//2+1``. .. seealso:: :func:`numpy.fft.ihfft` )rrrrrrrihfftsr?cCsDttjd|dddtjdtj|d dtjdf||S)zReturn the FFT sample frequencies. Args: n (int): Window length. d (scalar): Sample spacing. Returns: cupy.ndarray: Array of length ``n`` containing the sample frequencies. .. seealso:: :func:`numpy.fft.fftfreq` rrrr )r$hstackarangerr rmrrrrfftfreqs rcCs"tjd|ddtjd||S)a Return the FFT sample frequencies for real input. Args: n (int): Window length. d (scalar): Sample spacing. Returns: cupy.ndarray: Array of length ``n//2+1`` containing the sample frequencies. .. seealso:: :func:`numpy.fft.rfftfreq` rrrr)r$rrr rrrrrfftfreq-s" rcNt|durttj}nt|tr|f}tfdd|D|S)aSShift the zero-frequency component to the center of the spectrum. Args: x (cupy.ndarray): Input array. axes (int or tuple of ints): Axes over which to shift. Default is ``None``, which shifts all axes. Returns: cupy.ndarray: The shifted array. .. seealso:: :func:`numpy.fft.fftshift` Ncsg|] }j|dqSrrr{r)xrrr}Oszfftshift..r$asarrayrrr"r[introllrr'rrrfftshift=  rcr)a3The inverse of :meth:`fftshift`. Args: x (cupy.ndarray): Input array. axes (int or tuple of ints): Axes over which to shift. Default is ``None``, which shifts all axes. Returns: cupy.ndarray: The shifted array. .. seealso:: :func:`numpy.fft.ifftshift` Ncsg|] }j|d qSrrrrrrr}dszifftshift..rrrrr ifftshiftRrr)r)NNNr)r/FN)NNr/)NrNT)r/rNFN)NNNr/)NrN)NrN)r)3 functoolsrdnumpyrr$cupy.fftrcupy.fft._cacherreducer_coreinternalprodr_utilmemoizerrr,r8rurwrrrrrrrrrbrrrrrrrrrrrrrrrrrrrrrrrsh        -$ * X 3 '