o iy @sddlZddlmZddlmZddlZddlmZeej dZ gdZ ej e dfded ejd ej d ejfd d Zej e ddfded ejd ej deejd ejf ddZded efddZe de fded ejd ej ded eejf ddZdejd ejfddZdS)N)Path)Optional) safe_openzhadamards.safetensors)random_hadamard_matrixdeterministic_hadamard_matrixis_pow2cpusizedtypedevicereturncCs||dkrtdtt|}|d|krtdtjdgg||d}t|D]}tt||ft|| ff}q'|S)a Construct an n-by-n Hadamard matrix, using Sylvester's construction. `n` must be a power of 2. Adapated from https://github.com/scipy/scipy/blob/v1.15.2/scipy/linalg/_special_matrices.py # noqa: E501 :param size: order of the matrix, must be a power of 2 :param dtype: data type of matrix :param device: device to construct matrix on :return: hadamard matrix of size `size` rz4Cannot construct deterministic hadamard of size <= 0z6Cannot construct deterministic hadamard of size != 2^nr r ) ValueErrorintmathlog2torchtensorrangevstackhstack)r r r rH_r_/home/ubuntu/.local/lib/python3.10/site-packages/compressed_tensors/transform/utils/hadamard.pyr!s  &rgencCs@tjdd|f||d}|j|d}|dd}t|}t|S)a Produces a randomly generated Hadamard matrix. Differs from `deterministic_hadamard_matrix` in that this function supports non powers of 2 and randomization using a seeded generator Adapated from https://github.com/facebookresearch/SpinQuant/blob/main/utils/hadamard_utils.py # noqa: E501 Known matrices were retrieved from N. J. A. Sloane's Library of Hadamard Matrices http://www.neilsloane.com/hadamard/ # noqa: E501 :param size: The dimension of the hamadard matrix :param dtype: data type of matrix :param device: device to construct matrix on :param gen: Optional generator random values :return: randomly generated hadamard matrix rr )lowhighr generatorr r r)rrandinttodiag _matmul_hadU)r r r rQrrrrAs    rncCs|dko ||d@dkS)zt Check if a number is a power of 2 :param n: number to check :return: True iff `n` is a power of 2 rrr)r'rrrr\sr file_pathcCs|jdkr tdn|}t|dt|d<}tdd|Ddd}|D]#}||d krHt||rH|t|j ||d Wd Sq%Wd d S1sTwYd S) a{ Fetch a known hadamard matrix from the given file path. The returned matrix will be of of size `k` such that `n / k` is a power of two. Return None if no such matrix exists. Note: This function reopens the safetensors file every time it is called. This is technically inefficient, but a very small runtime cost and simpler than forcing callers to manage the file open context :param n: size of known hadamard matrix :param dtype: data type to move fetched hadamard to :param device: device to move fetched hadamard to :return: a known hadamard matrix of size `n` if one exists, else None metarpt) frameworkr css|]}t|VqdS)N)r).0keyrrr |sz*_fetch_hadamard_divisor..T)reverserrN) typerr rstrsortedkeysr get_tensorr#)r'r r r( open_devicefiledivisorsdivisorrrr_fetch_hadamard_divisorfs r9XcCs|d}|j}|j}t|||d}|durtd||d}|d|d}|}|jd|kr||jd|jddd|jd}||j}|dddddddf|dddddddf|dddddddf<|dddddddf|dddddddf|dddddddf<||jd|jdd}||}}|jd|ks6|jd|ksJ~|d||||}||jS)Nrr!z0Cannot construct random hadamard matrix of size rr ) r r r r9rcloneviewshaper#)r:r r r hadKKinputoutputrrrr%s*  & PP  r%)rpathlibrtypingrr safetensorsr__file__parent REPO_PATH__all__bfloat16r rr Tensorr Generatorrboolrr1r9r%rrrrs^     "