o ۾iA @sddlmZz ddlmZddlZWn"ey4Zzde evr"dndZ ede de dedZ[wwde d ej fd d Z d Zejjd ddgdgedZejdejdejd ejfddZdejdej d ejfddZdedej d dfddZdS))GuideNnumpymlxz3To use the kernels in `outlines_core.kernels.mlx`, z9 must be installed. You can install it with `pip install ` vocab_sizereturncCstjd|ddfdtjdS)N )dtype)npfullint32)rrM/home/ubuntu/.local/lib/python3.10/site-packages/outlines_core/kernels/mlx.pyallocate_token_bitmask s raC // Batch index uint batch = thread_position_in_grid.y; // Element index uint elem = thread_position_in_grid.x; uint bit = ((elem >> 5) < mask_shape[1]) && ((mask[batch * mask_shape[1] + (elem >> 5)] >> (elem & 31)) & 1); out[batch * inp_shape[1] + elem] = bit ? inp[batch * inp_shape[1] + elem] : -INFINITY; bitmask_apply_batchedinpmaskout)name input_names output_namessourcedatacCs>t||gd|jfg|jd|jddfd|jg|jgddS)NTrr)rr)inputstemplategrid threadgroup output_shapes output_dtypes)_KERNELr shape)rrrrr_apply_token_bitmask_kernel)s r&logitsmask_npcCst|}t|jdkr|ntj|dd}t|jdkr|ntj|dd}|jtjkr4td|jdt|jdkrDtd|jdt|jdkrTtd |jd|jd|jdkrotd |jdd |jdd t ||S) aa Apply a logits bitmask inplace, setting the probability of invalid tokens to -infinity. Arguments: logits (mx.array): The logits tensor. mask (mx.array): The token bitmask representing the validity of each token in the logits tensor. Raises: ValueError: If any of the following conditions are not met: - `mask.dtype` is not `mx.int32` - `mask` is not a 2D array - `logits` is not a 2D array - `mask.shape`shape does not match `logits.shape` Returns: None: Modifies the mask array in place. rr)axis2Invalid mask dtype: Expected `np.int32`, but got ``.6Invalid mask dimensions: Expected a 2D array, but got D.z8Invalid logits dimensions: Expected a 2D array, but got z.Invalid batch size: Expected `mask.shape[0]` (z) to match `logits.shape[0]` (z).) mxarraylenr% expand_dimsr r ValueErrorndimr&)r'r(rrrrapply_token_bitmask5s(        r5guidecCs|jtjkrtd|jd|jdkrtd|jd|jddkr-td|jd |jd s6td ||jj |j |j S) am Writes a bitmask to represent the tokens permissible by the current state of the `guide`. Each bit in the bitmask corresponds to a token ID, with a bit value of 1 indicating that the token is allowed and 0 indicating that it is disallowed. This function directly modifies the `mask` array in-place. Arguments: guide (Guide): An instance of the `Guide` class that provides the current guidance state. mask (torch.Tensor): A 2D tensor of type `torch.int32` where the bitmask will be written. The tensor must be contiguous, have a single batch dimension (shape[0] == 1), and reside on the CPU. Raises: ValueError: If any of the following conditions are not met: - `mask.dtype` is not `np.int32` - `mask` is not a 2D tensor - `mask` does not have a single batch dimension (shape[0] != 1) - `mask` is not contiguous in memory - `mask` is not on the CPU device Returns: None: Modifies the `mask` tensor in-place. r*r+r,r-r.rrz_Invalid batch size: Batch mask writes are not supported. Expected shape[0] == 1, but got shape . C_CONTIGUOUSzJMask array must be contiguous in memory. Use `np.ascontiguousarray(mask)`.) r r rr3r4r%flagswrite_mask_intoctypesrsizeitemsize)r6rrrrfill_next_token_bitmaskcs"      r>) outlines_corermlx.corecorer/rr ImportErrorestr missing_depintndarrayr_KERNEL_SOURCEfast metal_kernelr$compiler0r&r5r>rrrrs2     .