o [۷iR @sddlmZddlZddlZddlZddlZddlmZddlmZddl Z dej vrEee j j j j drEejdeee j j j ddlZedZdd Zd d Zd d ZddZddZejjeejjeejjeejjeejjeiZ dddZ!dddZ"e#dkre"dSdS)) annotationsN)tqdm)Path NO_LOCAL_GGUFzgguf-pyzgguf-convert-endiancCsdS)N)tensor block_offsrrV/home/ubuntu/vllm_env/lib/python3.10/site-packages/gguf/scripts/gguf_convert_endian.py byteswap_noopsr cC,|j||djtjd}|jdddSNdtypeTinplacedataviewnpuint16byteswaprrdeltarrr byteswap_q4_0rcCr r rrrrr byteswap_q8_0#rrcCsX|j||djtjd}|jdd|j|d|djtjd}|jdddS)Nr rTrrrrrr byteswap_q4_k+s  rcCs0|j|d|djtjd}|jdddS)NrTrrrrrr byteswap_q6_k6s r!readergguf.GGUFReaderargsargparse.NamespacereturnNonec CsZ|jj}|jdkr|dkrdnd}n|}|jdkr|n|j}td|d|d||kr>td|d td td |j D]$}|j t vrj|j t j jt j jt j jfvrjtd |j jd t|jqFtd|d||jr{dStdtd||krtdtdtdtd}|dkrtdtd tdt|jdt|jD]%\}}td|ddt|jdt|j|jD]} | jddqqtd t|j dtt|j d!d"} D]\}}d#t|jd$|j jd%|jd&} |jjD] } | jddq|j t vr|jj } d'} |jj D]}| |9} q.|j!| t j"j#|j d'}t |j }t|j|}tt$|d(d)d*}D] }||}||||d+d kr{|%d,|||d-q\|j!| n|j t j jkr|jj&t'j(d.jddn|jjdd| %| qtd/dS)0NSLITTLEBIGnativez * Host is z endian, GGUF file seems to be z endianz* File is already z endian. Nothing to do.rz/* Checking tensors for conversion compatibilityzCannot handle type z for tensor z* Preparing to convert from z to z&*** Warning *** Warning *** Warning **zH* This conversion process may damage the file. Ensure you have a backup.z]* Requested endian differs from host, you will not be able to load the model on this machine.zQ* The file will be modified immediately, so if conversion fails or is interruptedzX* the file will be corrupted. Enter exactly YES if you are positive you want to proceed:zYES, I am sure> YESz(You didn't enter YES. Okay then, see ya!z* Converting fields ()z- 4z: Converting field z, part count: Trz* Converting tensors (zConverting tensor)desczConverting tensor z, type=z , elements= zByte-swapping BlocksF)r/leaveizByte-swapping Blocks []rz * Completion)) endianessname byte_orderorderupperloggerinfosysexittensors tensor_typebyteswap_tensorsggufGGMLQuantizationTypeF32F16BF16 ValueErrorreprdry_runwarninginputlenfields enumeratevaluespartsrr n_elementsfieldrshaperesize constantsGGML_QUANT_SIZESrangeset_descriptionrrr)r"r$ file_endian host_endianr7rresponseidxrPpartpbar log_messageoldshapenewshapei block_size byteswap_funcn_blocks inner_pbar block_numrrrr convert_byteorderGs            *         rfcCstjdd}|jdtdd|jdtgddd |jd d d d |jdd dd |ttjdkr4dndg}tj |j r@tj ntj dt d|jt|j|jrWdnd}t||dS)NzConvert GGUF file byte order) descriptionmodelzGGUF format model filename)typehelpr7)biglittler+zRequested byte order)richoicesrjz --dry-run store_truezDon't actually change anything)actionrjz --verbosezincrease output verbosityr1z--help)levelz * Loading: rzr+)argparseArgumentParser add_argumentstr parse_argsrJr;argvlogging basicConfigverboseDEBUGINFOr9r:rhr@ GGUFReaderrGrf)parserr$r"rrr mains&  r__main__)r"r#r$r%r&r')r&r')$ __future__rrxrrosr;rpathlibrnumpyrenviron__file__parentexistspathinsertrur@ getLoggerr9r rrrr!rAQ4_0Q8_0Q4_KQ6_KMXFP4r?rfr__name__rrrr s8   "    Y