o ,wi9 @sUdZddlZddlZddlZddlZddlmZddlmZddl m Z ddl m Z m Z mZmZmZmZmZmZmZmZmZmZmZmZddlmZddlmZmZmZm Z m!Z!m"Z"d d l#m$Z$m%Z%m&Z&m'Z'd d l(m)Z)m*Z*e rdd l+m,Z,m-Z-ddl.m/Z/e0Z1ee0e fZ2ee3efZ4ee4e2e1dfZ5ddhZ6e7e8Z9eGdddZ:eGdddZ;ddZede5dee4fddZ?ededdeedfd dZ?edee5deee4ddffd!dZ?de5de0fd"d#Z@d$e0ddfd%d&ZAde3defd'd(ZBde3defd)d*ZCde3ddfd+d,ZDd-ee3efdefd.d/ZEd0ee3d1eFdeee0ee*ffd2d3ZGd0ee3d1eFdeee0ee*ffd4d5ZHd6e3d1eFdeee0e*ffd7d8ZId9ee3dee)fd:d;ZJd9ee3dee)fdd?ZLd@dAd-dBdee3fdCdDZMiZNeee0ee0feOdE<dFee0dGee0ddfdHdIZPdFee0dee0fdJdKZQdLedefdMdNZRdOee0dPee0de fdQdRZSdS)SzEContains utilities used by both the sync and async inference clients.N)contextmanager) dataclass)Path) TYPE_CHECKINGAny AsyncIterableBinaryIOContextManagerDict GeneratorIterableListLiteralNoReturnOptionalUnionoverload) HTTPError)GenerationErrorIncompleteGenerationErrorOverloadedErrorTextGenerationError UnknownErrorValidationError) get_sessionis_aiohttp_availableis_numpy_availableis_pillow_available)ChatCompletionStreamOutputTextGenerationStreamOutput)ClientResponse ClientSessionImager%z text-to-imagezimage-to-imagec@s\eZdZUeed<eed<eeed<eeeeefed<ee ed<eee fed<dS)RequestParametersurltaskmodeljsondataheadersN) __name__ __module__ __qualname__str__annotations__rrr r ContentTrr3r3^/home/ubuntu/sommelier/.venv/lib/python3.10/site-packages/huggingface_hub/inference/_common.pyr&Hs   r&c@s2eZdZUdZeed<eed<eed<eed<dS) ModelStatusas This Dataclass represents the model status in the HF Inference API. Args: loaded (`bool`): If the model is currently loaded into HF's Inference API. Models are loaded on-demand, leading to the user's first request taking longer. If a model is loaded, you can be assured that it is in a healthy state. state (`str`): The current state of the model. This can be 'Loaded', 'Loadable', 'TooBig'. If a model's state is 'Loadable', it's not too big and has a supported backend. Loadable models are automatically loaded when the user first requests inference on the endpoint. This means it is transparent for the user to load a model, except that the first call takes longer to complete. compute_type (`Dict`): Information about the compute resource the model is using or will use, such as 'gpu' type and number of replicas. framework (`str`): The name of the framework that the model was built with, such as 'transformers' or 'text-generation-inference'. loadedstate compute_type frameworkN)r-r.r/__doc__boolr1r0r r3r3r3r4r5Ss  r5cCststdddl}|S)NzMPlease install aiohttp to use `AsyncInferenceClient` (`pip install aiohttp`).r)r ImportErroraiohttp)r=r3r3r4_import_aiohttptr>cCststdddl}|S)z.Make sure `numpy` is installed on the machine.zGPlease install numpy to use deal with embeddings (`pip install numpy`).rN)rr<numpy)r@r3r3r4 _import_numpy}r?rAcCststdddlm}|S)z,Make sure `PIL` is installed on the machine.zPlease install Pillow to use deal with images (`pip install Pillow`). If you don't want the image to be post-processed, use `client.post(...)` and get the raw response from the server.rr$)rr<PILr%r$r3r3r4_import_pil_images  rCcontentreturncCdSNr3rDr3r3r4_open_as_binaryrIcCrFrGr3rHr3r3r4rIrJccst|tr2|ds|dr"td|t|jVdSt|}| s2t d|dt|trZtd|| d }|VWddS1sSwYdSt |d rytd t }|j|d d |d |VdS|VdS)zOpen `content` as a binary file, either from a URL, a local path, or raw bytes. Do nothing if `content` is None, TODO: handle base64 as input zhttps://zhttp://zDownloading content from NzFile not found at z. If `data` is a string, it must either be a URL or a path to a local file. To pass raw content, please encode it as bytes first.zOpening content from rbsavezConverting PIL Image to bytesPNG)formatr) isinstancer0 startswithloggerdebugrgetrDrexistsFileNotFoundErroropenhasattrioBytesIOrLseek)rDfbufferr3r3r4rIs.    "     cCsPt|}t|tr |n|}t|WdS1s!wYdS)z\Encode a raw file (image, audio) into base64. Can be bytes, an opened file, a path or a URL.N)rIrObytesreadbase64 b64encodedecode)rDr+ data_as_bytesr3r3r4 _b64_encodes  $rc encoded_imagecCst}|tt|S)z/Parse a base64-encoded string into a PIL Image.)rCrVrXrYr_ b64decode)rdr%r3r3r4 _b64_to_imagesrfcCt|S)acParse bytes from a Response object into a Python list. Expects the response body to be JSON-encoded data. NOTE: This is exactly the same implementation as `_bytes_to_dict` and will not complain if the returned data is a dictionary. The only advantage of having both is to help the user (and mypy) understand what kind of data to expect. r*loadsrarHr3r3r4_bytes_to_listrjcCrg)acParse bytes from a Response object into a Python dictionary. Expects the response body to be JSON-encoded data. NOTE: This is exactly the same implementation as `_bytes_to_list` and will not complain if the returned data is a list. The only advantage of having both is to help the user (and mypy) understand what kind of data to expect. rhrHr3r3r4_bytes_to_dictrkrlcCst}|t|S)zParse bytes from a Response object into a PIL Image. Expects the response body to be raw bytes. To deal with b64 encoded images, use `_b64_to_image` instead. )rCrVrXrY)rDr%r3r3r4_bytes_to_imagesrmresponsecCst|tr t|S|SrG)rOr]r*ri)rnr3r3r4_as_dictsrobytes_output_as_linesdetailsc csD|D]}zt||}Wn tyYdSw|dur|VqdS)z*Used in `InferenceClient.text_generation`.N%_format_text_generation_stream_output StopIterationrprq byte_payloadoutputr3r3r4 _stream_text_generation_responses rxc CsN|2z 3dHW}zt||}Wn tyYdSw|dur#|Vq6dS)z/Used in `AsyncInferenceClient.text_generation`.Nrrrur3r3r4&_async_stream_text_generation_response s ryrvcCs~|dsdS|dkrtd|d}t|dd}|ddur2t |d|dt |}|s=|j j S|S Nsdata:s data: [DONE]z[DONE] signal received.zutf-8zdata:z/nerror error_type)rPstriprtrar*rilstriprstriprS_parse_text_generation_errorr!parse_obj_as_instancetokentext)rvrqpayload json_payloadrwr3r3r4rss    rs bytes_linesc csB|D]}zt|}Wn tyYdSw|dur|VqdS)zFUsed in `InferenceClient.chat_completion` if model is served with TGI.N%_format_chat_completion_stream_outputrtritemrwr3r3r4 _stream_chat_completion_response.s  rc CsL|2z3dHW}zt|}Wn tyYdSw|dur"|Vq6dS)z/Used in `AsyncInferenceClient.chat_completion`.Nrrr3r3r4&_async_stream_chat_completion_response;s  rcCsn|dsdS|dkrtd|d}t|dd}|ddur2t |d|dt |Srz) rPr}rtrar*rir~rrSrr r)rvrrr3r3r4rHs    rclientr#r"cCs4|j2z 3dHW}|Vq6|IdHdSrG)rDr}close)rrnrvr3r3r4_async_yield_from]s  r#_UNSUPPORTED_TEXT_GENERATION_KWARGSr)unsupported_kwargscCst|g|dSrG)r setdefaultextend)r)rr3r3r4'_set_unsupported_text_generation_kwargsxsrcCs t|gSrG)rrS)r)r3r3r4'_get_unsupported_text_generation_kwargs|s r http_errorcCs^zt|ddp |j}|d}|d}Wn ty |w|dur-t||}|||)z Try to parse text-generation-inference error message and raise HTTPError in any case. Args: error (`HTTPError`): The HTTPError that have been raised. response_error_payloadNr{r|)getattrrnr*rS Exceptionr)rrr{r| exceptionr3r3r4raise_text_generation_errors    rr{r|cCsH|dkrt|S|dkrt|S|dkrt|S|dkr t|St|S)N generationincomplete_generation overloaded validation)rrrrr)r{r|r3r3r4rsr)Tr:r_rXr*logging contextlibr dataclassesrpathlibrtypingrrrrr r r r r rrrrrrequestsrhuggingface_hub.errorsrrrrrrutilsrrrr_generated.typesr r!r=r"r# PIL.Imager%r0UrlTPathTr]BinaryTr2TASKS_EXPECTING_IMAGES getLoggerr-rQr&r5r>rArCrIrcrfrjrlrmror;rxryrsrrrrrr1rrrrr3r3r3r4s   @              &$             "