o ٷiA@sUdZddlZddlmZmZmZmZmZddlm Z m Z m Z ddl m Z ddlmZmZddlmZmZdd lmZeeZdaeeed <daeejed <Gd d d eZGdddeZdede fddZdS)zFastAPI HTTP Server for cache-dit. Adapted from SGLang's HTTP server: https://github.com/sgl-project/sglang/blob/main/python/sglang/srt/entrypoints/http_server.py N)OptionalDictAnyListLiteral)FastAPI HTTPExceptionResponse) JSONResponse) BaseModelField) ModelManagerGenerateRequest) init_logger_global_model_manager_request_semaphorec@seZdZUdZedddZeed<edddZe eed<ed d d d d Z e ed<ed dd d d Z e ed<eddddd Z e ed<eddddd Zeed<edddZe eeed<edddZe e ed<edd dd!d Ze ed"<edd#dZe eeed$<edd%ddd Ze e ed&<ed'd(dd)d Ze e ed*<ed+d,dZeed-<ed.d/dZed0ed1<edd2dZe eed3<dS)4GenerateRequestAPIz-API request model for image/video generation..z Text prompt descriptionpromptzNegative promptnegative_promptizImage/Video width@i)rgelewidthzImage/Video heightheight2zNumber of inference stepsr num_inference_stepsg@zGuidance scalegg4@guidance_scaleNzdCustom sigma schedule (e.g. for turbo inference). Length should typically match num_inference_steps.sigmasz Random seedseedzNumber of images to generate num_imageszInput images for image editing. Supports: URLs (http/https), local file paths, base64 strings (with or without data URI prefix) image_urlsz%Number of frames for video generation num_framesz"Frames per second for video output<fpsFzInclude stats field in response include_statsbase64zOutput format: base64 or path)r,path output_formatz;Output directory when output_format=path (server-side path) output_dir)__name__ __module__ __qualname____doc__r rstr__annotations__rrrintrr r!floatr"rr#r%r&r'r*r+boolr.rr/r9r9N/home/ubuntu/.local/lib/python3.10/site-packages/cache_dit/serve/api_server.pyrs> rc@seZdZUdZedddZeeee d<edddZ eee d<edddZ ee ee fe d <edd dZeee d <edd dZeee d <edddZeee d<dS)GenerateResponseAPIz.API response model for image/video generation.Nz#Base64 encoded images or file pathsrimagesz'Base64 encoded video (mp4) or file pathvideozCache statisticsstatszGeneration time in seconds time_costz6Inference start time (local time with timezone offset)inference_start_timez4Inference end time (local time with timezone offset)inference_end_time)r0r1r2r3r r<rlistr4r5r=r>rrr?r7r@rAr9r9r9r:r;8s  r; model_managerreturncCst|atdatdddd}|ddd}|d d d }|jd td ddtfdd}|ddd}|S)zCreate FastAPI application.r zCache-DiT Serving APIz;Text-to-image model serving API with cache-dit accelerationz1.0.0)titlerversionz/healthcs,tdus tjdurtdddStdddS)zHealth check endpoint.NModel not loaded) status_codecontentrOK)rpiper r9r9r9r:healthSs  zcreate_app..healthz/get_model_infocs$tdur tdddttdS)zGet model information.NrGModel manager not initializedrIdetailrJ)rrr get_model_infor9r9r9r:rRZs z"create_app..get_model_infoz /generateT)response_modelresponse_model_exclude_nonerequestcs<tdur tdddtjdurtdddt4IdHuzLt|j|j|j|j|j |j |j |j |j |j|j|j|j|j|jd}t}|dtj|IdH}t|j|j|j|j|j|jdWWdIdHSty}zt !dt"|j#d t$|td d t$|dd}~ww1IdHswYdS) z!Generate images from text prompt.NrGrNrOrH)rrrrr r!r"r#r%r&r'r*r+r.r/)r<r=r>r?r@rAzError generating image: z: zGeneration failed: )%rrrLrrrrrrr r!r"r#r%r&r'r*r+r.r/asyncioget_event_looprun_in_executorgenerater;r<r=r>r?r@rA Exceptionloggererrortyper0r4)rU gen_requestloopresponseer9r9r:rZbsV    " zcreate_app..generatez /flush_cachec sttdus tjdurtdddztddidWSty9}ztdt|td d t|dd}~ww) z Flush cache.NrGrHrOmessagezCache flushed successfullyrQzError flushing cache: rVzFailed to flush cache: )rrLrr r[r\r]r4)rbr9r9r: flush_caches zcreate_app..flush_cache) rrW Semaphorerrgetpostr;r)rCapprMrRrZrdr9r9r: create_appGs    .  ri)r3rWtypingrrrrrfastapirrr fastapi.responsesr pydanticr r rCrrcache_dit.loggerrr0r\rr5rrerr;rir9r9r9r:s  "