o -i@sVddlmZddlmZddlmZmZmZmZda ddZ ddZ Gd d d Z d S) ) Awaitable) JSONResponse)ASGIAppReceiveScopeSendFcCstSN_scaling_elastic_epr r i/home/ubuntu/veenaModal/venv/lib/python3.10/site-packages/vllm/entrypoints/serve/elastic_ep/middleware.pyget_scaling_elastic_ep sr cCs|adSrr )valuer r r set_scaling_elastic_epsrc @s@eZdZdZdeddfddZdeded ede dfd d Z dS) ScalingMiddlewarez Middleware that checks if the model is currently scaling and returns a 503 Service Unavailable response if it is. This middleware applies to all HTTP requests and prevents processing when the model is in a scaling state. appreturnNcCs ||_dSr)r)selfrr r r __init__s zScalingMiddleware.__init__scopereceivesendcCsJ|ddkr ||||Strtddidd}||||S||||S)Ntypehttperrorz7The model is currently scaling. Please try again later.i)content status_code)rr r)rrrrresponser r r __call__"s  zScalingMiddleware.__call__) __name__ __module__ __qualname____doc__rrrrrrrr r r r rs"rN) collections.abcrfastapi.responsesrstarlette.typesrrrrr r rrr r r r s