o -i @sddlmZddlmZmZmZmZddlmZddl m Z ddl m Z e e Zdede fdd ZeZed ed ed fd edededefddZedd edefddZedd edefddZdefddZdS)) HTTPStatus) APIRouterFastAPIQueryRequest) JSONResponse) EngineClient) init_loggerrequestreturncCs |jjjSN)appstate engine_client)r rc/home/ubuntu/veenaModal/venv/lib/python3.10/site-packages/vllm/entrypoints/serve/rlhf/api_router.pyrs rz/pauseFT raw_requestwait_for_inflight_requests clear_cachec st|}z|j||dIdHtdditjjdWSty8}ztdt|itjjdWYd}~Sd}~wt y[}zt dtdd|itj jdWYd}~Sd}~ww) aOPause generation requests to allow weight updates. Args: wait_for_inflight_requests: When ``True`` waits for in-flight requests to finish before pausing. When ``False`` (default), aborts any in-flight requests immediately. clear_cache: Whether to clear KV/prefix caches after draining. )rrNstatuspausedcontent status_codeerrorzFailed to pause generationzFailed to pause generation: ) rpause_generationrrOKvalue ValueErrorstr BAD_REQUEST Exceptionlogger exceptionINTERNAL_SERVER_ERROR)rrrengineerrrrrrs2    rz/resumec sxt|}z|IdHtdditjjdWSty;}ztdtdd|itj jdWYd}~Sd}~ww)z Resume generation after a pause.NrresumedrzFailed to resume generationrzFailed to resume generation: ) rresume_generationrrrrr!r"r#r$)rr%r&rrrr(?s   r(z /is_pausedc stt|}z |IdH}Wn$ty2}ztdtdd|itjjdWYd}~Sd}~wwtd|idS)z Return the current pause status.NzFailed to fetch pause statusrzFailed to fetch pause status: r is_paused)r) rr)r!r"r#rrr$r)rr%rr&rrrr)Ss  r)r cCs|tdSr )include_routerrouter)r rrr attach_routeresr,N)httprfastapirrrrfastapi.responsesrvllm.engine.protocolr vllm.loggerr __name__r"rr+postboolrr(getr)r,rrrrs2     '