o Tim@sdddlmZddlmZddlmZddlmZGdddeZGdd d eZ Gd d d eZ d S) )Field)Optional)DeepSpeedConfigModel)DSStateManagerConfigc@seZdZUdZdZeed<dS)DeepSpeedTPConfig' Configure tensor parallelism settings rtp_sizeN)__name__ __module__ __qualname____doc__r int__annotations__rrT/home/ubuntu/.local/lib/python3.10/site-packages/deepspeed/inference/v2/config_v2.pyr s  rc@s"eZdZUdZdZeeed<dS)QuantizationConfigrNquantization_mode)r r r r rrstrrrrrrrs rc@sJeZdZUdZeiddZeed< eiddZe ed< iZ e ed<dS) RaggedInferenceEngineConfigz1 Sets parameters for DeepSpeed Inference Engine. tp)aliastensor_parallelmanager state_manager quantizationN) r r r r rrrrrrrrrrrrrs rN) pydanticrtypingrdeepspeed.runtime.config_utilsrraggedrrrrrrrrs