o ;iq@sddlmZddlmZddlmZddlmZGdddZGdd d eZ Gd d d eZ Gd d d eZ GdddeZ GdddeZ GdddeZGdddeZdZedeefZdedejfddZdS))Union)api_pb2)deprecation_warning) InvalidErrorc@s@eZdZUeed<eed<dedefddZdejfddZ dS) _GPUConfiggpu_typecountcCsJ|jj}|}|dkr|d|7}tdd|d|d||_||_dS)Nr:)iz`gpu=z (...)` is deprecated. Use `gpu="z "` instead.) __class____name__rrr )selfrr name str_valuer=/home/ubuntu/.local/lib/python3.10/site-packages/modal/gpu.py__init__s z_GPUConfig.__init__returncCstj|j|jdS)z?Convert this GPU config to an internal protobuf representation.rr )r GPUConfigrr rrrr _to_protosz_GPUConfig._to_protoN) r __module__ __qualname__str__annotations__intrrrrrrrrr s  rc2eZdZdZ ddeffdd ZddZZS) T4z [NVIDIA T4 Tensor Core](https://www.nvidia.com/en-us/data-center/tesla-t4/) GPU class. A low-cost data center GPU based on the Turing architecture, providing 16GB of GPU memory. rr ctd|dS)Nr superrrr r rrr&z T4.__init__cCd|jdS)NzGPU(T4, count=)r rrrr__repr__,z T4.__repr__rrrr__doc__rrr* __classcell__rrr%rr sr cr) L4z [NVIDIA L4 Tensor Core](https://www.nvidia.com/en-us/data-center/l4/) GPU class. A mid-tier data center GPU based on the Ada Lovelace architecture, providing 24GB of GPU memory. Includes RTX (ray tracing) support. rr cr!)Nr0r"r$r%rrr8r&z L4.__init__cCr')NzGPU(L4, count=r(r)rrrrr*>r+z L4.__repr__r,r-rrr%rr00s r0csBeZdZdZddddedeedfffddZd d ZZ S) A100z [NVIDIA A100 Tensor Core](https://www.nvidia.com/en-us/data-center/a100/) GPU class. The flagship data center GPU of the Ampere architecture. Available in 40GB and 80GB GPU memory configurations. rN)r sizer r2csH|dks|std|dS|dkrtd|dStd|d)N40GBz A100-40GB80GBz A100-80GBzsize='z@' is invalid. A100s can only have memory values of 40GB or 80GB.)r#r ValueError)rr r2r%rrrIs z A100.__init__cCsd|jd|jdS)NzGPU(z, count=r(rrrrrr*Vsz A100.__repr__) rrrr.rrrrr*r/rrr%rr1Bs   r1c4eZdZdZdddeffddZddZZS) A10Gau [NVIDIA A10G Tensor Core](https://www.nvidia.com/en-us/data-center/products/a10-gpu/) GPU class. A mid-tier data center GPU based on the Ampere architecture, providing 24 GB of memory. A10G GPUs deliver up to 3.3x better ML training performance, 3x better ML inference performance, and 3x better graphics performance, in comparison to NVIDIA T4 GPUs. rr)r cr!)Nr7r"r$r%rrrcz A10G.__init__cCr')NzGPU(A10G, count=r(r)rrrrr*lr+z A10G.__repr__r-rrr%rr7Z  r7cr6) H100a: [NVIDIA H100 Tensor Core](https://www.nvidia.com/en-us/data-center/h100/) GPU class. The flagship data center GPU of the Hopper architecture. Enhanced support for FP8 precision and a Transformer Engine that provides up to 4X faster training over the prior generation for GPT-3 (175B) models. rr)r cr!)Nr:r"r$r%rrryr8z H100.__init__cCr')NzGPU(H100, count=r(r)rrrrr*r+z H100.__repr__r-rrr%rr:pr9r:cr6) L40Sz [NVIDIA L40S](https://www.nvidia.com/en-us/data-center/l40s/) GPU class. The L40S is a data center GPU for the Ada Lovelace architecture. It has 48 GB of on-chip GDDR6 RAM and enhanced support for FP8 precision. rr)r cr!)Nr;r"r$r%rrrr8z L40S.__init__cCr')NzGPU(L40S, count=r(r)rrrrr*r+z L40S.__repr__r-rrr%rr;s  r;cr6) AnyzUSelects any one of the GPU classes available within Modal, according to availability.rr)r cr!)NANYr"r$r%rrrsz Any.__init__cCr')NzGPU(Any, count=r(r)rrrrr*r+z Any.__repr__r-rrr%rr<sr<a  **GPU configuration shortcodes** You can pass a wide range of `str` values for the `gpu` parameter of [`@app.function`](https://modal.com/docs/reference/modal.App#function). For instance: - `gpu="H100"` will attach 1 H100 GPU to each container - `gpu="L40S"` will attach 1 L40S GPU to each container - `gpu="T4:4"` will attach 4 T4 GPUs to each container You can see a list of Modal GPU options in the [GPU docs](https://modal.com/docs/guide/gpu). **Example** ```python @app.function(gpu="A100-80GB:4") def my_gpu_function(): ... # This will have 4 A100-80GB with each container ``` **Deprecation notes** An older deprecated way to configure GPU is also still supported, but will be removed in future versions of Modal. Examples: - `gpu=modal.gpu.H100()` will attach 1 H100 GPU to each container - `gpu=modal.gpu.T4(count=4)` will attach 4 T4 GPUs to each container - `gpu=modal.gpu.A100()` will attach 1 A100-40GB GPUs to each container - `gpu=modal.gpu.A100(size="80GB")` will attach 1 A100-80GB GPUs to each container NvaluercCst|tr |St|tr=d}d|vr2|dd\}}zt|}Wnty1td|dw|}t j ||dS|durEt Std|d)Nrr zInvalid GPU count: z. Value must be an integer.rzInvalid GPU config: zG. Value must be a string or `None` (or a deprecated `modal.gpu` object)) isinstancerrrsplitrr5rupperrr)r>r count_strrrrrparse_gpu_configs*     rC)typingr modal_protor_utils.deprecationr exceptionrrr r0r1r7r:r;r<r.rGPU_TrrCrrrrs     !