o ;iq@sddlZddlZddlZddlmZmZddlmZmZddlm Z m Z m Z m Z ddl Z ddlmZddlmZddlmZdd lmZdd lmZdd lmZdd lmZd ZdZejraddlZGdddej Z!eGdddZ"e #dZ$e j%dddZ&e j%dddZ'e e ge fe e ge ffZ(e e ge fZ)d[de"de*ddfddZ+Gdddej,e$e&e'fZ-d e.e d!e/de0e1e-ffd"d#Z2d$e d!e/de0e1e d%e fffd&d'Z3Gd(d)d)Z4 d\dd*d+e e*de4fd,d-Z5d\d.e ee1de6ej7fd/d0Z8 d\d1ddddd2d3e1d4e e1d.e ee1d5e*d6e*de e e-e$e&e&fe e$e&ffge-e$e&e&fff d7d8Z9 d\d1ddddd9d3e1d4e e1d5e*d.e ee1d6e*de e e-e$e&e&fe e$e&ffge-e$e&e&fff d:d;Z: d\dddd<d4e e1d.e ee1d6e*de e e-e(fge-ffd=d>Z; d\dddd<d4e e1d.e ee1d6e*de e e-e(fge-ffd?d@Z d\ddGdHe*de e e-e)fge-ffdIdJZ?d\de e)ge-ffdKdLZ@ d\dMe/dNe/de e e-e$e&e&fe e$e&ffge-e$e&e&fffdOdPZA d\dddQdRe e/dSe e/de e e e$e&fe-e$e&e&ffge-e$e&e&fffdTdUZB d]dVe/dWe*dXe*de e e e$e&fe-e$e&e&ffge-e$e&e&fffdYdZZCdS)^N) CoroutineIterable)asdict dataclass)AnyCallableOptionalUnion)api_pb2) _Function) synchronizer)deprecation_warning)callable_has_non_self_params)config) InvalidErrorii' c@sfeZdZdZdZdZdZdZdZdZ dZ d Z e d e fd d Ze d e fd dZe d e fddZdS)_PartialFunctionFlags @ireturncCs tdS)Nr)rrrK/home/ubuntu/.local/lib/python3.10/site-packages/modal/_partial_function.pyall3s z_PartialFunctionFlags.allcCstjtjBtjBSN)rENTER_PRE_SNAPSHOTENTER_POST_SNAPSHOTEXITrrrrlifecycle_flags7s z%_PartialFunctionFlags.lifecycle_flagscCs tjtjBSr)rCALLABLE_INTERFACE WEB_INTERFACErrrrinterface_flags?s z%_PartialFunctionFlags.interface_flagsN)__name__ __module__ __qualname__r r!r"r$r%BATCHED CONCURRENT CLUSTEREDHTTP_WEB_INTERFACE staticmethodintrr#r&rrrrr s rc@seZdZUdZeejed<dZee ed<dZ ee ed<dZ ee ed<dZ ee ed<dZee ed<dZee ed<dZee ed <dZee ed <dZee ed <dZeejed <dddZdS)_PartialFunctionParamsNwebhook_config is_generator force_buildbatch_max_size batch_wait_ms cluster_sizemax_concurrent_inputstarget_concurrent_inputs build_timeoutrdma http_configotherrcCsNt|D]\}}|dur$t||ddurtd|dt|||qdS)z%Update self with params set in other.Nz Cannot set `z` twice.)ritemsgetattrrsetattr)selfr<keyvalrrrupdateRs z_PartialFunctionParams.update)r<r0rN)r'r(r)r1rr WebhookConfig__annotations__r2boolr3r4r/r5r6r7r8r9r:r; HTTPConfigrCrrrrr0Ds r0P ReturnTypeT) covariantOriginalReturnTypeFparamsis_flashrcCs@dtddfdd}dtddfdd}|r||dS||dS)NrLrcSs(|jdur td|jdurtddS)Nz@modal.concurrent(max_inputs=...) is not yet supported for Flash functions. Use `@modal.concurrent(target_inputs=...)` instead.zA`@modal.concurrent()` missing required argument: `target_inputs`.)r7 TypeErrorr8rLrrr-_verify_concurrent_params_with_flash_settingscs  zOverify_concurrent_params.._verify_concurrent_params_with_flash_settingscSs|jdur tddS)Nz>`@modal.concurrent()` missing required argument: `max_inputs`.)r7rNrOrrr_verify_concurrent_paramsls z;verify_concurrent_params.._verify_concurrent_params)r0)rLrMrPrQrrrverify_concurrent_paramsbs    rRc @seZdZUdZeeeefed<dZ ee ed<e ed<e ed<de eeefe fde de fdd Zde de d ejfd d Zdd dZ ddededed dfddZd eeeffddZd efddZdd eeeeffddZdS)_PartialFunctionz}Object produced by a decorator in the `modal` namespace The object will eventually by consumed by an App decorator. raw_fNuser_clsflagsrLobjcCs<t|tr ||_d|_n||_d|_||_||_|dSr) isinstancetyperUrTrVrLvalidate_flag_composition)r@rWrVrLrrr__init__s  z_PartialFunction.__init__rcCs&|j|O_|j|||S)zBImplement decorator composition by combining the flags and params.)rVrLrCrZ)r@rVrLrrrstacks z_PartialFunction.stackcCs\|jt@}|jt@}|r|rtd|jtj@}|jtj@}|r*|r,tddSdS)z=Validate decorator composition based on PartialFunctionFlags.zBInterface decorators cannot be combined with lifecycle decorators.zECallable decorators cannot be combined with web interface decorators.N)rVrr&r#rr%r$)r@uses_interface_flagsuses_lifecycle_flagshas_web_interfacehas_callable_interfacerrrrZs  z*_PartialFunction.validate_flag_compositionFdecorator_name require_syncrequire_nullarycCsddlm}|jt@}|jt@}|jdur%|s|r%td|d|jp*|j}t |t r8td|dt ||rEtd|d|jdurvt |jsWtd |d |rgt |jrgtd |d |rxt|jrztd |ddSdSdS)zZEnforce compatibility with the wrapped object; called from individual decorator functions.r )_ClsNzCannot apply `@modal.z:` to a class. Hint: consider applying to a method instead.zCannot stack `@modal.zD` on top of `@app.function`. Hint: swap the order of the decorators.zA` on top of `@app.cls()`. Hint: swap the order of the decorators.zThe object wrapped by `@modal.z` must be callable.z The `@modal.z2` decorator can't be applied to an async function.zFunctions decorated by `@modal.z` can't have parameters.)clsrdrVrr#r&rUrrTrXr callableinspectiscoroutinefunctionr)r@rarbrcrdr^r]wrapped_objectrrrvalidate_obj_compatibilitys2         z+_PartialFunction.validate_obj_compatibilitycCs|jdusJ|jSr)rTr@rrr _get_raw_fsz_PartialFunction._get_raw_fcCs |jjdurdS|jjjtjkS)NF)rLr1rYr WEBHOOK_TYPE_UNSPECIFIEDrkrrr_is_web_endpoints z!_PartialFunction._is_web_endpointcCsD|jdusJ|jj}|r t|drt|d|S|j||S|S)N_modal_functions)rTr'hasattrr>__get__)r@rWobjtypekrrrrqs z_PartialFunction.__get__)rN)FFr)r'r(r)__doc__rrrHrIrErUrYrr0r r[typing_extensionsSelfr\rZstrrFrjrlrnr rKrqrrrrrSvs8     $rSrUrVcCspddlm}i}t|D]'}|tur5|jD]\}}t||r4t t t |}|j |@r4|||<qq|S)zPGrabs all method on a user class, and returns partials. Includes legacy methods.r )PartialFunction)partial_functionrxreversedmroobject__dict__r=rXtypingcastrSr _translate_inrV)rUrVrxpartial_functions parent_clsrsv_partial_functionrrr"_find_partial_methods_for_user_clss   ruser_obj.cs$t}fddt||DS)zs   z+_find_callables_for_obj..)rYrr=)rrVrUrrr_find_callables_for_objs  rc@s\eZdZej    dddZej    dd dZej   dd dZd dZdS)_MethodDecoratorTypefuncmmodal.partial_function.PartialFunction[typing_extensions.Concatenate[Any, P], ReturnType, OriginalReturnType]rImodal.partial_function.PartialFunction[P, ReturnType, OriginalReturnType]cCdSrrr@rrrr__call__ sz_MethodDecoratorType.__call__PCallable[typing_extensions.Concatenate[Any, P], Coroutine[Any, Any, ReturnType]]Vmodal.partial_function.PartialFunction[P, ReturnType, Coroutine[Any, Any, ReturnType]]cCrrrrrrrr;Callable[typing_extensions.Concatenate[Any, P], ReturnType]Amodal.partial_function.PartialFunction[P, ReturnType, ReturnType]cCrrrrrrrrrcCrrrrrrrrsN)rrrr)rrrr)rrrr)r'r(r)r~overloadrrrrrrs(   rr2r2cs:|durtddttdtftfdtffdd }|S)zDecorator for methods that should be transformed into a Modal Function registered against this class's App. **Usage:** ```python @app.cls(cpu=8) class MyCls: @modal.method() def f(self): ... ``` Nz`Positional arguments are not allowed. Did you forget parentheses? Suggestion: `@modal.method()`.rW.rcsrtj}durt|tr|jn|}t|pt|td}t|tr,| ||}nt|||}| d|S)Nrmethod) rr$rXrSrTrgisgeneratorfunctionisasyncgenfunctionr0r\rj)rWrVrfrLrrrrwrapper6s    z_method..wrapper)rr rrrS)_warn_parentheses_missingr2rrrr_methods &rcustom_domainscCs>t|tr Jdg}|dur|D] }|tj|dq|S)Nz.wrapper)rXrwrr rDWEBHOOK_TYPE_FUNCTIONrgetWEBHOOK_ASYNC_MODE_AUTOrrr%r0r rSrHrIr)rrrrrrr1rrrr_fastapi_endpointTs4     r)rrrrrc st|tr td|durtdtddtjtj|||pdtdtj t ||d}t j t |d d tttttftttffd ttttfffd d }|S)aRegister a basic web endpoint with this application. DEPRECATED: This decorator has been renamed to `@modal.fastapi_endpoint`. This is the simple way to create a web endpoint on Modal. The function behaves as a [FastAPI](https://fastapi.tiangolo.com/) handler and should return a response object to the caller. Endpoints created with `@modal.web_endpoint` are meant to be simple, single request handlers and automatically have [CORS](https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS) enabled. For more flexibility, use `@modal.asgi_app`. To learn how to use Modal with popular web frameworks, see the [guide on web endpoints](https://modal.com/docs/guide/webhooks). zVPositional arguments are not allowed. Suggestion: `@modal.web_endpoint(method="GET")`.NzfPositional arguments are not allowed. Did you forget parentheses? Suggestion: `@modal.web_endpoint()`.)izRThe `@modal.web_endpoint` decorator has been renamed to `@modal.fastapi_endpoint`.rrrrrWrcr)N web_endpointrrrrrrrz_web_endpoint..wrapper)rXrwrrr rDrrrrrrr%r0r rSrHrIr)rrrrrrr1rrrr _web_endpoints6    r)rrrct|tr td|d|durtdtjtj|pdtdtjt ||d}t j t |dd t ttfd tffd d }|S) a~Decorator for registering an ASGI app with a Modal function. Asynchronous Server Gateway Interface (ASGI) is a standard for Python synchronous and asynchronous apps, supported by all popular Python web libraries. This is an advanced decorator that gives full flexibility in defining one or more web endpoints on Modal. **Usage:** ```python from typing import Callable @app.function() @modal.asgi_app() def create_asgi() -> Callable: ... ``` To learn how to use Modal with popular web frameworks, see the [guide on web endpoints](https://modal.com/docs/guide/webhooks). zJPositional arguments are not allowed. Suggestion: `@modal.asgi_app(label="rNzbPositional arguments are not allowed. Did you forget parentheses? Suggestion: `@modal.asgi_app()`.rrrYrrrrrrrWrc8t|tr |}nt|}|jdddd|S)Nasgi_appTrbrcrrrrrr  z_asgi_app..wrapper)rXrwrr rDWEBHOOK_TYPE_ASGI_APPrrrrrr%r0r rSNullaryFuncOrMethodrrrrr1rrrr _asgi_app$   rcr) aDecorator for registering a WSGI app with a Modal function. Web Server Gateway Interface (WSGI) is a standard for synchronous Python web apps. It has been [succeeded by the ASGI interface](https://asgi.readthedocs.io/en/latest/introduction.html#wsgi-compatibility) which is compatible with ASGI and supports additional functionality such as web sockets. Modal supports ASGI via [`asgi_app`](https://modal.com/docs/reference/modal.asgi_app). **Usage:** ```python from typing import Callable @app.function() @modal.wsgi_app() def create_wsgi() -> Callable: ... ``` To learn how to use this decorator with popular web frameworks, see the [guide on web endpoints](https://modal.com/docs/guide/webhooks). zJPositional arguments are not allowed. Suggestion: `@modal.wsgi_app(label="rNzbPositional arguments are not allowed. Did you forget parentheses? Suggestion: `@modal.wsgi_app()`.rrrrrWrcr)Nwsgi_appTrrrrrrrBrz_wsgi_app..wrapper)rXrwrr rDWEBHOOK_TYPE_WSGI_APPrrrrrr%r0r rSrrrrr _wsgi_apprrg@)startup_timeoutrrrportrc st|tr |dks |dkrtd|dkrtdtjtj|p dtdtjt ||||d}t j t |d d t ttfd tffd d }|S)aBDecorator that registers an HTTP web server inside the container. This is similar to `@asgi_app` and `@wsgi_app`, but it allows you to expose a full HTTP server listening on a container port. This is useful for servers written in other languages like Rust, as well as integrating with non-ASGI frameworks like aiohttp and Tornado. **Usage:** ```python import subprocess @app.function() @modal.web_server(8000) def my_file_server(): subprocess.Popen("python -m http.server -d / 8000", shell=True) ``` The above example starts a simple file server, displaying the contents of the root directory. Here, requests to the web endpoint will go to external port 8000 on the container. The `http.server` module is included with Python, but you could run anything here. Internally, the web server is transparently converted into a web endpoint by Modal, so it has the same serverless autoscaling behavior as other web endpoints. For more info, see the [guide on web endpoints](https://modal.com/docs/guide/webhooks). r izRFirst argument of `@web_server` must be a local port, such as `@web_server(8000)`.rzAThe `startup_timeout` argument of `@web_server` must be positive.rr)rYrrrrweb_server_portweb_server_startup_timeoutrrrWrcr)N web_serverTrrrrrrrrz_web_server..wrapper)rXr/rr rDWEBHOOK_TYPE_WEB_SERVERrrrrrr%r0r rSr)rrrrrr1rrrr _web_serverMs$"  r)snaprcsJ|durtd|r tjntjtdtttfdtffdd }|S)zDecorator for methods which should be executed when a new container is started. See the [lifeycle function guide](https://modal.com/docs/guide/lifecycle-functions#enter) for more information.Nz_Positional arguments are not allowed. Did you forget parentheses? Suggestion: `@modal.enter()`.rWrcr)Nenterrrrrrrs   z_enter..wrapper)rrr r!r0r rS NullaryMethod)rrrrrr_enters  rcs@|durtdtjtdtttfdtffdd }|S)zDecorator for methods which should be executed when a container is about to exit. See the [lifeycle function guide](https://modal.com/docs/guide/lifecycle-functions#exit) for more information.Nz^Positional arguments are not allowed. Did you forget parentheses? Suggestion: `@modal.exit()`.rWrcr)Nexitrrrrrrs   z_exit..wrapper)rrr"r0r rSr)rrrrr_exits rmax_batch_sizewait_mscs|durtd|dkrtd|tkrtdtd|dkr$td|tkr0td tdtjtjBt||d d ttt t t ft t t ffd tt t t fffd d }|S)a|Decorator for functions or class methods that should be batched. **Usage** ```python # Stack the decorator under `@app.function()` to enable dynamic batching @app.function() @modal.batched(max_batch_size=4, wait_ms=1000) async def batched_multiply(xs: list[int], ys: list[int]) -> list[int]: return [x * y for x, y in zip(xs, ys)] # call batched_multiply with individual inputs # batched_multiply.remote.aio(2, 100) # With `@app.cls()`, apply the decorator to a method (this may change in the future) @app.cls() class BatchedClass: @modal.batched(max_batch_size=4, wait_ms=1000) def batched_multiply(self, xs: list[int], ys: list[int]) -> list[int]: return [x * y for x, y in zip(xs, ys)] ``` See the [dynamic batching guide](https://modal.com/docs/guide/dynamic-batching) for more information. NzaPositional arguments are not allowed. Did you forget parentheses? Suggestion: `@modal.batched()`.r z*max_batch_size must be a positive integer.z&max_batch_size cannot be greater than .rz'wait_ms must be a non-negative integer.zwait_ms cannot be greater than )r4r5rWrcr)Nbatchedrrrrrrrz_batched..wrapper) rMAX_MAX_BATCH_SIZEMAX_BATCH_WAIT_MSrr$r*r0r rSrHrIr)rrrrrrr_batcheds(!    r) max_inputs target_inputsrrcs|durtd|dur|dur||krtdtjt||ddtttttftttffdttttfffdd }|S) aDecorator that allows individual containers to handle multiple inputs concurrently. The concurrency mechanism depends on whether the function is async or not: - Async functions will run inputs on a single thread as asyncio tasks. - Synchronous functions will use multi-threading. The code must be thread-safe. Input concurrency will be most useful for workflows that are IO-bound (e.g., making network requests) or when running an inference server that supports dynamic batching. When `target_inputs` is set, Modal's autoscaler will try to provision resources such that each container is running that many inputs concurrently, rather than autoscaling based on `max_inputs`. Containers may burst up to up to `max_inputs` if resources are insufficient to remain at the target concurrency, e.g. when the arrival rate of inputs increases. This can trade-off a small increase in average latency to avoid larger tail latencies from input queuing. **Examples:** ```python # Stack the decorator under `@app.function()` to enable input concurrency @app.function() @modal.concurrent(max_inputs=100) async def f(data): # Async function; will be scheduled as asyncio task ... # With `@app.cls()`, apply the decorator at the class level, not on individual methods @app.cls() @modal.concurrent(max_inputs=100, target_inputs=80) class C: @modal.method() def f(self, data): # Sync function; must be thread-safe ... ``` *Added in v0.73.148:* This decorator replaces the `allow_concurrent_inputs` parameter in `@app.function()` and `@app.cls()`. NzdPositional arguments are not allowed. Did you forget parentheses? Suggestion: `@modal.concurrent()`.z>`target_inputs` parameter cannot be greater than `max_inputs`.)r7r8rWrcr)N concurrentrrrrrr?rz_concurrent..wrapper) rrr+r0r rSrHrIr)rrrrrrr _concurrents2   rsize broadcastr:csn|sJd|dkrtdtjt||ddtttttftttffdttttfffdd }|S) aProvision clusters of colocated and networked containers for the Function. Parameters: size: int Number of containers spun up to handle each input. broadcast: bool = True If True, inputs will be sent simultaneously to each container. Otherwise, inputs will be sent only to the rank-0 container, which is responsible for delegating to the workers. z-broadcast=False has not been implemented yet!rz#cluster size must be greater than 0)r6r:rWrcr)N clusteredrrrrrrfrz_clustered..wrapper) ValueErrorrr,r0r rSrHrIr)rrr:rrrr _clusteredMs    r)Fr)TF)Denumrgr~collections.abcrr dataclassesrrrrrr ru modal_protor _functionsr _utils.async_utilsr _utils.deprecationr_utils.function_utilsrr exceptionrrr TYPE_CHECKINGmodal.partial_functionmodalIntFlagrr0 ParamSpecrHTypeVarrIrKrrrFrRGenericrSrYr/dictrwrrrrlistrrrrrrfloatrrrrrrrrrrsr       $ "x&  "-    A    F  ;  =  A   ?  P