o ;i&^ @sddlZddlmZmZddlmZddlmZddlmZm Z m Z ddl m Z ddl mZddlmZdd lmZdd lmZd d lmZd d lmZmZmZmZmZd dlmZd dlm Z m!Z!d dl"m#Z#m$Z$d dl%m&Z&m'Z'd dl(m)Z)d dl*m+Z+m,Z,d dl-m.Z.d dl/m0Z0d dl1m2Z2m3Z3m4Z4m5Z5m6Z6m7Z7GdddZ8e8Z9ddZ:ddde;defdd ZGd&d'd'Z?e$e?Z@Gd(dded)d*ZAe$eAZBdS)+N) AsyncIteratorMapping) dataclass)datetime)AnyOptionalUnion)Message) classproperty)asynccontextmanager)Retry)api_pb2) LoadContext) EPHEMERAL_OBJECT_HEARTBEAT_SLEEP_get_environment_name_Object live_methodlive_method_gen)Resolver) deserialize serialize) TaskContextsynchronize_api)deprecation_warningwarn_if_passing_namespace)check_object_name) as_timestamptimestamp_to_localized_dt)_Client)logger)AlreadyExistsErrorDeserializationErrorError InvalidError NotFoundErrorRequestSizeErrorc@seZdZdefddZdS)_NoDefaultSentinelreturncCsdS)Nz...selfr)r)>/home/ubuntu/.local/lib/python3.10/site-packages/modal/dict.py__repr__*sz_NoDefaultSentinel.__repr__N)__name__ __module__ __qualname__strr-r)r)r)r,r')sr'cCsdd|DS)NcSs&g|]\}}tjt|t|dqS))keyvalue)r DictEntryr).0kvr)r)r, 2s&z#_serialize_dict..)items)datar)r)r,_serialize_dict1sr;dict_Dictr:r(c Cs^zt||jWSty.}z|jrd|jdnd|j}td|d||d}~ww)NDict ''ephemeral Dict z!Failed to deserialize a key from : )r_clientr"name object_id)r<r:excdict_identifierr)r)r,_deserialize_dict_key5s rGr2c Cszzt||jWSty<}z)|turdnd|}|jr$d|jdnd|j}td|d|d||d}~ww) Nz for key r>r?r@zFailed to deserialize valuez from rA)rrBr" _NO_DEFAULTrCrD)r<r:r2rEkey_identifierrFr)r)r,_deserialize_dict_value=s rKc@s2eZdZUdZeeed<eed<eeed<dS)DictInfoz Information about a Dict object.rC created_at created_byN)r.r/r0__doc__rr1__annotations__rr)r)r)r,rLHs  rLc @seZdZdZedddddededeedeed df d d Z eddd dd dee dee e efdedeed e jdf ddZedddddededeedeefddZdS) _DictManagerz7Namespace with methods for managing named Dict objects.FN)allow_existingenvironment_nameclientrCrRrSrTr(cs~t|d|durtIdHn|}|rtjntj}tj|t||d}z |j |IdHWdSt y>|s;YdSw)aCreate a new Dict object. **Examples:** ```python notest modal.Dict.objects.create("my-dict") ``` Dicts will be created in the active environment, or another one can be specified: ```python notest modal.Dict.objects.create("my-dict", environment_name="dev") ``` By default, an error will be raised if the Dict already exists, but passing `allow_existing=True` will make the creation attempt a no-op in this case. ```python notest modal.Dict.objects.create("my-dict", allow_existing=True) ``` Note that this method does not return a local instance of the Dict. You can use `modal.Dict.from_name` to perform a lookup after creation. Added in v1.1.2. DictN)deployment_namerSobject_creation_type) rrfrom_envr &OBJECT_CREATION_TYPE_CREATE_IF_MISSING*OBJECT_CREATION_TYPE_CREATE_FAIL_IF_EXISTSDictGetOrCreateRequestrstubDictGetOrCreater!)rCrRrSrTrWreqr)r)r,createWs& # z_DictManager.createrH) max_objectscreated_beforerSrTr`rar=csdur tIdHndurdkrtdgdtdtffdd }|t|IdH} |r7n |d jjjIdH}q4fd d D}durX|dS|S) aReturn a list of hydrated Dict objects. **Examples:** ```python dicts = modal.Dict.objects.list() print([d.name for d in dicts]) ``` Dicts will be retreived from the active environment, or another one can be specified: ```python notest dev_dicts = modal.Dict.objects.list(environment_name="dev") ``` By default, all named Dict are returned, newest to oldest. It's also possible to limit the number of results and to filter by creation date: ```python dicts = modal.Dict.objects.list(max_objects=10, created_before="2025-01-01") ``` Added in v1.1.2. Nrzmax_objects cannot be negativerar(csdurdntdt}tj||d}tjt|d}j|IdH}|j t|j |kp?duo?tk}|S)Nd)r`ra)rS pagination) minlenr ListPaginationDictListRequestrr\DictListextenddicts)ra max_page_sizercr^respfinishedrTrSr9r`r)r, retrieve_pages "z(_DictManager.list..retrieve_pageTc s.g|]}tj|j|jdt|jdqS)Tis_another_apprep)r= _new_hydrateddict_idmetadata_reprrC)r5itemrTrSr)r,r8s z%_DictManager.list..) rrXr$floatboolrrv creation_inforM)r`rarSrTrormrjr)rnr,lists !   z_DictManager.list) allow_missingrSrTr~csbztj||d|IdH}Wn ty|sYdSwtj|jd}|jj |IdHdS)aDelete a named Dict. Warning: This deletes an *entire Dict*, not just a specific key. Deletion is irreversible and will affect any Apps currently using the Dict. **Examples:** ```python notest await modal.Dict.objects.delete("my-dict") ``` Dicts will be deleted from the active environment, or another one can be specified: ```python notest await modal.Dict.objects.delete("my-dict", environment_name="dev") ``` Added in v1.1.2. )rSNru) r= from_namehydrater%r DictDeleteRequestrDrBr\ DictDelete)rCr~rSrTobjr^r)r)r,deletes z_DictManager.delete)r.r/r0rO staticmethodr1r{rrr_intrrbuiltinsr}rr)r)r)r,rQTs\ 4 CrQc@seZdZUdZdZeeed<dZee j ed<ifddZ e de fdd Zedeefd d Zd eefd dZde j fddZeedddefdeddeedeedeedededf ddZe dHddddddedeedeededeeddf ddZe dHdedeeddfdd Z eddd!dedeedeefd"d#Z!e"de#fd$d%Z$e"dId&d'Z%e"dHd(e&d)ee&de&fd*d+Z'e"d(e&defd,d-Z(e"de)fd.d/Z*e"d(e&de&fd0d1Z+e"dHd2ee,ddfd3d4Z-e"dd5d(e&d6e&d7edefd8d9Z.e"d(e&d6e&ddfd:d;Z/e"e0fd(e&d)e&de&fdd?Z2e"d(e&defd@dAZ3e4dee&fdBdCZ5e4dee&fdDdEZ6e4dee7e&e&ffdFdGZ8dS)Jr=a Distributed dictionary for storage in Modal apps. Dict contents can be essentially any object so long as they can be serialized by `cloudpickle`. This includes other Modal objects. If writing and reading in different environments (eg., writing locally and reading remotely), it's necessary to have the library defining the data type installed, with compatible versions, on both sides. Additionally, cloudpickle serialization is not guaranteed to be deterministic, so it is generally recommended to use primitive types for keys. **Lifetime of a Dict and its items** An individual Dict entry will expire after 7 days of inactivity (no reads or writes). The Dict entries are written to durable storage. Legacy Dicts (created before 2025-05-20) will still have entries expire 30 days after being last added. Additionally, contents are stored in memory on the Modal server and could be lost due to unexpected server restarts. Eventually, these Dicts will be fully sunset. **Usage** ```python from modal import Dict my_dict = Dict.from_name("my-persisted_dict", create_if_missing=True) my_dict["some key"] = "some value" my_dict[123] = 456 assert my_dict["some key"] == "some value" assert my_dict[123] == 456 ``` The `Dict` class offers a few methods for operations that are usually accomplished in Python with operators, such as `Dict.put` and `Dict.contains`. The advantage of these methods is that they can be safely called in an asynchronous context by using the `.aio` suffix on the method, whereas their operator-based analogues will always run synchronously and block the event loop. For more examples, see the [guide](https://modal.com/docs/guide/dicts-and-queues#modal-dicts). N_name _metadatacCstd)z mdmd:hiddenz_`Dict(...)` constructor is not allowed. Please use `Dict.from_name` or `Dict.ephemeral` instead) RuntimeError)r+r:r)r)r,__init__&sz_Dict.__init__r(cCstSN)rQ)clsr)r)r,objects,sz _Dict.objectscCs|jSr)rr*r)r)r,rC0sz _Dict.namervcCs*|rt|tjs J||_|j|_dSdSr) isinstancer DictMetadatarrCr)r+rvr)r)r,_hydrate_metadata4s  z_Dict._hydrate_metadatacCs|jsJ|jSr)rr*r)r)r, _get_metadata:s z_Dict._get_metadatarr:rTrS_heartbeat_sleepc sdur tIdH|rtddt|dur|ni}tjtjt||djj t dddIdH}t 4IdH,}tj |j d|jfd d |d |j|j |jd d dVWdIdHdS1IdHsowYdS)aHCreates a new ephemeral Dict within a context manager: Usage: ```python from modal import Dict with Dict.ephemeral() as d: d["foo"] = "bar" ``` ```python notest async with Dict.ephemeral() as d: await d.put.aio("foo", "bar") ``` Nz_Passing data to `modal.Dict.ephemeral` is deprecated and will stop working in a future release.)rWrSr:g$@) total_timeout)retryrcs jSr)r\ DictHeartbeatr)rTrequestr)r,fs z!_Dict.ephemeral..)sleepTzmodal.Dict.ephemeral()rq)rrXrr;r r[OBJECT_CREATION_TYPE_EPHEMERALrr\r]r rDictHeartbeatRequestru infinite_looprtrv)rr:rTrSr serializedresponsetcr)rr, ephemeral>s4 .z_Dict.ephemeralF) namespacerScreate_if_missingrTrCrc srtdt|drtdddtdtdtdttffd d }t|}tj ||d d t||d d S)aPReference a named Dict, creating if necessary. This is a lazy method that defers hydrating the local object with metadata from Modal servers until the first time it is actually used. ```python d = modal.Dict.from_name("my-dict", create_if_missing=True) d[123] = 456 ``` rUzmodal.Dict.from_namerz_Passing data to `modal.Dict.from_name` is deprecated and will stop working in a future release.r+resolver load_contextexisting_object_idcsrtdurni}tj|jrtjnd|d}|jj|IdH}t d|j | |j |j|j dS)N)rVrSrWr:zCreated dict with id ) r;r r[rSrYrTr\r]r debugru_hydraterv)r+rrrrr^rrr:rCr)r,_loads z_Dict.from_name.._loadTrSrT)rrhydrate_lazilyrCload_context_overrides) rrrr=rrrr1rw _from_loader)rCr:rrSrrTrrsr)rr,ros"  &  z_Dict.from_nameruc sHdtdtdtdttffdd }dd}tj||d d t|d d S) aConstruct a Dict from an id and look up the Dict metadata. This is a lazy method that defers hydrating the local object with metadata from Modal servers until the first time it is actually used. The ID of a Dict object can be accessed using `.object_id`. **Example:** ```python notest @app.function() def my_worker(dict_id: str): d = modal.Dict.from_id(dict_id) d["key"] = "Hello from remote function!" with modal.Dict.ephemeral() as d: # Pass the dict ID to a remote function my_worker.remote(d.object_id) print(d["key"]) # "Hello from remote function!" ``` r+rrrcs:tjd}|jj|IdH}||j|j|jdS)Nr)r DictGetByIdRequestrTr\ DictGetByIdrrurv)r+rrrr^rrr)r,rs z_Dict.from_id.._loadz Dict.from_id()T)rT)rrrr)r=rrrr1r)rurTrrsr)rr,from_ids" z _Dict.from_idrycs(tddtjj|||dIdHdS)a>mdmd:hidden Delete a named Dict object. Warning: This deletes an *entire Dict*, not just a specific key. Deletion is irreversible and will affect any Apps currently using the Dict. DEPRECATED: This method is deprecated; we recommend using `modal.Dict.objects.delete` instead. )rrzZ`modal.Dict.delete` is deprecated; we recommend using `modal.Dict.objects.delete` instead.rN)rr=rr)rCrTrSr)r)r,rs z _Dict.deletecs0|}|j}t|jp dt|j|jpddS)z)Return information about the Dict object.N)rCrMrN)rr|rLrCrrMrN)r+rvr|r)r)r,infosz _Dict.infocs(tj|jd}|jj|IdHdS)zRemove all items from the Dict.rN)r DictClearRequestrDrBr\ DictClear)r+r^r)r)r,clearsz _Dict.clearr2defaultcsBtj|jt|d}|jj|IdH}|js|St||j |S)z_Get the value associated with a key. Returns `default` if key does not exist. rur2N) r DictGetRequestrDrrBr\DictGetfoundrKr3r+r2rr^rlr)r)r,gets z _Dict.getcs0tj|jt|d}|jj|IdH}|jS)zReturn if a key is present.rN)r DictContainsRequestrDrrBr\ DictContainsr)r+r2r^rlr)r)r,containssz_Dict.containscs*tj|jd}|jj|IdH}|jS)zvReturn the length of the Dict. Note: This is an expensive operation and will return at most 100,000. rN)r DictLenRequestrDrBr\DictLenrer+r^rlr)r)r,resz _Dict.lencs:t}|||IdH}||urt|d|j|S)zGet the value associated with a key. Note: this function will block the event loop when called in an async context. N not in dict )objectrKeyErrorrD)r+r2 NOT_FOUNDr3r)r)r, __getitem__s z_Dict.__getitem__otherc si}r|fddD|r||t|}tj|j|d}z |jj|IdHWdSt yL}z dt |vrFt d||d}~ww)z&Update the Dict with additional items.csi|]}||qSr)r))r5r6rr)r, #sz _Dict.update..)ruupdatesNstatus = '413'z Dict.update request is too large) updatekeysr;r DictUpdateRequestrDrBr\ DictUpdater#r1r&)r+rkwargscontentsrr^rEr)rr,rs    z _Dict.update)skip_if_existsr3rc st||i}t|}tj|j||d}z|jj|IdH}|jWSty9}z dt |vr3t d||d}~ww)zAdd a specific key-value pair to the Dict. Returns True if the key-value pair was added and False if it wasn't because the key already existed and `skip_if_exists` was set. )rur if_not_existsNrzDict.put request is too large) r;r rrDrBr\rcreatedr#r1r&) r+r2r3rrrr^rlrEr)r)r,put0s  z _Dict.putcs|||IdHS)zSet a specific key-value pair to the Dict. Note: this function will block the event loop when called in an async context. N)r)r+r2r3r)r)r, __setitem__Csz_Dict.__setitem__cs^tj|jt|d}|jj|IdH}|js(|tur|St |d|jt ||j |S)zRemove a key from the Dict, returning the value if it exists. If key is not found, return default if provided, otherwise raise KeyError. rNr) r DictPopRequestrDrrBr\DictPoprrIrrKr3rr)r)r,popKsz _Dict.popc||IdHS)z|Delete a key from the Dict. Note: this function will block the event loop when called in an async context. N)rr+r2r)r)r, __delitem__Yz_Dict.__delitem__cr)z|Return if a key is present. Note: this function will block the event loop when called in an async context. N)rrr)r)r, __contains__arz_Dict.__contains__cCsDtj|jdd}|jjj|2z 3dHW}t||jVq6dS)zReturn an iterator over the keys in this Dict. Note that (unlike with Python dicts) the return value is a simple iterator, and results are unordered. T)rurN) r DictContentsRequestrDrBr\ DictContents unary_streamrGr2rr)r)r,ris z _Dict.keysc Csntj|jdd}|jjj|2z"3dHW}zt||j}Wn t y*t }Ynwt ||j |Vq6dS)zReturn an iterator over the values in this Dict. Note that (unlike with Python dicts) the return value is a simple iterator, and results are unordered. T)ruvaluesN) r rrDrBr\rrrGr2r"rIrKr3)r+r^rl key_deserr)r)r,rts z _Dict.valuescCs\tj|jddd}|jjj|2z3dHW}t||j}t ||j |}||fVq6dS)zReturn an iterator over the (key, value) tuples in this Dict. Note that (unlike with Python dicts) the return value is a simple iterator, and results are unordered. T)rurrN) r rrDrBr\rrrGr2rKr3)r+r^rlr value_deserr)r)r,r9s  z _Dict.itemsr)r(N)9r.r/r0rOrrr1rPrr rrr rQrpropertyrCr rr classmethodr rtyper<rrzrrrr{rrrrrLrrrrrrrerrrrrrIrrrrrrtupler9r)r)r)r,r=s ) / 3 )     "   di) type_prefix)Crcollections.abcrr dataclassesrrtypingrrrgoogle.protobuf.messager synchronicityr synchronicity.async_wrapr modal._utils.grpc_utilsr modal_protor _load_contextr_objectrrrrr _resolverr_serializationrr_utils.async_utilsrr_utils.deprecationrr_utils.name_utilsr_utils.time_utilsrrrTrconfigr exceptionr!r"r#r$r%r&r'rIr;bytesrGrKrLrQ DictManagerr=rUr)r)r)r,sH                #