o i!@sUddlZddlZddlmZddlmZmZmZmZm Z m Z ddl m Z ddl mZmZddlmZmZmZmZmZdd lmZmZdd lmZdd lmZdd lmZdd lm Z ddl!m"Z"edddZ#eee"e$d<edidZ%ee&e$d<ddiZ'de(ddfddZ)d1ddZ*d1ddZ+ddZ,de(de"fddZ-d d!Z.ej/de(fd"d#Z0de"fd$d%Z1de"ddfd&d'Z2de3fd(d)Z4d*d+Z5ej6fd,ee(efd-e ej6fd.d/Z7gd0Z8dS)2N) ContextVar)AnyCallableDictOptionalTypecast)registry)cupyhas_cupy)assert_pytorch_installedassert_tensorflow_installedget_torch_default_device is_cupy_array require_cpu)cupy_pytorch_allocatorcupy_tensorflow_allocator) ParamServer)CupyOps)MPSOps)NumpyOps)Ops context_ops)default context_poolsops allocatorreturncCs4|dkr tdS|dkrtdStd|d)zRoute GPU memory allocation via PyTorch or tensorflow. Raise an error if the given argument does not match either of the two. pytorch tensorflowz#Invalid 'gpu_allocator' argument: 'z4'. Available allocators are: 'pytorch', 'tensorflow'N)use_pytorch_for_gpu_memoryuse_tensorflow_for_gpu_memory ValueErrorrr&K/home/ubuntu/.local/lib/python3.10/site-packages/thinc/backends/__init__.pyset_gpu_allocators   r(cCsNttjdkr dSt}d|vrtjjtd|d<tj |dj dS)atRoute GPU memory allocation via PyTorch. This is recommended for using PyTorch and cupy together, as otherwise OOM errors can occur when there's available memory sitting in the other library's pool. We'd like to support routing Tensorflow memory allocation via PyTorch as well (or vice versa), but do not currently have an implementation for it. cudaNr r%) r rtypergetr r) MemoryPoolr set_allocatormallocpoolsr&r&r'r",s r"cCs>tt}d|vrtjjtd|d<tj|djdS)azRoute GPU memory allocation via TensorFlow. This is recommended for using TensorFlow and cupy together, as otherwise OOM errors can occur when there's available memory sitting in the other library's pool. We'd like to support routing PyTorch memory allocation via Tensorflow as well (or vice versa), but do not currently have an implementation for it. r!r%N) rrr+r r)r,rr-r.r/r&r&r'r#As  r#cCsLzddlm}Wn tyYnwz ddlm}WdSty%YdSw)Nr)AppleOps) BigEndianOps)thinc_apple_opsr1 ImportErrorthinc_bigendian_opsr2)r1r2r&r&r'_import_extra_cpu_backendsRs  r6namecKs|ddtjD}d}|dkr't|d}|d|}|d|}n||}|dur7td||d i|S) zYGet a backend object. The special name "cpu" returns the best available CPU backend.cSsi|]}|j|qSr&)r7).0ops_clsr&r&r' bszget_ops..Ncpunumpyapple bigendianzInvalid backend: r&)r rget_allvaluesr6r+r$)r7kwargs ops_by_nameclsr&r&r'get_ops]s   rDcCst|rtStS)z4Return CupyOps for a cupy array, NumpyOps otherwise.)rrr)arrr&r&r' get_array_opsssrFcks<t}tt|fi|z dVWt|dSt|w)zsF