o rril @sddlmZddlZddlZddlZddlmZddlZddlm Z ddl Z ddl Z ddl mZddlmZddlmZeeZd d Zedd ededefddZejdfd edefddZdede jfddZd efddZdS))contextmanagerN)Path)rmtree) BaseContainer) OmegaConf)fatalcCsddl}t|trdd|DSt|ttfr dd|DSt||jr.| St|t r7t |S|dusDt|t t t tfrF|St|trPt|Stt|d)NrcSsi|] \}}|t|qSjsonable).0kvr r >/home/ubuntu/.local/lib/python3.10/site-packages/dora/utils.py szjsonable..cSsg|]}t|qSr r )r rr r r szjsonable..z is not jsonable.)torch isinstancedictitemslisttupleTensordetachcputolistrstrintfloatboolrr to_container ValueErrorrepr)valuerr r rr s     r wb.tmppathmodesuffixccsNt||}t|| }|VWdn1swYt||dS)a? Write to a temporary file with the given suffix, then rename it to the right filename. As renaming a file is usually much faster than writing it, this removes (or highly limits as far as I understand NFS) the likelihood of leaving a half-written checkpoint behind, if killed at the wrong time. N)ropenosrename)r&r'r(tmp_pathfr r rwrite_and_rename-s  r.rbc CsPz|t||WSttjttfy'}ztd||WYd}~dSd}~ww)zt Try to load from a path using torch.load, and handles various failure cases. Return None upon failure. zLAn error happened when trying to load from %s, this file will be ignored: %rN)r)OSErrorpickleUnpicklingError RuntimeErrorEOFErrorloggerwarning)r&loadr'excr r rtry_load<sr9 module_namereturncCsBzt|WSty tjd|ddtd|dYdSw)NzCould not import module %sT)exc_infozFailed to import module .) importlib import_module ImportErrorr5infor)r:r r rimport_or_fatalKs   rBcsxdfdd}t||dr4|sJtj|j|jddd}td|d |||d S|r:Jd S) zReliably delete the given folder, trying to remove while ignoring errors, and if any files remain, renaming to some trash folder.Fcsdtd|dS)NTzError deleting file )r5r6)func error_pathr<errorr r _on_errorXsz"reliable_rmtree.._on_error)onerror__trash)dirprefixr(z Deletion of z failed, moving to N) rexiststempfilemkdtempparentnamer5r6r+)r&rG target_namer rErreliable_rmtreeSs   rS)r$r%) contextlibrr>loggingr*pathlibrr1shutilrrNtypingtpomegaconf.basecontainerr omegaconfrlogr getLogger__name__r5r rr.r7r9AnyrBrSr r r rs&