o Ơi @sddlZddlmZmZmZddlZddlZeeZ dddZ de de dej j d ej j def d d Zde de d ej j dee ej j ffd d Zde de d ej j deee ej j ffddZdS)N)ListOptionalTuplereturncCs<tjjjstjdkrtdtj ddSdSdS)Na_Detected this engine is being instantitated in a multi-GPU system with multi-device safe mode disabled. For more on the implications of this as well as workarounds, see the linked documentation (https://pytorch.org/TensorRT/user_guide/runtime.html#multi-device-safe-mode). The engine is set to be instantiated on the current default cuda device, cuda:zd. If this is incorrect, please set the desired cuda device via torch.cuda.set_device(...) and retry.) torch_tensorrtruntime_multi_device_safe_mode_PY_RT_MULTI_DEVICE_SAFE_MODEtorchcuda device_countloggerwarningcurrent_devicerrQ/home/ubuntu/.local/lib/python3.10/site-packages/torch_tensorrt/runtime/_utils.pymulti_gpu_device_check srcurr_device_idengine_device_idcurr_device_propertiesengine_device_propertiescCs|j|jf|j|jfkr"td|j|jfd|j|jfddS|j|jkr8td|jd|jddS||krJtd|d |ddSd S) zODetermines whether a device switch is required based on input device parameterszConfigured SM capability z2 does not match with current device SM capability z. Switching device context.TzProgram compiled for z9 but current CUDA device is current device SM capability z?. Attempting to switch device context for better compatibility.zConfigured Device ID: z& is different than current device ID: F)majorminorrrname)rrrrrrr_is_switch_requireds2    rcCs t|||}|dusJd|S)z@Wraps compatible device check and raises error if none are foundNzBCould not find a compatible device on the system to run TRT Engine)_get_most_compatible_device)rrrnew_target_device_optrrr_select_rt_deviceBs rc sddttjD}td||j|jffdd|D}td|t|dkr9|r7|dSdSd}|D]"}|\}}|j |j kr_||krR|}|S||krY|}q=|dur_|}q=|S) z6Selects a runtime device based on compatibility checkscSsg|] }|tj|fqSr)r r get_device_properties).0irrr Ysz/_get_most_compatible_device..zAll available devices: cs(g|]\}}|j|jfkr||fqSr)rr)r r!device_propertiestarget_device_smrrr"as zFound candidate devices: rrN) ranger r r rdebugrrlenr) rrr all_devicescandidate_devices best_match candidater!r#rr$rrSs2      r)rN)loggingtypingrrrr r getLogger__name__rrint_C_CudaDevicePropertiesboolrrrrrrrsF   (