o }oi @sBddlmZddlZddZddZd deejdefd d ZdS) )IterableNcCstjo tjdkS)N)torch distributedis_initializedget_world_sizerr`/home/ubuntu/.local/lib/python3.10/site-packages/nemo/collections/tts/parts/utils/distributed.py_is_distributed*sr cCst|p t|S)N)ris_floating_point is_complex)tensorrrr _is_complex_or_float.srtensorssrccCsZtsdSdd|D}g}|D]}tjj|j|dd}||q|D]}|q$dS)z Broadcast the tensors from the given parameters to all workers. This can be used to ensure that all workers have the same model to start with. NcSsg|]}t|r|qSr)r).0r rrr :sz%broadcast_tensors..T)rasync_op)r rr broadcastdataappendwait)rrhandlesr handlerrr broadcast_tensors2s  r)r)typingrrr rTensorintrrrrr s $