o i @sdddlZddlZddlmZmZmZddlmZGdddeZ GdddeZ Gdd d eZ dS) N)MultiprocessIteratorSerialIteratorShuffleOrderSampler) Extensionc@s eZdZdZddZddZdS)ShufflingEnablerz.An extension enabling shuffling on an IteratorcCsd|_||_dS)zrInits the ShufflingEnabler :param list[Iterator] iterators: The iterators to enable shuffling on FN)set iterators)selfrr S/home/ubuntu/.local/lib/python3.10/site-packages/espnet/utils/training/iterators.py__init__ s zShufflingEnabler.__init__cCs(|js|jD]}|qd|_dSdS)zVCalls the enabler on the given iterator :param trainer: The iterator TN)rr start_shuffle)r traineriteratorr r r __call__s    zShufflingEnabler.__call__N)__name__ __module__ __qualname____doc__r rr r r r rs rcs*eZdZdZdfdd ZddZZS)!ToggleableShufflingSerialIteratorzHA SerialIterator having its shuffling property activated during trainingTcstt|||||dS)aInit the Iterator :param torch.nn.Tensor dataset: The dataset to take batches from :param int batch_size: The batch size :param bool repeat: Whether to repeat data (allow multiple epochs) :param bool shuffle: Whether to shuffle the batches N)superrr )r dataset batch_sizerepeatshuffle __class__r r r s z*ToggleableShufflingSerialIterator.__init__cCsXd|_ttjjddkrtjt|j |_ dSt |_ | t t|j d|_ dSz,Starts shuffling (or reshuffles) the batchesTrN)_shuffleintchainer_version __version__nprandom permutationlenr_orderr order_sampleraranger r r r r ,s z/ToggleableShufflingSerialIterator.start_shuffle)TTrrrrr r __classcell__r r rr rs rcs6eZdZdZ      d fdd Zdd ZZS) 'ToggleableShufflingMultiprocessIteratorzNA MultiprocessIterator having its shuffling property activated during trainingTNc s$tt|j||||||||ddS)a:Init the iterator :param torch.nn.Tensor dataset: The dataset to take batches from :param int batch_size: The batch size :param bool repeat: Whether to repeat batches or not (enables multiple epochs) :param bool shuffle: Whether to shuffle the order of the batches :param int n_processes: How many processes to use :param int n_prefetch: The number of prefetch to use :param int shared_mem: How many memory to share between processes :param int maxtasksperchild: Maximum number of tasks per child )rrrr n_processes n_prefetch shared_memmaxtasksperchildN)rr.r ) r rrrrr1r2r3r4rr r r 9s  z0ToggleableShufflingMultiprocessIterator.__init__cCs^d|_ttjjddkrtjt|j |_ nt |_ | t t|j d|_ |dSr)rr r!r"r#r$r%r&r'rr(rr)r*_set_prefetch_stater+r r r r Zs  z5ToggleableShufflingMultiprocessIterator.start_shuffle)TTNr/Nr0r,r r rr r.6s!r.) r!numpyr$chainer.iteratorsrrrchainer.training.extensionrrrr.r r r r s