o 2wi@sddlmZddlmZddlmZddlmZddlm Z m Z m Z m Z m Z ddlmZmZddlmZmZmZddlmZmZdd lmZmZe d eeeeeZe d eeeeZd e ee efd efddZdedede d efddZde ed e efddZ dS))ProcessPoolExecutor)reduce)chain)add)CallableIterableOptionalTypeVarUnion) Recording RecordingSet)CutSetMixedCutMonoCut)Features FeatureSet)SupervisionSegmentSupervisionSet ManifestItemManifest manifestsreturncGs,t|dkrt|ttfr|d}tt|S)a( Combine multiple manifests of the same type into one. Examples: >>> # Pass several arguments >>> combine(recording_set1, recording_set2, recording_set3) >>> # Or pass a single list/tuple of manifests >>> combine([supervision_set1, supervision_set2]) r)len isinstancetuplelistrr)rrP/home/ubuntu/sommelier/.venv/lib/python3.10/site-packages/lhotse/manipulation.pycombines  rnum_jobsmanifestfncsd|j|d}t|fdd|D}tdd|D}Wd|S1s+wY|S)a) Convenience wrapper that parallelizes the execution of functions that transform manifests. It splits the manifests into ``num_jobs`` pieces, applies the function to each split, and then combines the splits. This function is used internally in Lhotse to implement some parallel ops. .. hint: This way of parallel execution is the most optimal when each item in the manifest is being processed with "simple" operations that take a short amount of time -- a ``ProcessPoolExecutor`` would have added a lot of overhead for sending individual items over IPC. Example:: >>> from lhotse import CutSet, split_parallelize_combine >>> cuts = CutSet(...) >>> window_cuts = split_parallelize_combine( ... 16, ... cuts, ... CutSet.cut_into_windows, ... duration=30.0 ... ) :param num_jobs: The number of parallel jobs. :param manifest: The manifest to be processed. :param fn: Function or method that transforms the manifest; the first parameter has to be ``manifest`` (for methods, they have to be methods on that manifests type, e.g. ``CutSet.cut_into_windows``. :param args: positional arguments to ``fn``. :param kwargs keyword arguments to ``fn``. ) num_splitscs&g|]}j|gRiqSr)submit).0subsetargsexr"kwargsrr Fs&z-split_parallelize_combine..cSsg|]}|qSr)result)r%frrrr+GsN)splitrr)r r!r"r(r*splitsfuturesr,rr'rsplit_parallelize_combine!s #  r1itemscCst|}zt|}Wn tyYdSwt|g|}t|tr%t|St|tr/t |St|t t fr;t |St|trDtdtd|)z Take an iterable of data types in Lhotse such as Recording, SupervisonSegment or Cut, and create the manifest of the corresponding type. When the iterable is empty, returns None. NzFeatureSet generic construction from iterable is not possible, as the config information would have been lost. Call FeatureSet.from_features() directly instead.zUnknown type of manifest item: )iternext StopIterationrrr r from_recordingsrr from_segmentsrrr from_cutsr ValueError)r2 first_itemrrr to_manifestKs$         r;N)!concurrent.futuresr functoolsr itertoolsroperatorrtypingrrrr r lhotse.audior r lhotse.cutr rrlhotse.featuresrrlhotse.supervisionrrrrrintr1r;rrrrs0     *