o zi@ @sddlZddlmZddlmZmZmZmZmZm Z m Z m Z m Z ddl Z ddlmZddl mZddlmZmZmZmZmZmZddlmZddlZddlmZmZmZm Z dd l!m"Z"dd l#m$Z$dd l%m&Z&dd l'm(Z(dd l)m*Z*m+Z+e ee,eedfedfZ-e*Z.de-dee e/ddffddZ0de-de/fddZ1 d&de2ddde3dee fddZ4 d'dede eefde e&defddZ5 d'dede eefde e&de e eee,efffd d!Z6 d'dede eefde e&dee,effd"d#Z7de2de3fd$d%Z8dS)(N)fields) AnyDict GeneratorIterableMappingOptionalSizedTupleUnion)is_dataclass_instance)Tensor) BatchSampler DataLoaderIterableDataset RandomSamplerSamplerSequentialSampler) TypeGuard)_reinstantiate_wrapped_cls_replace_value_in_saved_argshas_iterable_dataset sized_len)PossibleUserWarning)_IndexBatchSamplerWrapper) RunningStage)MisconfigurationException) WarningCacherank_zero_warnBTypebatchreturnccst|tr|jdkrdVdS|dVdSt|ttfr;t|ts;t|tr-|}|D] }t|EdHq/dSt |rSt |D] }tt ||j EdHqCdSdVdS)Nr) isinstancer ndimsizerrstrvalues_extract_batch_sizer rgetattrname)r samplefieldr-T/home/ubuntu/.local/lib/python3.10/site-packages/pytorch_lightning/utilities/data.pyr(*s       r(cCsrd}d}zt|D]}|dur|}q ||kr!td|dnq Wn ty.t|w|dur7t||S)zUnpack a batch to find a ``torch.Tensor``. Returns: ``len(tensor)`` when found, or ``1`` when it hits an empty or non iterable. zWe could not infer the batch_size from the batch. Either simplify its structure or provide the batch_size as `self.log(..., batch_size=batch_size)`.NzZTrying to infer the `batch_size` from an ambiguous collection. The batch size we found is zK. To avoid any miscalculations, use `self.log(..., batch_size=batch_size)`.)r( warning_cachewarnRecursionErrorr)r error_msg batch_sizebsr-r-r.extract_batch_size=s,  r5F dataloaderstrategyzpl.strategies.Strategy2allow_zero_length_dataloader_with_multiple_devicescCst|}|dur dS|jtj||jddd}|dkr&tdt|jd|dkrE|dkrEt|j}|s=td |d td|d t |rMtd d S)zIChecks if a given object has ``__len__`` method implemented on all ranks.NF)devicesum) reduce_oprzTotal length of `zA` across ranks is zero. Please make sure this was your intention.`zW` within local rank has zero length. Please make sure that it returns at least 1 batch.zb` across ranks is zero, but local rank has zero length. Please be cautious of uneven batch length.zYour `IterableDataset` has `__len__` defined. In combination with multi-process data loading (when num_workers > 1), `__len__` could be inaccurate if each worker is not configured independently to avoid having duplicate data.T) rreducetorchtensor root_devicertype__name__ RuntimeErrorr)r6r7r8 local_length total_lengthdataloader_cls_namer-r-r.has_len_all_ranks\s,   rGsamplermodecCs&t|||\}}t|g|Ri|SN)$_get_dataloader_init_args_and_kwargsr)r6rHrIdl_args dl_kwargsr-r-r._update_dataloadersrNc st|ts td|dt|d}|r!|j}|j|j|j}nddt| Dd}|j d<dt t |jj}tdd |D}|ro|r_|d dt tjj Dn|t tjj|d d|sfd d | Ddfdd Dd}d|}t|trdd<dd<n t|||fdd |D} | rt| } |jj} ddd | D} td| d| d| d| d |sttB|} | rt| }|jj} td| d|d| d|fS)NzThe dataloader z0 needs to subclass `torch.utils.data.DataLoader`__pl_saved_argscSs i|] \}}|ds||qS)_) startswith.0kvr-r-r. s z8_get_dataloader_init_args_and_kwargs..multiprocessing_contextr-css|] }|j|juVqdSrJ)kind VAR_KEYWORDrSpr-r-r. z7_get_dataloader_init_args_and_kwargs..cSs"i|] \}}|j|jur||qSr-)defaultemptyrRr-r-r.rVsselfcs*h|]\}}|vr|j|ur|qSr-)r^)rSr*r[)attrsr-r. s*z7_get_dataloader_init_args_and_kwargs..datasetcsi|] \}}|vr||qSr-r-rR) non_defaultsr-r.rVs batch_samplerrHcsDh|]}|j|j|jfvr |j|jur |jvr |jvr|jqSr-)rXPOSITIONAL_ONLYPOSITIONAL_OR_KEYWORDr^r_r*rZ) arg_namesrMr-r.rbs   z, css|] }d|dVqdS)z`self.r<Nr-)rSarg_namer-r-r.r\r]z,Trying to inject custom `Sampler` into the `z` instance. This would fail as some of the `__init__` arguments are not available as instance attributes. The missing attributes are z. If you instantiate your `zZ` inside a `*_dataloader` hook of your module, we will do this for you. Otherwise, define z inside your `__init__`.z&Trying to inject parameters into the `z{` instance. This would fail as it doesn't expose all its attributes in the `__init__` signature. The missing arguments are z. HINT: If you wrote the `zA` class, add the `__init__` arguments or allow passing `**kwargs`)r#r ValueErrorhasattrrO__pl_saved_kwargs__pl_saved_arg_names __datasetvarsitemsrWdictinspect signature__init__ parametersanyr'updatepopaddgetr'_dataloader_init_kwargs_resolve_samplersorted __class__rBjoinrsetkeys)r6rHrI was_wrappedrLoriginal_datasetparamshas_variadic_kwargsrc required_argssorted_required_argsrFmissing_args_messagemissing_kwargssorted_missing_kwargsr-)rhrarMrdr.rKsx            rKc Cs|tjk}t|d}t|}|dur|tus|rt|dre|j}|j}|j}|j } |rBt dd|||| \} }}| sBt d|j dt d||||| \} }}| sYt d |j d t|g|Ri|}nLt|d rt|drz|||j|rxdn|jd }Wn2t y} zd dl} | dt| } | st d| d} ~ ww|rt d|j dtdnt d|rt|}dd|dddS|dddS)a.This function is used to handle the sampler, batch_sampler arguments associated within a DataLoader for its re- instantiation. If the dataloader is being used for prediction, the sampler will be wrapped into an `_IndexBatchSamplerWrapper`, so Lightning can keep track of its indices. reNrO drop_lastFzlTrying to inject `drop_last=False` into batch sampler since you are predicting, however it seems the class `z` does not support it. Your predictions might be incomplete. To mitigate this, expose `drop_last` in the `__init__` method of your custom class.rHzYTrying to inject a modified sampler into the batch sampler; however, it seems the class `z` does not have an argument called `sampler.` To mitigate this, expose an argument `sampler` in the `__init__` method of your custom class.r3)r3rrz:.*__init__\(\) (got multiple values)|(missing \d required)a Lightning can't inject a (distributed) sampler into your batch sampler, because it doesn't subclass PyTorch's `BatchSampler`. To mitigate this, either follow the API of `BatchSampler` and instantiate your custom batch sampler inside the `*_dataloader` hook of your module, or set `Trainer(use_distributed_sampler=False)`. If you choose the latter, you will be responsible for handling the distributed sampling within your batch sampler.z&You are using a custom batch sampler `z` for prediction. Lightning would normally set `drop_last=False` to ensure all samples are returned, but for custom samplers it can't guarantee this. Make sure your sampler is configured correctly to return all indices.)categorya\ Lightning can't inject a (distributed) sampler into your batch sampler, because it doesn't subclass PyTorch's `BatchSampler`. To mitigate this, either follow the API of `BatchSampler` or set `Trainer(use_distributed_sampler=False)`. If you choose the latter, you will be responsible for handling the distributed sampling within your batch sampler.r")rHshufflerer3r)rHrre)r PREDICTINGr)rArrkrOrl__pl_saved_default_kwargsrmrr __qualname__ TypeErrorrr3rrematchr&rr)r6rHrI is_predictingrebatch_sampler_clsargskwargsdefault_kwargsrhsuccessexrrr-r-r.r{s           r{cCst|drd|jvr|jdSd|jvr|j|jdSt|dr*t|jtr*dSt|ds1dS|j}t|t r;dSt|t S)NrlrrcFrH) rkrlrmrOindexr#rcrrHrr)r6rHr-r-r._is_dataloader_shuffledRs       r)FrJ)9rr dataclassesrtypingrrrrrrr r r r>#lightning_utilities.core.apply_funcr r torch.utils.datarrrrrrtyping_extensionsrpytorch_lightningpllightning_fabric.utilities.datarrrr#lightning_fabric.utilities.warningsr'pytorch_lightning.overrides.distributedr pytorch_lightning.trainer.statesr&pytorch_lightning.utilities.exceptionsr%pytorch_lightning.utilities.rank_zerorrr&rr/intr(r5objectboolrGrNrKr{rr-r-r-r.sx ,         " (     a   j