o 8wiA @sddlZddlmZmZmZmZddlmZddlm Z m Z m Z ddl Z ddl mZddl mZddlmZmZmZmZmZmZddlmZddlZdd lmZmZmZmZdd l m!Z!dd l"m#Z#dd l$m%Z%dd l&m'Z'ddl(m)Z)m*Z*e ee+ee dfedfZ,e)Z-de,dee e.ddffddZ/de,de.fddZ0 d'de1ddde2deefddZ3 d(dede eefde e%defdd Z4 d(dede eefde e%de5e5e e6e+e fffd!d"Z7 d(dede eefde e%de6e+e ffd#d$Z8de1de2fd%d&Z9dS))N) GeneratorIterableMappingSized)fields)AnyOptionalUnion)is_dataclass_instance)Tensor) BatchSampler DataLoaderIterableDataset RandomSamplerSamplerSequentialSampler) TypeGuard)_reinstantiate_wrapped_cls_replace_value_in_saved_argshas_iterable_dataset sized_len)PossibleUserWarning)_IndexBatchSamplerWrapper) RunningStage)MisconfigurationException) WarningCacherank_zero_warnBTypebatchreturnccst|tr|jdkrdVdS|dVdSt|ttfr;t|ts;t|tr-|}|D] }t|EdHq/dSt |rSt |D] }tt ||j EdHqCdSdVdS)Nr) isinstancer ndimsizerrstrvalues_extract_batch_sizer rgetattrname)rsamplefieldr+]/home/ubuntu/sommelier/.venv/lib/python3.10/site-packages/pytorch_lightning/utilities/data.pyr&+s       r&cCsrd}d}zt|D]}|dur|}q ||kr!td|dnq Wn ty.t|w|dur7t||S)zUnpack a batch to find a ``torch.Tensor``. Returns: ``len(tensor)`` when found, or ``1`` when it hits an empty or non iterable. zWe could not infer the batch_size from the batch. Either simplify its structure or provide the batch_size as `self.log(..., batch_size=batch_size)`.NzZTrying to infer the `batch_size` from an ambiguous collection. The batch size we found is zK. To avoid any miscalculations, use `self.log(..., batch_size=batch_size)`.)r& warning_cachewarnRecursionErrorr)r error_msg batch_sizebsr+r+r,extract_batch_size>s,  r3F dataloaderstrategyzpl.strategies.Strategy2allow_zero_length_dataloader_with_multiple_devicescCst|}|dur dS|jtj||jddd}|dkr&tdt|jd|dkrE|dkrEt|j}|s=td |d td|d t |rMtd d S)zIChecks if a given object has ``__len__`` method implemented on all ranks.NF)devicesum) reduce_oprzTotal length of `zA` across ranks is zero. Please make sure this was your intention.`zW` within local rank has zero length. Please make sure that it returns at least 1 batch.zb` across ranks is zero, but local rank has zero length. Please be cautious of uneven batch length.zYour `IterableDataset` has `__len__` defined. In combination with multi-process data loading (when num_workers > 1), `__len__` could be inaccurate if each worker is not configured independently to avoid having duplicate data.T) rreducetorchtensor root_devicertype__name__ RuntimeErrorr)r4r5r6 local_length total_lengthdataloader_cls_namer+r+r,has_len_all_ranks]s,   rEsamplermodecCs&t|||\}}t|g|Ri|SN)$_get_dataloader_init_args_and_kwargsr)r4rFrGdl_args dl_kwargsr+r+r,_update_dataloadersrLc st|ts td|dt|d}|r!|j}|j|j|j}nddt| Dd}|j d<dt t |jj}tdd |D}|ro|r_|d dt tjj Dn|t tjj|d d|sfd d | Ddfdd Dd}d|}t|trdd<dd<n t|||fdd |D} | rt| } |jj} ddd | D} td| d| d| d| d |sttB|} | rt| }|jj} td| d|d| d|fS)NzThe dataloader z0 needs to subclass `torch.utils.data.DataLoader`__pl_saved_argscSs i|] \}}|ds||qS)_) startswith.0kvr+r+r, s z8_get_dataloader_init_args_and_kwargs..multiprocessing_contextr+css|] }|j|juVqdSrH)kind VAR_KEYWORDrQpr+r+r, z7_get_dataloader_init_args_and_kwargs..cSs"i|] \}}|j|jur||qSr+)defaultemptyrPr+r+r,rTsselfcs*h|]\}}|vr|j|ur|qSr+)r\)rQr(rY)attrsr+r, s*z7_get_dataloader_init_args_and_kwargs..datasetcsi|] \}}|vr||qSr+r+rP) non_defaultsr+r,rTs batch_samplerrFcsDh|]}|j|j|jfvr |j|jur |jvr |jvr|jqSr+)rVPOSITIONAL_ONLYPOSITIONAL_OR_KEYWORDr\r]r(rX) arg_namesrKr+r,r`s   z, css|] }d|dVqdS)z`self.r:Nr+)rQarg_namer+r+r,rZr[z,Trying to inject custom `Sampler` into the `z` instance. This would fail as some of the `__init__` arguments are not available as instance attributes. The missing attributes are z. If you instantiate your `zZ` inside a `*_dataloader` hook of your module, we will do this for you. Otherwise, define z inside your `__init__`.z&Trying to inject parameters into the `z{` instance. This would fail as it doesn't expose all its attributes in the `__init__` signature. The missing arguments are z. HINT: If you wrote the `zA` class, add the `__init__` arguments or allow passing `**kwargs`)r!r ValueErrorhasattrrM__pl_saved_kwargs__pl_saved_arg_names __datasetvarsitemsrUdictinspect signature__init__ parametersanyr%updatepopaddgetr'_dataloader_init_kwargs_resolve_samplersorted __class__r@joinrsetkeys)r4rFrG was_wrappedrJoriginal_datasetparamshas_variadic_kwargsra required_argssorted_required_argsrDmissing_args_messagemissing_kwargssorted_missing_kwargsr+)rfr_rKrbr,rIsx            rIc Cs|tjk}t|d}t|}|dur|tus|rt|dre|j}|j}|j}|j } |rBt dd|||| \} }}| sBt d|j dt d||||| \} }}| sYt d |j d t|g|Ri|}nLt|d rt|drz|||j|rxdn|jd }Wn2t y} zd dl} | dt| } | st d| d} ~ ww|rt d|j dtdnt d|rt|}dd|dddS|dddS)a.This function is used to handle the sampler, batch_sampler arguments associated within a DataLoader for its re- instantiation. If the dataloader is being used for prediction, the sampler will be wrapped into an `_IndexBatchSamplerWrapper`, so Lightning can keep track of its indices. rcNrM drop_lastFzlTrying to inject `drop_last=False` into batch sampler since you are predicting, however it seems the class `z` does not support it. Your predictions might be incomplete. To mitigate this, expose `drop_last` in the `__init__` method of your custom class.rFzYTrying to inject a modified sampler into the batch sampler; however, it seems the class `z` does not have an argument called `sampler.` To mitigate this, expose an argument `sampler` in the `__init__` method of your custom class.r1)r1rrz:.*__init__\(\) (got multiple values)|(missing \d required)a Lightning can't inject a (distributed) sampler into your batch sampler, because it doesn't subclass PyTorch's `BatchSampler`. To mitigate this, either follow the API of `BatchSampler` and instantiate your custom batch sampler inside the `*_dataloader` hook of your module, or set `Trainer(use_distributed_sampler=False)`. If you choose the latter, you will be responsible for handling the distributed sampling within your batch sampler.z&You are using a custom batch sampler `z` for prediction. Lightning would normally set `drop_last=False` to ensure all samples are returned, but for custom samplers it can't guarantee this. Make sure your sampler is configured correctly to return all indices.)categorya\ Lightning can't inject a (distributed) sampler into your batch sampler, because it doesn't subclass PyTorch's `BatchSampler`. To mitigate this, either follow the API of `BatchSampler` or set `Trainer(use_distributed_sampler=False)`. If you choose the latter, you will be responsible for handling the distributed sampling within your batch sampler.r )rFshufflercr1r)rFrrc)r PREDICTINGr'r?r rirMrj__pl_saved_default_kwargsrkrr __qualname__ TypeErrorrr1rrematchr$rr)r4rFrG is_predictingrcbatch_sampler_clsargskwargsdefault_kwargsrfsuccessexrrr+r+r,rys           rycCst|drd|jvr|jdSd|jvr|j|jdSt|dr*t|jtr*dSt|ds1dS|j}|durCt|dr@|j n|}n|j }t|t rMdSt|t S)NrjrraFrF) rirjrkrMindexr!rarrcrFrr)r4rcrFr+r+r,_is_dataloader_shuffledSs       r)FrH):rpcollections.abcrrrr dataclassesrtypingrrr r<#lightning_utilities.core.apply_funcr r torch.utils.datar r rrrrtyping_extensionsrpytorch_lightningpllightning_fabric.utilities.datarrrr#lightning_fabric.utilities.warningsr'pytorch_lightning.overrides.distributedr pytorch_lightning.trainer.statesr&pytorch_lightning.utilities.exceptionsr%pytorch_lightning.utilities.rank_zerorrr$rr-intr&r3objectboolrErLtuplerorIryrr+r+r+r,sz           " (     a   j