o 8wi%I@sHddlmZddlmZddlmZmZmZddlZddl m Z ddl Z ddl mZddlmZddlmZmZdd lmZdd lmZdd lmZmZmZdd lmZdd lmZddl m!Z!ddl"m#Z#m$Z$m%Z%m&Z&m'Z'ddl(m)Z)m*Z*ddl+m,Z,ddl-m.Z.ddl/m0Z0ddl1m2Z2ddl3m4Z4ddl5m6Z6GdddeZ7dS) OrderedDict)Iterator)AnyOptionalUnionN) WarningCache)move_data_to_device)BasePredictionWriter) _DataFetcher_DataLoaderIterDataFetcher)_Loop) _Progress)_no_grad_context_select_data_fetcher"_verify_dataloader_idx_requirement)_IndexBatchSamplerWrapper)_MultiProcessingLauncher)call)_check_dataloader_iterable_DataLoaderSource_parse_num_batches_process_dataloader_request_dataloader) RunningStage TrainerFn)CombinedLoader)has_len_all_ranks)MisconfigurationException) _ModuleMode)is_param_in_hook_signature)_PREDICT_OUTPUTc seZdZdZdCdddeddffdd Zedefd d ZejdDd e eddfd d Zede e fddZ ede fddZedefddZede efddZdEddZdEddZdEddZde efddZdEddZd e d!e d"e d#e eddf d$d%Zd e d!e d"e e defd&d'Zd(ed)edefd*d+Zd,ede e e fd-d.Z d!e d"e defd/d0Z!dEd1d2Z"dEd3d4Z#dEd5d6Z$dEd7d8Z%dEd9d:Z&dEd;d<Z'de efd=d>Z(dEd?d@Z)dEdAdBZ*Z+S)F_PredictionLoopz'Top-level loop where prediction starts.Ttrainerz pl.Trainerinference_modereturnNcsjt|||_g|_g|_t|_g|_t|_ t dd|_ d|_ d|_ d|_g|_d|_t|_dS)Npredict_dataloaderF)super__init__r$epoch_batch_indicescurrent_batch_indicesrbatch_progress max_batchesr_warning_cacher _data_source_combined_loader _data_fetcher_results _predictions_return_predictionsr _module_mode)selfr#r$ __class__d/home/ubuntu/sommelier/.venv/lib/python3.10/site-packages/pytorch_lightning/loops/prediction_loop.pyr(2s   z_PredictionLoop.__init__cCs|jS)z)Whether to return the predictions or not.)r3r5r8r8r9return_predictionsDsz"_PredictionLoop.return_predictionsr;cCsTt|jjjt }|r|std|dt|jjd|dur%||_dS||_dS)Nzb`return_predictions` should be set to `False` when using the strategies that spawn or fork. Found z with strategy .) isinstancer#strategylauncherrrtyper3)r5r;return_supportedr8r8r9r;Is cCs*|jgkr|jS|jdkr|jdS|jS)zThe cached predictions.r)r2num_dataloadersr:r8r8r9 predictionsUs z_PredictionLoop.predictionscCs|j}|dus Jt|jS)z-Returns the number of prediction dataloaders.N)r/len flattened)r5combined_loaderr8r8r9rC\s  z_PredictionLoop.num_dataloaderscCst|jdkS)Nr)sumr,r:r8r8r9skipcsz_PredictionLoop.skipcCs||jr dS|||j}|dusJ zAz+t|tr0t|}|j}|j }|j }n d}t|\}}}|j |j _ |||||WntyWYWd|_|SwWd|_nd|_wq)NTF) setup_datarIreset on_run_startr0r=r next_batch _batch_idx_dataloader_idxdoner+ is_last_batch _predict_step StopIteration _restarting on_run_end)r5 data_fetcherdataloader_iterbatch batch_idxdataloader_idxr8r8r9rungs4    z_PredictionLoop.runc Cs|j}|jdkr dS|j}t|}|jdt|ts"t|d}n|}|jj }|j dur3||j j O}t j }t j }g}g|_|jD]0}t|||t||||}||t||j|r`t|ntd} t|| |j} |j| qA||_||_dS)Nrzpredict_dataloader() sequentialinf)r#limit_predict_batchesr.rr>barrierr=rlightning_module2allow_zero_length_dataloader_with_multiple_devices datamoduler PREDICTINGrr,rFrrappendrrEfloatrr/) r5r#source dataloadersrGallow_zero_length trainer_fnstagedllength num_batchesr8r8r9rJs2          z_PredictionLoop.setup_datacCs|j|jjjdusJt|j|jjj}|j}|dus J|jdkr)td|j |_ | |t ||j |jjj7_ |j|_|j|_||_|j}ddt|D|_ddt|D|_dS)z4Resets the internal state of the loop for a new run.Nr]zO`trainer.predict()` only supports the `CombinedLoader(mode="sequential")` mode.cSg|]}gqSr8r8.0_r8r8r9 z)_PredictionLoop.reset..cSror8r8rpr8r8r9rsrt)r+ reset_on_runr#staterkrr/_mode ValueErrorr,limitssetupiterfetchedcurrentready_on_before_fetch_start_profiler_on_after_fetch_stop_profilerr0rCranger)r2)r5rWrGrCr8r8r9rKs"    z_PredictionLoop.resetcCs$||||dS)z^Calls ``_on_predict_model_eval``, ``_on_predict_start`` and ``_on_predict_epoch_start`` hooks.N)r_on_predict_model_eval_on_predict_start_on_predict_epoch_startr:r8r8r9rLs z_PredictionLoop.on_run_startcCs|}|||S)zeCalls ``on_predict_epoch_end`` and ``on_predict_end`` hooks and returns results from all dataloaders.)_on_predict_epoch_end_on_predict_end_on_predict_model_train)r5resultsr8r8r9rVsz_PredictionLoop.on_run_endcCs"|jdur|jd|_dSdSN)r0teardownr:r8r8r9rs   z_PredictionLoop.teardownrYrZr[rXc Cs|j}|j}|dus Jt|t}s*|j|}|jj||d}tj |d||d}|j |s7| ||nd}| |||jdkrD|nd} tj|dg| Rtj|dg| R|j |sl|| dn|f} tj |dg| R} | dur|jd|j |r|j}|j}|j}| |||jdkr|nd} tj|d | g| Rtj|d | g| R|j |js|r|j|t| t d dSdS) aRuns the actual predict step together with all the necessary bookkeeping and the hooks tied to it. Args: batch: the current batch to run the prediction on batch_idx: The index of the current batch. dataloader_idx: the index of the dataloader producing the current batch. dataloader_iter: The iterator if using this step flavor. N)r[batch_to_deviceFrBon_predict_batch_start predict_stepzBpredict returned None if it was on purpose, ignore this warning...on_predict_batch_endcpu)!r#r0r=r precision_plugin convert_inputra_on_before_batch_transferr_call_strategy_hookr+increment_ready!_store_data_for_prediction_writer _build_kwargsrC_call_callback_hooksvalues_call_lightning_module_hookincrement_started!_build_step_args_from_hook_kwargsr-warnincrement_processedrNrOrPincrement_completedr3r2rer torchdevice) r5rYrZr[rXr#rWusing_dataloader_iter any_on_epoch hook_kwargs step_argsrDr8r8r9rSsB          z_PredictionLoop._predict_stepcCs(td|fd|fg}|dur||d<|S)aAssembles the keyword arguments for the ``predict_step`` Args: batch: the current batch to run the prediction on batch_idx: the index of the current batch. dataloader_idx: the index of the dataloader producing the current batch. None if not multiple dataloaders in sequential mode. Returns: the dictionary containing all the keyboard arguments for the predict step rYrZNr[r)r5rYrZr[ step_kwargsr8r8r9rs z_PredictionLoop._build_kwargsrstep_hook_namecCs<|}t|jj|}t|ddds|ddt|S)z/Helper method to build args for `predict_step`.rZ)min_argsN)copygetattrr#rar poptupler)r5rrkwargs step_hook_fxr8r8r9r&s   z1_PredictionLoop._build_step_args_from_hook_kwargs dataloadercCs:t|dd}t|ts|jdt|jdgS|jS)zReturns a reference to the seen batch indices if the dataloader has a batch sampler wrapped by our :class:`~pytorch_lightning.overrides.distributed._IndexBatchSamplerWrapper`. batch_samplerNz@Couldn't infer the batch indices fetched from your dataloader: ``)rr=rr-rr@__name__seen_batch_indices)r5rrr8r8r9_get_batch_indices.s  z"_PredictionLoop._get_batch_indicesc Csdd|jjD}tdd|D}tdd|D}|s|rI|j}|dus(J|j|}||}|s6|S||}|rD|j|||rI||_|S)NcSsg|] }t|tr|qSr8)r=r rqcbr8r8r9rs:szE_PredictionLoop._store_data_for_prediction_writer..cs|]}|jjVqdSr)intervalon_epochrr8r8r9 ;zD_PredictionLoop._store_data_for_prediction_writer..csrr)ron_batchrr8r8r9r<r) r# callbacksanyr/rFrr)rer*) r5rZr[prediction_writersr any_on_batchrGr batch_indicesr8r8r9r9s    z1_PredictionLoop._store_data_for_prediction_writercC |jjdt|jddSN[z].predict_next)r#profilerstartr@rr:r8r8r9rMs z _PredictionLoop._on_before_fetchcCrr)r#rstopr@rr:r8r8r9rPs z_PredictionLoop._on_after_fetchcCs.|j}t|dt|dt|ddS)z!Calls ``on_predict_start`` hooks.on_predict_startN)r#rrrrr5r#r8r8r9rUs  z!_PredictionLoop._on_predict_startcCs"|j|jjt|jddS)Non_predict_model_eval)r4capturer#rarrr:r8r8r9r\sz&_PredictionLoop._on_predict_model_evalcCs|j|jjdSr)r4restorer#rar:r8r8r9r`sz'_PredictionLoop._on_predict_model_traincCs"|j}t|dt|ddS)z'Calls ``on_predict_epoch_start`` hooks.on_predict_epoch_startN)r#rrrrr8r8r9rcs z'_PredictionLoop._on_predict_epoch_startcCs.|j}t|dt|d|jr|jSdS)zlCalls ``on_predict_epoch_end`` hook. Returns: the results for all dataloaders on_predict_epoch_endN)r#rrrr;rDrr8r8r9ris   z%_PredictionLoop._on_predict_epoch_endcCs@|jsg|_g|_|j}t|dt|dt|ddS)zBResets previous gradient status and calls ``on_predict_end`` hook.on_predict_endN)r;r2r)r#rrrrrr8r8r9rxs  z_PredictionLoop._on_predict_endcCsn|j}|jdus Jtd|jjdko|jdkot|jt tj |j td|jjdko/|jdktj |j dS)N)rr]rB)rr) r#r/rrwrCr=r0r rrdrarr8r8r9rs"  z2_PredictionLoop._verify_dataloader_idx_requirement)Tr)r%N),r __module__ __qualname____doc__boolr(propertyr;setterrlistrrDintrCrIrr!r\rJrKrLrVrrrSrrstrrrobjectrrrrrrrrrrr __classcell__r8r8r6r9r"/sX   #   ?        r")8 collectionsrcollections.abcrtypingrrrrlightning_utilitiesrpytorch_lightningpllightning_fabric.utilitiesr pytorch_lightning.callbacksr pytorch_lightning.loops.fetchersr r pytorch_lightning.loops.loopr pytorch_lightning.loops.progressr!pytorch_lightning.loops.utilitiesrrr'pytorch_lightning.overrides.distributedr&pytorch_lightning.strategies.launchersrpytorch_lightning.trainerr3pytorch_lightning.trainer.connectors.data_connectorrrrrr pytorch_lightning.trainer.statesrr+pytorch_lightning.utilities.combined_loaderr pytorch_lightning.utilities.datar&pytorch_lightning.utilities.exceptionsr)pytorch_lightning.utilities.model_helpersr+pytorch_lightning.utilities.signature_utilsr !pytorch_lightning.utilities.typesr!r"r8r8r8r9s0