o oiF#@sddlmZddlZddlmZddlmZddlZddlmm Z ddlm Z ddl m Z mZmZddlmZddlmZdd lmZdd lmZdd lmZmZdd lmZdd lmZee Z!GdddZ"dS)) annotationsN)datetime)cast)optim)ModuleTensortensor) KORNIA_CHECK) PinholeCamera)psnr)Images) RayDatasetinstantiate_ray_dataloader) NerfModel) deprecatedc@sneZdZdZd.dd Z d/d0ddZedd d! d/d0d"d#Zed1d%d&Z d2d'd(Z d3d4d+d,Z d-S)5 NerfSolverzNeRF solver class. Args: device: device for class tensors: Union[str, Device] dtype: type for all floating point calculations: torch.dtype device torch.devicedtype torch.dtypereturnNonecCsLd|_d|_d|_d|_d|_d|_d|_d|_d|_d|_ ||_ ||_ dS)NTr) _cameras _min_depth _max_depth_ndc_imgs _num_img_rays _batch_size_num_ray_points _nerf_model_nerf_optimizer_device_dtype)selfrrr&K/home/ubuntu/.local/lib/python3.10/site-packages/kornia/nerf/nerf_solver.py__init__1s zNerfSolver.__init__TMbP?camerasr min_depthfloat max_depthndcboolimgsr num_img_rays Tensor | int batch_sizeintnum_ray_pointsirregular_ray_samplinglog_space_encodinglrc Cs||_||_||_||_||_tt|to|dkdtt|to$|dkdt|dudt|tr=t|g|j |_ n t |rF||_ nt d||_t|| | d|_|jj|j|jdtj|j| d |_dS) aInitialize training settings and model. Args: cameras: Scene cameras in the order of input images. min_depth: sampled rays minimal depth from cameras. max_depth: sampled rays maximal depth from cameras. ndc: convert ray parameters to normalized device coordinates. imgs: Scene 2D images (one for each camera). num_img_rays: Number of rays to randomly cast from each camera: math: `(B)`. batch_size: Number of rays to sample in a batch. num_ray_points: Number of points to sample along rays. irregular_ray_sampling: Whether to sample ray points irregularly. log_space_encoding: Whether frequency sampling should be log spaced. lr: Learning rate. rz%batch_size must be a positive integerz)num_ray_points must be a positive integerNznum_img_rays must be specifiedz-num_img_rays can be either an int or a Tensor)r6r7rr)r8)rrrrrr isinstancer4rr3rtorch is_tensor TypeErrorrrr!tor#r$rAdam parametersr" r%r*r+r-r.r0r1r3r5r6r7r8r&r&r' setup_solverPs2  zNerfSolver.setup_solverrBz0.7.0) replace_withversionc Cs"|||||||||| | | dS)N)rBrAr&r&r' init_trainingszNerfSolver.init_training Module | NonecCs|jS)zReturns the NeRF model.)r!)r%r&r&r' nerf_modelszNerfSolver.nerf_modelcCsNt|jdudt|jdudt|jdudt|jdudt|jdudtt|j}tt|j}tt |j}tt |j}tt j |j}t ||j|j|j|j|jd}||||t||jdd }tjd |j|jd}d } |D](\} } } || | } t| | }|t| | d 7}|||| d 7} qvt|| d S)aTrains the NeRF model one epoch. 1) A dataset of rays is initialized, and sent over to a data loader. 2) The data loader sample a batch of rays randomly, and runs them through the NeRF model, to predict ray associated rgb model values. 3) The model rgb is compared with the image pixel rgb, and the loss between the two is back propagated to update the model weights. Implemented steps: - Create an object of class RayDataset - Initialize ray dataset with group of images on disk, and number of rays to randomly sample - Initialize a data loader with batch size info - Iterate over data loader -- Reset optimizer -- Run ray batch through Nerf model -- Find loss -- Back propagate loss -- Optimizer step Returns: Average psnr over all epoch rays. Nz!The model should be a NeRF model.z*The optimizer should be an Adam optimizer.z%The camera should be a PinholeCamera.z'The images should be a list of tensors.z/The number of images of Ray should be a tensor.r9T)shufflerrg?) r r!r"rrrrr rr rr Optimizerr rrrr#r$init_ray_datasetinit_images_for_trainingrrr;rFmse_lossr zero_gradbackwardstepr,)r%r*r1imagesrGnerf_optimizer ray_datasetray_data_loader total_psnri_batchorigins directionsrgbs rgbs_modellossr&r&r'_train_one_epochs6         zNerfSolver._train_one_epochrI num_epochscCsDt|D]}|}|ddkrtd}td|||qdS)zjRun training epochs. Args: num_epochs: number of epochs to run. Default: 1. rz%H:%M:%Sz$Epoch: %d: epoch_psnr = %f; time: %sN)ranger]rnowstrftimeloggerinfo)r%r^i_epoch epoch_psnr current_timer&r&r'runs  zNerfSolver.runN)rrrrrr)TTr))r*r r+r,r-r,r.r/r0r r1r2r3r4r5r4r6r/r7r/r8r,rr)rrF)rr,)rI)r^r4rr) __name__ __module__ __qualname____doc__r(rBrrEpropertyrGr]rhr&r&r&r'r(s  ) A   Ar)# __future__rloggingrtypingrr;torch.nn.functionalnn functionalrMr kornia.corerrrkornia.core.checkr kornia.geometry.camerar kornia.metricsr kornia.nerf.corer kornia.nerf.data_utilsr rkornia.nerf.nerf_modelr kornia.utilsr getLoggerrircrr&r&r&r's"