o oi @s*ddlmZmZmZmZmZmZddlZddlm Z m Z m Z m Z m Z ddlmZddlmZmZmZddlmZddlmZmZddlmZmZdd lmZmZmZeeeeefZ d ee!d eee"fd d Z#d ee!d eeefddZ$Gddde e Z%dde%de&de'd e e fddZ(dS)) TYPE_CHECKINGListOptionalSequenceTupleUnionN) BatchSampler DataLoaderDataset RandomSamplerSequentialSampler) TypeGuard)DeviceTensorstack) PinholeCamera) ImageLoadType load_image)Images ImageTensors)RandomRaySampler RaySamplerUniformRaySamplerlstreturncCt|to tdd|DS)Ncs|]}t|tVqdSN isinstancestr.0xr$J/home/ubuntu/.local/lib/python3.10/site-packages/kornia/nerf/data_utils.py "z"_is_list_of_str..rlistallrr$r$r%_is_list_of_str!r,cCr)Ncsrrrrr!r$r$r%r&&r'z&_is_list_of_tensors..r(r+r$r$r%_is_list_of_tensors%r-r/cseZdZdZdedededededej dd ffd d Z d"d e e dd fd dZ dedd fddZd e dd fddZd#ddZdedd fddZdedd fddZedeedee fddZdefddZdeeeefdefd d!ZZS)$ RayDatasetaClass to represent a dataset of rays. Args: cameras: scene cameras: PinholeCamera min_depth: sampled rays minimal depth from cameras: float max_depth: sampled rays maximal depth from cameras: float ndc: convert ray parameters to normalized device coordinates: bool device: device for ray tensors: Union[str, torch.device] dtype: type of ray tensors: torch.dtype cameras min_depth max_depthndcdevicedtyperNcs>td|_d|_||_||_||_||_||_||_ dSr) super__init__ _ray_sampler_imgs_cameras _min_depth _max_depth_ndc_device_dtype)selfr1r2r3r4r5r6 __class__r$r%r86s  zRayDataset.__init__ num_img_rayscCs"|dur |dS||dS)zInitialize a ray dataset. Args: num_img_rays: If not None, number of rays to randomly cast from each camera: math: `(B)`. N)_init_uniform_ray_dataset_init_random_ray_datasetrArDr$r$r%init_ray_datasetCs zRayDataset.init_ray_datasetimgscsb|t|r|}nt|r|}n tdt|d|fdd|D_dS)zInitialize images for training. Images can be either a list of tensors, or a list of paths to image disk locations. Args: imgs: List of image tensors or image paths: Images z8Expected a list of image tensors or image paths. Gotcha .csg|]}|jqSr$)tor?r"imgrAr$r% dsz7RayDataset.init_images_for_training..N)_check_image_type_consistencyr, _load_imagesr/ TypeErrortype_check_dimensionsr:)rArIimagesr$rNr%init_images_for_trainingOs   z#RayDataset.init_images_for_trainingcCs2t|j|j|j|j|jd|_|j|j|dS)zInitialize a random ray sampler and calculates dataset ray parameters. Args: num_img_rays: If not None, number of rays to randomly cast from each camers: math: `(B)`. r5r6N) rr<r=r>r?r@r9calc_ray_paramsr;rGr$r$r%rFfsz#RayDataset._init_random_ray_datasetcCs0t|j|j|j|j|jd|_|j|jdS)zGInitialize a uniform ray sampler and calculates dataset ray parameters.rWN) rr<r=r>r?r@r9rXr;rNr$r$r%rErsz$RayDataset._init_uniform_ray_datasetcCs4tdd|Dstdd|DstddSdS)NcsrrrrLr$r$r%r&zr'z;RayDataset._check_image_type_consistency..csrrr.rLr$r$r%r&zr'z9The list of input images can only be all paths or tensors)r* ValueError)rArIr$r$r%rPys$z(RayDataset._check_image_type_consistencyc Cst||jjkrtdt|d|jjtdd|Ds#tdtt||jj|jjD]-\}\}}}|j dd||fkr\td|d|j d|j d fd | | fq/dS) NzNumber of images z" does not match number of cameras css|] }|jddkVqdS)rN)shaperLr$r$r%r&sz/RayDataset._check_dimensions..z$Not all input images have 3 channelsz Image index z dimensions z4 are inconsistent with equivalent camera dimensions ) lenr; batch_sizerYr* enumeratezipheightwidthr[item)rArIirMrbrcr$r$r%rT}s&zRayDataset._check_dimensions img_pathscCs$g}|D] }|t|tjq|Sr)appendrr UNCHANGED)rfrIimg_pathr$r$r%rQszRayDataset._load_imagescCst|jtr t|jSdSNr)rr9rr^rNr$r$r%__len__s  zRayDataset.__len__idxscstjts tdjj|}jj|}jdur ||dfSjj|}jj|}d}fdd|D}t ddt ||D}|j j dd}|||fS)a*Get a dataset item. Args: idxs: An index or group of indices of ray parameter object: Union[int, List[int]] Return: A ray parameter object that includes ray origins, directions, and rgb values at the ray 2d pixel coordinates: RayGroup zRRay sampler is not initiate yet, please run self.init_ray_dataset() before use it.Ncsg|]}j|qSr$)r:)r"rerNr$r%rOsz*RayDataset.__getitem__..cSs2g|]\}}|dd|d|dfqS)Nr\r)rd)r"rMpoint2dr$r$r%rOs2)r6go@) rr9rrRorigins directionsr: camera_ids points_2drrarKr@)rArlrnro camerd_idsrqrgbs imgs_for_idsr$rNr% __getitem__s        zRayDataset.__getitem__r)rN)__name__ __module__ __qualname____doc__rfloatboolrtorchr6r8rrrHrrVrFrErPrrT staticmethodrr rQintrkrRayGroupru __classcell__r$r$rBr%r0)s6    &r0r\Tdatasetr_shufflecCsJdttdtfdd}trt|St|t|rt|nt||dd|dS)zInitialize a dataloader to manage a ray dataset. Args: dataset: A ray dataset: RayDataset batch_size: Number of rays to sample in a batch: int shuffle: Whether to shuffle rays or sample then sequentially: bool itemsrcSs|dSrjr$)rr$r$r% collate_rayssz0instantiate_ray_dataloader..collate_raysF) drop_last)sampler collate_fn)rrrr rr r )rr_rrr$r$r%instantiate_ray_dataloaders r)r\T))typingrrrrrrr|torch.utils.datarr r r r typing_extensionsr kornia.corerrrkornia.geometry.camerar kornia.iorrkornia.nerf.corerrkornia.nerf.samplersrrrrobjectr r,r/r0r~r{rr$r$r$r%s   $