o }oi/ @s4ddlmZmZddlZddlZddlmmZ dej dej de de dej f dd Z d"d ej d ee d ej fddZd ej dej dej dej d ej dej f ddZejjjdd   d#dej dej dededeedeej d ej deeej ffddZ d$dej dedededed ej dej fd d!ZdS)%)DictOptionalNthetasphisoverheadfrontreturncCs|jd}tj|tjd}|dtj}d|||dk|dtj|dkB<d|||dk|tj|dk@<d||tj|dk|tj|dk@<d||tj|dk|dtj|dk@<d|||k<d||tj|k<|S)az Get the view direction based on given theta and phi values. Parameters: - thetas (torch.Tensor): Array of theta values with shape [B,] - phis (torch.Tensor): Array of phi values with shape [B,] - overhead (float): Threshold for determining top and bottom views. - front (float): Threshold for determining front, back and side views. Returns: - torch.Tensor: Array of view directions. Values can be: 0: front 1: side (camera left) 2: back 3: side (camera right) 4: top 5: bottom Notes: - Phi and theta values are assumed to be in radians. r)dtype)shapetorchzeroslongnppi)rrrr num_samplesresr_/home/ubuntu/.local/lib/python3.10/site-packages/nemo/collections/multimodal/data/nerf/utils.pyget_view_directions &"(, rcudacenters jitter_updevicecCst|}tgd|dt|d}ttj||dd}|dur.t ||nd}ttj||dd|}|||fS)a Compute the look-at vectors for camera poses. Parameters: centers: The centers of the cameras. jitter_up: The noise range for the up vector of the camera. device: Device to allocate the output tensor. Returns: Tuple: Contains the following: - forward_vector: The forward vectors of the cameras, shape [B, 3]. - up_vector: The up vectors of the cameras, shape [B, 3]. - right_vector: The right vectors of the cameras, shape [B, 3]. )rr rrr dimN) F normalizer FloatTensorto unsqueezerepeatlencross randn_like)rrrforward_vector up_vector right_vectorup_noiserrrcompute_look_at_vectors@s & r.r,r+r*cCsjtjdtj|ddt|dd}tj|||gdd|ddddddf<||dddddf<|S) a Construct the 4x4 pose matrices. Args: size (int): Number of pose matrices to construct. centers (torch.Tensor): The Cartesian coordinates of the camera centers. right_vector (torch.Tensor): The right vectors of the cameras. up_vector (torch.Tensor): The up vectors of the cameras. forward_vector (torch.Tensor): The forward vectors of the cameras. device (torch.device): Device to allocate tensors on. Returns: torch.Tensor: The pose matrices, shape [size, 4, 4]. r )r rrr rrNr )reyefloat32r%r&r'stack)rr,r+r*rposesrrrconstruct_posesXs&*r3F)enabledr2 intrinsicsheightwidthr error_mapcCs|jd}|\}} } } tjtjd|d||dtjd|d||ddd\} } | d||g|||gd} | d||g|||gd} i}|durt|||}|durttjd|||g|d}|||g}nt |||||d \}}||d <t | d |} t | d |} ||d <ntj |||d|||g}t | d }| |  ||}| | | |}tj |||fd d}||ddddddfd d}|ddddfd|}|d ||d}|d ||d}||fS)a Generates rays from camera poses and intrinsics. Args: poses (torch.Tensor): Camera poses, shape [B, 4, 4] (cam2world). intrinsics (torch.Tensor): Intrinsic camera parameters [fx, fy, cx, cy]. height (int): Height of the image. width (int): Width of the image. num_samples: Number of rays to sample, default is None for all rays. error_map: Optional tensor to use for non-uniform sampling of rays. device (torch.device): Device on which to generate the rays. Returns: Dict[str, torch.Tensor]: A dictionary containing the following keys: - 'rays_o': Origin of the rays, shape [B, N, 3] - 'rays_d': Directions of the rays, shape [B, N, 3] - 'inds': Indices of the rays, shape [B, N] (if N > 0) - 'inds_coarse': Coarse indices of the rays, shape [B, N] (if error_map is not None) rr rij)indexingg?N)sizer)r8rr6r7rsampled_indices_coarsersampled_indicesgrr .)rrmeshgridlinspacetreshapeexpandminrandintnon_uniform_samplinggatherarange full_liker1 transposer% expand_asview)r2r5r6r7rr8r batch_sizefxfycxcyijresultsr>r=zsxsys directionsrays_drays_orrrget_raysts@   ((     &r\rNc Cstj|||dd}|d|d}}|d|d} } || tj|||d| j|dd}|| tj|||d| j|dd}|||} | |fS)a Perform non-uniform sampling based on the provided error_map. Parameters: error_map: The error map for non-uniform sampling. batch_size (int): Batch size of the generated samples. num_samples (int): Number of samples to pick. height (int): Height of the image. width (int): Width of the image. device: Device on which tensors are stored. Returns: A tensor containing the sampled indices. F) replacementr9r )max)r multinomialr$randrclamp) r8rNrr6r7rr=inds_xinds_ysxsyr>rrrrGs,, rG)Nr)NNr)r)typingrrnumpyrrtorch.nn.functionalnn functionalr!Tensorfloatrrr.r3rampautocastintstrr\rGrrrrsp$ *   K