ACloudViewer  3.9.4
A Modern Library for 3D Data Processing
Public Member Functions | Static Public Attributes | List of all members
cloudViewer::t::pipelines::slac::ControlGrid Class Reference

#include <ControlGrid.h>

Public Member Functions

 ControlGrid ()=default
 Default constructor. More...
 
 ControlGrid (float grid_size, int64_t grid_count=1000, const core::Device &device=core::Device("CPU:0"))
 Constructor with initial grid size and grid count estimation. More...
 
 ControlGrid (float grid_size, const core::Tensor &keys, const core::Tensor &values, const core::Device &device=core::Device("CPU:0"))
 
void Touch (const geometry::PointCloud &pcd)
 Allocate control grids in the shared camera space. More...
 
void Compactify ()
 
std::tuple< core::Tensor, core::Tensor, core::Tensor > GetNeighborGridMap ()
 
geometry::PointCloud Parameterize (const geometry::PointCloud &pcd)
 
geometry::PointCloud Deform (const geometry::PointCloud &pcd)
 Non-rigidly deform a point cloud using the control grid. More...
 
geometry::Image Deform (const geometry::Image &depth, const core::Tensor &intrinsics, const core::Tensor &extrinsics, float depth_scale, float depth_max)
 
geometry::RGBDImage Deform (const geometry::RGBDImage &rgbd, const core::Tensor &intrinsics, const core::Tensor &extrinsics, float depth_scale, float depth_max)
 
core::Tensor GetInitPositions ()
 Get control grid original positions directly from tensor keys. More...
 
core::Tensor GetCurrPositions ()
 
std::shared_ptr< core::HashMap > GetHashMap ()
 
int64_t Size ()
 
core::Device GetDevice ()
 
int64_t GetAnchorIdx ()
 

Static Public Attributes

static const std::string kGrid8NbIndices = "Grid8NbIndices"
 
static const std::string kGrid8NbVertexInterpRatios
 8 neighbor grid interpolation ratio for vertex per point. More...
 
static const std::string kGrid8NbNormalInterpRatios
 8 neighbor grid interpolation ratio for normal per point. More...
 

Detailed Description

ControlGrid is a spatially hashed voxel grid used for non-rigid point cloud registration and TSDF integration. Each grid stores a map from the initial grid location to the deformed location. You can imagine a control grid as a jelly that is warped upon perturbation with its overall shape preserved. Reference: https://github.com/qianyizh/ElasticReconstruction/blob/master/FragmentOptimizer/OptApp.cpp http://vladlen.info/papers/elastic-fragments.pdf

Definition at line 30 of file ControlGrid.h.

Constructor & Destructor Documentation

◆ ControlGrid() [1/3]

cloudViewer::t::pipelines::slac::ControlGrid::ControlGrid ( )
default

Default constructor.

◆ ControlGrid() [2/3]

cloudViewer::t::pipelines::slac::ControlGrid::ControlGrid ( float  grid_size,
int64_t  grid_count = 1000,
const core::Device &  device = core::Device("CPU:0") 
)

Constructor with initial grid size and grid count estimation.

Definition at line 24 of file ControlGrid.cpp.

References cloudViewer::core::Float32, and cloudViewer::core::Int32.

◆ ControlGrid() [3/3]

cloudViewer::t::pipelines::slac::ControlGrid::ControlGrid ( float  grid_size,
const core::Tensor &  keys,
const core::Tensor &  values,
const core::Device &  device = core::Device("CPU:0") 
)

Constructor with known keys (Int32 x 3 positions) and values (Float32 x 3 shifted positions).

Definition at line 33 of file ControlGrid.cpp.

References cloudViewer::core::Float32, cloudViewer::core::Tensor::GetLength(), and cloudViewer::core::Int32.

Member Function Documentation

◆ Compactify()

void cloudViewer::t::pipelines::slac::ControlGrid::Compactify ( )

Force rehashing, so that all entries are remapped to [0, size) and form a contiguous index map.

Definition at line 82 of file ControlGrid.cpp.

References cloudViewer::core::Int64, cloudViewer::core::eigen_converter::TensorToEigenVector3iVector(), and cloudViewer::core::Tensor::To().

Referenced by cloudViewer::t::pipelines::slac::pybind_slac(), and cloudViewer::t::pipelines::slac::RunSLACOptimizerForFragments().

◆ Deform() [1/3]

geometry::Image cloudViewer::t::pipelines::slac::ControlGrid::Deform ( const geometry::Image &  depth,
const core::Tensor &  intrinsics,
const core::Tensor &  extrinsics,
float  depth_scale,
float  depth_max 
)

Non-rigidly deform a depth image by

  • unprojecting the depth image to a point cloud;
  • deform the point cloud;
  • project the deformed point cloud back to the image.

Definition at line 290 of file ControlGrid.cpp.

References cloudViewer::t::geometry::PointCloud::CreateFromDepthImage(), Deform(), cloudViewer::t::geometry::Image::GetCols(), cloudViewer::t::geometry::Image::GetRows(), Parameterize(), and cloudViewer::t::geometry::PointCloud::ProjectToDepthImage().

◆ Deform() [2/3]

geometry::PointCloud cloudViewer::t::pipelines::slac::ControlGrid::Deform ( const geometry::PointCloud &  pcd)

Non-rigidly deform a point cloud using the control grid.

Definition at line 241 of file ControlGrid.cpp.

References cloudViewer::t::geometry::PointCloud::GetPointAttr(), cloudViewer::t::geometry::PointCloud::GetPointColors(), cloudViewer::t::geometry::PointCloud::HasPointAttr(), cloudViewer::t::geometry::PointCloud::HasPointColors(), cloudViewer::t::geometry::PointCloud::HasPointNormals(), cloudViewer::core::Tensor::IndexGet(), cloudViewer::core::Int64, kGrid8NbIndices, kGrid8NbNormalInterpRatios, kGrid8NbVertexInterpRatios, LogError, cloudViewer::t::geometry::PointCloud::SetPointColors(), cloudViewer::t::geometry::PointCloud::SetPointNormals(), Sqrt(), cloudViewer::core::Tensor::Sum(), and cloudViewer::core::Tensor::View().

Referenced by Deform(), cloudViewer::t::pipelines::slac::FillInSLACAlignmentTerm(), and cloudViewer::t::pipelines::slac::VisualizePointCloudDeformation().

◆ Deform() [3/3]

geometry::RGBDImage cloudViewer::t::pipelines::slac::ControlGrid::Deform ( const geometry::RGBDImage &  rgbd,
const core::Tensor &  intrinsics,
const core::Tensor &  extrinsics,
float  depth_scale,
float  depth_max 
)

Non-rigidly deform an RGBD image by

  • unprojecting the depth image to a point cloud;
  • deform the point cloud;
  • project the deformed point cloud back to the image.

Definition at line 306 of file ControlGrid.cpp.

References cloudViewer::t::geometry::RGBDImage::color_, cloudViewer::t::geometry::PointCloud::CreateFromRGBDImage(), Deform(), cloudViewer::t::geometry::RGBDImage::depth_, cloudViewer::t::geometry::Image::GetCols(), cloudViewer::t::geometry::Image::GetRows(), Parameterize(), and cloudViewer::t::geometry::PointCloud::ProjectToRGBDImage().

◆ GetAnchorIdx()

int64_t cloudViewer::t::pipelines::slac::ControlGrid::GetAnchorIdx ( )
inline

Definition at line 116 of file ControlGrid.h.

Referenced by cloudViewer::t::pipelines::slac::FillInSLACRegularizerTerm(), and cloudViewer::t::pipelines::slac::pybind_slac().

◆ GetCurrPositions()

core::Tensor cloudViewer::t::pipelines::slac::ControlGrid::GetCurrPositions ( )
inline

Get control grid shifted positions from tensor values (optimized in-place).

Definition at line 110 of file ControlGrid.h.

Referenced by cloudViewer::t::pipelines::slac::FillInSLACRegularizerTerm(), cloudViewer::t::pipelines::slac::pybind_slac(), cloudViewer::t::pipelines::slac::UpdateControlGrid(), cloudViewer::t::pipelines::slac::VisualizeGridDeformation(), cloudViewer::t::pipelines::slac::VisualizePointCloudDeformation(), and cloudViewer::t::pipelines::slac::VisualizePointCloudEmbedding().

◆ GetDevice()

core::Device cloudViewer::t::pipelines::slac::ControlGrid::GetDevice ( )
inline

Definition at line 115 of file ControlGrid.h.

Referenced by cloudViewer::t::pipelines::slac::InitializeControlGrid(), and cloudViewer::t::pipelines::slac::pybind_slac().

◆ GetHashMap()

std::shared_ptr<core::HashMap> cloudViewer::t::pipelines::slac::ControlGrid::GetHashMap ( )
inline

Definition at line 112 of file ControlGrid.h.

◆ GetInitPositions()

core::Tensor cloudViewer::t::pipelines::slac::ControlGrid::GetInitPositions ( )
inline

Get control grid original positions directly from tensor keys.

Definition at line 104 of file ControlGrid.h.

References cloudViewer::core::Float32.

Referenced by cloudViewer::t::pipelines::slac::FillInSLACRegularizerTerm(), cloudViewer::t::pipelines::slac::pybind_slac(), cloudViewer::t::pipelines::slac::VisualizeGridDeformation(), and cloudViewer::t::pipelines::slac::VisualizePointCloudDeformation().

◆ GetNeighborGridMap()

std::tuple< core::Tensor, core::Tensor, core::Tensor > cloudViewer::t::pipelines::slac::ControlGrid::GetNeighborGridMap ( )

Get the neighbor indices per grid to construct the regularizer.

Returns
A 6-way neighbor grid map for all the active entries of shape (N, ).
  • buf_indices Active indices in the buffer of shape (N, )
  • buf_indices_nb Neighbor indices (including non-allocated entries) for the active entries of shape (N, 6).
  • masks_nb Corresponding neighbor masks of shape (N, 6).

Definition at line 115 of file ControlGrid.cpp.

References cloudViewer::core::Tensor::Contiguous(), cloudViewer::core::Tensor::GetLength(), cloudViewer::core::Int32, cloudViewer::core::Int64, cloudViewer::core::Tensor::T(), cloudViewer::core::Tensor::To(), and cloudViewer::core::Tensor::View().

Referenced by cloudViewer::t::pipelines::slac::FillInSLACRegularizerTerm(), cloudViewer::t::pipelines::slac::pybind_slac(), and cloudViewer::t::pipelines::slac::VisualizeGridDeformation().

◆ Parameterize()

geometry::PointCloud cloudViewer::t::pipelines::slac::ControlGrid::Parameterize ( const geometry::PointCloud &  pcd)

Parameterize an input point cloud by embedding each point in the grid with 8 corners via indexing and interpolation.

Returns
A PointCloud with parameterization attributes:
  • neighbors: Index of 8 neighbor control grid points of shape (8, ) in Int64.
  • ratios: Interpolation ratios of 8 neighbor control grid points of shape (8, ) in Float32.

TODO(wei): allow entries with less than 8 neighbors, probably in a kernel. Now we simply discard them and only accepts points with all 8 neighbors in the control grid map.

Definition at line 150 of file ControlGrid.cpp.

References cloudViewer::core::Tensor::Contiguous(), cloudViewer::core::Float32, cloudViewer::core::Tensor::Floor(), cloudViewer::core::Tensor::GetItem(), cloudViewer::core::Tensor::GetLength(), cloudViewer::t::geometry::PointCloud::GetPointColors(), cloudViewer::t::geometry::PointCloud::GetPointNormals(), cloudViewer::t::geometry::PointCloud::GetPointPositions(), cloudViewer::t::geometry::PointCloud::HasPointColors(), cloudViewer::t::geometry::PointCloud::HasPointNormals(), cloudViewer::core::TensorKey::Index(), cloudViewer::core::Tensor::IndexGet(), cloudViewer::core::Int32, cloudViewer::core::Int64, kGrid8NbIndices, kGrid8NbNormalInterpRatios, kGrid8NbVertexInterpRatios, cloudViewer::core::None, cloudViewer::t::geometry::PointCloud::SetPointAttr(), cloudViewer::t::geometry::PointCloud::SetPointColors(), cloudViewer::t::geometry::PointCloud::SetPointNormals(), cloudViewer::t::geometry::PointCloud::SetPointPositions(), cloudViewer::core::TensorKey::Slice(), cloudViewer::core::Tensor::T(), cloudViewer::t::geometry::kernel::image::To(), cloudViewer::core::Tensor::To(), and cloudViewer::core::Tensor::View().

Referenced by Deform(), and cloudViewer::t::pipelines::slac::FillInSLACAlignmentTerm().

◆ Size()

int64_t cloudViewer::t::pipelines::slac::ControlGrid::Size ( )
inline

Definition at line 113 of file ControlGrid.h.

Referenced by cloudViewer::t::pipelines::slac::pybind_slac(), cloudViewer::t::pipelines::slac::RunSLACOptimizerForFragments(), cloudViewer::t::pipelines::slac::UpdateControlGrid(), cloudViewer::t::pipelines::slac::VisualizeGridDeformation(), and cloudViewer::t::pipelines::slac::VisualizePointCloudEmbedding().

◆ Touch()

void cloudViewer::t::pipelines::slac::ControlGrid::Touch ( const geometry::PointCloud &  pcd)

Allocate control grids in the shared camera space.

Definition at line 46 of file ControlGrid.cpp.

References cloudViewer::core::Float32, cloudViewer::core::Tensor::GetLength(), cloudViewer::t::geometry::PointCloud::GetPointPositions(), cloudViewer::core::Int32, cloudViewer::core::Tensor::To(), and cloudViewer::core::Tensor::View().

Referenced by cloudViewer::t::pipelines::slac::InitializeControlGrid().

Member Data Documentation

◆ kGrid8NbIndices

const std::string cloudViewer::t::pipelines::slac::ControlGrid::kGrid8NbIndices = "Grid8NbIndices"
static

Attributes used to extend the point cloud to support neighbor lookup. 8 neighbor grid index per point.

Definition at line 34 of file ControlGrid.h.

Referenced by Deform(), cloudViewer::t::pipelines::slac::FillInSLACAlignmentTerm(), Parameterize(), and cloudViewer::t::pipelines::slac::VisualizePointCloudEmbedding().

◆ kGrid8NbNormalInterpRatios

const std::string cloudViewer::t::pipelines::slac::ControlGrid::kGrid8NbNormalInterpRatios
static
Initial value:
=
"Grid8NbNormalInterpRatios"

8 neighbor grid interpolation ratio for normal per point.

Definition at line 38 of file ControlGrid.h.

Referenced by Deform(), and Parameterize().

◆ kGrid8NbVertexInterpRatios

const std::string cloudViewer::t::pipelines::slac::ControlGrid::kGrid8NbVertexInterpRatios
static
Initial value:
=
"Grid8NbVertexInterpRatios"

8 neighbor grid interpolation ratio for vertex per point.

Definition at line 36 of file ControlGrid.h.

Referenced by Deform(), cloudViewer::t::pipelines::slac::FillInSLACAlignmentTerm(), and Parameterize().

The documentation for this class was generated from the following files: