#include <HashMap.h>

Inheritance diagram for cloudViewer::core::HashMap:

Collaboration diagram for cloudViewer::core::HashMap:

Public Member Functions
	HashMap (int64_t init_capacity, const Dtype &key_dtype, const SizeVector &key_element_shape, const Dtype &value_dtype, const SizeVector &value_element_shapes, const Device &device, const HashBackendType &backend=HashBackendType::Default)
	Initialize a hash map given a key and a value dtype and element shape. More...

	HashMap (int64_t init_capacity, const Dtype &key_dtype, const SizeVector &key_element_shape, const std::vector< Dtype > &dtypes_value, const std::vector< SizeVector > &element_shapes_value, const Device &device, const HashBackendType &backend=HashBackendType::Default)

	~HashMap ()=default
	Default destructor. More...

void	Reserve (int64_t capacity)
	Reserve the internal hash map with the given capacity by rehashing. More...

std::pair< Tensor, Tensor >	Insert (const Tensor &input_keys, const Tensor &input_values)

std::pair< Tensor, Tensor >	Insert (const Tensor &input_keys, const std::vector< Tensor > &input_values_soa)

std::pair< Tensor, Tensor >	Activate (const Tensor &input_keys)

std::pair< Tensor, Tensor >	Find (const Tensor &input_keys)

Tensor	Erase (const Tensor &input_keys)

Tensor	GetActiveIndices () const

void	Insert (const Tensor &input_keys, const Tensor &input_values, Tensor &output_buf_indices, Tensor &output_masks)

void	Insert (const Tensor &input_keys, const std::vector< Tensor > &input_values_soa, Tensor &output_buf_indices, Tensor &output_masks)

void	Activate (const Tensor &input_keys, Tensor &output_buf_indices, Tensor &output_masks)

void	Find (const Tensor &input_keys, Tensor &output_buf_indices, Tensor &output_masks)

void	Erase (const Tensor &input_keys, Tensor &output_masks)

void	GetActiveIndices (Tensor &output_buf_indices) const

void	Clear ()
	Clear stored map without reallocating the buffers. More...

void	Save (const std::string &file_name)

HashMap	Clone () const
	Clone the hash map with buffers. More...

HashMap	To (const Device &device, bool copy=false) const
	Convert the hash map to another device. More...

int64_t	Size () const
	Get the size (number of active entries) of the hash map. More...

int64_t	GetCapacity () const
	Get the capacity of the hash map. More...

int64_t	GetBucketCount () const
	Get the number of buckets of the internal hash map. More...

Device	GetDevice () const override
	Get the device of the hash map. More...

Tensor	GetKeyTensor () const

std::vector< Tensor >	GetValueTensors () const

Tensor	GetValueTensor (size_t index=0) const

std::vector< int64_t >	BucketSizes () const
	Return number of elements per bucket. More...

float	LoadFactor () const
	Return size / bucket_count. More...

std::shared_ptr< DeviceHashBackend >	GetDeviceHashBackend () const
	Return the implementation of the device hash backend. More...

Public Member Functions inherited from cloudViewer::core::IsDevice
	IsDevice ()=default

virtual	~IsDevice ()=default

bool	IsCPU () const

bool	IsCUDA () const

bool	IsSYCL () const

Static Public Member Functions
static HashMap	Load (const std::string &file_name)

Protected Member Functions
void	Init (int64_t init_capacity, const Device &device, const HashBackendType &backend)

void	InsertImpl (const Tensor &input_keys, const std::vector< Tensor > &input_values_soa, Tensor &output_buf_indices, Tensor &output_masks, bool is_activate_op=false)

void	CheckKeyLength (const Tensor &input_keys) const

void	CheckKeyValueLengthCompatibility (const Tensor &input_keys, const std::vector< Tensor > &input_values_soa) const

void	CheckKeyCompatibility (const Tensor &input_keys) const

void	CheckValueCompatibility (const std::vector< Tensor > &input_values_soa) const

void	PrepareIndicesOutput (Tensor &output_buf_indices, int64_t length) const

void	PrepareMasksOutput (Tensor &output_masks, int64_t length) const

std::pair< int64_t, std::vector< int64_t > >	GetCommonValueSizeDivisor ()

Detailed Description

Definition at line 23 of file HashMap.h.

Constructor & Destructor Documentation

◆ HashMap() [1/2]

cloudViewer::core::HashMap::HashMap	(	int64_t	init_capacity,
		const Dtype &	key_dtype,
		const SizeVector &	key_element_shape,
		const Dtype &	value_dtype,
		const SizeVector &	value_element_shapes,
		const Device &	device,
		const HashBackendType &	backend = `HashBackendType::Default`
	)

Initialize a hash map given a key and a value dtype and element shape.

Definition at line 20 of file HashMap.cpp.

◆ HashMap() [2/2]

cloudViewer::core::HashMap::HashMap	(	int64_t	init_capacity,
		const Dtype &	key_dtype,
		const SizeVector &	key_element_shape,
		const std::vector< Dtype > &	dtypes_value,
		const std::vector< SizeVector > &	element_shapes_value,
		const Device &	device,
		const HashBackendType &	backend = `HashBackendType::Default`
	)

Initialize a hash map given a key dtype and element shape, and a vector of value dtypes and element shapes for values stored in structure of arrays.

Definition at line 34 of file HashMap.cpp.

References Init().

◆ ~HashMap()

cloudViewer::core::HashMap::~HashMap ( )

default

Default destructor.

Member Function Documentation

◆ Activate() [1/2]

std::pair< Tensor, Tensor > cloudViewer::core::HashMap::Activate ( const Tensor & input_keys )

Parallel activate arrays of keys in Tensor. Specifically useful for large value elements (e.g., a 3D tensor), where we can do in-place management after activation. Return: output_buf_indices and output_masks, their roles are the same as in Insert.

Definition at line 94 of file HashMap.cpp.

References cloudViewer::core::make_pair().

Referenced by cloudViewer::core::pybind_core_hashmap().

◆ Activate() [2/2]

void cloudViewer::core::HashMap::Activate	(	const Tensor &	input_keys,
		Tensor &	output_buf_indices,
		Tensor &	output_masks
	)

Same as Activate, but takes output_buf_indices and output_masks as input. If their shapes and types match, reallocation is not needed.

Definition at line 166 of file HashMap.cpp.

References GetCapacity(), cloudViewer::core::Tensor::GetLength(), InsertImpl(), length(), max(), Reserve(), and Size().

◆ BucketSizes()

std::vector< int64_t > cloudViewer::core::HashMap::BucketSizes ( ) const

Return number of elements per bucket.

Definition at line 313 of file HashMap.cpp.

◆ CheckKeyCompatibility()

void cloudViewer::core::HashMap::CheckKeyCompatibility ( const Tensor & input_keys ) const

protected

Definition at line 383 of file HashMap.cpp.

References cloudViewer::core::SmallVectorTemplateCommon< T, typename >::begin(), cloudViewer::core::Dtype::ByteSize(), cloudViewer::core::SmallVectorImpl< T >::erase(), cloudViewer::core::Tensor::GetDtype(), cloudViewer::core::Tensor::GetShape(), LogError, and cloudViewer::core::SizeVector::NumElements().

Referenced by Erase(), Find(), and InsertImpl().

◆ CheckKeyLength()

void cloudViewer::core::HashMap::CheckKeyLength ( const Tensor & input_keys ) const

protected

Definition at line 357 of file HashMap.cpp.

References cloudViewer::core::Tensor::GetLength(), and LogError.

Referenced by Erase(), and Find().

◆ CheckKeyValueLengthCompatibility()

void cloudViewer::core::HashMap::CheckKeyValueLengthCompatibility	(	const Tensor &	input_keys,
		const std::vector< Tensor > &	input_values_soa
	)		const

protected

Definition at line 364 of file HashMap.cpp.

References cloudViewer::core::Tensor::GetLength(), and LogError.

Referenced by InsertImpl().

◆ CheckValueCompatibility()

void cloudViewer::core::HashMap::CheckValueCompatibility ( const std::vector< Tensor > & input_values_soa ) const

protected

Definition at line 398 of file HashMap.cpp.

References cloudViewer::core::SmallVectorTemplateCommon< T, typename >::begin(), cloudViewer::core::Dtype::ByteSize(), cloudViewer::core::SmallVectorImpl< T >::erase(), cloudViewer::core::Tensor::GetDtype(), cloudViewer::core::Tensor::GetShape(), LogError, and cloudViewer::core::SizeVector::NumElements().

Referenced by InsertImpl().

◆ Clear()

void cloudViewer::core::HashMap::Clear ( )

Clear stored map without reallocating the buffers.

Definition at line 217 of file HashMap.cpp.

◆ Clone()

HashMap cloudViewer::core::HashMap::Clone ( ) const

Clone the hash map with buffers.

Definition at line 227 of file HashMap.cpp.

References GetDevice(), and To().

Referenced by cloudViewer::core::pybind_core_hashmap().

◆ Erase() [1/2]

Tensor cloudViewer::core::HashMap::Erase ( const Tensor & input_keys )

Parallel erase an array of keys in Tensor. Return: output_masks stores if the erase is a success or failure (key not found all already erased in another thread).

Definition at line 106 of file HashMap.cpp.

Referenced by cloudViewer::core::pybind_core_hashmap().

◆ Erase() [2/2]

void cloudViewer::core::HashMap::Erase	(	const Tensor &	input_keys,
		Tensor &	output_masks
	)

Same as Erase, but takes output_masks as input. If its shape and type matches, reallocation is not needed.

Definition at line 198 of file HashMap.cpp.

References CheckKeyCompatibility(), CheckKeyLength(), cloudViewer::core::Tensor::GetDataPtr(), cloudViewer::core::Tensor::GetLength(), length(), and PrepareMasksOutput().

◆ Find() [1/2]

std::pair< Tensor, Tensor > cloudViewer::core::HashMap::Find ( const Tensor & input_keys )

Parallel find an array of keys in Tensor. Return: output_buf_indices, its role is the same as in Insert. Return: output_masks stores if the finding is a success or failure (key not found).

Definition at line 100 of file HashMap.cpp.

References cloudViewer::core::make_pair().

Referenced by cloudViewer::core::pybind_core_hashmap().

◆ Find() [2/2]

void cloudViewer::core::HashMap::Find	(	const Tensor &	input_keys,
		Tensor &	output_buf_indices,
		Tensor &	output_masks
	)

Same as Find, but takes output_buf_indices and output_masks as input. If their shapes and types match, reallocation is not needed.

Definition at line 182 of file HashMap.cpp.

References CheckKeyCompatibility(), CheckKeyLength(), cloudViewer::core::Tensor::GetDataPtr(), cloudViewer::core::Tensor::GetLength(), length(), PrepareIndicesOutput(), and PrepareMasksOutput().

◆ GetActiveIndices() [1/2]

Tensor cloudViewer::core::HashMap::GetActiveIndices ( ) const

Parallel collect all indices in the buffer corresponding to the active entries in the hash map. Return output_buf_indices, collected buffer indices.

Definition at line 112 of file HashMap.cpp.

Referenced by cloudViewer::core::pybind_core_hashmap(), Reserve(), To(), and cloudViewer::t::io::WriteHashMap().

◆ GetActiveIndices() [2/2]

void cloudViewer::core::HashMap::GetActiveIndices ( Tensor & output_buf_indices ) const

Same as GetActiveIndices, but takes output_buf_indices as input. If its shape and type matches, reallocation is not needed.

Definition at line 209 of file HashMap.cpp.

References cloudViewer::core::Tensor::GetDataPtr(), length(), and PrepareIndicesOutput().

◆ GetBucketCount()

int64_t cloudViewer::core::HashMap::GetBucketCount ( ) const

Get the number of buckets of the internal hash map.

Definition at line 260 of file HashMap.cpp.

◆ GetCapacity()

int64_t cloudViewer::core::HashMap::GetCapacity ( ) const

Get the capacity of the hash map.

Definition at line 258 of file HashMap.cpp.

Referenced by Activate(), GetKeyTensor(), GetValueTensor(), GetValueTensors(), Insert(), cloudViewer::core::pybind_core_hashmap(), and To().

◆ GetCommonValueSizeDivisor()

std::pair<int64_t, std::vector<int64_t> > cloudViewer::core::HashMap::GetCommonValueSizeDivisor ( )

protected

◆ GetDevice()

Device cloudViewer::core::HashMap::GetDevice ( ) const

overridevirtual

Get the device of the hash map.

Implements cloudViewer::core::IsDevice.

Definition at line 264 of file HashMap.cpp.

Referenced by Clone(), PrepareIndicesOutput(), PrepareMasksOutput(), cloudViewer::core::pybind_core_hashmap(), and To().

◆ GetDeviceHashBackend()

std::shared_ptr<DeviceHashBackend> cloudViewer::core::HashMap::GetDeviceHashBackend ( ) const

inline

Return the implementation of the device hash backend.

Definition at line 184 of file HashMap.h.

◆ GetKeyTensor()

Tensor cloudViewer::core::HashMap::GetKeyTensor ( ) const

Get the key tensor buffer to be used along with buf_indices and masks. Example: GetKeyTensor().IndexGet({buf_indices.To(core::Int64).IndexGet{masks}})

Definition at line 266 of file HashMap.cpp.

References cloudViewer::core::SmallVectorTemplateCommon< T, typename >::begin(), cloudViewer::core::shape_util::DefaultStrides(), GetCapacity(), and cloudViewer::core::SmallVectorImpl< T >::insert().

Referenced by cloudViewer::core::pybind_core_hashmap(), Reserve(), To(), and cloudViewer::t::io::WriteHashMap().

◆ GetValueTensor()

Tensor cloudViewer::core::HashMap::GetValueTensor ( size_t index = 0 ) const

Get the i-th value tensor buffer to be used along with buf_indices and masks. Example: GetValueTensors(0).IndexGet({buf_indices.To(core::Int64).IndexGet{masks}})

Definition at line 294 of file HashMap.cpp.

References cloudViewer::core::SmallVectorTemplateCommon< T, typename >::begin(), cloudViewer::core::shape_util::DefaultStrides(), cloudViewer::core::Tensor::GetBlob(), GetCapacity(), cloudViewer::core::Tensor::GetDataPtr(), cloudViewer::core::SmallVectorImpl< T >::insert(), and LogError.

Referenced by cloudViewer::t::geometry::ConstructTensorMap(), and cloudViewer::core::pybind_core_hashmap().

◆ GetValueTensors()

std::vector< Tensor > cloudViewer::core::HashMap::GetValueTensors ( ) const

Get the values tensor buffers to be used along with buf_indices and masks. Example: GetValueTensors()[0].IndexGet({buf_indices.To(core::Int64).IndexGet{masks}})

Definition at line 275 of file HashMap.cpp.

References cloudViewer::core::SmallVectorTemplateCommon< T, typename >::begin(), cloudViewer::core::shape_util::DefaultStrides(), GetCapacity(), and cloudViewer::core::SmallVectorImpl< T >::insert().

Referenced by cloudViewer::core::pybind_core_hashmap(), Reserve(), To(), and cloudViewer::t::io::WriteHashMap().

◆ Init()

void cloudViewer::core::HashMap::Init	(	int64_t	init_capacity,
		const Device &	device,
		const HashBackendType &	backend
	)

protected

Definition at line 319 of file HashMap.cpp.

References cloudViewer::core::CreateDeviceHashBackend(), cloudViewer::core::Dtype::GetDtypeCode(), LogError, cloudViewer::core::SizeVector::NumElements(), and cloudViewer::core::Dtype::Undefined.

Referenced by HashMap().

◆ Insert() [1/4]

std::pair< Tensor, Tensor > cloudViewer::core::HashMap::Insert	(	const Tensor &	input_keys,
		const std::vector< Tensor > &	input_values_soa
	)

Parallel insert arrays of keys and a structure of value arrays in Tensors. Return: output_buf_indices and output_masks, their role are the same as in single value Insert interface.

Definition at line 87 of file HashMap.cpp.

References Insert(), and cloudViewer::core::make_pair().

◆ Insert() [2/4]

void cloudViewer::core::HashMap::Insert	(	const Tensor &	input_keys,
		const std::vector< Tensor > &	input_values_soa,
		Tensor &	output_buf_indices,
		Tensor &	output_masks
	)

Same as Insert with a SoA of values, but takes output_buf_indices and output_masks as input. If their shapes and types match, reallocation is not needed.

Definition at line 152 of file HashMap.cpp.

References GetCapacity(), cloudViewer::core::Tensor::GetLength(), InsertImpl(), length(), max(), Reserve(), and Size().

◆ Insert() [3/4]

std::pair< Tensor, Tensor > cloudViewer::core::HashMap::Insert	(	const Tensor &	input_keys,
		const Tensor &	input_values
	)

Parallel insert arrays of keys and values in Tensors. Return: output_buf_indices stores buffer indices that access buffer tensors obtained from GetKeyTensor() and GetValueTensor() via advanced indexing. NOTE: output_buf_indices are stored in Int32. A conversion to Int64 is required for further indexing. Return: output_masks stores if the insertion is a success or failure (key already exists).

Definition at line 80 of file HashMap.cpp.

References cloudViewer::core::make_pair().

Referenced by Insert(), cloudViewer::core::pybind_core_hashmap(), and To().

◆ Insert() [4/4]

void cloudViewer::core::HashMap::Insert	(	const Tensor &	input_keys,
		const Tensor &	input_values,
		Tensor &	output_buf_indices,
		Tensor &	output_masks
	)

Same as Insert with a single value array, but takes output_buf_indices and output_masks as input. If their shapes and types match, reallocation is not needed.

Definition at line 144 of file HashMap.cpp.

References Insert().

◆ InsertImpl()

void cloudViewer::core::HashMap::InsertImpl	(	const Tensor &	input_keys,
		const std::vector< Tensor > &	input_values_soa,
		Tensor &	output_buf_indices,
		Tensor &	output_masks,
		bool	is_activate_op = `false`
	)

protected

Definition at line 118 of file HashMap.cpp.

References CheckKeyCompatibility(), CheckKeyValueLengthCompatibility(), CheckValueCompatibility(), cloudViewer::core::Tensor::GetDataPtr(), cloudViewer::core::Tensor::GetLength(), length(), PrepareIndicesOutput(), and PrepareMasksOutput().

Referenced by Activate(), Insert(), and Reserve().

◆ Load()

HashMap cloudViewer::core::HashMap::Load ( const std::string & file_name )

static

Load active keys and values from a npz file that contains 'key', 'n_values', 'value_{:03d}'.

Definition at line 223 of file HashMap.cpp.

References cloudViewer::t::io::ReadHashMap().

Referenced by cloudViewer::core::pybind_core_hashmap().

◆ LoadFactor()

float cloudViewer::core::HashMap::LoadFactor ( ) const

Return size / bucket_count.

Definition at line 317 of file HashMap.cpp.

◆ PrepareIndicesOutput()

void cloudViewer::core::HashMap::PrepareIndicesOutput	(	Tensor &	output_buf_indices,
		int64_t	length
	)		const

protected

Definition at line 429 of file HashMap.cpp.

References GetDevice(), cloudViewer::core::Tensor::GetDevice(), cloudViewer::core::Tensor::GetDtype(), cloudViewer::core::Tensor::GetLength(), cloudViewer::core::Int32, and length().

Referenced by Find(), GetActiveIndices(), and InsertImpl().

◆ PrepareMasksOutput()

void cloudViewer::core::HashMap::PrepareMasksOutput	(	Tensor &	output_masks,
		int64_t	length
	)		const

protected

Definition at line 438 of file HashMap.cpp.

References cloudViewer::core::Bool, GetDevice(), cloudViewer::core::Tensor::GetDevice(), cloudViewer::core::Tensor::GetDtype(), cloudViewer::core::Tensor::GetLength(), and length().

Referenced by Erase(), Find(), and InsertImpl().

◆ Reserve()

void cloudViewer::core::HashMap::Reserve ( int64_t capacity )

Reserve the internal hash map with the given capacity by rehashing.

Definition at line 48 of file HashMap.cpp.

References count, GetActiveIndices(), GetKeyTensor(), GetValueTensors(), cloudViewer::core::Tensor::IndexGet(), InsertImpl(), cloudViewer::core::Int64, LogDebug, Size(), and cloudViewer::core::Tensor::To().

Referenced by Activate(), Insert(), and cloudViewer::core::pybind_core_hashmap().

◆ Save()

void cloudViewer::core::HashMap::Save ( const std::string & file_name )

Save active keys and values to a npz file at 'key' and 'value_{:03d}'. The number of values is stored in 'n_values'. The file name should end with 'npz', otherwise 'npz' will be added as an extension.

Definition at line 219 of file HashMap.cpp.

References cloudViewer::t::io::WriteHashMap().

Referenced by cloudViewer::core::pybind_core_hashmap().

◆ Size()

int64_t cloudViewer::core::HashMap::Size ( ) const

Get the size (number of active entries) of the hash map.

Definition at line 256 of file HashMap.cpp.

Referenced by Activate(), Insert(), cloudViewer::core::pybind_core_hashmap(), and Reserve().

◆ To()

HashMap cloudViewer::core::HashMap::To	(	const Device &	device,
		bool	copy = `false`
	)		const

Convert the hash map to another device.

Definition at line 229 of file HashMap.cpp.

References copy, GetActiveIndices(), GetCapacity(), GetDevice(), GetKeyTensor(), GetValueTensors(), cloudViewer::core::Tensor::IndexGet(), Insert(), cloudViewer::core::Int64, and cloudViewer::core::Tensor::To().

Referenced by Clone(), and cloudViewer::core::pybind_core_hashmap().

The documentation for this class was generated from the following files:

/root/ACloudViewer/libs/cloudViewer/core/hashmap/HashMap.h
/root/ACloudViewer/libs/cloudViewer/core/hashmap/HashMap.cpp

Public Member Functions

Static Public Member Functions

Protected Member Functions

Detailed Description

Constructor & Destructor Documentation

◆ HashMap() [1/2]

◆ HashMap() [2/2]

◆ ~HashMap()

Member Function Documentation

◆ Activate() [1/2]

◆ Activate() [2/2]

◆ BucketSizes()

◆ CheckKeyCompatibility()

◆ CheckKeyLength()

◆ CheckKeyValueLengthCompatibility()

◆ CheckValueCompatibility()

◆ Clear()

◆ Clone()

◆ Erase() [1/2]

◆ Erase() [2/2]

◆ Find() [1/2]

◆ Find() [2/2]

◆ GetActiveIndices() [1/2]

◆ GetActiveIndices() [2/2]

◆ GetBucketCount()

◆ GetCapacity()

◆ GetCommonValueSizeDivisor()

◆ GetDevice()

◆ GetDeviceHashBackend()

◆ GetKeyTensor()

◆ GetValueTensor()

◆ GetValueTensors()

◆ Init()

◆ Insert() [1/4]

◆ Insert() [2/4]

◆ Insert() [3/4]

◆ Insert() [4/4]

◆ InsertImpl()

◆ Load()

◆ LoadFactor()

◆ PrepareIndicesOutput()

◆ PrepareMasksOutput()

◆ Reserve()

◆ Save()

◆ Size()

◆ To()