Class foreign_storage::ParquetInPlaceEncoder

class ParquetInPlaceEncoder : public foreign_storage::ParquetScalarEncoder

Subclassed by foreign_storage::TypedParquetInPlaceEncoder< V, T, NullType >, foreign_storage::TypedParquetInPlaceEncoder< int64_t, int32_t, NullType >, foreign_storage::TypedParquetInPlaceEncoder< V, V >

Public Functions

ParquetInPlaceEncoder(Data_Namespace::AbstractBuffer *buffer, const size_t omnisci_data_type_byte_size, const size_t parquet_data_type_byte_size)
virtual void reserve(const size_t num_elements) = 0
void appendData(const int16_t *def_levels, const int16_t *rep_levels, const int64_t values_read, const int64_t levels_read, int8_t *values)

Appends Parquet data to the buffer using an in-place algorithm. Any necessary transformation or validation of the data and decoding of nulls is part of appending the data. Each class inheriting from this abstract class must implement the functionality to copy, nullify and encode the data.

Note that the Parquet format encodes nulls using Dremel encoding.

Parameters

  • def_levels: - an array containing the Dremel encoding definition levels

  • rep_levels: - an array containing the Dremel encoding repetition levels

  • values_read: - the number of non-null values read

  • levels_read: - the total number of values (non-null & null) that are read

  • values: - values that are read

Protected Attributes

const size_t omnisci_data_type_byte_size_
const size_t parquet_data_type_byte_size_

Private Functions

void decodeNullsAndEncodeData(int8_t *data_ptr, const int16_t *def_levels, const int64_t values_read, const int64_t levels_read, const bool do_encoding)