Class FixedLengthEncoder

template<typename T, typename V>
class FixedLengthEncoder : public Encoder

Public Functions

FixedLengthEncoder(Data_Namespace::AbstractBuffer *buffer)
size_t getNumElemsForBytesEncodedDataAtIndices(const int8_t *index_data, const std::vector<size_t> &selected_idx, const size_t byte_limit)

Compute the maximum number of variable length encoded elements given a byte limit

NOTE: optional parameters above may be ignored by the implementation, but may or may not be required depending on the encoder type backing the implementation.

Return

the number of elements

Parameters

  • index_data: - (optional) index data for the encoded type

  • selected_idx: - which indices in the encoded data to consider

  • byte_limit: - byte limit that must be respected

std::shared_ptr<ChunkMetadata> appendEncodedDataAtIndices(const int8_t *index_data, int8_t *data, const std::vector<size_t> &selected_idx)

Append selected encoded data to the chunk buffer backing this encoder.

NOTE:

index_data must be non-null for varlen encoder types.
Return

updated chunk metadata for the chunk buffer backing this encoder

Parameters

  • index_data: - (optional) the index data of data to append

  • data: - the data to append

  • selected_idx: - which indices in the encoded data to append

std::shared_ptr<ChunkMetadata> appendEncodedData(const int8_t *index_data, int8_t *data, const size_t start_idx, const size_t num_elements)

Append encoded data to the chunk buffer backing this encoder.

NOTE:

index_data must be non-null for varlen encoder types.
Return

updated chunk metadata for the chunk buffer backing this encoder

Parameters

  • index_data: - (optional) the index data of data to append

  • data: - the data to append

  • start_idx: - the position to start encoding from in the data array

  • num_elements: - the number of elements to encode from the data array

std::shared_ptr<ChunkMetadata> appendData(int8_t *&src_data, const size_t num_elems_to_append, const SQLTypeInfo &ti, const bool replicating = false, const int64_t offset = -1)

Append data to the chunk buffer backing this encoder.

Parameters

  • src_data: Source data for the append

  • num_elems_to_append: Number of elements to append

  • ti: SQL Type Info for the column TODO(adb): used?

  • replicating: Pass one value and fill the chunk with it

  • offset: Write data starting at a given offset. Default is -1 which indicates an append, an offset of 0 rewrites the chunk up to num_elems_to_append.

void getMetadata(const std::shared_ptr<ChunkMetadata> &chunkMetadata)
std::shared_ptr<ChunkMetadata> getMetadata(const SQLTypeInfo &ti)
void updateStats(const int64_t val, const bool is_null)
void updateStats(const double val, const bool is_null)
void updateStats(const int8_t *const src_data, const size_t num_elements)

Update statistics for data without appending.

Parameters

  • src_data: - the data with which to update statistics

  • num_elements: - the number of elements to scan in the data

void updateStatsEncoded(const int8_t *const dst_data, const size_t num_elements)

Update statistics for encoded data without appending.

Parameters

  • dst_data: - the data with which to update statistics

  • num_elements: - the number of elements to scan in the data

void updateStats(const std::vector<std::string> *const src_data, const size_t start_idx, const size_t num_elements)

Update statistics for string data without appending.

Parameters

  • src_data: - the string data with which to update statistics

  • start_idx: - the offset into src_data to start the update

  • num_elements: - the number of elements to scan in the string data

void updateStats(const std::vector<ArrayDatum> *const src_data, const size_t start_idx, const size_t num_elements)

Update statistics for array data without appending.

Parameters

  • src_data: - the array data with which to update statistics

  • start_idx: - the offset into src_data to start the update

  • num_elements: - the number of elements to scan in the array data

void reduceStats(const Encoder &that)
void copyMetadata(const Encoder *copyFromEncoder)
void writeMetadata(FILE *f)
void readMetadata(FILE *f)
bool resetChunkStats(const ChunkStats&)

: Reset chunk level stats (min, max, nulls) using new values from the argument.

Return

: True if an update occurred and the chunk needs to be flushed. False otherwise. Default false if metadata update is unsupported. Only reset chunk stats if the incoming stats differ from the current stats.

void resetChunkStats()

Resets chunk metadata stats to their default values.

Public Members

T dataMin
T dataMax
bool has_nulls

Private Functions

std::shared_ptr<ChunkMetadata> appendEncodedOrUnencodedData(int8_t *&src_data, const size_t num_elems_to_append, const SQLTypeInfo &ti, const bool replicating, const int64_t offset, const bool is_encoded)
void updateStatsWithAlreadyEncoded(const V &encoded_data)
V encodeDataAndUpdateStats(const T &unencoded_data)