Class Encoder

class Encoder

Subclassed by ArrayNoneEncoder, DateDaysEncoder< T, V >, FixedLengthArrayNoneEncoder, FixedLengthEncoder< T, V >, NoneEncoder< T >, StringNoneEncoder

Public Functions

Encoder(Data_Namespace::AbstractBuffer *buffer)
virtual ~Encoder()
virtual size_t getNumElemsForBytesEncodedDataAtIndices(const int8_t *index_data, const std::vector<size_t> &selected_idx, const size_t byte_limit) = 0

Compute the maximum number of variable length encoded elements given a byte limit

NOTE: optional parameters above may be ignored by the implementation, but may or may not be required depending on the encoder type backing the implementation.

Return

the number of elements

Parameters

  • index_data: - (optional) index data for the encoded type

  • selected_idx: - which indices in the encoded data to consider

  • byte_limit: - byte limit that must be respected

virtual std::shared_ptr<ChunkMetadata> appendEncodedDataAtIndices(const int8_t *index_data, int8_t *data, const std::vector<size_t> &selected_idx) = 0

Append selected encoded data to the chunk buffer backing this encoder.

NOTE:

index_data must be non-null for varlen encoder types.
Return

updated chunk metadata for the chunk buffer backing this encoder

Parameters

  • index_data: - (optional) the index data of data to append

  • data: - the data to append

  • selected_idx: - which indices in the encoded data to append

virtual std::shared_ptr<ChunkMetadata> appendEncodedData(const int8_t *index_data, int8_t *data, const size_t start_idx, const size_t num_elements) = 0

Append encoded data to the chunk buffer backing this encoder.

NOTE:

index_data must be non-null for varlen encoder types.
Return

updated chunk metadata for the chunk buffer backing this encoder

Parameters

  • index_data: - (optional) the index data of data to append

  • data: - the data to append

  • start_idx: - the position to start encoding from in the data array

  • num_elements: - the number of elements to encode from the data array

virtual std::shared_ptr<ChunkMetadata> appendData(int8_t *&src_data, const size_t num_elems_to_append, const SQLTypeInfo &ti, const bool replicating = false, const int64_t offset = -1) = 0

Append data to the chunk buffer backing this encoder.

Parameters

  • src_data: Source data for the append

  • num_elems_to_append: Number of elements to append

  • ti: SQL Type Info for the column TODO(adb): used?

  • replicating: Pass one value and fill the chunk with it

  • offset: Write data starting at a given offset. Default is -1 which indicates an append, an offset of 0 rewrites the chunk up to num_elems_to_append.

void getMetadata(const std::shared_ptr<ChunkMetadata> &chunkMetadata)
virtual std::shared_ptr<ChunkMetadata> getMetadata(const SQLTypeInfo &ti) = 0
virtual void updateStats(const int64_t val, const bool is_null) = 0
virtual void updateStats(const double val, const bool is_null) = 0
virtual void updateStats(const int8_t *const src_data, const size_t num_elements) = 0

Update statistics for data without appending.

Parameters

  • src_data: - the data with which to update statistics

  • num_elements: - the number of elements to scan in the data

virtual void updateStatsEncoded(const int8_t *const dst_data, const size_t num_elements)

Update statistics for encoded data without appending.

Parameters

  • dst_data: - the data with which to update statistics

  • num_elements: - the number of elements to scan in the data

virtual void updateStats(const std::vector<std::string> *const src_data, const size_t start_idx, const size_t num_elements) = 0

Update statistics for string data without appending.

Parameters

  • src_data: - the string data with which to update statistics

  • start_idx: - the offset into src_data to start the update

  • num_elements: - the number of elements to scan in the string data

virtual void updateStats(const std::vector<ArrayDatum> *const src_data, const size_t start_idx, const size_t num_elements) = 0

Update statistics for array data without appending.

Parameters

  • src_data: - the array data with which to update statistics

  • start_idx: - the offset into src_data to start the update

  • num_elements: - the number of elements to scan in the array data

virtual void reduceStats(const Encoder&) = 0
virtual void copyMetadata(const Encoder *copyFromEncoder) = 0
virtual void writeMetadata(FILE *f) = 0
virtual void readMetadata(FILE *f) = 0
virtual bool resetChunkStats(const ChunkStats&)

: Reset chunk level stats (min, max, nulls) using new values from the argument.

Return

: True if an update occurred and the chunk needs to be flushed. False otherwise. Default false if metadata update is unsupported. Only reset chunk stats if the incoming stats differ from the current stats.

virtual void resetChunkStats() = 0

Resets chunk metadata stats to their default values.

size_t getNumElems() const
void setNumElems(const size_t num_elems)

Public Static Functions

Encoder *Create(Data_Namespace::AbstractBuffer *buffer, const SQLTypeInfo sqlType)

Protected Attributes

size_t num_elems_
Data_Namespace::AbstractBuffer *buffer_
DecimalOverflowValidator decimal_overflow_validator_
DateDaysOverflowValidator date_days_overflow_validator_