Class foreign_storage::FileReader

class FileReader

Subclassed by foreign_storage::MultiFileReader, foreign_storage::SingleFileReader

Public Functions

FileReader(const std::string &file_path, const import_export::CopyParams &copy_params)
virtual ~FileReader()
virtual size_t read(void *buffer, size_t max_size) = 0

Read up to max_size bytes from archive into buffer starting starting from the end of the last read

Return

number of bytes actually read

Parameters

  • buffer: - buffer to load into

  • max_size: - maximum number of bytes to read into the buffer

virtual bool isScanFinished() = 0

Return

true if the entire file has been read

virtual size_t readRegion(void *buffer, size_t offset, size_t size) = 0

Read up to max_size bytes from archive, starting at given offset isScanFinished() must return true to use readRegion

Return

number of bytes actually read

Parameters

  • buffer: - buffer to load into

  • offset: - starting point into the archive to read

  • size: - maximum number of bytes to read into the buffer

virtual size_t getRemainingSize() = 0

Return

size of the remaining content to be read

virtual bool isRemainingSizeKnown() = 0

Return

if remaining size is known

virtual void checkForMoreRows(size_t file_offset, const ForeignServer *server_options = nullptr, const UserMapping *user_mapping = nullptr)

Rescan the target files Throws an exception if the rescan fails (ie files are not in a valid appended state or not supported)

Parameters

  • file_offset: - where to resume the scan from (end of the last row) as not all of the bytes may have been consumed by the upstream compoennet

  • server_options: - only needed for S3 backed files

  • user_mapping: - only needed for S3 backed files

virtual void serialize(rapidjson::Value &value, rapidjson::Document::AllocatorType &allocator) const = 0

Serialize internal state to given json object This Json will later be used to restore the reader state through a constructor must be called when isScanFinished() is true

Parameters

  • value: - json object to store needed state to this function can store any needed data or none

  • allocator: - allocator to use for json contruction

virtual FirstLineByFilePath getFirstLineForEachFile() const = 0

Returns a map containing the first line for each file that will be read.

virtual bool isEndOfLastFile() = 0

Returns a boolean indicating whether the reader is at the end of the last file that was read.

Protected Attributes

import_export::CopyParams copy_params_
std::string file_path_