Class foreign_storage::SingleTextFileReader

class SingleTextFileReader : public foreign_storage::SingleFileReader

Public Functions

SingleTextFileReader(const std::string &file_path, const import_export::CopyParams &copy_params)
SingleTextFileReader(const std::string &file_path, const import_export::CopyParams &copy_params, const rapidjson::Value &value)
~SingleTextFileReader()
SingleTextFileReader(const SingleTextFileReader&)
SingleTextFileReader &operator=(const SingleTextFileReader&)
size_t read(void *buffer, size_t max_size)

Read up to max_size bytes from archive into buffer starting starting from the end of the last read

Return

number of bytes actually read

Parameters

  • buffer: - buffer to load into

  • max_size: - maximum number of bytes to read into the buffer

size_t readRegion(void *buffer, size_t offset, size_t size)

Read up to max_size bytes from archive, starting at given offset isScanFinished() must return true to use readRegion

Return

number of bytes actually read

Parameters

  • buffer: - buffer to load into

  • offset: - starting point into the archive to read

  • size: - maximum number of bytes to read into the buffer

bool isScanFinished()

Return

true if the entire file has been read

size_t getRemainingSize()

Return

size of the remaining content to be read

bool isRemainingSizeKnown()

Return

if remaining size is known

void checkForMoreRows(size_t file_offset, const ForeignServer *server_options, const UserMapping *user_mapping)

Rescan the target files Throws an exception if the rescan fails (ie files are not in a valid appended state or not supported)

Parameters

  • file_offset: - where to resume the scan from (end of the last row) as not all of the bytes may have been consumed by the upstream compoennet

  • server_options: - only needed for S3 backed files

  • user_mapping: - only needed for S3 backed files

void serialize(rapidjson::Value &value, rapidjson::Document::AllocatorType &allocator) const

Serialize internal state to given json object This Json will later be used to restore the reader state through a constructor must be called when isScanFinished() is true

Parameters

  • value: - json object to store needed state to this function can store any needed data or none

  • allocator: - allocator to use for json contruction

Private Functions

std::string getFirstLine() const
void skipHeader()

Private Members

std::FILE *file_
size_t data_size_
bool scan_finished_
size_t header_offset_
size_t total_bytes_read_