4.4. External API

OmniSciDB uses Apache’s Thrift software framework for all external client communication. The open source clients shipped with the repository include OmniSci’s omnisql command line process, JDBC driver and SQLImporter utility. For a full list of Thrift enabled clients supported by OmniSci see Table OmniSci Open Source Clients below. The full list of thrift API methods can be found in the omnisci.thrift file in the root of the OmniSciDB source directory.

All internal communication between the discrete processes within OmniSciDB also use the Thrift communication framework. For example, communications between OmniSciDB and Apache Calcite are facilitated via Thrift.

4.4.1. Design

OmniSci has developed a rich set of Thrift based API calls. The bindings for the these calls can be found in the file omnisci.thrift in the open source repository.

The classes generated from the Thrift bindings include OmniSciProcessor, OmniSciClient and OmniSciIf. The OmniSci code sitting ‘behind’ these classes can be found in the class DBHandler, in the file ThriftHandler/DBhandler.(cpp/h).

The class OmniSci class DBHandler implements the generated interface OmniSciIf. The OmniSci server (in MapDServer.cpp) create an instance of DBHandler, using this object to initialize two instances of apache::ThreadedServer objects - one per Thrift protocol supported by the server (see section below for protocol details).

OmniSci clients such as the JDBC driver use the generated OmniSciClient class to make calls through to the server. To maintain state across client calls to the Thrift bindings, an initial connection/validation process returns a session identifier as a TSessionId to the connecting client; subsequent calls need to include the session identifier as part of the serialised data structure.

In the case of Thrift communication with the Calcite server, Calcite acts as the communication server and the database acts as the client. The Calcite Thrift bindings can be found in java/thrift/calciteserver.thrift in the repository. The DBHandler class (called from MapdServer.cpp) creates an instance of the Calcite class, a class generated by Thrift from the calciteserver.thrift binding which implements the client portion of the framework. The Java Calcite server creates an apache::TThreadPoolServer to handle API calls. The OmniSci code written to process the API calls can be found in the file CalciteServerHandler.java under java/calcite in the repository.

4.4.2. Protocols

The OmniSci database server presents the Thrift interface using either Thrift’s ‘binary’ or ‘json’ protocols. When using json, Thrift will wrap the payload in a either a http or https header. For simplicity, in this document reference to http and https will be used to mean http/json and https/json.

The Thrift framework offers support for SSL/TLS encryption of links between clients and servers. OmniSci uses this feature to secure/encrypted communications between both its external clients and the main server and its internal process. The database system can process binary encrypted connections and https connections. To manage https communications, OmniSci provides a web_server to act as a proxy to the normal http port opened by the server. Table OmniSci Open Source Clients below lists the protocols supported by each of the clients.

As mentioned in the ‘Design’ section the server opens two ports, a binary port and a http port - whether or not these ports and encrypted and which specific ports number are used is configurable at startup; either through command line parameters or a configuration files. The server can not open a port for a specific protocol in both both encrypted and non-encrypted format; for example it can not open a binary port and binary encrypted port.

OmniSci Open Source Clients

Client process	Protocols
JDBC Driver	Binary/Binary Encrypted/HTTP/HTTPS
Omnisql	Binary/Binary Encrypted/HTTP/HTTPS
SQLImporter	Binary/HTTP/HTTPS
KafkaImporter	Binary/HTTP/HTTPS
StreamInserter	Binary/HTTP/HTTPS
Pymapd	Binary/Binary Encrypted/HTTP/HTTPS
Julia	Binary