Menu
Simba Technologies
Simba Technologies

SimbaEngine X SDK 10.1.3
Developing Drivers for Data Stores Without SQL

Overview of Collaborative Query Execution

SQL Engine has features that allow a DSII to alter and optimize the execution of a query according to the strengths of the data store. This takes place by providing access to the Algebraic Expression Tree (AE-Tree), which is an object-oriented representation of the operations necessary to perform the query. The ability to optimize the tree comes in two different forms:

  • You have full access to the AE-Tree. You can analyze it and add, remove, or alter the nodes.
  • Or, the SQL Engine can analyze the tree for you. It will use pass-down handlers for the operations that can often be executed in the data store, thus eliminating the need for them to be processed in the SQL engine.

Note:

  • For a JDBC driver, the functionality described in this section can be accomplished using the Java SQL Engine.
  • The Codebase sample implements some of the Collaborative Query Execution (CQE) optimizations. You can use it as a reference for implementing your own optimizations.

The advantage of using Collaborative Query Execution is that it allows your DSII to take over execution of the parts of the SQL query that your data store excels at, while leaving the rest to the Simba SQLEngine. For instance, if your data store can join tables extremely quickly, then this operation can be executed by your DSII while Simba SQLEngine takes care of the rest of the operations. When Simba SQLEngine and your DSII use Collaborative Query Execution, your driver supports all of the SQL that Simba SQLEngine supports, while still exposing the strengths of your data store.

If your data store cannot perform any additional operations, then Collaborative Query Execution does not need to be used. Simba SQLEngine will still support the full range of SQL in a fast and efficient manner.

Passing Down Processing to a Data Store

Before it executes a SQL statement, Simba SQLEngine can pass an optimized representation of the SQL statement to the DSI implementation. This optimized representation is called an Algebraic Expression Tree, or AE-Tree. The SQL statement takes this form just before Simba SQLEngine transforms it into an execution plan and executes it. When Simba SQLEngine passes the AE-Tree to the DSI implementation, the DSI implementation can choose to execute any part of the AE-Tree itself. It signals its intentions by modifying the AE-Tree before returning it to Simba SQLEngine.

For example, if your data store can filter data, or join data, or execute aggregate functions quickly, it can modify those nodes of the AE-Tree to point to the DSI implementation for execution. The DSI implementation can modify any part of the AE-Tree if it can perform the execution quickly, or it can replace the entire tree and execute the whole query itself.

The following diagram illustrates an example of creating an AE-Tree passing down a filter to the data store, then replacing the original table node with a new filtered table node:

The diagram above shows three views of a notional AE-Tree corresponding to the SQL query SELECT Col1 FROM Table1 WHERE Col2 = 23.

  1. The first view shows the AE-Tree originally created by Simba SQLEngine. All the columns in the projection are retrieved and filtered by the Simba SQL execution engine.
  2. The second tree is the same as the first. “Table1” and sub-tree rooted from “=” , shown in the highlighted section, are passed down to the data store. That is, the SQL Engine provides access to those nodes so that the DSII can determine whether it can handle the filter. The original tree is not changed until the DSII tells the SQL Engine that the filter can be passed down.
  3. If pass-down succeeds, it is converted into the tree shown in the third view, where “Selection” and “=” are replaced by “Table1’”.

After the DSI implementation passes back the AE-Tree, Simba SQLEngine transforms the modified AE-Tree into an execution plan and executes it. Simba SQLEngine execution engine, and the DSI implementation and data store collaborate on the execution of the SQL statement, with the data store executing the parts it can do quickly, and Simba SQLEngine executing the rest. Of course, the DSI implementation does not have to modify the AE-Tree at all. Simba SQLEngine can execute the entire SQL statement relatively quickly and efficiently by itself.

 

Related Links

Algebraic Expression Tree and Optimization

Pass-Down Operation Handlers