Clone
Abdullah Alamoudi <bamousaa@gmail.com>
committed
on 17 Dec 17
[ASTERIXDB-2194][COMP] Introduce datasource functions
- user model changes: yes
Some functions can be datasources
- storage format changes… Show more
[ASTERIXDB-2194][COMP] Introduce datasource functions

- user model changes: yes

 Some functions can be datasources

- storage format changes: no

- interface changes: yes

 - Add IDatasourceFunction: A function that is also a datasource

 - Add IFunctionToDataSourceTransformer: transform an unnest

   function into a datascan during compilation

Details:

- Currently, functions are location agnostic and are run on

 parameters that are either passed through them during compile

 time or runtime.

- An exception to this is the dataset function which has

 an associated location constraints running on the nodes

 which host the dataset.

- In this change, we introduce a general framework that allows

 creation of new functions similar to the dataset function.

- Such functions are called datasource Functions.

- A datasource function takes constant parameters and run on

 a set of partitions similar to the dataset function.

- The first example of such functions is the DatasetResources

 function.

- The DatasetResources function takes two parameters, a dataverse

 and a dataset. It is then run on all nodes and returns a set

 of dataset resources.

- Test cases are added for this function.

Change-Id: Ibcf807ac713a21e8f4d59868525467386e801303

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2216

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

Tested-by: abdullah alamoudi <bamousaa@gmail.com>

Show less

master + 3 more