Clone
 

abdullah alamoudi <alamouda@uci.edu> in asterixdb

remove global metadata lock

Change-Id: Id05ff463fee356b3270b53d0b3137c4b1bc3d830

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/84

Reviewed-by: Sattam Alsubaiee <salsubaiee@gmail.com>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -2
    • +25
    /asterix-aql/src/main/javacc/AQL.jj
Fix ADM Parser Bug

This change make sure that each adm parser instance use its own objects for parsing

and not share it with other parser running on the same NC.

Change-Id: Ib54dd2f9f8474ddb8dc2d785f819dd62c7ce7ca3

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/73

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <westmann@gmail.com>

Add external indexes

This change include the following:

1. additional data parser for external data that parse hdfs records using Hive serdes.

2. allow users to create external data. this includes:

a) changes in metadata external dataset details.

b) addition of a new metadata index to store external file's statuses.

c) the pipeline for building the B-Tree and R-Tree indexes.

d) hyracks operators to fetch records with their RIDs using different formats.

e) hyracks operators to lookup and parse external records.

f) test cases for indexing and index access of different hdfs file formats.

g) exposing the secondary indexes over external data to the compiler.

3. adding a new aql command to refresh external datasets. this includes

a) global recovery on system startup.

b) changes in the aql parser.

c) construction of bulk modify pipelines and additional operators to perform local commit and abort operations (using 2PC protocol).

4. Added copyright header to all new files

5. Added additional test cases to test left outer join on external data

Change-Id: I1065a473299f6027eb073aeeba3a56d137f6f98e

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/70

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@uci.edu>

    • binary
    /asterix-app/data/hdfs/external-indexing-test.rc
    • binary
    /asterix-app/data/hdfs/external-indexing-test.seq
    • -0
    • +11
    /asterix-app/data/hdfs/external-indexing-test.txt
    • -0
    • +21
    /asterix-app/data/hdfs/spatialData.json
    • -0
    • +250
    /asterix-app/data/hdfs/tw_for_indexleftouterjoin.adm
  1. … 136 more files in changeset.
Adding external indexes

In Hyracks side, this change include the following:

1. The addition of three indexes:

a) external b-tree index

b) external r-tree index

c) external b-tree with buddy b-tree index

2. creating an additional logical operator in algebricks for performing lookup operations over external data and modify the different visitors to work with this operator

3. Added copyright header to all new files

Change-Id: Iecfbd86f06aff3caaf3a9652b63420666745ebb9

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/69

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Zachary Heilbron <zheilbron@gmail.com>

Reviewed-by: Sattam Alsubaiee <salsubaiee@gmail.com>

Reviewed-by: Ian Maxon <imaxon@uci.edu>

  1. … 51 more files in changeset.
Fix the rtree bulkload pipeline

Fix a bug related to merges in LSMRTree

Fix the LSM Rtree force modify

Merge branch 'master' of https://code.google.com/p/hyracks

Adjust creation of Rtree to reflect the new LSM Rtree Optimization

LSM Rtree Optimization

  1. … 41 more files in changeset.
fix merge issue 131 in LSM R-Tree and LSM Inverted Index

Revert "Merge external indexes into master"

This reverts commit 1a0d2b4b80677ddadb524503ad9c84e2f03aade7, reversing

changes made to 5eba6a0795f2278f4ea7747f8b54c858529b5502.

  1. … 42 more files in changeset.
Revert "fix merge issues"

This reverts commit 6440e891f1e3d53be18156a3b51ff3f5f5ba2847.

Revert "Added jdo2-api in asterix-external pom"

This reverts commit 13ce098d527a3ebf648d6c240d59fc8f968aed24.

Revert "updated test results to reflect changes in metadata"

This reverts commit 3629af665d3d3d3d285c5a00495a6a8288b7e436.

Revert "Merge branch 'alamouda/adding-external-indexes'"

This reverts commit 50c09981d993ca75c316b388041f0a497efeff86, reversing

changes made to 7ec10c1640fd072cabc985dc42c2b176a0439ddf.

updated test results to reflect changes in metadata

Added jdo2-api in asterix-external pom

fix merge issues

Merge external indexes into master

Merge branch 'alamouda/adding-external-indexes'

Correct behaviour of drop dataverse and drop dataset to drop external data indexes

Bug fixes

add R-Tree support for external indexes and indexes optimizations

  1. … 13 more files in changeset.
Introduced changes needed for external data indexing

  1. … 30 more files in changeset.
Introduced changes needed for external data indexing