asterixdb

Clone Tools
  • last updated a few minutes ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
Introduce Strategy Based Replication and Fault-Tolerance

This change includes the following:

- Introduce new APIs for Replication and Fault-Tolerance Strategies.

- Add configuration in cluster description file for high-availability.

- Add built-in replication strategies (Metadata_Only, Chained_Declustering)

- Add built-in fault-tolerance strategies (Auto, Metadata_Node)

- Remove none cluster state functionality from ClusterStateManager.

- Add customizable NC startup sequence

Change-Id: I1d1012f5541ce786f127866efefb9f3db434fedd

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1405

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

  1. … 97 more files in changeset.
Feed Connection Refactoring

1. The feed subscription network using FeedJoint is removed.

2. FeedConnection metadata dataset is added (pkeys: dataverseName,

feedName, datasetName).

3. Replaced the old intake job + collect job combination with one single

job using SplitOperator.

4. Now one feed can connect to multiple datasets.

5. The disconnect feed job is replaced by ActiveManagerMessage.

6. The new feed life cycle is:

- Create feed

- Connect feed to dataset0, dataset1, dataset2, etc.

- Start feed

- Stop feed

- Disconnect feed

7. New feedEventListner framework by Abdullah

Change-Id: Ic36267eb9a10df21734ce1cc1f38583e23c9e8f0

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1259

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

  1. … 183 more files in changeset.
Use rsync instead of wget to distribute the benchmark source.

Change-Id: I58d2553eb7866d6cc947f9a8f0443cb82fc654f6

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1511

Reviewed-by: Ian Maxon <imaxon@apache.org>

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Ian Maxon <imaxon@apache.org>

Added some info about open and close types to the data model doc. Change-Id: I1262090a2a568d935fd96e00db1da5cd836f25a5 Reviewed-on: https://asterix-gerrit.ics.uci.edu/1510 Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu> Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu> Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

Fix the names of Let to Servlet

Change-Id: Ib2f63f37ee9ef063930478028e3e05ca9dd4742c

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1486

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 14 more files in changeset.
ASTERIXDB-1747 Implemented full lifecycle capabilities for pre-distributed jobs

Added distribute and destroy functionality

Removed serialization and bytes when running pred-distributed jobs

Cleaned up methods

Enabled Mockito testing for CCS and NCS

Added Unit Test for Distributed Jobs

Change-Id: I59c3422d5c1ab7756a6a4685ac527dfe50434954

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1377

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

  1. … 27 more files in changeset.
Add cluster performance testing scripts.

- Wait SSH instead of sleep in the AWS installation script;

- Remove Java code and generate config files in Ansible;

- TPC-H is one benchmark, and more benchmarks could be added into the framework.

Change-Id: Ie17f20dc87694a865aebc561d8eded064c66f5bb

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1504

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

    • -0
    • +51
    /asterixdb/asterix-benchmark/pom.xml
  1. … 74 more files in changeset.
More flexibility for the LangExecutionUtil

Change-Id: Id2632dce25519995798016b5be1ca650f8af9586

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1507

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

NCService documentation improvements

Change-Id: I8a775498e758e900013db1116fef593d292cf8af

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1490

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

    • -0
    • +33
    /asterixdb/asterix-doc/src/main/ncservice.tex
    • -0
    • +1
    /asterixdb/asterix-doc/src/site/site.xml
Add error message for malformed job ids.

Change-Id: I315e3aa798af791f41535279e8a983e857f2b14e

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1495

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@apache.org>

Fix asterix-docker

Change-Id: Ifac17caf603f1d4f1c53413996d833869d68a266

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1506

Reviewed-by: Michael Blow <mblow@apache.org>

Integration-Tests: Michael Blow <mblow@apache.org>

Tested-by: Michael Blow <mblow@apache.org>

Truncate a temporary file before deleting it.

- Before RunFileReader tries to delete a temporary file,

make sure to truncate the file since OS might keep the

deleted file for a while, thus taking disk space.

Change-Id: Ie906c9f950e2f31af6f1b5ecc9cb35829d3edf8a

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1501

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

AQLPlus Refactoring 2: AQL+ grammar generation and AQLPlusExpressionToPlanTranslator

- Apply a systematic way of generating AQL+ grammar from AQL grammar instead of

having a separate grammar file and updating it by hand.

- Refactor AQLPlusExpressionToPlanTranslator so that it extends

AQLExpressionToPlanTranslator. This makes this class now follows

the current translation logic and it doesn't have to manually updated for

AQL expressions.

Change-Id: I444dbf4f615c23ccd69a5e4bb1ead300d0a81451

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1434

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

    • -0
    • +175
    /asterixdb/asterix-algebra/src/main/javacc/AQLPlusExtension.jj
Add a description of MANAGIX_HOME to the installation docs.

A more complex description was removed as part of a previous

cleanup, but the term is still used throughout the page.

Change-Id: I519dcd1fa2c0baab2aeabbd29c2d474fc32de0bf

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1500

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

Remove some broken links to the old AsterixDB site.

Change-Id: I54e42c3ecca6e4b002dec1f4922b7255b5a49593

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1499

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Encode queries with encodeURIComponent()

Change-Id: I6a04445b22258d9cc3bb258f60e91ed970e85d6a

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1497

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@apache.org>

Report integer metrics as numbers

Change-Id: Ie330ba0e030d188adbf675741c382cf2754853f1

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1496

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@apache.org>

Add necesasary plugins for simple release method

Change-Id: I8768ff760235f2f7f0c33d5af633bcdefa0b51e5

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1493

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

Fix async result delivery

- add new result status "started" for async requests

- add support for result status to the test framework

- stabilize result distribution for error in async requests

- add support for "pollget" to test async execution

- add sleep function

- allow inject-failure function to return any first argument

- use URIs instead on Strings in the TestExecutor to ensure that URIs are

correctly escaped

- add a few tests

Change-Id: Iafba65d9c7bd8643c42e5126c8d89164ae328908

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1394

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

  1. … 70 more files in changeset.
Debian Packaging for NCService

Change-Id: I1b5d0895ed5a34700d4e2f666715b857ce275127

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1492

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

    • -0
    • +24
    /asterixdb/asterix-server/src/deb/control/control
    • -0
    • +17
    /asterixdb/asterix-server/src/deb/control/postinst
    • -0
    • +15
    /asterixdb/asterix-server/src/deb/control/postrm
    • -0
    • +15
    /asterixdb/asterix-server/src/deb/control/preinst
    • -0
    • +15
    /asterixdb/asterix-server/src/deb/control/prerm
    • -0
    • +26
    /asterixdb/asterix-server/src/deb/systemd/asterix-cc.service
    • -0
    • +26
    /asterixdb/asterix-server/src/deb/systemd/asterix-nc.service
Change "samples" to "opt" in the distribution package.

This change also does the following things:

- make AWS script more robust: (1) can be run from anywhere,

(2) eliminate the need of customize_deployment.yml.

- cleanup documentation to reflect the directory structure

change.

- fix Managix zip to be able to directories deeper than 2 levels.

Change-Id: Ie7876376d089b9ebebba3d37ac90ac346a741be3

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1482

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

Tested-by: Ian Maxon <imaxon@apache.org>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -54
    • +0
    /asterixdb/asterix-server/src/main/aws/README
    • -23
    • +0
    /asterixdb/asterix-server/src/main/aws/bin/stop.sh
    • -0
    • +55
    /asterixdb/asterix-server/src/main/opt/aws/README
    • -0
    • +35
    /asterixdb/asterix-server/src/main/opt/aws/ansible/aws_stop.yml
  1. … 18 more files in changeset.
Change name resolution order to Node name -> DNS

Change-Id: I2bbfc8d9d8469c444f6c70bd3a77654892aeb42f

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1491

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Fix ASTERIXDB-1776

The source of the issue was a deprecated method that failed when two

IODevices share a common prefix in their absolute path

Change-Id: Iba7837b433ce57f99e2c547e8bd1fb0bfc5a31df

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1489

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Fix a bug in secondary index entry matter/antimatter reconciliation during upsert

Change-Id: I5d5c4c44f49308757e05bed3a20475abdd3d38e7

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1488

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

ASTERIXDB-1778: Optimize the edit-distance-check function

- Only calculate 2 * (threshold + 1) cells, rather than all cells per row.

- Terminate the calculation steps early when it become obvious that

the possible edit-distance value is greater than the given threshold.

There is no reason to compute all cells in the 2 dimensional array.

- Move the location of IListIterator to Hyracks since we now have

a CharacterIterator in a String. Change the name to ISequenceIterator.

- Add the section for the function in the manual.

- Remove letter counting filtering method since it is only applicable for

the string in ASCII range (0 ~ 127).

Change-Id: Ibc8729c4514bb87c347dd7d50358fd897b769977

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1481

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Jianfeng Jia <jianfeng.jia@gmail.com>

Enable commit runtime extension

Change-Id: I98083ea5e93cb5f45d92c5dfbacfee1020fad57a

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1485

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

  1. … 10 more files in changeset.
ASTERIXDB-1785: Remove servlet dependencies

Change-Id: I12bbd0754755982c95b522a9ade19e9f9690d95a

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1484

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -5
    • +0
    /asterixdb/asterix-client-helper/pom.xml
    • -19
    • +13
    /asterixdb/asterix-common/pom.xml
Improve reading from and writing to Metadata pages

This change introduces a new interface IComponentMetadata.

Each LSM component is associated with a metadata object

which can be used to read and write arbitrary data to

the metadata pages of components. When flushing a

component, data in its metadata component is automatically

flushed to the disk component. For merge operations,

the IO Callback is responsible for merging the components'

metadata pages.

Change-Id: Id95ef33c0a0bc1abb3fc3ecdea5611ee4acd6dfa

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1476

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@apache.org>

Integration-Tests: Ian Maxon <imaxon@apache.org>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 138 more files in changeset.
Improve the github README.

Change-Id: I676ec5ee6bcca4f1dc36d6c65f1127c1fddef340

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1480

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@apache.org>

Polish the github README.

Change-Id: I93b4eca1ef9a27dc2f14cd4a0e4bb384159731ce

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1479

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>