asterixdb

Clone Tools
  • last updated 16 mins ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
Change logical plan to apply filter from 2ndary index

- Changes the IntroduceLSMComponentFilterRule to

replace the constant filter value from the query to the value

carried from 2ndary index search.

- Can use 2ndary index filter even the query doens't contain

any filter related condition.

Change-Id: I0e2fe0208662e5dcd49d1a22bfb58f96533e9497

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1727

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

    • -0
    • +5000
    /asterixdb/asterix-app/data/twitter/real.2.adm
    • -0
    • +5000
    /asterixdb/asterix-app/data/twitter/real.adm
  1. … 70 more files in changeset.
Include dead / failed nodes in cluster state json

Change-Id: I01f169088238aa777e2793b8dd4b73f79a0a01b3

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1808

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@apache.org>

Add internal function for type casting in 'lax' mode

- Add internal function 'cast-lax' which performs type

demotion in 'lax' mode and returns 'missing' if cast fails

- Fixed incorrect boundary check when converting from double to float

Change-Id: Id929f1e66853f0603d033cf0f824349296e83521

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1806

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

Fix UUID printing for JSON and CSV

Change-Id: If36a71e81cf7ba53ce50e8728d09a3eb7577a946

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1805

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Steven Jacobs <sjaco002@ucr.edu>

Add Disk Component Scan operation for primary LSM index

-Added disk component scan operation for primary LSMBTree index,

which would be used by creating new secondary index

-This operation scans all disk components of the primary index,

and return all tuples. Thus, tuples with the same primary key

in different in components would be returned separately.

-The returned tuple has an extra int field, which indicates

which component this tuple comes from, and a boolean flag,

which indicates whether this tuple is an anti-matter tuple or not.

Change-Id: I31b2c67c58cb0a440c1d2c26400af322e2f1c1e5

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1791

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

Minor cleanup, no PENDING on cluster state refresh

Change-Id: I34221471f9aafa4ae970b2f44f80577ebb8bbba7

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1803

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Fix typo in Index.isEnforcingKeyFields

Change-Id: Ibcc8c171bf9d1ad7ff5ab51a5baf804b9a23ed40

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1801

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

Allow insert anti-matter tuples when bulk loading LSM index

Previously, when we bulk load an LSM index, we are not allowed to insert

anti-matter tuples to the disk component. However, creating secondary

index for correlated datasets requires anti-matter tuples to be inserted

as well. Thus, this patch mainly contains the following changes:

- When bulk loading LSM index, allow the user to switch between insert

mode and delete mode

- Extended the LSMDiskComponentBulkLoader with the delete method. For

LSM index with anti-matter tuples, the delete method simply sets the

TupleWriter to delete mode, and inserts the anti-matter tuple. For LSM

index with buddy btree, it simply inserts the deleted tuple into the

buddy btree.

- Since the LSMDiskComponentBulkLoader would have a delete method

anyway, added a new ILSMDiskComponentBulkLoader interface containing the

delete method.

Change-Id: I6665f56a5d2183697197298fa24824eeb827686a

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1796

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 6 more files in changeset.
Clean up numeric type conversion code

- Add new conversion methods to ITypeConvertorComputer interface

- Move remaining type conversion code from ATypeHierarchy

to ITypeConvertorComputer implementations

- Add type demotion parameter to chose whether it fails

if input value is out of range for the target type ('strict' mode)

or not ('lax' mode)

- Clean up error messages

Change-Id: I68b78e2dc9ebf78799698a1c90bcd7cb5150137d

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1800

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

  1. … 14 more files in changeset.
Remove default node group.

In this way, CREATE DATASET statement can adjust to dynamic

cluster topology.

When we create a dataset:

- if the node group name is not given, we create a new node group

using all currently available nodes;

- if the node group name is give, we use the given node group for

the dataset.

When we drop a dataset:

- if no other dataset depends on the node group of the dataset to

be dropped, we also drop the node group.

Change-Id: If68dc6a7c1270ab1f5049c9334e3318425fd8287

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1799

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

  1. … 37 more files in changeset.
Remove unnecessary integer boxing in AInt*

Change-Id: I6ecefc4e8617371de00fb430edfe24fca2b1f92f

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1797

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

Enable Adding Nodes to Running *DB Cluster

Also ability to configure unique partition ids without having access to

complete cluster topology

Change-Id: If978442a95687c00ef78c89ed1b4440f5e308b99

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1785

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 9 more files in changeset.
Added 'source-format' profile to Hyracks pom to enable code reformatting at build time using AsterixCodeFormatProfile

Change-Id: I9d8a4a82f1152204d24eae5507f4516ac091620c

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1795

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

    • -0
    • +298
    /hyracks-fullstack/AsterixCodeFormatProfile.xml
Add LSMDiskComponentBulkLoader

-Added LSMDiskComponentBulkLoader implementations, which are used to bulk

load an LSMDiskComponent with anti-matters

-Added LSMDiskComponentWithBuddyBTreeBulkLoader implementations,

which are used to bulk load an LSMDiskComponent with deleted-keys btrees

-Refactored LSM flush/merge/index bulk load operations to use

the LSMDiskComponentBulkLoader

Change-Id: I772a6d68761fcbb85982a1c9f72f2d186e1d1ffb

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1773

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Implemented Disk Components Alignment Based on IDs

- Added IDs (using LSN) for disk components. When a disk component is

flushed, its initial ID is set as the LSN. When components are merged,

the result ID is the union of all IDs to be merged.

- Changed the correlated merge policy to correlate disk components of

the primary and secondaries using IDs.

Change-Id: I768ee9ac0a8d3c99c631086093a6b778b2e7588e

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1761

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Support rebalancing all datasets or a given dataverse.

Change-Id: Iad2740fd53b36bf122fd469beeca759d887e40fb

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1793

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

  1. … 53 more files in changeset.
Change IO directory for integration util to be a directory in target

Change-Id: I3dfb74dd4228725fb624eb1d21f621b7855f3d37

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1789

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

Reviewed-by: Till Westmann <tillw@apache.org>

Fixed race condition during ncMap lookup

Change-Id: I1bfbe712c100f48011a516c373ac8994028dc3dd

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1792

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

Fix potential class cast exception in UnorderedListTypeDefinition

Change-Id: I583d681ffe84bd73eebcb4acd07062f6b4a1460e

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1794

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Support rebalancing datasets with indexes.

- Remove type arguments from the methods in IndexUtils

that generate index operation (e.g., create, load, compact)

jobs. Do type extraction inside SecondaryIndexOperationsHelper.

Change-Id: I9c0720382440ae44441a8f8847e75649a3822fa2

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1790

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 21 more files in changeset.
Fix uniontype compatibility issue in TweetParser

1. Fix compatibility issue of UnionType in TweetParser.

2. Add unit test for TweetParser, covering missing attribute record

filtering.

Change-Id: Id7e316d20f929236b3fa17618bb8f87d28ac7efc

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1786

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Steven Jacobs <sjaco002@ucr.edu>

    • -0
    • +1
    /asterixdb/asterix-external-data/pom.xml
Add query/aql endpoint.

Change-Id: Ib17f8d33adf49d75403c1cbdabebea2e0f3aeb4a

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1653

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Taewoo Kim <wangsaeu@gmail.com>

Add a dataset rebalance REST API.

- Failures during rebalance are not handled;

- Indexes are not built for the rebalance target.

Change-Id: Ibda35252031fc4940972f0f19bbf796cadfa53d6

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1768

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

  1. … 61 more files in changeset.
Rename Component Filter Factory

Change-Id: I1b3d9bef3fb6118c66195630cec23a7e41fc5861

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1779

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

  1. … 3 more files in changeset.
Reduce exception wrapping for casting and printing

Change-Id: I1911125b2fa93b743bea8cd16211e3bd464dbf1e

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1782

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

  1. … 15 more files in changeset.
Fix potential race condition in ATypeHierarchy

Change-Id: I804ae58571ecdf5829b70073457480c4540657db

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1781

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Fix typo in feed ingestion tutorial

Make the datatypes in data and DDL consistent.

Change-Id: I5e8da3e6aa2ae267d15b23318f5c653267ccd062

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1780

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Unexpose enter and exit components

Change-Id: Ib4ef7b432bbe6ac9cf2bbe9244cfe2b406f4cb93

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1778

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Fix upsert deadlock and upsert with filtered primary only

This change fixes a deadlock that happens when 3 operations

an upsert, a search and a flush happen simulteniously.

If all the memory components are full, the upsert

gets blocked, the upsert could've obtained a lock on the

search key which would block the search not allowing it

to exit the components and not allowing the components

to be cleared and reused.

In addition, the change refactors common LSM index code.

Change-Id: I93fac0f27ab0b3cc071ff38aef90d850cbbce488

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1762

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

  1. … 72 more files in changeset.
Use JRE for AWS installation.

Change-Id: I25ca959f53c49c7aa1e2841227d27d7c254578be

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1776

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@apache.org>

Reviewed-by: Michael Blow <mblow@apache.org>

    • -0
    • +26
    /asterixdb/asterix-server/src/main/opt/aws/yaml/install_jre.yml