asterixdb

Clone Tools
  • last updated a few minutes ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
Improvment on Cursor for Delimited Data

This change allows the parser to parse records in addition to streams.

Change-Id: I84ff40db664633c633277e9cc0ffa534cda9f26a

Reviewed-on: https://asterix-gerrit.ics.uci.edu/567

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

Divide Cluster into Unique Partitions

The change includes the following:

- Fix passing NC stores to AsterixConfiguration.

- Unify storage direcotry name in the instance level rather than the node level.

- Divide the cluster into unique storage partitions based on the number of stores.

- Refactored FileSplits and moved out of AqlMetadataProvider.

- Make AsterixHyracksIntegrationUtil use the passed configuration file.

- Make File Splits pass relative index paths of partitions rather than absolute paths.

- Remove unused AqlCompiledMetadataDeclarations class.

Change-Id: I8c7fbca5113dd7ad569a46dfa2591addb5bf8655

Reviewed-on: https://asterix-gerrit.ics.uci.edu/564

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

  1. … 27 more files in changeset.
Make Index Path Relative to NC IO Devices

This change includes the following:

- Store relative index path in persisted local resource as resource name.

- Fetch index absolute path based on FileSplit IO device and index relative path.

This change will enable a node to perform operations on an index that was created

on another node by finding the absolve path based on the node IO deviec and

the index relative path. This is needed in case a node leaves the cluster and

a replica node is assigned to take over and recover the index partitions

that belonged to the failed node.

Change-Id: I09fabd097b8a995ca959fe60286acdb43edfcfa1

Reviewed-on: https://asterix-gerrit.ics.uci.edu/563

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

  1. … 5 more files in changeset.
Fix for ASTERIXDB-1018, ASTERIXDB-1017, ASTERIXDB-1019, ASTERIXDB-1020, ASTERIXDB-1029, ASTERIXDB-1030, ASTERIXDB-1034

1. Inline SubplanOperator input and hence remove SubplanOperator.

2. Use boolean return from initFromSubTree to confirm that the

result should actually be used in IntroduceJoinAccessMethodRule

Prevent accidental attempted cast of null variable

in AbstractIntroduceAccessMethodrule.

Change-Id: Ifc1f844ac6e06e00da3f1618235b6ce9bfd48e9f

Reviewed-on: https://asterix-gerrit.ics.uci.edu/489

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

  1. … 22 more files in changeset.
Fix for ASTERIXDB-1018, ASTERIXDB-1017, ASTERIXDB-1019, ASTERIXDB-1020, ASTERIXDB-1029, ASTERIXDB-1030, ASTERIXDB-1034:

1. Let the keys of introduced nested group-bys (group-bys in a subplan) be only the

variables that are produced in the subplan;

2. In PushSelectIntoJoinRule, push independent operators (e.g., a current-datetime() call)

into the first branch from which the join condition refers some variables.

3. In SimpleUnnestToJoinRule, move the boundary between the two join branches of a added join

which results from pipelined datascans to be below operators that doesn't use any variables

(e.g., a current-datetime() call), therefore potentially, the upper (left) join branch

can be rewritten to index lookups.

Change-Id: I18cfa3875d676f71b26e91433ff101a7e725c890

Reviewed-on: https://asterix-gerrit.ics.uci.edu/488

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

  1. … 22 more files in changeset.
Change doclint mask flag to only enable on Java 8

Change-Id: I402cef4206fa0610eaff899cf005875323fb18d4

Reviewed-on: https://asterix-gerrit.ics.uci.edu/565

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@apache.org>

ASTERIXDB-54: s/IHyracksCommonContext/IHyracksTaskContext/

Change-Id: Id98f3d94e8036199dcbdbdb059c97c0f99ed9205

Reviewed-on: https://asterix-gerrit.ics.uci.edu/534

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

ASTERIXDB-54: s/IHyracksCommonContext/IHyracksTaskContext/

Change-Id: I5f2daf1d8f933d86e0254be443f3ebcef17ecbcf

Reviewed-on: https://asterix-gerrit.ics.uci.edu/533

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

ASTERIXDB-1229: - Fixed RemoveRedundantListifyRule to consider general expressions

Change-Id: I5e7b6f5ca4ed51e91de371b9d0b4e4dabdd2f2df

Reviewed-on: https://asterix-gerrit.ics.uci.edu/555

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Preston Carman <prestonc@apache.org>

Depend on stable Hyracks version

Here I am depending on the next Hyracks version, by

fetching from the Apache staging repository.

Change-Id: I402cef4206fa0610eaff899cf005875323fb18d3

Reviewed-on: https://asterix-gerrit.ics.uci.edu/562

Reviewed-by: Ian Maxon <imaxon@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Adapt Replication to Append Only LSM Components

This change includes the following:

- Make LSN recording in a single tree in LSM components.

- Pass LSN byte offset with every tree index file being replicated.

- Add LSN to remote recovery logs to check which logs should be replayed.

- Update ILogRecord method names to better describe their operation.

Change-Id: I8bd2656746e1c293b981d5f43e80928314ccbad0

Reviewed-on: https://asterix-gerrit.ics.uci.edu/561

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

Reviewed-by: Ian Maxon <imaxon@apache.org>

Refactored External Data

This change re-arrange asterix module's order. asterix-

external-data is moved in front of asterix-metadata.

Change-Id: I46b60b5e1cc37fd59adc0dd89f374d96502091b2

Reviewed-on: https://asterix-gerrit.ics.uci.edu/559

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

    • -177
    • +167
    /asterix-external-data/pom.xml
  1. … 79 more files in changeset.
Add Method to Get LSN Byte Offset in LSM Disk Components

The LSN byte offset is needed to support updating LSNs in remote replicas.

Change-Id: I2eb407e607157fe9c71a9cf22b04accc832be31b

Reviewed-on: https://asterix-gerrit.ics.uci.edu/560

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@apache.org>

Asterix MessageBroker implementation

This change includes the following:

- Add implementation for CC/NC MessageBroker.

- Implement GlobalResourceIdFactory using MessageBroker.

- Change resource id factory to GlobalResourceIdFactory.

- Refactor metadata indexes fixed properties.

- Use fixed resource ids for metadata indexes.

Change-Id: If4320e2c5a0130d2f86a4be6ae61f5cee43e30af

Reviewed-on: https://asterix-gerrit.ics.uci.edu/486

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 12 more files in changeset.
Changes required to support MessageBroker implementation

This change includes the following:

- API changes to support MessageBroker implementation.

- IResourceIdFactory interface to support application dependent implementation.

Change-Id: Ib9f49234eebe912c48e7f71980433a9b42595741

Reviewed-on: https://asterix-gerrit.ics.uci.edu/485

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

  1. … 6 more files in changeset.
patch 2

  1. … 141 more files in changeset.
patch 2

Changed the IFrameWriter Contract

Updated existing operators and added a test case for BTreeSearchOperatorNodePushable.

With this change, calling the open method itself moves it to the open state and

hence, close must be called.

Change-Id: I03da090002f79f4db7b5b31454ce3ac2b9e40c7f

Reviewed-on: https://asterix-gerrit.ics.uci.edu/551

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

  1. … 27 more files in changeset.
Changed the IFrameWriter Contract

Updated existing operators to conform to the new contract.

These operators are either index operators or Feed operators.

The rest of the operator already follow the new contract.

Change-Id: Ibcebe876340a25be0f561945582a95211c140e10

Reviewed-on: https://asterix-gerrit.ics.uci.edu/552

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

Fix ASTERIXDB-1201

The reason the nulls were hitting the bulk loader was because

the filter was always set to null. It seems to work fine when

it is passed in just as it is in IndexInsertDeletePOperator

Change-Id: I0fbfaeb8a98316f4d148285832ad31d991481e4b

Reviewed-on: https://asterix-gerrit.ics.uci.edu/553

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

Reviewed-by: Taewoo Kim <wangsaeu@gmail.com>

Run ExecutionTest w/integration test framework

I changed the default local.xml to use 2 NCs with

2 partitions each like AsterixHyracksIntegrationUtil

so that the results will match despite using simple

string compare to determine correctness.

There is also one test in particular (big-object-join)

that I had to add an artificial order by clause to

to deal with different results.

Issues outstanding:

- HDFS in ClusterExecutionIT needs to be thought out

Change-Id: I423f2a7c77839b999d466dd5cace302574d956c0

Reviewed-on: https://asterix-gerrit.ics.uci.edu/525

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Chris Hillery <ceej@lambda.nu>

  1. … 3852 more files in changeset.
Disable non-JDK 8 build profiles.

We use Java 8 specific features now, so we shouldn't bother

with any profiles for compiler settings older than that.

Change-Id: I97b03f76bd7afa66e9be5a20ff64477383525868

Reviewed-on: https://asterix-gerrit.ics.uci.edu/545

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

Reviewed-by: Chris Hillery <ceej@lambda.nu>

patch 1

    • -0
    • +7
    /asterix-app/data/tinycollege/staff.adm
    • -0
    • +7
    /asterix-app/data/tinycollege/students.adm
  1. … 75 more files in changeset.
patch 1

  1. … 45 more files in changeset.
ASTERIXDB-1110 Fix count keeping in primary index opTracker

This change fixes the counter of number of active operations in primary

index opTracker. This is accomplished by sharing of counter between multiple

partitions. Each transaction will have that counter as part of its

context object

Change-Id: I0dc34b9a3aa69d39ac2eda11c17e7ad0ccc1a661

Reviewed-on: https://asterix-gerrit.ics.uci.edu/536

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

ASTERIX-1211: Malformed metadata record for enforced open index - Changed type on the open olist field in index metadata to [ ANY ] - Fixed heterogeneous list deserialization

Change-Id: Ib587a138f59f37c369ba01e627a1ef9dd5a4989e

Reviewed-on: https://asterix-gerrit.ics.uci.edu/532

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Add a test case for Hyracks error reporting.

Change-Id: Ic3ac4c7b0a07eacbc448ce5724085eb8d1e6bc4d

Reviewed-on: https://asterix-gerrit.ics.uci.edu/527

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

ASTERIXDB-1198: make ISerializerDeserializer implementations in AsterixDB stateless.

Change-Id: I808b6583db6b48a0e9000dcf0ac88c03c336c3a5

Reviewed-on: https://asterix-gerrit.ics.uci.edu/522

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Jianfeng Jia <jianfeng.jia@gmail.com>

  1. … 50 more files in changeset.
ASTERIXDB-1206: call initialize()/deinitialize() in parallel for OperatorNodePushables in SuperActivityOperatorNodePushable.

Change-Id: I8700d5258d658ebbf711b4233bb0def1e8cf7c39

Reviewed-on: https://asterix-gerrit.ics.uci.edu/526

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Jianfeng Jia <jianfeng.jia@gmail.com>

ASTERIXDB-1198: make ISerializerDeserializer implementations in Hyracks stateless.

Change-Id: I1ec86d0a93d8f15d88d68fab24dbe858c5ba8842

Reviewed-on: https://asterix-gerrit.ics.uci.edu/521

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Jianfeng Jia <jianfeng.jia@gmail.com>