asterixdb

Clone Tools
  • last updated 17 mins ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
Fix Error in Aborting Task in Super Activity

When aborting a task, its thread gets interrupted. This creates

a problem when interrupting

Change-Id: I603d3c101e0a4de4816eb5a6a7fd4320df317ce4

Reviewed-on: https://asterix-gerrit.ics.uci.edu/582

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

Stop Network Output Channel Sending Extra Bytes

This change re-introduce an optimization to the output channel

which ensures sending of data bytes only. This optimization

was broken when the big object change was introduced.

Change-Id: I896daf80deb23bcae5d5e934565cf5493ed828ba

Reviewed-on: https://asterix-gerrit.ics.uci.edu/581

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

ASTERIXDB-1187, ASTERIXDB-1162 and ASTERIXDB-1252 fixes, plus new internal functions

This change contains the following updates and fixes:

- New deep-equality function which also handles numeric equality,

e.g., [2.0, 2, 1.0] == [2, 2, 1].

- New internal functions for record manipulation, including add and remove fields from records

- Fixes for ASTERIXDB-1187: The RecordMergeDescriptor does not fully support

open record merge. Merging records with open fields results in "NullPointerException".

- Fixes to address issue ASTERIXDB-1162: dealing with closed or partly closed record

added to an open record field.

- Fixes issue ASTERIXDB-1252: Problems with Autogenerated Key

Change-Id: I3621ebdd71c7cd91b50d77a972ad863cea7fcbc2

Reviewed-on: https://asterix-gerrit.ics.uci.edu/298

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 70 more files in changeset.
ASTERIXDB-1256: wait for job completion on SYNC results delivery

Change-Id: Ib8a214ba4ff636d12927720c2311a9a5ca247d9e

Reviewed-on: https://asterix-gerrit.ics.uci.edu/580

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

Fix for ASTERIXDB-1247

It seems like the root of this is the testing harness closing the index/deleting the file before it has

had a chance to flush all of its pages. There are also some changes to cover potential

corner cases where confiscated pages could be lost, but this doesn't seem to directly

affect the bug.

Change-Id: Ia580242b3f7753fc2f793f879332de3270ee3fee

Reviewed-on: https://asterix-gerrit.ics.uci.edu/575

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

Give names to parameterized JUnit tests

Change-Id: Ifb53be6df9ad0efb1032b03ad8622c1bb7b7afbe

Reviewed-on: https://asterix-gerrit.ics.uci.edu/568

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

Change LICENSE creation mode for binary assemblies

Change-Id: I402cef4206fa0610eaff899cf005875323fb18d6

Reviewed-on: https://asterix-gerrit.ics.uci.edu/571

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Chris Hillery <ceej@lambda.nu>

    • -0
    • +1761
    /asterix-installer/src/main/licenses/LICENSE
    • -0
    • +403
    /asterix-installer/src/main/licenses/NOTICE
    • -0
    • +1739
    /asterix-server/src/main/licenses/LICENSE
    • -0
    • +403
    /asterix-server/src/main/licenses/NOTICE
    • -0
    • +1785
    /asterix-yarn/src/main/licenses/LICENSE
    • -0
    • +403
    /asterix-yarn/src/main/licenses/NOTICE
    • -0
    • +815
    /asterix-yarn/src/main/resources/LICENSE
Remove deprecated binary assemblies and fix others

Change-Id: I402cef4206fa0610eaff899cf005875323fb18d5

Reviewed-on: https://asterix-gerrit.ics.uci.edu/570

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Chris Hillery <ceej@lambda.nu>

Fix licenses in Hyracks binary assemblies

Change-Id: I2a6c4fb809b6fcad578007b8f48bf8bce179cd81

Reviewed-on: https://asterix-gerrit.ics.uci.edu/569

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Chris Hillery <ceej@lambda.nu>

ASTERIXDB-1044 Allow Reading large records from HDFS

This change tests reading of HDFS records larger than 8192.

The fix for the issue was merged previously.

Change-Id: I3cf52be4bc0fd8af2555062eeb421d7235088d98

Reviewed-on: https://asterix-gerrit.ics.uci.edu/529

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

    • -0
    • +11
    /asterix-app/data/hdfs/large-record.txt
Reduce Object Creation in Index Operations

Removed proportioanl object creation in btrees and rtrees during insert and

search operations. Instead, we re-use objects in OpContexts.

Change-Id: I7fab280372951522db02f8c2ff1d7d5b15529cd7

Reviewed-on: https://asterix-gerrit.ics.uci.edu/245

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

  1. … 12 more files in changeset.
First stage of external data cleanup

In this change, different parts of external data were refactored.

The goal was to make it more modular, easier to maintain and allow

higher flexibility for extension in addition to reducing code redundancy.

Change-Id: I04a8c4e494d8d1363992b6fe0bdbe6b2b3b7b767

Reviewed-on: https://asterix-gerrit.ics.uci.edu/566

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

  1. … 329 more files in changeset.
Improvment on Cursor for Delimited Data

This change allows the parser to parse records in addition to streams.

Change-Id: I84ff40db664633c633277e9cc0ffa534cda9f26a

Reviewed-on: https://asterix-gerrit.ics.uci.edu/567

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

Divide Cluster into Unique Partitions

The change includes the following:

- Fix passing NC stores to AsterixConfiguration.

- Unify storage direcotry name in the instance level rather than the node level.

- Divide the cluster into unique storage partitions based on the number of stores.

- Refactored FileSplits and moved out of AqlMetadataProvider.

- Make AsterixHyracksIntegrationUtil use the passed configuration file.

- Make File Splits pass relative index paths of partitions rather than absolute paths.

- Remove unused AqlCompiledMetadataDeclarations class.

Change-Id: I8c7fbca5113dd7ad569a46dfa2591addb5bf8655

Reviewed-on: https://asterix-gerrit.ics.uci.edu/564

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

  1. … 27 more files in changeset.
Make Index Path Relative to NC IO Devices

This change includes the following:

- Store relative index path in persisted local resource as resource name.

- Fetch index absolute path based on FileSplit IO device and index relative path.

This change will enable a node to perform operations on an index that was created

on another node by finding the absolve path based on the node IO deviec and

the index relative path. This is needed in case a node leaves the cluster and

a replica node is assigned to take over and recover the index partitions

that belonged to the failed node.

Change-Id: I09fabd097b8a995ca959fe60286acdb43edfcfa1

Reviewed-on: https://asterix-gerrit.ics.uci.edu/563

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

  1. … 5 more files in changeset.
Fix for ASTERIXDB-1018, ASTERIXDB-1017, ASTERIXDB-1019, ASTERIXDB-1020, ASTERIXDB-1029, ASTERIXDB-1030, ASTERIXDB-1034

1. Inline SubplanOperator input and hence remove SubplanOperator.

2. Use boolean return from initFromSubTree to confirm that the

result should actually be used in IntroduceJoinAccessMethodRule

Prevent accidental attempted cast of null variable

in AbstractIntroduceAccessMethodrule.

Change-Id: Ifc1f844ac6e06e00da3f1618235b6ce9bfd48e9f

Reviewed-on: https://asterix-gerrit.ics.uci.edu/489

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

  1. … 22 more files in changeset.
Fix for ASTERIXDB-1018, ASTERIXDB-1017, ASTERIXDB-1019, ASTERIXDB-1020, ASTERIXDB-1029, ASTERIXDB-1030, ASTERIXDB-1034:

1. Let the keys of introduced nested group-bys (group-bys in a subplan) be only the

variables that are produced in the subplan;

2. In PushSelectIntoJoinRule, push independent operators (e.g., a current-datetime() call)

into the first branch from which the join condition refers some variables.

3. In SimpleUnnestToJoinRule, move the boundary between the two join branches of a added join

which results from pipelined datascans to be below operators that doesn't use any variables

(e.g., a current-datetime() call), therefore potentially, the upper (left) join branch

can be rewritten to index lookups.

Change-Id: I18cfa3875d676f71b26e91433ff101a7e725c890

Reviewed-on: https://asterix-gerrit.ics.uci.edu/488

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

  1. … 22 more files in changeset.
Change doclint mask flag to only enable on Java 8

Change-Id: I402cef4206fa0610eaff899cf005875323fb18d4

Reviewed-on: https://asterix-gerrit.ics.uci.edu/565

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@apache.org>

ASTERIXDB-54: s/IHyracksCommonContext/IHyracksTaskContext/

Change-Id: Id98f3d94e8036199dcbdbdb059c97c0f99ed9205

Reviewed-on: https://asterix-gerrit.ics.uci.edu/534

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

ASTERIXDB-54: s/IHyracksCommonContext/IHyracksTaskContext/

Change-Id: I5f2daf1d8f933d86e0254be443f3ebcef17ecbcf

Reviewed-on: https://asterix-gerrit.ics.uci.edu/533

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

ASTERIXDB-1229: - Fixed RemoveRedundantListifyRule to consider general expressions

Change-Id: I5e7b6f5ca4ed51e91de371b9d0b4e4dabdd2f2df

Reviewed-on: https://asterix-gerrit.ics.uci.edu/555

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Preston Carman <prestonc@apache.org>

Depend on stable Hyracks version

Here I am depending on the next Hyracks version, by

fetching from the Apache staging repository.

Change-Id: I402cef4206fa0610eaff899cf005875323fb18d3

Reviewed-on: https://asterix-gerrit.ics.uci.edu/562

Reviewed-by: Ian Maxon <imaxon@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Adapt Replication to Append Only LSM Components

This change includes the following:

- Make LSN recording in a single tree in LSM components.

- Pass LSN byte offset with every tree index file being replicated.

- Add LSN to remote recovery logs to check which logs should be replayed.

- Update ILogRecord method names to better describe their operation.

Change-Id: I8bd2656746e1c293b981d5f43e80928314ccbad0

Reviewed-on: https://asterix-gerrit.ics.uci.edu/561

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

Reviewed-by: Ian Maxon <imaxon@apache.org>

Refactored External Data

This change re-arrange asterix module's order. asterix-

external-data is moved in front of asterix-metadata.

Change-Id: I46b60b5e1cc37fd59adc0dd89f374d96502091b2

Reviewed-on: https://asterix-gerrit.ics.uci.edu/559

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

    • -177
    • +167
    /asterix-external-data/pom.xml
  1. … 79 more files in changeset.
Add Method to Get LSN Byte Offset in LSM Disk Components

The LSN byte offset is needed to support updating LSNs in remote replicas.

Change-Id: I2eb407e607157fe9c71a9cf22b04accc832be31b

Reviewed-on: https://asterix-gerrit.ics.uci.edu/560

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@apache.org>

Asterix MessageBroker implementation

This change includes the following:

- Add implementation for CC/NC MessageBroker.

- Implement GlobalResourceIdFactory using MessageBroker.

- Change resource id factory to GlobalResourceIdFactory.

- Refactor metadata indexes fixed properties.

- Use fixed resource ids for metadata indexes.

Change-Id: If4320e2c5a0130d2f86a4be6ae61f5cee43e30af

Reviewed-on: https://asterix-gerrit.ics.uci.edu/486

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 12 more files in changeset.
Changes required to support MessageBroker implementation

This change includes the following:

- API changes to support MessageBroker implementation.

- IResourceIdFactory interface to support application dependent implementation.

Change-Id: Ib9f49234eebe912c48e7f71980433a9b42595741

Reviewed-on: https://asterix-gerrit.ics.uci.edu/485

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

  1. … 6 more files in changeset.
patch 2

  1. … 141 more files in changeset.
patch 2

Changed the IFrameWriter Contract

Updated existing operators and added a test case for BTreeSearchOperatorNodePushable.

With this change, calling the open method itself moves it to the open state and

hence, close must be called.

Change-Id: I03da090002f79f4db7b5b31454ce3ac2b9e40c7f

Reviewed-on: https://asterix-gerrit.ics.uci.edu/551

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

  1. … 27 more files in changeset.