asterixdb

Clone Tools
  • last updated 29 mins ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
some finals + exception cleanup

Change-Id: Ie0252fef069ee93db6e32e13a13ebb11df5ce913

Reviewed-on: https://asterix-gerrit.ics.uci.edu/877

Reviewed-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

ASTERIXDB-1460: Fix log level crashing CC on start

The Hyracks CC would nullpoint on startup if the log level was

greater than FINEST. This works around the core issue that's in

ASTERIXDB-1460 since in this instance the way the log level was used

would cause the CC to fail on startup where the log level was

equal to or greater than FINEST.

Change-Id: I720eca41fac312fc6cbdbb880162a5bc8b0357dc

Reviewed-on: https://asterix-gerrit.ics.uci.edu/870

Reviewed-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Remove PowerMock from asterix-external-data

PowerMock prevents jacoco from collecting coverage information.

Hence, it is better not to use it. However, using Mockito is fine.

In addition, this change fixes the BTree frame writer unit test.

Change-Id: I69cff2827b1f2cc57e0fec0316b2b997f806787c

Reviewed-on: https://asterix-gerrit.ics.uci.edu/875

Reviewed-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

    • -12
    • +0
    /asterixdb/asterix-external-data/pom.xml
Fixes and tweaks for running Asterix atop new config-management framework.

Change-Id: Ie3027c8c839f25ea858790bd3340187f4b11f213

Reviewed-on: https://asterix-gerrit.ics.uci.edu/874

Reviewed-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

Reviewed-by: Michael Blow <michael.blow@couchbase.com>

ASTERIXDB-1425 & ASTERIXDB-1450: Fix LogReader random reads

- Fix random reads for truncated logs (ASTERIXDB-1425).

- Fix log file partition size boundary check (ASTERIXDB-1450).

- Fix deadlock between LogReader and LogFlusher.

- Prevent checkpoints from deleting log files being accessed by rollbacks.

- Make rollbacks start from LSN = max(txnFirstLSN, minMemoryLSN).

- Make default log partition size 256MB instead of 2GB.

Change-Id: I1c75ca4a7c8fe197451126392389d4baecbd7e45

Reviewed-on: https://asterix-gerrit.ics.uci.edu/867

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@apache.org>

stop application entry point

Change-Id: Ia4a8fab765f4878b30b6f7adb430f01ef2757605

Reviewed-on: https://asterix-gerrit.ics.uci.edu/827

Reviewed-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

fewer copies

Change-Id: Ia409bf61140816c8246ab79c11732654613b20bf

Reviewed-on: https://asterix-gerrit.ics.uci.edu/823

Reviewed-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

Fix Input Handler Test

The test was checking that the data was fully consumed before it closes

the input handler. The test was fixed to close it before it checks.

Change-Id: I6e0e8f130a8250d9cddeb9077f9c4b79491f6fd9

Reviewed-on: https://asterix-gerrit.ics.uci.edu/872

Reviewed-by: Michael Blow <michael.blow@couchbase.com>

Reviewed-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Remove Log-Based Remote Recovery

This change removes the code that supports log-based remote recovery.

Remote recovery was replaced by the failback process which depends on

copying LSM disk component instead of logs.

Change-Id: I86e3b5832b52207e36c8409a072ccbda564d78b5

Reviewed-on: https://asterix-gerrit.ics.uci.edu/871

Reviewed-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

Fix Intermittent Hangs In Recovery Tests

Update recovery script to exact match on target PID

Change-Id: I48d1f1a8c882e6f45d09c37df1c129463f2eaaf4

Reviewed-on: https://asterix-gerrit.ics.uci.edu/869

Reviewed-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

Remove Key-Value Adapter

Change-Id: Iaa7d8d70b0869242d1a872f55f0c6928fda94dcb

Reviewed-on: https://asterix-gerrit.ics.uci.edu/868

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <michael.blow@couchbase.com>

Add Unit Tests for Feed Runtime Input Handler

Change-Id: I7088f489a7d53dee8cf6cdbf5baa7cd8d3884f55

Reviewed-on: https://asterix-gerrit.ics.uci.edu/866

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <michael.blow@couchbase.com>

Merge with master.

    • -68
    • +68
    /asterixdb/asterix-runtime/pom.xml
Fix Apache version

The latest org.apache.apache version seems to break the build.

This sets it from LATEST to 16, which it was at not long ago.

Change-Id: I4b5ad96336a85b2295ff6986fc2c99b27a0ddcf7

Reviewed-on: https://asterix-gerrit.ics.uci.edu/863

Reviewed-by: Michael Blow <michael.blow@couchbase.com>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

ASTERIXDB-1455: Scale CharRecord Buffer Expansions

Reading a 30 MB record requires > 1500 buffer reallocations when increasing by

a fixed amount. Updated to scale up the buffer increases to reduce the number

of reallocations in case of large records.

Change-Id: I4822b2a78e1f411de94d2c717523df6b2b4a381e

Reviewed-on: https://asterix-gerrit.ics.uci.edu/861

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

ASTERIXDB-1287: Fixed Admin Console pages.

From: Pritom Ahmed <pritom.11@gmail.com>:

Previously job details page of the admin console was not working correctly.

Now It is fixed. Also fixed some internal issues like wrong time shown.

Now JSON data in adminconsole for Job Details is shown in tabular form.

Previously, in job details page of admin console, all the data was shown

in raw JSON format. Now using a MIT licensed js library, it is shown in

tabular form.

Added Headers for the data shown in Index page of adminconsole.

Previously in job summary, create time, start time and end time was never

updated. As a result they all were 0. Fixed it.

In adminconsole's Index page, the job create time, start time and end

time all were being assigned with create time. Fixed it. The same

problem was also present in the REST API as well. Also fixed it.

Showing a message in job details page in admin console for jobs that

were deleted because the maximum size of job archive was reached.

Beautified the job details page in adminconsole by adding tabs.

Fixed the job details page in adminconsole. Previously it was broken.

Now it shows the raw JSON data sent to it.

Change-Id: Iefbdd9ab564cff12ab75e00922299fbbcde1c08f

Reviewed-on: https://asterix-gerrit.ics.uci.edu/860

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@apache.org>

Fix Maven Warnings

This change eliminates the following Maven warnings in the build:

- [WARNING] 'parent.relativePath' of POM org.apache.hyracks:apache-asterixdb-hyr

acks:0.2.18-SNAPSHOT (/home/jenkins/workspace/asterix-gerrit-notopic/hyracks-f

ullstack/pom.xml) points at org.apache:apache-asterixdb-fullstack instead of o

rg.apache:apache, please verify your project structure @ org.apache.hyracks:ap

ache-asterixdb-hyracks:0.2.18-SNAPSHOT, /home/jenkins/workspace/asterix-gerrit

-notopic/hyracks-fullstack/pom.xml, line 28, column 11

- [WARNING] 'dependencies.dependency.(groupId:artifactId:type:classifier)' must

be unique: org.apache.hadoop:hadoop-hdfs:jar -> duplicate declaration of versi

on (?) @ org.apache.asterix:asterix-app:[unknown-version], /home/jenkins/works

pace/asterix-gerrit-notopic/asterixdb/asterix-app/pom.xml, line 258, column 17

- [WARNING] 'build.plugins.plugin.(groupId:artifactId)' must be unique but found

duplicate declaration of plugin org.apache.maven.plugins:maven-jar-plugin @ o

rg.apache.asterix:asterix-app:[unknown-version], /home/jenkins/workspace/aster

ix-gerrit-notopic/asterixdb/asterix-app/pom.xml, line 54, column 15

- [WARNING] 'dependencies.dependency.(groupId:artifactId:type:classifier)' must

be unique: org.apache.asterix:asterix-app:test-jar -> duplicate declaration of

version 0.8.9-SNAPSHOT @ org.apache.asterix:asterix-installer:[unknown-versio

n], /home/jenkins/workspace/asterix-gerrit-notopic/asterixdb/asterix-installer

/pom.xml, line 319, column 17

Change-Id: Ie984b7f0ea50b9688399fafdfc48057fdf46012d

Reviewed-on: https://asterix-gerrit.ics.uci.edu/859

Tested-by: Michael Blow <michael.blow@couchbase.com>

Reviewed-by: Till Westmann <tillw@apache.org>

Fix Installer Test Regression For Big Object Tests

copy target/data for installer tests

Change-Id: I012ff4968b97cb93cb3f58580f3ae5e288a5645c

Reviewed-on: https://asterix-gerrit.ics.uci.edu/858

Reviewed-by: Till Westmann <tillw@apache.org>

Tested-by: Michael Blow <michael.blow@couchbase.com>

remove AsterixRuntimeException

Change-Id: Ica9d828bffceabe3b614f68886bc860e34f593b4

Reviewed-on: https://asterix-gerrit.ics.uci.edu/856

Tested-by: Michael Blow <michael.blow@couchbase.com>

Reviewed-by: Michael Blow <michael.blow@couchbase.com>

Make job recovery memory configurable

- Make memory allocated per job during recovery configurable.

- Exclude bloom filter file from LSMBTree with no bloom filter.

Change-Id: Ief359eae2352408c0cbfd37b0e3a1e758c78e0dd

Reviewed-on: https://asterix-gerrit.ics.uci.edu/852

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Young-Seok Kim <kisskys@gmail.com>

ASTERIXDB-1439: Add Test Cases For Big Objects

Two cases involving 20MB values, using load mechanism

Change-Id: I26abeba3db348f08de5b936a791fb3e3814519c9

Reviewed-on: https://asterix-gerrit.ics.uci.edu/849

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <michael.blow@couchbase.com>

    • -0
    • +1
    /asterixdb/asterix-maven-plugins/pom.xml
  1. … 3 more files in changeset.
Remove the not-existing interval-from functions

Change-Id: I9c22aaad20f4ca4ab594160ba69dec913d9956c2

Reviewed-on: https://asterix-gerrit.ics.uci.edu/854

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Preston Carman <prestonc@apache.org>

SONAR: Synchronization On Boxed Primitives

Synchronization should not be based on Strings or boxed primitives

Fixes:

- https://asterix-sonar.ics.uci.edu/issues/search#issues=AVR98lUKuXNBBqjTzg_N

- https://asterix-sonar.ics.uci.edu/issues/search#issues=AVR98lUKuXNBBqjTzg_O

- https://asterix-sonar.ics.uci.edu/issues/search#issues=AVR98lUKuXNBBqjTzg_P

Change-Id: I520582cb5a03d775d696ec5b4390b571322e18f7

Reviewed-on: https://asterix-gerrit.ics.uci.edu/853

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

fix typo

Change-Id: Iabd78359ae7cdb924a87444dd03968e6f428ab8e

Reviewed-on: https://asterix-gerrit.ics.uci.edu/786

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

Run Assembly Plugin Only When Assembly Defined

Enable incremental builds in hyracks by only gating assembly under a

file-presence activation profile.

Change-Id: I97268a79f89b043a9b9b32ef195012a928f5ba9f

Reviewed-on: https://asterix-gerrit.ics.uci.edu/851

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

Cleanup Feed CodeBase

In order to expedite cleaning feeds up and reaching a maintainable

state, we disabled the following:

1. Policies (At least once, throttling, discarding, elasticity).

2. Statistics Reporting.

3. Load management.

4. Feed re-activation upon System reboot.

Right now on master, none of these features work reliably.

We will re-introduce them one feature at a time.

The rules followed in this change:

1. Keep X if X is tested in a test case.

2. Remove X if X is not used in test cases.

After a few meetings with Mike and Till, the policies

1. Buffering

2. Spill

3. Discard

4. Throttle

have been fixed and unit tests have been added.

Change-Id: I545bc4f8560564e4c868a80d27c77a4edd97a8b8

Reviewed-on: https://asterix-gerrit.ics.uci.edu/798

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

  1. … 127 more files in changeset.
ASTERIXDB-1436: Big Object Support For Storage

Preliminary submit for big object storage support (values only).

Enable hyracks frames up to Integer.MAX_VALUE

Change-Id: Ie0a3cb855768dfd0cd1da4a5fc9f62aedebcbc5f

Reviewed-on: https://asterix-gerrit.ics.uci.edu/840

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

  1. … 84 more files in changeset.
ASTERIXDB-1434: fixed RecoveryIT failure test

- Fixed Recovery IT failure test. There has been a lurking bug

which always set the isPrimaryIndex flag to true incorrectly for

secondary LSMBTree indexes. The bug surfaced out from the

recent change which removed a Bloom filter from secondary

LSMBTree indexes.

Change-Id: I85bc3c8445c10da8f2a8c2f334d1eeefb1c0662a

Reviewed-on: https://asterix-gerrit.ics.uci.edu/845

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <michael.blow@couchbase.com>

Run git-commit-id-plugin Once

Only run the git-commit-id-plugin on the asterix-app build, to improve build

performnace. Generate the git.properties directly into the target directory.

Change-Id: I21122d45c9b3e5909ee0634a68dd02df6d6d9313

Reviewed-on: https://asterix-gerrit.ics.uci.edu/844

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@apache.org>

Overhaul of Hyracks configuration management.

Includes Asterix changes to make use of new conf management as a

Hyracks application.

Change-Id: Ie3027c8c839f25ea858790bd3340187f4b11f212

Reviewed-on: https://asterix-gerrit.ics.uci.edu/336

Tested-by: Chris Hillery <ceej@lambda.nu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <michael.blow@couchbase.com>

Reviewed-by: Ian Maxon <imaxon@apache.org>

  1. … 22 more files in changeset.