asterixdb

Clone Tools
  • last updated 29 mins ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
ASTERIXDB-1425 & ASTERIXDB-1450: Fix LogReader random reads

- Fix random reads for truncated logs (ASTERIXDB-1425).

- Fix log file partition size boundary check (ASTERIXDB-1450).

- Fix deadlock between LogReader and LogFlusher.

- Prevent checkpoints from deleting log files being accessed by rollbacks.

- Make rollbacks start from LSN = max(txnFirstLSN, minMemoryLSN).

- Make default log partition size 256MB instead of 2GB.

Change-Id: I1c75ca4a7c8fe197451126392389d4baecbd7e45

Reviewed-on: https://asterix-gerrit.ics.uci.edu/867

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@apache.org>

stop application entry point

Change-Id: Ia4a8fab765f4878b30b6f7adb430f01ef2757605

Reviewed-on: https://asterix-gerrit.ics.uci.edu/827

Reviewed-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

fewer copies

Change-Id: Ia409bf61140816c8246ab79c11732654613b20bf

Reviewed-on: https://asterix-gerrit.ics.uci.edu/823

Reviewed-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

Fix Input Handler Test

The test was checking that the data was fully consumed before it closes

the input handler. The test was fixed to close it before it checks.

Change-Id: I6e0e8f130a8250d9cddeb9077f9c4b79491f6fd9

Reviewed-on: https://asterix-gerrit.ics.uci.edu/872

Reviewed-by: Michael Blow <michael.blow@couchbase.com>

Reviewed-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Remove Log-Based Remote Recovery

This change removes the code that supports log-based remote recovery.

Remote recovery was replaced by the failback process which depends on

copying LSM disk component instead of logs.

Change-Id: I86e3b5832b52207e36c8409a072ccbda564d78b5

Reviewed-on: https://asterix-gerrit.ics.uci.edu/871

Reviewed-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

Fix Intermittent Hangs In Recovery Tests

Update recovery script to exact match on target PID

Change-Id: I48d1f1a8c882e6f45d09c37df1c129463f2eaaf4

Reviewed-on: https://asterix-gerrit.ics.uci.edu/869

Reviewed-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

Remove Key-Value Adapter

Change-Id: Iaa7d8d70b0869242d1a872f55f0c6928fda94dcb

Reviewed-on: https://asterix-gerrit.ics.uci.edu/868

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <michael.blow@couchbase.com>

Add Unit Tests for Feed Runtime Input Handler

Change-Id: I7088f489a7d53dee8cf6cdbf5baa7cd8d3884f55

Reviewed-on: https://asterix-gerrit.ics.uci.edu/866

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <michael.blow@couchbase.com>

Merge with master.

    • -68
    • +68
    /asterixdb/asterix-runtime/pom.xml
Fix Apache version

The latest org.apache.apache version seems to break the build.

This sets it from LATEST to 16, which it was at not long ago.

Change-Id: I4b5ad96336a85b2295ff6986fc2c99b27a0ddcf7

Reviewed-on: https://asterix-gerrit.ics.uci.edu/863

Reviewed-by: Michael Blow <michael.blow@couchbase.com>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

ASTERIXDB-1455: Scale CharRecord Buffer Expansions

Reading a 30 MB record requires > 1500 buffer reallocations when increasing by

a fixed amount. Updated to scale up the buffer increases to reduce the number

of reallocations in case of large records.

Change-Id: I4822b2a78e1f411de94d2c717523df6b2b4a381e

Reviewed-on: https://asterix-gerrit.ics.uci.edu/861

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

ASTERIXDB-1287: Fixed Admin Console pages.

From: Pritom Ahmed <pritom.11@gmail.com>:

Previously job details page of the admin console was not working correctly.

Now It is fixed. Also fixed some internal issues like wrong time shown.

Now JSON data in adminconsole for Job Details is shown in tabular form.

Previously, in job details page of admin console, all the data was shown

in raw JSON format. Now using a MIT licensed js library, it is shown in

tabular form.

Added Headers for the data shown in Index page of adminconsole.

Previously in job summary, create time, start time and end time was never

updated. As a result they all were 0. Fixed it.

In adminconsole's Index page, the job create time, start time and end

time all were being assigned with create time. Fixed it. The same

problem was also present in the REST API as well. Also fixed it.

Showing a message in job details page in admin console for jobs that

were deleted because the maximum size of job archive was reached.

Beautified the job details page in adminconsole by adding tabs.

Fixed the job details page in adminconsole. Previously it was broken.

Now it shows the raw JSON data sent to it.

Change-Id: Iefbdd9ab564cff12ab75e00922299fbbcde1c08f

Reviewed-on: https://asterix-gerrit.ics.uci.edu/860

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@apache.org>

Fix Maven Warnings

This change eliminates the following Maven warnings in the build:

- [WARNING] 'parent.relativePath' of POM org.apache.hyracks:apache-asterixdb-hyr

acks:0.2.18-SNAPSHOT (/home/jenkins/workspace/asterix-gerrit-notopic/hyracks-f

ullstack/pom.xml) points at org.apache:apache-asterixdb-fullstack instead of o

rg.apache:apache, please verify your project structure @ org.apache.hyracks:ap

ache-asterixdb-hyracks:0.2.18-SNAPSHOT, /home/jenkins/workspace/asterix-gerrit

-notopic/hyracks-fullstack/pom.xml, line 28, column 11

- [WARNING] 'dependencies.dependency.(groupId:artifactId:type:classifier)' must

be unique: org.apache.hadoop:hadoop-hdfs:jar -> duplicate declaration of versi

on (?) @ org.apache.asterix:asterix-app:[unknown-version], /home/jenkins/works

pace/asterix-gerrit-notopic/asterixdb/asterix-app/pom.xml, line 258, column 17

- [WARNING] 'build.plugins.plugin.(groupId:artifactId)' must be unique but found

duplicate declaration of plugin org.apache.maven.plugins:maven-jar-plugin @ o

rg.apache.asterix:asterix-app:[unknown-version], /home/jenkins/workspace/aster

ix-gerrit-notopic/asterixdb/asterix-app/pom.xml, line 54, column 15

- [WARNING] 'dependencies.dependency.(groupId:artifactId:type:classifier)' must

be unique: org.apache.asterix:asterix-app:test-jar -> duplicate declaration of

version 0.8.9-SNAPSHOT @ org.apache.asterix:asterix-installer:[unknown-versio

n], /home/jenkins/workspace/asterix-gerrit-notopic/asterixdb/asterix-installer

/pom.xml, line 319, column 17

Change-Id: Ie984b7f0ea50b9688399fafdfc48057fdf46012d

Reviewed-on: https://asterix-gerrit.ics.uci.edu/859

Tested-by: Michael Blow <michael.blow@couchbase.com>

Reviewed-by: Till Westmann <tillw@apache.org>

Fix Installer Test Regression For Big Object Tests

copy target/data for installer tests

Change-Id: I012ff4968b97cb93cb3f58580f3ae5e288a5645c

Reviewed-on: https://asterix-gerrit.ics.uci.edu/858

Reviewed-by: Till Westmann <tillw@apache.org>

Tested-by: Michael Blow <michael.blow@couchbase.com>

remove AsterixRuntimeException

Change-Id: Ica9d828bffceabe3b614f68886bc860e34f593b4

Reviewed-on: https://asterix-gerrit.ics.uci.edu/856

Tested-by: Michael Blow <michael.blow@couchbase.com>

Reviewed-by: Michael Blow <michael.blow@couchbase.com>

Make job recovery memory configurable

- Make memory allocated per job during recovery configurable.

- Exclude bloom filter file from LSMBTree with no bloom filter.

Change-Id: Ief359eae2352408c0cbfd37b0e3a1e758c78e0dd

Reviewed-on: https://asterix-gerrit.ics.uci.edu/852

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Young-Seok Kim <kisskys@gmail.com>

ASTERIXDB-1439: Add Test Cases For Big Objects

Two cases involving 20MB values, using load mechanism

Change-Id: I26abeba3db348f08de5b936a791fb3e3814519c9

Reviewed-on: https://asterix-gerrit.ics.uci.edu/849

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <michael.blow@couchbase.com>

    • -0
    • +1
    /asterixdb/asterix-maven-plugins/pom.xml
  1. … 3 more files in changeset.
Remove the not-existing interval-from functions

Change-Id: I9c22aaad20f4ca4ab594160ba69dec913d9956c2

Reviewed-on: https://asterix-gerrit.ics.uci.edu/854

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Preston Carman <prestonc@apache.org>

SONAR: Synchronization On Boxed Primitives

Synchronization should not be based on Strings or boxed primitives

Fixes:

- https://asterix-sonar.ics.uci.edu/issues/search#issues=AVR98lUKuXNBBqjTzg_N

- https://asterix-sonar.ics.uci.edu/issues/search#issues=AVR98lUKuXNBBqjTzg_O

- https://asterix-sonar.ics.uci.edu/issues/search#issues=AVR98lUKuXNBBqjTzg_P

Change-Id: I520582cb5a03d775d696ec5b4390b571322e18f7

Reviewed-on: https://asterix-gerrit.ics.uci.edu/853

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

fix typo

Change-Id: Iabd78359ae7cdb924a87444dd03968e6f428ab8e

Reviewed-on: https://asterix-gerrit.ics.uci.edu/786

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

Run Assembly Plugin Only When Assembly Defined

Enable incremental builds in hyracks by only gating assembly under a

file-presence activation profile.

Change-Id: I97268a79f89b043a9b9b32ef195012a928f5ba9f

Reviewed-on: https://asterix-gerrit.ics.uci.edu/851

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

Cleanup Feed CodeBase

In order to expedite cleaning feeds up and reaching a maintainable

state, we disabled the following:

1. Policies (At least once, throttling, discarding, elasticity).

2. Statistics Reporting.

3. Load management.

4. Feed re-activation upon System reboot.

Right now on master, none of these features work reliably.

We will re-introduce them one feature at a time.

The rules followed in this change:

1. Keep X if X is tested in a test case.

2. Remove X if X is not used in test cases.

After a few meetings with Mike and Till, the policies

1. Buffering

2. Spill

3. Discard

4. Throttle

have been fixed and unit tests have been added.

Change-Id: I545bc4f8560564e4c868a80d27c77a4edd97a8b8

Reviewed-on: https://asterix-gerrit.ics.uci.edu/798

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

  1. … 127 more files in changeset.
ASTERIXDB-1436: Big Object Support For Storage

Preliminary submit for big object storage support (values only).

Enable hyracks frames up to Integer.MAX_VALUE

Change-Id: Ie0a3cb855768dfd0cd1da4a5fc9f62aedebcbc5f

Reviewed-on: https://asterix-gerrit.ics.uci.edu/840

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

  1. … 84 more files in changeset.
ASTERIXDB-1434: fixed RecoveryIT failure test

- Fixed Recovery IT failure test. There has been a lurking bug

which always set the isPrimaryIndex flag to true incorrectly for

secondary LSMBTree indexes. The bug surfaced out from the

recent change which removed a Bloom filter from secondary

LSMBTree indexes.

Change-Id: I85bc3c8445c10da8f2a8c2f334d1eeefb1c0662a

Reviewed-on: https://asterix-gerrit.ics.uci.edu/845

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <michael.blow@couchbase.com>

Run git-commit-id-plugin Once

Only run the git-commit-id-plugin on the asterix-app build, to improve build

performnace. Generate the git.properties directly into the target directory.

Change-Id: I21122d45c9b3e5909ee0634a68dd02df6d6d9313

Reviewed-on: https://asterix-gerrit.ics.uci.edu/844

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@apache.org>

Overhaul of Hyracks configuration management.

Includes Asterix changes to make use of new conf management as a

Hyracks application.

Change-Id: Ie3027c8c839f25ea858790bd3340187f4b11f212

Reviewed-on: https://asterix-gerrit.ics.uci.edu/336

Tested-by: Chris Hillery <ceej@lambda.nu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <michael.blow@couchbase.com>

Reviewed-by: Ian Maxon <imaxon@apache.org>

  1. … 22 more files in changeset.
Normalize XML Whitespace

Normalize whitespace in project XML files (e.g. pom.xml), to two-spaces

per indentation level.

Cleanup rewriter error messages for user-defined functions

Change-Id: Ic06e3ff4431508d878e0cc614fbf957887a3c682

Reviewed-on: https://asterix-gerrit.ics.uci.edu/836

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

    • -146
    • +146
    /asterixdb/asterix-algebra/pom.xml
    • -232
    • +232
    /asterixdb/asterix-app/pom.xml
  1. … 153 more files in changeset.
ASTERIXDB-1067: removed Bloomfilters from internal datasets' secondary LSM Btree indexes

Change-Id: I45ee757641e167dfd7ff73436111ba12b335dc93

Reviewed-on: https://asterix-gerrit.ics.uci.edu/843

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@apache.org>

Deadlock-free locking protocol is enabled

- Added EntityCommitProfiler class in TransactionSubsystem.java file:

This profiler takes a report interval (in seconds) parameter and

reports entity level commit count every report interval (in seconds)

only if IS_PROFILE_MODE is set to true. The profiler runs in a separate

thread. However, the profiler thread doesn't start reporting the count

until the entityCommitCount > 0. The profiler can be used to measure

1) IPS (Inserts Per Second) and

2) IIPS (instantaneous IPS) for the every report interval.

Change-Id: Ie58ae2f519baa53599e99b51bd61ea5f8366dafd

Reviewed-on: https://asterix-gerrit.ics.uci.edu/825

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

  1. … 49 more files in changeset.
Fix asterix-docker module

Change-Id: Ifeaa71689f329a2572150ab2457181df0c8ad63b

Reviewed-on: https://asterix-gerrit.ics.uci.edu/842

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <michael.blow@couchbase.com>