asterixdb

Clone Tools
  • last updated 22 mins ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
Make job recovery memory configurable

- Make memory allocated per job during recovery configurable.

- Exclude bloom filter file from LSMBTree with no bloom filter.

Change-Id: Ief359eae2352408c0cbfd37b0e3a1e758c78e0dd

Reviewed-on: https://asterix-gerrit.ics.uci.edu/852

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Young-Seok Kim <kisskys@gmail.com>

ASTERIXDB-1439: Add Test Cases For Big Objects

Two cases involving 20MB values, using load mechanism

Change-Id: I26abeba3db348f08de5b936a791fb3e3814519c9

Reviewed-on: https://asterix-gerrit.ics.uci.edu/849

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <michael.blow@couchbase.com>

    • -0
    • +1
    /asterixdb/asterix-maven-plugins/pom.xml
  1. … 3 more files in changeset.
Remove the not-existing interval-from functions

Change-Id: I9c22aaad20f4ca4ab594160ba69dec913d9956c2

Reviewed-on: https://asterix-gerrit.ics.uci.edu/854

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Preston Carman <prestonc@apache.org>

SONAR: Synchronization On Boxed Primitives

Synchronization should not be based on Strings or boxed primitives

Fixes:

- https://asterix-sonar.ics.uci.edu/issues/search#issues=AVR98lUKuXNBBqjTzg_N

- https://asterix-sonar.ics.uci.edu/issues/search#issues=AVR98lUKuXNBBqjTzg_O

- https://asterix-sonar.ics.uci.edu/issues/search#issues=AVR98lUKuXNBBqjTzg_P

Change-Id: I520582cb5a03d775d696ec5b4390b571322e18f7

Reviewed-on: https://asterix-gerrit.ics.uci.edu/853

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

fix typo

Change-Id: Iabd78359ae7cdb924a87444dd03968e6f428ab8e

Reviewed-on: https://asterix-gerrit.ics.uci.edu/786

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

Run Assembly Plugin Only When Assembly Defined

Enable incremental builds in hyracks by only gating assembly under a

file-presence activation profile.

Change-Id: I97268a79f89b043a9b9b32ef195012a928f5ba9f

Reviewed-on: https://asterix-gerrit.ics.uci.edu/851

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

Cleanup Feed CodeBase

In order to expedite cleaning feeds up and reaching a maintainable

state, we disabled the following:

1. Policies (At least once, throttling, discarding, elasticity).

2. Statistics Reporting.

3. Load management.

4. Feed re-activation upon System reboot.

Right now on master, none of these features work reliably.

We will re-introduce them one feature at a time.

The rules followed in this change:

1. Keep X if X is tested in a test case.

2. Remove X if X is not used in test cases.

After a few meetings with Mike and Till, the policies

1. Buffering

2. Spill

3. Discard

4. Throttle

have been fixed and unit tests have been added.

Change-Id: I545bc4f8560564e4c868a80d27c77a4edd97a8b8

Reviewed-on: https://asterix-gerrit.ics.uci.edu/798

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

  1. … 127 more files in changeset.
ASTERIXDB-1436: Big Object Support For Storage

Preliminary submit for big object storage support (values only).

Enable hyracks frames up to Integer.MAX_VALUE

Change-Id: Ie0a3cb855768dfd0cd1da4a5fc9f62aedebcbc5f

Reviewed-on: https://asterix-gerrit.ics.uci.edu/840

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

  1. … 84 more files in changeset.
ASTERIXDB-1434: fixed RecoveryIT failure test

- Fixed Recovery IT failure test. There has been a lurking bug

which always set the isPrimaryIndex flag to true incorrectly for

secondary LSMBTree indexes. The bug surfaced out from the

recent change which removed a Bloom filter from secondary

LSMBTree indexes.

Change-Id: I85bc3c8445c10da8f2a8c2f334d1eeefb1c0662a

Reviewed-on: https://asterix-gerrit.ics.uci.edu/845

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <michael.blow@couchbase.com>

Run git-commit-id-plugin Once

Only run the git-commit-id-plugin on the asterix-app build, to improve build

performnace. Generate the git.properties directly into the target directory.

Change-Id: I21122d45c9b3e5909ee0634a68dd02df6d6d9313

Reviewed-on: https://asterix-gerrit.ics.uci.edu/844

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@apache.org>

Overhaul of Hyracks configuration management.

Includes Asterix changes to make use of new conf management as a

Hyracks application.

Change-Id: Ie3027c8c839f25ea858790bd3340187f4b11f212

Reviewed-on: https://asterix-gerrit.ics.uci.edu/336

Tested-by: Chris Hillery <ceej@lambda.nu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <michael.blow@couchbase.com>

Reviewed-by: Ian Maxon <imaxon@apache.org>

  1. … 22 more files in changeset.
Normalize XML Whitespace

Normalize whitespace in project XML files (e.g. pom.xml), to two-spaces

per indentation level.

Cleanup rewriter error messages for user-defined functions

Change-Id: Ic06e3ff4431508d878e0cc614fbf957887a3c682

Reviewed-on: https://asterix-gerrit.ics.uci.edu/836

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

    • -146
    • +146
    /asterixdb/asterix-algebra/pom.xml
    • -232
    • +232
    /asterixdb/asterix-app/pom.xml
  1. … 153 more files in changeset.
ASTERIXDB-1067: removed Bloomfilters from internal datasets' secondary LSM Btree indexes

Change-Id: I45ee757641e167dfd7ff73436111ba12b335dc93

Reviewed-on: https://asterix-gerrit.ics.uci.edu/843

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@apache.org>

Deadlock-free locking protocol is enabled

- Added EntityCommitProfiler class in TransactionSubsystem.java file:

This profiler takes a report interval (in seconds) parameter and

reports entity level commit count every report interval (in seconds)

only if IS_PROFILE_MODE is set to true. The profiler runs in a separate

thread. However, the profiler thread doesn't start reporting the count

until the entityCommitCount > 0. The profiler can be used to measure

1) IPS (Inserts Per Second) and

2) IIPS (instantaneous IPS) for the every report interval.

Change-Id: Ie58ae2f519baa53599e99b51bd61ea5f8366dafd

Reviewed-on: https://asterix-gerrit.ics.uci.edu/825

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

  1. … 49 more files in changeset.
Fix asterix-docker module

Change-Id: Ifeaa71689f329a2572150ab2457181df0c8ad63b

Reviewed-on: https://asterix-gerrit.ics.uci.edu/842

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <michael.blow@couchbase.com>

Misc. Coverage Fixes

- Collect coverage data for hyracks modules

- Generate aggregate coverage report for SonarQube

Change-Id: Ib44d8ed4fe81d2e26380e1bf6e02b7458910be0a

Reviewed-on: https://asterix-gerrit.ics.uci.edu/841

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@apache.org>

    • -96
    • +132
    /asterixdb/asterix-coverage/pom.xml
ASTERIXDB-1415 Fix Feed Freeze with Indexed Dataset

A feed might cause the system to freeze when connected to

a dataset that has secondary indexes. This case happen if

a frame contains all duplicate records pass the primary index

insert operator. The root cause is that the assign runtime in

the pipeline assumes at least a single record. Hence, a corrupted

tuple will reach the commit operator which will create a bad log

record with a PK of size 0. The log record will cause the log

flusher thread to die.

Change-Id: Ib0fa0bc08cf4224dcf64f83ea72c227f63affc73

Reviewed-on: https://asterix-gerrit.ics.uci.edu/839

Reviewed-by: Till Westmann <tillw@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Set LSMRTreeWithAntiMatterTuples as the default RTree for interal dataset

Change-Id: I19323ef38335409bc914265502ae56655db6d106

Reviewed-on: https://asterix-gerrit.ics.uci.edu/838

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@apache.org>

BufferCache Concurrency Fixes

1. Fix thread-safety issues in ClockPageReplacementStrategy.findVictimByEviction

2. Fix race-condition between page evicition & file deletion

Change-Id: I01b4ab3000ae6f481f226c0df9fe876c6b16c7aa

Reviewed-on: https://asterix-gerrit.ics.uci.edu/835

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

Enable Fully Qualified Class Names for Adapter Data Sources

Change-Id: I49953d4570b828b7cecb2ec926e017d008b13ddc

Reviewed-on: https://asterix-gerrit.ics.uci.edu/837

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

MBR size optimization for point data in R-Tree

Change-Id: Ic48304a221f462c66bb1417f291b0ce4f2ce7581

Reviewed-on: https://asterix-gerrit.ics.uci.edu/831

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@apache.org>

  1. … 31 more files in changeset.
Update the Key Value Reader

Change-Id: I54f8a313c871428b2468b74c0760e8d28e810699

Reviewed-on: https://asterix-gerrit.ics.uci.edu/805

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

    • -6
    • +1
    /asterixdb/asterix-external-data/pom.xml
Add Timestamp to Feed Logs

As per the request of users. This will help determine the time it

takes to ingest data.

Change-Id: I3bb8bb0679108d1eba879d18af1e2119fe7e336a

Reviewed-on: https://asterix-gerrit.ics.uci.edu/833

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

Fix the ASTERIXDB-1336 and ASTERIXDB-1413

Change-Id: Iffc60490aea144cec9abd38349853764024b0ce0

Reviewed-on: https://asterix-gerrit.ics.uci.edu/830

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

ASTERIXDB-1409: fix the type check for functions with two string arguments.

Change-Id: Idd36d9f5ee0f7d6d20afdff2e0669d6be6c6769b

Reviewed-on: https://asterix-gerrit.ics.uci.edu/829

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

ASTERIXDB-1407: let the build branch to broadcast for NestedLoopJoin.

-Change the broadcast branch;

-Fix a bug in SuperActivityOperatorNodePushable;

-Fix jobbuilder to use a fixed location (within query) for operators

with "count=1" constraint;

-Fix OptimizerTest to generate the same directory structure for

actual files as expected files.

-Updates the test query plans.

Change-Id: I0988624406d2f7460f0ee5ac7b4829d81e48c652

Reviewed-on: https://asterix-gerrit.ics.uci.edu/828

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Jianfeng Jia <jianfeng.jia@gmail.com>

  1. … 83 more files in changeset.
Bytecode generation for null-handling in SQL++ functions.

Change-Id: I75d8abfad6b0a8c0d7faaac2e7fc7d4fea638e37

Reviewed-on: https://asterix-gerrit.ics.uci.edu/824

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 21 more files in changeset.
Add List of Supported Adapters to Doc

Change-Id: I2bb98477e144e78e9983d33f9dd2f89a547aeccf

Reviewed-on: https://asterix-gerrit.ics.uci.edu/802

Reviewed-by: Till Westmann <tillw@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Squashed changes for interval index join.

  1. … 39 more files in changeset.
merged interval join into one branch, one commit