asterixdb

Clone Tools
  • last updated 19 mins ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
Run Assembly Plugin Only When Assembly Defined

Enable incremental builds in hyracks by only gating assembly under a

file-presence activation profile.

Change-Id: I97268a79f89b043a9b9b32ef195012a928f5ba9f

Reviewed-on: https://asterix-gerrit.ics.uci.edu/851

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

Cleanup Feed CodeBase

In order to expedite cleaning feeds up and reaching a maintainable

state, we disabled the following:

1. Policies (At least once, throttling, discarding, elasticity).

2. Statistics Reporting.

3. Load management.

4. Feed re-activation upon System reboot.

Right now on master, none of these features work reliably.

We will re-introduce them one feature at a time.

The rules followed in this change:

1. Keep X if X is tested in a test case.

2. Remove X if X is not used in test cases.

After a few meetings with Mike and Till, the policies

1. Buffering

2. Spill

3. Discard

4. Throttle

have been fixed and unit tests have been added.

Change-Id: I545bc4f8560564e4c868a80d27c77a4edd97a8b8

Reviewed-on: https://asterix-gerrit.ics.uci.edu/798

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

  1. … 127 more files in changeset.
ASTERIXDB-1436: Big Object Support For Storage

Preliminary submit for big object storage support (values only).

Enable hyracks frames up to Integer.MAX_VALUE

Change-Id: Ie0a3cb855768dfd0cd1da4a5fc9f62aedebcbc5f

Reviewed-on: https://asterix-gerrit.ics.uci.edu/840

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

  1. … 84 more files in changeset.
ASTERIXDB-1434: fixed RecoveryIT failure test

- Fixed Recovery IT failure test. There has been a lurking bug

which always set the isPrimaryIndex flag to true incorrectly for

secondary LSMBTree indexes. The bug surfaced out from the

recent change which removed a Bloom filter from secondary

LSMBTree indexes.

Change-Id: I85bc3c8445c10da8f2a8c2f334d1eeefb1c0662a

Reviewed-on: https://asterix-gerrit.ics.uci.edu/845

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <michael.blow@couchbase.com>

Run git-commit-id-plugin Once

Only run the git-commit-id-plugin on the asterix-app build, to improve build

performnace. Generate the git.properties directly into the target directory.

Change-Id: I21122d45c9b3e5909ee0634a68dd02df6d6d9313

Reviewed-on: https://asterix-gerrit.ics.uci.edu/844

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@apache.org>

Overhaul of Hyracks configuration management.

Includes Asterix changes to make use of new conf management as a

Hyracks application.

Change-Id: Ie3027c8c839f25ea858790bd3340187f4b11f212

Reviewed-on: https://asterix-gerrit.ics.uci.edu/336

Tested-by: Chris Hillery <ceej@lambda.nu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <michael.blow@couchbase.com>

Reviewed-by: Ian Maxon <imaxon@apache.org>

  1. … 22 more files in changeset.
Normalize XML Whitespace

Normalize whitespace in project XML files (e.g. pom.xml), to two-spaces

per indentation level.

Cleanup rewriter error messages for user-defined functions

Change-Id: Ic06e3ff4431508d878e0cc614fbf957887a3c682

Reviewed-on: https://asterix-gerrit.ics.uci.edu/836

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

    • -146
    • +146
    /asterixdb/asterix-algebra/pom.xml
    • -232
    • +232
    /asterixdb/asterix-app/pom.xml
  1. … 153 more files in changeset.
ASTERIXDB-1067: removed Bloomfilters from internal datasets' secondary LSM Btree indexes

Change-Id: I45ee757641e167dfd7ff73436111ba12b335dc93

Reviewed-on: https://asterix-gerrit.ics.uci.edu/843

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@apache.org>

Deadlock-free locking protocol is enabled

- Added EntityCommitProfiler class in TransactionSubsystem.java file:

This profiler takes a report interval (in seconds) parameter and

reports entity level commit count every report interval (in seconds)

only if IS_PROFILE_MODE is set to true. The profiler runs in a separate

thread. However, the profiler thread doesn't start reporting the count

until the entityCommitCount > 0. The profiler can be used to measure

1) IPS (Inserts Per Second) and

2) IIPS (instantaneous IPS) for the every report interval.

Change-Id: Ie58ae2f519baa53599e99b51bd61ea5f8366dafd

Reviewed-on: https://asterix-gerrit.ics.uci.edu/825

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

  1. … 49 more files in changeset.
Fix asterix-docker module

Change-Id: Ifeaa71689f329a2572150ab2457181df0c8ad63b

Reviewed-on: https://asterix-gerrit.ics.uci.edu/842

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <michael.blow@couchbase.com>

Misc. Coverage Fixes

- Collect coverage data for hyracks modules

- Generate aggregate coverage report for SonarQube

Change-Id: Ib44d8ed4fe81d2e26380e1bf6e02b7458910be0a

Reviewed-on: https://asterix-gerrit.ics.uci.edu/841

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@apache.org>

    • -96
    • +132
    /asterixdb/asterix-coverage/pom.xml
ASTERIXDB-1415 Fix Feed Freeze with Indexed Dataset

A feed might cause the system to freeze when connected to

a dataset that has secondary indexes. This case happen if

a frame contains all duplicate records pass the primary index

insert operator. The root cause is that the assign runtime in

the pipeline assumes at least a single record. Hence, a corrupted

tuple will reach the commit operator which will create a bad log

record with a PK of size 0. The log record will cause the log

flusher thread to die.

Change-Id: Ib0fa0bc08cf4224dcf64f83ea72c227f63affc73

Reviewed-on: https://asterix-gerrit.ics.uci.edu/839

Reviewed-by: Till Westmann <tillw@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Set LSMRTreeWithAntiMatterTuples as the default RTree for interal dataset

Change-Id: I19323ef38335409bc914265502ae56655db6d106

Reviewed-on: https://asterix-gerrit.ics.uci.edu/838

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@apache.org>

BufferCache Concurrency Fixes

1. Fix thread-safety issues in ClockPageReplacementStrategy.findVictimByEviction

2. Fix race-condition between page evicition & file deletion

Change-Id: I01b4ab3000ae6f481f226c0df9fe876c6b16c7aa

Reviewed-on: https://asterix-gerrit.ics.uci.edu/835

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

Enable Fully Qualified Class Names for Adapter Data Sources

Change-Id: I49953d4570b828b7cecb2ec926e017d008b13ddc

Reviewed-on: https://asterix-gerrit.ics.uci.edu/837

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

MBR size optimization for point data in R-Tree

Change-Id: Ic48304a221f462c66bb1417f291b0ce4f2ce7581

Reviewed-on: https://asterix-gerrit.ics.uci.edu/831

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@apache.org>

  1. … 31 more files in changeset.
Update the Key Value Reader

Change-Id: I54f8a313c871428b2468b74c0760e8d28e810699

Reviewed-on: https://asterix-gerrit.ics.uci.edu/805

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

    • -6
    • +1
    /asterixdb/asterix-external-data/pom.xml
Add Timestamp to Feed Logs

As per the request of users. This will help determine the time it

takes to ingest data.

Change-Id: I3bb8bb0679108d1eba879d18af1e2119fe7e336a

Reviewed-on: https://asterix-gerrit.ics.uci.edu/833

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

Fix the ASTERIXDB-1336 and ASTERIXDB-1413

Change-Id: Iffc60490aea144cec9abd38349853764024b0ce0

Reviewed-on: https://asterix-gerrit.ics.uci.edu/830

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

ASTERIXDB-1409: fix the type check for functions with two string arguments.

Change-Id: Idd36d9f5ee0f7d6d20afdff2e0669d6be6c6769b

Reviewed-on: https://asterix-gerrit.ics.uci.edu/829

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

ASTERIXDB-1407: let the build branch to broadcast for NestedLoopJoin.

-Change the broadcast branch;

-Fix a bug in SuperActivityOperatorNodePushable;

-Fix jobbuilder to use a fixed location (within query) for operators

with "count=1" constraint;

-Fix OptimizerTest to generate the same directory structure for

actual files as expected files.

-Updates the test query plans.

Change-Id: I0988624406d2f7460f0ee5ac7b4829d81e48c652

Reviewed-on: https://asterix-gerrit.ics.uci.edu/828

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Jianfeng Jia <jianfeng.jia@gmail.com>

  1. … 83 more files in changeset.
Bytecode generation for null-handling in SQL++ functions.

Change-Id: I75d8abfad6b0a8c0d7faaac2e7fc7d4fea638e37

Reviewed-on: https://asterix-gerrit.ics.uci.edu/824

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 21 more files in changeset.
Add List of Supported Adapters to Doc

Change-Id: I2bb98477e144e78e9983d33f9dd2f89a547aeccf

Reviewed-on: https://asterix-gerrit.ics.uci.edu/802

Reviewed-by: Till Westmann <tillw@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Squashed changes for interval index join.

  1. … 39 more files in changeset.
merged interval join into one branch, one commit

Refactored the AbstractComparisonEvaluator.

The refactoring allows the comparison code to be used from other locations,

specifically, for generic interval logic.

Change-Id: I42e0e8cf71207bb862334cd0629e8c024ff0556c

Reviewed-on: https://asterix-gerrit.ics.uci.edu/801

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

  1. … 8 more files in changeset.
ASTERIXDB-1383: reduce the number of lines for "AS".

Change-Id: Ic655bf25701f33230787d175b78bedc489909cb0

Reviewed-on: https://asterix-gerrit.ics.uci.edu/822

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 1705 more files in changeset.
Avoid 1 copy of the data when returning results

Change-Id: I8728b218e8f8e20d3e58be46c704f75ef2288933

Reviewed-on: https://asterix-gerrit.ics.uci.edu/808

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

Remove Unnecessary Flush on Index Drop

Change-Id: I2e41b5d0aee13cb3b2d3a23f064c35d28837da10

Reviewed-on: https://asterix-gerrit.ics.uci.edu/804

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

Reviewed-by: Young-Seok Kim <kisskys@gmail.com>

need just 1 interface method as byte[] is Serializable

Change-Id: I471c800037781b678d2a6ba2ba0281f78623e7bf

Reviewed-on: https://asterix-gerrit.ics.uci.edu/800

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>