hyracks

Clone Tools
  • last updated 21 mins ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
moved files for the merge

  1. … 162 more files in changeset.
great merge - hyracks

  1. … 64 more files in changeset.
add min and max to range map.

Move merged files

  1. … 1 more file in changeset.
Range partition has been updated with new tests.

  1. … 4 more files in changeset.
Add Compatibility for FileRemoveOperatorDescriptor

Change-Id: Ic08ba645db6936df3c4c59be0e104dbb18088370

Reviewed-on: https://asterix-gerrit.ics.uci.edu/784

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

Enable Remove Quietly for Remove Directory Jobs

Change-Id: I00a60bf4ac6d64295224c2cebc28fb22f92e77fd

Reviewed-on: https://asterix-gerrit.ics.uci.edu/775

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

Exclude Temporary Indexes From Replication

- Exclude non-durable LSM indexes from replication.

- Stop heartbeat task after NC has stopped to avoid false failures detection.

Change-Id: Icce91a203e04cb068a7a5aa541720bbd0289eacb

Reviewed-on: https://asterix-gerrit.ics.uci.edu/777

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

ASTERIXDB-1377: Prevent Operations from Exiting Components Multiple Times

Change-Id: I5907b1b4c76ce48366f5447f2940f3561c474cfe

Reviewed-on: https://asterix-gerrit.ics.uci.edu/769

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

Move Hyracks to subfolder

  1. … 4413 more files in changeset.
CR-LF inconsistencies

    • -7308
    • +7308
    ./hyracks-dataflow-std/src/test/resources/data/beer.txt
    • -13052
    • +13052
    ./hyracks-examples/text-example/textserver/data/file1.txt
    • -10216
    • +10216
    ./hyracks-examples/text-example/textserver/data/file2.txt
Interator with delete skip.

Provide a Way to Create UTF8 Serde with Reader and Writer

Change-Id: Ia1a0a2f240ab780e21e7170e03767d8e9981899c

Reviewed-on: https://asterix-gerrit.ics.uci.edu/757

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

Fixed Bugs Caused by Disabling Installer Tests

Change-Id: Ic77f4ee76a77f4b49cac6835bbb976eb9dc92d3a

Reviewed-on: https://asterix-gerrit.ics.uci.edu/736

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@apache.org>

Merge branch 'release-0.2.17' into master

Change-Id: I8fcf41c009f9faf51bc8bccc0c3c7217b7769147

    • -5
    • +7
    ./hyracks-data/hyracks-data-std/pom.xml
    • -15
    • +18
    ./hyracks-dataflow-common/pom.xml
  1. … 46 more files in changeset.
Update for tuple based memory iterator.

  1. … 3 more files in changeset.
Add Another Signature to Send Messages to CC

The existing interface to send messages from NC

to CC expects a byte[]. The new method takes

a Serializable and serialize it.

Change-Id: I0ec4e0898c341723f29af32043cec8c042377ed9

Reviewed-on: https://asterix-gerrit.ics.uci.edu/727

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

Support Change Feeds and Ingestion of Records with MetaData

This change allows feeds to perform upserts and deletes

in order to perform replication of an external data source.

The change does so by performing the following:

1. The adapter produces [PK][Record]. (Record == null --> delete)

2. The insert is replaced by an upsert operator.

Change-Id: I3749349e2b9f1b03c8b310eb99d3f44d08be77df

Reviewed-on: https://asterix-gerrit.ics.uci.edu/620

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -0
    • +7308
    ./hyracks-dataflow-std/src/test/resources/data/beer.txt
  1. … 15 more files in changeset.
Fix the ExternalGroupby for fuzzy join

Change-Id: I5a3852d08f9a3e2738ea2952f05aa082a2e6dbbb

Reviewed-on: https://asterix-gerrit.ics.uci.edu/713

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

ASTERIXDB-1337: Dataset Memory Management on Multi-Partition NC

As sugggested in ASTERIXDB-1337, this change is to maintain a per-

partition MultitenantVirtualBufferCache budget, as opposed to sharing

the budget across the dataset.

Change-Id: Ie554ab6d9b0d01ed149ab0729d20cdac3c969ec1

Reviewed-on: https://asterix-gerrit.ics.uci.edu/711

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

a few formatting issues.

name change updates from merge.

ASTERIXDB-1330: fix NPE in ExternalGroupByOperator.

Change-Id: I2279221abbef1440179a31df180a24f6a642c641

Reviewed-on: https://asterix-gerrit.ics.uci.edu/687

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@apache.org>

Misc. Coverity Issues

Clean up a few issues encountered in Coverity scan

Change-Id: Ie90a91682ae0ee2443fce1677fca0e00fa45c5f0

Reviewed-on: https://asterix-gerrit.ics.uci.edu/686

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

  1. … 2 more files in changeset.
Fix Double-Checked Locking (Coverity)

Fix CIDs 68208,68209,68210 - eliminate problematic double-checked locking

Change-Id: I3383035b5e8e0aa3cfd54f803f12d56090f006cf

Reviewed-on: https://asterix-gerrit.ics.uci.edu/667

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@apache.org>

Implemented the memory-bounded HashGroupby and HashJoin for BigObject

It contains both hash grouby and hash join changes.

The main change is

1. update the ExternalGroupby to Hash-based groupby

2. update the Join operators to use the Buffermanager.

The buffer manager part is moved from the Sort package to upper

level so that it can be shared by all the operators.

Change-Id: I248f3a374fdacad7d57e49cf18d8233745e55460

Reviewed-on: https://asterix-gerrit.ics.uci.edu/398

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

  1. … 133 more files in changeset.
More Coverity Fixes

- CID 68210 (#1 of 1): Check of thread-shared field evades lock

acquisition (LOCK_EVASION)

6. thread1_overwrites_value_in_field: Thread1 sets index to a new

value. Now the two threads have an inconsistent view of index and

updates to fields of index or fields correlated with index may be

lost.

- CID 68209 (#1 of 1): Check of thread-shared field evades lock

acquisition (LOCK_EVASION)

6. thread1_overwrites_value_in_field: Thread1 sets index to a new

value. Now the two threads have an inconsistent view of index and

updates to fields of index or fields correlated with index may be

lost.

- CID 68208 (#1 of 1): Check of thread-shared field evades lock

acquisition (LOCK_EVASION)

6. thread1_overwrites_value_in_field: Thread1 sets index to a new

value. Now the two threads have an inconsistent view of index and

updates to fields of index or fields correlated with index may be

lost.

- CID 68435 (#1 of 1): UR: Uninitialized read of field in constructor

(FB.UR_UNINIT_READ)

1. defect: Uninitialized read of btreeCmp.

- CID 68416 (#1 of 1): UL: Lock not released on all paths

(FB.UL_UNRELEASED_LOCK_EXCEPTION_PATH)

1. defect:

org.apache.hyracks.storage.am.lsm.invertedindex.inmemory.PartitionedIn

MemoryInvertedIndex.updatePartitionIndexes(short) does not release

lock on all exception paths.

- CID 68415 (#1 of 1): UL: Lock not released on all paths

(FB.UL_UNRELEASED_LOCK)

1. defect:

org.apache.hyracks.storage.common.buffercache.BufferCache.sweepAndFlus

h(int, boolean) does not release lock on all paths.

- CID 68414 (#1 of 1): UL: Lock not released on all paths

(FB.UL_UNRELEASED_LOCK)

1. defect:

org.apache.hyracks.storage.common.buffercache.BufferCache.dumpState()

does not release lock on all paths.

- CID 68412 (#1 of 1): UL: Lock not released on all paths

(FB.UL_UNRELEASED_LOCK)

1. defect:

org.apache.hyracks.storage.am.lsm.common.impls.VirtualBufferCache.dele

teFile(int, boolean) does not release lock on all paths.

- CID 68337 (#1 of 1): RV: Bad use of return value

(FB.RV_ABSOLUTE_VALUE_OF_RANDOM_INT)

1. defect: Bad attempt to compute absolute value of signed random

integer.

- CID 68336 (#1 of 1): RV: Bad use of return value

(FB.RV_ABSOLUTE_VALUE_OF_RANDOM_INT)

1. defect: Bad attempt to compute absolute value of signed random

integer.

- CID 68335 (#1 of 1): RV: Bad use of return value

(FB.RV_ABSOLUTE_VALUE_OF_RANDOM_INT)

1. defect: Bad attempt to compute absolute value of signed random

integer.

- CID 68334 (#1 of 1): RV: Bad use of return value

(FB.RV_ABSOLUTE_VALUE_OF_RANDOM_INT)

1. defect: Bad attempt to compute absolute value of signed random

integer.

- CID 68259 (#1 of 1): Result is not floating-point

(UNINTENDED_INTEGER_DIVISION)

integer_division: Dividing integer expressions 2147483647 and 2, and

then converting the integer quotient to type double. Any remainder,

or fractional part of the quotient, is ignored.

Change-Id: I0f63f7fb75b1ac8ffc1fc1d70e66bb19b788fe8e

Reviewed-on: https://asterix-gerrit.ics.uci.edu/661

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@apache.org>

Fix Remove File Operator

The operator didn't get the correct path of the folder to be

deleted. After the fix, the operator gets the correct path

from the IO manager.

Change-Id: I5910409d79e9c1f8e7192a1050b2be0f823e912f

Reviewed-on: https://asterix-gerrit.ics.uci.edu/659

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

Fixes for Coverity Issues

(https://scan.coverity.com/projects/hyracks)

- CID 68257 (#1 of 1): Resource leak (RESOURCE_LEAK)

5. leaked_resource: Variable outputStream going out of scope leaks the

resource it refers to.

- CID 68260 (#1 of 1): Useless call (USELESS_CALL)

side_effect_free: Calling equivalentVars.get(0) is only useful for its

return value, which is ignored.

- CID 68261 (#1 of 1): BC: Bad casts of object references

(FB.BC_EQUALS_METHOD_SHOULD_WORK_FOR_ALL_OBJECTS)

1. defect: Equals method for org.apache.hyracks.algebricks.core.

algebra.properties.LocalOrderProperty assumes the argument is of type

LocalOrderProperty.

- CID 68262 (#1 of 1): BC: Bad casts of object references

(FB.BC_EQUALS_METHOD_SHOULD_WORK_FOR_ALL_OBJECTS)

1. defect: Equals method for org.apache.hyracks.storage.am.rtree.

RTreeCheckTuple assumes the argument is of type RTreeCheckTuple.

Change-Id: I8ba6e578f173b1081a8f55b228704744f08588df

Reviewed-on: https://asterix-gerrit.ics.uci.edu/658

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

Reviewed-by: Till Westmann <tillw@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 2 more files in changeset.
Remove Unneeded Suppress Warnings

When the classes in this change were created, we were using

deprecated Hadoop library. Since we updated the library,

there is no need to suppress deprecation warnings.

Change-Id: I710c1efbe006ee6de22299f8e6e833a566fb884d

Reviewed-on: https://asterix-gerrit.ics.uci.edu/656

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

Reviewed-by: Till Westmann <tillw@apache.org>

Reviewed-by: Michael Blow <michael.blow@couchbase.com>