Clone Tools
  • last updated 28 mins ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
Exclude Temporary Indexes From Replication

- Exclude non-durable LSM indexes from replication.

- Stop heartbeat task after NC has stopped to avoid false failures detection.

Change-Id: Icce91a203e04cb068a7a5aa541720bbd0289eacb

Reviewed-on: https://asterix-gerrit.ics.uci.edu/777

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

  1. … 3 more files in changeset.
Move Hyracks to subfolder

    • -405
    • +0
    ./org/apache/hyracks/control/nc/Task.java
    • -37
    • +0
    ./org/apache/hyracks/control/nc/dataset/Page.java
    • -94
    • +0
    ./org/apache/hyracks/control/nc/io/FileHandle.java
    • -291
    • +0
    ./org/apache/hyracks/control/nc/io/IOManager.java
  1. … 4413 more files in changeset.
Add Another Signature to Send Messages to CC

The existing interface to send messages from NC

to CC expects a byte[]. The new method takes

a Serializable and serialize it.

Change-Id: I0ec4e0898c341723f29af32043cec8c042377ed9

Reviewed-on: https://asterix-gerrit.ics.uci.edu/727

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

    • -0
    • +7
    ./org/apache/hyracks/control/nc/Task.java
  1. … 2 more files in changeset.
Misc. Coverity Issues

Clean up a few issues encountered in Coverity scan

Change-Id: Ie90a91682ae0ee2443fce1677fca0e00fa45c5f0

Reviewed-on: https://asterix-gerrit.ics.uci.edu/686

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

  1. … 6 more files in changeset.
Remove Trailing White Spaces

This change removes all trailing white spaces in

java files in hyracks

Change-Id: I1c2ea4ff1190a965a857c0c3760cbebdf21860b3

Reviewed-on: https://asterix-gerrit.ics.uci.edu/645

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <michael.blow@couchbase.com>

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

    • -1
    • +1
    ./org/apache/hyracks/control/nc/Task.java
    • -1
    • +1
    ./org/apache/hyracks/control/nc/io/FileHandle.java
  1. … 643 more files in changeset.
Support Sending Messages Alongside Frame Data

This change support sending messages with records. The tuple Appender

reserves 100 bytes for a message. Before sending the frame, it appends

The message in the last tuple position. The message is read from the

task context as the shared object between different operators in the

pipeline. The first use of this feature will be within feeds to request

acks for at least once semantics.

Change-Id: I56ae8124052c13a52ca42965b8d00e18ecf35a28

Reviewed-on: https://asterix-gerrit.ics.uci.edu/604

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <michael.blow@couchbase.com>

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

    • -0
    • +12
    ./org/apache/hyracks/control/nc/Task.java
  1. … 18 more files in changeset.
Intersect the 2ndary indexes before primary search

The following commits from your working branch will be included:

Change-Id: Ic16c67c529ca19d8b1a5439ddef22760945fd0d7

Reviewed-on: https://asterix-gerrit.ics.uci.edu/577

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

  1. … 34 more files in changeset.
remove end-of-line whitespace

Change-Id: If3542874d7f3a536a6fe524aba2fffaf6447a614

Reviewed-on: https://asterix-gerrit.ics.uci.edu/600

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

  1. … 252 more files in changeset.
Add flush() to IFrameWriter

This change introduces flush() method in frame writers. It is

intended to be used for long running jobs (Feeds) to flush contents

of frames all the way to storage.

Change-Id: I85424bab7965b71aac709280af066e1655457aa3

Reviewed-on: https://asterix-gerrit.ics.uci.edu/584

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

  1. … 80 more files in changeset.
Fix Error in Aborting Task in Super Activity

When aborting a task, its thread gets interrupted. This creates

a problem when interrupting

Change-Id: I603d3c101e0a4de4816eb5a6a7fd4320df317ce4

Reviewed-on: https://asterix-gerrit.ics.uci.edu/582

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

    • -1
    • +1
    ./org/apache/hyracks/control/nc/Task.java
  1. … 1 more file in changeset.
Make Index Path Relative to NC IO Devices

This change includes the following:

- Store relative index path in persisted local resource as resource name.

- Fetch index absolute path based on FileSplit IO device and index relative path.

This change will enable a node to perform operations on an index that was created

on another node by finding the absolve path based on the node IO deviec and

the index relative path. This is needed in case a node leaves the cluster and

a replica node is assigned to take over and recover the index partitions

that belonged to the failed node.

Change-Id: I09fabd097b8a995ca959fe60286acdb43edfcfa1

Reviewed-on: https://asterix-gerrit.ics.uci.edu/563

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

  1. … 19 more files in changeset.
Changes required to support MessageBroker implementation

This change includes the following:

- API changes to support MessageBroker implementation.

- IResourceIdFactory interface to support application dependent implementation.

Change-Id: Ib9f49234eebe912c48e7f71980433a9b42595741

Reviewed-on: https://asterix-gerrit.ics.uci.edu/485

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

    • -2
    • +2
    ./org/apache/hyracks/control/nc/Task.java
  1. … 18 more files in changeset.
Changed the IFrameWriter Contract

Updated existing operators and added a test case for BTreeSearchOperatorNodePushable.

With this change, calling the open method itself moves it to the open state and

hence, close must be called.

Change-Id: I03da090002f79f4db7b5b31454ce3ac2b9e40c7f

Reviewed-on: https://asterix-gerrit.ics.uci.edu/551

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

  1. … 41 more files in changeset.
ASTERIXDB-1206: call initialize()/deinitialize() in parallel for OperatorNodePushables in SuperActivityOperatorNodePushable.

Change-Id: I8700d5258d658ebbf711b4233bb0def1e8cf7c39

Reviewed-on: https://asterix-gerrit.ics.uci.edu/526

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Jianfeng Jia <jianfeng.jia@gmail.com>

    • -11
    • +14
    ./org/apache/hyracks/control/nc/Task.java
  1. … 4 more files in changeset.
Make LSM bulkload append-only and write-once.

Allows for usage of LSM indexes with underlying storage that is append-only.

This also results in a small improvement for LSM component bulk load speed.

- Tree metadata (filters, etc) now lie at the back of the tree file in

append-only mode.

-- Note that you should *not* ever give the append-only flag on bulk-load,

if the tree is ever to be modified in place.

- Append-only operations bypass the buffer cache for writes, but utilize

the buffer cache for memory allocation and reads.

- Addresses ASTERIXDB-1059

Change-Id: I80fb891b5310252143854a336b591bf3f8cd4ba7

Reviewed-on: https://asterix-gerrit.ics.uci.edu/255

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Young-Seok Kim <kisskys@gmail.com>

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

  1. … 96 more files in changeset.
ASTERIXDB-1137: Prevent file from being deleted while a page is being flushed.

Change-Id: I256a84c416bf9d7251b1ee788d028d6b2928e747

Reviewed-on: https://asterix-gerrit.ics.uci.edu/499

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@apache.org>

  1. … 1 more file in changeset.
ASTERIXDB-1145: Fix error propagating in operators/connectors: 1. When an AbstractUnarySourceOperator instance runs into an exception, it should call writer.fail() first and then throw the exception. 2. An IFrameWriter.fail() implementation should not throw yet-another exception, instead, it should just propgate the failure to its downstream operators and optionally set a "failed" state so that in the close()/nextFrame() method it can potentially behave differently from usual close()/nextFrame().

Change-Id: Ifb538155423687c4aa01a0485adeaab87f291547

Reviewed-on: https://asterix-gerrit.ics.uci.edu/491

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 4 more files in changeset.
ASTERIXDB-1002: Fix exception handling in EmptyTupleSourceRuntimeFactory

Revisiting the previous fix by calling fail() on pipeline

Change-Id: I19f8c8485e483e4d4efeff939e6bd82c7a04a101

Reviewed-on: https://asterix-gerrit.ics.uci.edu/443

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

Reviewed-by: Ian Maxon <imaxon@apache.org>

  1. … 2 more files in changeset.
ASTERIXDB-139: Add temp workspace files deletion to IOManager

This change includes the following:

- Add a method to delete temp workspace files (WAF)

- Expose LSMComponents files suffixes to Asterix

Change-Id: I760074764755e7aee100ff33c14b13bf4b29ec2e

Reviewed-on: https://asterix-gerrit.ics.uci.edu/337

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

    • -4
    • +22
    ./org/apache/hyracks/control/nc/io/IOManager.java
  1. … 22 more files in changeset.
Let a run file reader have an option to delete the run file after it is read.

Change-Id: Iabbd7c3e00489e2dbd8b1d4b87c7c9f81e8116b9

Reviewed-on: https://asterix-gerrit.ics.uci.edu/397

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Jianfeng Jia <jianfeng.jia@gmail.com>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 16 more files in changeset.
Set the priority of the following threads to be Thread.MAX_PRIORITY: 1. heartbeat thread at NC 2. IPC network thread 3. work queue thread in CC

Change-Id: I4e53a85e21a6bdee48a3ca8d004569700f911fbd

Reviewed-on: https://asterix-gerrit.ics.uci.edu/389

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 5 more files in changeset.
Change license headers

Change-Id: I98b18f24a20dcd8dc75e828e47fb0ab88179a5be

Reviewed-on: https://asterix-gerrit.ics.uci.edu/386

Reviewed-by: Till Westmann <tillw@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -12
    • +16
    ./org/apache/hyracks/control/nc/NCDriver.java
    • -12
    • +16
    ./org/apache/hyracks/control/nc/Task.java
    • -12
    • +16
    ./org/apache/hyracks/control/nc/dataset/Page.java
    • -12
    • +16
    ./org/apache/hyracks/control/nc/io/FileHandle.java
    • -12
    • +16
    ./org/apache/hyracks/control/nc/io/IOManager.java
  1. … 2055 more files in changeset.
Change folder structure for Java repackage

Change only the folders, not the files, for our package name change.

This will break the build, and needs to be followed by a change to

the package name in all of the source files. However performing

the folder move and file change in two steps lets Git understand

that the files are the same, and lets us track revisions across

those files.

Change-Id: I08aff75e25ac7c6298c32cf3402febbc4a318c2a

Reviewed-on: https://asterix-gerrit.ics.uci.edu/307

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Chris Hillery <ceej@lambda.nu>

    • -341
    • +0
    ./edu/uci/ics/hyracks/control/nc/Joblet.java
    • -379
    • +0
    ./edu/uci/ics/hyracks/control/nc/Task.java
    • -33
    • +0
    ./edu/uci/ics/hyracks/control/nc/dataset/Page.java
    • -255
    • +0
    ./edu/uci/ics/hyracks/control/nc/io/IOManager.java
  1. … 3879 more files in changeset.
Change Java package from edu.uci.ics to org.apache

Change-Id: I99172d856e88954b00cf7cfb24d33bb400f53994

Reviewed-on: https://asterix-gerrit.ics.uci.edu/308

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

    • -33
    • +33
    ./org/apache/hyracks/control/nc/Task.java
    • -4
    • +4
    ./org/apache/hyracks/control/nc/io/FileHandle.java
  1. … 2019 more files in changeset.
Add Apache RAT License Auditor plugin, and fix missing licenses

Change-Id: I39d92ec6654c73b4e6b8ba76dd66770bb60c7b79

Reviewed-on: https://asterix-gerrit.ics.uci.edu/260

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Chris Hillery <ceej@lambda.nu>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 31 more files in changeset.
VariableSizeFrame(VSizeFrame) support for Hyracks.

This patch replaced Frame/Accessor/Appender with the new API which

supports BigObject.

The ExternalSorter/TopKSorter/ExternalGroupSorter

have been implemented to support big object.

The Groupby && Join should work with BigObject also. But it will break the

memory budget when it encounter a big object. I will fix the memory

problem later in a separate CR.

The design about the frame allocation is

here:https://docs.google.com/presentation/d/15h9iQf5OYsgGZoQTbGHkj1yS2G9q2fd0s1lDAD1EJq0/edit?usp=sharing

Suggest review order:

Patch 12: It includes all of the sorting operators.

Patch 13: It applys the new IFrame API to all Hyracks codes.

Patch 14: Some bug fixes to pass all Asterix's tests.

Patch 15: Skip it!

Patch 16: Some bug fixes to the Asterix's tests in small frame setting.

Later Patch: address the comments

Change-Id: I2e08692078683f6f2cf17387e39037ad851fc05b

Reviewed-on: https://asterix-gerrit.ics.uci.edu/234

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

    • -12
    • +23
    ./edu/uci/ics/hyracks/control/nc/Joblet.java
    • -6
    • +18
    ./edu/uci/ics/hyracks/control/nc/Task.java
    • -1
    • +10
    ./edu/uci/ics/hyracks/control/nc/io/IOManager.java
  1. … 211 more files in changeset.
Allow retries on IPCSystem.getHandle().

NC will retry indefinitely to connect CC.

Change-Id: I0f4c15cacd265c3fbe85307af9f5c33577035447

Reviewed-on: https://asterix-gerrit.ics.uci.edu/250

Reviewed-by: Chris Hillery <ceej@lambda.nu>

Tested-by: Chris Hillery <ceej@lambda.nu>

  1. … 4 more files in changeset.
Adding hash join logging comments.

commit 513c3a7899dc64af3c3cdec96fad9093a4ca2c5f

Merge: b27e9b5 82609d9

Author: Eldon Carman <ecarm002@ucr.edu>

Date: Thu Feb 5 12:47:52 2015 -0800

Adding hash join logging comments.

Change-Id: Iade2c53436e5ae82c31305d6f618c780cd72568b

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/219

Reviewed-by: Ian Maxon <imaxon@uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Pouria Pirzadeh <pouria.pirzadeh@gmail.com>

  1. … 6 more files in changeset.
Add ability to configure network ports for NCs and CC.

This includes the ability to specify different "public" ports, for situations

when a node is behind a NAT firewall.

Also eliminate IP check for data and dataset network addresses, which

should allow DNS names to be used as well.

Some internal cleanup regarding handling network addresses.

Change-Id: I9947fe3cec59daef3458cdc14d33c9353449da27

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/204

Reviewed-by: Ian Maxon <imaxon@uci.edu>

Tested-by: Ian Maxon <imaxon@uci.edu>

  1. … 21 more files in changeset.
Add Shutdown via API to Hyracks

This change adds a method to HyracksConnection called stopCluster().

When the CC recieves a message from this, it asks all NC tasks to close

and acknowledge that they have recieved the message and are closing.

If all NCs have closed, or a 10 second timeout elapses, the CC then

exits with a 0 return code if all NCs closed, or a 1 if some did

not acknowledge the shutdown request.

Change-Id: Iaf3d395dc7964e114d4929830f40063f58e0d5da

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/76

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Vinayak Borkar <vinayakb@gmail.com>

    • -0
    • +63
    ./edu/uci/ics/hyracks/control/nc/work/ShutdownWork.java
  1. … 22 more files in changeset.