Clone Tools
  • last updated a few minutes ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
Move merged files

  1. … 159 more files in changeset.
ASTERIXDB-1356: report file not found

Change-Id: Ia12bf84adfd8442103567df42ac22e5750e1360f

Reviewed-on: https://asterix-gerrit.ics.uci.edu/771

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

  1. … 3 more files in changeset.
Fix Number of LocalFS Readers

This change makes the number of readers in localfs

adapter always equals to one. This is a temporary

change and is done to avoid sporadic failures for

decorrelation with unique id.

Change-Id: Id948bfd2c1a4e79863378ed98fb27f662fffba93

Reviewed-on: https://asterix-gerrit.ics.uci.edu/782

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

Control Number of Readers for LocalFS Data

Change-Id: Ib9d5ece656220d5f562cc385f882c5ddfd3283a6

Reviewed-on: https://asterix-gerrit.ics.uci.edu/776

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

  1. … 10 more files in changeset.
ASTERIXDB-1378 Fix NPE on Feed Connect After Restart

This issue was caused by the way type traits are computed for each

index. since we have duplicate code to create the index dataflow

helper, we had to fix the way type traits are created in all of

these different places. We propably need to do further refactoring

to have common code for creating index dataflow helper instances.

Change-Id: If8f8696d252868a8cce0afdbaeda0dd046f99186

Reviewed-on: https://asterix-gerrit.ics.uci.edu/766

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

  1. … 17 more files in changeset.
ASTERIXDB-1360 Fix Error Message for Unknown Format

This change includes some refactoring for external

data. This refactoring makes the code less error prone

and separate data source selection from data parser

selection. It also fixes issue ASTERIXDB-1366 and adds

a test case for it as well.

Change-Id: Ib4aac833e30bd7c5a7706f5c8116383c2362c964

Reviewed-on: https://asterix-gerrit.ics.uci.edu/767

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 19 more files in changeset.
Fix Asterix Stream Reader

The reset method in the asterix stream reader wasn't working.

Calling clear() on a byte buffer only reset the position but

not the limit. hence, the reader still thinks that there are

some leftover data. This has been fixed to set the limit to 0

Change-Id: I41a84baacdd233120ca08eeebbd2d757b4e5c154

Reviewed-on: https://asterix-gerrit.ics.uci.edu/761

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 1 more file in changeset.
Improve Error Handling in Local Directory Feeds

This change improves handling of two error types for filesystem

based feeds. The first one is the handling of IO Errors which

causes the input stream to be closed, and the second one is

reacting to missed filesystem events. In both cases, we scan the

directory and compare it with the history we have in order to

resume from where we last left off.

In addition, this change includes some refactoring in external

data. Particularly, we get rid of the stream provider layer and

instead, stream factories create input streams directly. This

is consistent with record reader factories which create readers

directly without reader providers.

Change-Id: I08d89229e33c91532b1038ba9f7a372f7ca1fdb5

Reviewed-on: https://asterix-gerrit.ics.uci.edu/720

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

  1. … 130 more files in changeset.
Fixed Feed Connect Statement

This change includes two fixes:

1. Feed connect doesn't return until the connection is complete.

2. When using wait for completion, it waits until all the jobs

complete.

Change-Id: I416bf4917b1f5cea687d1202c435f7183136cf1f

Reviewed-on: https://asterix-gerrit.ics.uci.edu/726

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

  1. … 36 more files in changeset.
Fix Asterix Installer Test Cases

Many tests started to fail in installer after being

disabled for a long time. This change fixes the following:

1. asterix-installer: external library tests.

2. asterix-installer: recovery: query-after-restart.

Change-Id: Iefad5ac3ff754693025312f1e2fa01b08832514c

Reviewed-on: https://asterix-gerrit.ics.uci.edu/737

Reviewed-by: Ian Maxon <imaxon@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 29 more files in changeset.
Improve KV Generator

Currently, KV Generator produces the same documents when

running in parallel where the only difference is in the

key of the record. With this change, it produces different

Documents in each partition to produce more diverse set of

records.

Change-Id: Ia337be57f8394d74a98f3c72ebcaa584f277f34b

Reviewed-on: https://asterix-gerrit.ics.uci.edu/724

Reviewed-by: Michael Blow <michael.blow@couchbase.com>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

  1. … 1 more file in changeset.
Support Change Feeds and Ingestion of Records with MetaData

This change allows feeds to perform upserts and deletes

in order to perform replication of an external data source.

The change does so by performing the following:

1. The adapter produces [PK][Record]. (Record == null --> delete)

2. The insert is replaced by an upsert operator.

Change-Id: If136a03d424970132dfb09f0dda56e160d4c0078

Reviewed-on: https://asterix-gerrit.ics.uci.edu/621

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -21
    • +2
    ./org/apache/asterix/external/api/IDataParser.java
    • -50
    • +0
    ./org/apache/asterix/external/api/IFeedAdapter.java
    • -0
    • +26
    ./org/apache/asterix/external/api/IRecordConverter.java
  1. … 256 more files in changeset.
Coverity Fixes

FeedMessageOperatorNodePushable

- CID 68721 (#1 of 1): Useless call (USELESS_CALL)

FeedLoadManager

- CID 68719 (#1 of 1): Useless call (USELESS_CALL)

RuntimeRecordTypeInfo

- CID 68717 (#1 of 1): Unused value (UNUSED_VALUE)

HelpCommand

- CID 68591 (#1 of 1): Missing break in switch (MISSING_BREAK)

- CID 68716 (#1 of 1): Unused value (UNUSED_VALUE)

SimilarityJaccardEvaluator

- CID 68714 (#1 of 1): Arguments in wrong order (SWAPPED_ARGUMENTS)

DatasetLifecycleManager

- CID 68713 (#1 of 1): Dereference before null check (REVERSE_INULL)

FeedMetaComputeNodePushable

- CID 68712 (#1 of 2): Dereference before null check (REVERSE_INULL)

- CID 68712 (#2 of 2): Dereference before null check (REVERSE_INULL)

FeedRuntimeInputHandler

- CID 68709 (#1 of 1): Dereference before null check (REVERSE_INULL)

DataflowControllerProvider

- CID 68708 (#1 of 1): Resource leak on an exceptional path

(RESOURCE_LEAK)

TokenLoad

- CID 68697 (#1 of 1): Resource leak (RESOURCE_LEAK)

RecordGroupLengthRange

- CID 68696 (#1 of 1): Resource leak (RESOURCE_LEAK)

RecordManagerGeneratorMojo

- CID 68691 (#1 of 1): Resource leak (RESOURCE_LEAK)

APIServlet

- CID 68687 (#1 of 1): Resource leak (RESOURCE_LEAK)

Change-Id: I8927126f505481e83dd5b9e6e5ccd75bb67d8188

Reviewed-on: https://asterix-gerrit.ics.uci.edu/715

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

  1. … 9 more files in changeset.
Flush Socket Feed Before Waiting for More Data

Change-Id: Ic37085114592e97b6c5b2d29c5451d816a154aa8

Reviewed-on: https://asterix-gerrit.ics.uci.edu/709

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

ASTERIXDB-1325, ASTERIXDB-1326: fixed leftouterjoin on an external dataset

- Fixed the issue that optimizer ignores index_nl hint on an index on an external dataset.

- Fixed the issue that index-nested-loop-leftouterjoin on an external dataset returns an empty results.

- Modified OptimizerTest to enable mockup HDFSCluster.

- Added an optimizer test on an external dataset.

Change-Id: I91b2e57f0bd1beaa58dd28a84d8e08d308246b26

Reviewed-on: https://asterix-gerrit.ics.uci.edu/672

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 4 more files in changeset.
ASTERIXDB-1302 ASTERIXDB-1301 Fix Socket Feed Connection

A bug causes a read lock to never be released when a feed is

connected with "wait-for-completion" set to false. The bug

was fixed and a test case was added.

Another bug was causing the socket feed to not receive

connections correctly. The bug was fixed and a test case

was added.

Additionally, this change ensures that adapters have absolute

partitions to ensure consistency with regards to feed log

manager.

Change-Id: I8f6e982440d3577343f2479c3779653a9c3db614

Reviewed-on: https://asterix-gerrit.ics.uci.edu/660

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ildar Absalyamov <ildar.absalyamov@gmail.com>

  1. … 35 more files in changeset.
Remove Unnecessary Prints in Tests

Change-Id: I4def109c0a8b40343d59d5689f8a791249174478

Reviewed-on: https://asterix-gerrit.ics.uci.edu/655

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 8 more files in changeset.
Coverity Fix for Bad Lock Object

CID 68477: Bad choice of lock object (BAD_LOCK_OBJECT)

- boxed_lock:

Boxing a primitive may or may not return a canonical boxed

representation depending upon the value of the primitive being boxed.

Thus, using a boxed primitive as a lock is dangerous.

Change-Id: Ib993d94bfae6b788b5b56d388fa7a33ec958dee4

Reviewed-on: https://asterix-gerrit.ics.uci.edu/665

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@apache.org>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

Enabled Feed Tests and Added External Library tests

Feed tests had been switched off for a while due to having too many

sporadic failures. Now, we are switching them back on.

In addition, a new set of tests have been added to test that external

library works as expected.

Change-Id: Idd1fccd136fa2645b2707bbf7c04e60991ae8d4a

Reviewed-on: https://asterix-gerrit.ics.uci.edu/625

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

  1. … 190 more files in changeset.
Remove Trailing Whitespace

This change removes all trailing whitespace in

java files in asterixdb

Change-Id: I32cda384f71dd5237a3c63e83c661f1cf64dc0eb

Reviewed-on: https://asterix-gerrit.ics.uci.edu/646

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 312 more files in changeset.
ASTERIXDB-1304: remove interfaces and consolidate

Change-Id: Ibb8819f51c655fadaf47fdbad0af5f66b261b2b5

Reviewed-on: https://asterix-gerrit.ics.uci.edu/643

Reviewed-by: Michael Blow <michael.blow@couchbase.com>

Reviewed-by: Till Westmann <tillw@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -0
    • +47
    ./org/apache/asterix/external/api/ITupleForwarder.java
  1. … 5 more files in changeset.
Merge asterix-experiments to master

Change-Id: I79b91e045bd056e62adb6e48795a6625baee0062

Reviewed-on: https://asterix-gerrit.ics.uci.edu/642

Reviewed-by: Ian Maxon <imaxon@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 254 more files in changeset.
Support Sending Messages Alongside Frame Data

This change supports sending messages with records. The tuple Appender

reserves 100 bytes for a message. Before sending the frame, it appends

The message in the last tuple position. The message is read from the

task context as the shared object between different operators in the

pipeline. The first use of this feature will be within feeds to request

acks for at least once semantics.

Change-Id: Iaa23e9f8a909ddcafc1c3ee95181092eb04ee1ad

Reviewed-on: https://asterix-gerrit.ics.uci.edu/605

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

    • -0
    • +29
    ./org/apache/asterix/external/util/FeedMessageUtils.java
  1. … 4 more files in changeset.
Add compatibility for Socket adapter

This change allows the use of alias for socket adapter

Change-Id: I8c8c78598c0cfea36c18503a73b002ece700be71

Reviewed-on: https://asterix-gerrit.ics.uci.edu/609

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

Move to non-copy-based evaluator interfaces for all function implementations, including: - scalar functions, - aggregate functions, - running aggregate functions, - unnesting functions

Change-Id: If132857cf8d707ce96951f12ba6dd86f54aa4069

Reviewed-on: https://asterix-gerrit.ics.uci.edu/615

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 312 more files in changeset.
ASTERIXDB-1271: Revisit UUID Implementation

- Updated UUID implementation to eliminate marshalling to/from a pair

of longs, replacing with byte [].

- Eliminate all but one call to SecureRandom at init time, avoid risk

of entropy exhaustion

- Fix exceptions thrown

- Incorporate constant to represent number of bytes, String chars in

an instance of AUUID

- Split generated from parsed AUUID

- Eliminate intermediate StringBuilder construction on priting paths,

optimize imports

Change-Id: I9e90d42f6b62f80ad180dbd0c8c852db85b14173

Reviewed-on: https://asterix-gerrit.ics.uci.edu/610

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 10 more files in changeset.
ASTERIXDB-1281 - Interval format update to AQL and ADM

The new interval format takes a more generic approach to representing intervals.

Here is an example for a date interval:

interval(date("2012-01-01”), date(”2013-04-01”))

Note that the interval type is defined by the arguments to the interval expression.

Currently only date, time, and datetime types are supported for intervals. The new

format is used for ADM and AQL.

In addition to the format change, the internal byte structure of an interval has been

updated. The format looks like the following:

byte tag, T start, T end (where T is a date, time or datetime type)

Note how the tag has been moved to the front. Also with the new sturcture, an

interval is variable length, not fixed length as before.

Change-Id: I009c71b7a445d141e228ba15d56d0b6cf3c8a3f5

Reviewed-on: https://asterix-gerrit.ics.uci.edu/602

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 70 more files in changeset.
Some exception cleanup

- Use HyracksDataException and IOException for errors ingesting external

data (instead of a mixture of AlgebricksException, AsterixException, and

HyracksDataException).

- Avoid unnecessary wrapping of exceptions.

- Wrap exception at boundaries (e.g. HiveRecordParser)

- ADMLexerException and ADMDataParser.ParseException extend

HyracksDataException

- Tighter exception declarations and handling for internal methods.

Change-Id: Icf9591046c44f5fa2281874ff0c98d780e741267

Reviewed-on: https://asterix-gerrit.ics.uci.edu/603

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

  1. … 50 more files in changeset.
Add Support for Upsert Operation

This change allows users to execute upsert commands which

couples delete if found with insert. It locks the primary

keys before doing the search ensuring consistency.

Change-Id: I8999000331795a5949d621d2dd003903e057a521

Reviewed-on: https://asterix-gerrit.ics.uci.edu/477

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 249 more files in changeset.
Add flush() to IFrameWriter

This method is expected to be used with feeds to push

frames all the way to storage when needed. As of now, it is

needed in two cases:

1. No activities in ingestion node and need to push content

so it can be stored.

2. When the ingestion node needs to move the checkpoint ahead

if the at least once semantics are used.

Two feeds make use of this function. The filesystem feed and

couchbase feed which was introduced as well in this change.

Change-Id: Id862ce9e9b1360864c6976f2aea2137092f51203

Reviewed-on: https://asterix-gerrit.ics.uci.edu/585

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

  1. … 89 more files in changeset.