Clone Tools
  • last updated 11 mins ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
avoid some exception wrapping

Change-Id: Ia8fff04c314eb146ce0a73beb0b8ec412cb2d280

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1179

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 13 more files in changeset.
Small Cleanup

Change-Id: Ib527f2eee283faf75c04323b6961c31a6d739d07

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1145

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -4
    • +10
    ./apache/asterix/external/api/IAdapterFactory.java
  1. … 5 more files in changeset.
Refactor Messaging

In this refactoring, each message implementation includes

a handle method. This avoids bloating of message brokers

and enable better extensibility for messaging.

Change-Id: I7c918bf504058c98ecf89f5b019503278e9aa01f

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1128

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

  1. … 112 more files in changeset.
Refactor Active Listeners

Change-Id: I260c8608329523f56dc54780d87d796f838505cf

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1118

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

  1. … 24 more files in changeset.
Introduce MessagingNetworkManager for NC2NC AppMessaging

This change introduces MessagingNetworkManager to NodeControllerService.

The MessagingNetworkManager is used to open channels that are used for NC2NC

application messaging. The read and write interfaces of the messaging channels

are set by a MessagingChannelInterfaceFactory which is set by the application.

Change-Id: I5c0bd7c11c1e78954ebceff49cb274d8073a64bd

Reviewed-on: https://asterix-gerrit.ics.uci.edu/897

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

  1. … 74 more files in changeset.
TweetParser Extension

This patch includes following changes:

1. ExtendedTweetParser to parse more than fix attributes.

2. Changed the twitter feeds message unit from Status to String.

Change-Id: I7021e7b779de05b9ec999a8d5f8464fb0ab413c0

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1002

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

    • -83
    • +208
    ./apache/asterix/external/parser/TweetParser.java
    • -14
    • +19
    ./apache/asterix/external/util/TwitterUtil.java
  1. … 6 more files in changeset.
Add Asterix Extension Manager

More extension support is added. A user can now provide implementations

for the IExtension interface which will give them more control over

the behavior of the system and give them the ability to add custom

features.

Initial customizations include:

1. Metadata Tuple Translators

2. Metadata Datasets

3. Query Translators

4. Statement Handlers

5. Lang Compilation Provider

Change-Id: I280268495cc3aad00f898cba21f7299f7120ce5c

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1017

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

  1. … 218 more files in changeset.
ASTERIXDB-1478: fix the utf8 reader.

1. Fix the ASTERIXDB-1478.

2. Add the utf8 testCases.

Change-Id: Idb302dc604fcd71811de550d3d4bd727c81a13ee

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1077

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

  1. … 7 more files in changeset.
small refactoring

Change-Id: I37eab1645416e3aad6119bba527c5e3b4b98fddc

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1052

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

Fix Synchronization Issue in FeedRecordDataflowController

Change-Id: Ia290499132c320ce99402e698c76ba6d944f8f3d

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1051

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

small ADMParser/AdmLexer cleanup

- do not declare IOExceptions that are not thrown

- improve formatting of lexer template

Change-Id: Id032f0c83ebef108a9eddb7242c586d5481217e4

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1032

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 2 more files in changeset.
Add Maven Plugin for Grammar Extension

This change introduces a new asterix maven plugin for grammar

extension. The plugin takes a base grammar file and an extension

file and combine them to generate an extended grammar. A base

grammar can be extended by overriding production nodes, adding

extending existing nodes, or introducing new terminals and

non terminals.

In addition, a mvn verify goal was added to generate the parser

java files from the grammar generated by the test.

Change-Id: Iaa2d11782d43dd8f27d69e347ed0fc8797d79dad

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1011

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

  1. … 56 more files in changeset.
Cast Record Before Insert for Open Indexes

This change modifies the insert plan to perform cast for open

indexes before inserting to primary index. This avoids modification

of indexes and then doing complex work to try and undo the insert.

In addition, feeds can simply log and remove tuples which have

open fields of incompatible type.

Change-Id: I0eef5813ddbfe1b9c518cd7f92d37c95e8486914

Reviewed-on: https://asterix-gerrit.ics.uci.edu/997

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

  1. … 30 more files in changeset.
Add Test NodeController, Test Data Generator, and Marker Logs

This test enable creating a node controller for unit test purposes.

The Node controller is identical to the regular node controller

except that it doesn't communicate with a cluster controller at all.

In this change, Test Data Generator is introduced which should

facilitate writing unit test cases which requires data generation.

The change also includes enabling feeds to send progress data. progress

information can then be sent through the pipeline and persisted in the

transaction logs and primary index component. A Unit test case has

been created to test adding progress markers to logs and index

components and then reading them.

The last part of this change is the addition of marker logs and their

callbacks. They enable components to create arbitrary logs and get a

callback when they are written to the transaction logs. Initial set of

unit tests were added for marker logs.

Change-Id: I3b9aa8de758b7d26ca34868b16e5ce693e0c0243

Reviewed-on: https://asterix-gerrit.ics.uci.edu/962

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

    • -0
    • +35
    ./apache/asterix/external/api/IFeedMarker.java
  1. … 68 more files in changeset.
Refactor General Active Classes

This change is the first of a series of changes aiming to refactor

feed related classes into generic active and feed specific.

In this change, we introduce asterix-active which will be one level

below asterix-external-data. The ultimate goal is to have a very

flexible framewrok for long running activities within asterix.

Change-Id: I0a8f33ee5b45c5e090b08c24a102e369aae43c04

Reviewed-on: https://asterix-gerrit.ics.uci.edu/977

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

    • -2
    • +2
    ./apache/asterix/external/feed/api/IFeed.java
  1. … 116 more files in changeset.
ASTERIXDB-1494: refactor ExternalLibraryManager and make it non-static.

Change-Id: I5c287a35ff90c3aea639d3069d5a842e28c5e508

Reviewed-on: https://asterix-gerrit.ics.uci.edu/980

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 20 more files in changeset.
Fix Decoding of byte[] Records

Change-Id: I71c3d8b8dfa5a98123725f139247d2b5ce10012e

Reviewed-on: https://asterix-gerrit.ics.uci.edu/951

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

Reviewed-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 3 more files in changeset.
Better logging for generic record

Change-Id: I6840881c51c073b527f38e74703c0a0364051bc6

Reviewed-on: https://asterix-gerrit.ics.uci.edu/927

Reviewed-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

Improve External Data

This change enable an adapter to specify its data parser and not have

the user specify it and pass it as part of the adapter configurations.

In addition, it introduces a new parameter "parser-factory" that can be

used to specifies a parser factory instead of using parser to specify

a parser factory.

Change-Id: Iae2560c73fa63e9454f731b8e893ae779a2ac7d9

Reviewed-on: https://asterix-gerrit.ics.uci.edu/906

Reviewed-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

    • -13
    • +7
    ./apache/asterix/external/api/IAdapterFactory.java
    • -0
    • +30
    ./apache/asterix/external/api/IRecordWithMetadataParser.java
  1. … 21 more files in changeset.
Supports Left Outer Join and Left Outer Unnest in SQL++.

- change record format to allow MISSING value for an optional field;

- support roundtrips of MISSING valued closed fields for load and insert;

- suppress fields with MISSING values in result printer;

- fix ByNameToByIndexFieldAccessRule and LoadRecordFieldsRule;

- update test queries and results.

Change-Id: Ie0caea9c1842d93541b067a1193d117af30d8dfc

Reviewed-on: https://asterix-gerrit.ics.uci.edu/899

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

    • -12
    • +4
    ./apache/asterix/external/parser/ADMDataParser.java
  1. … 583 more files in changeset.
Remove Test Classes From Production Binaries

Change-Id: I1da72566e7d140a1fb965c893dcea6881b99ca26

Reviewed-on: https://asterix-gerrit.ics.uci.edu/901

Reviewed-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

  1. … 15 more files in changeset.
ASTERIXDB-1228: Add MISSING into the data model.

1. MISSING repsents the value of a non-existing field in a record

or an out-of-bound index access of a collection;

2. NULL represents that the value of an optional field in a record

is unknown or the value of existing collection entry is unknown.

3. Unit tests for all missing/null-in-missing/null-out scalar functions.

Change-Id: Ia49ed8474bfc5d6604231819065117468c5b0897

Reviewed-on: https://asterix-gerrit.ics.uci.edu/846

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

    • -18
    • +12
    ./apache/asterix/external/parser/ADMDataParser.java
  1. … 537 more files in changeset.
Improve Messaging Connector and Add Unit Tests

Before this change, messaging connector always reserves 100 bytes

for messages which are mostly un-used. With this change, it only

reserves two bytes and sends null messages by default. In case a

new message doesn't fit in the leftover space of a frame, it sends

the frame with a null message, followed by a dedicated frame for

the message.

Change-Id: If4336e9c234e8d282798cfba9f48432b46cccfca

Reviewed-on: https://asterix-gerrit.ics.uci.edu/880

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

Reviewed-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 15 more files in changeset.
some finals + exception cleanup

Change-Id: Ie0252fef069ee93db6e32e13a13ebb11df5ce913

Reviewed-on: https://asterix-gerrit.ics.uci.edu/877

Reviewed-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

  1. … 6 more files in changeset.
Remove PowerMock from asterix-external-data

PowerMock prevents jacoco from collecting coverage information.

Hence, it is better not to use it. However, using Mockito is fine.

In addition, this change fixes the BTree frame writer unit test.

Change-Id: I69cff2827b1f2cc57e0fec0316b2b997f806787c

Reviewed-on: https://asterix-gerrit.ics.uci.edu/875

Reviewed-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

  1. … 6 more files in changeset.
Remove Key-Value Adapter

Change-Id: Iaa7d8d70b0869242d1a872f55f0c6928fda94dcb

Reviewed-on: https://asterix-gerrit.ics.uci.edu/868

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <michael.blow@couchbase.com>

Add Unit Tests for Feed Runtime Input Handler

Change-Id: I7088f489a7d53dee8cf6cdbf5baa7cd8d3884f55

Reviewed-on: https://asterix-gerrit.ics.uci.edu/866

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <michael.blow@couchbase.com>

  1. … 12 more files in changeset.
ASTERIXDB-1455: Scale CharRecord Buffer Expansions

Reading a 30 MB record requires > 1500 buffer reallocations when increasing by

a fixed amount. Updated to scale up the buffer increases to reduce the number

of reallocations in case of large records.

Change-Id: I4822b2a78e1f411de94d2c717523df6b2b4a381e

Reviewed-on: https://asterix-gerrit.ics.uci.edu/861

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

Cleanup Feed CodeBase

In order to expedite cleaning feeds up and reaching a maintainable

state, we disabled the following:

1. Policies (At least once, throttling, discarding, elasticity).

2. Statistics Reporting.

3. Load management.

4. Feed re-activation upon System reboot.

Right now on master, none of these features work reliably.

We will re-introduce them one feature at a time.

The rules followed in this change:

1. Keep X if X is tested in a test case.

2. Remove X if X is not used in test cases.

After a few meetings with Mike and Till, the policies

1. Buffering

2. Spill

3. Discard

4. Throttle

have been fixed and unit tests have been added.

Change-Id: I545bc4f8560564e4c868a80d27c77a4edd97a8b8

Reviewed-on: https://asterix-gerrit.ics.uci.edu/798

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

  1. … 127 more files in changeset.
ASTERIXDB-1436: Big Object Support For Storage

Preliminary submit for big object storage support (values only).

Enable hyracks frames up to Integer.MAX_VALUE

Change-Id: Ie0a3cb855768dfd0cd1da4a5fc9f62aedebcbc5f

Reviewed-on: https://asterix-gerrit.ics.uci.edu/840

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

  1. … 95 more files in changeset.