asterix-common

Clone Tools
  • last updated 12 mins ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
Support Sending Messages Alongside Frame Data

This change supports sending messages with records. The tuple Appender

reserves 100 bytes for a message. Before sending the frame, it appends

The message in the last tuple position. The message is read from the

task context as the shared object between different operators in the

pipeline. The first use of this feature will be within feeds to request

acks for at least once semantics.

Change-Id: Iaa23e9f8a909ddcafc1c3ee95181092eb04ee1ad

Reviewed-on: https://asterix-gerrit.ics.uci.edu/605

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

  1. … 13 more files in changeset.
[maven-release-plugin] prepare for next development iteration

  1. … 21 more files in changeset.
[maven-release-plugin] prepare release apache-asterixdb-0.8.8-incubating-rc1

  1. … 21 more files in changeset.
Address RC0 Comments

Change-Id: I402cef4206fa0610eaff899cf005875323fb18d9

Reviewed-on: https://asterix-gerrit.ics.uci.edu/608

Reviewed-by: Till Westmann <tillw@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 28 more files in changeset.
Revert "[maven-release-plugin] prepare release asterix-0.8.8-incubating-rc0"

This reverts commit 753b4231a370cf36ec20a1124c71eeebbcbf9e92.

  1. … 21 more files in changeset.
Revert "[maven-release-plugin] prepare for next development iteration"

This reverts commit 1b18e778ed8c4028e80ec828c655523e5b2eff2b.

  1. … 21 more files in changeset.
Add Support for Upsert Operation

This change allows users to execute upsert commands which

couples delete if found with insert. It locks the primary

keys before doing the search ensuring consistency.

Change-Id: I8999000331795a5949d621d2dd003903e057a521

Reviewed-on: https://asterix-gerrit.ics.uci.edu/477

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 245 more files in changeset.
remove end-of-line whitespace

Change-Id: I5c0415f47d4c3a9827574fbdab949b45718d9ea4

Reviewed-on: https://asterix-gerrit.ics.uci.edu/601

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

  1. … 124 more files in changeset.
[maven-release-plugin] prepare for next development iteration

  1. … 21 more files in changeset.
[maven-release-plugin] prepare release asterix-0.8.8-incubating-rc0

  1. … 21 more files in changeset.
Asterix NCs Fault Tolerance

This change includes the following:

- Adapt replication to unique partitions storage.

- Implement auto failover for failing NCs.

- Implement auto failover for metadata node.

- Fix for ASTERIXDB-1251 using proper error message.

- Basic replication test cases using vagrant virtual cluster for:

1. LSM bulkload components replication.

2. LSM Memory components replication and recovery.

3. Metadata node takeover.

These test cases will be part of the cluster test profile.

Change-Id: Ice26d980912a315fcb3efdd571d6ce88717cfea4

Reviewed-on: https://asterix-gerrit.ics.uci.edu/573

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

  1. … 57 more files in changeset.
Feed Fixes and Cleanup

1. Introduce filesystem feed data source.

2. Fix the order of closing feed stages on disconnection.

3. Added Twitter feed to the compatibility utility to allow

using it with its alias.

4. First part of the feed log space.

5. Fixed the handling of duplicate key exception.

Change-Id: I4e8db26a810efd1fbaa52ceeb3efd0c8328ab070

Reviewed-on: https://asterix-gerrit.ics.uci.edu/574

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

  1. … 403 more files in changeset.
Improve the Testing Framework

This change includes the following improvements to the test framework

1. Checking each expected failure against the failure exception.

2. Checking against multiple causes of failure in case of multiple

expected errors.

3. Check if a test that is expected to fail pass.

4. Make temp datasets use different storage path.

Change-Id: I957ecf19bf7209981e010e0e50fb882442a525dd

Reviewed-on: https://asterix-gerrit.ics.uci.edu/537

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

  1. … 27 more files in changeset.
First stage of external data cleanup

In this change, different parts of external data were refactored.

The goal was to make it more modular, easier to maintain and allow

higher flexibility for extension in addition to reducing code redundancy.

Change-Id: I04a8c4e494d8d1363992b6fe0bdbe6b2b3b7b767

Reviewed-on: https://asterix-gerrit.ics.uci.edu/566

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

  1. … 329 more files in changeset.
Divide Cluster into Unique Partitions

The change includes the following:

- Fix passing NC stores to AsterixConfiguration.

- Unify storage direcotry name in the instance level rather than the node level.

- Divide the cluster into unique storage partitions based on the number of stores.

- Refactored FileSplits and moved out of AqlMetadataProvider.

- Make AsterixHyracksIntegrationUtil use the passed configuration file.

- Make File Splits pass relative index paths of partitions rather than absolute paths.

- Remove unused AqlCompiledMetadataDeclarations class.

Change-Id: I8c7fbca5113dd7ad569a46dfa2591addb5bf8655

Reviewed-on: https://asterix-gerrit.ics.uci.edu/564

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

    • -1
    • +0
    ./src/main/resources/schema/cluster.xsd
  1. … 34 more files in changeset.
ASTERIXDB-54: s/IHyracksCommonContext/IHyracksTaskContext/

Change-Id: Id98f3d94e8036199dcbdbdb059c97c0f99ed9205

Reviewed-on: https://asterix-gerrit.ics.uci.edu/534

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

  1. … 12 more files in changeset.
Adapt Replication to Append Only LSM Components

This change includes the following:

- Make LSN recording in a single tree in LSM components.

- Pass LSN byte offset with every tree index file being replicated.

- Add LSN to remote recovery logs to check which logs should be replayed.

- Update ILogRecord method names to better describe their operation.

Change-Id: I8bd2656746e1c293b981d5f43e80928314ccbad0

Reviewed-on: https://asterix-gerrit.ics.uci.edu/561

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

Reviewed-by: Ian Maxon <imaxon@apache.org>

  1. … 9 more files in changeset.
Asterix MessageBroker implementation

This change includes the following:

- Add implementation for CC/NC MessageBroker.

- Implement GlobalResourceIdFactory using MessageBroker.

- Change resource id factory to GlobalResourceIdFactory.

- Refactor metadata indexes fixed properties.

- Use fixed resource ids for metadata indexes.

Change-Id: If4320e2c5a0130d2f86a4be6ae61f5cee43e30af

Reviewed-on: https://asterix-gerrit.ics.uci.edu/486

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 16 more files in changeset.
patch 2

  1. … 155 more files in changeset.
Changed the IFrameWriter Contract

Updated existing operators to conform to the new contract.

These operators are either index operators or Feed operators.

The rest of the operator already follow the new contract.

Change-Id: Ibcebe876340a25be0f561945582a95211c140e10

Reviewed-on: https://asterix-gerrit.ics.uci.edu/552

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

  1. … 7 more files in changeset.
Run ExecutionTest w/integration test framework

I changed the default local.xml to use 2 NCs with

2 partitions each like AsterixHyracksIntegrationUtil

so that the results will match despite using simple

string compare to determine correctness.

There is also one test in particular (big-object-join)

that I had to add an artificial order by clause to

to deal with different results.

Issues outstanding:

- HDFS in ClusterExecutionIT needs to be thought out

Change-Id: I423f2a7c77839b999d466dd5cace302574d956c0

Reviewed-on: https://asterix-gerrit.ics.uci.edu/525

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Chris Hillery <ceej@lambda.nu>

  1. … 3866 more files in changeset.
patch 1

  1. … 88 more files in changeset.
ASTERIXDB-1110 Fix count keeping in primary index opTracker

This change fixes the counter of number of active operations in primary

index opTracker. This is accomplished by sharing of counter between multiple

partitions. Each transaction will have that counter as part of its

context object

Change-Id: I0dc34b9a3aa69d39ac2eda11c17e7ad0ccc1a661

Reviewed-on: https://asterix-gerrit.ics.uci.edu/536

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 2 more files in changeset.
Fix ASTERIXDB-1194,ASTERIXDB-1195,ASTERIXDB-1196,ASTERIXDB-1197.

Change-Id: I7d167b64bf9ec754182b5b2fe44dfc7e5908c686

Reviewed-on: https://asterix-gerrit.ics.uci.edu/325

Reviewed-by: Till Westmann <tillw@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 24 more files in changeset.
ASTERIXDB-1105 - LICENSE per submodule

This is a good part of what we need to have a specific set of

dependencies per binary assembly. The main key here is using the

maven remote resources plugin, and supplemental resources to

fill in info from the POM of dependencies where it is lacking.

For things that aren't described in Maven dependencies,

like Javascript files, they are added as LICENSE fragements

that get included in binary assemblies.

Also fix AsterixEventServiceUtil to not die when

the asterix-server bundle has files in the root of

the archive.

Change-Id: I2a6c4fb809b6fcad578007b8f48bf8bce179cd82

Reviewed-on: https://asterix-gerrit.ics.uci.edu/518

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 37 more files in changeset.
Clean up compilation warnings.

Change-Id: Idbfcd9c67f91d373c5f7269125778a5681021227

Reviewed-on: https://asterix-gerrit.ics.uci.edu/505

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

  1. … 95 more files in changeset.
Make LSN recording in components compatible with append-only

Change-Id: I5610e903be3347893e676c915b98316b06073c25

Reviewed-on: https://asterix-gerrit.ics.uci.edu/265

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

  1. … 6 more files in changeset.
ASTERIXDB-1152: Delete storage data of old instances

Change-Id: Ibb6c6949bdf2ed6c3e491fa66a23491ff34fc830

Reviewed-on: https://asterix-gerrit.ics.uci.edu/469

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

Reviewed-by: Taewoo Kim <wangsaeu@gmail.com>

Reviewed-by: Ian Maxon <imaxon@apache.org>

  1. … 5 more files in changeset.
ASTERIXDB-1176: remove unused feeds code

Change-Id: I2de2d7c7fd816ddbd53a80c855f64923c02efe35

Reviewed-on: https://asterix-gerrit.ics.uci.edu/494

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 3 more files in changeset.
SQL++ support in AsterixDB: 1. implemented SQL++ expression to logical plan translator; 2. refactored REST API to be agnostic of query languages; 3. disabled fuzzy join queries for SQL++ runtime tests; 4. fixed several rewriting rules.

Change-Id: I82919c4527b304325059519d819a2c30cf2902a9

Reviewed-on: https://asterix-gerrit.ics.uci.edu/479

Reviewed-by: Till Westmann <tillw@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 1378 more files in changeset.