Clone Tools
  • last updated a few seconds ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
Add an integer ordered list to dataset "Dataset" and "Index" optinally.

For the case the primary or secondary index key contains fields from the meta

record associated with a dataset record, we need an integer ordered

list to indicate which record a key field comes from.

Change-Id: I979c642ebd60e53213369f1e2070146b4c26e805

Reviewed-on: https://asterix-gerrit.ics.uci.edu/683

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

Reviewed-by: Michael Blow <michael.blow@couchbase.com>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 48 more files in changeset.
ASTERIXDB-1325, ASTERIXDB-1326: fixed leftouterjoin on an external dataset

- Fixed the issue that optimizer ignores index_nl hint on an index on an external dataset.

- Fixed the issue that index-nested-loop-leftouterjoin on an external dataset returns an empty results.

- Modified OptimizerTest to enable mockup HDFSCluster.

- Added an optimizer test on an external dataset.

Change-Id: I91b2e57f0bd1beaa58dd28a84d8e08d308246b26

Reviewed-on: https://asterix-gerrit.ics.uci.edu/672

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 5 more files in changeset.
Add DDL support for optionally associating meta record with dataset record.

-Fixed the TestExecutor.runScriptAndCompareWithResult() to compare fields

after "Time"-prefixed fields.

Change-Id: I613316985dbd7f553d946f95c5b1a1df56914902

Reviewed-on: https://asterix-gerrit.ics.uci.edu/666

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

  1. … 66 more files in changeset.
Add the filesize parameter to estimate a better partition number for ExternalHashGroupby

Change-Id: I3aadec904af64cc2b7b03e6dff399a2288c21a7b

Reviewed-on: https://asterix-gerrit.ics.uci.edu/451

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

  1. … 1 more file in changeset.
ASTERIXDB-1205: fix union queries.

Fixed the AqlExpressionToPlanTranslator for union expression;

Eliminate shared operator references in translated logical plans;

Removed IntroduceUnionRule;

Fixed tuple source for SQL++ join clause;

Added regression tests.

Change-Id: Ib79a899ec4b35ece6f8f9f59b80126fed0c48851

Reviewed-on: https://asterix-gerrit.ics.uci.edu/629

Reviewed-by: Till Westmann <tillw@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 64 more files in changeset.
Remove Trailing Whitespace

This change removes all trailing whitespace in

java files in asterixdb

Change-Id: I32cda384f71dd5237a3c63e83c661f1cf64dc0eb

Reviewed-on: https://asterix-gerrit.ics.uci.edu/646

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 313 more files in changeset.
patch 7

  1. … 17 more files in changeset.
ASTERIXDB-1304: remove interfaces and consolidate

Change-Id: Ibb8819f51c655fadaf47fdbad0af5f66b261b2b5

Reviewed-on: https://asterix-gerrit.ics.uci.edu/643

Reviewed-by: Michael Blow <michael.blow@couchbase.com>

Reviewed-by: Till Westmann <tillw@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 14 more files in changeset.
Asterix NCs Failback Support

- Allow Failed NCs to failback and takeover their partitions.

- New cluster API servlet for cluster state description.

- Remove nodeId from txn logs except remote FLUSH_LOG.

- Add partition id in UPDATE and Entity_COMMIT logs.

- Adapt remote recovery to new logs format.

- Refactor RecoveryManager and split Analysis and Redo phase.

- Spill remote recover logs to temporary file.

- Replicate files to remote replicas for partitions of interest only.

- Introduce NC active/inactive partitions concept.

- Test case for failback.

Change-Id: Id17819542d6b9c4e32647e64737c4a467b630f24

Reviewed-on: https://asterix-gerrit.ics.uci.edu/613

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

  1. … 88 more files in changeset.
Support Sending Messages Alongside Frame Data

This change supports sending messages with records. The tuple Appender

reserves 100 bytes for a message. Before sending the frame, it appends

The message in the last tuple position. The message is read from the

task context as the shared object between different operators in the

pipeline. The first use of this feature will be within feeds to request

acks for at least once semantics.

Change-Id: Iaa23e9f8a909ddcafc1c3ee95181092eb04ee1ad

Reviewed-on: https://asterix-gerrit.ics.uci.edu/605

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

  1. … 13 more files in changeset.
ASTERIXDB-1231, ASTERIXDB-636: fixed Self-join

- Fixed self-join with index-out-of-boud exception during the compilation

- Added a test case for ASTERIXDB-636

Change-Id: I8d5d9cb0cb54473fbe7a5e43934e9608548c1dbb

Reviewed-on: https://asterix-gerrit.ics.uci.edu/635

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

  1. … 10 more files in changeset.
ASTERIXDB-1272 Remove ExternalLookupOperator

ExternalLookupOperator has been removed in this change

and UnnestMapOperator was used instead. different visitors

and methods in the UnnestMapOperator were changed to

support both external and internal data.

Change-Id: Ic9a9400586b75fd8cb3c0b8f4aeb45aee76cdb92

Reviewed-on: https://asterix-gerrit.ics.uci.edu/592

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

  1. … 1 more file in changeset.
Intersect the secondary indexes before primary search

Change-Id: Ie167918fb23e39c8728840e4a90c1b85bf1bde85

Reviewed-on: https://asterix-gerrit.ics.uci.edu/578

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Jianfeng Jia <jianfeng.jia@gmail.com>

  1. … 17 more files in changeset.
Move to non-copy-based evaluator interfaces for all function implementations, including: - scalar functions, - aggregate functions, - running aggregate functions, - unnesting functions

Change-Id: If132857cf8d707ce96951f12ba6dd86f54aa4069

Reviewed-on: https://asterix-gerrit.ics.uci.edu/615

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 314 more files in changeset.
Fix ASTERIXDB-1298

Handle the case that the original join operator in a subplan

operator is a left-outer join such that no null-check

variable needs to be added.

Change-Id: Ibb594e161d7236131a7a6a0a290e76071d4587f4

Reviewed-on: https://asterix-gerrit.ics.uci.edu/630

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Steven Jacobs <sjaco002@ucr.edu>

  1. … 7 more files in changeset.
ASTERIXDB-1157: Pushdown Limit

- Limit Pushdown into an Order (ExternalSort) operator.

- ConstantFoldingRule is not applied on the record-related operations.

Change-Id: I19aa4ce402b1834d9f68320acb72d7635a41a837

Reviewed-on: https://asterix-gerrit.ics.uci.edu/617

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 34 more files in changeset.
Fix for ASTERIXDB-1291: added regression tests.

Change-Id: I1c8bbaf2522d421dff9a22af28c160f406dd2888

Reviewed-on: https://asterix-gerrit.ics.uci.edu/623

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Wenhai Li <lwhaymail@yahoo.com>

Reviewed-by: Taewoo Kim <wangsaeu@gmail.com>

  1. … 7 more files in changeset.
patch 4

  1. … 94 more files in changeset.
Fixed ASTERIXDB-1249 and ASTERIXDB-1250: self index-nested-loop join correctly identifies the outer and the inner branch.

The first dataset becomes the outer branch.

The second dataset becomes the inner branch.

The optimizer for index-nested-loop join now only try to use an index from the inner branch.

Change-Id: I0d4291197c2bcfbcdcde998c5952af41960c4ad7

Reviewed-on: https://asterix-gerrit.ics.uci.edu/576

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 270 more files in changeset.
Some exception cleanup

- Use HyracksDataException and IOException for errors ingesting external

data (instead of a mixture of AlgebricksException, AsterixException, and

HyracksDataException).

- Avoid unnecessary wrapping of exceptions.

- Wrap exception at boundaries (e.g. HiveRecordParser)

- ADMLexerException and ADMDataParser.ParseException extend

HyracksDataException

- Tighter exception declarations and handling for internal methods.

Change-Id: Icf9591046c44f5fa2281874ff0c98d780e741267

Reviewed-on: https://asterix-gerrit.ics.uci.edu/603

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

  1. … 63 more files in changeset.
Add Support for Upsert Operation

This change allows users to execute upsert commands which

couples delete if found with insert. It locks the primary

keys before doing the search ensuring consistency.

Change-Id: I8999000331795a5949d621d2dd003903e057a521

Reviewed-on: https://asterix-gerrit.ics.uci.edu/477

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 237 more files in changeset.
Add flush() to IFrameWriter

This method is expected to be used with feeds to push

frames all the way to storage when needed. As of now, it is

needed in two cases:

1. No activities in ingestion node and need to push content

so it can be stored.

2. When the ingestion node needs to move the checkpoint ahead

if the at least once semantics are used.

Two feeds make use of this function. The filesystem feed and

couchbase feed which was introduced as well in this change.

Change-Id: Id862ce9e9b1360864c6976f2aea2137092f51203

Reviewed-on: https://asterix-gerrit.ics.uci.edu/585

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

  1. … 103 more files in changeset.
remove end-of-line whitespace

Change-Id: I5c0415f47d4c3a9827574fbdab949b45718d9ea4

Reviewed-on: https://asterix-gerrit.ics.uci.edu/601

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

  1. … 130 more files in changeset.
ASTERIXDB-1005, ASTERIXDB-1263: Clean up subplan flattening: 1. Inline NestedTupleSource and remove SubplanOperator for special cases that join operators inside the SubplanOperator can be re-targeted for correlations; 2. Blindly inline NestedTupleSource and remove SubplanOperator for general cases where the condition of the special case is not met.

Change-Id: I4dd130a25f3c81272cc23f844ea20e376e990612

Reviewed-on: https://asterix-gerrit.ics.uci.edu/579

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 77 more files in changeset.
patch 3

  1. … 96 more files in changeset.
Feed Fixes and Cleanup

1. Introduce filesystem feed data source.

2. Fix the order of closing feed stages on disconnection.

3. Added Twitter feed to the compatibility utility to allow

using it with its alias.

4. First part of the feed log space.

5. Fixed the handling of duplicate key exception.

Change-Id: I4e8db26a810efd1fbaa52ceeb3efd0c8328ab070

Reviewed-on: https://asterix-gerrit.ics.uci.edu/574

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

  1. … 412 more files in changeset.
Enabled Datasets to use Datatypes from foreign Dataverses

Removed broken Metadata Secondary Indexes

Added DatatypeDataverse field to Dataset

Cleaned up Metadata Tests

Added tests for dropping used Datatypes

User-facing changes are:

1) Allow qualified datatype when creating dataset

2) Metadata queries for datasets will now

include the type dataverse

Change-Id: I24dbc04dcb2a4126fc8361ebe3104877a0d1f2bb

Reviewed-on: https://asterix-gerrit.ics.uci.edu/558

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

  1. … 132 more files in changeset.
some cleanup

- remove redundant code around DataSources

- fix typo in constant name

Change-Id: I48602e1acbbc22642d4467ab37f27f337de20c5e

Reviewed-on: https://asterix-gerrit.ics.uci.edu/583

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

  1. … 6 more files in changeset.
Fix for ASTERIXDB-1018, ASTERIXDB-1017, ASTERIXDB-1019, ASTERIXDB-1020, ASTERIXDB-1029, ASTERIXDB-1030, ASTERIXDB-1034

1. Inline SubplanOperator input and hence remove SubplanOperator.

2. Use boolean return from initFromSubTree to confirm that the

result should actually be used in IntroduceJoinAccessMethodRule

Prevent accidental attempted cast of null variable

in AbstractIntroduceAccessMethodrule.

Change-Id: Ifc1f844ac6e06e00da3f1618235b6ce9bfd48e9f

Reviewed-on: https://asterix-gerrit.ics.uci.edu/489

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

  1. … 24 more files in changeset.
ASTERIXDB-1229: - Fixed RemoveRedundantListifyRule to consider general expressions

Change-Id: I5e7b6f5ca4ed51e91de371b9d0b4e4dabdd2f2df

Reviewed-on: https://asterix-gerrit.ics.uci.edu/555

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Preston Carman <prestonc@apache.org>