Clone Tools
  • last updated a few minutes ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
Control Number of Readers for LocalFS Data

Change-Id: Ib9d5ece656220d5f562cc385f882c5ddfd3283a6

Reviewed-on: https://asterix-gerrit.ics.uci.edu/776

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

  1. … 24 more files in changeset.
Allow Replication to be Enabled on Virtual Cluster

- Allow replication port assignment per NC.

- Allow replication to be enabled on virtual cluster.

- Wait for JOB_ABORT ACK from remote replicas.

- Fix LSM component mask file name.

- Fix index directory deletion on index drop.

- Eliminate multiple partition takeover requests.

- Free LogFlusher thread from sending replication ACKs.

- Fix possible deadlock between LogFlusher and Logs Replication Thread.

- Remove wait for FLUSH_LOG for replicated LSM components:

This wait is not needed since on node failure, complete remote recovery is done.

Change-Id: I34a38f59c4915a19242adb6a4eaa6ee1c82d2372

Reviewed-on: https://asterix-gerrit.ics.uci.edu/743

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@apache.org>

  1. … 20 more files in changeset.
ASTERIXDB-865: fix query compilation for if-else expression.

Change-Id: I80e7995e814180fe567818263c79493901af284c

Reviewed-on: https://asterix-gerrit.ics.uci.edu/703

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

    • -2
    • +3
    ./typecomputer/impl/TypeCompatibilityChecker.java
  1. … 21 more files in changeset.
Support Change Feeds and Ingestion of Records with MetaData

This change allows feeds to perform upserts and deletes

in order to perform replication of an external data source.

The change does so by performing the following:

1. The adapter produces [PK][Record]. (Record == null --> delete)

2. The insert is replaced by an upsert operator.

Change-Id: If136a03d424970132dfb09f0dda56e160d4c0078

Reviewed-on: https://asterix-gerrit.ics.uci.edu/621

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -0
    • +3
    ./functions/AsterixBuiltinFunctions.java
  1. … 269 more files in changeset.
Coverity Fixes

FeedMessageOperatorNodePushable

- CID 68721 (#1 of 1): Useless call (USELESS_CALL)

FeedLoadManager

- CID 68719 (#1 of 1): Useless call (USELESS_CALL)

RuntimeRecordTypeInfo

- CID 68717 (#1 of 1): Unused value (UNUSED_VALUE)

HelpCommand

- CID 68591 (#1 of 1): Missing break in switch (MISSING_BREAK)

- CID 68716 (#1 of 1): Unused value (UNUSED_VALUE)

SimilarityJaccardEvaluator

- CID 68714 (#1 of 1): Arguments in wrong order (SWAPPED_ARGUMENTS)

DatasetLifecycleManager

- CID 68713 (#1 of 1): Dereference before null check (REVERSE_INULL)

FeedMetaComputeNodePushable

- CID 68712 (#1 of 2): Dereference before null check (REVERSE_INULL)

- CID 68712 (#2 of 2): Dereference before null check (REVERSE_INULL)

FeedRuntimeInputHandler

- CID 68709 (#1 of 1): Dereference before null check (REVERSE_INULL)

DataflowControllerProvider

- CID 68708 (#1 of 1): Resource leak on an exceptional path

(RESOURCE_LEAK)

TokenLoad

- CID 68697 (#1 of 1): Resource leak (RESOURCE_LEAK)

RecordGroupLengthRange

- CID 68696 (#1 of 1): Resource leak (RESOURCE_LEAK)

RecordManagerGeneratorMojo

- CID 68691 (#1 of 1): Resource leak (RESOURCE_LEAK)

APIServlet

- CID 68687 (#1 of 1): Resource leak (RESOURCE_LEAK)

Change-Id: I8927126f505481e83dd5b9e6e5ccd75bb67d8188

Reviewed-on: https://asterix-gerrit.ics.uci.edu/715

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

    • -1
    • +1
    ./types/runtime/RuntimeRecordTypeInfo.java
  1. … 12 more files in changeset.
ASTERIXDB-1331: fix the type computer for string-length.

Change-Id: Ia21a29049364bd2f99bfd766efdd1518aeffe79f

Reviewed-on: https://asterix-gerrit.ics.uci.edu/689

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <michael.blow@couchbase.com>

Reviewed-by: Ian Maxon <imaxon@apache.org>

    • -5
    • +8
    ./typecomputer/impl/UnaryStringInt64OrNullTypeComputer.java
  1. … 6 more files in changeset.
ASTERIXDB-1329: fix substring type computer for arguments with ANY type.

Added a TinySocial test suite for open datasets.

Change-Id: I23b7d2865cf0bc58b9f402ea602969b326d46d93

Reviewed-on: https://asterix-gerrit.ics.uci.edu/688

Reviewed-by: Ian Maxon <imaxon@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -5
    • +13
    ./typecomputer/impl/Substring2TypeComputer.java
  1. … 74 more files in changeset.
Supprt querying meta record with meta().

Change-Id: Ie65417b6baf209ca0ab413cfa4a5f7fc5156ca63

Reviewed-on: https://asterix-gerrit.ics.uci.edu/685

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

    • -0
    • +7
    ./functions/AsterixBuiltinFunctions.java
    • -11
    • +6
    ./typecomputer/impl/FieldAccessNestedResultType.java
    • -0
    • +43
    ./typecomputer/impl/OptionalOpenARecordTypeComputer.java
  1. … 61 more files in changeset.
Add the filesize parameter to estimate a better partition number for ExternalHashGroupby

Change-Id: I3aadec904af64cc2b7b03e6dff399a2288c21a7b

Reviewed-on: https://asterix-gerrit.ics.uci.edu/451

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

  1. … 1 more file in changeset.
ASTERIXDB-1205: fix union queries.

Fixed the AqlExpressionToPlanTranslator for union expression;

Eliminate shared operator references in translated logical plans;

Removed IntroduceUnionRule;

Fixed tuple source for SQL++ join clause;

Added regression tests.

Change-Id: Ib79a899ec4b35ece6f8f9f59b80126fed0c48851

Reviewed-on: https://asterix-gerrit.ics.uci.edu/629

Reviewed-by: Till Westmann <tillw@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -3
    • +0
    ./functions/AsterixBuiltinFunctions.java
  1. … 72 more files in changeset.
Add single-query-wide unique id generator.

Change-Id: I302f1547c5668250cf4801dbe26ae2aa6f455aa6

Reviewed-on: https://asterix-gerrit.ics.uci.edu/647

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

    • -3
    • +8
    ./functions/AsterixBuiltinFunctions.java
  1. … 7 more files in changeset.
remove unused pagacke org/apache/asterix/om/io

Change-Id: Ib1239b5e6c83b6b93da4f02d83492118bd0e97c2

Reviewed-on: https://asterix-gerrit.ics.uci.edu/652

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Remove Trailing Whitespace

This change removes all trailing whitespace in

java files in asterixdb

Change-Id: I32cda384f71dd5237a3c63e83c661f1cf64dc0eb

Reviewed-on: https://asterix-gerrit.ics.uci.edu/646

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

    • -1
    • +1
    ./functions/FunctionManagerHolder.java
    • -1
    • +1
    ./typecomputer/impl/ARectangleTypeComputer.java
    • -1
    • +1
    ./typecomputer/impl/NonTaggedCollectionMemberResultType.java
    • -1
    • +1
    ./typecomputer/impl/NonTaggedFieldAccessByNameResultType.java
    • -1
    • +1
    ./typecomputer/impl/NonTaggedGetItemResultType.java
    • -1
    • +1
    ./typecomputer/impl/NonTaggedUnaryMinusTypeComputer.java
  1. … 312 more files in changeset.
Asterix NCs Failback Support

- Allow Failed NCs to failback and takeover their partitions.

- New cluster API servlet for cluster state description.

- Remove nodeId from txn logs except remote FLUSH_LOG.

- Add partition id in UPDATE and Entity_COMMIT logs.

- Adapt remote recovery to new logs format.

- Refactor RecoveryManager and split Analysis and Redo phase.

- Spill remote recover logs to temporary file.

- Replicate files to remote replicas for partitions of interest only.

- Introduce NC active/inactive partitions concept.

- Test case for failback.

Change-Id: Id17819542d6b9c4e32647e64737c4a467b630f24

Reviewed-on: https://asterix-gerrit.ics.uci.edu/613

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

    • -71
    • +336
    ./util/AsterixClusterProperties.java
  1. … 92 more files in changeset.
Fix ASTERIXDB-1292

Change-Id: Iee8e9b87145d9b3511874cd11d61ae87acdc94c7

Reviewed-on: https://asterix-gerrit.ics.uci.edu/636

Reviewed-by: Till Westmann <tillw@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

    • -8
    • +8
    ./base/temporal/DateTimeFormatUtils.java
  1. … 4 more files in changeset.
ASTERIXDB-1272 Remove ExternalLookupOperator

ExternalLookupOperator has been removed in this change

and UnnestMapOperator was used instead. different visitors

and methods in the UnnestMapOperator were changed to

support both external and internal data.

Change-Id: Ic9a9400586b75fd8cb3c0b8f4aeb45aee76cdb92

Reviewed-on: https://asterix-gerrit.ics.uci.edu/592

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

    • -6
    • +6
    ./functions/AsterixBuiltinFunctions.java
  1. … 12 more files in changeset.
Move to non-copy-based evaluator interfaces for all function implementations, including: - scalar functions, - aggregate functions, - running aggregate functions, - unnesting functions

Change-Id: If132857cf8d707ce96951f12ba6dd86f54aa4069

Reviewed-on: https://asterix-gerrit.ics.uci.edu/615

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

    • -11
    • +11
    ./functions/AbstractFunctionDescriptor.java
    • -11
    • +11
    ./functions/IFunctionDescriptor.java
    • -74
    • +77
    ./types/hierachy/ATypeHierarchy.java
  1. … 311 more files in changeset.
ASTERIXDB-1271: Revisit UUID Implementation

- Updated UUID implementation to eliminate marshalling to/from a pair

of longs, replacing with byte [].

- Eliminate all but one call to SecureRandom at init time, avoid risk

of entropy exhaustion

- Fix exceptions thrown

- Incorporate constant to represent number of bytes, String chars in

an instance of AUUID

- Split generated from parsed AUUID

- Eliminate intermediate StringBuilder construction on priting paths,

optimize imports

Change-Id: I9e90d42f6b62f80ad180dbd0c8c852db85b14173

Reviewed-on: https://asterix-gerrit.ics.uci.edu/610

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

Reviewed-by: Till Westmann <tillw@apache.org>

    • -0
    • +62
    ./base/AGeneratedUUID.java
  1. … 9 more files in changeset.
ASTERIXDB-1281 - Interval format update to AQL and ADM

The new interval format takes a more generic approach to representing intervals.

Here is an example for a date interval:

interval(date("2012-01-01”), date(”2013-04-01”))

Note that the interval type is defined by the arguments to the interval expression.

Currently only date, time, and datetime types are supported for intervals. The new

format is used for ADM and AQL.

In addition to the format change, the internal byte structure of an interval has been

updated. The format looks like the following:

byte tag, T start, T end (where T is a date, time or datetime type)

Note how the tag has been moved to the front. Also with the new sturcture, an

interval is variable length, not fixed length as before.

Change-Id: I009c71b7a445d141e228ba15d56d0b6cf3c8a3f5

Reviewed-on: https://asterix-gerrit.ics.uci.edu/602

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

    • -9
    • +3
    ./functions/AsterixBuiltinFunctions.java
    • -13
    • +3
    ./pointables/AListVisitablePointable.java
  1. … 67 more files in changeset.
Some exception cleanup

- Use HyracksDataException and IOException for errors ingesting external

data (instead of a mixture of AlgebricksException, AsterixException, and

HyracksDataException).

- Avoid unnecessary wrapping of exceptions.

- Wrap exception at boundaries (e.g. HiveRecordParser)

- ADMLexerException and ADMDataParser.ParseException extend

HyracksDataException

- Tighter exception declarations and handling for internal methods.

Change-Id: Icf9591046c44f5fa2281874ff0c98d780e741267

Reviewed-on: https://asterix-gerrit.ics.uci.edu/603

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

    • -11
    • +6
    ./typecomputer/impl/RecordAddFieldsTypeComputer.java
    • -18
    • +14
    ./typecomputer/impl/RecordMergeTypeComputer.java
    • -21
    • +13
    ./typecomputer/impl/RecordRemoveFieldsTypeComputer.java
  1. … 56 more files in changeset.
Add Support for Upsert Operation

This change allows users to execute upsert commands which

couples delete if found with insert. It locks the primary

keys before doing the search ensuring consistency.

Change-Id: I8999000331795a5949d621d2dd003903e057a521

Reviewed-on: https://asterix-gerrit.ics.uci.edu/477

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

    • -9
    • +9
    ./pointables/ARecordVisitablePointable.java
    • -22
    • +22
    ./pointables/cast/ARecordCaster.java
  1. … 248 more files in changeset.
Add flush() to IFrameWriter

This method is expected to be used with feeds to push

frames all the way to storage when needed. As of now, it is

needed in two cases:

1. No activities in ingestion node and need to push content

so it can be stored.

2. When the ingestion node needs to move the checkpoint ahead

if the at least once semantics are used.

Two feeds make use of this function. The filesystem feed and

couchbase feed which was introduced as well in this change.

Change-Id: Id862ce9e9b1360864c6976f2aea2137092f51203

Reviewed-on: https://asterix-gerrit.ics.uci.edu/585

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

  1. … 103 more files in changeset.
remove end-of-line whitespace

Change-Id: I5c0415f47d4c3a9827574fbdab949b45718d9ea4

Reviewed-on: https://asterix-gerrit.ics.uci.edu/601

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

    • -3
    • +3
    ./base/temporal/ADateParserFactory.java
    • -6
    • +6
    ./base/temporal/ATimeParserFactory.java
    • -1
    • +1
    ./functions/IFunctionDescriptorFactory.java
    • -1
    • +1
    ./pointables/AListVisitablePointable.java
  1. … 126 more files in changeset.
Asterix NCs Fault Tolerance

This change includes the following:

- Adapt replication to unique partitions storage.

- Implement auto failover for failing NCs.

- Implement auto failover for metadata node.

- Fix for ASTERIXDB-1251 using proper error message.

- Basic replication test cases using vagrant virtual cluster for:

1. LSM bulkload components replication.

2. LSM Memory components replication and recovery.

3. Metadata node takeover.

These test cases will be part of the cluster test profile.

Change-Id: Ice26d980912a315fcb3efdd571d6ce88717cfea4

Reviewed-on: https://asterix-gerrit.ics.uci.edu/573

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

    • -12
    • +177
    ./util/AsterixClusterProperties.java
  1. … 70 more files in changeset.
ASTERIXDB-1261, ASTERIXDB-1274, ASTERIXDB-1275 UUID Fixes UUIDPrinter no longer has a UUID object UUIDComparator is used by AbstractComparisonEValuator Removed UUID_STRING type Added test for uuid comparator

Change-Id: Ie33dc8b02e32b3e85a3cb64ef08cb028a50f4ca0

Reviewed-on: https://asterix-gerrit.ics.uci.edu/587

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 20 more files in changeset.
Feed Fixes and Cleanup

1. Introduce filesystem feed data source.

2. Fix the order of closing feed stages on disconnection.

3. Added Twitter feed to the compatibility utility to allow

using it with its alias.

4. First part of the feed log space.

5. Fixed the handling of duplicate key exception.

Change-Id: I4e8db26a810efd1fbaa52ceeb3efd0c8328ab070

Reviewed-on: https://asterix-gerrit.ics.uci.edu/574

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

    • -13
    • +11
    ./util/AsterixClusterProperties.java
  1. … 417 more files in changeset.
ASTERIXDB-1187, ASTERIXDB-1162 and ASTERIXDB-1252 fixes, plus new internal functions

This change contains the following updates and fixes:

- New deep-equality function which also handles numeric equality,

e.g., [2.0, 2, 1.0] == [2, 2, 1].

- New internal functions for record manipulation, including add and remove fields from records

- Fixes for ASTERIXDB-1187: The RecordMergeDescriptor does not fully support

open record merge. Merging records with open fields results in "NullPointerException".

- Fixes to address issue ASTERIXDB-1162: dealing with closed or partly closed record

added to an open record field.

- Fixes issue ASTERIXDB-1252: Problems with Autogenerated Key

Change-Id: I3621ebdd71c7cd91b50d77a972ad863cea7fcbc2

Reviewed-on: https://asterix-gerrit.ics.uci.edu/298

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

    • -8
    • +23
    ./functions/AsterixBuiltinFunctions.java
    • -5
    • +5
    ./pointables/ARecordVisitablePointable.java
    • -0
    • +185
    ./typecomputer/impl/RecordAddFieldsTypeComputer.java
    • -46
    • +38
    ./typecomputer/impl/RecordMergeTypeComputer.java
    • -0
    • +328
    ./typecomputer/impl/RecordRemoveFieldsTypeComputer.java
    • -0
    • +75
    ./typecomputer/impl/TypeComputerUtils.java
    • -7
    • +11
    ./types/hierachy/ATypeHierarchy.java
  1. … 77 more files in changeset.
First stage of external data cleanup

In this change, different parts of external data were refactored.

The goal was to make it more modular, easier to maintain and allow

higher flexibility for extension in addition to reducing code redundancy.

Change-Id: I04a8c4e494d8d1363992b6fe0bdbe6b2b3b7b767

Reviewed-on: https://asterix-gerrit.ics.uci.edu/566

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

  1. … 343 more files in changeset.
Divide Cluster into Unique Partitions

The change includes the following:

- Fix passing NC stores to AsterixConfiguration.

- Unify storage direcotry name in the instance level rather than the node level.

- Divide the cluster into unique storage partitions based on the number of stores.

- Refactored FileSplits and moved out of AqlMetadataProvider.

- Make AsterixHyracksIntegrationUtil use the passed configuration file.

- Make File Splits pass relative index paths of partitions rather than absolute paths.

- Remove unused AqlCompiledMetadataDeclarations class.

Change-Id: I8c7fbca5113dd7ad569a46dfa2591addb5bf8655

Reviewed-on: https://asterix-gerrit.ics.uci.edu/564

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

    • -25
    • +87
    ./util/AsterixClusterProperties.java
  1. … 41 more files in changeset.
Asterix MessageBroker implementation

This change includes the following:

- Add implementation for CC/NC MessageBroker.

- Implement GlobalResourceIdFactory using MessageBroker.

- Change resource id factory to GlobalResourceIdFactory.

- Refactor metadata indexes fixed properties.

- Use fixed resource ids for metadata indexes.

Change-Id: If4320e2c5a0130d2f86a4be6ae61f5cee43e30af

Reviewed-on: https://asterix-gerrit.ics.uci.edu/486

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -0
    • +12
    ./util/AsterixClusterProperties.java
  1. … 26 more files in changeset.