Clone Tools
  • last updated 26 mins ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
Allow Replication to be Enabled on Virtual Cluster

- Allow replication port assignment per NC.

- Allow replication to be enabled on virtual cluster.

- Wait for JOB_ABORT ACK from remote replicas.

- Fix LSM component mask file name.

- Fix index directory deletion on index drop.

- Eliminate multiple partition takeover requests.

- Free LogFlusher thread from sending replication ACKs.

- Fix possible deadlock between LogFlusher and Logs Replication Thread.

- Remove wait for FLUSH_LOG for replicated LSM components:

This wait is not needed since on node failure, complete remote recovery is done.

Change-Id: I34a38f59c4915a19242adb6a4eaa6ee1c82d2372

Reviewed-on: https://asterix-gerrit.ics.uci.edu/743

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@apache.org>

  1. … 20 more files in changeset.
Coverity Fixes

FeedMessageOperatorNodePushable

- CID 68721 (#1 of 1): Useless call (USELESS_CALL)

FeedLoadManager

- CID 68719 (#1 of 1): Useless call (USELESS_CALL)

RuntimeRecordTypeInfo

- CID 68717 (#1 of 1): Unused value (UNUSED_VALUE)

HelpCommand

- CID 68591 (#1 of 1): Missing break in switch (MISSING_BREAK)

- CID 68716 (#1 of 1): Unused value (UNUSED_VALUE)

SimilarityJaccardEvaluator

- CID 68714 (#1 of 1): Arguments in wrong order (SWAPPED_ARGUMENTS)

DatasetLifecycleManager

- CID 68713 (#1 of 1): Dereference before null check (REVERSE_INULL)

FeedMetaComputeNodePushable

- CID 68712 (#1 of 2): Dereference before null check (REVERSE_INULL)

- CID 68712 (#2 of 2): Dereference before null check (REVERSE_INULL)

FeedRuntimeInputHandler

- CID 68709 (#1 of 1): Dereference before null check (REVERSE_INULL)

DataflowControllerProvider

- CID 68708 (#1 of 1): Resource leak on an exceptional path

(RESOURCE_LEAK)

TokenLoad

- CID 68697 (#1 of 1): Resource leak (RESOURCE_LEAK)

RecordGroupLengthRange

- CID 68696 (#1 of 1): Resource leak (RESOURCE_LEAK)

RecordManagerGeneratorMojo

- CID 68691 (#1 of 1): Resource leak (RESOURCE_LEAK)

APIServlet

- CID 68687 (#1 of 1): Resource leak (RESOURCE_LEAK)

Change-Id: I8927126f505481e83dd5b9e6e5ccd75bb67d8188

Reviewed-on: https://asterix-gerrit.ics.uci.edu/715

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

  1. … 12 more files in changeset.
Remove Trailing Whitespace

This change removes all trailing whitespace in

java files in asterixdb

Change-Id: I32cda384f71dd5237a3c63e83c661f1cf64dc0eb

Reviewed-on: https://asterix-gerrit.ics.uci.edu/646

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 326 more files in changeset.
Asterix NCs Fault Tolerance

This change includes the following:

- Adapt replication to unique partitions storage.

- Implement auto failover for failing NCs.

- Implement auto failover for metadata node.

- Fix for ASTERIXDB-1251 using proper error message.

- Basic replication test cases using vagrant virtual cluster for:

1. LSM bulkload components replication.

2. LSM Memory components replication and recovery.

3. Metadata node takeover.

These test cases will be part of the cluster test profile.

Change-Id: Ice26d980912a315fcb3efdd571d6ce88717cfea4

Reviewed-on: https://asterix-gerrit.ics.uci.edu/573

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

  1. … 71 more files in changeset.
Divide Cluster into Unique Partitions

The change includes the following:

- Fix passing NC stores to AsterixConfiguration.

- Unify storage direcotry name in the instance level rather than the node level.

- Divide the cluster into unique storage partitions based on the number of stores.

- Refactored FileSplits and moved out of AqlMetadataProvider.

- Make AsterixHyracksIntegrationUtil use the passed configuration file.

- Make File Splits pass relative index paths of partitions rather than absolute paths.

- Remove unused AqlCompiledMetadataDeclarations class.

Change-Id: I8c7fbca5113dd7ad569a46dfa2591addb5bf8655

Reviewed-on: https://asterix-gerrit.ics.uci.edu/564

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

  1. … 40 more files in changeset.
Clean up compilation warnings.

Change-Id: Idbfcd9c67f91d373c5f7269125778a5681021227

Reviewed-on: https://asterix-gerrit.ics.uci.edu/505

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

  1. … 109 more files in changeset.
Introducing Data Replication To AsterixDB

This change includes the following:

- Add data replication properties to cluster properties and Managix validate command.

- Introduce Data Replication components.

- Add data replication required fields to LogRecord.

- Specialized LogManager for data replication.

- Fix for invalid cluster state on nodes failure.

- ASTERIXDB-139: Fix for cleaning workspace files on startup/shutdown.

- Fix for temp datasets storage reclamation.

- Allow MetadataNode rebinding with CC.

- Add flag to checkpoint to identify sharp checkpoints.

- ASTERIXDB-1170: Fix shutdown sequence

Change-Id: I729fdd1144dbc9ff039b4bc414494860d7553810

Reviewed-on: https://asterix-gerrit.ics.uci.edu/338

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Young-Seok Kim <kisskys@gmail.com>

  1. … 73 more files in changeset.
This change fixes the sporadic connection refused errors

This can mostly be seen during asterix-installer tests.

The fix basically lets managix wait and listens

through zookeeper for a signal from the cluster controller that

the cluster is up and ready.

Once the cluster controller sends the signal, Managix can proceed.

Change-Id: Ib730f50ab2fb492f3cf973d1cf2f03b34e24e5b3

Reviewed-on: https://asterix-gerrit.ics.uci.edu/366

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

  1. … 10 more files in changeset.
Change License Headers

Also tweak the NOTICE file with some extras.

Change-Id:I09bc388089e515d7f51fd39c31bfbbc9f00cf84f

Reviewed-on: https://asterix-gerrit.ics.uci.edu/388

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

    • -10
    • +14
    ./org/apache/asterix/installer/command/ICommand.java
  1. … 2004 more files in changeset.
Change Java package from edu.uci.ics to org.apache

Change-Id: I2f01d2b5614e9e9c94fda4bf1294a8eba6a26c54

Reviewed-on: https://asterix-gerrit.ics.uci.edu/309

Reviewed-by: Till Westmann <tillw@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 2590 more files in changeset.
Change folder structure for Java repackage

Change only the folders, not the files, for our package name change.

This will break the build, and needs to be followed by a change to

the package name in all of the source files. However performing

the folder move and file change in two steps lets Git understand

that the files are the same, and lets us track revisions across

those files.

Change-Id: Iefd2a576415ebc1416cba2a3334d2b64f042ba92

Reviewed-on: https://asterix-gerrit.ics.uci.edu/306

Tested-by: Ian Maxon <imaxon@apache.org>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 3781 more files in changeset.
Add Apache RAT License Auditor plugin, and fix missing licenses

Change-Id: Ic0fa0b7836e8a69d0d8beed83f9c3b41c168afd4

Reviewed-on: https://asterix-gerrit.ics.uci.edu/263

Reviewed-by: Chris Hillery <ceej@lambda.nu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 1530 more files in changeset.
Added a flag to identify first NC start during creation

Change-Id: If1b76ec83ae19f9c643a223e44712228ed7037f5

Reviewed-on: https://asterix-gerrit.ics.uci.edu/272

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@uci.edu>

  1. … 5 more files in changeset.
Issue 705: Installation documentation

Fix errors in installer.md. Reorganize and expand installer.md

slightly. Add check for txn_log_dir to 'managix validate -c'.

Change-Id: Ib3396425c372c4970b86a792678e26f2e37b3630

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/154

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@uci.edu>

  1. … 1 more file in changeset.
improvements for demo

- add demo cluster

- add example mugshot.com data and queries

- automatic configuration of a cluster

- set MANAGIX_HOME implicitly

- better AQL lexer error reporting

- better eror reporting in ADM parser

- fix and optimize construction of asterix zip for deployment

Change-Id: I9d80402cef72a15271766031b6f06c7220e4ad5a

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/95

Reviewed-by: Ian Maxon <imaxon@uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Raman Grover <ramang@uci.edu>

  1. … 35 more files in changeset.
checkpoint

  1. … 24 more files in changeset.
Merge remote-tracking branch 'origin/raman/master_feeds_merge' into zheilbron/asterix_msr_demo

Conflicts:

.gitignore

asterix-algebra/src/main/java/edu/uci/ics/asterix/optimizer/rules/FeedScanCollectionToUnnest.java

asterix-algebra/src/main/java/edu/uci/ics/asterix/optimizer/rules/IntroduceMaterializationForInsertWithSelfScanRule.java

asterix-algebra/src/main/java/edu/uci/ics/asterix/optimizer/rules/UnnestToDataScanRule.java

asterix-app/src/main/java/edu/uci/ics/asterix/api/common/AsterixAppRuntimeContext.java

asterix-app/src/main/java/edu/uci/ics/asterix/api/common/AsterixAppRuntimeContextProdiverForRecovery.java

asterix-app/src/main/java/edu/uci/ics/asterix/aql/translator/AqlTranslator.java

asterix-app/src/test/resources/metadata/results/basic/issue_251_dataset_hint_2/issue_251_dataset_hint_2.1.adm

asterix-app/src/test/resources/metadata/results/basic/issue_251_dataset_hint_3/issue_251_dataset_hint_3.1.adm

asterix-app/src/test/resources/metadata/results/basic/issue_251_dataset_hint_4/issue_251_dataset_hint_4.1.adm

asterix-app/src/test/resources/metadata/results/basic/meta02/meta02.1.adm

asterix-app/src/test/resources/metadata/results/basic/meta09/meta09.1.adm

asterix-app/src/test/resources/metadata/results/basic/meta16/meta16.1.adm

asterix-app/src/test/resources/metadata/results/basic/meta17/meta17.1.adm

asterix-app/src/test/resources/metadata/results/basic/metadata_dataset/metadata_dataset.1.adm

asterix-app/src/test/resources/metadata/results/basic/metadata_datatype/metadata_datatype.1.adm

asterix-app/src/test/resources/runtimets/results/cross-dataverse/cross-dv02/cross-dv02.1.adm

asterix-app/src/test/resources/runtimets/results/cross-dataverse/cross-dv04/cross-dv04.1.adm

asterix-app/src/test/resources/runtimets/results/cross-dataverse/cross-dv19/cross-dv19.1.adm

asterix-app/src/test/resources/runtimets/results/user-defined-functions/udf23/udf23.1.adm

asterix-aql/src/main/java/edu/uci/ics/asterix/aql/expression/FeedDetailsDecl.java

asterix-aql/src/main/java/edu/uci/ics/asterix/aql/expression/InternalDetailsDecl.java

asterix-aql/src/main/java/edu/uci/ics/asterix/aql/expression/visitor/AQLPrintVisitor.java

asterix-aql/src/main/javacc/AQL.jj

asterix-common/src/main/java/edu/uci/ics/asterix/common/transactions/IAsterixAppRuntimeContextProvider.java

asterix-external-data/src/main/java/edu/uci/ics/asterix/external/dataset/adapter/FileSystemBasedAdapter.java

asterix-installer/src/main/java/edu/uci/ics/asterix/installer/driver/InstallerUtil.java

asterix-metadata/src/main/java/edu/uci/ics/asterix/metadata/MetadataNode.java

asterix-metadata/src/main/java/edu/uci/ics/asterix/metadata/bootstrap/MetadataBootstrap.java

asterix-metadata/src/main/java/edu/uci/ics/asterix/metadata/bootstrap/MetadataRecordTypes.java

asterix-metadata/src/main/java/edu/uci/ics/asterix/metadata/declared/DatasetDataSource.java

asterix-metadata/src/main/java/edu/uci/ics/asterix/metadata/entities/InternalDatasetDetails.java

asterix-metadata/src/main/java/edu/uci/ics/asterix/metadata/entitytupletranslators/DatasetTupleTranslator.java

asterix-metadata/src/main/java/edu/uci/ics/asterix/metadata/feeds/FeedMetaOperatorDescriptor.java

asterix-metadata/src/main/java/edu/uci/ics/asterix/metadata/feeds/FeedUtil.java

asterix-om/src/main/java/edu/uci/ics/asterix/om/functions/AsterixExternalFunctionInfo.java

asterix-om/src/main/java/edu/uci/ics/asterix/om/util/AsterixAppContextInfo.java

asterix-transactions/src/main/java/edu/uci/ics/asterix/transaction/management/opcallbacks/SecondaryIndexModificationOperationCallbackFactory.java

asterix-transactions/src/main/java/edu/uci/ics/asterix/transaction/management/resource/LSMBTreeLocalResourceMetadata.java

asterix-transactions/src/main/java/edu/uci/ics/asterix/transaction/management/service/logging/LogManager.java

asterix-transactions/src/main/java/edu/uci/ics/asterix/transaction/management/service/transaction/AsterixRuntimeComponentsProvider.java

  1. … 53 more files in changeset.
cosmetic changes

  1. … 213 more files in changeset.
merge from zheilbron/asterix_msr (checkpoint 3)

  1. … 3 more files in changeset.
merge from zheilbron/asterix_msr (checkpoint 1)

  1. … 15 more files in changeset.
Managix was overly strict in not allowing multiple instances share an IP as it can potentially cause port conflicts (all ports (except 3) are non-configurable). This changes relaxes the constraint to allow multiple instances share an IP. However the instance cannot be ACTIVE concurrently.

checkpoint

  1. … 15 more files in changeset.
added test case for fault-tolerance

  1. … 11 more files in changeset.
added managix commands for starting/stopping a node

  1. … 40 more files in changeset.
added managix commands for starting/stopping a node

  1. … 8 more files in changeset.
checkpoint

  1. … 56 more files in changeset.
configured appassembler to only use booter scripts for windows; renamed lib --> repo since appassembler's scripts don't support configurable repo names for booters

  1. … 2 more files in changeset.
merged raman/asterix_lsm_stabilization_udfs

  1. … 33 more files in changeset.
resolved conflicts post merge

  1. … 4 more files in changeset.
merge from master

  1. … 41 more files in changeset.