Clone Tools
  • last updated a few minutes ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
Added filters to AsterixDB's primary and secondary indexes.

Change-Id: If16b2758bb25392c8dd0405d7d59ae503efad3f0

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/82

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

  1. … 120 more files in changeset.
Adds a delivery mode called async-deferred

I added a new result delivery mode called async-deferred, this mode returns a handle once the query completed.

context: sync is a blocking call which returns the result once the query completed, whereas async returns a handle immediately to pull the results later.

async-defer is a hybrid of these two as it returns a handle but only once the query completed.

Change-Id: I8480d22ea9b107edf6d340073581c2613b017599

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/77

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <westmann@gmail.com>

  1. … 9 more files in changeset.
Add external indexes

This change include the following:

1. additional data parser for external data that parse hdfs records using Hive serdes.

2. allow users to create external data. this includes:

a) changes in metadata external dataset details.

b) addition of a new metadata index to store external file's statuses.

c) the pipeline for building the B-Tree and R-Tree indexes.

d) hyracks operators to fetch records with their RIDs using different formats.

e) hyracks operators to lookup and parse external records.

f) test cases for indexing and index access of different hdfs file formats.

g) exposing the secondary indexes over external data to the compiler.

3. adding a new aql command to refresh external datasets. this includes

a) global recovery on system startup.

b) changes in the aql parser.

c) construction of bulk modify pipelines and additional operators to perform local commit and abort operations (using 2PC protocol).

4. Added copyright header to all new files

5. Added additional test cases to test left outer join on external data

Change-Id: I1065a473299f6027eb073aeeba3a56d137f6f98e

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/70

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@uci.edu>

    • -39
    • +538
    ./translator/AqlTranslator.java
  1. … 150 more files in changeset.
Support sort-based group-by, add test coverage for out-of-core code paths, and adapt to the new buffer cache interface. -- add the support for sort-based group-by -- add test coverages for disk-based code path, including multi-pass code paths -- populate framesize and group-by buffer size into asterix -- adapt to new interface for buffer cache

Change-Id: I4af9eaa6fa6a8ae76b8ecaa39184785a90b32710

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/36

Tested-by: Ian Maxon <imaxon@uci.edu>

Reviewed-by: Till Westmann <westmann@gmail.com>

  1. … 39 more files in changeset.
Moved the skip secondary index search hint closer to the predicates.

  1. … 22 more files in changeset.
Added a hint to AQL to exclude secondary indexes from the query plan.

  1. … 29 more files in changeset.
reenable feed jobspec alteration + logging to metadata (bug filed)

  1. … 1 more file in changeset.
rename LoadFromFileStatement to LoadStatement

  1. … 15 more files in changeset.
ensure rest api returns a single json object with a single 'results' field for each query

  1. … 2 more files in changeset.
Merge remote-tracking branch 'origin/raman/master_feeds_merge' into zheilbron/asterix_msr_demo

Conflicts:

.gitignore

asterix-algebra/src/main/java/edu/uci/ics/asterix/optimizer/rules/FeedScanCollectionToUnnest.java

asterix-algebra/src/main/java/edu/uci/ics/asterix/optimizer/rules/IntroduceMaterializationForInsertWithSelfScanRule.java

asterix-algebra/src/main/java/edu/uci/ics/asterix/optimizer/rules/UnnestToDataScanRule.java

asterix-app/src/main/java/edu/uci/ics/asterix/api/common/AsterixAppRuntimeContext.java

asterix-app/src/main/java/edu/uci/ics/asterix/api/common/AsterixAppRuntimeContextProdiverForRecovery.java

asterix-app/src/main/java/edu/uci/ics/asterix/aql/translator/AqlTranslator.java

asterix-app/src/test/resources/metadata/results/basic/issue_251_dataset_hint_2/issue_251_dataset_hint_2.1.adm

asterix-app/src/test/resources/metadata/results/basic/issue_251_dataset_hint_3/issue_251_dataset_hint_3.1.adm

asterix-app/src/test/resources/metadata/results/basic/issue_251_dataset_hint_4/issue_251_dataset_hint_4.1.adm

asterix-app/src/test/resources/metadata/results/basic/meta02/meta02.1.adm

asterix-app/src/test/resources/metadata/results/basic/meta09/meta09.1.adm

asterix-app/src/test/resources/metadata/results/basic/meta16/meta16.1.adm

asterix-app/src/test/resources/metadata/results/basic/meta17/meta17.1.adm

asterix-app/src/test/resources/metadata/results/basic/metadata_dataset/metadata_dataset.1.adm

asterix-app/src/test/resources/metadata/results/basic/metadata_datatype/metadata_datatype.1.adm

asterix-app/src/test/resources/runtimets/results/cross-dataverse/cross-dv02/cross-dv02.1.adm

asterix-app/src/test/resources/runtimets/results/cross-dataverse/cross-dv04/cross-dv04.1.adm

asterix-app/src/test/resources/runtimets/results/cross-dataverse/cross-dv19/cross-dv19.1.adm

asterix-app/src/test/resources/runtimets/results/user-defined-functions/udf23/udf23.1.adm

asterix-aql/src/main/java/edu/uci/ics/asterix/aql/expression/FeedDetailsDecl.java

asterix-aql/src/main/java/edu/uci/ics/asterix/aql/expression/InternalDetailsDecl.java

asterix-aql/src/main/java/edu/uci/ics/asterix/aql/expression/visitor/AQLPrintVisitor.java

asterix-aql/src/main/javacc/AQL.jj

asterix-common/src/main/java/edu/uci/ics/asterix/common/transactions/IAsterixAppRuntimeContextProvider.java

asterix-external-data/src/main/java/edu/uci/ics/asterix/external/dataset/adapter/FileSystemBasedAdapter.java

asterix-installer/src/main/java/edu/uci/ics/asterix/installer/driver/InstallerUtil.java

asterix-metadata/src/main/java/edu/uci/ics/asterix/metadata/MetadataNode.java

asterix-metadata/src/main/java/edu/uci/ics/asterix/metadata/bootstrap/MetadataBootstrap.java

asterix-metadata/src/main/java/edu/uci/ics/asterix/metadata/bootstrap/MetadataRecordTypes.java

asterix-metadata/src/main/java/edu/uci/ics/asterix/metadata/declared/DatasetDataSource.java

asterix-metadata/src/main/java/edu/uci/ics/asterix/metadata/entities/InternalDatasetDetails.java

asterix-metadata/src/main/java/edu/uci/ics/asterix/metadata/entitytupletranslators/DatasetTupleTranslator.java

asterix-metadata/src/main/java/edu/uci/ics/asterix/metadata/feeds/FeedMetaOperatorDescriptor.java

asterix-metadata/src/main/java/edu/uci/ics/asterix/metadata/feeds/FeedUtil.java

asterix-om/src/main/java/edu/uci/ics/asterix/om/functions/AsterixExternalFunctionInfo.java

asterix-om/src/main/java/edu/uci/ics/asterix/om/util/AsterixAppContextInfo.java

asterix-transactions/src/main/java/edu/uci/ics/asterix/transaction/management/opcallbacks/SecondaryIndexModificationOperationCallbackFactory.java

asterix-transactions/src/main/java/edu/uci/ics/asterix/transaction/management/resource/LSMBTreeLocalResourceMetadata.java

asterix-transactions/src/main/java/edu/uci/ics/asterix/transaction/management/service/logging/LogManager.java

asterix-transactions/src/main/java/edu/uci/ics/asterix/transaction/management/service/transaction/AsterixRuntimeComponentsProvider.java

    • -31
    • +124
    ./translator/AqlTranslator.java
  1. … 53 more files in changeset.
cosmetic changes

  1. … 214 more files in changeset.
miscellaneous/minor changes + support for installing adapters

  1. … 101 more files in changeset.
minor cosmetic changes

  1. … 19 more files in changeset.
merged master, resolved conflicts

  1. … 26 more files in changeset.
Merge branch 'raman/master_feeds_ext' of https://code.google.com/p/asterixdb into raman/master_feeds_ext

checkpoint

  1. … 43 more files in changeset.
merge from master

  1. … 20 more files in changeset.
Allowed user to specify the merging policy

  1. … 59 more files in changeset.
checkpoint

  1. … 2 more files in changeset.
Support for full compaction command.

  1. … 32 more files in changeset.
Revert "Merge external indexes into master"

This reverts commit 1a0d2b4b80677ddadb524503ad9c84e2f03aade7, reversing

changes made to 5eba6a0795f2278f4ea7747f8b54c858529b5502.

  1. … 56 more files in changeset.
Merge external indexes into master

  1. … 10 more files in changeset.
Correct behaviour of drop dataverse and drop dataset to drop external data indexes

  1. … 5 more files in changeset.
Merge remote-tracking branch 'origin/raman/master_feeds' into zheilbron/asterix_msr_demo

Conflicts:

asterix-aql/src/main/java/edu/uci/ics/asterix/aql/expression/ConnectFeedStatement.java

asterix-metadata/src/main/java/edu/uci/ics/asterix/metadata/feeds/FeedUtil.java

  1. … 8 more files in changeset.
disable feed statistic collection

add R-Tree support for external indexes and indexes optimizations

  1. … 27 more files in changeset.
added UUID type; added autogenerate syntax and AST info

  1. … 42 more files in changeset.
Merge branch 'master' into dev/asterix_msr

Conflicts:

asterix-app/src/main/java/edu/uci/ics/asterix/api/common/AsterixAppRuntimeContext.java

asterix-app/src/main/java/edu/uci/ics/asterix/api/common/AsterixAppRuntimeContextProviderForRecovery.java

asterix-common/src/main/java/edu/uci/ics/asterix/common/api/IAsterixAppRuntimeContext.java

asterix-common/src/main/java/edu/uci/ics/asterix/common/transactions/IAsterixAppRuntimeContextProvider.java

asterix-common/src/main/java/edu/uci/ics/asterix/common/transactions/ITransactionSubsystem.java

asterix-common/src/main/java/edu/uci/ics/asterix/common/transactions/LogUtil.java

asterix-metadata/src/main/java/edu/uci/ics/asterix/metadata/bootstrap/MetadataBootstrap.java

asterix-om/src/main/java/edu/uci/ics/asterix/om/util/AsterixAppContextInfo.java

asterix-transactions/src/main/java/edu/uci/ics/asterix/transaction/management/resource/LSMBTreeLocalResourceMetadata.java

asterix-transactions/src/main/java/edu/uci/ics/asterix/transaction/management/resource/LSMInvertedIndexLocalResourceMetadata.java

asterix-transactions/src/main/java/edu/uci/ics/asterix/transaction/management/resource/LSMRTreeLocalResourceMetadata.java

asterix-transactions/src/main/java/edu/uci/ics/asterix/transaction/management/service/logging/IndexLoggerRepository.java

asterix-transactions/src/main/java/edu/uci/ics/asterix/transaction/management/service/logging/IndexResourceManager.java

asterix-transactions/src/main/java/edu/uci/ics/asterix/transaction/management/service/logging/LogManager.java

asterix-transactions/src/main/java/edu/uci/ics/asterix/transaction/management/service/recovery/RecoveryManager.java

asterix-transactions/src/main/java/edu/uci/ics/asterix/transaction/management/service/transaction/TransactionSubsystem.java

asterix-transactions/src/test/java/edu/uci/ics/asterix/transaction/management/test/TransactionSimulator.java

  1. … 18 more files in changeset.
another change to stop the LogFlusher thread

  1. … 3 more files in changeset.
removed the information of the numOfPartitions and refined synchronization blocks in transaction related classes.

  1. … 10 more files in changeset.