Clone Tools
  • last updated a few minutes ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
Add the Binary data type and corresponding helper functions to Asterix.

The binary data type is implemented as a bytearray. Its storage format

follows the String type which has 2 bytes for length and then store the

bytes contents.

Binary data will take hex("") or base64("") as the constructor method to

passing a hex string or base64 string into Asterix. For output we use

hex("") format.

The parse-[hex|base64](string) function will parse the corresponding hex

or base64 string to binary type. The print-[hex|base64](binary)

functions will print the binary to hex or base64 STRING format.

The sub-binary(binary, offset, [length]) function works the same as

substring(string, offset, [length])

The find-binary(srcbinary, targetbinary, [start-offset]) will find the

position of the targetbinary in the srcbinary.

Change-Id: I5ecf0cc115c44070fb5c1fc5b0ec12a95d4243a4

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/175

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

  1. … 127 more files in changeset.
Implement TokenizeOperator in addition to the changes made by Zachary Heilbron. Now, the compiler builds the bulk-load plan. For other details, please refer to the corresponding Hyracks changes.

Change-Id: I646539d88ea2bdc6da4fbf2b6e9460a6189125ff

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/79

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Young-Seok Kim <kisskys@gmail.com>

  1. … 116 more files in changeset.
cleanup

- refactor getNumberOfIODevices

- remove unused inner class AsterixCluster

- replace NPE with an IllegalStateException with a message

- cleaner log message

Change-Id: Iecad33440bb154b7e438f49755a1d8a2b0c8d677

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/117

Reviewed-by: Ian Maxon <imaxon@uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 1 more file in changeset.
Add external indexes

This change include the following:

1. additional data parser for external data that parse hdfs records using Hive serdes.

2. allow users to create external data. this includes:

a) changes in metadata external dataset details.

b) addition of a new metadata index to store external file's statuses.

c) the pipeline for building the B-Tree and R-Tree indexes.

d) hyracks operators to fetch records with their RIDs using different formats.

e) hyracks operators to lookup and parse external records.

f) test cases for indexing and index access of different hdfs file formats.

g) exposing the secondary indexes over external data to the compiler.

3. adding a new aql command to refresh external datasets. this includes

a) global recovery on system startup.

b) changes in the aql parser.

c) construction of bulk modify pipelines and additional operators to perform local commit and abort operations (using 2PC protocol).

4. Added copyright header to all new files

5. Added additional test cases to test left outer join on external data

Change-Id: I1065a473299f6027eb073aeeba3a56d137f6f98e

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/70

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@uci.edu>

  1. … 150 more files in changeset.
some small cleanup

  1. … 4 more files in changeset.
Merge remote-tracking branch 'origin/raman/master_feeds_merge' into zheilbron/asterix_msr_demo

Conflicts:

.gitignore

asterix-algebra/src/main/java/edu/uci/ics/asterix/optimizer/rules/FeedScanCollectionToUnnest.java

asterix-algebra/src/main/java/edu/uci/ics/asterix/optimizer/rules/IntroduceMaterializationForInsertWithSelfScanRule.java

asterix-algebra/src/main/java/edu/uci/ics/asterix/optimizer/rules/UnnestToDataScanRule.java

asterix-app/src/main/java/edu/uci/ics/asterix/api/common/AsterixAppRuntimeContext.java

asterix-app/src/main/java/edu/uci/ics/asterix/api/common/AsterixAppRuntimeContextProdiverForRecovery.java

asterix-app/src/main/java/edu/uci/ics/asterix/aql/translator/AqlTranslator.java

asterix-app/src/test/resources/metadata/results/basic/issue_251_dataset_hint_2/issue_251_dataset_hint_2.1.adm

asterix-app/src/test/resources/metadata/results/basic/issue_251_dataset_hint_3/issue_251_dataset_hint_3.1.adm

asterix-app/src/test/resources/metadata/results/basic/issue_251_dataset_hint_4/issue_251_dataset_hint_4.1.adm

asterix-app/src/test/resources/metadata/results/basic/meta02/meta02.1.adm

asterix-app/src/test/resources/metadata/results/basic/meta09/meta09.1.adm

asterix-app/src/test/resources/metadata/results/basic/meta16/meta16.1.adm

asterix-app/src/test/resources/metadata/results/basic/meta17/meta17.1.adm

asterix-app/src/test/resources/metadata/results/basic/metadata_dataset/metadata_dataset.1.adm

asterix-app/src/test/resources/metadata/results/basic/metadata_datatype/metadata_datatype.1.adm

asterix-app/src/test/resources/runtimets/results/cross-dataverse/cross-dv02/cross-dv02.1.adm

asterix-app/src/test/resources/runtimets/results/cross-dataverse/cross-dv04/cross-dv04.1.adm

asterix-app/src/test/resources/runtimets/results/cross-dataverse/cross-dv19/cross-dv19.1.adm

asterix-app/src/test/resources/runtimets/results/user-defined-functions/udf23/udf23.1.adm

asterix-aql/src/main/java/edu/uci/ics/asterix/aql/expression/FeedDetailsDecl.java

asterix-aql/src/main/java/edu/uci/ics/asterix/aql/expression/InternalDetailsDecl.java

asterix-aql/src/main/java/edu/uci/ics/asterix/aql/expression/visitor/AQLPrintVisitor.java

asterix-aql/src/main/javacc/AQL.jj

asterix-common/src/main/java/edu/uci/ics/asterix/common/transactions/IAsterixAppRuntimeContextProvider.java

asterix-external-data/src/main/java/edu/uci/ics/asterix/external/dataset/adapter/FileSystemBasedAdapter.java

asterix-installer/src/main/java/edu/uci/ics/asterix/installer/driver/InstallerUtil.java

asterix-metadata/src/main/java/edu/uci/ics/asterix/metadata/MetadataNode.java

asterix-metadata/src/main/java/edu/uci/ics/asterix/metadata/bootstrap/MetadataBootstrap.java

asterix-metadata/src/main/java/edu/uci/ics/asterix/metadata/bootstrap/MetadataRecordTypes.java

asterix-metadata/src/main/java/edu/uci/ics/asterix/metadata/declared/DatasetDataSource.java

asterix-metadata/src/main/java/edu/uci/ics/asterix/metadata/entities/InternalDatasetDetails.java

asterix-metadata/src/main/java/edu/uci/ics/asterix/metadata/entitytupletranslators/DatasetTupleTranslator.java

asterix-metadata/src/main/java/edu/uci/ics/asterix/metadata/feeds/FeedMetaOperatorDescriptor.java

asterix-metadata/src/main/java/edu/uci/ics/asterix/metadata/feeds/FeedUtil.java

asterix-om/src/main/java/edu/uci/ics/asterix/om/functions/AsterixExternalFunctionInfo.java

asterix-om/src/main/java/edu/uci/ics/asterix/om/util/AsterixAppContextInfo.java

asterix-transactions/src/main/java/edu/uci/ics/asterix/transaction/management/opcallbacks/SecondaryIndexModificationOperationCallbackFactory.java

asterix-transactions/src/main/java/edu/uci/ics/asterix/transaction/management/resource/LSMBTreeLocalResourceMetadata.java

asterix-transactions/src/main/java/edu/uci/ics/asterix/transaction/management/service/logging/LogManager.java

asterix-transactions/src/main/java/edu/uci/ics/asterix/transaction/management/service/transaction/AsterixRuntimeComponentsProvider.java

  1. … 53 more files in changeset.
cosmetic changes

    • -1
    • +0
    ./ResettableByteArrayOutputStream.java
  1. … 213 more files in changeset.
checkpoint

  1. … 13 more files in changeset.
merge from master

  1. … 20 more files in changeset.
added UUID type; added autogenerate syntax and AST info

  1. … 42 more files in changeset.
Merge branch 'master' into dev/asterix_msr

Conflicts:

asterix-app/src/main/java/edu/uci/ics/asterix/api/common/AsterixAppRuntimeContext.java

asterix-app/src/main/java/edu/uci/ics/asterix/api/common/AsterixAppRuntimeContextProviderForRecovery.java

asterix-common/src/main/java/edu/uci/ics/asterix/common/api/IAsterixAppRuntimeContext.java

asterix-common/src/main/java/edu/uci/ics/asterix/common/transactions/IAsterixAppRuntimeContextProvider.java

asterix-common/src/main/java/edu/uci/ics/asterix/common/transactions/ITransactionSubsystem.java

asterix-common/src/main/java/edu/uci/ics/asterix/common/transactions/LogUtil.java

asterix-metadata/src/main/java/edu/uci/ics/asterix/metadata/bootstrap/MetadataBootstrap.java

asterix-om/src/main/java/edu/uci/ics/asterix/om/util/AsterixAppContextInfo.java

asterix-transactions/src/main/java/edu/uci/ics/asterix/transaction/management/resource/LSMBTreeLocalResourceMetadata.java

asterix-transactions/src/main/java/edu/uci/ics/asterix/transaction/management/resource/LSMInvertedIndexLocalResourceMetadata.java

asterix-transactions/src/main/java/edu/uci/ics/asterix/transaction/management/resource/LSMRTreeLocalResourceMetadata.java

asterix-transactions/src/main/java/edu/uci/ics/asterix/transaction/management/service/logging/IndexLoggerRepository.java

asterix-transactions/src/main/java/edu/uci/ics/asterix/transaction/management/service/logging/IndexResourceManager.java

asterix-transactions/src/main/java/edu/uci/ics/asterix/transaction/management/service/logging/LogManager.java

asterix-transactions/src/main/java/edu/uci/ics/asterix/transaction/management/service/recovery/RecoveryManager.java

asterix-transactions/src/main/java/edu/uci/ics/asterix/transaction/management/service/transaction/TransactionSubsystem.java

asterix-transactions/src/test/java/edu/uci/ics/asterix/transaction/management/test/TransactionSimulator.java

  1. … 18 more files in changeset.
Added many fixes. Checkpointpoing.

  1. … 24 more files in changeset.
fault-tolerance checkpoint

  1. … 34 more files in changeset.
merge from master

  1. … 39 more files in changeset.
checkpoint

  1. … 8 more files in changeset.
added managix commands for starting/stopping a node

  1. … 43 more files in changeset.
checkpoint

  1. … 56 more files in changeset.
merge up-to-date master to jimahn/master_invix_internal_renaming

  1. … 10 more files in changeset.
Pass absolute directoey info to the lsm indexes.

  1. … 10 more files in changeset.
checkpoint

  1. … 15 more files in changeset.
code cleanup for adaptors/factories

  1. … 35 more files in changeset.
checkpoint post merge from master

  1. … 28 more files in changeset.
completed merge from master

  1. … 39 more files in changeset.
checkpoint

  1. … 18 more files in changeset.
merge from master

  1. … 149 more files in changeset.
add/update license headers

    • -0
    • +14
    ./ResettableByteArrayOutputStream.java
  1. … 1231 more files in changeset.
Internal renaming for inverted index to have prefix SINGLE_PARTITOIN and LENGTH_PARTITOINED

  1. … 127 more files in changeset.
merge from master

  1. … 402 more files in changeset.
Addressed code review comments.

  1. … 5 more files in changeset.
resolved dependency issue around AsterixClusterProperties that escaped the master fix, as it was hiding in a private branch.

    • -0
    • +82
    ./AsterixClusterProperties.java
  1. … 5 more files in changeset.