Clone Tools
  • last updated 20 mins ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
Change folder structure for Java repackage

Change only the folders, not the files, for our package name change.

This will break the build, and needs to be followed by a change to

the package name in all of the source files. However performing

the folder move and file change in two steps lets Git understand

that the files are the same, and lets us track revisions across

those files.

Change-Id: Iefd2a576415ebc1416cba2a3334d2b64f042ba92

Reviewed-on: https://asterix-gerrit.ics.uci.edu/306

Tested-by: Ian Maxon <imaxon@apache.org>

Reviewed-by: Till Westmann <tillw@apache.org>

    • -153
    • +0
    ./factory/CNNFeedAdapterFactory.java
    • -342
    • +0
    ./factory/HDFSAdapterFactory.java
    • -208
    • +0
    ./factory/HDFSIndexingAdapterFactory.java
    • -123
    • +0
    ./factory/HiveAdapterFactory.java
    • -164
    • +0
    ./factory/NCFileSystemAdapterFactory.java
    • -169
    • +0
    ./factory/PullBasedAzureTwitterAdapterFactory.java
    • -116
    • +0
    ./factory/PullBasedTwitterAdapterFactory.java
    • -105
    • +0
    ./factory/PushBasedTwitterAdapterFactory.java
    • -111
    • +0
    ./factory/RSSFeedAdapterFactory.java
    • -45
    • +0
    ./factory/StreamBasedAdapterFactory.java
  1. … 3786 more files in changeset.
Changed metadata storage format for nullable field types. Moved field name generation to the client out of metadata node code. Changed naming scheme for autogenerated types. Moved GroupName, CompactionPolicy & CompactionPolicyProperties fields up from External\InternalDetails to Dataset record type definition

Change-Id: I223aded8aaf80f0688358899c0e8b0d6988fac93

Reviewed-on: https://asterix-gerrit.ics.uci.edu/323

Reviewed-by: Till Westmann <tillw@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -4
    • +3
    ./factory/HDFSIndexingAdapterFactory.java
    • -2
    • +0
    ./factory/StreamBasedAdapterFactory.java
  1. … 205 more files in changeset.
whitespace fix for commit b361b06

Change-Id: I9b01a2cacb2a8efe0a4c0bb97436a9c59edb8e42

Reviewed-on: https://asterix-gerrit.ics.uci.edu/357

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@apache.org>

    • -15
    • +14
    ./factory/PushBasedTwitterAdapterFactory.java
  1. … 3 more files in changeset.
Fix for Issue 929

commit 58317b37a3a7b2546b4780f4427ae1ed21a4ece9

Author: Ubuntu <raman@ramangro.ramangro.d3.internal.cloudapp.net>

Date: Fri Aug 14 10:47:06 2015 +0000

Fix for Issue 929:

a) Added documenation for use of OAuth keys and tokens when using the built-in Twitter adaptor

b) Modified RSS feed adaptor and added documenation

Change-Id: I5521287a4fa1818c78a4f83b1a3cabeea8e6096d

Reviewed-on: https://asterix-gerrit.ics.uci.edu/354

Reviewed-by: Ian Maxon <imaxon@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -0
    • +23
    ./factory/PushBasedTwitterAdapterFactory.java
  1. … 7 more files in changeset.
This change includes the following: 1. allowing short-circuit reads to be used with external data. 2. enhancing behaviour of index access for Text data. 3. updating the external data documentation.

Rebased on current Master.

The following commits from your working branch will be included:

commit 3176d741b8ef187703c7346044c9531a99f2b716

Author: Abdullah Alamoudi <bamousaa@gmail.com>

Date: Wed Feb 4 18:32:24 2015 +0300

added changes from external experiments and reflected external data documentation to include external data indexing

Change-Id: I041c71391d8704cd800c4446a085beef197e7acf

Reviewed-on: https://asterix-gerrit.ics.uci.edu/218

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Raman Grover <ramangrover29@gmail.com>

  1. … 10 more files in changeset.
Add Apache RAT License Auditor plugin, and fix missing licenses

Change-Id: Ic0fa0b7836e8a69d0d8beed83f9c3b41c168afd4

Reviewed-on: https://asterix-gerrit.ics.uci.edu/263

Reviewed-by: Chris Hillery <ceej@lambda.nu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -0
    • +14
    ./factory/PullBasedAzureTwitterAdapterFactory.java
    • -2
    • +2
    ./factory/PullBasedTwitterAdapterFactory.java
    • -0
    • +14
    ./factory/PushBasedTwitterAdapterFactory.java
  1. … 1539 more files in changeset.
Introduces Feeds 2.0

commit c3f577861fc705d848c1641605689cadd6973bae

Merge: ebc4cae fc0c2c0

Author: ramangrover29 <ramangrover29@gmail.com>

Date: Fri Jun 26 13:04:05 2015 -0700

Merge branch 'raman/feeds_2_release' of https://code.google.com/p/asterixdb-sandbox into raman/feeds_2_release

Conflicts:

asterix-app/src/main/java/edu/uci/ics/asterix/api/http/servlet/FeedServlet.java

asterix-external-data/src/main/java/edu/uci/ics/asterix/external/library/java/JObjectAccessors.java

commit ebc4cae21a7302869f953df1ebda601e798d12d2

Author: ramangrover29 <ramangrover29@gmail.com>

Date: Sat Jun 20 17:14:45 2015 -0700

Introduces Feeds 2.0

Some of the prominent chnages introduced are as follows

a) Support for building a cascade network of feeds (via secondary feeds feature)

b) Feed Management Console for tracking active feeds and associated metrics

c) Support for elastic runtime for data ingestion

d) Improved fault-tolerance with support for logging of failed records

Documentation has been added at asterix-doc/src/site/markdown/feeds/

commit fc0c2c0549a6ee8b202e57607d2e110478cd57bb

Author: ramangrover29 <ramangrover29@gmail.com>

Date: Sat Jun 20 17:14:45 2015 -0700

Introduces Feeds 2.0

Some of the prominent chnages introduced are as follows

a) Support for building a cascade network of feeds (via secondary feeds feature)

b) Feed Management Console for tracking active feeds and associated metrics

c) Support for elastic runtime for data ingestion

d) Improved fault-tolerance with support for logging of failed records

Documentation has been added at asterix-doc/src/site/markdown/feeds/

Change-Id: I498f01c591a229aaf51cec43ab20f3e5c4f072f4

Reviewed-on: https://asterix-gerrit.ics.uci.edu/297

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Steven Jacobs <sjaco002@ucr.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

    • -15
    • +18
    ./factory/CNNFeedAdapterFactory.java
    • -21
    • +35
    ./factory/HDFSAdapterFactory.java
    • -12
    • +28
    ./factory/HDFSIndexingAdapterFactory.java
    • -11
    • +20
    ./factory/HiveAdapterFactory.java
    • -10
    • +20
    ./factory/NCFileSystemAdapterFactory.java
    • -13
    • +29
    ./factory/PullBasedAzureTwitterAdapterFactory.java
    • -28
    • +55
    ./factory/PullBasedTwitterAdapterFactory.java
    • -0
    • +69
    ./factory/PushBasedTwitterAdapterFactory.java
    • -18
    • +25
    ./factory/RSSFeedAdapterFactory.java
    • -151
    • +8
    ./factory/StreamBasedAdapterFactory.java
  1. … 374 more files in changeset.
Added support of typed indexes over open fields & indexes over nested fields

Open indexes requires user to provide a type along with a indexed field name.

This type would be enforced for all the indexed records, i.e. index cannot be created if in some records a field with provided name has a different type.

Index-specific rewrite rules match provided type with the inferred types of other arguments in join\select statements and trigger index rewrite.

Nested indexes use the same semantics as the regular indexes, with exception that field could be located arbitrarily deep inside nested structure

Change-Id: I53d00aba243ccf7cf79cf7d775dd305813d24f98

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/97

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Steven Jacobs <sjaco002@ucr.edu>

    • -5
    • +3
    ./factory/HDFSIndexingAdapterFactory.java
  1. … 1791 more files in changeset.
This change covers three points: 1. move the generation of UUIDs out of parsers 2. fixes a bug in the metadata lock when performing load operations 3. the external data scan operator is again a leaf operator 4. made sure the startup call doesn't return until the servers are ready to receive requests Merged master into branch for the second time.

The following commits from your working branch will be included:

commit da929c990249fe7f0699417fd7f7ea34908355af

Author: Abdullah Alamoudi <bamousaa@gmail.com>

Date: Fri Jan 16 11:51:14 2015 +0300

Moved generation of UUIDs out of parsers

Change-Id: I32437493cf48c79417f4280e9d8a85cfb1559b62

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/205

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@uci.edu>

Reviewed-by: Taewoo Kim <wangsaeu@gmail.com>

    • -9
    • +5
    ./factory/HDFSIndexingAdapterFactory.java
    • -41
    • +5
    ./factory/NCFileSystemAdapterFactory.java
    • -17
    • +10
    ./factory/StreamBasedAdapterFactory.java
  1. … 56 more files in changeset.
Introduce ("header"="true") option for CSV parsing

Change-Id: I473bc7b66d0c0d1355b5fb3e392cb9ece151e50e

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/226

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@uci.edu>

Tested-by: Chris Hillery <ceej@lambda.nu>

    • -5
    • +6
    ./factory/HDFSIndexingAdapterFactory.java
    • -9
    • +15
    ./factory/StreamBasedAdapterFactory.java
  1. … 10 more files in changeset.
Add the Binary data type and corresponding helper functions to Asterix.

The binary data type is implemented as a bytearray. Its storage format

follows the String type which has 2 bytes for length and then store the

bytes contents.

Binary data will take hex("") or base64("") as the constructor method to

passing a hex string or base64 string into Asterix. For output we use

hex("") format.

The parse-[hex|base64](string) function will parse the corresponding hex

or base64 string to binary type. The print-[hex|base64](binary)

functions will print the binary to hex or base64 STRING format.

The sub-binary(binary, offset, [length]) function works the same as

substring(string, offset, [length])

The find-binary(srcbinary, targetbinary, [start-offset]) will find the

position of the targetbinary in the srcbinary.

Change-Id: I5ecf0cc115c44070fb5c1fc5b0ec12a95d4243a4

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/175

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

    • -1
    • +2
    ./factory/StreamBasedAdapterFactory.java
  1. … 127 more files in changeset.
Fixed CSV parser to recognize quote and delimiter inside a string

Change-Id: Iaaabc23e86df4f9bbee9f06b7976d7fbdcbb0f3f

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/135

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <westmann@gmail.com>

    • -11
    • +19
    ./factory/HDFSIndexingAdapterFactory.java
    • -11
    • +44
    ./factory/StreamBasedAdapterFactory.java
  1. … 74 more files in changeset.
Implement TokenizeOperator in addition to the changes made by Zachary Heilbron. Now, the compiler builds the bulk-load plan. For other details, please refer to the corresponding Hyracks changes.

Change-Id: I646539d88ea2bdc6da4fbf2b6e9460a6189125ff

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/79

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Young-Seok Kim <kisskys@gmail.com>

    • -8
    • +8
    ./factory/HDFSIndexingAdapterFactory.java
    • -6
    • +43
    ./factory/NCFileSystemAdapterFactory.java
    • -10
    • +20
    ./factory/StreamBasedAdapterFactory.java
  1. … 112 more files in changeset.
Add external indexes

This change include the following:

1. additional data parser for external data that parse hdfs records using Hive serdes.

2. allow users to create external data. this includes:

a) changes in metadata external dataset details.

b) addition of a new metadata index to store external file's statuses.

c) the pipeline for building the B-Tree and R-Tree indexes.

d) hyracks operators to fetch records with their RIDs using different formats.

e) hyracks operators to lookup and parse external records.

f) test cases for indexing and index access of different hdfs file formats.

g) exposing the secondary indexes over external data to the compiler.

3. adding a new aql command to refresh external datasets. this includes

a) global recovery on system startup.

b) changes in the aql parser.

c) construction of bulk modify pipelines and additional operators to perform local commit and abort operations (using 2PC protocol).

4. Added copyright header to all new files

5. Added additional test cases to test left outer join on external data

Change-Id: I1065a473299f6027eb073aeeba3a56d137f6f98e

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/70

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@uci.edu>

    • -8
    • +141
    ./factory/HDFSAdapterFactory.java
    • -0
    • +190
    ./factory/HDFSIndexingAdapterFactory.java
    • -0
    • +8
    ./factory/NCFileSystemAdapterFactory.java
    • -0
    • +3
    ./factory/StreamBasedAdapterFactory.java
  1. … 146 more files in changeset.
clean up azure adapters

    • -2
    • +47
    ./factory/PullBasedAzureTwitterAdapterFactory.java
  1. … 2 more files in changeset.
minor changes to pull based twitter adapter

    • -12
    • +15
    ./factory/PullBasedTwitterAdapterFactory.java
  1. … 6 more files in changeset.
checkpoint

    • -77
    • +10
    ./factory/HiveAdapterFactory.java
  1. … 24 more files in changeset.
Merge remote-tracking branch 'origin/raman/master_feeds_merge' into zheilbron/asterix_msr_demo

Conflicts:

.gitignore

asterix-algebra/src/main/java/edu/uci/ics/asterix/optimizer/rules/FeedScanCollectionToUnnest.java

asterix-algebra/src/main/java/edu/uci/ics/asterix/optimizer/rules/IntroduceMaterializationForInsertWithSelfScanRule.java

asterix-algebra/src/main/java/edu/uci/ics/asterix/optimizer/rules/UnnestToDataScanRule.java

asterix-app/src/main/java/edu/uci/ics/asterix/api/common/AsterixAppRuntimeContext.java

asterix-app/src/main/java/edu/uci/ics/asterix/api/common/AsterixAppRuntimeContextProdiverForRecovery.java

asterix-app/src/main/java/edu/uci/ics/asterix/aql/translator/AqlTranslator.java

asterix-app/src/test/resources/metadata/results/basic/issue_251_dataset_hint_2/issue_251_dataset_hint_2.1.adm

asterix-app/src/test/resources/metadata/results/basic/issue_251_dataset_hint_3/issue_251_dataset_hint_3.1.adm

asterix-app/src/test/resources/metadata/results/basic/issue_251_dataset_hint_4/issue_251_dataset_hint_4.1.adm

asterix-app/src/test/resources/metadata/results/basic/meta02/meta02.1.adm

asterix-app/src/test/resources/metadata/results/basic/meta09/meta09.1.adm

asterix-app/src/test/resources/metadata/results/basic/meta16/meta16.1.adm

asterix-app/src/test/resources/metadata/results/basic/meta17/meta17.1.adm

asterix-app/src/test/resources/metadata/results/basic/metadata_dataset/metadata_dataset.1.adm

asterix-app/src/test/resources/metadata/results/basic/metadata_datatype/metadata_datatype.1.adm

asterix-app/src/test/resources/runtimets/results/cross-dataverse/cross-dv02/cross-dv02.1.adm

asterix-app/src/test/resources/runtimets/results/cross-dataverse/cross-dv04/cross-dv04.1.adm

asterix-app/src/test/resources/runtimets/results/cross-dataverse/cross-dv19/cross-dv19.1.adm

asterix-app/src/test/resources/runtimets/results/user-defined-functions/udf23/udf23.1.adm

asterix-aql/src/main/java/edu/uci/ics/asterix/aql/expression/FeedDetailsDecl.java

asterix-aql/src/main/java/edu/uci/ics/asterix/aql/expression/InternalDetailsDecl.java

asterix-aql/src/main/java/edu/uci/ics/asterix/aql/expression/visitor/AQLPrintVisitor.java

asterix-aql/src/main/javacc/AQL.jj

asterix-common/src/main/java/edu/uci/ics/asterix/common/transactions/IAsterixAppRuntimeContextProvider.java

asterix-external-data/src/main/java/edu/uci/ics/asterix/external/dataset/adapter/FileSystemBasedAdapter.java

asterix-installer/src/main/java/edu/uci/ics/asterix/installer/driver/InstallerUtil.java

asterix-metadata/src/main/java/edu/uci/ics/asterix/metadata/MetadataNode.java

asterix-metadata/src/main/java/edu/uci/ics/asterix/metadata/bootstrap/MetadataBootstrap.java

asterix-metadata/src/main/java/edu/uci/ics/asterix/metadata/bootstrap/MetadataRecordTypes.java

asterix-metadata/src/main/java/edu/uci/ics/asterix/metadata/declared/DatasetDataSource.java

asterix-metadata/src/main/java/edu/uci/ics/asterix/metadata/entities/InternalDatasetDetails.java

asterix-metadata/src/main/java/edu/uci/ics/asterix/metadata/entitytupletranslators/DatasetTupleTranslator.java

asterix-metadata/src/main/java/edu/uci/ics/asterix/metadata/feeds/FeedMetaOperatorDescriptor.java

asterix-metadata/src/main/java/edu/uci/ics/asterix/metadata/feeds/FeedUtil.java

asterix-om/src/main/java/edu/uci/ics/asterix/om/functions/AsterixExternalFunctionInfo.java

asterix-om/src/main/java/edu/uci/ics/asterix/om/util/AsterixAppContextInfo.java

asterix-transactions/src/main/java/edu/uci/ics/asterix/transaction/management/opcallbacks/SecondaryIndexModificationOperationCallbackFactory.java

asterix-transactions/src/main/java/edu/uci/ics/asterix/transaction/management/resource/LSMBTreeLocalResourceMetadata.java

asterix-transactions/src/main/java/edu/uci/ics/asterix/transaction/management/service/logging/LogManager.java

asterix-transactions/src/main/java/edu/uci/ics/asterix/transaction/management/service/transaction/AsterixRuntimeComponentsProvider.java

    • -1
    • +1
    ./factory/PullBasedAzureTwitterAdapterFactory.java
  1. … 53 more files in changeset.
miscellaneous/minor changes + support for installing adapters

    • -2
    • +2
    ./factory/NCFileSystemAdapterFactory.java
    • -3
    • +14
    ./factory/StreamBasedAdapterFactory.java
  1. … 98 more files in changeset.
re-orchestrated feeds fault tolerance integration test

    • -2
    • +15
    ./factory/StreamBasedAdapterFactory.java
  1. … 10 more files in changeset.
1) modified tweet schema to have tweet ids as int64. 2) Added GUID (Long) generator for generating cluster wide unique Ids

    • -1
    • +1
    ./factory/NCFileSystemAdapterFactory.java
    • -1
    • +1
    ./factory/PullBasedTwitterAdapterFactory.java
  1. … 11 more files in changeset.
Revert "Merge external indexes into master"

This reverts commit 1a0d2b4b80677ddadb524503ad9c84e2f03aade7, reversing

changes made to 5eba6a0795f2278f4ea7747f8b54c858529b5502.

    • -195
    • +72
    ./factory/HDFSAdapterFactory.java
    • -187
    • +106
    ./factory/HiveAdapterFactory.java
    • -27
    • +0
    ./factory/IGenericDatasetAdapterFactory.java
    • -30
    • +14
    ./factory/NCFileSystemAdapterFactory.java
  1. … 53 more files in changeset.
allow for absolute constraints or *all NCs* rather than count constraints

    • -5
    • +8
    ./factory/PullBasedAzureTwitterAdapterFactory.java
enable feeds to ingest partitions of the data stored in azure

    • -1
    • +4
    ./factory/PullBasedAzureTwitterAdapterFactory.java
  1. … 2 more files in changeset.
rename variables for clarity

    • -3
    • +3
    ./factory/PullBasedAzureTwitterAdapterFactory.java
Correct behaviour of drop dataverse and drop dataset to drop external data indexes

  1. … 5 more files in changeset.
add feed ingestion for posting metadata

    • -21
    • +1
    ./factory/PullBasedAzureTwitterAdapterFactory.java
  1. … 3 more files in changeset.
add R-Tree support for external indexes and indexes optimizations

    • -2
    • +3
    ./factory/IGenericDatasetAdapterFactory.java
    • -2
    • +3
    ./factory/NCFileSystemAdapterFactory.java
  1. … 24 more files in changeset.
syncing azure adapter

    • -0
    • +108
    ./factory/PullBasedAzureTwitterAdapterFactory.java
  1. … 6 more files in changeset.
Introduced changes needed for external data indexing

    • -72
    • +187
    ./factory/HDFSAdapterFactory.java
    • -106
    • +187
    ./factory/HiveAdapterFactory.java
    • -0
    • +26
    ./factory/IGenericDatasetAdapterFactory.java
    • -14
    • +29
    ./factory/NCFileSystemAdapterFactory.java
  1. … 41 more files in changeset.