Clone Tools
  • last updated 24 mins ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
Introducing Data Replication To AsterixDB

This change includes the following:

- Add data replication properties to cluster properties and Managix validate command.

- Introduce Data Replication components.

- Add data replication required fields to LogRecord.

- Specialized LogManager for data replication.

- Fix for invalid cluster state on nodes failure.

- ASTERIXDB-139: Fix for cleaning workspace files on startup/shutdown.

- Fix for temp datasets storage reclamation.

- Allow MetadataNode rebinding with CC.

- Add flag to checkpoint to identify sharp checkpoints.

- ASTERIXDB-1170: Fix shutdown sequence

Change-Id: I729fdd1144dbc9ff039b4bc414494860d7553810

Reviewed-on: https://asterix-gerrit.ics.uci.edu/338

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Young-Seok Kim <kisskys@gmail.com>

  1. … 73 more files in changeset.
SQL++ parser: 1. refactored asterix-aql to become asterix-lang-common and asterix-lang-aql, where the former is the common part for different languages; 2. added asterix-lang-sqlpp on top of asterix-lang-common; 3. ported parser tests, optimizer tests and runtime tests in asterix-app to their sql++ version, and added parser tests for all the queries.

Change-Id: Ie5af4e3b692ca017ec047a1ba3b404a51beb3a2e

Reviewed-on: https://asterix-gerrit.ics.uci.edu/466

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 10209 more files in changeset.
ASTERIXDB-1102: VarSize Encoding to store length of String and ByteArray

This patch is to change the encoding format that stores the length value

of

the variable length type (e.g. String, ByteArray) from fix-size encoding

(2bytes) to variable-size encoding ( 1 to 5bytes)

It will solve the issue 1102 to enable us to store a String that longer

than 64K. Also for the common case of storing the short string ( <=

127), it will save one byte per string.

Some important changes include:

1. The UTF8StringSerDer and ByteArraySerDer is not Singleton instance

any more. I need some state to speedup the serialization and avoid the

object creatation. Luckily, 99% percent of Serializer were used as

factory way. The other 1% has been fixed.

A separate Test support, the ExcutionTest now can produce the only.xml

which stores the previous failed runtime test.xml. It can speedup the

debug process.

Change-Id: I41fff780f5c071742ef10129d83c8f945d5886d7

Reviewed-on: https://asterix-gerrit.ics.uci.edu/450

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Jianfeng Jia <jianfeng.jia@gmail.com>

  1. … 321 more files in changeset.
Convert comparisons with current-datetime, current-date, and current-time to applicable index searches as though they were constants

Change-Id: Ied64276873afcfbdd31dac313009e47429d8f9b0

Reviewed-on: https://asterix-gerrit.ics.uci.edu/367

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

  1. … 8 more files in changeset.
ASTERIXDB-920: Fixed consecutive index-nested-loop joins failure

Index-nested-loop join now tries to use an index from the inner relation first.

Change-Id: Ie7bcc70c19d2c1e70ce274eaacfe2c3ce89c8de8

Reviewed-on: https://asterix-gerrit.ics.uci.edu/434

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

  1. … 28 more files in changeset.
Fix .jj and Markdown Licenses

Change-Id: Iadb5074d0e0d4fb2b7d4303aa405e9fc91f4bd21

Reviewed-on: https://asterix-gerrit.ics.uci.edu/426

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 55 more files in changeset.
Change License Headers

Also tweak the NOTICE file with some extras.

Change-Id:I09bc388089e515d7f51fd39c31bfbbc9f00cf84f

Reviewed-on: https://asterix-gerrit.ics.uci.edu/388

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 2004 more files in changeset.
Change Java package from edu.uci.ics to org.apache

Change-Id: I2f01d2b5614e9e9c94fda4bf1294a8eba6a26c54

Reviewed-on: https://asterix-gerrit.ics.uci.edu/309

Reviewed-by: Till Westmann <tillw@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 2590 more files in changeset.
Change folder structure for Java repackage

Change only the folders, not the files, for our package name change.

This will break the build, and needs to be followed by a change to

the package name in all of the source files. However performing

the folder move and file change in two steps lets Git understand

that the files are the same, and lets us track revisions across

those files.

Change-Id: Iefd2a576415ebc1416cba2a3334d2b64f042ba92

Reviewed-on: https://asterix-gerrit.ics.uci.edu/306

Tested-by: Ian Maxon <imaxon@apache.org>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 3781 more files in changeset.
Move ADM-specific printers and related classes into .adm packages.

Change-Id: I5c37d3b5aad7f286eba1cb8cb5f05ad456a6521b

Reviewed-on: https://asterix-gerrit.ics.uci.edu/360

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@apache.org>

  1. … 139 more files in changeset.
Changed metadata storage format for nullable field types. Moved field name generation to the client out of metadata node code. Changed naming scheme for autogenerated types. Moved GroupName, CompactionPolicy & CompactionPolicyProperties fields up from External\InternalDetails to Dataset record type definition

Change-Id: I223aded8aaf80f0688358899c0e8b0d6988fac93

Reviewed-on: https://asterix-gerrit.ics.uci.edu/323

Reviewed-by: Till Westmann <tillw@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 199 more files in changeset.
Abstracting algebricks rules out of AsterixDB.

Change-Id: Id184cc1cdfa738964705007ac42027d90b84cf58

Reviewed-on: https://asterix-gerrit.ics.uci.edu/353

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Preston Carman <prestonc@apache.org>

Add Apache RAT License Auditor plugin, and fix missing licenses

Change-Id: Ic0fa0b7836e8a69d0d8beed83f9c3b41c168afd4

Reviewed-on: https://asterix-gerrit.ics.uci.edu/263

Reviewed-by: Chris Hillery <ceej@lambda.nu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 1530 more files in changeset.
Introduces Feeds 2.0

commit c3f577861fc705d848c1641605689cadd6973bae

Merge: ebc4cae fc0c2c0

Author: ramangrover29 <ramangrover29@gmail.com>

Date: Fri Jun 26 13:04:05 2015 -0700

Merge branch 'raman/feeds_2_release' of https://code.google.com/p/asterixdb-sandbox into raman/feeds_2_release

Conflicts:

asterix-app/src/main/java/edu/uci/ics/asterix/api/http/servlet/FeedServlet.java

asterix-external-data/src/main/java/edu/uci/ics/asterix/external/library/java/JObjectAccessors.java

commit ebc4cae21a7302869f953df1ebda601e798d12d2

Author: ramangrover29 <ramangrover29@gmail.com>

Date: Sat Jun 20 17:14:45 2015 -0700

Introduces Feeds 2.0

Some of the prominent chnages introduced are as follows

a) Support for building a cascade network of feeds (via secondary feeds feature)

b) Feed Management Console for tracking active feeds and associated metrics

c) Support for elastic runtime for data ingestion

d) Improved fault-tolerance with support for logging of failed records

Documentation has been added at asterix-doc/src/site/markdown/feeds/

commit fc0c2c0549a6ee8b202e57607d2e110478cd57bb

Author: ramangrover29 <ramangrover29@gmail.com>

Date: Sat Jun 20 17:14:45 2015 -0700

Introduces Feeds 2.0

Some of the prominent chnages introduced are as follows

a) Support for building a cascade network of feeds (via secondary feeds feature)

b) Feed Management Console for tracking active feeds and associated metrics

c) Support for elastic runtime for data ingestion

d) Improved fault-tolerance with support for logging of failed records

Documentation has been added at asterix-doc/src/site/markdown/feeds/

Change-Id: I498f01c591a229aaf51cec43ab20f3e5c4f072f4

Reviewed-on: https://asterix-gerrit.ics.uci.edu/297

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Steven Jacobs <sjaco002@ucr.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

  1. … 373 more files in changeset.
VariableSizeFrame(VSizeFrame) support for Asterix (Runtime Only)

Apply the https://asterix-gerrit.ics.uci.edu/#/c/234/ API changes to

Asterix level.

Change-Id: I5459e877707a1494fc1bebf03d4457a7427e9e0f

Reviewed-on: https://asterix-gerrit.ics.uci.edu/259

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

  1. … 43 more files in changeset.
AsterixDB changes for fixing issue873.

For example, in the following query plan, the change lets the optimizer recognize that $12 and $20 are equivalent.

Therefore, HASH_PARTITION_EXCHANGE [$$12] can be replaced by ONE_TO_ONE_EXCHANGE.

-- COMMIT |PARTITIONED|

project ([$$12])

-- STREAM_PROJECT |PARTITIONED|

exchange

-- ONE_TO_ONE_EXCHANGE |PARTITIONED|

delete from TinySocial:TweetMessages from %0->$$4 partitioned by [%0->$$12]

-- INSERT_DELETE |PARTITIONED|

exchange

-- ONE_TO_ONE_EXCHANGE |PARTITIONED|

materialize

-- MATERIALIZE |PARTITIONED|

exchange

-- HASH_PARTITION_EXCHANGE [$$12] |PARTITIONED|

assign [$$12] <- [function-call: asterix:field-access-by-index, Args:[%0->$$4, AInt32: {0}]]

-- ASSIGN |PARTITIONED|

project ([$$4])

-- STREAM_PROJECT |PARTITIONED|

assign [$$4] <- [function-call: asterix:open-record-constructor, Args:[AString: {tweetid}, %0->$$14, AString: {user}, function-call: asterix:field-access-by-index, Args:[%0->$$0, AInt32: {1}], AString: {sender-location}, function-call: asterix:field-access-by-index, Args:[%0->$$0, AInt32: {2}], AString: {send-time}, function-call: asterix:field-access-by-index, Args:[%0->$$0, AInt32: {3}], AString: {referred-topics}, function-call: asterix:field-access-by-index, Args:[%0->$$0, AInt32: {4}], AString: {message-text}, function-call: asterix:field-access-by-index, Args:[%0->$$0, AInt32: {5}]]]

-- ASSIGN |PARTITIONED|

exchange

-- ONE_TO_ONE_EXCHANGE |PARTITIONED|

unnest-map [$$14, $$0] <- function-call: asterix:index-search, Args:[AString: {TweetMessages}, AInt32: {0}, AString: {TinySocial}, AString: {TweetMessages}, ABoolean: {false}, ABoolean: {false}, ABoolean: {false}, AInt32: {1}, %0->$$20, AInt32: {1}, %0->$$21, TRUE, TRUE, TRUE]

-- BTREE_SEARCH |PARTITIONED|

exchange

-- ONE_TO_ONE_EXCHANGE |PARTITIONED|

assign [$$20, $$21] <- [AString: {15}, AString: {15}]

-- ASSIGN |PARTITIONED|

empty-tuple-source

-- EMPTY_TUPLE_SOURCE |PARTITIONED|

Change-Id: Ife8c378a62cdbbcd8c19b521de246162f1f3d6ec

Reviewed-on: https://asterix-gerrit.ics.uci.edu/267

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Wenhai Li <lwhaymail@yahoo.com>

Reviewed-by: Ildar Absalyamov <ildar.absalyamov@gmail.com>

  1. … 10 more files in changeset.
Temp dataset support: 1. DDLs for creating a temporary dataset 2. Garbage collection for temporary dataset 3. Reading, inserting, and deleting data from (to) a temporary dataset is locking-free (except metadata locks) and logging-free (except flush/merge/job-commit logs).

Change-Id: Id0960acd1b4dfbfa7685dba227634572ca50b126

Reviewed-on: https://asterix-gerrit.ics.uci.edu/241

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Young-Seok Kim <kisskys@gmail.com>

  1. … 248 more files in changeset.
fix multi-delete statement issue

Change-Id: I32c55e225fb8cc3f164da7029b4c31c26c6e574d

Reviewed-on: https://asterix-gerrit.ics.uci.edu/271

Reviewed-by: Ian Maxon <imaxon@uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 5 more files in changeset.
Make index selection deterministic when multiple avaialbe indexes are available.

The priority of indexes is as follows:

1. First choice primary B+ tree

2. Second choice some other B+ tree

3. Third choice some R tree

4. Finally, choose based on index/dataset/dataverse name strings.

Change-Id: I5a24d9ab407568c3241cdaf9255643e912874384

Reviewed-on: https://asterix-gerrit.ics.uci.edu/264

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@uci.edu>

  1. … 6 more files in changeset.
Range connector update with order by hint.

Change-Id: I1b36f6dea801950c39bf63f27a0ec70d3584c5b7

Reviewed-on: https://asterix-gerrit.ics.uci.edu/254

Reviewed-by: Ildar Absalyamov <ildar.absalyamov@gmail.com>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Steven Jacobs <sjaco002@ucr.edu>

  1. … 35 more files in changeset.
Temporal update based on a merge from an old branch.

Includes the following new functions: overlap-bins, get-overlapping-interval, get-interval-start-date, get-interval-start-datetime, get-interval-start-time, get-interval-end-date, get-interval-end-datetime, get-interval-end-time

Change-Id: Ie15ed39ae7de83ce71c63c4e7490f2ebf5911540

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/244

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ildar Absalyamov <ildar.absalyamov@gmail.com>

  1. … 107 more files in changeset.
Implementation of Union Change-Id: I9049c89f7e3cd84cab1b4080af049f6042be9fb1 Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/236 Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu> Reviewed-by: Ildar Absalyamov <ildar.absalyamov@gmail.com> Reviewed-by: Preston Carman <ecarm002@ucr.edu>

  1. … 7 more files in changeset.
Added support of typed indexes over open fields & indexes over nested fields

Open indexes requires user to provide a type along with a indexed field name.

This type would be enforced for all the indexed records, i.e. index cannot be created if in some records a field with provided name has a different type.

Index-specific rewrite rules match provided type with the inferred types of other arguments in join\select statements and trigger index rewrite.

Nested indexes use the same semantics as the regular indexes, with exception that field could be located arbitrarily deep inside nested structure

Change-Id: I53d00aba243ccf7cf79cf7d775dd305813d24f98

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/97

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Steven Jacobs <sjaco002@ucr.edu>

  1. … 1777 more files in changeset.
This change covers three points: 1. move the generation of UUIDs out of parsers 2. fixes a bug in the metadata lock when performing load operations 3. the external data scan operator is again a leaf operator 4. made sure the startup call doesn't return until the servers are ready to receive requests Merged master into branch for the second time.

The following commits from your working branch will be included:

commit da929c990249fe7f0699417fd7f7ea34908355af

Author: Abdullah Alamoudi <bamousaa@gmail.com>

Date: Fri Jan 16 11:51:14 2015 +0300

Moved generation of UUIDs out of parsers

Change-Id: I32437493cf48c79417f4280e9d8a85cfb1559b62

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/205

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@uci.edu>

Reviewed-by: Taewoo Kim <wangsaeu@gmail.com>

  1. … 57 more files in changeset.
- Fixed Type Casting issue - Reorganized duplicated internal class in the DelimitedDataParser and DelimitedDataParserFactory - Prevented a user from creating an inverted index on a dataset with a variable-length PK - INT64 is now the default type - Issue 852 fixed

Change-Id: I2d71e8a21da4f709c3259a3d3f678c640f9e1160

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/192

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <westmann@gmail.com>

  1. … 833 more files in changeset.
Fix issue849.

Change-Id: I4d2933c94c4139a6c8e2cf15e3cd1f6a52335f6b

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/217

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Young-Seok Kim <kisskys@gmail.com>

Reviewed-by: Steven Jacobs <sjaco002@ucr.edu>

  1. … 14 more files in changeset.
Fix for issues 838, 841, 527

These fixes help to recognize and rewrite plans that should use index lookups rather than nested loops or outer joins

Change-Id: Icad72c10e7c6afbc46cfb014c6198429d4e338e2

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/215

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

  1. … 2 more files in changeset.
Tracks the variables coming from the outer branch of loj to later decide whether they are still nullable

Change-Id: I65ef3ec64e231d4ee4b9cb18ac3292f126ee16f8

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/209

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

  1. … 1 more file in changeset.
Fix issue 562.

Change-Id: If5829d66910294ef5c764c6cba8743effda68b28

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/203

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Pouria Pirzadeh <pouria.pirzadeh@gmail.com>

  1. … 9 more files in changeset.
Issue 548: Doc for CSV

Change-Id: Ia343a7a795876835607be024bb7b8ad3fec33933

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/194

Reviewed-by: Till Westmann <westmann@gmail.com>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 5 more files in changeset.