meta19.1.adm

Removed broken Metadata Secondary Indexes

Added DatatypeDataverse field to Dataset

Cleaned up Metadata Tests

Added tests for dropping used Datatypes

User-facing changes are:

1) Allow qualified datatype when creating dataset

2) Metadata queries for datasets will now

include the type dataverse

Change-Id: I24dbc04dcb2a4126fc8361ebe3104877a0d1f2bb

Reviewed-on: https://asterix-gerrit.ics.uci.edu/558

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

Introduce "wrapper-array" parameter to HTTP API which selects (for ADM and

JSON) whether to wrap the result sequence in a generated outer array. For

JSON this defaults to "true" as before. For ADM this defaults to false,

resulting in a large number of expected-results changes.

Also introduce ability to have AQL tests which provide HTTP parameters.

Change-Id: I3122f136ff9ca8a2c2268238c57bb5eddab7b27e

Reviewed-on: https://asterix-gerrit.ics.uci.edu/473

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Chris Hillery <ceej@lambda.nu>

Change-Id: I223aded8aaf80f0688358899c0e8b0d6988fac93

Reviewed-on: https://asterix-gerrit.ics.uci.edu/323

Reviewed-by: Till Westmann <tillw@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

commit c3f577861fc705d848c1641605689cadd6973bae

Merge: ebc4cae fc0c2c0

Author: ramangrover29 <ramangrover29@gmail.com>

Date: Fri Jun 26 13:04:05 2015 -0700

Merge branch 'raman/feeds_2_release' of https://code.google.com/p/asterixdb-sandbox into raman/feeds_2_release

Conflicts:

asterix-app/src/main/java/edu/uci/ics/asterix/api/http/servlet/FeedServlet.java

asterix-external-data/src/main/java/edu/uci/ics/asterix/external/library/java/JObjectAccessors.java

commit ebc4cae21a7302869f953df1ebda601e798d12d2

Author: ramangrover29 <ramangrover29@gmail.com>

Date: Sat Jun 20 17:14:45 2015 -0700

Introduces Feeds 2.0

Some of the prominent chnages introduced are as follows

a) Support for building a cascade network of feeds (via secondary feeds feature)

b) Feed Management Console for tracking active feeds and associated metrics

c) Support for elastic runtime for data ingestion

d) Improved fault-tolerance with support for logging of failed records

Documentation has been added at asterix-doc/src/site/markdown/feeds/

commit fc0c2c0549a6ee8b202e57607d2e110478cd57bb

Author: ramangrover29 <ramangrover29@gmail.com>

Date: Sat Jun 20 17:14:45 2015 -0700

Introduces Feeds 2.0

Some of the prominent chnages introduced are as follows

a) Support for building a cascade network of feeds (via secondary feeds feature)

b) Feed Management Console for tracking active feeds and associated metrics

c) Support for elastic runtime for data ingestion

d) Improved fault-tolerance with support for logging of failed records

Documentation has been added at asterix-doc/src/site/markdown/feeds/

Change-Id: I498f01c591a229aaf51cec43ab20f3e5c4f072f4

Reviewed-on: https://asterix-gerrit.ics.uci.edu/297

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Steven Jacobs <sjaco002@ucr.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

Open indexes requires user to provide a type along with a indexed field name.

This type would be enforced for all the indexed records, i.e. index cannot be created if in some records a field with provided name has a different type.

Index-specific rewrite rules match provided type with the inferred types of other arguments in join\select statements and trigger index rewrite.

Nested indexes use the same semantics as the regular indexes, with exception that field could be located arbitrarily deep inside nested structure

Change-Id: I53d00aba243ccf7cf79cf7d775dd305813d24f98

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/97

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Steven Jacobs <sjaco002@ucr.edu>

Elimiate the JSON "results" wrapper from API calls. Rename

DisplayFormat -> OutputFormat, TEXT -> ADM. Cleaned up some output and

added some useful comments. Fixed a few latent bugs in testdriver string

comparison routines. Refactored HTTP error handling in TestsUtils. Had

to update many expected test results.

Change-Id: I2d7ead038512455b2ab7844021cb62222400447b

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/137

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@uci.edu>

Reviewed-by: Till Westmann <westmann@gmail.com>

This change include the following:

1. additional data parser for external data that parse hdfs records using Hive serdes.

2. allow users to create external data. this includes:

a) changes in metadata external dataset details.

b) addition of a new metadata index to store external file's statuses.

c) the pipeline for building the B-Tree and R-Tree indexes.

d) hyracks operators to fetch records with their RIDs using different formats.

e) hyracks operators to lookup and parse external records.

f) test cases for indexing and index access of different hdfs file formats.

g) exposing the secondary indexes over external data to the compiler.

3. adding a new aql command to refresh external datasets. this includes

a) global recovery on system startup.

b) changes in the aql parser.

c) construction of bulk modify pipelines and additional operators to perform local commit and abort operations (using 2PC protocol).

4. Added copyright header to all new files

5. Added additional test cases to test left outer join on external data

Change-Id: I1065a473299f6027eb073aeeba3a56d137f6f98e

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/70

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@uci.edu>

This reverts commit 3629af665d3d3d3d285c5a00495a6a8288b7e436.

git-svn-id: https://asterixdb.googlecode.com/svn/trunk/asterix@1622 eaa15691-b419-025a-1212-ee371bd00084

git-svn-id: https://asterixdb.googlecode.com/svn/branches/asterix_stabilization@1423 eaa15691-b419-025a-1212-ee371bd00084