Clone Tools
  • last updated a few minutes ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
Move merged files

  1. … 154 more files in changeset.
Support Parsing UNIX Time for ClassAd data

Change-Id: Iba0a687475edd30078b1e5cda1810244eed76219

Reviewed-on: https://asterix-gerrit.ics.uci.edu/783

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

  1. … 30 more files in changeset.
ASTERIXDB-1378 Fix NPE on Feed Connect After Restart

This issue was caused by the way type traits are computed for each

index. since we have duplicate code to create the index dataflow

helper, we had to fix the way type traits are created in all of

these different places. We propably need to do further refactoring

to have common code for creating index dataflow helper instances.

Change-Id: If8f8696d252868a8cce0afdbaeda0dd046f99186

Reviewed-on: https://asterix-gerrit.ics.uci.edu/766

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

  1. … 7 more files in changeset.
ASTERIXDB-1360 Fix Error Message for Unknown Format

This change includes some refactoring for external

data. This refactoring makes the code less error prone

and separate data source selection from data parser

selection. It also fixes issue ASTERIXDB-1366 and adds

a test case for it as well.

Change-Id: Ib4aac833e30bd7c5a7706f5c8116383c2362c964

Reviewed-on: https://asterix-gerrit.ics.uci.edu/767

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 33 more files in changeset.
Improve Error Handling in Local Directory Feeds

This change improves handling of two error types for filesystem

based feeds. The first one is the handling of IO Errors which

causes the input stream to be closed, and the second one is

reacting to missed filesystem events. In both cases, we scan the

directory and compare it with the history we have in order to

resume from where we last left off.

In addition, this change includes some refactoring in external

data. Particularly, we get rid of the stream provider layer and

instead, stream factories create input streams directly. This

is consistent with record reader factories which create readers

directly without reader providers.

Change-Id: I08d89229e33c91532b1038ba9f7a372f7ca1fdb5

Reviewed-on: https://asterix-gerrit.ics.uci.edu/720

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

  1. … 144 more files in changeset.
Fix Asterix Installer Test Cases

Many tests started to fail in installer after being

disabled for a long time. This change fixes the following:

1. asterix-installer: external library tests.

2. asterix-installer: recovery: query-after-restart.

Change-Id: Iefad5ac3ff754693025312f1e2fa01b08832514c

Reviewed-on: https://asterix-gerrit.ics.uci.edu/737

Reviewed-by: Ian Maxon <imaxon@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 27 more files in changeset.
Merge branch 'release-0.8.8' into master

Change-Id: I8fcf41c009f9faf51bc8bccc0c3c7217b7769149

  1. … 36 more files in changeset.
Support Change Feeds and Ingestion of Records with MetaData

This change allows feeds to perform upserts and deletes

in order to perform replication of an external data source.

The change does so by performing the following:

1. The adapter produces [PK][Record]. (Record == null --> delete)

2. The insert is replaced by an upsert operator.

Change-Id: If136a03d424970132dfb09f0dda56e160d4c0078

Reviewed-on: https://asterix-gerrit.ics.uci.edu/621

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 265 more files in changeset.
use localfs instead of the class name of the adapter

Change-Id: If0c8fafbbc52bbe0f7ac3ec8b5f23781a8017390

Reviewed-on: https://asterix-gerrit.ics.uci.edu/706

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

  1. … 1136 more files in changeset.
Enabled Feed Tests and Added External Library tests

Feed tests had been switched off for a while due to having too many

sporadic failures. Now, we are switching them back on.

In addition, a new set of tests have been added to test that external

library works as expected.

Change-Id: Idd1fccd136fa2645b2707bbf7c04e60991ae8d4a

Reviewed-on: https://asterix-gerrit.ics.uci.edu/625

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

  1. … 202 more files in changeset.
Asterix NCs Failback Support

- Allow Failed NCs to failback and takeover their partitions.

- New cluster API servlet for cluster state description.

- Remove nodeId from txn logs except remote FLUSH_LOG.

- Add partition id in UPDATE and Entity_COMMIT logs.

- Adapt remote recovery to new logs format.

- Refactor RecoveryManager and split Analysis and Redo phase.

- Spill remote recover logs to temporary file.

- Replicate files to remote replicas for partitions of interest only.

- Introduce NC active/inactive partitions concept.

- Test case for failback.

Change-Id: Id17819542d6b9c4e32647e64737c4a467b630f24

Reviewed-on: https://asterix-gerrit.ics.uci.edu/613

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

  1. … 78 more files in changeset.
Asterix NCs Fault Tolerance

This change includes the following:

- Adapt replication to unique partitions storage.

- Implement auto failover for failing NCs.

- Implement auto failover for metadata node.

- Fix for ASTERIXDB-1251 using proper error message.

- Basic replication test cases using vagrant virtual cluster for:

1. LSM bulkload components replication.

2. LSM Memory components replication and recovery.

3. Metadata node takeover.

These test cases will be part of the cluster test profile.

Change-Id: Ice26d980912a315fcb3efdd571d6ce88717cfea4

Reviewed-on: https://asterix-gerrit.ics.uci.edu/573

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

    • -0
    • +63
    ./clusterts/cluster_with_replication.xml
  1. … 57 more files in changeset.
Improve the Testing Framework

This change includes the following improvements to the test framework

1. Checking each expected failure against the failure exception.

2. Checking against multiple causes of failure in case of multiple

expected errors.

3. Check if a test that is expected to fail pass.

4. Make temp datasets use different storage path.

Change-Id: I957ecf19bf7209981e010e0e50fb882442a525dd

Reviewed-on: https://asterix-gerrit.ics.uci.edu/537

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

  1. … 28 more files in changeset.
Run ExecutionTest w/integration test framework

I changed the default local.xml to use 2 NCs with

2 partitions each like AsterixHyracksIntegrationUtil

so that the results will match despite using simple

string compare to determine correctness.

There is also one test in particular (big-object-join)

that I had to add an artificial order by clause to

to deal with different results.

Issues outstanding:

- HDFS in ClusterExecutionIT needs to be thought out

Change-Id: I423f2a7c77839b999d466dd5cace302574d956c0

Reviewed-on: https://asterix-gerrit.ics.uci.edu/525

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Chris Hillery <ceej@lambda.nu>

    • -0
    • +255
    ./integrationts/asterix-configuration.xml
  1. … 3852 more files in changeset.
ASTERIXDB-1126 Correct comparators and type traits for external index

In case of external data, the type definition does not contain information

about primary keys and we should get them based on the data input format.

Change-Id: I71d924d7e2b7a7e6c752bc97679e612946afc17c

Reviewed-on: https://asterix-gerrit.ics.uci.edu/504

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@apache.org>

  1. … 6 more files in changeset.
ASTERIXDB-1148: Selectable array-wrapping of results

Introduce "wrapper-array" parameter to HTTP API which selects (for ADM and

JSON) whether to wrap the result sequence in a generated outer array. For

JSON this defaults to "true" as before. For ADM this defaults to false,

resulting in a large number of expected-results changes.

Also introduce ability to have AQL tests which provide HTTP parameters.

Change-Id: I3122f136ff9ca8a2c2268238c57bb5eddab7b27e

Reviewed-on: https://asterix-gerrit.ics.uci.edu/473

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Chris Hillery <ceej@lambda.nu>

  1. … 1426 more files in changeset.
Tweak more licenses

Change-Id: Iadb5074d0e0d4fb2b7d4103aa405e9fc91f44bc6

Reviewed-on: https://asterix-gerrit.ics.uci.edu/430

Reviewed-by: Till Westmann <tillw@apache.org>

Tested-by: Ian Maxon <imaxon@apache.org>

  1. … 13 more files in changeset.
Fix .jj and Markdown Licenses

Change-Id: Iadb5074d0e0d4fb2b7d4303aa405e9fc91f4bd21

Reviewed-on: https://asterix-gerrit.ics.uci.edu/426

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

    • -0
    • +18
    ./integrationts/library/testsuite.xml
  1. … 54 more files in changeset.
Fix shell script and javascript license issues

Change-Id: Iadb5074d0e0d4fb2b7d4303aa405e9fc91f4bd56

Reviewed-on: https://asterix-gerrit.ics.uci.edu/425

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 102 more files in changeset.
ASTERIXDB-1113, Add ASF license header to AQL files

Change-Id: I462cb156b5cd8507ac0625d4a40863ec9383ce82

Reviewed-on: https://asterix-gerrit.ics.uci.edu/400

Reviewed-by: Ian Maxon <imaxon@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 5758 more files in changeset.
Change License Headers

Also tweak the NOTICE file with some extras.

Change-Id:I09bc388089e515d7f51fd39c31bfbbc9f00cf84f

Reviewed-on: https://asterix-gerrit.ics.uci.edu/388

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

    • -12
    • +16
    ./integrationts/lifecycle/testsuite.xml
  1. … 2017 more files in changeset.
Change Java package from edu.uci.ics to org.apache

Change-Id: I2f01d2b5614e9e9c94fda4bf1294a8eba6a26c54

Reviewed-on: https://asterix-gerrit.ics.uci.edu/309

Reviewed-by: Till Westmann <tillw@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -1
    • +1
    ./integrationts/lifecycle/testsuite.xml
  1. … 2590 more files in changeset.
Introduces Feeds 2.0

commit c3f577861fc705d848c1641605689cadd6973bae

Merge: ebc4cae fc0c2c0

Author: ramangrover29 <ramangrover29@gmail.com>

Date: Fri Jun 26 13:04:05 2015 -0700

Merge branch 'raman/feeds_2_release' of https://code.google.com/p/asterixdb-sandbox into raman/feeds_2_release

Conflicts:

asterix-app/src/main/java/edu/uci/ics/asterix/api/http/servlet/FeedServlet.java

asterix-external-data/src/main/java/edu/uci/ics/asterix/external/library/java/JObjectAccessors.java

commit ebc4cae21a7302869f953df1ebda601e798d12d2

Author: ramangrover29 <ramangrover29@gmail.com>

Date: Sat Jun 20 17:14:45 2015 -0700

Introduces Feeds 2.0

Some of the prominent chnages introduced are as follows

a) Support for building a cascade network of feeds (via secondary feeds feature)

b) Feed Management Console for tracking active feeds and associated metrics

c) Support for elastic runtime for data ingestion

d) Improved fault-tolerance with support for logging of failed records

Documentation has been added at asterix-doc/src/site/markdown/feeds/

commit fc0c2c0549a6ee8b202e57607d2e110478cd57bb

Author: ramangrover29 <ramangrover29@gmail.com>

Date: Sat Jun 20 17:14:45 2015 -0700

Introduces Feeds 2.0

Some of the prominent chnages introduced are as follows

a) Support for building a cascade network of feeds (via secondary feeds feature)

b) Feed Management Console for tracking active feeds and associated metrics

c) Support for elastic runtime for data ingestion

d) Improved fault-tolerance with support for logging of failed records

Documentation has been added at asterix-doc/src/site/markdown/feeds/

Change-Id: I498f01c591a229aaf51cec43ab20f3e5c4f072f4

Reviewed-on: https://asterix-gerrit.ics.uci.edu/297

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Steven Jacobs <sjaco002@ucr.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

  1. … 372 more files in changeset.
Temp dataset support: 1. DDLs for creating a temporary dataset 2. Garbage collection for temporary dataset 3. Reading, inserting, and deleting data from (to) a temporary dataset is locking-free (except metadata locks) and logging-free (except flush/merge/job-commit logs).

Change-Id: Id0960acd1b4dfbfa7685dba227634572ca50b126

Reviewed-on: https://asterix-gerrit.ics.uci.edu/241

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Young-Seok Kim <kisskys@gmail.com>

  1. … 238 more files in changeset.
- Fixed Type Casting issue - Reorganized duplicated internal class in the DelimitedDataParser and DelimitedDataParserFactory - Prevented a user from creating an inverted index on a dataset with a variable-length PK - INT64 is now the default type - Issue 852 fixed

Change-Id: I2d71e8a21da4f709c3259a3d3f678c640f9e1160

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/192

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <westmann@gmail.com>

  1. … 840 more files in changeset.
Issue 804: Spell "Adapter" consistently

Change-Id: I3141c1ddd2357ba8dd8ad94d6b98abc78151cd72

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/148

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@uci.edu>

  1. … 37 more files in changeset.
Initial support for true ADM output (content type application/x-adm).

Elimiate the JSON "results" wrapper from API calls. Rename

DisplayFormat -> OutputFormat, TEXT -> ADM. Cleaned up some output and

added some useful comments. Fixed a few latent bugs in testdriver string

comparison routines. Refactored HTTP error handling in TestsUtils. Had

to update many expected test results.

Change-Id: I2d7ead038512455b2ab7844021cb62222400447b

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/137

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@uci.edu>

Reviewed-by: Till Westmann <westmann@gmail.com>

  1. … 1076 more files in changeset.
support for JSON escapes in ADM and AQL parser fix https://code.google.com/p/asterixdb/issues/detail?id=752 fix https://code.google.com/p/asterixdb/issues/detail?id=753 introduce PrintTools.writeUTF8StringWithEscapes to replace Hyracks' WriteValueTools.writeUTF8String as this serialization is ADM-specific new method AQLParser.parse for better exception handling more robust extraction of line numbers from error messages

Change-Id: I2e9bea7658d00032a3ac7a1d107eba8f17423eeb

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/75

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Chris Hillery <ceej@lambda.nu>

  1. … 34 more files in changeset.
Basis for Cluster integration testing

This branch adds cluster testing via Vagrant.

Requires my branch of the vagrant-maven plugin to work,

which can be sourced here:

https://github.com/parshimers/vagrant-maven-plugin

It is enabled with -DclusterTest=true in mvn verify.

A virtualized cluster with 4 nodes is started, and

then Asterix is started via managix on this cluster,

and then stopped.

Change-Id: I7e3cdcd4162ada19ee1e15f532be7447b4f34367

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/31

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@uci.edu>

Reviewed-by: Zachary Heilbron <zheilbron@gmail.com>

Reviewed-by: Till Westmann <westmann@gmail.com>

Reviewed-by: Chris Hillery <ceej@lambda.nu>

    • -0
    • +52
    ./clusterts/Vagrantfile
    • -0
    • +45
    ./clusterts/cluster.xml
    • -0
    • +11
    ./clusterts/known_hosts
  1. … 3 more files in changeset.
Change feeds_ingest test case to "slow"

  1. … 1 more file in changeset.