Clone Tools
  • last updated 28 mins ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
Support Parsing UNIX Time for ClassAd data

Change-Id: Iba0a687475edd30078b1e5cda1810244eed76219

Reviewed-on: https://asterix-gerrit.ics.uci.edu/783

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

    • -99
    • +99
    ./feed-with-external-parser/feed-with-external-parser.1.adm
  1. … 30 more files in changeset.
Improve Error Handling in Local Directory Feeds

This change improves handling of two error types for filesystem

based feeds. The first one is the handling of IO Errors which

causes the input stream to be closed, and the second one is

reacting to missed filesystem events. In both cases, we scan the

directory and compare it with the history we have in order to

resume from where we last left off.

In addition, this change includes some refactoring in external

data. Particularly, we get rid of the stream provider layer and

instead, stream factories create input streams directly. This

is consistent with record reader factories which create readers

directly without reader providers.

Change-Id: I08d89229e33c91532b1038ba9f7a372f7ca1fdb5

Reviewed-on: https://asterix-gerrit.ics.uci.edu/720

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

    • -99
    • +99
    ./feed-with-external-parser/feed-with-external-parser.1.adm
  1. … 144 more files in changeset.
Fix Asterix Installer Test Cases

Many tests started to fail in installer after being

disabled for a long time. This change fixes the following:

1. asterix-installer: external library tests.

2. asterix-installer: recovery: query-after-restart.

Change-Id: Iefad5ac3ff754693025312f1e2fa01b08832514c

Reviewed-on: https://asterix-gerrit.ics.uci.edu/737

Reviewed-by: Ian Maxon <imaxon@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -0
    • +12
    ./feed-with-external-function/feed-with-external-function.1.adm
  1. … 34 more files in changeset.
Add test queries over ingested feeds with meta records.

Change-Id: I7f18282a4c8e79a2d2fcd596a79c8819a951b219

Reviewed-on: https://asterix-gerrit.ics.uci.edu/723

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

    • -0
    • +1
    ./change-feed-with-meta-pk-in-meta/change-feed-with-meta-pk-in-meta.4.adm
    • -0
    • +1
    ./change-feed-with-meta-pk-in-meta/change-feed-with-meta-pk-in-meta.5.adm
    • -0
    • +5
    ./change-feed-with-meta-pk-in-meta/change-feed-with-meta-pk-in-meta.6.adm
    • -50
    • +0
    ./feed-with-meta-pk-in-meta/feed-with-meta-pk-in-meta.1.adm
    • -0
    • +1
    ./feed-with-meta-pk-in-meta/feed-with-meta-pk-in-meta.4.adm
    • -0
    • +5
    ./feed-with-meta-pk-in-meta/feed-with-meta-pk-in-meta.5.adm
  1. … 25 more files in changeset.
Support Change Feeds and Ingestion of Records with MetaData

This change allows feeds to perform upserts and deletes

in order to perform replication of an external data source.

The change does so by performing the following:

1. The adapter produces [PK][Record]. (Record == null --> delete)

2. The insert is replaced by an upsert operator.

Change-Id: If136a03d424970132dfb09f0dda56e160d4c0078

Reviewed-on: https://asterix-gerrit.ics.uci.edu/621

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -0
    • +10
    ./change-feed/change-feed.1.adm
    • -99
    • +99
    ./feed-with-external-parser/feed-with-external-parser.1.adm
    • -0
    • +50
    ./feed-with-meta-pk-in-meta/feed-with-meta-pk-in-meta.1.adm
  1. … 268 more files in changeset.
ASTERIXDB-1333 Fix Feed Ingestion for Dataset with Filters

This change fixes a bug in the compilation of feed connection

when the dataset has filters. The bug is caused by not setting

the filter expressions during initial plan generation.

Change-Id: Ic323c9a74504921b984f68790bff2d3a7140b85b

Reviewed-on: https://asterix-gerrit.ics.uci.edu/716

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -0
    • +7
    ./feed-with-filtered-dataset/feed-with-filtered-dataset.1.adm
  1. … 6 more files in changeset.
Add DDL support for optionally associating meta record with dataset record.

-Fixed the TestExecutor.runScriptAndCompareWithResult() to compare fields

after "Time"-prefixed fields.

Change-Id: I613316985dbd7f553d946f95c5b1a1df56914902

Reviewed-on: https://asterix-gerrit.ics.uci.edu/666

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

    • -99
    • +99
    ./feed-with-external-parser/feed-with-external-parser.1.adm
  1. … 72 more files in changeset.
ASTERIXDB-1302 ASTERIXDB-1301 Fix Socket Feed Connection

A bug causes a read lock to never be released when a feed is

connected with "wait-for-completion" set to false. The bug

was fixed and a test case was added.

Another bug was causing the socket feed to not receive

connections correctly. The bug was fixed and a test case

was added.

Additionally, this change ensures that adapters have absolute

partitions to ensure consistency with regards to feed log

manager.

Change-Id: I8f6e982440d3577343f2479c3779653a9c3db614

Reviewed-on: https://asterix-gerrit.ics.uci.edu/660

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ildar Absalyamov <ildar.absalyamov@gmail.com>

    • -0
    • +10
    ./feed-push-socket/feed-push-socket.1.adm
  1. … 49 more files in changeset.
Enabled Feed Tests and Added External Library tests

Feed tests had been switched off for a while due to having too many

sporadic failures. Now, we are switching them back on.

In addition, a new set of tests have been added to test that external

library works as expected.

Change-Id: Idd1fccd136fa2645b2707bbf7c04e60991ae8d4a

Reviewed-on: https://asterix-gerrit.ics.uci.edu/625

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

    • -0
    • +99
    ./feed-with-external-parser/feed-with-external-parser.1.adm
  1. … 203 more files in changeset.
Feed Fixes and Cleanup

1. Introduce filesystem feed data source.

2. Fix the order of closing feed stages on disconnection.

3. Added Twitter feed to the compatibility utility to allow

using it with its alias.

4. First part of the feed log space.

5. Fixed the handling of duplicate key exception.

Change-Id: I4e8db26a810efd1fbaa52ceeb3efd0c8328ab070

Reviewed-on: https://asterix-gerrit.ics.uci.edu/574

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

  1. … 417 more files in changeset.
ASTERIXDB-1148: Selectable array-wrapping of results

Introduce "wrapper-array" parameter to HTTP API which selects (for ADM and

JSON) whether to wrap the result sequence in a generated outer array. For

JSON this defaults to "true" as before. For ADM this defaults to false,

resulting in a large number of expected-results changes.

Also introduce ability to have AQL tests which provide HTTP parameters.

Change-Id: I3122f136ff9ca8a2c2268238c57bb5eddab7b27e

Reviewed-on: https://asterix-gerrit.ics.uci.edu/473

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Chris Hillery <ceej@lambda.nu>

    • -13
    • +12
    ./issue_230_feeds/issue_230_feeds.1.adm
    • -12
    • +11
    ./issue_711_feeds/issue_711_feeds.1.adm
  1. … 1427 more files in changeset.
Introduces Feeds 2.0

commit c3f577861fc705d848c1641605689cadd6973bae

Merge: ebc4cae fc0c2c0

Author: ramangrover29 <ramangrover29@gmail.com>

Date: Fri Jun 26 13:04:05 2015 -0700

Merge branch 'raman/feeds_2_release' of https://code.google.com/p/asterixdb-sandbox into raman/feeds_2_release

Conflicts:

asterix-app/src/main/java/edu/uci/ics/asterix/api/http/servlet/FeedServlet.java

asterix-external-data/src/main/java/edu/uci/ics/asterix/external/library/java/JObjectAccessors.java

commit ebc4cae21a7302869f953df1ebda601e798d12d2

Author: ramangrover29 <ramangrover29@gmail.com>

Date: Sat Jun 20 17:14:45 2015 -0700

Introduces Feeds 2.0

Some of the prominent chnages introduced are as follows

a) Support for building a cascade network of feeds (via secondary feeds feature)

b) Feed Management Console for tracking active feeds and associated metrics

c) Support for elastic runtime for data ingestion

d) Improved fault-tolerance with support for logging of failed records

Documentation has been added at asterix-doc/src/site/markdown/feeds/

commit fc0c2c0549a6ee8b202e57607d2e110478cd57bb

Author: ramangrover29 <ramangrover29@gmail.com>

Date: Sat Jun 20 17:14:45 2015 -0700

Introduces Feeds 2.0

Some of the prominent chnages introduced are as follows

a) Support for building a cascade network of feeds (via secondary feeds feature)

b) Feed Management Console for tracking active feeds and associated metrics

c) Support for elastic runtime for data ingestion

d) Improved fault-tolerance with support for logging of failed records

Documentation has been added at asterix-doc/src/site/markdown/feeds/

Change-Id: I498f01c591a229aaf51cec43ab20f3e5c4f072f4

Reviewed-on: https://asterix-gerrit.ics.uci.edu/297

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Steven Jacobs <sjaco002@ucr.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

    • -0
    • +2
    ./feeds_09/feeds_09.1.adm
    • -0
    • +2
    ./feeds_10/feeds_10.1.adm
    • -0
    • +13
    ./feeds_11/feeds_11.1.adm
    • -0
    • +12
    ./feeds_12/feeds_12.1.adm
  1. … 378 more files in changeset.
Issue 804: Spell "Adapter" consistently

Change-Id: I3141c1ddd2357ba8dd8ad94d6b98abc78151cd72

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/148

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@uci.edu>

  1. … 38 more files in changeset.
Initial support for true ADM output (content type application/x-adm).

Elimiate the JSON "results" wrapper from API calls. Rename

DisplayFormat -> OutputFormat, TEXT -> ADM. Cleaned up some output and

added some useful comments. Fixed a few latent bugs in testdriver string

comparison routines. Refactored HTTP error handling in TestsUtils. Had

to update many expected test results.

Change-Id: I2d7ead038512455b2ab7844021cb62222400447b

Reviewed-on: http://fulliautomatix.ics.uci.edu:8443/137

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@uci.edu>

Reviewed-by: Till Westmann <westmann@gmail.com>

    • -12
    • +13
    ./issue_230_feeds/issue_230_feeds.1.adm
    • -11
    • +12
    ./issue_711_feeds/issue_711_feeds.1.adm
  1. … 1081 more files in changeset.
added test case for inverted index + minor change to the c'tor (putting indexName as the last arg)

    • -0
    • +1
    ./feeds_07/feeds_07.1.adm
    • -0
    • +1
    ./feeds_08/feeds_08.1.adm
  1. … 7 more files in changeset.
fix for issue 711

    • -0
    • +11
    ./issue_711_feeds/issue_711_feeds.1.adm
  1. … 12 more files in changeset.
added test case for generic socket based feed adaptor

    • -0
    • +100
    ./feeds_06/feeds_06.1.adm
  1. … 15 more files in changeset.
merged master, resolved conflicts

  1. … 25 more files in changeset.
checkpoint

  1. … 42 more files in changeset.
Allowed user to specify the merging policy

  1. … 58 more files in changeset.
merge from master

  1. … 125 more files in changeset.
checkpoint

    • -0
    • +1
    ./feeds_05/feeds_05.1.adm
  1. … 16 more files in changeset.
code cleanup for adaptors/factories

  1. … 32 more files in changeset.
Merged asterix_lsm_stabilization upto r1547

git-svn-id: https://asterixdb.googlecode.com/svn/trunk/asterix@1622 eaa15691-b419-025a-1212-ee371bd00084

    • -0
    • +12
    ./issue_230_feeds/issue_230_feeds.1.adm
  1. … 6216 more files in changeset.
New directory structure for tests to run using REST framework.

git-svn-id: https://asterixdb.googlecode.com/svn/branches/asterix_stabilization_result_distribution@1319 eaa15691-b419-025a-1212-ee371bd00084

    • -0
    • +1
    ./feeds_01/feeds_01.1.adm
    • -0
    • +12
    ./feeds_02/feeds_02.1.adm
    • -0
    • +1
    ./feeds_03/feeds_03.1.adm
    • -0
    • +11
    ./feeds_04/feeds_04.1.adm
    • -0
    • +12
    ./issue_230_feeds/issue_230_feeds.1.adm
  1. … 4543 more files in changeset.
merged asterix_stabilization r620:1109

git-svn-id: https://asterixdb.googlecode.com/svn/branches/asterix_stabilization_temporal_functionality@1113 eaa15691-b419-025a-1212-ee371bd00084

  1. … 1428 more files in changeset.
Support for hints in create dataset statement.

ASTERIX allows user to give additional information in form of hints.

These hints can come handy in scenarios such as determining other parameters like the size of bloom filter to hold data.

To begin with, the onyl hint supported by Asterix is the 'CARDINALITY' hint.

CARDINALITY gives the expected number of tuples in the dataset.

An example create dataset statement that provides hints is given below:-

create dataset X(TypeY)

partitioned by key id

hints (CARDINALITY=2500);

Please note that hints are case-insensitive.

Test Cases:-

Positive

asterix-app/src/test/resources/metadata/queries/basic/issue_251_dataset_hint_1.aql

asterix-app/src/test/resources/metadata/queries/basic/issue_251_dataset_hint_2.aql

asterix-app/src/test/resources/metadata/queries/basic/issue_251_dataset_hint_3.aql

asterix-app/src/test/resources/metadata/queries/basic/issue_251_dataset_hint_4.aql

Negative

asterix-app/src/test/resources/metadata/queries/exception/issue_251_dataset_hint_error_1.aql

asterix-app/src/test/resources/metadata/queries/exception/issue_251_dataset_hint_error_2.aql

git-svn-id: https://asterixdb.googlecode.com/svn/branches/asterix_stabilization_issue_251@1105 eaa15691-b419-025a-1212-ee371bd00084

  1. … 29 more files in changeset.
Reviewer:Yingyi 1) Fix issues 14, 230, 238, 239, 245 2) Simplication of ADM/Delimited parser factories. 3) Additional test cases that use simulated feeds. 4) Additional test cases for external datasets over HDFS. 5) Code refactoring in asterix-external-data module.

git-svn-id: https://asterixdb.googlecode.com/svn/branches/asterix_stabilization@1094 eaa15691-b419-025a-1212-ee371bd00084

    • -0
    • +12
    ./issue_230_feeds.adm
  1. … 121 more files in changeset.