Clone Tools
  • last updated 20 mins ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
[NO ISSUE][FUN] Fix type inference and casting in UDFs

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

The current UDF framework handles the argument types in a sloppy way. It

takes in the arguments and reads them as the expected data types in the

configuration. This could cause exception at runtime when passing in

arguments with unexpected datatypes. When setting the arguments, it did

a type casting for numeric values only to make sure the int64 from query

interface can be evaluated properly. However, this is not robust enough.

This patch fixes the type inference for UDFs during the complation time.

The ExternalTypeComputer is refactored to return defined data type, and

meanwhile checks the argument data types. Also, the

IntroduceDynamicTypeCastForExternalFunctionRule is modified to cover the

type castings for data types besides record type.

Change-Id: I40506fcca3cd8f14bbd6412359683433256c4c1f

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3270

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Xikui Wang <xkkwww@gmail.com>

    • -0
    • +31
    ./java/org/apache/asterix/external/library/MySumFactory.java
    • -0
    • +47
    ./java/org/apache/asterix/external/library/MySumFunction.java
  1. … 25 more files in changeset.
[NO ISSUE] Apply / enforce java import order

The process-sources target will now sort imports as well as

format source code; the source-format job will likewise verify

import order in addition to source code format

Change-Id: I55d976c4df10d9919c6a25683be2a3e3304e65d9

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3288

Integration-Tests: Michael Blow <mblow@apache.org>

Tested-by: Michael Blow <mblow@apache.org>

Reviewed-by: Till Westmann <tillw@apache.org>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 617 more files in changeset.
[NO ISSUE][FUN] Fix type inference in JListAccessor

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

As the list element type is available, there is no need to deserialize

element types from the data. Doing so, for example, will cause the

object type to be fully open type and closed fields to be recognized as

open fields. This will further cause NPE in UDFs. Modified the upperCase

test case to check this issue.

Change-Id: I4f437857779e5c0af175c695938fbd7208da244d

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3264

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

  1. … 9 more files in changeset.
[ASTERIXDB-2495][ING] Avoid LET in applying functions to feeds

- user model changes: no

- storage format changes: no

- interface changes: no

LET is not necessary for applying functions to data feeds. We could

inline the function calls when constructing the pipeline query.

Change-Id: I65842f9ac84891b363d7e0a02425258d0df794e7

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3072

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

  1. … 8 more files in changeset.
[NO ISSUE][ING] Tweets ingestion related refactoring

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

As the incoming Tweets are merely JSON records, the Twitter adapter

should be able to use JSON parser. For that purpose, we changed the

record record to be consistent with JSONDataParser which uses char[]

instead of String. Also, "JSON" should also be a valid way to specify

using JSON parser.

Change-Id: Id6d656a4af974499ef0df2e389152ca205c2d078

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3032

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Wail Alkowaileet <wael.y.k@gmail.com>

  1. … 11 more files in changeset.
[NO ISSUE][FUN] Add more data types for external library

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

1. Added more types for external library.

2. Added test case for data types.

3. Fixed minor bug where the parameter type is not trimmed in UDF.

Change-Id: I6ce73e791533b0617074536e0d841242d9e0ee31

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2871

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

  1. … 10 more files in changeset.
[NO ISSUE] User-defined Function Documentation update

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

1. Updated the UDF documentation to be consisten with current master.

2. Cleaned default UDF package to remove useless UDFs.

3. Added the example in documentation as a test case for IT.

4. Reorganized the documentation to keep up with the new structure.

5. Minor changes to other documentation pages to keep style consistent.

Change-Id: I17b1b4d639ca38689298ce88145257e794eb90e1

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2804

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Taewoo Kim <wangsaeu@gmail.com>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 22 more files in changeset.
[NO ISSUE][EXT] Prevent NPE in close() when using TestTypedAdapter

WARN ...CleanupUtils - Failure closing a closeable resource

java.lang.NullPointerException: null

at org.apache.asterix.external.dataset.adapter.FeedAdapter.close(FeedAdapter.java:63)

Change-Id: If2d62ce00858ff9a9f8033bd21d5da5f1f207c56

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2903

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

[NO ISSUE][OTH] Cleanup Unnecessary Semicolons

- user model changes: no

- storage format changes: no

- interface changes: no

Change-Id: I73141e6a5d7950c074a80ea664fa6109ff94cdc4

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2888

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 44 more files in changeset.
Merge commit '2a182c495d311757f1aa9291ca407c8068f00ff2' from release-0.9.4-pre-rc

Change-Id: I69cf41237858ecbca4d89e8f0403e0f443101539

[NO ISSUE][TEST] Use reflection in ADMDataParserTest

Change-Id: I1f88684693230156f27327882f65f05ea3105a8e

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2685

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

[ASTERIXDB-2357] ADMParser Improvements

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- The current ADMParser heavily relies on string operations,

which results in a lot of objects being created. This patch optimize

this by directly operating on char[].

- Improved exception handling in ADMParser.

Change-Id: I106b58e79746b0a6f3d8b79473202653341a7009

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2573

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 11 more files in changeset.
[NO ISSUE] Incremental cleanup of deprecated exception ctors

Change-Id: I1e7c3655828fc6530cef83ea502a6cfbf41acddf

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2533

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

  1. … 239 more files in changeset.
[ASTERIXDB-2216] Disable flaky test which depends on external site

The feed http://lorem-rss.herokuapp.com/feed seems to be down, disabling

the test which relies on it. Also fixed the assert.

Change-Id: Idcf9acf410aa4b8094a93e40681db284a2156345

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2527

Reviewed-by: Michael Blow <mblow@apache.org>

Tested-by: Michael Blow <mblow@apache.org>

[ASTERIXDB-1371][FUN][AQL][SQL] Add standard geometry data type and functions

- A new geometry data type that represents standard OGC objects

- A new set of spatial functions that operate on geometry objects including

spatial analysis, spatial predicates, and spatial aggregate functions

- Parse both Well-Known Text (WKT) and GeoJSON standard formats

- All geometry functions are implemented through the open source library

Esri Geometry API

Change-Id: I9cddeffea42e85469b6fc38f361bd98e64025289

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2056

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

  1. … 126 more files in changeset.
[NO ISSUE][EXT] Java UDF framework refactoring

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

1. Separated the JObjects and its subtypes.

2. Add JBuiltinTypes so we don't have to create an object to get JType.

3. Dead code removal.

4. Memory usage optimization in JRecord Serialization.

5. Several fixes about incomplete type implementations in Java UDF and

getting JObjects in UDF examples.

Change-Id: I3b648191b73fe4aad4f2a6ba1c2066c872fa16a9

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2405

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 42 more files in changeset.
[NO ISSUE][ING] Follow the IFrameWriter contract in Feed Pipeline

- user model changes: no

- storage format changes: no

- interface changes: yes

- it removes unneeded ITupleForwarder interface.

Details:

- Previously, if a failure happens at the writer.open

call in the FeedIntake operator, the IFrameWriter

protocol is not followed since we don't fail before

close is called.

- Previously, fail calls can be done as well inside

of the feed adapter.

- This change moves failing the pipeline outside the

adapter and so the adapter's responsibility is

getting records from outside the system while

the operator takes care of opening, failing, and

closing the writer.

Change-Id: Ife679fb9643dc6b39d035e0eecdb915b227503a5

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2463

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

  1. … 22 more files in changeset.
[ASTERIXDB-2148][FUN] Add init parameter for external UDF

- user model changes: no

- storage format changes: no

- interface changes: yes

Added parameters to LibraryFunction in library.xsd.

Details:

By enabling init function in external UDF, a user can reuse the same UDF

implementation with different paramters. One example can be to reuse the

same Machine Learning algorithm with different model files by assigning

different file paths to the UDF parameter.

Change-Id: I567ce0bcac288267595b2565e53fea61e16fbd65

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2107

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -15
    • +33
    ./resources/library_descriptor.xml
  1. … 33 more files in changeset.
[ASTERIXDB-2256] Reformat sources using code format template

Change-Id: I4faa141c1a8c9700d5e9ac50b839acc9d1eede73

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2310

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

  1. … 970 more files in changeset.
[ASTERIXDB-1564][CONF] Consistently Use Log4j2

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Replace java.util.logging by Log4j2.

- Excluded classes due to their tests:

- IoUtil

- Tracer

Change-Id: Ic137571292f45de1f1994c61d328b97185012197

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2226

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 389 more files in changeset.
[ASTERIXDB-2158] Remove AsterixException (step 1)

- user model changes: no

- storage format changes: no

- interface changes: replace AsterixException on signatures

Details:

This change is the first step towards removing AsterixException from

all interfaces.

Change-Id: I3e858576f39f671cbf5eb14adc7a22aab1335faf

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2135

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Murtadha Hubail <mhubail@apache.org>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 49 more files in changeset.
[NO ISSUE][HYR][*DB] Minor refactoring / address SonarQube comments

Change-Id: Icf10b6df0fdc006675d8f0da6fd06d50200c6b6a

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2098

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

  1. … 60 more files in changeset.
[ISSUE PENDING][FUN] Prevent blind result write in external UDF

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

Current the result from UDF doesn't have type check. It's possible to

write data into result frame with a different datatype. This will cause

unpredicted behavior when reading the data. Also, to keep things

simpler, when return data type is not valid, we throw exception instead

of return null.

Change-Id: Ifce4a9e882febe4529f5c53e54db78e0cfa326fe

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2082

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Steven Jacobs <sjaco002@ucr.edu>

  1. … 10 more files in changeset.
[NO ISSUE][OTH] Fix memory component release trace call

- user model changes: no

- storage format changes: no

- interface changes: no

details:

- The trace for releasing memory component produces

malformed json which makes it fail to load in

chrome tracer. This change fixes it

Change-Id: I1bb6105ffebe01b6ece78983b7b56603f0787060

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2065

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 1 more file in changeset.
[NO ISSUE][*DB] Explicitly create ClusterStateManager

- user model changes: no

- storage format changes: no

- interface changes: no

details:

- Previously, we create the cluster state manager using the

singleton pattern.

- After this change, cluster state manager is created

as part of the cc application start.

- To access the cluster state manager after this change,

the cc application context is used.

Change-Id: Id6532245033ac4c6f6aa9f193539944eecb832f7

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1944

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

  1. … 38 more files in changeset.
[ASTERIXDB-2064][ING] Timeout Stop Feed

- user model changes: no

- storage format changes: no

- interface changes: no

details:

- The abort feed message stops the reader and wait for the

dataflow controller to signal end of life.

- If the reader returns true to stop but the dataflow controller

never signal ends, it can get stuck.

- This change adds a timeout after which, the task thread is

interrupted.

Change-Id: If609a8343767ee7a80689a58af35a1b3fca2964b

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1964

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 7 more files in changeset.
[ASTERIXDB-2008][CLUS] Only add pending removal if node known

[ASTERIXDB-2023][ING] Introduce Enums instead of using bytes

- user model changes: no

- storage format changes: no

- interface changes: no

details:

- Only nodes which are known to cluster manager are added

to the list of nodes pending removal. Other nodes are ignored

- Enums introduced:

- ActiveEvent.Kind

- ActivePartitionMessage.Event

- Remove AdapterRuntimeManager

- Remove AdapterExecutor

Change-Id: I7044896559798426c04a3f46861bc5335b25d140

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1921

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

  1. … 100 more files in changeset.
[ING] Add RSS test using external test feed

- user model changes: no

- storage format changes: no

- interface changes: no

details:

- use a simple feed from http://lorem-rss.herokuapp.com/

Change-Id: Ia020052efbe0fe0de5f78017202f3f7d61b7bff3

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1897

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Xikui Wang <xkkwww@gmail.com>

[ING] Add test case for RSS feed

- user model changes: no

- storage format changes: no

- interface changes: no

details:

- Added a simple test case for RSSRecordReader which fetches records

from dummy website.

Change-Id: I146a54ab069deda56f5ee611d761ec645d2497e6

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1890

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

Remove unnecessary integer boxing in AInt*

Change-Id: I6ecefc4e8617371de00fb430edfe24fca2b1f92f

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1797

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 5 more files in changeset.