Clone Tools
  • last updated 24 mins ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
[NO ISSUE][FUN] Fix type inference and casting in UDFs

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

The current UDF framework handles the argument types in a sloppy way. It

takes in the arguments and reads them as the expected data types in the

configuration. This could cause exception at runtime when passing in

arguments with unexpected datatypes. When setting the arguments, it did

a type casting for numeric values only to make sure the int64 from query

interface can be evaluated properly. However, this is not robust enough.

This patch fixes the type inference for UDFs during the complation time.

The ExternalTypeComputer is refactored to return defined data type, and

meanwhile checks the argument data types. Also, the

IntroduceDynamicTypeCastForExternalFunctionRule is modified to cover the

type castings for data types besides record type.

Change-Id: I40506fcca3cd8f14bbd6412359683433256c4c1f

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3270

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Xikui Wang <xkkwww@gmail.com>

    • -0
    • +1
    ./my_array_sum/my_array_sum.1.adm
    • -1
    • +2
    ./validate-default-library/validate-default-library.1.adm
  1. … 29 more files in changeset.
[NO ISSUE][FUN] Fix type inference in JListAccessor

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

As the list element type is available, there is no need to deserialize

element types from the data. Doing so, for example, will cause the

object type to be fully open type and closed fields to be recognized as

open fields. This will further cause NPE in UDFs. Modified the upperCase

test case to check this issue.

Change-Id: I4f437857779e5c0af175c695938fbd7208da244d

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3264

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

  1. … 9 more files in changeset.
[ASTERIXDB-2495][ING] Avoid LET in applying functions to feeds

- user model changes: no

- storage format changes: no

- interface changes: no

LET is not necessary for applying functions to data feeds. We could

inline the function calls when constructing the pipeline query.

Change-Id: I65842f9ac84891b363d7e0a02425258d0df794e7

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3072

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

    • -0
    • +1
    ./udf_filter_on_feed/udf_filter_on_feed.1.adm
    • -0
    • +1
    ./validate-default-library/validate-default-library.1.adm
  1. … 9 more files in changeset.
[NO ISSUE][FUN] Add more data types for external library

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

1. Added more types for external library.

2. Added test case for data types.

3. Fixed minor bug where the parameter type is not trimmed in UDF.

Change-Id: I6ce73e791533b0617074536e0d841242d9e0ee31

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2871

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

    • -0
    • +1
    ./type_validation/type_validation.1.adm
    • -0
    • +1
    ./validate-default-library/validate-default-library.1.adm
  1. … 11 more files in changeset.
[NO ISSUE] User-defined Function Documentation update

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

1. Updated the UDF documentation to be consisten with current master.

2. Cleaned default UDF package to remove useless UDFs.

3. Added the example in documentation as a test case for IT.

4. Reorganized the documentation to keep up with the new structure.

5. Minor changes to other documentation pages to keep style consistent.

Change-Id: I17b1b4d639ca38689298ce88145257e794eb90e1

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2804

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Taewoo Kim <wangsaeu@gmail.com>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -1
    • +1
    ./validate-default-library/validate-default-library.1.adm
  1. … 27 more files in changeset.
[ASTERIXDB-2148][FUN] Add init parameter for external UDF

- user model changes: no

- storage format changes: no

- interface changes: yes

Added parameters to LibraryFunction in library.xsd.

Details:

By enabling init function in external UDF, a user can reuse the same UDF

implementation with different paramters. One example can be to reuse the

same Machine Learning algorithm with different model files by assigning

different file paths to the UDF parameter.

Change-Id: I567ce0bcac288267595b2565e53fea61e16fbd65

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2107

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -0
    • +5
    ./keyword_detector/keyword_detector.1.adm
    • -0
    • +2
    ./validate-default-library/validate-default-library.1.adm
  1. … 34 more files in changeset.
[ASTERIXDB-2180][FUN] Prevent dropping of entities used by functions

Add dependencies to Functional Metadata

Check dependencies before dropping datasets or functions

Add tests

Change-Id: I2f08ff150dfd57432b88381c507814ddb57bd67b

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2253

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

    • -8
    • +8
    ./validate-default-library/validate-default-library.1.adm
  1. … 29 more files in changeset.
[ASTERIXDB-2096][COMP] Fix type casting for ExternalFunction

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

1. The current IntroduceDynamicTypeCastForExternalFunctionRule

cannot handle external function calls in nested record constructor.

This patch fix this issue by visiting all nested parameters for

external functions.

2. The ResultExtractor should be able to handle multiple queries

in single statement file as AQL does.

Change-Id: I65c298def75b18fab01f513012e28fc44fdc2fd4

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2010

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

  1. … 9 more files in changeset.
[ASTERIXDB-2098][TEST] Adding UDF execution tests for SQLPP

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

1. Added test cases for UDFs to SQLPP.

2. Added SqlppExecutionIT to test UDFs in SQLPP.

Change-Id: I3dae4300589f1b16ea5abbf13d81b686a2283add

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2009

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

    • -0
    • +8
    ./validate-default-library/validate-default-library.1.adm
  1. … 18 more files in changeset.
ASTERIXDB-1901 Fix IntroduceDynamicTypeCastForExternalFunctionRule

1. Instead of pattern matching, now we will inspect every paramter of

UDF. If there is a type mismatch, a cast function will be added.

2. Fixed the issue that type casting only applies to the first argument.

3. Added test case for this.

Change-Id: I6f44b2460ae3322fc52451e7939b6b5e711790a7

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1730

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -0
    • +2
    ./upperCase/upperCase.1.adm
  1. … 6 more files in changeset.
Update adm.grammar and printers for double/float

adm.grammar was not accepting suffixed doubles. This was breaking

a lot of round-trip ADM cases. I also changed the default print for

float and double to not have a suffix, because this makes the output

very odd (NaNd, NaNf, -Infinityf). The print mode for integers

has also been changed to omit the suffix, but, similarly the old

format is still accepted.

Change-Id: I6164d74f6c286fa160a30cff9ad47e97a30e0770

Reviewed-on: https://asterix-gerrit.ics.uci.edu/932

Reviewed-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <michael.blow@couchbase.com>

    • -100
    • +100
    ./classad-parser-new/classad-parser-new.1.adm
    • -5
    • +5
    ./classad-parser-old/classad-parser-old.1.adm
  1. … 500 more files in changeset.
Move merged files

    • -0
    • +100
    ./classad-parser-new/classad-parser-new.1.adm
    • -0
    • +5
    ./classad-parser-old/classad-parser-old.1.adm
  1. … 162 more files in changeset.
Merge branch 'master' into hyracks-merge2

    • -100
    • +0
    ./classad-parser/classad-parser.1.adm
    • -5
    • +0
    ./classad-parser2/classad-parser2.1.adm
  1. … 483 more files in changeset.