Clone Tools
  • last updated 16 mins ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
[NO ISSUE][FUN] Fix type inference and casting in UDFs

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

The current UDF framework handles the argument types in a sloppy way. It

takes in the arguments and reads them as the expected data types in the

configuration. This could cause exception at runtime when passing in

arguments with unexpected datatypes. When setting the arguments, it did

a type casting for numeric values only to make sure the int64 from query

interface can be evaluated properly. However, this is not robust enough.

This patch fixes the type inference for UDFs during the complation time.

The ExternalTypeComputer is refactored to return defined data type, and

meanwhile checks the argument data types. Also, the

IntroduceDynamicTypeCastForExternalFunctionRule is modified to cover the

type castings for data types besides record type.

Change-Id: I40506fcca3cd8f14bbd6412359683433256c4c1f

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3270

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Xikui Wang <xkkwww@gmail.com>

  1. … 29 more files in changeset.
[NO ISSUE] Fix logging for ncservice-based tests

Change-Id: Id264aede0f62558ad6e34355047c623a1d594692

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3311

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@uci.edu>

    • -2
    • +1
    ./NCServiceExecutionIT/ncservice1.conf
    • -2
    • +1
    ./NCServiceExecutionIT/ncservice2.conf
  1. … 27 more files in changeset.
Merge commit '963a544' from 'stabilization-f69489' to 'master'

Change-Id: Iedb57cd217d3115b94d13761d8779e81293003da

  1. … 7 more files in changeset.
[NO ISSUE] Fix serialization of surrogate pairs on non-UTF8 platforms

Change-Id: I17228e7dfdcd220e1c0caf5fcc8207fb8f3a7733

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3300

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

  1. … 2 more files in changeset.
[ASTERIXDB-2491][TXN] Recovery fixes

- user model changes: no

- storage format changes: yes[1]

- interface changes: no

Details:

- Change field offset type from integer16 to integer32

- Add recovery test

- Add version to log entries

[1] LogRecord format change with this patch; old LogRecord format can still

be processed, however old instances cannot read new log format

Change-Id: Iaf14b9a73a0239763bfeb0ce2d81cf952e6d72d3

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3065

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

  1. … 13 more files in changeset.
[NO ISSUE] Fix logging in NCServiceExecutionIT

- remove GC options not supported by OpenJDK

- don't run HDFSCluster by default (not compatible w/ Java 11)

Change-Id: I17ca6ec4605f6d2ceb2c727e9479e56becc29670

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3156

Contrib: Michael Blow <mblow@apache.org>

Tested-by: Michael Blow <mblow@apache.org>

Integration-Tests: Michael Blow <mblow@apache.org>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

  1. … 1 more file in changeset.
[NO ISSUE][FUN] Add more data types for external library

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

1. Added more types for external library.

2. Added test case for data types.

3. Fixed minor bug where the parameter type is not trimmed in UDF.

Change-Id: I6ce73e791533b0617074536e0d841242d9e0ee31

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2871

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

  1. … 12 more files in changeset.
[NO ISSUE] User-defined Function Documentation update

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

1. Updated the UDF documentation to be consisten with current master.

2. Cleaned default UDF package to remove useless UDFs.

3. Added the example in documentation as a test case for IT.

4. Reorganized the documentation to keep up with the new structure.

5. Minor changes to other documentation pages to keep style consistent.

Change-Id: I17b1b4d639ca38689298ce88145257e794eb90e1

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2804

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Taewoo Kim <wangsaeu@gmail.com>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 27 more files in changeset.
[NO ISSUE] Put asterix-server components in folder

Change-Id: I6281454bcd6b3f4a6c0f14d0c5b1d082efdd0955

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2880

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3198

Tested-by: Michael Blow <mblow@apache.org>

  1. … 14 more files in changeset.
[NO ISSUE] Put asterix-server components in folder

Change-Id: I6281454bcd6b3f4a6c0f14d0c5b1d082efdd0955

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2880

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

  1. … 14 more files in changeset.
[NO ISSUE] Remove instance.name, storage.subdir config properties

Change-Id: Ic530559e96eb4c744499f46a5595e3a4c59f7683

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2585

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Integration-Tests: Murtadha Hubail <mhubail@apache.org>

Tested-by: Murtadha Hubail <mhubail@apache.org>

Reviewed-by: Michael Blow <mblow@apache.org>

  1. … 12 more files in changeset.
[NO ISSUE][TEST] Retry on file copy, ignore failures after 3 retries

- for NCServiceExecutionIT based tests, print JVM final flags, help ensure

max gc wait time is being configured properly by NCService

Change-Id: I7127ee0dcba6ead1a19a373b62e4d6aa22467909

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2454

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

  1. … 5 more files in changeset.
[ASTERIXDB-2148][FUN] Add init parameter for external UDF

- user model changes: no

- storage format changes: no

- interface changes: yes

Added parameters to LibraryFunction in library.xsd.

Details:

By enabling init function in external UDF, a user can reuse the same UDF

implementation with different paramters. One example can be to reuse the

same Machine Learning algorithm with different model files by assigning

different file paths to the UDF parameter.

Change-Id: I567ce0bcac288267595b2565e53fea61e16fbd65

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2107

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 35 more files in changeset.
[NO ISSUE] Remove Pregelix

Change-Id: I7d5f789ee6c3ca3ffa41edcc36fa7f494b04924b

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2381

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Integration-Tests: Murtadha Hubail <mhubail@apache.org>

Tested-by: Murtadha Hubail <mhubail@apache.org>

    • -261
    • +0
    ./integrationts/asterix-configuration.xml
  1. … 34 more files in changeset.
[ASTERIXDB-2172] Increase heartbeat max misses

Avoid GC causing intermittent false dead-node detection by increasing

heartbeat duration from 10 to 50s.

Change-Id: I3178e61b13185eea0d74187e882ad1d207702f74

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2318

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 7 more files in changeset.
[ASTERIXDB-2172] collect GC logs to assist in diagnosis

Change-Id: I6841da6ffb756843ba4265c64aa42faff3acfc09

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2297

Reviewed-by: Till Westmann <tillw@apache.org>

Tested-by: Michael Blow <mblow@apache.org>

  1. … 1 more file in changeset.
[ASTERIXDB-2236][NET] Reestablish Nc2Nc Closed Connection

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Enable keep alive on Nc2Nc sockets.

- Open Nc2Nc connection when closed.

- Add "port" command to test framework to

ensure a port is not used anymore.

- Add test case for reestablishing connection

between nodes.

Change-Id: I3af4af37de6c1b36d7032c78ee60f1e5ca5f7c1c

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2273

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

    • -0
    • +48
    ./NcLifecycleIT/cc.conf
    • -0
    • +20
    ./NcLifecycleIT/ncservice1.conf
    • -0
    • +21
    ./NcLifecycleIT/ncservice2.conf
  1. … 8 more files in changeset.
[ASTERIXDB-2195][REPL] Replace Static Replication

- user model changes: no

- storage format changes: no

- interface changes: yes

- Redesigned all replication interfaces

Details:

- Replace static replication and fault tolerance by

dynamic storage API.

- Remove static based fault tolerance strategies.

- Redesign replication APIs and classes to smaller

maintainable parts.

- Clean up replication properties.

- Unify logic for checkpoints when replication is

enabled.

- Remove static replication test cases.

- Add replication runtime test cases for:

- Bulkload component replication.

- Memory component recovery.

- Flushed component replication.

- Add replication integration test for:

- Resync failed replica.

Change-Id: Ic5c4b0ac199a4530c807e558c8aebb1eb1284048

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2252

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

    • -20
    • +0
    ./MetadataReplicationIT/ncservice1.conf
    • -21
    • +0
    ./MetadataReplicationIT/ncservice2.conf
  1. … 238 more files in changeset.
[ASTERIXDB-2182][ING] Use JSON-like syntax for feed configuration

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

To be consistent with the other JSON-like syntax in defining

configurations, CREATE FEED will change to use JSON object instead of

key-value pairs. Also, the <USING> adapter_name syntax is removed. The

adapter name becomes as part of the configuration. After this change,

the feed configuration are specified as:

<CREATE> <FEED> testFeed <WITH> {

"adapter-name" : "localfs",

"path" : "localhost://../../../../../tweets.json",

"format" : "adm",

"type-name" : "typeTweet"

};

This patch also fixes two minor mistakes when drop feed and adapter in

Metadata.

Change-Id: I2dfb0fdf74d28ae0c45dafcf6babb624bd8b2149

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2209

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 103 more files in changeset.
[ASTERIXDB-1723] Delete managix

- user model changes: yes

- storage format changes: no

- interface changes: yes

Details:

- Delete managix installer method and all dependencies

- Remove xml-based configuration

Change-Id: I5c710b6ec870bedd49dd46b74ed96a3665670e9b

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2155

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -0
    • +52
    ./MetadataReplicationIT/cc.conf
    • -0
    • +20
    ./MetadataReplicationIT/ncservice1.conf
    • -0
    • +21
    ./MetadataReplicationIT/ncservice2.conf
    • -0
    • +51
    ./ReplicationIT/cc.conf
    • -0
    • +20
    ./ReplicationIT/ncservice1.conf
    • -0
    • +21
    ./ReplicationIT/ncservice2.conf
    • -0
    • +261
    ./integrationts/asterix-configuration.xml
  1. … 1364 more files in changeset.
[NO ISSUE] Fix duplicate -Xmx args being passed

The -Xmx override was looking for " -Xmx" instead of "-Xmx".

Added a test to stop regression.

Change-Id: I7e59be72fcc9aea3c315c2cee5339c1c7573ef51

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2206

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

Tested-by: Michael Blow <mblow@apache.org>

  1. … 4 more files in changeset.
[NO ISSUE][API] Add NC Storage API

- user model changes: no

- storage format changes: no

- interface changes: yes

Add IStorageSubsystem to track storage partitions

replicas.

Details:

- Add NC API port.

- Add storage API to NCs.

- Add StorageSubsystem to track storage partitions

replicas.

Change-Id: I120d9892bc9fe5a73395cd5a2ddc30b51b73ced2

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2190

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 25 more files in changeset.
Re-register NC with CC on reestablished IPCHandle

In case of failed NC -> CC IPCHandle due to CC crash/restart, the NC

needs to re-register with the CC in order to rejoin the cluster, as the

CC ignore heartbeats from unregistered nodes.

- Improve toString on IPCHandle

- Add tests for killing & restarting CC / NCs to NCServiceExecutionIT

- Retrigger NCService on detected dead node

- Ensure jobIds are not reused on CC restart

- NCService shouldn't truncate NC log

Change-Id: I6f93ca9ab37e56e02bafcdecd1e2d0cf664faef6

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1830

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 23 more files in changeset.
Unify structure of Cluster Controller and Node Controller

Change-Id: Ife3a002371a07ee9cdd32e1ffd50cc775bf1d453

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1514

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 160 more files in changeset.
Configuration Revamp

- Ini section of node / cc details now returns ini param names instead of

managix option names

- Normalized command line -vs- ini file configuration parameter names

- Eliminated unused parameters

- Ini validation

- Migrate *DB parameters out of [app] and into nc / cc sections as

appropriate

- Eliminate [app] section. Cluster-wide configuration lives in [common]

- Sort properties alphabetically when returned by HTTP api

Change-Id: I95b7e0bd4538ef42817c8826e76412150074b754

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1487

Reviewed-by: Michael Blow <mblow@apache.org>

Integration-Tests: Michael Blow <mblow@apache.org>

Tested-by: Michael Blow <mblow@apache.org>

  1. … 174 more files in changeset.
Fix Issues in 0.9.0 RC1 Release Assemblies

- Unable to build *DB from source assembly due to:

o Git plugin requiring build from within Git clone

o License automation templates referenced hyracks sources (expected

co-location)

- Missing APLv2 license text from head of binary assembly LICENSE files

- Added license text to head of NCService conf files

- Exclude useless DEPENDENCIES from asterixdb source assembly

Change-Id: I1821416abb0387ee05b9add9f9c15798deed25e1

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1452

Reviewed-by: Till Westmann <tillw@apache.org>

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -0
    • +17
    ./NCServiceExecutionIT/ncservice1.conf
    • -0
    • +17
    ./NCServiceExecutionIT/ncservice2.conf
  1. … 21 more files in changeset.
[asterix] -> [app] for NCService configs

Change-Id: Ie7bcc249b117bffb5671d087cd1760ef6a85110a

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1246

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 2 more files in changeset.
ASTERIXDB-1482: Added NCServiceExecutionIT, HyracksVirtualCluster.

NCServiceExecutionIT runs all execution tests against a local cluster

managed by the NCService deployment framework.

HyracksVirtualCluster offers programmatic NCService deployment

control along with improved HyracksNCProcess/HyracksCCProcess.

Further fixes and improvements:

1. Fix handling of iodevices/storagedir (ASTERIXDB-1482)

2. Proper handling of [nc] default section in all cases

3. Ensure asterixnc, etc. scripts are executable

4. Consolidate Ini handling

5. Pruned some dead code, including VirtualClusterDriver

6. A bit of refactoring and extended commenting

Change-Id: If3eb450782a595cf85d04a2c2e9cc732564e65e6

Reviewed-on: https://asterix-gerrit.ics.uci.edu/958

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@apache.org>

    • -0
    • +25
    ./NCServiceExecutionIT/cc.conf
    • -0
    • +3
    ./NCServiceExecutionIT/ncservice1.conf
    • -0
    • +4
    ./NCServiceExecutionIT/ncservice2.conf
  1. … 19 more files in changeset.