Clone Tools
  • last updated 26 mins ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
[ASTERIXDB-2435][*DB][CLUS] Block unknown nodes

Provide mechanism by which unauthorized nodes can be prevented from

joining hyracks cluster. In *DB, only authorize configured nodes to

join.

Change-Id: I3a45a41f69e8e9968dd65bb2268d3d3a2ced9664

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2851

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 4 more files in changeset.
[NO ISSUE] Set JobId in thread name waiting for job completion

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- To allow correlation between running tasks and requests waiting

for job completion, we set the job id in the request thread and

we also add it to the name of the parallel threads in Super

Activity Operator Node Pushable.

Change-Id: I003432da0109c253e976f130e91f1f1f5ff628f3

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2837

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -0
    • +2
    ./cc/work/WaitForJobCompletionWork.java
  1. … 2 more files in changeset.
[NO ISSUE][OTH] Add CC/NC Ping Function

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Add ping function that can be used to ask a node

to ping the CC.

Change-Id: I676e523dccbf94d1e5af4ea408e026af260c9b06

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2831

Reviewed-by: Michael Blow <mblow@apache.org>

Tested-by: Michael Blow <mblow@apache.org>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 7 more files in changeset.
[NO ISSUE][CONF] Add Configuration For Max Http Request Size

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Add configuration for max http request size and default

it to 50 MB.

- Close client connection if request exceeds max size.

- Add test case for large request.

Change-Id: I11153490fc022d3a94b5b6f7dc3e20204fb105a0

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2826

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

  1. … 14 more files in changeset.
[ASTERIXDB-2314][HYR] Dataset in class names in Hyracks

- user model changes: no

- storage format changes: no

- interface changes: yes

Change-Id: I260add3a2dfb4a722440af5008a1a75f9446579e

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2478

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

    • -10
    • +10
    ./cc/ClusterControllerService.java
    • -316
    • +0
    ./cc/dataset/DatasetDirectoryService.java
    • -50
    • +0
    ./cc/dataset/IDatasetDirectoryService.java
    • -0
    • +50
    ./cc/result/IResultDirectoryService.java
    • -0
    • +316
    ./cc/result/ResultDirectoryService.java
    • -45
    • +0
    ./cc/work/GetDatasetDirectoryServiceInfoWork.java
    • -0
    • +45
    ./cc/work/GetResultDirectoryAddressWork.java
    • -7
    • +7
    ./cc/work/GetResultPartitionLocationsWork.java
    • -2
    • +2
    ./cc/work/RegisterResultPartitionLocationWork.java
    • -2
    • +2
    ./cc/work/ReportResultPartitionWriteCompletionWork.java
  1. … 107 more files in changeset.
[NO ISSUE][RT] Improve logging in task related works

Change-Id: Ia4e24a95aaac37b1d3d0d9a35266109ae0315293

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2803

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 5 more files in changeset.
[NO ISSUE][HYR] Ignore Job Notifications For Unknown Jobs

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Ignore notifications for unknown jobs.

Change-Id: I743b469f56f6126c2fdd4161fcaa48fc5f4d7218

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2793

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

[NO ISSUE] Use public address/port for client advertisement

Change-Id: Ia88912f144fa3b0e4b79341c245bd45ae0453f80

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2706

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

[NO ISSUE] Expose public address / ports for CC services

Change-Id: I61f8988b32f13a1eba029fa7120d6e753aacc151

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2704

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 1 more file in changeset.
[ASTERIXDB-2397][*DB] Enable execution on Java 9/10

(cherry picked from commit 5c28118b5341306f4774d32937770f107f3a015b)

Change-Id: I221dfba1b2d15e33b312feb3fb50e70d536027fd

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2700

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

    • -7
    • +11
    ./cc/work/GetNodeDetailsJSONWork.java
  1. … 13 more files in changeset.
[ASTERIXDB-2397][*DB] Enable execution on Java 9/10

Change-Id: I720507856c235443d64b6024b75f931656badc6d

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2686

Reviewed-by: Michael Blow <mblow@apache.org>

Contrib: Michael Blow <mblow@apache.org>

Integration-Tests: Michael Blow <mblow@apache.org>

Tested-by: Michael Blow <mblow@apache.org>

    • -7
    • +11
    ./cc/work/GetNodeDetailsJSONWork.java
  1. … 13 more files in changeset.
[ASTERIXDB-2386][CLUS] Allow extension of the global recovery manager

Allow the Asterix GlobalRecoveryManager to be extended

Cleanup some deployed job spec methods

Change-Id: I1213e702a77ededde18ee0b50bc105212f43480d

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2640

Reviewed-by: Till Westmann <tillw@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Steven Jacobs <sjaco002@ucr.edu>

  1. … 11 more files in changeset.
[ASTERIXDB-2373][HYR,CLUS] Allow upsert of JobSpecs for Deployed Jobs

- user model changes: none

- storage format changes: none

- interface changes: new methods added

This change adds the upsertDeployedJobSpec method, enabling

a Deployed Job to update its Job Specification.

Added call in test.

Removed DeployedJobService (moved methods to BAD codebase).

Change-Id: I01fd5d43896d520fe75e1007d7bd39324f6f6e4b

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2619

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Xikui Wang <xkkwww@gmail.com>

  1. … 13 more files in changeset.
[NO ISSUE][HYR] Ensure IJobLifecycleListener is notified on cancelled queued jobs

Change-Id: I7e26c9d1015725f895876f5873eccd3f86b17653

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2624

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

  1. … 1 more file in changeset.
[NO ISSUE][NET] IPC Connections Improvements

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Refactor IPCConnectionManager.

- Halt on IPC message serialization failures.

- Ensure channels and handles are closed on connection

failures.

- Remove IPCHandle unneeded CONNECT_FAILED state.

- Fix RegisterNodeWork failure handling.

- Consistently use NodeControllerRemoteProxy for NC RPC.

Change-Id: I4049b16573c13fcdb1b12c0b6b2a97ee1fcc709e

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2617

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

  1. … 11 more files in changeset.
[NO ISSUE] Incremental cleanup of deprecated exception ctors

Change-Id: I1e7c3655828fc6530cef83ea502a6cfbf41acddf

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2533

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

  1. … 238 more files in changeset.
[NO ISSUE][RT] Add job start timestamp to the joblet context

- user model changes: no

- storage format changes: no

- interface changes: yes

Details:

- add job start timestamp to the joblet context

- make CommonFunctionMapUtil extensible by products

- enhance DateTimeFormatUtils.parseDateTime() to return

timezone information and not throw exception on invalid data

- add GregorianCalendarSystem.getDayOfYear(), getWeekOfYear()

- change sleep() to sleep first then evaluate the argument

- skip fractional trailing 0s when printing nano-duration

- fix documentation about using column aliases in GroupBy

Change-Id: I190663ec5e709584ef449f8279b1d2a5a0b099dd

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2511

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 26 more files in changeset.
[NO ISSUE][RT] Ensure all NC tasks are aborted before joining

- user model changes: no

- storage format changes: no

- interface changes: yes

- Add getApplication() to NodeControllerService

details:

- This change ensures that all previous tasks

of a CC on an NC are completed before completion of

registration.

Change-Id: I0517e5a390d50e8703ffdbecbb84467c22edda85

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2507

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 7 more files in changeset.
[ASTERIXDB-2320][CLUS] Don't delay removing dead node on max heartbeat misses

Once heartbeat miss threshold miss has occurred, we attempt to contact the failed nc

to confirm it is down, and if not, force him to fail and reconnect. This contact

attempt can take some time, ensure we do not delay marking the node as dead due to

this.

Change-Id: I89c0241fbc88fb6c4150201e4bcba07a3548d3f5

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2475

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

  1. … 1 more file in changeset.
[NO ISSUE] Refactor shutdown watchdog to ExitUtil

Change-Id: Id27635f90587c64ee29ab1a4ac2b1a6182042900

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2470

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 7 more files in changeset.
[NO ISSUE][HYR] Add 'getTimer' to IControllerService

Change-Id: I9b33cfbc345d15d95a99ea2c993ad173816bada8

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2469

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 2 more files in changeset.
[ASTERIXDB-2229][OTR] Restore Thread Names in Thread Pool

- user model changes: no

- storage format changes: no

- interface changes: no

- Restore thread names to their original names

before returning them to the pool of the NC

app.

- Remove explicit thread name resets.

- Delete unused ThreadExecutor class.

Change-Id: I3bda1b65e7aefd35d2b8cfa814f73369c3bf5a18

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2447

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 9 more files in changeset.
[NO ISSUE] Decouple heartbeat data calculation from transmission

- also, generate native JSON objects instead of POJOs for node config & stats

- fix illegal regex on Windows for node config

Change-Id: Iccb02350b56328ba1adbca97a1cb5efeb9d9ad14

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2439

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

  1. … 17 more files in changeset.
[NO ISSUE] Set MaxGCPauseMillis to not exceed 1/2 of dead node detection threshold

Help prevent nodes under heavy gc from missing too many heartbeats

Change-Id: I7e51db5ccfbb4771ba1f6e0264abfd69f833e7e7

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2426

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

[NO ISSUE] Make NodeManager.remove() idempotent

Change-Id: I0e2b14526be49c9817c0d64836fcf57647ac59b2

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2414

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Integration-Tests: Michael Blow <mblow@apache.org>

Tested-by: Michael Blow <mblow@apache.org>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 1 more file in changeset.
[NO ISSUE] Add support for NodeGroup upsert, etc.

Also, enable resource id generation when only pending removal nodes are absent

from the cluster

Change-Id: I15cfb74bc345680102cedafa99f7ff4f144860bc

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2389

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Integration-Tests: Murtadha Hubail <mhubail@apache.org>

Tested-by: Murtadha Hubail <mhubail@apache.org>

  1. … 12 more files in changeset.
[ASTERIXDB-2284][CLUS] Ensure Node Failure on Heartbeat Miss

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Request the node which exceeded its heartbeat misses

to shutdown to ensure its failures.

- Ensure thread safety of lastHeartbeatNanoTime in

NodeControllerState.

Change-Id: I121f85fd858484377a9d888d18c3069c239f00fc

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2390

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

[ASTERIXDB-2204][STO] Fix implementations and usages of IIndexCursor

- user model changes: no

- storage format changes: no

- interface changes: yes

- IIndexCursor.close() is now idempotent and can be called on

a closed cursor.

- IIndexCursor.destroy() is now idempotent and can be called

on a destroyed cursor.

- Add IIndexAccessor.destroy() letting the accessor know it is

safe to destroy its reusable cursors and operation contexts.

- Add IIndexOperationContext.destroy() letting the context

know that the user is done with it and allow it to release

resources

details:

- Previously, implementations of the IIndexCursor interface

didn't enforce the interface contract. This change enforces

the contract for all the implementations.

- With the enforcement of the contract, all the users of the

cursors are expected to follow and enforce the expected lifecycle.

- Test cases were added.

Change-Id: I98a7a8b931eb24dbe11bf2bdc61b754ca28ebdf9

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2324

Reviewed-by: Michael Blow <mblow@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 132 more files in changeset.
[NO ISSUE] More multi-CC support, ConfigManager updates

- add ability for OptionTypes to natively parse JsonNodes

- allow all options to be overridden at the NC level, not just NC options (i.e. common, cc)

- accept controller id from the CC, avoid configuring this on NCs

- update all CCs with metadata bootstrap, not just the primary CC

- remove TxnIdFactory static singleton, management by metadata node

- remove unused build-properties style test configs

- cleanup test iodevices

Change-Id: Iff60887bf71ce3f3ed7201afd9499612bfc83485

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2344

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

    • -4
    • +0
    ./cc/application/CCServiceContext.java
  1. … 65 more files in changeset.
[ASTERIXDB-1554][CONF] Do Not Override Logger Config

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Do not override logger config for hyracks/asterix

roots if they are already configured.

Change-Id: Iff7ed7c094d9044f5959f9e24713302af9774786

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2335

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

  1. … 8 more files in changeset.