asterixdb

Clone Tools
  • last updated 29 mins ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
[NO ISSUE][STO] Misc fixes of inverted index

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

This patch mainly improves the inverted index merge perf

in the following ways:

- Fix the bloom filter check on deleted btree

- Remove unnecessary btree search during inverted index range search

- Remove unnecessary page pins when merging inverted lists by

making the cursor remembering the last pinned page

- Fix close/destroy operations of range search cursor

Change-Id: I7bbf6fe2d332db0e0368a4ce0f1ab218ba6da81d

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2514

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

[NO ISSUE] IOCounter improvements

- cache results for short period

- split iostat, proc fs impls

Change-Id: I7789171db6b6d7eea3561c24467af63f065f5dc6

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2515

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Tested-by: Murtadha Hubail <mhubail@apache.org>

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

[NO ISSUE][TEST] Add timeout multiplier for test executor timeouts

Change-Id: I340098ce840f31f2afcef5dd66e43a21ed7ceb9d

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2513

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Tested-by: Michael Blow <mblow@apache.org>

[NO ISSUE][STO] Fix opening of inverted index deleted keys cursor

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- LSMInvertedIndexDeletedKeysBTreeMergeCursor.open used to open

their inner cursors twice. This change fixes it so they are open

only once.

Change-Id: Ib325348265d8cc203aea6cd22591a49221f95411

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2510

Reviewed-by: Luo Chen <cluo8@uci.edu>

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Merge commit 'e6587f628a72f10594e676ec375c1cf26e73ec53' from release-0.9.4-pre-rc into master

Change-Id: I6f23feac3c249745d561359b117cf1e0d573942a

[NO ISSUE][RT] Ensure all NC tasks are aborted before joining

- user model changes: no

- storage format changes: no

- interface changes: yes

- Add getApplication() to NodeControllerService

details:

- This change ensures that all previous tasks

of a CC on an NC are completed before completion of

registration.

Change-Id: I0517e5a390d50e8703ffdbecbb84467c22edda85

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2507

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

[ASTERIXDB-1708][TX] Prevent log deletion during scan

Right now there is a potential for a soft checkpoint to delete a

log file that is about to be read as part of a transaction rollback.

This patch stops the soft checkpoint from proceeding if a rollback

is about to take place and vice-versa.

Change-Id: Icff1a520af24c8fac8e5836cdbf46425b78b1260

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2508

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

[NO ISSUE][CLUS] Asynchronous reregistration with CC

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Perform CC reregistration asynchronously on a

separate thread when connection is restored.

- Ensure a single registration is sent to CC when

connection is restored.

- Restart JVM on unexpected CC registration request

failures.

- Halt JVM on registration failures or timeout.

Change-Id: I404256e6f550c42a6eaf17c0ae4defb7ffb9cb2f

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2505

Reviewed-by: Michael Blow <mblow@apache.org>

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

[NO ISSUE] Simplify IoUtil delete API

Change-Id: I0dabcb642fa3007b3b6e9d30be9911a22ed8f252

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2502

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

[ASTERIXDB-2321][STO] Follow the contract in IIndexCursor.open calls

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- The index cursor contract says that an open call which returns

successfully, leaves the cursor in the open state, otherwise,

the cursor remains in the closed state.

- The LSM cursors have many cursors inside. In the

case where one of the cursors fails to open, and an exception

is about to be thrown, we must close all previously open cursors

since the LSM cursor will be in the closed state

and close will not be called.

Change-Id: I19db2afd2d6ca4a2ca1056cd95ae504b2be69813

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2501

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

[NO ISSUE][RT] Report all errors on SuperActivityOperatorNodePushable

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Currently, if a failure happens in SuperActivityOperatorNodePushable,

we only report that failure and miss the rest of the failures.

This is especially critical in case of job cancellation since we

don't know where each thread was interrupted.

- After this change, we suppress all other failures in the root

failure for reporting purposes.

Change-Id: Ibbf31dd91303ce2f606734fcccb19270875266b3

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2500

Reviewed-by: Michael Blow <mblow@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

[NO ISSUE][TX] Log LogBufferTailReader Read Failures

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- on read failures, include the read attempt status

and the last log record that was read.

Change-Id: Iba2edd8808e1d6de33b1b0fedd27263d99171b39

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2499

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -0
    • +4
    /asterixdb/asterix-transactions/pom.xml
[ASTERIXDB-2332][RT] Fix concurrency issue with RecordMerge and RecordRemoveFields

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

Removed shared instance of PointableHelper.STRING_BINARY_COMPARATOR

which was not thread safe.

Change-Id: I4c5db58184235474f7aed7e5a4b91b6b5685fc06

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2492

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

[NO ISSUE][RT] Remove runtime from ActiveManager after it stops

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

The runtime should not be removed before it's fully stopped. This may

causes issues when downstream operators need to access runtime during

shutdown.

Change-Id: I07c6398f46d4bb6d29b270aa71dee36e0d0917ae

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2494

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

[NO ISSUE][OTH] Cancel queries on abrupt close of user connection

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Since there is no point of processing user queries when the user

disconnects before getting the response of their requests, the

QueryServiceServlet now cancels the user request when that happens.

Change-Id: I902b52322153d114f7ca5f0f06e0af9c5ead8f2a

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2490

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

[NO ISSUE][RT] Fix wait for IO operations

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Some operations such as close dataset, delete components,

and drop index need to wait for IO operations.

- Before this change, the wait for IO operation would just check

the count of IO operations on the dataset info. This is not

enough as a flush might have started by writing the flush log to

the log tail but only on the flush of that log, we trigger the

flush operation and the count of IO operation increases.

- To address this problem, we write a wait log before we check the

IO operation count ensuring that any flush logs in the log tail

have been flushed and counts incremented.

Change-Id: Ibfa883410cd24e0af54732f7ea6f1b4eb2184e8e

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2495

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

[NO ISSUE][SQL] SQLPP UDF test cases revisit

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

1. Revisited all SQLPP UDF test cases. Fixed several testcases that were

disabled due to different issues that were no longer exist.

2. Minor fix to SQLPP so we don't need to nest function body into

subquery.

Change-Id: I844883b99dd9ff729ac518c60f6c47941017f24e

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2493

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

  1. … 5 more files in changeset.
[ASTERIXDB-2319][TEST] Split Queries in start-feed Test

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Split queries to multiple files to avoid

job cancellation of the second query after

the result of the first query has been read.

Change-Id: I06978c83bca612a41337944c61027a1cf3849ac0

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2491

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

[ASTERIXDB-2308][STO] Prevent Race To Allocate Memory Components

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Ensure concurrent threads will not attempt

to allocate memory components twice.

- Synchronize index lifecycle operations.

- Remove unused methods.

Change-Id: Ibd424ba6a2a68939f6ab8a4338c2f6c0c8057ed1

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2489

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@apache.org>

[ASTERIXDB-2285][TEST] Increase Poll Time on Test

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Previous runs showed sporadic failures due to

short poll time and slow execution on jenkins

in test flwor/at00.

Change-Id: Icccdb0a4a961116653e367a8434639488916320d

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2488

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

[NO ISSUE][NET] Improve Logging in TCPEndpoint

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Include end point local address in IO thread name.

- Log TCPEndpoint/Connection addresses on failures.

- Fix multiple IO Threads initialization.

- Fix synchronization for closed connection notification.

Change-Id: I5e78755150a6424fd22f587c4311da7c60b3b55a

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2487

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

[NO ISSUE][API] Add Storage Stats API

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Add a new API on NCs for storage stats.

The API returns the total size of each index

as well as the id and size of each disk component.

Change-Id: Ie0345b00cfd825cbb77664c7617e8301c4f84e6d

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2486

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

[NO ISSUE] Avoid sleep, shutdown log manager on halt

Change-Id: I64749ffd98563099bd68926cd7ca0e697f066baf

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2485

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

[ASTERIXDB-2317] Intermittent Failure in Kill CC NCServiceExecutionIT

Increase timeouts waiting for cluster state transitions, to accomodate

graceful shutdown

Change-Id: I0392cae1d2e9b6bfd9455ba2795711ecf7f1ebe3

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2484

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

[NO ISSUE][OTH] Log open file count on unexpected HttpServer close

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Recently, there has been some cases where the Http server

repeatedly drops its listener. One of the possible causes

for this is a leak in open file descriptor. To determine if

this is the case, we log the open files count when that happens.

Change-Id: Ie43f3392268b3994839d1f98f4de9fe669c7be62

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2483

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

[ASTERIXDB-2329][MTD] Remove Invalid Find Dataset

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Remove invalid findDataset from MetadataManager.

- Add test case for dropping and recreating a dataverse

with indexes and data.

Change-Id: I59b084c85279d926aa60fb7bfcd1a9f6e5935f2a

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2482

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Luo Chen <cluo8@uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

[NO ISSUE][TEST] Refactor licensing test support

Change-Id: I11385a73302b22f9aaaf91c426df8320f52ebb72

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2479

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

[ASTERIXDB-2330][*DB][RT] Add IFunctionRegistrant for dynamic function registration

- user model changes: no

- storage format changes: no

- interface changes: yes

Details:

- Look for IFunctionRegistrant service instances at runtime, and use these to dynamically

register non-core functions with *DB

- Extract fuzzyjoin functions from core runtime

Change-Id: Ia88590280cbf476e08b905d9e1d62c68667a2569

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2480

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

Tested-by: Michael Blow <mblow@apache.org>

    • -12
    • +52
    /asterixdb/asterix-fuzzyjoin/pom.xml
  1. … 63 more files in changeset.
[NO ISSUE] Minor refactoring/cleanup of nc detail servlets

Change-Id: Ib62cd5a6e193e61d52121f45be3ae6067b16f29e

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2476

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

[ASTERIXDB-2320][CLUS] Don't delay removing dead node on max heartbeat misses

Once heartbeat miss threshold miss has occurred, we attempt to contact the failed nc

to confirm it is down, and if not, force him to fail and reconnect. This contact

attempt can take some time, ensure we do not delay marking the node as dead due to

this.

Change-Id: I89c0241fbc88fb6c4150201e4bcba07a3548d3f5

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2475

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>