Clone Tools
  • last updated 29 mins ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
[NO ISSUE][SQLPP] Support dataset synonyms

- user model changes: yes

- storage format changes: no

- interface changes: no

Details:

- Add CREATE / DROP SYNONYM statements to SQL++

- Follow synonyms when resolving dataset references

in SELECT, INSERT, UPSERT, DELETE, LOAD statements

- Add metadata dataset for synonyms: Metadata.Synonym

- Add documentation and tests

Change-Id: Idf806914664205fb186ee3da8b0cebf234266f37

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/4484

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 65 more files in changeset.
[NO ISSUE] Support multipart dataverse names

- user model changes: yes

- storage format changes: no

- interface changes: yes

Details:

- Support dataverse names that consist of multiple parts

- Introduce DataverseName class and adopt it throughout the system

- Add testcases for multipart dataverse names

Change-Id: I1313b2dc0e8df6a9b0ded48dea122afc3ba5dbe7

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/4004

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

  1. … 281 more files in changeset.
[NO ISSUE][STO] Persist Bloom Filter Existence in Index Metadata

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- To clearly identify whether an index has a bloom filter or

not for BTree indexes, persist this information in the index's

metadata stored on each NC.

- For backward compatibility, when reading an index's metadata that

was created before adding the hasBloomFilter field, default its

value based on whether or not the index is a primary key index.

- Remove unused special readObject from LSMBTreeLocalResource.

Change-Id: Icec570d490987de401c036790ee9567238a60301

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/3804

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Reviewed-by: Michael Blow <mblow@apache.org>

  1. … 20 more files in changeset.
[ASTERIXDB-2600][STO] Introduce ConcurrentMergePolicy

- user model changes: yes. Add a new merge policy and make it as default

- storage format changes: no.

- interface changes: no.

Details:

- Introduce ConcurrentMergePolicy that performs concurrent merges

without the maximum component size.

- Make this merge policy as the default merge policy in AsterixDB since

the PrefixMergePolicy has made some wrong design decisions.

Change-Id: I2ed79847584b9fe846d62ad56ee094863538a2a2

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3463

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Luo Chen <cluo8@uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

  1. … 32 more files in changeset.
[ASTERIXDB-2540][STO] Optimize Storage Disk I/O

- user model changes: yes. Add a new storage option:

storage.disk.force.bytes (default 16MB),

- storage format changes: no.

- interface changes: yes.

Introduced IPageWriteCallback to LSM indexes

Details:

- Bypass all queuing (from BufferCache and IOManager) for disk writes.

This queuing is unnecessary but destroys fairness among multiple

writers.

- Introduce IPageWriteCallback to control the behavior of disk page

writes. Currently, this interface is used to perform disk forces

regularly for each writer thread.

Change-Id: I1f618dc7c186623e860239b4d97640fe3528e75b

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3285

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

  1. … 142 more files in changeset.
[NO ISSUE][OTR] GlobalConfig Cleanup

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Remove usage of DEBUG flag in GlobalConfig.

- Remove unused methods/constants in GlobalConfig.

- Move storage constants from GlobalConfig to StorageConstants.

Change-Id: Ie59a646d7d2eedaf1360c97c355f827d80917adf

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3376

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

  1. … 13 more files in changeset.
[NO ISSUE][HYR] Binary compatibility enhancements

Infrastructure & changes to enable binary compatibility with 0.9.4

Change-Id: I77d4919be4853d9afe9b0137861cff3b1d751e20

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3128

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

  1. … 33 more files in changeset.
[ASTERIXDB-2490][NET] Support Encrypted RMI Connections

- user model changes: no

- storage format changes: no

- interface changes: yes

Details:

- Use RMIServer/ClientFactory in metadata node RMI connections

to support both unencrypted and encrypted sockets.

- Add config getter to network security manager.

Change-Id: I11577b7d26d8002d4182255fee0dd769945ca389

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3093

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Reviewed-by: Michael Blow <mblow@apache.org>

  1. … 6 more files in changeset.
[NO ISSUE][API][MTD] Refactor query service servlet

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

Refactor query service servlet in addition to some code

clean-ups.

Change-Id: Idbc81219cbe2f337b3c4aaf4a242ef2ebeb165c5

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3038

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

  1. … 9 more files in changeset.
[ASTERIXDB-2422][STO] Introduce compressed storage

- user model changes: yes

- Add new configuration in the with-caluse to enable compression

- Add new nc configuration in the config file

- storage format changes: yes

- Pages of the primary index can be compressed

- Add a companion file (Look Aside File) with the compressed index

- Allow optional values in the LocalResource

- Add compression information in Metadata.Dataset

- interface changes: yes

- ICCApplicationContext:

- Add getCompressionManager()

- IBufferCache:

- Add getCompressedFileWriter(int fileId)

- ICachedPageInternal:

- Add setCompressedPageOffset(long offset)

- Add getCompressedPageOffset()

- Add setCompressedPageSize(int size)

- Add getCompressedPageSize()

Details:

- Add new integration test for this patch

- Fix ASTERIXDB-2464

- Add ddl-with-clause type validator

Additional details in the design document:

https://cwiki.apache.org/confluence/display/ASTERIXDB/Compression+in+AsterixDB

Change-Id: Idde6f37c810c30c7f1a5ee8bcbc1e3e5f4410031

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2857

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

  1. … 190 more files in changeset.
[NO ISSUE][STO] Add consistency to flush lifecycle

- user model changes: no

- storage format changes: yes

- renamed AbstractLSMIOOperationCallbackFactory

to LSMIOOperationCallbackFactory

- useless classes have been removed.

- LSMBTreeIOOperationCallbackFactory

- LSMBTreeWithBuddyIOOperationCallbackFactory

- LSMInvertedIndexIOOperationCallbackFactory

- LSMRTreeIOOperationCallbackFactory

- interface changes: yes

Details:

- Previously, flushes have different lifecycle depending

on the memory component state

- not allocated

- allocated

- modified

- In certain cases, flush operations are skipped alltogether

- IO Operation callbacks became complicated and difficult

to maintain since calls are done differently in different

cases.

- In certain cases, afterFinalize is called on the IO

Operation callbacks even if beforeOperation was never

called.

- In this change, flushes go through the same lifecycle

events regardless of the state of the memory component.

- In addition, primary and secondary memory components

would reside in different virtual buffer caches due

to skipped flushes, or due to having the secondary

index created when the primary index's memory component

is residing on the virtual buffer cache with index !=0.

- Moreover, when flushes are lagging and all memory

components are being flushed, search operations assumes

the oldest of the memory component is the newest and

produces incorrect results.

- In addition, in case of a failed flush of a component,

the IO scheduler would skip it and flush the next

component. This would produce a bad state on disk.

- In this change, a failed flush can be retried. otherwise,

all future flushes of the component fail due to the failure

of the previously failed flush.

- Previously, when a component fails to modify an index due

to flush failures, it assumes disk is full.

- With this change, the modification failure reports the

original cause of the failed flush.

Change-Id: I29f7992ec6c0f71c5b63d45800b2fb590d651e4b

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2584

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Tested-by: Murtadha Hubail <mhubail@apache.org>

  1. … 161 more files in changeset.
[NO ISSUE][MD] Remove Legacy Metadata Latch

- user model changes: no

- storage format changes: no

- interface changes: yes

Details:

- Remove legacy metadata latch that was used before

the metadata entity locking was introduced.

Change-Id: I5b6b7153c8bc7bb6b489242c5d651879a09b57e0

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2556

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

  1. … 3 more files in changeset.
[NO ISSUE][TX] Remove Invalid Metadata Locking API

- user model changes: no

- storage format changes: no

- interface changes: yes

Details:

- All metadata locking is performed by the CC in

MetadataLockManager. Having the locking API in

MetadataNode is only causing confusion.

Change-Id: I9fae8bd32c8312ef69b38016ad00f170fca9a1c5

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2474

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

  1. … 4 more files in changeset.
[ASTERIXDB-2227][ING] Enabling filitering incoming data in feed

- user model changes: yes

Add syntax support for specifying predicate in connect feed

- storage format changes: no

- interface changes: no

Details:

In some use cases, a user may want to filter the incombing data with

certain attributes. One example can be only store the incoming tweets

with geo locations. This patch enables the <WHERE> clause in connect

feed statement. User can subset the incoming data using following

syntax:

connect feed feeds.TweetFeed to dataset Tweets3 using policy `Basic`

WHERE id NOT LIKE 'nc1:10%' OR username = 'BronsonMike';

Change-Id: I0b3cc6fe9d7fb5f5645dd9c759da448bfe1e88f1

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2255

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

  1. … 13 more files in changeset.
[NO ISSUE][OTR] Clean up INcApplicationContext

Change-Id: Ide17ab0ea7553a6e8e22487b52ba0af379431f81

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2372

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

  1. … 2 more files in changeset.
[NO ISSUE] Metadata partition bootstrap, replication

- Send metadata partition id as part of metadata bootstrap

- Ignore requests to add replicas to myself

- Avoid hard checkpoint at shutdown

Change-Id: I08c91de570bbc90f5532329f72db21a424993fed

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2369

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Integration-Tests: Murtadha Hubail <mhubail@apache.org>

Tested-by: Murtadha Hubail <mhubail@apache.org>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 9 more files in changeset.
[NO ISSUE] More multi-CC support, ConfigManager updates

- add ability for OptionTypes to natively parse JsonNodes

- allow all options to be overridden at the NC level, not just NC options (i.e. common, cc)

- accept controller id from the CC, avoid configuring this on NCs

- update all CCs with metadata bootstrap, not just the primary CC

- remove TxnIdFactory static singleton, management by metadata node

- remove unused build-properties style test configs

- cleanup test iodevices

Change-Id: Iff60887bf71ce3f3ed7201afd9499612bfc83485

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2344

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

  1. … 67 more files in changeset.
[ASTERIXDB-2231][STO] Separate primary op tracker for each partition

- user model changes: no

- storage format changes: no.

- interface changes: yes.

Details:

- Separate primary index operation tracker for each partition, instead

of having a global one on each NC to achieve better scalability.

- As a coordinated change, separate component id generator for each

partition as well.

- Add partition to transaction context so that transaction operations

can operate on proper op tracker.

- Fixes [ASTERIXDB-2232] to calculate dataset partitions correctly.

Change-Id: I9eb3854d2343e45beeccb87b0d434e5f4efd69c9

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2263

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

  1. … 62 more files in changeset.
[ASTERIXDB-2256] Reformat sources using code format template

Change-Id: I4faa141c1a8c9700d5e9ac50b839acc9d1eede73

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2310

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

  1. … 983 more files in changeset.
[ASTERIXDB-2180][FUN] Prevent dropping of entities used by functions

Add dependencies to Functional Metadata

Check dependencies before dropping datasets or functions

Add tests

Change-Id: I2f08ff150dfd57432b88381c507814ddb57bd67b

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2253

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

  1. … 29 more files in changeset.
[ASTERIXDB-2182][ING] Use JSON-like syntax for feed configuration

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

To be consistent with the other JSON-like syntax in defining

configurations, CREATE FEED will change to use JSON object instead of

key-value pairs. Also, the <USING> adapter_name syntax is removed. The

adapter name becomes as part of the configuration. After this change,

the feed configuration are specified as:

<CREATE> <FEED> testFeed <WITH> {

"adapter-name" : "localfs",

"path" : "localhost://../../../../../tweets.json",

"format" : "adm",

"type-name" : "typeTweet"

};

This patch also fixes two minor mistakes when drop feed and adapter in

Metadata.

Change-Id: I2dfb0fdf74d28ae0c45dafcf6babb624bd8b2149

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2209

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 107 more files in changeset.
[ASTERIXDB-2194][COMP] Introduce datasource functions

- user model changes: yes

Some functions can be datasources

- storage format changes: no

- interface changes: yes

- Add IDatasourceFunction: A function that is also a datasource

- Add IFunctionToDataSourceTransformer: transform an unnest

function into a datascan during compilation

Details:

- Currently, functions are location agnostic and are run on

parameters that are either passed through them during compile

time or runtime.

- An exception to this is the dataset function which has

an associated location constraints running on the nodes

which host the dataset.

- In this change, we introduce a general framework that allows

creation of new functions similar to the dataset function.

- Such functions are called datasource Functions.

- A datasource function takes constant parameters and run on

a set of partitions similar to the dataset function.

- The first example of such functions is the DatasetResources

function.

- The DatasetResources function takes two parameters, a dataverse

and a dataset. It is then run on all nodes and returns a set

of dataset resources.

- Test cases are added for this function.

Change-Id: Ibcf807ac713a21e8f4d59868525467386e801303

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2216

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

Tested-by: abdullah alamoudi <bamousaa@gmail.com>

  1. … 41 more files in changeset.
[ASTERIXDB-1564][CONF] Consistently Use Log4j2

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Replace java.util.logging by Log4j2.

- Excluded classes due to their tests:

- IoUtil

- Tracer

Change-Id: Ic137571292f45de1f1994c61d328b97185012197

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2226

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 389 more files in changeset.
[ASTERIXDB-1723] Delete managix

- user model changes: yes

- storage format changes: no

- interface changes: yes

Details:

- Delete managix installer method and all dependencies

- Remove xml-based configuration

Change-Id: I5c710b6ec870bedd49dd46b74ed96a3665670e9b

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2155

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 1378 more files in changeset.
[ASTERIXDB-2177][STO] Use Fixed Storage Root Dir Name

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Eliminate the need to read the storage root dir name

from cluster properties and use a fixed name (storage).

- Eliminate the need to maintain root_metadata file.

Change-Id: I4e9772e9da10cff33f11353610788ba541a35571

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2182

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 14 more files in changeset.
[ASTERIXDB-2171][STO] Remove Temporary Datasets

- user model changes: no

- storage format changes: no

- interface changes: yes

- Removed all reference to temporary datasets.

Details:

- Remove temporary datasets grammar but reserve

<TEMPORARY> token.

- Remove special callbacks for temp datasets.

- Remove metadata garbage collector thread.

- Remove isTemp flag from datasets.

- Remove special storage path for temp datasets.

- Remove temp datasets test cases.

Change-Id: Ib50ea27e2ec9c114508cd4a109cf646e2f20dcb0

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2172

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

  1. … 369 more files in changeset.
[NO ISSUE][FAIL] Replace MetadataException by AlgebricksException in APIs

Change-Id: Ia543ac84b698042582249837912d21ddaa48453f

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2134

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 39 more files in changeset.
[ASTERIXDB-2115] Add Component Ids to LSM Indexes

- user model changes: no

- storage format changes: no

- interface changes: yes

Details:

- Add LSMComponentId to all LSM components. Component Ids are managed

through IO operation callbacks.

- For memory component, it's ID is reset every time it's recycled.

- For disk component, it's ID is copied from the source component(s)

during flush/merge

- For indexes of a dataset, we need to guarantee all their memory

components should recieve the same ID. This is achieved using a shared

component Id generator.

- Fix memory component recycled callback, make sure it's called only

when we've indeed recycled the memory component

A design wiki for this patch: https://cwiki.apache.org/confluence/display/

ASTERIXDB/Component+Id-based+secondary-to-primary+index+acceleration

Change-Id: I8aec6261a84a0729ce35f4b1cb708be299ddb98d

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2025

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 62 more files in changeset.
[ASTERIXDB-2081][STO] Introduce DatasetMemoryManager

- user model changes: no

- storage format changes: no

- interface changes: yes

Added IDatasetMemoryManager to manage datasets memory

reservation and allocation.

Details:

- Reserve metadata datasets memory to allow them to be opened

when needed.

- Add UngracefulShutdownNCApplication to force recovery

to run on AsterixHyracksIntegrationUtil.

- Refactor the use of firstAvilableUserDatasetID to check

for metadata datasets.

- Add ThreadSafe annotation.

- Add test case for RecoveryManager after creating multiple

datasets.

Change-Id: Ica76b3c8eca6f7d2ad1d962fb5ef84267c258571

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2112

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 12 more files in changeset.
[ASTERIXDB-1949][FUN][ING] Function usage check refactoring in feed

- user model changes: no

- storage format changes: no

- interface changes: yes

Removed several interefaces for maintaining reference count.

Added getFunctions and getFeeds method for MetadataNode.

Details:

1. Change the usage check from reference count to scan feed connections

in all dataverses.

2. Add test case for drop dataverse which has function that is being

used in other dataverse. This operation should be blocked.

3. Fix small bug in apply function metadata which the function dataverse

was not recorded properly.

Change-Id: I568f7af18e69216ff2a5fd3b623a1667809b2f95

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1787

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

  1. … 25 more files in changeset.