Clone Tools
  • last updated 15 mins ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
Separate index build from index access

This change separates index build from index access.

All indexes now have a single dataflow helper which

uses the index path to locate the resource on the nc

to read the resource from memory or disk.

Existing resource metadata and dataflow helpers were

combined into resource builders eliminating lots of

duplicated code.

Change-Id: Ie4ea3aaa63dff8d246fa43ca7c7359729bc8cf47

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1728

Integration-Tests: Ian Maxon <imaxon@apache.org>

Tested-by: Ian Maxon <imaxon@apache.org>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

    • -16
    • +16
    ./ExternalFileIndexAccessor.java
  1. … 568 more files in changeset.
Cleanup storage exceptions

Change-Id: I020d2b4b1f4ae48fc2df0b720e70a1ce95867d34

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1619

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

  1. … 188 more files in changeset.
Improve reading from and writing to Metadata pages

This change introduces a new interface IComponentMetadata.

Each LSM component is associated with a metadata object

which can be used to read and write arbitrary data to

the metadata pages of components. When flushing a

component, data in its metadata component is automatically

flushed to the disk component. For merge operations,

the IO Callback is responsible for merging the components'

metadata pages.

Change-Id: Id95ef33c0a0bc1abb3fc3ecdea5611ee4acd6dfa

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1476

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@apache.org>

Integration-Tests: Ian Maxon <imaxon@apache.org>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -15
    • +11
    ./ExternalFileIndexAccessor.java
  1. … 152 more files in changeset.
Introduce IStorageComponentProvider

Change-Id: If86750cdb2436c713f6598e54d4aaaf23d9f7bbf

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1451

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 425 more files in changeset.
Subset hive-exec, make twitter4j a provided dep.

Change-Id: Iee4276f540ec8552181bfb452882654b5faa17df

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1427

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

  1. … 17 more files in changeset.
Runtime ErrorCode fix in external data

1. Refactored most of the exceptions in external-data package to use

ErrorCode.

2. This change only covers RuntimeDataException/HyracksDataExcetion.

3. Refactor errorcode names

Change-Id: Ida810a56bf4aef1394879f088a6a5e8f82c60b74

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1374

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

  1. … 96 more files in changeset.
ASTERIXDB-1711: remove some more Aql-prefixes

Change-Id: I041341fe76e9338b7c568a9eacaf826f8fa0ed46

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1360

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 227 more files in changeset.
Cleanup FileSplit and FileReference

This change gives FileSplit and FileReference specific meaning to

avoid confusion of an absolute vs relative, local vs global, inside

an IO device vs outside IO devices.

In addition, it enables better abstraction of global partitions and

delegate the responsibility of choosing which partition goes to which

IO device to the IO Manager through the introduction of FileDeviceComputer

In details:

Previously, the LocalResource in Hyracks had partition (storage partition)

and there is no such thing in Hyracks. This scope leak is bad. In addition

The local resource had a name and a path. they were always the same and so

the name was removed.

The storage partition was instead moved to asterixdb implementation of the

serialized object in the local resource.

With all of these changes, the cluster controller (compiler) only needs to

know about partitions and relative paths. It doesn't need to worry about

heterogenous Node setups and different io device configurations. For File

assignment to IO devices, a new interface (IFileDeviceComputer) was

introduced which can be overriden by applications to have their own

strategy for distributing files among IO devices.

Change-Id: I4fac508bf9af5a3bed41a3cf4464d2cbfecf2f61

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1352

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

  1. … 287 more files in changeset.
avoid some exception wrapping

Change-Id: Ia8fff04c314eb146ce0a73beb0b8ec412cb2d280

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1179

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 13 more files in changeset.
Add Asterix Extension Manager

More extension support is added. A user can now provide implementations

for the IExtension interface which will give them more control over

the behavior of the system and give them the ability to add custom

features.

Initial customizations include:

1. Metadata Tuple Translators

2. Metadata Datasets

3. Query Translators

4. Statement Handlers

5. Lang Compilation Provider

Change-Id: I280268495cc3aad00f898cba21f7299f7120ce5c

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1017

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

  1. … 232 more files in changeset.
ASTERIXDB-1228: Add MISSING into the data model.

1. MISSING repsents the value of a non-existing field in a record

or an out-of-bound index access of a collection;

2. NULL represents that the value of an optional field in a record

is unknown or the value of existing collection entry is unknown.

3. Unit tests for all missing/null-in-missing/null-out scalar functions.

Change-Id: Ia49ed8474bfc5d6604231819065117468c5b0897

Reviewed-on: https://asterix-gerrit.ics.uci.edu/846

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 551 more files in changeset.
Deadlock-free locking protocol is enabled

- Added EntityCommitProfiler class in TransactionSubsystem.java file:

This profiler takes a report interval (in seconds) parameter and

reports entity level commit count every report interval (in seconds)

only if IS_PROFILE_MODE is set to true. The profiler runs in a separate

thread. However, the profiler thread doesn't start reporting the count

until the entityCommitCount > 0. The profiler can be used to measure

1) IPS (Inserts Per Second) and

2) IIPS (instantaneous IPS) for the every report interval.

Change-Id: Ie58ae2f519baa53599e99b51bd61ea5f8366dafd

Reviewed-on: https://asterix-gerrit.ics.uci.edu/825

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <hubailmor@gmail.com>

  1. … 63 more files in changeset.