Clone Tools
  • last updated 22 mins ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
[ASTERIXDB-2532][RT] per-operator profiling

Enables profiling in queries at the operator-level when the analyze

variable is set in a query.

Change-Id: Ie16f3901ae5b32920d8552d5fd1ec8bb6e2ec8ae

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3226

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

    • -0
    • +5
    ./AlgebricksMetaOperatorDescriptor.java
  1. … 59 more files in changeset.
[NO ISSUE][RT] Window operator runtime optimization

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Runtime optimization for window operators with accumulating

frames (unbounded preceding to current row or n following)

- Refactor window function properties into generic

builtin function properties

Change-Id: I8d1574defc73076ad960c4067432da29ead160a5

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3151

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

  1. … 30 more files in changeset.
[ASTERIXDB-2488][COMP] Support aggregate window functions

- user model changes: yes

- storage format changes: no

- interface changes: no

Details:

- Implement aggregate window functions:

agg_func() OVER (frame_var AS)? (PARTITION BY ... ORDER BY ... frame_spec)

- Where agg_func is a SQL/SQL++ aggregate function

- Fix percent_rank() to always return 0 for the first tuple

- Fix ntile() to handle NULL argument

- Log query after each rewrite rule in SqlppQueryRewriter

- Implement toString() for ADayTimeDuration, fix it for AYearMonthDuration

- Add seek() method to RunFileReader

Change-Id: If0f71118a04c2dbd3462070673d52e67f076b7e1

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3049

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

  1. … 854 more files in changeset.
[NO ISSUE][RT] Follow IFrameWriter protocol in AbstractOneInputPushRuntime

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Many implementations of AbstractOneInputPushRuntime didn't

follow the IFrameWriter protocol causing many unexpected

runtime exceptions.

- This change ensures that all of the subclasses implement the

protocol correctly.

Change-Id: I5133007f298366f58b53acc9f48bc553724dd7b5

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2884

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

  1. … 15 more files in changeset.
[ASTERIXDB-2244][RT] Implement micro union-all operator

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- implement support for binary micro operators in subplans

- implement micro union-all operator

- fix free variables visitor

Change-Id: I11be926f175889978c144dd4483ec565d3d86e2d

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2277

Reviewed-by: Till Westmann <tillw@apache.org>

Contrib: Till Westmann <tillw@apache.org>

Integration-Tests: Till Westmann <tillw@apache.org>

Tested-by: Till Westmann <tillw@apache.org>

    • -2
    • +9
    ./AlgebricksMetaOperatorDescriptor.java
  1. … 39 more files in changeset.
[ASTERIXDB-2256] Reformat sources using code format template

Change-Id: I4faa141c1a8c9700d5e9ac50b839acc9d1eede73

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2310

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

  1. … 984 more files in changeset.
[NO ISSUE] Misc logging cleanup

Change-Id: I2b951e11a9716eb11832d18eba57b124faf146b1

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2295

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

    • -7
    • +1
    ./AlgebricksMetaOperatorDescriptor.java
  1. … 1 more file in changeset.
[ASTERIXDB-2169][STO][TX] Unblock modifications during full scan

- user model changes: no

- storage format changes: no

- interface changes: yes

- added ILSMHarness.replaceMemoryComponentsWithDiskComponents

details:

- During a long running query aka full scan, two things block

incoming modifications:

1) Memory component gets full, is flushed but can't be recycled

because of the search operation inside the component.

2) Read latches on the memory component not being released and

the memory component search cursor is not advancing.

The two cases are addressed in this change for the LSMBTree but

not yet addressed for other indexes.

The proposed solution for case (1) is to poll memory components

states every n records during the search operation. If a memory

component was found to have been flushed, its cursor is moved

to the corresponding disk component allowing the memory

component to be recycled.

The proposed solution for case (2) is to check memory component

cursor every n records. If the cursor has not advanced and the

component has writers, then the latches over the leaf page are

released, and the cursor re-do the operation entering from the

tree root.

- Added a test case.

- Added performance traces for enter and exit components.

Change-Id: I37ba52f6324ed1c5a78465c3a8cbcd351f1ed5bc

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2166

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Luo Chen <cluo8@uci.edu>

    • -17
    • +32
    ./AlgebricksMetaOperatorDescriptor.java
  1. … 51 more files in changeset.
[ASTERIXDB-1897][COMP][RT] Fix complex group-by.

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Fix type computer for numeric aggregations;

- Fix error reporting for SubplanRuntimeFactory;

- Add a negative test query.

Change-Id: Iebb393820a8edd0c54d80248b2a33c77d4f6fd7b

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1923

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 6 more files in changeset.
Support IFrameWriter contract check.

- add a instance-level flag for injecting operators to

check IFrameWriter contract violations;

- check contract violations in runtime tests.

Change-Id: I9827b06f640858f27ec1bcca2a39991780bee3b1

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1618

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

    • -2
    • +8
    ./AlgebricksMetaOperatorDescriptor.java
  1. … 42 more files in changeset.
Separate index build from index access

This change separates index build from index access.

All indexes now have a single dataflow helper which

uses the index path to locate the resource on the nc

to read the resource from memory or disk.

Existing resource metadata and dataflow helpers were

combined into resource builders eliminating lots of

duplicated code.

Change-Id: Ie4ea3aaa63dff8d246fa43ca7c7359729bc8cf47

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1728

Integration-Tests: Ian Maxon <imaxon@apache.org>

Tested-by: Ian Maxon <imaxon@apache.org>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

    • -3
    • +3
    ./AlgebricksMetaOperatorDescriptor.java
  1. … 571 more files in changeset.
Cleanup storage exceptions

Change-Id: I020d2b4b1f4ae48fc2df0b720e70a1ce95867d34

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1619

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

    • -12
    • +10
    ./AlgebricksMetaOperatorDescriptor.java
  1. … 188 more files in changeset.
ASTERIXDB-1791, ASTERIXDB-1796: fix failure handling in runtime operators.

This change includes the following parts:

- Fix the implementation of fail() and close() in several runtime operators

to avoid file handle leak and job hang;

- Add an erase method to RunFileWriter which closes files before deleting

them in order release the holding disk space;

- Call RunFileWriter.close() and RunFileReader.close() in "finally" blocks.

- Fix RunFileReader to not truncate files to be deleted - it is not the root

cause of un-released disk space - open deleted files are the root cuase;

- Check file handle leaks in LangExecutionUtil.tearDown().

Change-Id: I203168171e6dac16b57d2eda960823e3810e22a3

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1513

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

    • -2
    • +2
    ./AlgebricksMetaOperatorDescriptor.java
  1. … 53 more files in changeset.
ASTERIXDB-1714: Eliminate dependency on org.json

Change-Id: Ie9c5400fd134ae75d43385255af7794e968b1c7e

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1392

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -9
    • +6
    ./AlgebricksMetaOperatorDescriptor.java
  1. … 133 more files in changeset.
Unify runtime type exceptions by using error code and message template.

-- fixed string_join to be able to handle self-described list;

-- fixed the input arity of day_of_week;

-- added tests for exceptions.

RuntimeDataException is a newly added super class for errors in the data in the runtime.

It has the following subclasses:

-- IncompatibleTypeException, e.g., 1 + "2"

-- InvalidDataFormatException, e.g., boolean(1)

-- OverflowException, e.g., int8(1024)

-- TypeMismatchException, e.g., substr(1, 2)

-- UnderflowException, e.g., int8(-1024)

-- UnsupportedItemTypeException, e.g., string_concat(["a", 1, "c"])

-- UnsupportedTypeException, e.g., rectangle("1,2 3,4") + rectangle("2,5 7,8")

All "root-cause" exceptions thrown from runtime functions have an error code.

Going forward, all "root-cause" exceptions thrown from asterixdb

should have an error code.

Change-Id: Ie4fff8f5e64ffb027910a4899c0246b37ed5bce7

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1313

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

    • -14
    • +3
    ./AlgebricksMetaOperatorDescriptor.java
  1. … 432 more files in changeset.
ASTERIXDB-1228: Add MISSING into the data model.

1. MISSING repsents the value of a non-existing field in a record

or an out-of-bound index access of a collection;

2. NULL represents that the value of an optional field in a record

is unknown or the value of existing collection entry is unknown.

3. Unit tests for all missing/null-in-missing/null-out scalar functions.

Change-Id: Ia49ed8474bfc5d6604231819065117468c5b0897

Reviewed-on: https://asterix-gerrit.ics.uci.edu/846

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 551 more files in changeset.
ASTERIXDB-1407: let the build branch to broadcast for NestedLoopJoin.

-Change the broadcast branch;

-Fix a bug in SuperActivityOperatorNodePushable;

-Fix jobbuilder to use a fixed location (within query) for operators

with "count=1" constraint;

-Fix OptimizerTest to generate the same directory structure for

actual files as expected files.

-Updates the test query plans.

Change-Id: I0988624406d2f7460f0ee5ac7b4829d81e48c652

Reviewed-on: https://asterix-gerrit.ics.uci.edu/828

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Jianfeng Jia <jianfeng.jia@gmail.com>

    • -0
    • +5
    ./AlgebricksMetaOperatorDescriptor.java
  1. … 97 more files in changeset.
Move Hyracks to subfolder

    • -0
    • +167
    ./AlgebricksMetaOperatorDescriptor.java
    • -0
    • +70
    ./PipelineAssembler.java
    • -0
    • +174
    ./SubplanRuntimeFactory.java
  1. … 4425 more files in changeset.