Clone Tools
  • last updated a few minutes ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
[NO ISSUE][FUN] Introduce IEvaluatorContext

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Introduce IEvaluatorContext - a context for evaluators

that is available both at runtime and during constant

folding at compile time

- IEvaluatorContext provides access to IHyracksTaskContext

which is only available at runtime. At compile time this

accessor returns null

- Instantiate evaluators with IEvaluatorContext

instead of IHyracksTaskContext

Change-Id: Iff90c7739e46d6602fcd0609e67f6a8cd9714ca5

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3481

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 685 more files in changeset.
[ASTERIXDB-2552][RT] Reuse objects on reseting buffer manager

- user model changes: no

- storage format changes: no

- interface changes: yes

Details:

The "VariableFrameMemoryManager" releases the physical frames

objects, the logical frames objects and the free slots objects

on reset(). This patch is to retain those objects and only

release them on close() (as also stated in the interface).

- added close() to IFrameFreeSlotPolicy to make the interface similar

to the other interfaces (IFrameBufferManager, IFramePool, ... etc)

where close() is supposed to release while reset() is supposed to only

reset but retain resources.

- renamed "InMemorySortRuntimeFactory" to "MicroSortRuntimeFactory".

Change-Id: I88bea27e5024c621412ef609475e2a7ba1913afa

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3372

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 10 more files in changeset.
[ASTERIXDB-2552][RT] Implement micro external sort for subplans

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

This patch is to change the in memory sort used in subplans to

be a micro external sort to avoid out of memory error.

- added reset() to the runs merger to allow reusing the runs merger.

- renamed "InMemoryStableSortPOperator" to "MicroStableSortPOperator".

- changed the tag from "IN_MEMORY_STABLE_SORT" to "MICRO_STABLE_SORT".

- added test cases.

Change-Id: I930849d644c60d461d2869c9773b85e49b46fbdb

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3353

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

  1. … 33 more files in changeset.
[NO ISSUE][COMP] Refactor physical operators for GroupBy and DistinctBy

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Introduce AbstractGroupByPOperator -

base class for physical GroupBy operators

- Introduce AbstractDistinctByPOperator -

base class for physical DistinctBy operators

- Set memory limit for MicroPreclusteredGroupByPOperator

- Remove unused PhysicalOperatorTag.HASH_GROUP_BY

- Rename GroupByOperator.getGbyVarList() to getGroupByVarList()

- Remove constructors that do not specify memory limit from

PreclusteredGroupOperatorDescriptor and PreclusteredGroupWriter

Change-Id: I93f17a6a3e0df6587e518e18eea01ca54a289b4c

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3364

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

  1. … 28 more files in changeset.
[ASTERIXDB-2554][HYR] Add UTF8 and byte array comparator factories

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

Add comparator factories for UTF8StringPointable, UTF8StringLowercasePointable,

UTF8StringLowercaseTokenPointable and ByteArrayPointable instead of using

PointableBinaryComparatorFactory, a wrapping factory that will create a factory

each time (which also creates a comparator each time).

Change-Id: Ied6a29210a3dc1ba9fd553fb0a67ff4340e4571f

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3355

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

  1. … 48 more files in changeset.
[ASTERIXDB-2516][RT] Move primitive comparators to Hyracks and make singleton

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- moved 2 comparators, boolean and long comparators from asterix to hyracks.

- added byte, short, integer, float and double comparator

factories to Hyracks to replace PointableBinaryComparatorFactory.

- removed checking lengths of 0 from PointableBinaryComparatorFactory.

- changed tests to use the primitive factories.

Change-Id: If15dc4e0dd0db942a4cadb15abbe56cbfe617b48

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3294

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

  1. … 52 more files in changeset.
[NO ISSUE] Apply / enforce java import order

The process-sources target will now sort imports as well as

format source code; the source-format job will likewise verify

import order in addition to source code format

Change-Id: I55d976c4df10d9919c6a25683be2a3e3304e65d9

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3288

Integration-Tests: Michael Blow <mblow@apache.org>

Tested-by: Michael Blow <mblow@apache.org>

Reviewed-by: Till Westmann <tillw@apache.org>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 624 more files in changeset.
[ASTERIXDB-2488][COMP] Support aggregate window functions

- user model changes: yes

- storage format changes: no

- interface changes: no

Details:

- Implement aggregate window functions:

agg_func() OVER (frame_var AS)? (PARTITION BY ... ORDER BY ... frame_spec)

- Where agg_func is a SQL/SQL++ aggregate function

- Fix percent_rank() to always return 0 for the first tuple

- Fix ntile() to handle NULL argument

- Log query after each rewrite rule in SqlppQueryRewriter

- Implement toString() for ADayTimeDuration, fix it for AYearMonthDuration

- Add seek() method to RunFileReader

Change-Id: If0f71118a04c2dbd3462070673d52e67f076b7e1

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3049

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

  1. … 854 more files in changeset.
[NO ISSUE][OTH] Fix hyracks-api Dependences

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Ensure hyracks-api module depends only on hyracks-util. This way

new APIs can be added to hyracks-api and used on all other modules

without facing cyclic dependency issues.

Change-Id: I7f4329b3dad99c256fb2e10a7863aaca41990ce0

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3047

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 47 more files in changeset.
[ASTERIXDB-2466][FUN] Implement window functions

- user model changes: yes

- storage format changes: no

- interface changes: no

Details:

- Implement window functions with SQL syntax:

function() OVER ((PARTITION BY expr1, expr2, ...)? ORDER BY exprA, exprB, ...)

- Where supported functions are:

ROW_NUMBER(), RANK(), DENSE_RANK(), PERCENT_RANK(), NTILE()

Change-Id: Ia28af8773cb11049c38d440c51b9c3cd1ed2bab4

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3002

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

  1. … 167 more files in changeset.
[NO ISSUE][COMP] Check logging level before writing a log message

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Cleanup: Check logging level before writing to the logger

Change-Id: Ib126397c390f0d32436a80a306981106afa2a794

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2543

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

  1. … 7 more files in changeset.
[NO ISSUE] Incremental cleanup of deprecated exception ctors

Change-Id: I1e7c3655828fc6530cef83ea502a6cfbf41acddf

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2533

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

  1. … 239 more files in changeset.
[ASTERIXDB-1280][TEST] JUnit cleanup

- user model changes: no

- storage format changes: no

- interface changes: no

Change-Id: I2a57643682d45097ae5ac4adc1fe97fbb3111385

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2461

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

  1. … 15 more files in changeset.
[ASTERIXDB-2244][RT] Implement micro union-all operator

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- implement support for binary micro operators in subplans

- implement micro union-all operator

- fix free variables visitor

Change-Id: I11be926f175889978c144dd4483ec565d3d86e2d

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2277

Reviewed-by: Till Westmann <tillw@apache.org>

Contrib: Till Westmann <tillw@apache.org>

Integration-Tests: Till Westmann <tillw@apache.org>

Tested-by: Till Westmann <tillw@apache.org>

  1. … 41 more files in changeset.
[ASTERIXDB-2256] Reformat sources using code format template

Change-Id: I4faa141c1a8c9700d5e9ac50b839acc9d1eede73

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2310

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

  1. … 981 more files in changeset.
[ASTERIXDB-2149] Enable multiple normalized keys in sort

- user model changes: no

- storage format changes: no

- interface changes: yes. The interface of sort is changed.

Currently, during the (in-memory) sort, we use an int normalized keys to

speed up comparisions by avoiding random memory accesses. However, this

technique is inefficient if the first 4 bytes of the sorting keys are

not distinctive. From performance point of view, it's better to use

longer normalized keys when it's possible (2-3x improvements).

This is enabled by this patch by:

- Allowing multiple normalized keys during sort, and the length of each

normalized key can be longer (multiple integers).

- Enable memory budgeting of pointer directories as well during sort

(but for performance, we still use int[], instead of byte[] from frame).

The next patch will enable the AsterixDB layer to use this feature to

speed up sort performance.

Change-Id: I4354242ff731b4b006b8446b58f65873047dde78

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2127

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

  1. … 29 more files in changeset.
AsterixDB Rat Execution Audit & Fixes

Change-Id: I3d771c5b63eaef4b6a0b4da769039ed5bca17736

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1609

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

  1. … 493 more files in changeset.
Configuration Revamp

- Ini section of node / cc details now returns ini param names instead of

managix option names

- Normalized command line -vs- ini file configuration parameter names

- Eliminated unused parameters

- Ini validation

- Migrate *DB parameters out of [app] and into nc / cc sections as

appropriate

- Eliminate [app] section. Cluster-wide configuration lives in [common]

- Sort properties alphabetically when returned by HTTP api

Change-Id: I95b7e0bd4538ef42817c8826e76412150074b754

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1487

Reviewed-by: Michael Blow <mblow@apache.org>

Integration-Tests: Michael Blow <mblow@apache.org>

Tested-by: Michael Blow <mblow@apache.org>

  1. … 174 more files in changeset.
Continue Cleaning Up File References and Splits

1. Make FileSplit an abstract class with two subclasses;

Managed and Unmanaged. A Managed FileSplit can be mapped

in a new subclass MappedFileSplit that maps a relative path to an

IO device. UnmanagedFileSplit is for files outside the io devices.

2. Remove all usages of absolute paths in file split in test cases. The

only remaining place is the write statement.

3. Fix some of the hidden issues in the tests that were working because

of our use of the absolute paths.

4. Revert the decision of selecting the IO device to the CC.

Change-Id: I166af8f9b3a2257f94d7b05db94888fb7cb4c79e

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1359

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

  1. … 252 more files in changeset.
Cleanup FileSplit and FileReference

This change gives FileSplit and FileReference specific meaning to

avoid confusion of an absolute vs relative, local vs global, inside

an IO device vs outside IO devices.

In addition, it enables better abstraction of global partitions and

delegate the responsibility of choosing which partition goes to which

IO device to the IO Manager through the introduction of FileDeviceComputer

In details:

Previously, the LocalResource in Hyracks had partition (storage partition)

and there is no such thing in Hyracks. This scope leak is bad. In addition

The local resource had a name and a path. they were always the same and so

the name was removed.

The storage partition was instead moved to asterixdb implementation of the

serialized object in the local resource.

With all of these changes, the cluster controller (compiler) only needs to

know about partitions and relative paths. It doesn't need to worry about

heterogenous Node setups and different io device configurations. For File

assignment to IO devices, a new interface (IFileDeviceComputer) was

introduced which can be overriden by applications to have their own

strategy for distributing files among IO devices.

Change-Id: I4fac508bf9af5a3bed41a3cf4464d2cbfecf2f61

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1352

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

  1. … 287 more files in changeset.
Unify runtime type exceptions by using error code and message template.

-- fixed string_join to be able to handle self-described list;

-- fixed the input arity of day_of_week;

-- added tests for exceptions.

RuntimeDataException is a newly added super class for errors in the data in the runtime.

It has the following subclasses:

-- IncompatibleTypeException, e.g., 1 + "2"

-- InvalidDataFormatException, e.g., boolean(1)

-- OverflowException, e.g., int8(1024)

-- TypeMismatchException, e.g., substr(1, 2)

-- UnderflowException, e.g., int8(-1024)

-- UnsupportedItemTypeException, e.g., string_concat(["a", 1, "c"])

-- UnsupportedTypeException, e.g., rectangle("1,2 3,4") + rectangle("2,5 7,8")

All "root-cause" exceptions thrown from runtime functions have an error code.

Going forward, all "root-cause" exceptions thrown from asterixdb

should have an error code.

Change-Id: Ie4fff8f5e64ffb027910a4899c0246b37ed5bce7

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1313

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

  1. … 430 more files in changeset.
Index-only plan step 2: Added SplitOperator

- Introduced SplitOperator that sends a tuple to only one output frame unlike the ReplicateOperator

that propagates a tuple into all outputs frames.

- Removed PartitioningSplitOperator and PartitioningSplitOperatorDescriptor that are not functional

(lacking physical operator)

- Added a unit test case of SplitOperatorDescriptor in PushRuntimeTest.

Change-Id: Ice190827513cd8632764b52c9d0338d65c830740

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1196

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

  1. … 38 more files in changeset.
Hard Tabs++

Eliminate more hard tabs, widen scope of checkstyle rules to prevent regressions.

Relax whitespace expectations in polygon ctor

Change-Id: Ia3f78fc6705bc79a21cfa97fdfcad30998ab1058

Reviewed-on: https://asterix-gerrit.ics.uci.edu/967

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 455 more files in changeset.
Supports Left Outer Join and Left Outer Unnest in SQL++.

- change record format to allow MISSING value for an optional field;

- support roundtrips of MISSING valued closed fields for load and insert;

- suppress fields with MISSING values in result printer;

- fix ByNameToByIndexFieldAccessRule and LoadRecordFieldsRule;

- update test queries and results.

Change-Id: Ie0caea9c1842d93541b067a1193d117af30d8dfc

Reviewed-on: https://asterix-gerrit.ics.uci.edu/899

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 585 more files in changeset.
ASTERIXDB-1228: Add MISSING into the data model.

1. MISSING repsents the value of a non-existing field in a record

or an out-of-bound index access of a collection;

2. NULL represents that the value of an optional field in a record

is unknown or the value of existing collection entry is unknown.

3. Unit tests for all missing/null-in-missing/null-out scalar functions.

Change-Id: Ia49ed8474bfc5d6604231819065117468c5b0897

Reviewed-on: https://asterix-gerrit.ics.uci.edu/846

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 551 more files in changeset.
Move Hyracks to subfolder

    • -0
    • +63
    ./main/scripts/run.cmd
    • -0
    • +99
    ./main/scripts/run.sh
    • -0
    • +25
    ./test/resources/results/scanMicroSortWrite.out
  1. … 4416 more files in changeset.