Clone Tools
  • last updated a few minutes ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
[NO ISSUE][COMP] Introduce Index Only Compiler Property

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Introduce "compiler.indexonly" compiler property to

enable/disable index-only plans.

- Replace usage of 'noindexonly' query property by

the new compiler property.

- Update docs and test cases.

Change-Id: I82d05bbfacbaca17eb8d8a2b6d825aa180e9ab02

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/5567

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

  1. … 33 more files in changeset.
[ASTERIXDB-2635][*DB] Fix JSON plan pretty printing

- user model changes: no

- storage format changes: no

- interface changes: yes

Details:

Use Jackson to pretty print the logical plan as JSON.

- fixed JsonLogicalPlanTest to validate JSON plans

produced by optimizer tests.

Change-Id: Ic5ada2f31afebc3f219b0584b5ae527ddf9e326e

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3567

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 19 more files in changeset.
[ASTERIXDB-2631][COMP][RT] Report count of all warnings

- user model changes: no

- storage format changes: no

- interface changes: yes

Details:

Currently, only runtime warnings are counted. Include

parser & compile-time warnings, as well.

- removed the warn limit as a compiler option and made it

a request parameter.

- the warning collector of the parser collects all warnings

but allow pulling warnings up to a maximum desired.

- the warning collector of the compiler/executor collects up

to the request max-warnings and allow pulling up to a max desired.

- servlets collect from the parser and query compiler &

executor up to the request max-warnings.

Change-Id: Ia7e559313d32a8ba6f450f254a3a7b4b1e662a50

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3564

Reviewed-by: Till Westmann <tillw@apache.org>

Tested-by: Till Westmann <tillw@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

  1. … 90 more files in changeset.
[ASTERIXDB-2631][COMP][RT] Limit number of reported warnings

- user model changes: no

- storage format changes: no

- interface changes: yes

Details:

Limit number of reported warnings.

- added a new compiler property "COMPILER.RUNTIME.WARNINGS"

with default value set to 0.

- updated Stat class to have a field for the count of runtime

warnings.

- updated few classes to propagate the warnings limit number

to Hyracks Task class.

- added new Option to parse unsigned long values.

- TaskProfile has a field for the count.

- updated TestExecutor to allow specifying parameters in

the body of the POST method.

Change-Id: Ie8756f1b9dcb7bbce92bd77692490fbada9a2482

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3535

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -13
    • +21
    ./PhysicalOptimizationConfig.java
  1. … 57 more files in changeset.
[NO ISSUE][COMP] Add support for compiler warnings

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Add support for compiler warnings during parsing,

AST rewriting and query optimization phases

- SQL++ parser emits warnings for unknown hints in

relational expression, between expression,

groupby clause and function call.

- Optimizer emits warning if hash group by hint cannot

be applied because there's no serializable aggregate

- Added testcases for these new warnings

Change-Id: Ic3c1e98c183cd214eea3e4fee24b2b7c46366b52

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3476

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

  1. … 37 more files in changeset.
[NO ISSUE][COMP] Stop reusing variables of Intersect operator

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Intersect operator's input variables should not be reused as

its output variables (fixed IntroduceSelectAccessMethodRule

and IntroduceLSMComponentFilterRule)

- Remove HeuristicOptimizer.isHyracksOp() and updated

ExtractCommonOperatorsRule to use isMicroOperator() instead

- When copying Intersect operator OperatorDeepCopyVisitor now

properly handles its two kinds input variables

- Clear additional state in IntroduceSelectAccessMethodRule.clear()

and OptimizableOperatorSubTree.reset()

- Added testcases for index intersection

Change-Id: I551819b828450760797f6b29af02aad4e124646c

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3450

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

  1. … 28 more files in changeset.
[NO ISSUE][COMP] Change plan printing for window operators

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Print window operators as 'window-aggregate' in the query plan

- Add testcase for a window function call used in WHERE clause,

and a negative testcase

- Implement reset() for LogicalOperatorPrettyPrintVisitorJson

- Make IOptimizationContext.getPrettyPrintVisitor() return

AbstractLogicalOperatorPrettyPrintVisitor

Change-Id: I1446238a7c550abf618a72650c2a811fbc4dd710

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3397

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

  1. … 9 more files in changeset.
[NO ISSUE][COMP] Reduce memory requirements for metadata only queries

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Queries over metadata datasets should run with minimal memory requirements

- Introduce AsterixOptimizationContext which extends AlgebricksOptimizationContext

  and contains information required by Asterix optimizer rules

- Implement equals()/hashCode() for DataSourceId

- Add "plans" testcase type

Change-Id: I0d8902f88f61953e839e2fa0b06a77c324edbc57

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3385

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

Reviewed-by: Till Westmann <tillw@apache.org>

    • -10
    • +0
    ./AlgebricksOptimizationContext.java
  1. … 16 more files in changeset.
[NO ISSUE][COMP] Refactor physical operators for GroupBy and DistinctBy

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Introduce AbstractGroupByPOperator -

base class for physical GroupBy operators

- Introduce AbstractDistinctByPOperator -

base class for physical DistinctBy operators

- Set memory limit for MicroPreclusteredGroupByPOperator

- Remove unused PhysicalOperatorTag.HASH_GROUP_BY

- Rename GroupByOperator.getGbyVarList() to getGroupByVarList()

- Remove constructors that do not specify memory limit from

PreclusteredGroupOperatorDescriptor and PreclusteredGroupWriter

Change-Id: I93f17a6a3e0df6587e518e18eea01ca54a289b4c

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3364

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

  1. … 28 more files in changeset.
[NO ISSUE][RT] Window operator runtime optimization

- user model changes: yes

- storage format changes: no

- interface changes: no

Details:

- Improve memory management for window operators

- Add "compiler.windowmemory" property that specifies memory

budget for each window operator (default is 4MB, min is 160KB)

- Consolidated negative window operator testcases into a single one

Change-Id: I6756e92046883f79db339ef490cca8bc8b7b1fb8

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3227

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

  1. … 58 more files in changeset.
[NO ISSUE][OTH] Support log redaction

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

Support log redaction.

Change-Id: I602c833ba2a055da8fbe8782ec62be683ff4581b

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3105

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

  1. … 28 more files in changeset.
[NO ISSUE] Refactoring plan optimization method

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Refactoring plan optimization method.

Change-Id: I8275bca00b41969444824cffd12f542d802bf26f

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3101

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

[ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

- user model changes: yes

- storage format changes: no

- interface changes: yes

details:

- new plan for sort operation which includes sampling and

replicating the stream of data to be sorted. Sort-merge connector

is removed from the plan. The sorted result now is in multiple partitions.

- new optimization rule to check whether full parallel sort is applicable.

- new Forward operator to read the replicated sort input stream and

to receive the ouput of the sampling.

- new sequential merge connector to merge a globally ordered result residing

in multiple partitions (in addition to the connector's partition computer).

- "asterix-lang-aql/pom.xml" is changed as a result of refactoring

code related to the range map handling.

- new private sampling function to generate the range map object

(local & global functions) & their type computers.

user model changes:

- new compiler property is added to enable and disable parallel sort.

interface changes:

- "ILogicalOperatorVisitor.java" includes Forward Operator.

- "ITuplePartitionComputer.java" includes initialize() to enable partitioner

to do some initialization. FieldRangePartitionComputerFactory uses it to

pick a range map.

- "ITuplePartitionComputerFactory.java". createPartitioner() is changed to

createPartitioner(IHyracksTaskContext hyracksTaskContext). Context is needed

for transferring the range map throught the context.

Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2393

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

  1. … 356 more files in changeset.
[NO ISSUE][OTH] Remove Unused Imports

- user model changes: no

- storage format changes: no

- interface changes: no

Change-Id: Iafff39073d0fedaff74a26ef7e3260008a79ff0c

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2893

Reviewed-by: Michael Blow <mblow@apache.org>

Tested-by: Michael Blow <mblow@apache.org>

  1. … 67 more files in changeset.
[NO ISSUE][OTH] Fix Job + Algebricks Logging Level

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Move job execution + Algebricks trace logs to trace

log level.

Change-Id: I3a94fb251523ff7056fa4ba7db503c8f49bfbfbb

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2853

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

  1. … 13 more files in changeset.
[NO ISSUE][COMP] Check logging level before writing a log message

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Cleanup: Check logging level before writing to the logger

Change-Id: Ib126397c390f0d32436a80a306981106afa2a794

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2543

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

  1. … 7 more files in changeset.
[ASTERIXDB-2083][COMP][RT][IDX][SITE] Budget-Constrained Inverted index search

- user-model changes: add text.searchmemory parameter

- storage format changes: no

- interface changes: IInvertedIndexSearcher, IInPlaceInvertedIndex,

IInvertedIndexAccessor, IInvertedListCursor

IObjectFactory, IPartitionedInvertedIndex,

IIndexAccessor

Details:

- Introduce text.searchmemory parameter in the configuration

to conduct budget-constrained inverted index search to prevent

a possible OOM exception

- Remove non-standard hyracks task context from the inverted-index-search

Change-Id: Ib2b2ef7c0b8c55ef66a5322be5d97ebbbf287bf5

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2251

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: abdullah alamoudi <bamousaa@gmail.com>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 100 more files in changeset.
[ASTERIXDB-2256] Reformat sources using code format template

Change-Id: I4faa141c1a8c9700d5e9ac50b839acc9d1eede73

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2310

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

  1. … 984 more files in changeset.
[ASTERIXDB-1564][CONF] Consistently Use Log4j2

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Replace java.util.logging by Log4j2.

- Excluded classes due to their tests:

- IoUtil

- Tracer

Change-Id: Ic137571292f45de1f1994c61d328b97185012197

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2226

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 389 more files in changeset.
[ASTERIXDB-1812][RT] Budget the memory usage for pre-clustered group-by.

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- let pre-clustered group-by consider memory budget.

Change-Id: I670269b0b8f446d06d8dd73202194574aa524e85

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1940

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 27 more files in changeset.
Make query plans more user-friendly.

Change-Id: I91c15913009295352cf7cccf4a4028f1ff271e3f

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1721

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Steven Jacobs <sjaco002@ucr.edu>

  1. … 97 more files in changeset.
Fix ASTERIXDB-1867: get Plan String only when context is not null

- Fix AbstractRuleController so that getPlanString() is

only executed when context is not null.

Change-Id: I4c4dfb50f16c48e4b09e8148d13f0debbd2de659

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1651

Reviewed-by: Xikui Wang <xkkwww@gmail.com>

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Changed the physical tag of ReplicatePOperator (SPLIT -> REPLICATE)

Change-Id: Ic298f90c5bc9875cea1017aff17a524214596b1e

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1219

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 93 more files in changeset.
Reduced logging

Change-Id: I3a3756427999c4b79a01cd5a9cbfe79adf44b95b

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1204

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

  1. … 3 more files in changeset.
merge master

    • -11
    • +19
    ./AlgebricksOptimizationContext.java
  1. … 21 more files in changeset.
snapshot for range state transition

    • -15
    • +25
    ./AlgebricksOptimizationContext.java
  1. … 70 more files in changeset.
Support UNION ALL.

- support heterogenous input types for UNION ALL and the output is resolved to a mimimally generalized type;

- fix the partitioning property inference for UnionAllPOperator;

- fix for ASTERIXDB-1354, a bug in IsomorphismVariableMappingVisitor.

Change-Id: Ifadf1707bb2b6bed22f8fc5792c635e87291a468

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1035

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

    • -24
    • +32
    ./AlgebricksOptimizationContext.java
  1. … 71 more files in changeset.
some pretty printing refactoring/cleanup

- introduce AlgebricksAppendable

- use AlgebricksAppendable instead of PrintWriter and StringBuilder for plan

printing

- avoid creation of intermediate strings for each Operator

- only log plans in the optimizer if the log-level requires it

- change param to indent output JSON from INDENT to PRETTY

Change-Id: Ied0203adc51e9710690ace74fe1e152eb7a716e8

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1004

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

  1. … 10 more files in changeset.
merge with master

  1. … 13 more files in changeset.
Fixed the nested loop join algorithm to use join memory parameter.

Change-Id: I50e24ea023c9ae9aa043698716e5021d4dafc327

Reviewed-on: https://asterix-gerrit.ics.uci.edu/953

Reviewed-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@apache.org>

    • -10
    • +10
    ./PhysicalOptimizationConfig.java
  1. … 2 more files in changeset.