Clone Tools
  • last updated 23 mins ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
[NO ISSUE][COMP] Refactor physical operators for GroupBy and DistinctBy

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Introduce AbstractGroupByPOperator -

base class for physical GroupBy operators

- Introduce AbstractDistinctByPOperator -

base class for physical DistinctBy operators

- Set memory limit for MicroPreclusteredGroupByPOperator

- Remove unused PhysicalOperatorTag.HASH_GROUP_BY

- Rename GroupByOperator.getGbyVarList() to getGroupByVarList()

- Remove constructors that do not specify memory limit from

PreclusteredGroupOperatorDescriptor and PreclusteredGroupWriter

Change-Id: I93f17a6a3e0df6587e518e18eea01ca54a289b4c

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3364

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

  1. … 24 more files in changeset.
[ASTERIXDB-2554][HYR] Add UTF8 and byte array comparator factories

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

Add comparator factories for UTF8StringPointable, UTF8StringLowercasePointable,

UTF8StringLowercaseTokenPointable and ByteArrayPointable instead of using

PointableBinaryComparatorFactory, a wrapping factory that will create a factory

each time (which also creates a comparator each time).

Change-Id: Ied6a29210a3dc1ba9fd553fb0a67ff4340e4571f

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3355

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

  1. … 34 more files in changeset.
[ASTERIXDB-2310][STO]Enforce Key Uniquness using PKIndex

- user model changes: no

- storage format changes: yes. Primary key index

now has bloom filters.

- interface changes: no

Details:

- Add bloom filters to primary key index.

- Introduce LSMPrimaryInsertOperator to separate uniqueness check from

the primary index. When the primary key index is available, it will be

used for uniqueness check. This implementation of this operation is

similar to LSMPrimaryUpsertOperator.

Change-Id: I7a52bb75ee5b14521972999df2f45ba62adc5af1

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2453

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

  1. … 48 more files in changeset.
[NO ISSUE] Create an abstraction for the ForwardOperatorDescriptor

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Abstract the activities out of the ForwardOperatorDescriptor so we can reuse

the same basic framework for multiple forward Operators.

- Abstract the ForwardOperatorDescriptor out of the ForwardPOperator so we can

reuse the same basic framework for multiple forward Operators.

Change-Id: Icc3db4b386e69a98c2a1c40dadc96eb3e1a5d4fa

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3338

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 15 more files in changeset.
[NO ISSUE][RT] Improve PreclusteredGroupWriter

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Modified PreclusteredGroupWriter to only save group fields

from a last tuple in a frame instead of the whole frame

- move PermutingFrameTupleReference and PermutingTupleReference

from 'hyracks-storage-am-common' to 'hyracks-dataflow-common'

Change-Id: Ic75de2e6b64d0aacaf48096ecc9d47fc8e95c9cf

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3351

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

  1. … 23 more files in changeset.
[NO ISSUE] Make IOManager more configurable

Change-Id: I1c8ad11c2b8b983ef4bf7cf78c2f068accddfff4

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3133

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

Contrib: Michael Blow <mblow@apache.org>

  1. … 1 more file in changeset.
[ASTERIXDB-2516][RT] Prevent passing null type to comparator provider

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

This change is to disallow passing null as IAType to the comparator

provider when asking for a comparator. It also includes few fixes

for issues appearing as a result of this change.

- changed the generic comparator to check whether the tag of data

at runtime is a valid tag and throw an exception if not.

- modified the comparator provider to return non-tagged comparator

for IAType SHORTWITHOUTINFOTYPE which is a short without tag.

SHORTWITHOUTINFOTYPE should not use the generic comparator since

the input data has no tag.

- fixed Dataset class to consider external dataset when getting

the IAType of the primary keys. The primary keys for external

datasets are different from regular datasets. They are not

part of the record type. Previously, null would be returned.

This would cause a failure when getting a comparator for the

primary keys of an external dataset since the type passed is null.

- fixed the TypeTraitProvider to give the correct lengths for

duration, day_time_duration, and year_month_duration.

Change-Id: I37767a3f3d1e3b29597d2a4998c0b60005cadb09

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3336

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 6 more files in changeset.
[NO ISSUE][RT] Window operator performance improvement

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Improve performance of window operator: save only

last values of partitioning columns from the

previous frame instead of the whole frame

Change-Id: Ib8d6b3b99ee24c73b76fd118040ed6972e6798d9

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3346

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

  1. … 6 more files in changeset.
[NO ISSUE][HYR] Use System.nanotime for StopWatch

Change-Id: Ic64f1b938c26161f5fb482af5ee1da6121f49643

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3339

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@uci.edu>

[ASTERIXDB-2516][RT] Move primitive comparators to Hyracks and make singleton

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- moved 2 comparators, boolean and long comparators from asterix to hyracks.

- added byte, short, integer, float and double comparator

factories to Hyracks to replace PointableBinaryComparatorFactory.

- removed checking lengths of 0 from PointableBinaryComparatorFactory.

- changed tests to use the primitive factories.

Change-Id: If15dc4e0dd0db942a4cadb15abbe56cbfe617b48

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3294

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

  1. … 38 more files in changeset.
[ASTERIXDB-2516][RT] Modifty range map function of parallel sort

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

Currently, the range map function of parallel sort passes "null"

to get a comparator to sort the samples and produce the range map.

The comparator provider will enforce providing a type and prevent

passing "null". The range map function needs to be updated to get

the types of the sort fields and use them to get comparators.

- changed the output type of the local sampling function from

list of list of ANY to binary. The old type computer was removed.

- added null writer aggregate function that just produces null as

an aggregate value. this is needed in order to propagate the type

of the sort fields from the local step to the global step so that

the range map function can know the types of the sort fields.

Change-Id: I7edbb10906cc4464210af87a5b1630ba3aecbde0

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3333

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

  1. … 12 more files in changeset.
Merge commit '153b2ce970' from 'stabilization-f69489'

Change-Id: I5b0d2756d839b0bc2e7c59723306921e698a7630

  1. … 6 more files in changeset.
[NO ISSUE] Use RuntimeMXBean.getPid() when available

(reapplies 7cc982e5 which was inadvertantly reverted)

Change-Id: I0f4575affd9689f0044bac9959404ad76e133358

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3320

Reviewed-by: Ian Maxon <imaxon@uci.edu>

Integration-Tests: Michael Blow <mblow@apache.org>

Tested-by: Michael Blow <mblow@apache.org>

[NO ISSUE] Fix logging for ncservice-based tests

Change-Id: Id264aede0f62558ad6e34355047c623a1d594692

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3311

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ian Maxon <imaxon@uci.edu>

  1. … 21 more files in changeset.
[NO ISSUE] Don't generate trace log on CC

Change-Id: Ic6d7e55ef7dd4af16a77167a400b0c6ab4cb23cb

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3318

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Reviewed-by: Ian Maxon <imaxon@uci.edu>

[NO ISSUE] Use RuntimeMXBean.getPid() when available

Change-Id: Ic2371171dfc01423921c2e2d04fbc68a38978f2e

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3312

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

[NO ISSUE][RT] Fix Secondary Indexes Bulkload/Upsert

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Prevent null values from being upserted into

secondary indexes.

- Use secondary index tuple filter when bulkloading

to prevent null/missing values from being inserted

into indexes of optional fields.

Change-Id: I7b847aece06a4387cc41389b0566d0c0dc98487d

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3304

Reviewed-by: Luo Chen <cluo8@uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 10 more files in changeset.
[NO ISSUE] SonarQube fixes

Change-Id: I9f135fa10a0f78c967f2803bb2c3bce83e96a123

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3307

Integration-Tests: Michael Blow <mblow@apache.org>

Tested-by: Michael Blow <mblow@apache.org>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 1 more file in changeset.
Merge commit '963a544' from 'stabilization-f69489' to 'master'

Change-Id: Iedb57cd217d3115b94d13761d8779e81293003da

  1. … 3 more files in changeset.
[NO ISSUE] Fix serialization of surrogate pairs on non-UTF8 platforms

Change-Id: I17228e7dfdcd220e1c0caf5fcc8207fb8f3a7733

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3300

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

  1. … 2 more files in changeset.
[NO ISSUE] Bump versions to 0.[39].4.2-SNAPSHOT

Change-Id: If0cf05002c8019c1e1af43a18e75e617c0f7d4c9

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3299

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Tested-by: Michael Blow <mblow@apache.org>

    • -1
    • +1
    ./hyracks-data/hyracks-data-std/pom.xml
  1. … 87 more files in changeset.
[NO ISSUE] Prevent .* imports

Change-Id: I00309aa789d2e10ccc7bbe79cb2c0bdfd6e2925c

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3296

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

Tested-by: Michael Blow <mblow@apache.org>

  1. … 9 more files in changeset.
[NO ISSUE][NET] Do Not Track Incoming IPC Connections Handles

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Incoming IPC connections use a random port every time

they are established even if they are coming from the

same remote node. We should not track any incoming

connections handles as the IPCConnectionManager only

needs to keep track of out-going connections to avoid

establishing multiple out-going connections to the same

destination.

- Unregister IPC handles from IPCConnectionManager when

they are closed.

Change-Id: I8e2328c3baf4d208bc9f4de7ce6c8d49ac7d4a61

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3291

Reviewed-by: Michael Blow <mblow@apache.org>

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

[ASTERIXDB-2516][RT] Make asterix types comparators singleton and stateless

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

Avoid creating asterix types comparators when they could be made singleton.

Change-Id: I65972162ed2cf14d1f2654dbf00442aa35dd5c01

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3255

Reviewed-by: Michael Blow <mblow@apache.org>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 18 more files in changeset.
[NO ISSUE] Apply / enforce java import order

The process-sources target will now sort imports as well as

format source code; the source-format job will likewise verify

import order in addition to source code format

Change-Id: I55d976c4df10d9919c6a25683be2a3e3304e65d9

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3288

Integration-Tests: Michael Blow <mblow@apache.org>

Tested-by: Michael Blow <mblow@apache.org>

Reviewed-by: Till Westmann <tillw@apache.org>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

  1. … 611 more files in changeset.
[NO ISSUE][OTH] IApplication interface change to allow configuring loggers

- user model changes: no

- storage format changes: no

- interface changes: yes

Added configureLoggingLevel() as part of the IApplication to allow configuring

the application loggers.

Change-Id: Ib9724d9f12650f8e2c1f9d25c098028c75da7606

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3281

Reviewed-by: Till Westmann <tillw@apache.org>

Tested-by: Till Westmann <tillw@apache.org>

  1. … 2 more files in changeset.
[ASTERIXDB-2516][COMP] Avoid writing field names & values when comparing records

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

ARecordVisitablePointable writes field names, field tags, and field values

first before giving access to the record information. This is not ideal for

comparison. A different record accessor is needed for comparison. Also, the

field names should be sorted which ARecordVisitablePointable does not provide.

- avoid this writing when a pointable to the name & value can be obtained

(especially when the field value already includes the tag)

- use UTF8Pointable cached values (string length, meta length) to compare

instead of using the string comparator which would recalculate these values

- refactored some common code.

- removed not used methods in ARecordPointable

Change-Id: I19ac95a91749b2983bf06f763e463521a97a261c

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3280

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

  1. … 13 more files in changeset.
[NO ISSUE][OTH] Introduce NC CacheManager

- user model changes: no

- storage format changes: no

- interface changes: yes

Details:

- Introduce NC CacheManager that can be used to cache

values and reevaluate them based on cache policy.

- Add test case for CacheManager.

Change-Id: I8cba7753af11469e8a45cb1d59dfd70c17cd05f6

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3277

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

  1. … 2 more files in changeset.
[NO ISSUE][RT] Eliminate Excessive Object Creation in Sort Merge

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Currently, every binary search for frame lookup during

the sort merge phase results in auto-boxing to an Integer.

In addition, every call to InnerFrameInfo#compareTo during

the binary search results another auto-boxing. This change

eliminates these objects creation by using a MutableInt and

and using a primitive int comparison while comparing.

Change-Id: I72b498d476d242f85966caf67d2543978e5c88bb

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3276

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

[NO ISSUE][HYR] Log thread dump at shutdown at DEBUG level

Change-Id: I29b3e01caa5b8747163fb6b2dbcee268549455c6

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3275

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>