Clone Tools
  • last updated 21 mins ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
[ASTERIXDB-2634][FUN] String functions to return null on data/type errors

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- substring(string, int, int)

- codepoint-to-string([int])

- string-concat([string])

Change-Id: I17905ad9e767b43822d28a46123bc1390a159cc7

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/4083

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Hussain Towaileb <hussainht@gmail.com>

    • -0
    • +3
    ./nontagged/serde/AOrderedListSerializerDeserializer.java
  1. … 19 more files in changeset.
[ASTERIXDB-2613][*DB] Fix serialization of schemaless records

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

When serializing schemaless records, the record serializer should

continue serializing the fields as schemaless fields to handle fields

that are also records (nested records).

Change-Id: Ibc7257987d24905e82d5e81f27840fba81281706

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3501

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

    • -41
    • +16
    ./nontagged/serde/ARecordSerializerDeserializer.java
  1. … 3 more files in changeset.
[ASTERIXDB-2610][RT][*DB] Improve deep comparison runtime

- user model changes: no

- storage format changes: no

- interface changes: yes

Details:

Avoid data copies during comparison (no visitable pointables).

Allow comparison of non-tagged values to avoid copying the tag

when accessing an array element or record field.

- added TaggedValueReference.

- changed ILogicalBinaryComparator to use TaggedValueReference

instead of IPointable.

- removed numberOfItems() from ListAccessorUtil and used

AOrderedListSerializerDeserializer.getNumberOfItems().

- removed storage & IPointable pools from AbstractAGenericBinaryComparator

& LogicalComplexBinaryComparator and used TaggedValueReference pool.

- removed compareNumbers() from ComparatorUtil since IPointable's

have been replaced with TaggedValueReference.

- adapted record comparison logic in LogicalComplexBinaryComparator

to match the logic in the physical comparator.

Change-Id: Id9ece93c704f566d7bdb7fd17b1ba92713c917d5

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3466

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

Reviewed-by: Till Westmann <tillw@apache.org>

    • -4
    • +3
    ./common/ILogicalBinaryComparator.java
    • -0
    • +57
    ./common/TaggedValueReference.java
    • -85
    • +73
    ./nontagged/comparators/AbstractAGenericBinaryComparator.java
    • -156
    • +96
    ./nontagged/comparators/LogicalComplexBinaryComparator.java
    • -19
    • +10
    ./nontagged/comparators/LogicalGenericBinaryComparator.java
    • -17
    • +16
    ./nontagged/comparators/LogicalScalarBinaryComparator.java
    • -1
    • +2
    ./nontagged/hash/AMurmurHash3BinaryHashFunctionFamily.java
    • -20
    • +4
    ./nontagged/serde/AOrderedListSerializerDeserializer.java
    • -19
    • +4
    ./nontagged/serde/AUnorderedListSerializerDeserializer.java
    • -0
    • +31
    ./nontagged/serde/SerializerDeserializerUtil.java
  1. … 6 more files in changeset.
[ASTERIXDB-2575][RT][FUN] Remove usages of ComparisonHelper

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

Remove usages of ComparisonHelper and switch to logical or physical

comparators.

- updated RecordReplace test cases that used to throw an exception

when comparing complex types.

- cleaned up interval classes.

- fixed getStartOffset() in AIntervalPointable.

- removed ComparisonHelper

Change-Id: Ib2a05f35c752e1d2ecfdbabbf5d303ea73c7c9db

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3470

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

    • -15
    • +10
    ./nontagged/serde/AIntervalSerializerDeserializer.java
  1. … 12 more files in changeset.
[ASTERIXDB-2574][COMP] Fix min/max functions

- user model changes: no

- storage format changes: no

- interface changes: no

This change is mainly for 2 things. The first thing is to not throw

an exception when the type of the aggregated field is invalid for min/max

(e.g. record or rectange) or min/max get incompatible data like string

and int. The result in this case would be NULL. The second thing is to

enable comparing ARRAYs correctly by using logical comparison. When

a partition runs into type invalidity, it will output NULL. The global

aggregator interprets NULL received from a partition as type invalidity

and outputs NULL as the final result. Both SQL and SQL++ will do that.

Special treatment is needed for scalar and distinct version of SQL since

SQL min/max ignores NULL values and continue aggregation and the scalar

and distinct version of SQL are normally setup as a global aggregator

since they behave like the global aggregator in a two-step aggregation.

Currently, there is only a local min and max functions. The other

min/max functions are used for everything, the global function of

two-step aggregation, and for scalar and distinct min/max. In order to

differentiate, a global min/max functions are added that will be used

for the two-step aggregation.

Details:

- fixed listify to open up elements when adding them to the collection

and the collection item type is of type ANY and changed the type inferer

of listify to enable that.

- fixed AbstractCollectionType to make sure itemType is never null.

- changed MinMaxAggTypeComputer to not throw an exception but return

NULL for invalid types.

- changed min/max descriptors to implement inferer to propagate the

type of the field and pass that when getting a comparator.

- switched min/max comparison to the logical comparison.

- refactored method inequalityUndefined to be shared by logical comparison

and min/max functions.

- added global max/min functions to enable differentiating between

scalar min/max, distinct min/max and two-step min/max (global & local).

- code clean-up for LogicalScalarBinaryComparator; created two INSTANCES

and re-used.

Change-Id: I1231cfe558099d167bae0b2fa7fc4879b756baf0

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3427

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

    • -0
    • +21
    ./common/ILogicalBinaryComparator.java
  1. … 147 more files in changeset.
[ASTERIXDB-2458][COMP] Fix InjectTypeCastForFunctionArgumentsRule

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

InjectTypeCastForFunctionArgumentsRule is for functions that

can potentially return any of their arguments. switch and

if_null(expr1, expr2, ...) are examples. All the arguments

need to be casted (opened) to the type that the function

will return which is the generalized type of all arguments.

Some functions like if_null can determine the exact expression

they will return, e.g. if_null(1, {"id": 3}) in which case

the return type is always integer. The rule tries to cast

th 2nd argument, the record, to integer and fails. In such

cases, these functions do not need to cast their arguments.

If the function determines its output type to be ANY, then

all arguments need to be casted (opened). If the function

determines its output to be a dervied type, then casting is

also needed since that output type should be the generalized

type of all arguments.

Change-Id: I2fee234d883b59319e4ec4df58d61ecd498373fd

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3406

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

  1. … 4 more files in changeset.
[ASTERIXDB-2567][RT] Fix serializing nullable record fields

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

When serializing a record, a field should be checked if it's nullable.

The value to be serialized should be checked if it's null or value to

serialize it correctly. Exceptions could happen when the record is in

an array if record fields are not checked.

Change-Id: I42105965e69c60bc4b123337296476b4d75f0ad8

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3398

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -9
    • +19
    ./nontagged/serde/ARecordSerializerDeserializer.java
  1. … 17 more files in changeset.
[ASTERIXDB-2555][RT][COMP] Make hash join use logical comparison

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

This patch changes the hash join operator to use the join condition

to evaluate if tuples are equal when joining. Binary physical comparators

have been removed. The join condition evaluator is in TuplePairEvaluator.

- extraced TuplePairEvaluatorFactory out of nested loop join class

into a separate class so that it is shared among nested loop and

hash join.

- switched from FrameTuplePairComparator to ITuplePairComparator in

in OptimizedHybridHashJoin and InMemoryHashJoin.

- moved debugging code from OptimizedHybridHashJoin into a separate

class, JoinUtil.

- temporarily made the logical comparison of multisets use raw binary

comparison instead of returning null until the logic is implemented.

- made IBinaryBooleanInspector a functional interface and updated

the implementations.

- updated record and array test cases to reflect the new

behaviour of hash join where logical comparison could produce null.

Also, updated sorting, group by and distinct test cases since

the input data has been modified.

- added two new input files arrays1nulls.adm & arrays2nulls.adm

to be used by the open dataset. previous arrays1.adm & arrays2.adm

are used by the closed dataset since it cannot accept arrays with

null values.

Change-Id: If1834967fdd913fdc76003f09636b2450d07cd5e

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3387

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

  1. … 48 more files in changeset.
[ASTERIXDB-2554][HYR] Add UTF8 and byte array comparator factories

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

Add comparator factories for UTF8StringPointable, UTF8StringLowercasePointable,

UTF8StringLowercaseTokenPointable and ByteArrayPointable instead of using

PointableBinaryComparatorFactory, a wrapping factory that will create a factory

each time (which also creates a comparator each time).

Change-Id: Ied6a29210a3dc1ba9fd553fb0a67ff4340e4571f

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3355

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Ali Alsuliman <ali.al.solaiman@gmail.com>

  1. … 45 more files in changeset.
[ASTERIXDB-2516][RT] Prevent passing null type to comparator provider

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

This change is to disallow passing null as IAType to the comparator

provider when asking for a comparator. It also includes few fixes

for issues appearing as a result of this change.

- changed the generic comparator to check whether the tag of data

at runtime is a valid tag and throw an exception if not.

- modified the comparator provider to return non-tagged comparator

for IAType SHORTWITHOUTINFOTYPE which is a short without tag.

SHORTWITHOUTINFOTYPE should not use the generic comparator since

the input data has no tag.

- fixed Dataset class to consider external dataset when getting

the IAType of the primary keys. The primary keys for external

datasets are different from regular datasets. They are not

part of the record type. Previously, null would be returned.

This would cause a failure when getting a comparator for the

primary keys of an external dataset since the type passed is null.

- fixed the TypeTraitProvider to give the correct lengths for

duration, day_time_duration, and year_month_duration.

Change-Id: I37767a3f3d1e3b29597d2a4998c0b60005cadb09

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3336

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

    • -24
    • +17
    ./nontagged/comparators/AbstractAGenericBinaryComparator.java
    • -16
    • +13
    ./nontagged/comparators/AbstractAGenericBinaryComparatorFactory.java
    • -10
    • +4
    ./nontagged/hash/AMurmurHash3BinaryHashFunctionFamily.java
  1. … 11 more files in changeset.
[ASTERIXDB-2516][RT] Move primitive comparators to Hyracks and make singleton

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- moved 2 comparators, boolean and long comparators from asterix to hyracks.

- added byte, short, integer, float and double comparator

factories to Hyracks to replace PointableBinaryComparatorFactory.

- removed checking lengths of 0 from PointableBinaryComparatorFactory.

- changed tests to use the primitive factories.

Change-Id: If15dc4e0dd0db942a4cadb15abbe56cbfe617b48

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3294

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

    • -34
    • +16
    ./nontagged/comparators/AbstractAGenericBinaryComparator.java
    • -59
    • +0
    ./nontagged/comparators/BooleanBinaryComparatorFactory.java
    • -34
    • +21
    ./nontagged/comparators/LogicalScalarBinaryComparator.java
  1. … 46 more files in changeset.
[NO ISSUE] SonarQube fixes

Change-Id: I9f135fa10a0f78c967f2803bb2c3bce83e96a123

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3307

Integration-Tests: Michael Blow <mblow@apache.org>

Tested-by: Michael Blow <mblow@apache.org>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

    • -11
    • +22
    ./nontagged/printers/PrintTools.java
  1. … 4 more files in changeset.
[NO ISSUE] Fix serialization of surrogate pairs on non-UTF8 platforms

Change-Id: I17228e7dfdcd220e1c0caf5fcc8207fb8f3a7733

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3300

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

  1. … 2 more files in changeset.
[ASTERIXDB-2516][RT] Make asterix types comparators singleton and stateless

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

Avoid creating asterix types comparators when they could be made singleton.

Change-Id: I65972162ed2cf14d1f2654dbf00442aa35dd5c01

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3255

Reviewed-by: Michael Blow <mblow@apache.org>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -45
    • +28
    ./nontagged/comparators/ACirclePartialBinaryComparatorFactory.java
    • -23
    • +15
    ./nontagged/comparators/ADurationPartialBinaryComparatorFactory.java
    • -23
    • +18
    ./nontagged/comparators/AIntervalAscPartialBinaryComparatorFactory.java
    • -24
    • +19
    ./nontagged/comparators/AIntervalDescPartialBinaryComparatorFactory.java
    • -55
    • +29
    ./nontagged/comparators/ALinePartialBinaryComparatorFactory.java
    • -42
    • +23
    ./nontagged/comparators/APoint3DPartialBinaryComparatorFactory.java
    • -27
    • +18
    ./nontagged/comparators/APointPartialBinaryComparatorFactory.java
    • -48
    • +30
    ./nontagged/comparators/APolygonPartialBinaryComparatorFactory.java
    • -26
    • +24
    ./nontagged/comparators/ARectanglePartialBinaryComparatorFactory.java
    • -14
    • +11
    ./nontagged/comparators/AUUIDPartialBinaryComparatorFactory.java
    • -13
    • +7
    ./nontagged/comparators/BooleanBinaryComparatorFactory.java
  1. … 9 more files in changeset.
[NO ISSUE] Apply / enforce java import order

The process-sources target will now sort imports as well as

format source code; the source-format job will likewise verify

import order in addition to source code format

Change-Id: I55d976c4df10d9919c6a25683be2a3e3304e65d9

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3288

Integration-Tests: Michael Blow <mblow@apache.org>

Tested-by: Michael Blow <mblow@apache.org>

Reviewed-by: Till Westmann <tillw@apache.org>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -2
    • +1
    ./common/MergeAggregationExpressionFactory.java
    • -2
    • +0
    ./nontagged/serde/ARecordSerializerDeserializer.java
    • -1
    • +1
    ./nontagged/serde/AStringSerializerDeserializer.java
  1. … 622 more files in changeset.
[ASTERIXDB-2516][COMP] Avoid writing field names & values when comparing records

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

ARecordVisitablePointable writes field names, field tags, and field values

first before giving access to the record information. This is not ideal for

comparison. A different record accessor is needed for comparison. Also, the

field names should be sorted which ARecordVisitablePointable does not provide.

- avoid this writing when a pointable to the name & value can be obtained

(especially when the field value already includes the tag)

- use UTF8Pointable cached values (string length, meta length) to compare

instead of using the string comparator which would recalculate these values

- refactored some common code.

- removed not used methods in ARecordPointable

Change-Id: I19ac95a91749b2983bf06f763e463521a97a261c

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3280

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

    • -91
    • +42
    ./nontagged/comparators/AbstractAGenericBinaryComparator.java
    • -47
    • +25
    ./nontagged/hash/AMurmurHash3BinaryHashFunctionFamily.java
  1. … 11 more files in changeset.
[ASTERIXDB-2516][COMP] Avoid writing into buffer when comparing numbers

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

Avoid writing into buffer when comparing and promoting between numbers.

- made seed the initial hash for arrays and records.

- renamed & refactored LogicalComparatorUtil to share code between logical

and physical comparators

- minor code clean-ups

Change-Id: Ie089d386a9ab8271f2833c05ffdfb0d484937b51

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3272

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

Reviewed-by: Till Westmann <tillw@apache.org>

    • -175
    • +32
    ./nontagged/comparators/AbstractAGenericBinaryComparator.java
    • -0
    • +217
    ./nontagged/comparators/ComparatorUtil.java
    • -14
    • +11
    ./nontagged/comparators/LogicalScalarBinaryComparator.java
    • -6
    • +3
    ./nontagged/hash/AMurmurHash3BinaryHashFunctionFamily.java
  1. … 2 more files in changeset.
[ASTERIXDB-2516][COMP] Change logical comparator interface

- user model changes: no

- storage format changes: no

- interface changes: yes

Details:

Use IPointable instead of bytes, offset, and length.

Change-Id: I8150ccb4b18550fd8fba10ffca9c88d45ebc625b

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3263

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

    • -4
    • +4
    ./common/ILogicalBinaryComparator.java
    • -46
    • +35
    ./nontagged/comparators/LogicalComplexBinaryComparator.java
    • -14
    • +15
    ./nontagged/comparators/LogicalGenericBinaryComparator.java
    • -16
    • +17
    ./nontagged/comparators/LogicalScalarBinaryComparator.java
  1. … 1 more file in changeset.
[ASTERIXDB-2516][COMP] Change logical comparators handling of null/missing

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

Change the result of comparing against null and missing.

For arrays, comparing against null or missing item will result in incomparable.

For records, comparing against a null field will result in incomparable.

- modified test cases accordingly

- moved few object factories to the common place.

- modified compareRecords to avoid string construction for field names

- few code clean-ups.

Change-Id: Id93bea76e13658768e08a98fd373c71a901ceec5

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3259

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

    • -103
    • +34
    ./nontagged/comparators/LogicalComplexBinaryComparator.java
    • -2
    • +6
    ./nontagged/hash/AMurmurHash3BinaryHashFunctionFamily.java
  1. … 6 more files in changeset.
[ASTERIXDB-2523][RT][COMP] add support for hashing record fields

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

Add support for hashing record fields.

- Added hashing records in GenericHashFunction.

- Extracted common functionality from comparators and hash functions into

CompareHashUtil.

- Added test cases.

Change-Id: I9d18a17089afdc6e28ff01091d890b375868969c

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3249

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

    • -0
    • +93
    ./nontagged/CompareHashUtil.java
    • -64
    • +8
    ./nontagged/comparators/AbstractAGenericBinaryComparator.java
    • -2
    • +57
    ./nontagged/hash/AMurmurHash3BinaryHashFunctionFamily.java
  1. … 80 more files in changeset.
[ASTERIXDB-2523][RT][COMP] add support for hashing array fields

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

Add support for hashing array fields.

- Modified AMurmurHash3BinaryHashFunctionFamily and extracted the hashing function

into a private named hashing function "GenericHashFunction". Added hashing arrays.

- Modified hash join to include generating hash functions for the right branch

since now hash functions are type-dependent and cannot use the same hash functions

generated for the left branch.

- Added test cases.

Change-Id: Ibd0dc7f270730140226f54445705822049f5c863

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3241

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -22
    • +9
    ./nontagged/comparators/AbstractAGenericBinaryComparator.java
    • -60
    • +109
    ./nontagged/hash/AMurmurHash3BinaryHashFunctionFamily.java
  1. … 53 more files in changeset.
[ASTERIXDB-2516][RT] record deep comparison (ordering)

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

Add support for physical record deep comparison.

- Added test cases

Change-Id: I186f853e0b16acdb3170f7b51bb8a5707d34f9d8

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3234

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

Reviewed-by: Till Westmann <tillw@apache.org>

    • -8
    • +153
    ./nontagged/comparators/AbstractAGenericBinaryComparator.java
  1. … 20 more files in changeset.
[ASTERIXDB-2516][RT] array deep comparison (ordering)

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

Add support for physical array deep comparison.

- Refactored and renamed AObjectAscBinaryComparatorFactory/AObjectDescBinaryComparatorFactory

- Refactored AGenericDescBinaryComparator and AGenericAscBinaryComparator

- Added test cases

Change-Id: Ib369a4dd22f32bfca4bd2c0cbf987529c0bafa6d

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3233

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

    • -288
    • +3
    ./nontagged/comparators/AGenericAscBinaryComparator.java
    • -0
    • +51
    ./nontagged/comparators/AGenericAscBinaryComparatorFactory.java
    • -0
    • +51
    ./nontagged/comparators/AGenericDescBinaryComparatorFactory.java
    • -86
    • +0
    ./nontagged/comparators/AObjectAscBinaryComparatorFactory.java
    • -45
    • +0
    ./nontagged/comparators/AObjectDescBinaryComparatorFactory.java
    • -0
    • +381
    ./nontagged/comparators/AbstractAGenericBinaryComparator.java
    • -0
    • +64
    ./nontagged/comparators/AbstractAGenericBinaryComparatorFactory.java
  1. … 18 more files in changeset.
[ASTERIXDB-2516][RT] prepare physical comparators for deep comparison

- user model changes: no

- storage format changes: no

- interface changes: yes

Details:

This change is to make physical comparators type-aware in order to do

deep comparison of complex types like arrays and records. The IAType

is propagated to the comparators.

- added new methods in IBinaryComparatorFactoryProvider to accept the

type of left and right inputs for operations like hash join where

the join key types come from different dataset sources.

- defaulted some arrays functions to use the old comparator behaviour temporarily

until complex comparison is implemented

- modified AObjectAscBinaryComparatorFactory & AObjectDescBinaryComparatorFactory to

create a comparator with IAType information. Changed the serialization/deserialization

of their instances to take care of the newly added fields since they are not

present in old instances.

Change-Id: I02011e7151398d5f5f9ba9c1e1db6518484b9fe5

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3229

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

Reviewed-by: Till Westmann <tillw@apache.org>

    • -0
    • +319
    ./nontagged/comparators/AGenericAscBinaryComparator.java
    • -0
    • +47
    ./nontagged/comparators/AGenericDescBinaryComparator.java
    • -295
    • +41
    ./nontagged/comparators/AObjectAscBinaryComparatorFactory.java
    • -36
    • +8
    ./nontagged/comparators/AObjectDescBinaryComparatorFactory.java
  1. … 21 more files in changeset.
[ASTERIXDB-2516][RT] add support for record deep comparison

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

Add support for record deep comparison.

- modified LogicalComplexBinaryComparator to allow record comparison

- added test cases for record comparison

- modified PointableAllocator to allow freeing record visitable pointables

Change-Id: I3e8bfbb014b86295749e980d123b0d3edf079beb

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3222

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

    • -19
    • +172
    ./nontagged/comparators/LogicalComplexBinaryComparator.java
  1. … 39 more files in changeset.
[ASTERIXDB-2516][RT] add support for array deep comparison

- user model changes: no

- storage format changes: no

- interface changes: yes

Details:

Add support for array deep comparison.

- modified LogicalComplexBinaryComparator.java to allow array comparison

- added test cases for array comparison

- changed interface IObjectPool.java to allow freeing a specific object

Change-Id: I2fef48d7c6189362f44786b8d89d89c5f91d4b10

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3173

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Michael Blow <mblow@apache.org>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

Reviewed-by: Till Westmann <tillw@apache.org>

    • -1
    • +2
    ./common/ILogicalBinaryComparator.java
    • -0
    • +90
    ./common/ListAccessorUtil.java
    • -16
    • +178
    ./nontagged/comparators/LogicalComplexBinaryComparator.java
  1. … 47 more files in changeset.
[NO ISSUE] Cleanup / refactor upgrade code

Change-Id: Ic81e87e70eecf49b71f9d96b1ac7c7180a314564

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3174

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

  1. … 6 more files in changeset.
[ASTERIXDB-2516][RT] add support for deep comparison 1

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

Add support for deep comparison.

- add interface ILogicalBinaryComparator

- switch EQ,LT,GT and other functions to use ILogicalBinaryComparator

instead of ComparisonHelper

Change-Id: I623662861e6f3b1fdcff78f8edc0e3216ca10fe1

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3145

Reviewed-by: Till Westmann <tillw@apache.org>

Reviewed-by: Dmitry Lychagin <dmitry.lychagin@couchbase.com>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Contrib: Michael Blow <mblow@apache.org>

Integration-Tests: Michael Blow <mblow@apache.org>

Tested-by: Michael Blow <mblow@apache.org>

    • -0
    • +46
    ./common/ILogicalBinaryComparator.java
    • -0
    • +218
    ./nontagged/comparators/LogicalComparatorUtil.java
    • -0
    • +97
    ./nontagged/comparators/LogicalComplexBinaryComparator.java
    • -0
    • +79
    ./nontagged/comparators/LogicalGenericBinaryComparator.java
    • -0
    • +233
    ./nontagged/comparators/LogicalScalarBinaryComparator.java
  1. … 28 more files in changeset.
[ASTERIXDB-2507][RT] Objects Creation in ARecordSerializerDeserializer

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Reuse UTF8 hash function and comparator from

evaluators when extracting a field's offset.

Change-Id: Icdb47db9e909e49d65ebf1cd667e27e2467e4085

Reviewed-on: https://asterix-gerrit.ics.uci.edu/3109

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -12
    • +5
    ./nontagged/serde/ARecordSerializerDeserializer.java
  1. … 3 more files in changeset.
[ASTERIXDB-2447][COMP] Parameter rewriting for if_missing, if_null functions

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Introduce cast operation for parameters of if_missing, if_null functions

(as performed for parameters of the switch_case function)

- Fix union type handling in TypeResolverUtil.generalizeTypes()

Change-Id: I768d8236f5b0ccb9a850304ffedd3686d911702d

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2935

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

  1. … 7 more files in changeset.