Clone Tools
  • last updated 28 mins ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
[ASTERIXDB-2686][RT] Delete run files eagerly

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

Delete run files eagerly.

Change-Id: Idc780b44b0554bf36e2378700aae1b38b8e57353

Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/4403

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

    • -2
    • +1
    ./ExternalGroupWriteOperatorNodePushable.java
  1. … 9 more files in changeset.
[NO ISSUE][OTH] Cleanup Redundant Type Cast

- user model changes: no

- storage format changes: no

- interface changes: no

Change-Id: I9c93ce90ad4aa789ae46284431e172ae4777bf55

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2887

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

Reviewed-by: Till Westmann <tillw@apache.org>

    • -1
    • +1
    ./ExternalGroupWriteOperatorNodePushable.java
  1. … 30 more files in changeset.
[NO ISSUE] Incremental cleanup of deprecated exception ctors

Change-Id: I1e7c3655828fc6530cef83ea502a6cfbf41acddf

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2533

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

  1. … 240 more files in changeset.
[ASTERIXDB-2256] Reformat sources using code format template

Change-Id: I4faa141c1a8c9700d5e9ac50b839acc9d1eede73

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2310

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Murtadha Hubail <mhubail@apache.org>

    • -2
    • +2
    ./ExternalGroupBuildOperatorNodePushable.java
    • -2
    • +2
    ./ExternalGroupOperatorDescriptor.java
    • -2
    • +2
    ./ExternalGroupWriteOperatorNodePushable.java
  1. … 981 more files in changeset.
[ASTERIXDB-1564][CONF] Consistently Use Log4j2

- user model changes: no

- storage format changes: no

- interface changes: no

Details:

- Replace java.util.logging by Log4j2.

- Excluded classes due to their tests:

- IoUtil

- Tracer

Change-Id: Ic137571292f45de1f1994c61d328b97185012197

Reviewed-on: https://asterix-gerrit.ics.uci.edu/2226

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Michael Blow <mblow@apache.org>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

    • -5
    • +5
    ./ExternalGroupBuildOperatorNodePushable.java
    • -5
    • +5
    ./ExternalGroupWriteOperatorNodePushable.java
  1. … 389 more files in changeset.
Separate index build from index access

This change separates index build from index access.

All indexes now have a single dataflow helper which

uses the index path to locate the resource on the nc

to read the resource from memory or disk.

Existing resource metadata and dataflow helpers were

combined into resource builders eliminating lots of

duplicated code.

Change-Id: Ie4ea3aaa63dff8d246fa43ca7c7359729bc8cf47

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1728

Integration-Tests: Ian Maxon <imaxon@apache.org>

Tested-by: Ian Maxon <imaxon@apache.org>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

    • -1
    • +1
    ./ExternalGroupBuildOperatorNodePushable.java
    • -2
    • +2
    ./ExternalGroupOperatorDescriptor.java
    • -1
    • +1
    ./ExternalGroupWriteOperatorNodePushable.java
  1. … 569 more files in changeset.
ASTERIXDB-1892: Sets a proper hash table cardinality during hash-group by

- Set a proper hash table cardinality during the merge phase

of the external hash group-by operator.

- Currently, the number of tuples in a spilled partition is

used as the hash table cardinality. And this can cause an issue

since compiler.groupmemory size is not considered.

- So, like the initial group-by build phase, the hash table

cardinality will be set properly based on the memory budget for

the group-by operator.

Change-Id: I651139b2b559ad4d2f6137a5c844814606516a90

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1702

Sonar-Qube: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

    • -0
    • +48
    ./ExternalGroupOperatorDescriptor.java
    • -1
    • +7
    ./ExternalGroupWriteOperatorNodePushable.java
  1. … 5 more files in changeset.
ASTERIXDB-1791, ASTERIXDB-1796: fix failure handling in runtime operators.

This change includes the following parts:

- Fix the implementation of fail() and close() in several runtime operators

to avoid file handle leak and job hang;

- Add an erase method to RunFileWriter which closes files before deleting

them in order release the holding disk space;

- Call RunFileWriter.close() and RunFileReader.close() in "finally" blocks.

- Fix RunFileReader to not truncate files to be deleted - it is not the root

cause of un-released disk space - open deleted files are the root cuase;

- Check file handle leaks in LangExecutionUtil.tearDown().

Change-Id: I203168171e6dac16b57d2eda960823e3810e22a3

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1513

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Till Westmann <tillw@apache.org>

    • -3
    • +2
    ./ExternalGroupBuildOperatorNodePushable.java
    • -3
    • +8
    ./ExternalGroupWriteOperatorNodePushable.java
  1. … 51 more files in changeset.
ASTERIXDB-1556, ASTERIXDB-1733: Hash Group By and Hash Join conform to the memory budget

- External Hash Group By and Hash Join now conform to the memory budget (compiler.groupmemory and compiler.joinmemory)

- For Optimzed Hybrid Hash Join, we calculate the expected hash table size when the build phase is done and

try to spill one or more partitions if the freespace can't afford the hash table size.

- For External Hash Group By, the number of hash entries (hash table size) is calculated based on

an estimation of the aggregated tuple size and possible hash values for the given field size in that tuple.

- Garbage Collection feature has been added to SerializableHashTable. For external hash group-by,

whenever we spill a data partition to the disk, we also check the ratio of garbage in the hash table.

If it's greater than the given threshold, we conduct a GC on Hash Table.

Change-Id: I2b323e9a2141b4c1dd1652a360d2d9354d3bc3f5

Reviewed-on: https://asterix-gerrit.ics.uci.edu/1056

Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

BAD: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Reviewed-by: Yingyi Bu <buyingyi@gmail.com>

    • -1
    • +1
    ./ExternalGroupBuildOperatorNodePushable.java
    • -3
    • +4
    ./ExternalGroupOperatorDescriptor.java
  1. … 40 more files in changeset.
Move Hyracks to subfolder

    • -0
    • +141
    ./ExternalGroupBuildOperatorNodePushable.java
    • -0
    • +153
    ./ExternalGroupOperatorDescriptor.java
    • -0
    • +73
    ./ExternalGroupState.java
    • -0
    • +167
    ./ExternalGroupWriteOperatorNodePushable.java
    • -0
    • +97
    ./ExternalHashGroupBy.java
    • -0
    • +27
    ./IRunFileWriterGenerator.java
  1. … 4422 more files in changeset.