seatrans

Clone Tools
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
Additional fix for [Trafodion-3244] update Makefile

White space adjustment

Added Apache license to new file copy_proto

Adding apache license to generated protobuf files

Fix for [Trafodion-3244] Need to provide support for HDP 2.5.x and 2.6.x Various files needed to correctly build modified protos and remain backward compatible

    • -0
    • +11
    ./hbase-trx/copy_proto
    • -0
    • +27
    ./hbase-trx/hbase-protocol_pb/README.txt
  1. … 73 more files in changeset.
add copy jars

move target ALL to the header

Not necessary for package before install for maven

Ignore *.log in hbase-trx

optimize hbase-trx Makefile

optimize hbasetlib2 Makefile

Do not make again before MAKE CLEAN

  1. … 8 more files in changeset.
TRAFODION-3165 HDFS storage option not work well if table use namespace

Added blank line to test commit permission

TRAFODION-3111 CentOS 7 build error

  1. … 2 more files in changeset.
refactor the retry logic

refactor the code for review comments

add retry logic

TRAFODION-3076 update pom.xml.hdp

[TRAFODION-2821] Trafodion core code base needs to be thread safe

TMLIB JNI interaction was not thread safe causing T2 driver application

to dump core at times.

Cleaned up TMLIB JNI interaction and added lock mechanism such that JNI

methods are initialized by one thread in a thread safe manner.

    • -2
    • +1
    ./tm/hbasetmlib2/javaobjectinterfacetm.cpp
    • -0
    • +1
    ./tm/hbasetmlib2/javaobjectinterfacetm.h
  1. … 3 more files in changeset.
add license header

use coprocessor to set storage policy

    • -0
    • +13
    ./hbase-trx/src/main/protobuf/TrxRegion.proto
  1. … 2 more files in changeset.
Update pom.xml.cdh54

fix a typo caused by a testing leftover, just detected after the first commit.

[TRAFODION-3054] update pom.xml.cdh for hbase-trx to build protobuf

fix the bufoverrun Critical error checked by TScanCode

  1. … 1 more file in changeset.
[TRAFODION-2733] Provide an improved memory quota assignment for big memory operators (BMO)

Added SplitBanceHelper.java. This file was deleted inadvertently

[TRAFODION-2733] Provide an improved memory quota assignment for big memory operators (BMO)

Enabled memory quota per node. The CQD BMO_MEMORY_LIMIT_PER_NODE

(renamed from EXE_MEMORY_LIMIT_PER_CPU) is set to 10240 MB by default.

Old attribute Old value Renamed Attribute New value

EXE_MEMORY_LIMIT_PER_CPU 0 BMO_MEMORY_LIMIT_PER_NODE 10240

EXE_MEMORY_LIMIT_LOWER_BOUND_HASHGROUPBY 10 BMO_MEMORY_LIMIT_LOWER_BOUND_HASHGROUPBY 25

EXE_MEMORY_LIMIT_LOWER_BOUND_HASHJOIN 10 BMO_MEMORY_LIMIT_LOWER_BOUND_HASHJOIN 25

EXE_MEMORY_LIMIT_LOWER_BOUND_PROBE_CACHE 10 BMO_MEMORY_LIMIT_LOWER_BOUND_PROBE_CACHE 25

EXE_MEMORY_LIMIT_LOWER_BOUND_SORT 10 BMO_MEMORY_LIMIT_LOWER_BOUND_SORT 200

Changes in EXPLAIN

Estimated memory per node for all BMOs at ROOT operator

Estimated memory per instance for every BMO operator

Memory quota per instance for every BMO operator

BMO TDB contains the memory quota per esp instance now.

Root TDB now contains the limit per node and estimated memory per node.

This can be used by WMS to change the memory allocation during

runtime without compilation. - Not yet implemented.

Added a CQD BMO_MEMORY_LIMIT_UPPER_BOUND to gap the memory

consumed by BMO by the same queries with less number of

bmos.

The unused memory quota is yielded to other fragments in the process

also.

Removed the code to limit the ESPs from being assigned to a fragement

based on the BMO memory quota.

Added a new CQD BMO_MEMORY_ESTIMATE_RATIO_CAP to gap the memory

estimate skew by any one BMO operator to 0.7.

To disable the memory quota per node, set BMO_MEMORY_LIMIT_PER_NODE to 0.

This memory quota is distributed proportionally based on the estimated memory

taking into consideration the number of bmo instances per operator and

the number of nodes available in the cluster to host these instances.

Hence, this memory quota should be valid in multi-fragments independent of the

number of fragments in an ESP.

Removed the CQD EXE_MEMORY_LIMIT_NONBMOS_PERCENT and EXE_MEMORY_RESERVED_FOR_MXOSRVR_IN_MB

Fixed BMO stats WM to be at least the allocated memory.

Changed the sort operator to account the bmo memory correctly.

  1. … 7 more files in changeset.
Following changes are done in BMO memory quota

1) Enabled memory quota per node. The CQD BMO_MEMORY_LIMIT_PER_NODE

(renamed from EXE_MEMORY_LIMIT_PER_CPU) is set to 10240 MB by default.

Old attribute Old value Renamed Attribute New value

EXE_MEMORY_LIMIT_PER_CPU 0 BMO_MEMORY_LIMIT_PER_CPU 10240

EXE_MEMORY_LIMIT_LOWER_BOUND_HASHGROUPBY 10 BMO_MEMORY_LIMIT_LOWER_BOUND_HASHGROUPBY 25

EXE_MEMORY_LIMIT_LOWER_BOUND_HASHJOIN 10 BMO_MEMORY_LIMIT_LOWER_BOUND_HASHJOIN 25

EXE_MEMORY_LIMIT_LOWER_BOUND_PROBE_CACHE 10 BMO_MEMORY_LIMIT_LOWER_BOUND_PROBE_CACHE 25

EXE_MEMORY_LIMIT_LOWER_BOUND_SORT 10 BMO_MEMORY_LIMIT_LOWER_BOUND_SORT 200

2) Changes in EXPLAIN

Estimated memory per node for all BMOs at ROOT operator

Estimated memory per instance for every BMO operator

Memory quota per instance for every BMO operator

3) BMO TDB contains the memory quota per esp instance now.

4) Root TDB now contains the limit per node and estimated memory per node.

This can be used by WMS to change the memory allocation during

runtime without compilation. - Not yet implemented.

4) Added a CQD BMO_MEMORY_LIMIT_UPPER_BOUND to gap the memory

consumed by BMO by the same queries with less number of

bmos.

5) The unused memory quota is yielded to other fragments in the process

also.

6) Removed the code to limit the ESPs from being assigned to a fragement

based on the BMO memory quota.

7) Added a new CQD BMO_MEMORY_ESTIMATE_RATIO_CAP to gap the memory

estimate skew by any one BMO operator to 0.7.

8) To disable the memory quota per node, set BMO_MEMORY_LIMIT_PER_NODE to 0.

9) This memory quota is distributed proportionally based on the estimated memory

taking into consideration the number of bmo instances per operator and

the number of nodes available in the cluster to host these instances.

Hence, this memory quota should be valid in multi-fragments independent of the

number of fragments in an ESP.

10) Removed the CQD EXE_MEMORY_LIMIT_NONBMOS_PERCENT and EXE_MEMORY_RESERVED_FOR_MXOSRVR_IN_MB

11) Fixed BMO stats WM to be at least the allocated memory.

12) Changed the sort operator to account the bmo memory correctly.

(cherry picked from commit ba19c04a58890fdd845b03f8d915abdd487b6407)

Conflicts:

core/sqf/src/seatrans/hbase-trx/src/main/java/org/apache/hadoop/hbase/coprocessor/transactional/SplitBalanceHelper.java

core/sql/cli/Context.cpp

core/sql/executor/ex_frag_rt.cpp

core/sql/executor/ex_sort.cpp

core/sql/regress/executor/EXPECTED131

core/sql/regress/executor/EXPECTED140

core/sql/regress/hive/EXPECTED009

core/sql/regress/hive/EXPECTED030

core/sql/regress/hive/FILTER009

core/sql/regress/seabase/EXPECTED010

core/sql/regress/seabase/EXPECTED011

core/sql/regress/seabase/EXPECTED016

core/sql/sqlcomp/DefaultConstants.h

  1. … 43 more files in changeset.
Reworks, addressing Sean's and Selva's comments

  1. … 1 more file in changeset.
Put back Apache license in generated file