Clone Tools
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
White space adjustment

Adding apache license to generated protobuf files

    • -0
    • +23
    ./generated/SsccRegionProtos.java.nopb
    • -0
    • +23
    ./generated/SsccRegionProtos.java.pb
    • -0
    • +23
    ./generated/TrxRegionProtos.java.nopb
    • -0
    • +23
    ./generated/TrxRegionProtos.java.pb
Fix for [Trafodion-3244] Need to provide support for HDP 2.5.x and 2.6.x Various files needed to correctly build modified protos and remain backward compatible

    • -0
    • +424
    ./SplitBalanceHelper.java.tmpl
    • -0
    • +35657
    ./generated/SsccRegionProtos.java.nopb
    • -0
    • +35669
    ./generated/SsccRegionProtos.java.pb
    • -0
    • +52700
    ./generated/TrxRegionProtos.java.nopb
    • -0
    • +52716
    ./generated/TrxRegionProtos.java.pb
  1. … 77 more files in changeset.
refactor the code for review comments

  1. … 1 more file in changeset.
add license header

use coprocessor to set storage policy

    • -109
    • +1396
    ./generated/TrxRegionProtos.java
  1. … 4 more files in changeset.
[TRAFODION-2733] Provide an improved memory quota assignment for big memory operators (BMO)

Added SplitBanceHelper.java. This file was deleted inadvertently

Following changes are done in BMO memory quota

1) Enabled memory quota per node. The CQD BMO_MEMORY_LIMIT_PER_NODE

(renamed from EXE_MEMORY_LIMIT_PER_CPU) is set to 10240 MB by default.

Old attribute Old value Renamed Attribute New value

EXE_MEMORY_LIMIT_PER_CPU 0 BMO_MEMORY_LIMIT_PER_CPU 10240

EXE_MEMORY_LIMIT_LOWER_BOUND_HASHGROUPBY 10 BMO_MEMORY_LIMIT_LOWER_BOUND_HASHGROUPBY 25

EXE_MEMORY_LIMIT_LOWER_BOUND_HASHJOIN 10 BMO_MEMORY_LIMIT_LOWER_BOUND_HASHJOIN 25

EXE_MEMORY_LIMIT_LOWER_BOUND_PROBE_CACHE 10 BMO_MEMORY_LIMIT_LOWER_BOUND_PROBE_CACHE 25

EXE_MEMORY_LIMIT_LOWER_BOUND_SORT 10 BMO_MEMORY_LIMIT_LOWER_BOUND_SORT 200

2) Changes in EXPLAIN

Estimated memory per node for all BMOs at ROOT operator

Estimated memory per instance for every BMO operator

Memory quota per instance for every BMO operator

3) BMO TDB contains the memory quota per esp instance now.

4) Root TDB now contains the limit per node and estimated memory per node.

This can be used by WMS to change the memory allocation during

runtime without compilation. - Not yet implemented.

4) Added a CQD BMO_MEMORY_LIMIT_UPPER_BOUND to gap the memory

consumed by BMO by the same queries with less number of

bmos.

5) The unused memory quota is yielded to other fragments in the process

also.

6) Removed the code to limit the ESPs from being assigned to a fragement

based on the BMO memory quota.

7) Added a new CQD BMO_MEMORY_ESTIMATE_RATIO_CAP to gap the memory

estimate skew by any one BMO operator to 0.7.

8) To disable the memory quota per node, set BMO_MEMORY_LIMIT_PER_NODE to 0.

9) This memory quota is distributed proportionally based on the estimated memory

taking into consideration the number of bmo instances per operator and

the number of nodes available in the cluster to host these instances.

Hence, this memory quota should be valid in multi-fragments independent of the

number of fragments in an ESP.

10) Removed the CQD EXE_MEMORY_LIMIT_NONBMOS_PERCENT and EXE_MEMORY_RESERVED_FOR_MXOSRVR_IN_MB

11) Fixed BMO stats WM to be at least the allocated memory.

12) Changed the sort operator to account the bmo memory correctly.

(cherry picked from commit ba19c04a58890fdd845b03f8d915abdd487b6407)

Conflicts:

core/sqf/src/seatrans/hbase-trx/src/main/java/org/apache/hadoop/hbase/coprocessor/transactional/SplitBalanceHelper.java

core/sql/cli/Context.cpp

core/sql/executor/ex_frag_rt.cpp

core/sql/executor/ex_sort.cpp

core/sql/regress/executor/EXPECTED131

core/sql/regress/executor/EXPECTED140

core/sql/regress/hive/EXPECTED009

core/sql/regress/hive/EXPECTED030

core/sql/regress/hive/FILTER009

core/sql/regress/seabase/EXPECTED010

core/sql/regress/seabase/EXPECTED011

core/sql/regress/seabase/EXPECTED016

core/sql/sqlcomp/DefaultConstants.h

  1. … 44 more files in changeset.
Reworks, addressing Sean's and Selva's comments

  1. … 1 more file in changeset.
Put back Apache license in generated file

[TRAFODION-2617] Optionally push estimate row count logic into HBase coproc

    • -192
    • +1743
    ./generated/TrxRegionProtos.java
  1. … 9 more files in changeset.
[TRAFODION-2654] Change the location of trafodion-site.xml from $TRAF_HOME/etc to config [TRAFODION-2653] Sort operator loops at times

All HBase client connections from Trafodion uses Trafodion configuration

concept. Any client side property can now be set in trafodion-site.xml.

The scratch files are now open in blocking mode. Hence any scratch file

operation shouldn't return EAGAIN.

The internal table TDDL of table is now created without any table

coprocessors attached to it.

  1. … 23 more files in changeset.
[Trafodion-2608] Skips file flush in case of insert drop same transaction.

Manual merge fixes

  1. … 4 more files in changeset.
Merge fixes

  1. … 2 more files in changeset.
Added copyright to the generated protobufs

This is a large contribution of changes from Esgyn TransactionManager and libraries that are collectively much better tested and hardened than Trafodion, but are too numerous and complex to cherry pick individually.

The changes include:

Dozens of bug fixes

Improved recovery after HBase failures

Better error reporting and tracing, which includes region keys for easier debugging

Reduced UnknownTransactionExceptions resulting from duplicate transaction registration

Better TLOG management including reducing the default number of tables

    • -1578
    • +1859
    ./TrxRegionEndpoint.java.tmpl
    • -3016
    • +3812
    ./generated/TrxRegionProtos.java
  1. … 19 more files in changeset.
[TRAFODION-2448] Change TM SplitHelper to get configured parent znode instead of hard-coded /hbase

[TRAFODION-2150] and [TRAFODION-2151] TM long prepare and chore thread issues.

  1. … 1 more file in changeset.
[TRAFODION-2414] enhance 8616 commit conflict error message to include tx conflict details.

    • -0
    • +61
    ./CommitConflictException.java
  1. … 30 more files in changeset.
Fix for JIRA [TRAFODION-2305] After a region split the transactions to check against list is not fully populated

[TRAFODION-2236] Fix to handle bad TM ZK node entries during recovery This is the second fix for JIRA [TRAFODION-2236] [Author: Sean Broeder <sbroeder@edev06.esgyn.local>]

Squashed commit of the following:

commit 66030b519c49b4abe66b8b49b9ddc63a9543d8fc

Author: Sean Broeder <sbroeder@edev06.esgyn.local>

Date: Thu Sep 29 18:55:39 2016 +0000

Addressing review comments

commit c1b22cf41eeb56c3002a2ca86d2b204772667b9d

Merge: 6ee98ca a62e76f

Author: Sean Broeder <sbroeder@edev06.esgyn.local>

Date: Thu Sep 29 16:14:16 2016 +0000

Merge remote branch 'sbroeder_inctraf/sean_traf' into sean_traf

commit 6ee98cab2a8adf07df365131f431f1ded3e15d3a

Merge: 98a1ec6 b352dd8

Author: Sean Broeder <sbroeder@edev06.esgyn.local>

Date: Thu Sep 29 15:14:38 2016 +0000

Merge remote branch 'sbroeder_inctraf/sean_traf' into sean_traf

Conflicts:

core/sqf/src/seatrans/tm/hbasetmlib2/src/main/java/org/trafodion/dtm/HBaseTxClient.java

commit a62e76faf1753a02dd9095aa1061829b537da49f

Merge: 98a1ec6 b352dd8

Author: Sean Broeder <sbroeder@edev06.esgyn.local>

Date: Thu Sep 29 15:14:38 2016 +0000

Merge remote branch 'sbroeder_inctraf/sean_traf' into sean_traf

Conflicts:

core/sqf/src/seatrans/tm/hbasetmlib2/src/main/java/org/trafodion/dtm/HBaseTxClient.java

commit 98a1ec6bdf0eb9b69fbb6f4d8c908e657638ba54

Merge: d7232d3 fd4fadf

Author: Sean Broeder <sbroeder@edev06.esgyn.local>

Date: Wed Sep 28 23:20:57 2016 +0000

Fix to handle bad TM ZK node entries during recovery

commit b352dd8f1fdab8f73bf2f4dea3b0ac757dd6ad29

Merge: d7232d3 fd4fadf

Author: Sean Broeder <sbroeder@edev06.esgyn.local>

Date: Wed Sep 28 23:20:57 2016 +0000

Fix to handle bad TM ZK node entries during recovery

commit d7232d3f816129464a3449cea56a10919794ddd2

Merge: 0ccafa2 1e94882

Author: Sean Broeder <sbroeder@edev06.esgyn.local>

Date: Wed Sep 28 23:19:20 2016 +0000

Merge branch 'master' of github.com:apache/incubator-trafodion into sean_traf

commit fd4fadffab1f969dd1e3e735b01a9c62d9bb9766

Author: Sean Broeder <sean.broeder@esgyn.com>

Date: Tue Sep 27 10:22:02 2016 -0700

Update TransactionManager.java

commit 0ccafa240eb1ee225226c18c1396204830bea8ad

Author: Sean Broeder <sbroeder@edev06.esgyn.local>

Date: Tue Sep 27 03:13:23 2016 +0000

Fix to handle bad TM ZK node entries during recovery

This is the second fix for JIRA [TRAFODION-2236]

This closes #730

  1. … 4 more files in changeset.
Fix to handle bad TM ZK node entries during recovery This is the second fix for JIRA [TRAFODION-2236]

  1. … 4 more files in changeset.
Recovery changes for region transaction changes Fixes are part of JIRA [TRAFODION-1435]

  1. … 2 more files in changeset.
Adding expected file for test116 for JIRA [TRAFODION-1435]

  1. … 3 more files in changeset.
Merging HEAD and addressing review comments for JIRA [TRAFODION-1435]

  1. … 2 more files in changeset.
Second set of changes for JIRA [TRAFODION-1435] Region operations performed transactionally for delete, chackAndDelete Improved endpoint coprocessor exception logging

    • -45
    • +592
    ./SsccRegionEndpoint.java.tmpl
    • -139
    • +378
    ./TrxRegionEndpoint.java.tmpl
    • -459
    • +2330
    ./generated/SsccRegionProtos.java
    • -867
    • +2567
    ./generated/TrxRegionProtos.java
  1. … 7 more files in changeset.
Merge [TRAFODION-1435] PR-663 Initial region transaction changes (currently off by default)

  1. … 2 more files in changeset.
Update TrxRegionEndpoint.java.tmpl

Update TrxRegionEndpoint.java.tmpl