Trafodion

Clone Tools
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
fix cardinality errors for better DoP for OE

Problems fixed are as follows.

1. TableAnalysis::getLocalPredsOnPrefixOfList() prematurely breaks

out of the loop to collect predicates on key columns.

2. TableDesc::getSaltColumnAsSet() does not return the result in

the form of base columns valueIds.

3. When the dop is estimated lower than the low bound of ESPs, the

low bound value is used.

4. SimpleFileScanOptimizer::computeSingleSubsetSize() incorrectly

applies the computed predicates on salted column which reduces the

subset size by CQD HIST_NO_STATS_UEC (default to 2). A subset size

specifies the # of rows processed.

Change-Id: I85bebf0d31d8f3e2db5bfe7c00cb4467b53308ac

    • -2
    • +8
    /sql/optimizer/SimpleScanOptimizer.cpp
    • -189
    • +188
    /sql/regress/seabase/EXPECTED010
Merge "fixed an upgrade bug"

Merge "check jdbcT2.jar/jdbcT4.jar."

fixed an upgrade bug

Change-Id: Iff000c23425a252e7393dd67edaff534ffb203c3

    • -5
    • +13
    /sql/sqlcomp/CmpSeabaseDDLupgrade.cpp
Merge "Fix for bug 1392090"

support fully qualified domain name in hostnames.

Change-Id: I2bc291cb1bac6a5e355661cfbfc2a1ddadff5576

Fix for bug 1392090

Earlier JNI objects in Trafodion were made thread independent. HBaseClient_JNI

and HIveClient_JNI objects are the creator/user of other Trafodion JNI objects.These objects couldn't be used in a different thread because the

thread specific JNIEnv was not setup. This change initializes the JNIENV when

it is not initialized in the thread.

Change-Id: I827002a495c3373b66fb2c88f80675d9da188193

check jdbcT2.jar/jdbcT4.jar.

Change-Id: I4570c70fa707241f7d650c052a0140e4c150240a

Merge "mxosrvr exit when heapsize become large, code changed after last review"

Initial changes for ORC file support.

Access to ORC (optimized row columnar) format tables is not enabled by

default yet. This checkin is initial and infrastructure changes for

that support.

Change-Id: I683c1b63c502dd4d2c736181952cb40f9f299cfd

    • -83
    • +148
    /sql/comexe/ComTdbHdfsScan.cpp
  1. … 39 more files in changeset.
Enabling runtime stats for hbase tables and operators

This is the second set of changes to collect the runtime stats info for

hbase tables and operators. It contains:

1) Stats for hbase IUD operations

2) Moved incActualRowsReturned() call to

ExHbaseAccessTcb::moveRowToUpQueue()

3) Added Hbase call counter

4) Display full hbase operator names instead of generic

"EX_HBASE_ACCESS" for hbase operator runtime stats

Change-Id: I94d727c897876a429b588f9acb3fec465dd56fe5

    • -128
    • +120
    /sql/executor/ExHbaseIUD.cpp
    • -15
    • +92
    /sql/executor/HBaseClient_JNI.cpp
Enable RMS monitoring

The query execution statistics can now be obtained while the query is running

via

a) GET STATISTICS FOR QID <qid>

b) Using STATISTICS table-valued function

c) Using CLI calls – SQL_EXEC_GetStatistics2 and SQL_EXEC_GetStatisticsItems

The statistics can be viewed in accumulated, pertable, progress and operator

stats format.

You can also obtain statistics for each fragment instance and process wise

statistics for each operator of an active query.

Light-weight SQL Offender (LSO) feature is also enabled. With LSO, you can get

1. Queries offending CPU resource

2. Queries blocked in Trafodion Engine

3. Queries that are not active

4. Queries that haven’t finished but blocked in client

5. Queries offending memory resource

Explain in RMS feature can be enabled by setting the

cqd explain_in_rms ‘on’ before compiling a query.

EXPLAIN.. FOR QID <qid> command is now supported.

GET PROCESS STATISTICS FOR <pid> is also supported

Change-Id: I59eb54dd4fd347a37e0b7e7842ee242e553ef60d

    • -0
    • +283
    /sqf/export/limited-support-tools/LSO/README
    • -0
    • +202
    /sqf/export/limited-support-tools/LSO/offender
  1. … 34 more files in changeset.
Enable Phoenix T2 tests to run with Maven

Enable Phoenix T2 tests to be run with Maven. This will standarize

the framework used by the T2 and the T4 Phoenix tests.

Part of blueprint infra-add-t2-phoenix-tests

Change-Id: I69efee7033f43c8b3033b5aba2cb5399c7c0b9db

Uninstall command line parsing fix

Since a shift command occurs later in the loop, the --all option does

not need to to shift arguments. This caused a parsing error in the case

when the --all option is followed by other options.

Also removed quotes around a command line that caused shell to try to

find the entire command line as a single file name.

Change-Id: I0498c3aa7aa3dec4e4deca4d2466849feee77a6e

Merge "Index-join scan trimming heuristics rework II"

Use HBase estimated row count if no stats exist

In the absence of statistics, the row count for an HBase table

is estimated using information in its HFiles. Previouly, this

estimate was calculated and assigned to the NATable's

originalCardinality_ member variable, but this did not result

in it being used by the optimizer (the correct row count did

not appear in the results of an Explain). The code to calculate

the estimate is now invoked from HSHbaseTableDef::getRowCount(),

which has the effect of it being used as intended.

Change-Id: I3794e8368082b6f17e47fe686ec9a004aed27976

Closes-Bug: #1388918

Merge "Adding version information for hbase-trx"

Merge "Prevent deferring control points indefinitely"

Adding version information for hbase-trx

Version information is added to the Manifest

file for the hbase-trx JAR files, the 1.6, 1.7, and

hdp2_1. The version information is obtained from

the SCMBuildJava.sh script run during Make

Change-Id: Ic7ff4ad508535b0c0c016eb3d8d5f18b5c979d1b

Warning if using Cloudera Parcels and cores bug

traf_setup: Cloudera parcels are not currently supported by install or

Trafodion core. Because of this an error has been added that will not

allow the user to run trafodion_setup if they have installed via

parcels. This is a temporary fix until parcels are supported.

traf_add_user: Trafodion only supports a max of 15 cores and if more

than 15 cores are detected then the range is changed to 0-15 in the

sqconfig file.

trafodion_setup: Installer verion now included. Hard coded temporarily

until better solution is found.

traf_cloudera_uninstall: Removing yum cache and rebuilding rpm database

during cloudera uninstall.

Change-Id: Ifaae9c346b5bfa06f2d682a70e3dd7d22a26d169

    • -0
    • +5
    /installer/tools/traf_cloudera_uninstall
Prevent deferring control points indefinitely

Change-Id: I733e36bf31eedd85c871699b82c023ef1fcf643e

Various changes done to upgrade metadata from v23 to v30.

-- initialize trafodion, upgrade; will upgrade metadata from v23 to v30.

-- Following columns were added/dropped/modified in metadata tables:

-- COLUMNS: added sql_data_type

-- OBJECTS: added DROPPABLE column to indicated droppable table.

added schema_owner column.

-- TABLES: added num_salt_partns, table_format, comment, key_length,

row_data_length, row_total_length.

Removed hbase_create_options and moved it to TEXT.

-- SEQ_GEN: redef_ts and upd_ts added.

-- TEXT: text_type, sub_id added.

-- TABLE_CONSTRAINTS: added IS_ENFORCED, DROPPABLE, DISABLED,

VALIDATED, LAST_VALIDATED_TIME

-- reserved flags column added to metadata tables, initialize to zero.

-- default_value in COLUMNS is now UTF8 instead of UCS2

-- new columns populated during upgrade for existing objects.

-- values that moved from one metadata table to another populated during upgrade.

-- new columns populated during create operation for new objects.

-- step added to drop/create metadata views as part of upgrade

Change-Id: I173c33cbe7ee9314c66b84d16a6ef8b2d7394335

    • -175
    • +20
    /sql/generator/GenExpGenerator.cpp
    • -192
    • +257
    /sql/sqlcomp/CmpSeabaseDDLcommon.cpp
  1. … 6 more files in changeset.
Set up security keys while constructing NATable

This change corrects a problem in NATable by setting up the

secKeySet_ ComSecurityKeySet while the NATable is constructed.

In this way, the NATable::resetAfterStatement method will not

flag it as a leak. This change also includes a new regression

test for REVOKE and query invalidation.

Change-Id: Ie2ac5a2380007a53ab13c95f154e02ad1bde191e

Closes-Bug: #1371199

    • -0
    • +302
    /sql/regress/catman1/EXPECTED131
    • -0
    • +270
    /sql/regress/catman1/TEST131
    • -2
    • +2
    /sql/regress/tools/runregr_catman1.ksh
Bug 1390644

Change-Id: I87a3473ad7923cd82a8af0878fe73196686ff395

Merge "Txn path perf improvements based on code review by the DTM team"

Merge "Set authorization enabled/Sequence generator privs"

    • -107
    • +186
    /sql/sqlcomp/CmpSeabaseDDLcommon.cpp
    • -4
    • +71
    /sql/sqlcomp/CmpSeabaseDDLindex.cpp
    • -27
    • +109
    /sql/sqlcomp/CmpSeabaseDDLtable.cpp
Txn path perf improvements based on code review by the DTM team

Change-Id: I9e32f9e9ed5c5ac207edf93d3c510ef97e0f61f3

Merge "Adding README.txt for compiling tm protocol buffers"

Merge "Fixes bug 1383877"

Adding README.txt for compiling tm protocol buffers

Change-Id: I422c1888460084eb400adaadfe83ce5ad5b72d1c

    • -0
    • +14
    /sqf/src/seatrans/hbase-trx/src/main/protobuf/README.txt