Trafodion

Clone Tools
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
fix lp 1392099 (cqd hive_max_varchar_length)

There are two contribuing factors.

1. A fully specified partition requirement (partition of 1) in

FileScan::synthHiveScanPhysicalProperty() is not recognized

and a modification of the ESPs renders the partition requirement

not satisfiable.

2. The reduction in CQD value for hive_max_varchar_length reduces

the dop from 5 to 1. In Exchange::createContextForChild(), a dop of

1 renders the parallel version of plan infeasible (see line near 2004

on test "rppForMe->getCountOfPipelines() > 1").

The fix is to address issue #1.

Added a safety check for ApproximatelyNPartitions and improvement

for hive sampling logic.

remove update to cg/TEST011

Change-Id: Idbf24be0b00b336dbac93afdddb6896ed0937e23

    • -17
    • +33
    /sql/optimizer/OptPhysRelExpr.cpp
LP and other fixes. Details below.

-- LP 1408504: Any sql operation done when trafodion is uninitialized or

needs to be upgraded will return error. Until now, some commands

(like get schemas, invoke) were not returning an error.

-- LP 1408506: metadata upgrade was not handling repository tables

and was failing. That has been fixed.

-- index related commands (create, populate) now run in multiple phases.

Metadata update within a xn and row population without a xn.

-- common methods have been added that can be called to begin and

end transactions.

-- HYBRID_QUERY_CACHE cqd is now off by default

Change-Id: I99ef2548998b1a6830d4332db09080df5bcfc1c1

    • -180
    • +151
    /sql/sqlcomp/CmpSeabaseDDLcommon.cpp
    • -45
    • +43
    /sql/sqlcomp/CmpSeabaseDDLindex.cpp
    • -6
    • +54
    /sql/sqlcomp/CmpSeabaseDDLrepos.cpp
    • -6
    • +37
    /sql/sqlcomp/CmpSeabaseDDLtable.cpp
    • -2
    • +61
    /sql/sqlcomp/CmpSeabaseDDLupgrade.cpp
    • -53
    • +11
    /sql/sqlcomp/CmpSeabaseDDLview.cpp
Skip running catman1/TEST138

Sometimes catman1/TEST138 fails with authorization errors on the

Hortonworks platform. There is no plan to fix this issue for release

9.1 so we are skipping the test. LP bug 1408155 has been created to

track this issue.

Change-Id: If7cfd67ed85be40a581de79a89a9d1dd1bc7c564

    • -2
    • +1
    /sql/regress/tools/runregr_catman1.ksh
Merge "Fixes for SQL security"

Fix for bug 1409128

Reworked the code to delete Compiler Contexts (CmpContext) from

the Trafodion SQL Engine CLI contexts (ContextCli). Now all the

CmpContexts created in a ContextCli are deallocated when it is deleted

as part of JDBC disconnect call.

Earlier, the code was deallocating the current CmpContext alone. However

the current CmpContext may not be from the current ContextCli. This was

causing the deallocated memory to be referenced again.

Change-Id: Ia957f17a8733e528495107b097d29bdc92d7bf95

Merge "Fix build errors for SQL compiler GUI"

Fixes for SQL security

LP bugs fixed:

1392805 – DB_ROOT incorrectly gets “NOT AUTHORIZED” messages

1398546 – revoke priv from role fails when view is present

1401233 – USAGE privilege not checked when creating procedure (and

revoking privileges)

1403995 – Update stats failures due to schema PUBLIC_ACCESS_SCHEMA

1401683 – (Partial) DDLoperations see error 8841 about transaction

started by SQL

Regressions updated:

catman1/TEST135 & EXPECTED138

catman1/EXPECTED138 (fix in common/ComUser.cpp)

Bug descriptions:

1392805:

Changed create view code to allow DB__ROOT to create views. Some

reorganization required to make sure create view sets the updatatable

and insertable privilege correctly. This also fixed the problem where

the incorrect privileges were set when created by DB__ROOT.

Sqlcomp/CmpSeabaseDDLview.cpp

Sqlcomp/PrivMgrPrivileges.h (sets default privileges)

1398546:

The check to see if the "select" privilege is still in existence needed

to be changed until after all the privilege descriptors were analyzed.

Sqlcomp/PrivMgrPrivileges.cpp (gatherViewPrivileges)

Sqlcomp/PrivMgrDesc.h

1401233:

Missing checks at create UDR and revoke USAGE privilege were added.

Sqlcomp/CmpSeabaseDDLroutine.cpp

Sqlcomp/PrivMgrMD (getUdrsThatReferenceLibraries)

Sqlcomp/PrivMgrPrivileges.cpp (dealWithUdrs)

1403995:

This is a critical case QA filed because the PUBLIC_ACCESS_SCHEMA does

not exist for temporary sample tables during Update Statistics. If the

PUBLIC_ACCESS_SCHEMA does not exist, the temporary sample table will be

created in the same schema as the source table. Also fixed an issue for

private schemas not owned by DB__ROOT to make the histogram table's

owner the current user.

ustat/hs_cli.cpp

ustat/hs_globals.cpp

1401683:

There are several 8841 issues being detected. This is a fix for one of

them related to Update Statistics where an embedded "get" statement

causes a transaction to be started in a child tdm_arkcmp process. The

fix is to not automatically start a transaction for the get request.

generator/GenRelScan.cpp

Change-Id: Ied42fdea6c6f8c43f29dab661b06b74f0f07ff99

    • -49
    • +368
    /sql/regress/catman1/EXPECTED135
    • -22
    • +22
    /sql/regress/catman1/EXPECTED138
    • -0
    • +33
    /sql/sqlcomp/CmpSeabaseDDLroutine.cpp
    • -61
    • +66
    /sql/sqlcomp/CmpSeabaseDDLview.cpp
    • -16
    • +90
    /sql/sqlcomp/PrivMgrPrivileges.cpp
Fixes for log file reader UDF

Following issues have been resolved

1) Multi-line error messages are now supported.

2) Default event log file name of master_exec_* is now supported

3) When an event log file name other the default is specified in the config

file, the log file reader UDF will now pick it up. The directory is still

expected to be $MY_SQROOT/logs.

Patch Set 2

Address comment issue found by Dave

Change-Id: Ia54543bb8a6f6f8122988424761b52416192794e

    • -42
    • +160
    /sql/sqludr/PredefUdrReadfile.cpp
Fix build errors for SQL compiler GUI

Recently, the compiler GUI started giving error messages during the build.

Note that this does not cause the overall build to fail.

Thanks to Howard for suggesting the fix, making sure that we use

/usr/bin/g++ to compile the GUI as we do for the rest of the code.

I tried using $(CC) instead of /usr/bin/g++ but that did not work.

Here are some of the error messages we saw, complaining about C++

syntax that was either outdated or relatively new:

In file included from ../export/NAStringDef.h:64,

from ../common/NAString.h:39,

from ../common/ComObjectName.h:55,

from ../common/CmpCommon.h:48,

from ../optimizer/ObjectNames.h:41,

from ../optimizer/ValueDesc.h:38,

from ../optimizer/GroupAttr.h:41,

from CommonSqlCmpDbg.h:34,

from MainWindow.h:24,

from MainWindow.cpp:18:

../export/FBString.h: In function ‘void folly::fbstring_detail::pod_fill(Pod*, Pod*, T)’:

../export/FBString.h:133: error: ISO C++ forbids declaration of ‘ee’ with no type

auto const ee = b + ((e – b) & ~7u);

In file included from ../common/ComSecurityKey.h:24,

from ../optimizer/NATable.h:43,

from ../optimizer/ColStatDesc.h:42,

from ../optimizer/EstLogProp.h:49,

from ../optimizer/GroupAttr.h:42,

from CommonSqlCmpDbg.h:34,

from MainWindow.h:24,

from MainWindow.cpp:18:

../sqlcomp/PrivMgrDefs.h: At global scope:

../sqlcomp/PrivMgrDefs.h:61: error: expected identifier before ‘class’

enum class PrivAuthClass {

Change-Id: I5e220b031af988077271179a969bc0724e2ee584

    • -0
    • +1
    /sql/SqlCompilerDebugger/SqlCmpDbg.pro
Use LITERAL for region state and print out in-doubt trnasactions as needed

Change-Id: Ib6c8f8530a153c7bf162b0069f0356a992e28d0d

Merge "Fix Bug #1366987 T2 Version no. and T4 name"

Second part of change to disable mxosrvr repos stats

After first checkin it was discovered that change was not affected in

dcs-default.xml resource file.

Closes bug: #1409091

Change-Id: Ib52049087db243939e405d74c30ea3a72cac6b9d

Merge "Modified sqstart script to invoke sqcheck script after dcsstart"

Fix for install_local_hadoop bug# 1409189

Added a conditional check to look for existence to trafci/bin

folder

Change-Id: Ie2a90f9df30e9dad335a233ed040594bc23a6f33

    • -3
    • +10
    /sqf/sql/scripts/install_local_hadoop
Modified sqstart script to invoke sqcheck script after dcsstart

Fixed Bug#1408470 - sqstart - sqcheck must be run after dcsstart

This also address the '0' display that was seen for MXOSRVRs

when 'sqcheck' script executed

Change-Id: I0d6709f792124aac087569b5dbecf134c8da2622

Merge "Disable mxosrvr port map Closes-Bug: #1408454"

Merge "Fix for bug 1408187 missing commas in SQL event log files"

Disable workload statistics publishing Closes-Bug: #1409091

Disable the publishing of workload statistics in mxosrvr.

This is set in the following property:

dcs.server.user.program.statistics.enabled

Change-Id: I7d6227dbe04f50aa670ceb0a21aaf95dcd4454be

Disable mxosrvr port map Closes-Bug: #1408454

In order to improve mxosrvr start time performance 2 new properties were

added:

dcs.server.user.program.port.map.timeout.seconds = 60

dcs.server.user.program.port.bind.timeout.seconds = 30

They are passed to mxosrvr at start time. The DcsMaster would create a

port map in the servers/registered znode so mxosrvrs could identify

their DcsServer instance/child id and discover the TCP/IP port to use.

During performance testing, it was found that because DCS/MXOSRVRs usage

of ports in the ephemeral range DCS couldn't guarantee dedicated use of

a given range of ports. Setting the 2 properties to -1 disables port

mapping and returns to original herding style port identification

dcs.server.user.program.port.map.timeout.seconds = -1

dcs.server.user.program.port.bind.timeout.seconds = -1

Change-Id: I0497bfbc58878378d67a8cba9293708e88dc271e

Merge "Fix LP1408148 and LP1408485"

Reduced number of rows used in batch tests.

Reduced from to 6000 from 60000 to avoid tests timing out.

Also changed TOTALUPDATECOUNT to not be hardcoded.

Change-Id: I61741b9127b9041124995addade8b6df41d34150

Fix for bug 1408187 missing commas in SQL event log files

The QRLogger::log method did not provide values for SQLCode and Query ID

that the event log reader TMUDF expects. In some cases, the calls to

QRLogger::log provided these values in the text, or they provided

two commas to indicate NULL values, but there are hundreds of other

calls to QRLogger::log not doing either of these. Added a second

QRLogger::log method and handled SQLCode and QueryID logging inside

those methods, so only a few calls had to be changed.

Change-Id: I9d147398df3056ed5c8b04be20ceebd3f42fe006

    • -1
    • +1
    /sql/SqlCompilerDebugger/SqlCmpDbg.pro
    • -19
    • +10
    /sql/sqlmxevents/logmxevent_traf.cpp
Merge "Fix for LP bug 1404951"

Merge "Resubmitting split delay work, additional checks"

Fix for LP bug 1404951

This is an intermittent problem that appears on the build machine,

caused by Change-Id I5b570c42712d4c38157181c3b76bf9a3ab6e2ed9. In this

delivery we are increasing the number of rows fetched by hbase scan call

to two rows, up from the previous value of one row. This is only for the

scan call used to determine mdam probe keys.

Change-Id: I12f20084bf53c188000db24ed8a698b3fdc7f41b

Resubmitting split delay work, additional checks

Split delay portion of LP bug 1378544.

Added a check/put mechanism for putting objects into

shared map. Depends on which coprocessor is started

first according to what was entered in the HBase

configuration under 'hbase.coprocessor.region.classes'

Change-Id: I7deeb91b7c608b7d4ae902590e6aa753e01530b9

Cleaning of the TrxTransactionState structures moved to chore thread

Change-Id: Ib46ae1171bd5632c477d3a8be78b2c8d7c94fbc1

Fix LP1408148 and LP1408485

Change-Id: I1577f0e8b3d016b45599add5007672d227cd969e

    • -986
    • +1226
    /sql/regress/compGeneral/EXPECTED042
    • -5
    • +164
    /sql/regress/compGeneral/TEST042
Merge "Fix TEST042 failure."

Seamonster preliminary changes.

Change-Id: I57e110e51ebc46cd83ed330bf40be9a2ee005644