hs_cli.cpp

Clone Tools
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
Misleading error when Hivestats can't be created

When Update Statistics is performed on a Hive table, the

trafodion.hivestats schema is created if it doesn't already

exist. If the user issuing the ustat statement does not

have the create_schema component privilege, this will fail, but

the code as it existed ignored the error, assuming it was

a "schema already exists" error, which is expected in most

cases. If the schema does not exist, an error is then given

when an attempt is made to create the histograms table in

the non-existing schema.

This has been changed to check the error when the schema

creation attempt fails, so the correct error is generated.

This is a partial fix, because a user with manage_statistics

privilege should not be prevented from executing Update

Statistics just because the trafodion.hivestats schema did

not already exist. The likely full solution would be to

create the schema automatically at Trafodion initialization.

Change-Id: I26561100bab9f6a59f3f5d0d56ce9d2aab380fa3

Partial-Bug: #1425748

  1. … 4 more files in changeset.
Errors reported for Update Stats on Hive tables

Attempts to execute Update Statistics statements on Hive

tables fail with error 1002 (catalog does not exist). The

catalog used to store Hive stats was no longer being

successfully created on demand. The Trafodion catalog is

now used instead, with a schema for Hive stats created

in it on first use. The tables for histograms and

histogram intervals for Hive tables are created in that

schema on first use.

Change-Id: Ib57d0af4a3da6f52f0544d6c2fce3e77d2e823c1

Closes-Bug: #1320397

  1. … 3 more files in changeset.
Fix for LP bugs 1404442 & 1401683

These are duplicate problems.

A QA test suite (arkcase/arkt1112.log) contains some update statistics

tests which are starting a transaction but the transaction is never

ended.

Since the rogue transaction is started in a secondary arkcmp, the master

arkcmp is unaware of this transaction.

What differentiates this from other tests is that QA test turns off

autocommit before performing a series of update statistics.

The fix is to encapsulate a begin/commit work around DDL operations

executed by update statistics.

Change-Id: Ibbed2133e5a79f7393c9ec1b25c884a7dd8b0c3e

Fixes for SQL security

LP bugs fixed:

1392805 – DB_ROOT incorrectly gets “NOT AUTHORIZED” messages

1398546 – revoke priv from role fails when view is present

1401233 – USAGE privilege not checked when creating procedure (and

revoking privileges)

1403995 – Update stats failures due to schema PUBLIC_ACCESS_SCHEMA

1401683 – (Partial) DDLoperations see error 8841 about transaction

started by SQL

Regressions updated:

catman1/TEST135 & EXPECTED138

catman1/EXPECTED138 (fix in common/ComUser.cpp)

Bug descriptions:

1392805:

Changed create view code to allow DB__ROOT to create views. Some

reorganization required to make sure create view sets the updatatable

and insertable privilege correctly. This also fixed the problem where

the incorrect privileges were set when created by DB__ROOT.

Sqlcomp/CmpSeabaseDDLview.cpp

Sqlcomp/PrivMgrPrivileges.h (sets default privileges)

1398546:

The check to see if the "select" privilege is still in existence needed

to be changed until after all the privilege descriptors were analyzed.

Sqlcomp/PrivMgrPrivileges.cpp (gatherViewPrivileges)

Sqlcomp/PrivMgrDesc.h

1401233:

Missing checks at create UDR and revoke USAGE privilege were added.

Sqlcomp/CmpSeabaseDDLroutine.cpp

Sqlcomp/PrivMgrMD (getUdrsThatReferenceLibraries)

Sqlcomp/PrivMgrPrivileges.cpp (dealWithUdrs)

1403995:

This is a critical case QA filed because the PUBLIC_ACCESS_SCHEMA does

not exist for temporary sample tables during Update Statistics. If the

PUBLIC_ACCESS_SCHEMA does not exist, the temporary sample table will be

created in the same schema as the source table. Also fixed an issue for

private schemas not owned by DB__ROOT to make the histogram table's

owner the current user.

ustat/hs_cli.cpp

ustat/hs_globals.cpp

1401683:

There are several 8841 issues being detected. This is a fix for one of

them related to Update Statistics where an embedded "get" statement

causes a transaction to be started in a child tdm_arkcmp process. The

fix is to not automatically start a transaction for the get request.

generator/GenRelScan.cpp

Change-Id: Ied42fdea6c6f8c43f29dab661b06b74f0f07ff99

  1. … 14 more files in changeset.
Fixed typos in user-visible messages and sample code

Fixed spellings and minor grammar in strings used in

in output, and sample code supplied to users.

One possibly harmful misspelling was found in source code,

file sqf/monitor/linux/montim.cxx used "Contianer_ExitProcess"

where it should have used "Container_ExitProcess".

Five files which could be changed for this reason are being held back

because they are also in open change 875. They will be checked in after

that change is merged to the main branch:

conn/odbc/src/odbc/dependencies/linux/smxoevl.h

conn/trafci/src/org/trafodion/ci/InterfaceQuery.java

sqf/src/sp_proxy/sp_common.cpp

sqf/src/tm/tmlibmsg.h

sqf/src/tm/tmtimer.cpp

In patch set 2, modified these files as notde by reviewer:

conn/jdbc_type2/samples/CreateDataSource.java

sqf/monitor/test/montestutil.cxx

sql/cli/Statement.h

sql/optimizer/RelExpr.h

sql/optimizer/RelScan.h

sql/optimizer/ScmCostMethod.cpp

Change-Id: I3761e9e1518ab39415806bba0f4ed93d14e1d41c

  1. … 59 more files in changeset.
Automated collection of necessary statistics

When the optimizer requests a histogram for a given column, and that

histogram does not exist, it may (depending on the cqds in effect) register

a request for the histogram to be created at a later time, or utilize a

small sample to generate a rudimentary histogram on the fly. In either

case, when a subsequent Update Statistics statements specifies the ON

NECESSARY COLUMNS clause, any column of the target table that has been

the subject of one of these actions will have a bona fide histogram

created.

blueprint ustat-automation

Change-Id: Ieceac3e1d84bf8091a5bf340dc1739a447ad2436

  1. … 6 more files in changeset.
remove WITH DIVISION clause

Change-Id: I986fe56a8559f88f2781adf700933d5b03c9e697

Authorization checks for DDL & utilities

Fixed issues from code comments.

LOAD/UNLOAD authorization checks:

Code was added during code generation to make sure user has privileges,

if the user had necessary privileges, then the EXEUTIL parser flag is

turned on to avoid further privilege checks. When load/unload

completes, the parser flag is reset.

Update/showstats Statistics authorization checks:

Added a new error message

Changed hs_globals to support a new isAuthorized method and store

parser flags when class is instantiated and reset them when done

Changed hs_cli.cpp to use new IF NOT EXISTS syntax when creating

histogram tables, make owner of histogram tables DB__ROOT

(will need to adjust when schema privileges happen), and clean up

CreateHistTables method to remove old authorization mechanism

Changed hs_update.cpp which controls the update and showstats operation

to add authorization checks

Purgedata and populate index changes:

Changed CmpSeabaseDDLcommon.cpp to check privileges for purgedata

Changed CmpSeabaseDDLindex.cpp to check privileges for popindex

Additional component privileges and checks:

Added support for new component privileges in PrivMgrMD.h/.cpp

Added support for MANAGE_COMPONENTS

Added support for CREATE_INDEX and DROP_INDEX component privs

Fixes from last delivery that were postponed:

Context.cpp - fix for previous code review

CmpSeabaseDDLtable - added calls to deallocEHI

PrivMgrMD - fixed wording in a comment

Miscellaneous changes:

ComUser - added new convenience method - isRootUserID()

NATable.cpp (setupPrivInfo) to always set up privInfo_ and to call

the embedded compiler while extracting privileges

Privilege adjustments to take advantage of privInfo stored in NATable:

Added code to mark and rewind errors in diags.

Fix for LP bug 1392895

Change-Id: I6f7245ae7e66086769c0e92d901399c99e8f2af3

  1. … 33 more files in changeset.
Launchpad fixes, details follow.

-- index descriptor was not setting index row length causing opt assertion failure.

LP 1351476

-- purgedata on index supported. LP 1354108

-- An update stmt that was causing failure if index maintenance was being used.

LP 1353116

-- new clause ‘not serialized’ has been added to force columns to not be serialized even

if serialization is set to ON as the default.

Used by metadata table definitions as its columns are not serialized.

Change-Id: I3b10616823fd36af199e742c4ccd58d8656c5eba

  1. … 15 more files in changeset.
enable embedded compiler support for fetch histogram queries

The fetch histogram logic executes queries against the user metadata

tables HISTOGAMS and HISTORAGM_INTERVALS that require the spawning of

external arkcmp processes. The work in this check-in enables embedded

compiler for these queries so the spawning of external arkcmps is avoided.

The change introduces a "switch context" to a meta data context when

compiling the user metadata queries and "switch back" to the user context

prior to fetching the rows from the cursors on the user metadata tables.

The main change is implemented in the "readHistogram" function in

hs_read.cpp file.

A new method (switchBackContext) was added to class CmpContext class

to enable switching back to a previous context. The purpose of the new

method is to ensure that thread globals are appropriately reset when

reverting back to a previous context.

The checkin also turns OFF CQD HBASE_SERIALIZATION in file regress/tools/sbdefs

as the hbase serialization feature is not enabled yet. The feature (CQD)

was enabled during regression testing prior to this checkin.

Change-Id: Iee1b8152dee3fea0ce3d3354719b5b7aa8fb198f

  1. … 12 more files in changeset.
Update stats performance fix

Two issues fixed to address the update stats slow running issue.

1. Sample table is not created if source table is salted

(partitioned) which will lead to failure when

construct the histogram for the salt column

2. Cardinality hints are not inserted in 3 locations (serial plan).

When update stats is executed on a table with no stats, cardinality

hints are neeeded to improve the plans for update stats internal

queries on the table.

3. Correctly display the DOP in the explain output of update stats

internal queries in the update stats log

Change-Id: I49edd281cd426ea210b01b4cd6a2287a9e6d0198

  1. … 1 more file in changeset.
Various launchpad fixes.

-- upshift datatype in default clause is now handled correctly

-- ctas creation was returning error if constraints were used. Fixed by not starting a transaction for the root query.

-- compiler crash fixed in parser due to a null ptr reference for a ctas stmt

-- hist stats tables are correctly dropped from schema during a drop schema stmt

-- unique constraint is now created for a table with column level primary key clause

-- sometimes tables with constraints were not dropped correctly during drop schema. That has been fixed.

-- get indexes in schema is now supported

Change-Id: Ic05649a9f279dc4660d1dd33e8960142bb7b3430

  1. … 10 more files in changeset.
Initial code drop of Trafodion

  1. … 4886 more files in changeset.