Statement.cpp

Clone Tools
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
Migrate from log4cpp to log4cxx

This change is a wholesale removal of log4cpp from source tree.

log4cxx is an external library installed via RPM, or user build

to default /usr/lib64 and /usr/include directories. Some of the

QRLogger and CommonLogger code was changed to use the new log4cxx

APIs.

Change-Id: I248bac0a8ffbfea6cbc1ba847867b30638892eae

  1. … 208 more files in changeset.
Move core into subdir to combine repos

  1. … 10768 more files in changeset.
Move core into subdir to combine repos

  1. … 10622 more files in changeset.
Move core into subdir to combine repos

Use: git log --follow -- <file>

to view file history thru renames.

  1. … 10837 more files in changeset.
Remove some dead code

Remove dead code concerned with constraint and schema labels.

This is an anachronism from pre-open-source versions of the code.

Most of the code removed is in the compiler, with a small amount

of cli and executor code removed.

Change-Id: Ic8a833bb15d1ca9a0e2e2683f2d4644b44c4f96b

  1. … 13 more files in changeset.
explain enhancements and fixes.

-- support to return explain details from packed explain plan.

select * from table(explain(null, 'EXPLAIN_PLAN=<packed-plan>'))

This enables caller to retrieve the packed explain plan, ship

it to another process and then format it there.

DSM will be using this functionality.

-- sqlci syntax added to test explain enhancement functionality:

get qid for statement s;

store explain for s in repository;

set qid <qid> for s;

-- new tests added to seabase/TEST011

-- some bug fixes to handle 4 byte lengths for

explain plan greater than short max.

-- changed err enums CLI_DESC_NOT_EXSISTS and

CLI_STMT_NOT_EXSISTS to the right EXISTS spelling

(this is just for you, Dave).

-- added missing copyrights

Change-Id: Ic60758fe49790516be125cca7f7e23fe1265feb7

  1. … 32 more files in changeset.
Fix child/parent query IDs for update stats

Failure to setup child/parent query IDs for update stats and

other non-embedded compiler scenarios caused problems in cancel

including frequest but intermittent failure of executor/TEST106.

This change fixes the problem.

Change-Id: I0d6b80aeddf9906f08d3dab09f9775414ea7bfc3

Closes-Bug: #1404963

  1. … 10 more files in changeset.
Support cancel for DDL, update stats, and utils

This change propagates parent query ID to queries created

by DDL, update stats and utils. In this way, preexisting

code sets up child query IDs and the preexisting cancel

broker code in RMS cancels the entire query tree below

the indicated parent. Also added is logic to check for a

canceled parent at the time a child query ID is setup in

Statement::execute, to make the scheme more robust and

to support a common scenario where the parent is compiling

a child query at the time of cancel.

This change also makes ex_root_tcb::cancel to wait for

cancel broker message completion. This fixes a problem where

a query (e.g., CREATE INDEX or DROP INDEX) is prepared,

executed once and gets an error, then re-execed. In this

case, sqlci was not closing the statement after the error.

To improve robustness, after an error we will now wait for

message completion, if needed.

Since we do not yet have transaction protection for DDL,

it will usually be required to cleanup canceled DDL

operations by using the CLEANUP command.

Change-Id: I8940f7108906d5d8d1a8aa4574aacf2b9ffcf0b6

  1. … 27 more files in changeset.
Fixes for security gaps

Fix summary:

1389791 – Create table with 128 character-long schema & table names hangs on HortonWorks

fix 1 - Privilege checks not working for UDRs

fix 2 - QI not working when UDR's are involved

fix 3 - Routines are not being removed from NARoutineDB cache

Code cleanup

Miscellaneous changes

1389791: Create table with 128 character-long schema & table names hangs on HortonWorks

Check to make sure the total name length is not longer than supported value,

see: https://issues.apache.org/jira/browse/HDFS-6055

bin/SqlciErrors.txt - new error message

sqlcomp/CmpCatSqlErrorCodes.h - new error message

sqlcomp/CmpSeabaseDDLmd.h - new literal describing length of generated HBase name

sqlcomp/CmpSeabaseDDLcommon.cpp - new check for maxmum HBase name length

fix 1: privilege checks are not working correctly for UDR's

The method RelRoot::checkPrivileges is called to verify privileges for all object types.

However, some UDR objects checks were skipped because they were not added to the UDR Stoi list.

optimizer/BindItemExpr.cpp - add function to Stoi list

optimizer/BindRelExpr.cpp - add procedures to Stoi List

optimzier/RelMisc.h - signature changes for privilege related work

optimizer/BindRelExpr.cpp - rewrote checkPrivileges

optimizer/NARoutine.h/NARoutineDB.cpp - added method

moveRoutineToDeleteList

fix 2: QI is not working when UDR's are dropped

Code to drop items from NARoutineDB cache was missing.

Code to set security keys for the user in the plan was missing

Code to set objectUIDs in the plan was missing

When security keys were added, they were incorrect

sqlcomp/CmpMain.h (.cpp) - added calls to compare invalidation keys with objects stored in

NARoutineDB cache; if found, then remove item from cache by

calling helper methods in NARoutineDB class.

optimizer/NARoutineDB.h (NARoutine.cpp) - added helper method to remove entries from the cache

free_entries_with_QI_key - based off of similar method for table cache

ComSecurityKey.h (.cpp) - new method to check invalidation keys shared by tables/routines

qiCheckForInvalidObject

optimizer/NATable.cpp - rewrote table invalidation code so it could be shared with routines.

generator/GenUdr.cpp - add the routine's object UID to the query plan

sqlcomp/CmpSeabaseDDLroutine.cpp - code to send invalidations keys during drop routine

common/ComSmallDefs.h - new QI actions for USAGE and REFERENCES

common/ComDistribution.cpp - add EXECUTE as a privilege for QI, also added USAGE and REFERENCES

sqlcomp/PrivMgrPrivileges.cpp - not generating correct security keys

fix 3: Routines were not being removed from NARoutineDB cache

Added new fields to the various routine structures for objectOwnerID, schemaOwnerID, and privInfo.

Set up the correct routineID in various routine structures

At drop time, made sure routine was removed from NARoutineDB cache

comexe/ComTdb.h - added new fields to routine descriptor and TDB

generator/Generator.cpp - new fields for routines

optimizer/NARoutine.h (.cpp) - new fields for routines

removeNARoutine - based off similar method for table cache

optimizer/NARoutine.cpp - added new field to store privilege information in NARoutine,

which also gets security keys needed for query invalidation

sqlcat/desc.h - new fields for routines

sqlcomp/CmpSeabaseDDLtable.cpp - set up new values in NARoutine structure

sqlcomp/CmpSeabaseDDLroutine.cpp - code to remove entries from cache at drop time

Other changes:

sqlcomp/PrivMgrCommand.h (.cpp) - performance change, don't check authorization enabled

sqlcomp/PrivMgrMD.h (.cpp) - performance change, don't check authorization enabled

sqlcomp/PrivMgrDesc.cpp - missing object_type

parser/sqlparser.y - incorrect object type set for grant/revoke on UDRs

ustat/hs_globals.cpp - incorrect error returned

Code cleanup:

cli/Statement.h - remove obsolete code

cli/Statement.cpp - remove obsolete code

common/Collections.h - remove obsolete code

generator/GenRelMisc.cpp - remove obsolete code

optimizer/ItemCache.cpp - remove obsolete code

optimizer/RelCache.cpp - remove obsolete code

optimizer/NARoutine.h - remove obsolete code

optimizer/NARoutine.cpp - remove obsolete code

executor/SqlTableOpenInfo.h - new helper methods to check privileges

sqlcomp/PrivMgrMD.h - new helper methods to check privileges and get text for error

sqlcomp/PrivMgrDefs.h - simplification of code for checkPrivileges method

Change-Id: I981ad7f094b79a25f5e0aca30dedea4601b424ea

  1. … 39 more files in changeset.
Fix shared segment corruption from seabase/TEST024

With this change, we never remove a StmtStats object

from StatsGlobals::stmtStatsList_ when the statement

is deallocated during AQR. The StmtStats object will

be removed after the AQR completes and the statement

is deallocated, or in general process cleanup.

The change also re-enables seabase/TEST024.

Change-Id: Icd5a23c4a88334892ca292fe2ff514fbb9913616

Closes-Bug: #1417748

  1. … 2 more files in changeset.
Manageability changes - event mgmt and stats publication

Implements changes to support event management using log4cpp.

Configuration files are located in $MY_SQROOT/conf folder and all logs

files are located in $MY_SQROOT/logs folder

For more information see the blueprint at:

https://blueprints.launchpad.net/trafodion/+spec/eventmanagement

Implements changes for publication of statistics to repository. For more

information see the blueprint at:

https://blueprints.launchpad.net/trafodion/+spec/repositorymetrics

Note:

In this initial delivery publication of statistics is disabled by

default and it can be enabled via DCS property. This code has been

reviewed internally prior to merging with mainline

Documentation:

https://wiki.trafodion.org/wiki/index.php/Trafodion_Manageability

Included timestamp to be part of the primarykey for metric aggregation

table

Addressed some of the comments and incorporated Anoop's change for

repository

Changed the queryBuf size in sql/sqlcomp/CmpSeabaseDDLrepos.cpp to 20000

Modified the sql/regress/seabase/EXPECTED024

Change-Id: I517575233c10b2a8683cdd1d53a2eec96d7c2a6f

  1. … 781 more files in changeset.
More fixes to make SQL thread safe

Fix for bug 1403334 – T2 multithread tests core dump in SQLCLI_DropContext

NAMemory was not threadsafe in derived memory management. The calls

registerMemory, unRegisterMemory was not threadSafe when the heap is marked

as threadsafe heap. Also, the mutex is now dropped as part of NAHeap

destructor.

Fix for bug 1403410 - With 128 threads thru T2 driver, MTTrans and YCSB tests

crash and generate this core

The context list was being manipulated without acquiring a mutex in some places

These obsolete codes are now removed.

Some of the globals were wrongly declared as thread globals.

Other Fixes

SQL_EXEC_GetDiagnosticsCondInfo2 CLI call is now protected by mutexes

like any other CLI call.

Also removed some of the obsolete codes in the cli layer.

Currently commented out the code to close all the open cursors in all

the contexts when a change in transaction id is detected. We need to detect

the change in transaction at context level and close cursors belonging to

that context only.

Change-Id: I0edb7b7f0a844c8c49995af4c130dd319472a783

  1. … 9 more files in changeset.
SQL syntax to cancel executing query, phase 2

This check-in causes queries to register with the cancel

broker using a nowait Seabed message, and to process the

completion of the message by canceling the executing query

if the cancel broker so indicates in the message reply. The

check-in includes new test cases in the regression test to

cancel different types of SELECT queries.

These types of queries do not yet respond to cancel:

1. Queries that are decomposed into child queries, e.g.,

CREATE INDEX, CREATE TABLE AS, UPDATE STATISTICS, etc;

2. UPDATE and DELETE

3. INSERT queries might work now, but haven't been tested.

By design, unique queries cannot be canceled.

Implements: blueprint sql-query-cancel

Change-Id: I95aaf9b42de645b60ba68cdce0da729b84f1eb82

Change-Id: I1ee1f05fd33c2cfb72405a5ddddb9518ab52076d

  1. … 10 more files in changeset.
Query Invalidation triggered by DDL, phase 3

This check-in allows invalidation of queries which have been prepared

and are held by the EXE for execution or re-execution. It does not

invalidate running queries.

When an attempt is made to (re)execute an invalidated query, a

special SQLCODE, 8738, is raised and the query is sent ack to the

compiler via the AQR mechanism. The check-in include a new test cases

in the executor/TEST122 regression test which demonstrate the

functionality.

Change-Id: I98ea484f9d7fc9e43372e3057d8e8288162865c1

Implements: blueprint ddl-query-invalidation

  1. … 17 more files in changeset.
Enable RMS monitoring

The query execution statistics can now be obtained while the query is running

via

a) GET STATISTICS FOR QID <qid>

b) Using STATISTICS table-valued function

c) Using CLI calls – SQL_EXEC_GetStatistics2 and SQL_EXEC_GetStatisticsItems

The statistics can be viewed in accumulated, pertable, progress and operator

stats format.

You can also obtain statistics for each fragment instance and process wise

statistics for each operator of an active query.

Light-weight SQL Offender (LSO) feature is also enabled. With LSO, you can get

1. Queries offending CPU resource

2. Queries blocked in Trafodion Engine

3. Queries that are not active

4. Queries that haven’t finished but blocked in client

5. Queries offending memory resource

Explain in RMS feature can be enabled by setting the

cqd explain_in_rms ‘on’ before compiling a query.

EXPLAIN.. FOR QID <qid> command is now supported.

GET PROCESS STATISTICS FOR <pid> is also supported

Change-Id: I59eb54dd4fd347a37e0b7e7842ee242e553ef60d

  1. … 48 more files in changeset.
Security changes to support authorization

Added support for authorization commands:

- initialize authorization [, drop]

- create/drop roles

- register/unregister components

- create/drop component operations

- grant/revoke object privileges

- grant/revoke role privileges

- grant/revoke component privileges

- updates to GET and SHOWDDL statements

- checking of privileges for DML requests

- checking of privileges for DDL requests

- regression tests added to catman1 library

Fixed a testware problem in catman1 TEST135 and TEST139

Fixed a parser problem introduced by compGeneral/TESTTOK2 which was recently

introduced.

More details:

This delivery was part of code worked on by many people for several

months on a remote branch. This team held bi-weekly meetings

for several months to design and implement these features. These

meetings also included extensive code reviews.

The security features which include authentication (which was delivered

in June) and authorization is turned off by default. The

traf_authentication_setup script located in $MY_SQROOT/sql/scripts needs

to be run to enable both authentication and authorization. This

procedure is described on the Trafodion Twiki page and will be updated once this

delivery completed to include authorzation.

Delivery updates:

Updated traf_authentication_setup to return consistent error messages

and added a comment to ComSmallDefs.h to address a buf size issue for

metadata tables.

Change-Id: I896f1ee006590284653b2c9882901c05b5f2ba22

  1. … 100 more files in changeset.
Using embedded compiler will not get error 2031

There is no need to raise error 2031 (Could not use embedded compiler to

compile the statement: ...) when failed to compile a query. Removed the

AQR entry for SQL error 2031.

Change-Id: I3979574ea9405d0b35c0c7924e30ad64475c6c10

  1. … 2 more files in changeset.
Code Drop Update - 5/23/14

Change-Id: If478e8857cbfa9652227af7ed83cd61dd075a889

  1. … 163 more files in changeset.
Initial code drop of Trafodion

    • -0
    • +9163
    ./Statement.cpp
  1. … 4886 more files in changeset.