RelExeUtil.h

Clone Tools
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
TRAFODION-3121 Enhance 'initialize trafodion' to return progress status

-- init traf will now return status during various steps.

'Started' indication will be returned when that step starts.

'Completed' will be returned when it completes successfully.

'Failed' will be returned if that step fails.

Elapsed time for each step will be shown in {ET hh:mm:ss} format.

-- method CmpSeabaseDDL::initSeabaseMD is rewritten and moved from

file CmpSeabaseDDLcommon.cpp to new file CmpSeabaseDDLinitraf.cpp.

-- priv mgr tables are now always created during init traf.

-- a 'no return status' option has been added if status is not to be

returned during init traf. This is used when init traf is called

internally, for ex, during upgrade.

It is not an externalized option.

-- Output of init traf is shown below.

Note: output is not a printf/cout to stdout.

It is returned as selected rows to the caller.

Apache Trafodion Conversational Interface 2.3.0

Copyright (c) 2015-2017 Apache Software Foundation

>>initialize trafodion;

Initialize Trafodion: Started

Verify User: Started

Verify User: Completed {ET: 00:00:00}

Version Check: Started

Version Check: Completed {ET: 00:00:00}

Create Metadata Tables: Started

Create Metadata Tables: Completed {ET: 00:00:49}

Update Metadata Tables: Started

Update Metadata Tables: Completed {ET: 00:00:06}

Create Schema Objects: Started

Create Schema Objects: Completed {ET: 00:00:20}

Create Metadata Views: Started

Create Metadata Views: Completed {ET: 00:00:05}

Create Repository Tables: Started

Create Repository Tables: Completed {ET: 00:00:16}

Create PrivMgr Tables: Started

Create PrivMgr Tables: Completed {ET: 00:00:29}

Create Library Manager: Started

Create Library Manager: Completed {ET: 00:00:51}

Initialize Trafodion: Completed {ET: 00:03:09}

--- SQL operation complete.

  1. … 36 more files in changeset.
TRAFODION-3086 additional fixes

  1. … 7 more files in changeset.
TRAFODION-3086 Further enhancements to 'DDL on Hive objects'

-- Hive CTAS enhancements

-- CTAS will do create through Hive and insert...select in Traf

-- a cqd can be set if CTAS need to be passed in completely to Hive

-- CTAS 'no load' option can be used to create a Hive

table LIKE a traf table.

Older 'create hive like traf' is removed.

-- explain for CTAS to show ddl, insert...select, upd...stats stmts

-- Truncate enhancements.

-- traf: purgedata, truncate, truncate table

-- Hive: truncate, truncate table

-- 'if exists' option has been added

-- Hive truncate is through new exeutil operator

-- older operator uses LOB interface to clear data.

Maintained as Legacy operator, will be removed.

-- new operator uses Hive to truncate tables

-- explain for truncate to show trunc query that will be sent

-- removed obsolete parallel label op

-- removed obsolete purgedata (fast delete) op

  1. … 30 more files in changeset.
CTAS fixes for ddl on hive objects

  1. … 26 more files in changeset.
TRAFODION-3086 Traf support for DDL operations on Hive objects

-- Support for TRAFODION-3086. Details in document attached to jira.

Other changes:

-- support for "if not exists", "if exists" clause for create/drop view

-- Support for: truncate T, truncate table T.

-- same as purgedata

-- showddl <tab>, detail

-- unregister hive schema <sch>

-- will unregister all objects in specified schema

-- Support for "if not registered", "if registered" clauss for

register/unregister command.

  1. … 90 more files in changeset.
New syntax to retrieve the LOB HDFS filename for both external and internal LOBs . Also added syntax to return starting offset of a particular LOB handle in the LOB Hdfs data file.

  1. … 15 more files in changeset.
Few fixes, details listed below.

-- fix an issue where multiple values inserted from a list would return

error but each value inserted on its own would succeed.

ex: create table ts (a timestamp);

insert into ts values ('2017-01-01 10:10:10'), ('2018-01-01 10:10:10');

-- sometimes errors returned from child during hive inserts were not

being returned. That has been fixed.

-- TRAFODION-2683 extension.

added a 'p' (prune) option which would cleanse and filter unneeded

explain output. This helps in reducing output especially

for larger explains.

Ex:

>>explain option 'p' select * from dual;

------------------------------------------------------ PLAN SUMMARY

STATEMENT_NAME ........... NOT NAMED

STATEMENT ................ select * from dual;

------------------------------------------------------- NODE LISTING

ROOT ====================================== SEQ_NO 2 ONLY CHILD 1

DESCRIPTION

fragment_id ............ 0

parent_frag ............ (none)

fragment_type .......... master

xn_access_mode ......... read_only

auto_query_retry ....... enabled

embedded_arkcmp ........ used

select_list ............ %(0)

input_variables ........ %(0), %(0), %(0)

VALUES ==================================== SEQ_NO 1 NO CHILDREN

DESCRIPTION

fragment_id ............ 0

parent_frag ............ (none)

fragment_type .......... master

tuple_expr ............. %(0)

--- SQL operation complete.

>>

  1. … 19 more files in changeset.
TRAFODION-2683 add a new explain option to mask variant fields in output

--Syntax extension to cleanse and mask:

explain options 'c' select ...

explain options 'fc' select...

prepare s from select ...

explain option 'c' s;

--Masked fields will show up as "###" in explain output.

--Both option and options keyword can be used in explain and showplan.

-- explain option/options ...

-- showplan option/options ...

-- showplan can be used on explain statement.

-- Here is an example of what the output will look like:

>>explain option 'c' select * from dual;

----------------------------------------------- PLAN SUMMARY

MODULE_NAME .............. DYNAMICALLY COMPILED

STATEMENT_NAME ........... NOT NAMED

PLAN_ID ................ ###

ROWS_OUT ............... ###

EST_TOTAL_COST ......... ###

STATEMENT ................ select * from dual;

------------------------------------------- NODE LISTING

ROOT ================================ SEQ_NO 2 ONLY CHILD 1

REQUESTS_IN ............ ###

ROWS_OUT ............... ###

EST_OPER_COST .......... ###

EST_TOTAL_COST ......... ###

DESCRIPTION

max_card_est ......... ###

fragment_id ............ 0

parent_frag ............ (none)

fragment_type .......... master

statement_index ........ 0

affinity_value ....... ###

max_max_cardinality ###

total_overflow_size ###

xn_access_mode ......... read_only

xn_autoabort_interval 0

auto_query_retry ....... enabled

plan_version ....... 2,600

embedded_arkcmp ........ used

select_list ............ %(0)

input_variables ........ %(0), %(0), %(0)

VALUES ======================== SEQ_NO 1 NO CHILDREN

REQUESTS_IN ............ ###

ROWS_OUT ............... ###

EST_OPER_COST .......... ###

EST_TOTAL_COST ......... ###

DESCRIPTION

max_card_est ......... ###

fragment_id ............ 0

parent_frag ............ (none)

fragment_type .......... master

tuple_expr ............. %(0)

--- SQL operation complete.

>>

  1. … 21 more files in changeset.
Multiple fixes: REPLACE func maxlen, hbase unregister, DDLExpr cleanup

-- REPLACE function max length is no longer limited to 32K

-- HBase object unregister was not working correctly. That is fixed.

-- class DDLExpr has been cleaned up to have one constructor for all cases.

All options are now set using flags instead of constructor params.

-- "showddl <tab>, explain " command now returns an error instead of crash.

  1. … 10 more files in changeset.
[TRAFODION-2603] Remove obsolete utilities and component privileges

  1. … 15 more files in changeset.
[TRAFODION-2420] RMS enhancements Introduced 2 new counters HeapWM: - "SQL Heap WM" - Tracks the high water mark of the used memory including space buffer and bmo operator memory. It is enough to monitor this counter alone to track the memory utilization of the query.

scrIOTime: "Scr. IO Time" - Tracks the IO time taken to read/write to

the scratch files at BMO operator level

Introduced a 'Single line Format' option to display BMO stats and SE stats

in a single line format to enable exporting the output to Excel easily.

GET STATISTICS FOR QID CURRENT PROGRESS, OPTIONS 'SL'

Removed and cleaned up the code further

Also fixed memory corruption issues seen with GET STATISTICS command.

  1. … 16 more files in changeset.
Various fixes, details below

-- max length limited to 16777216 bytes (16M)

for char cols and functions (repeat, concat).

(optimizer/SynthType.cpp, common/ComSmallDefs.h, sqlcomp/nadefaults.cpp)

-- previous max length change requires adding of a new hbase property

called hbase.client.keyvalue.maxsize so large key/values could be

handled in hbase cell.

Following scripts have been updated to handle that.

(sqf/sql/scripts/install_local_hadoop,

install/ambari-installer/traf-mpack/common-services/TRAFODION/2.1/

service_advisor.py

install/installer/traf_hortonworks_mods

install/python-installer/configs/mod_cfgs.json)

Developers can also update hbase-site.xml with this property if

they dont want to reinstall local hadoop.

<property>

<name>hbase.client.keyvalue.maxsize</name>

<value>0</value>

</property>

-- while accessing a hive table as an external table, the hive table and

corresponding external table definitions are validated to be the

same. This validates that corresponding columns have the same

data attributes (type, length, scale, etc).

This check causes failures if hive column is of 'string' datatype.

That is because hive 'string' column length can be changed by

a cqd but the corresponding external table has predefined length

set when the table is created.

The validation check now ignores the length attribute if the hive column is

of 'string' datatype.

(optimizer/BindRelExpr.cpp, common/CharType.*, NAType.*)

-- data moved into direct buffer would sometimes cause overflow and crash.

The max direct buffer length used to send/retrieve hbase data

is now limited to 1G (executor/ExHbaseAccess.cpp)

-- errors during vsbb upsert are now handled correctly

(executor/ExHbaseIUD.cpp)

-- support for GET CATALOGS command

(generator/GenRelExeUtil.cpp, executor/ExExeUtilGet.cpp)

-- An incorrect computation would sometimes cause group by rollup to

crash compiler in NAHeap::unlinkLargeFragment()

(generator/GenRelGrby.cpp)

-- sort of a large row that exceeded sort pre-set buffer size would crash

if the row size exceeded the max buffer size.

That has been fixed by allocating space for atleast one row.

(generator/GenRelMisc.cpp)

-- purgedata on a delimited name was failing. That has been fixed.

(optimizer/RelExeUtil.h)

-- regress/tools/runregr_privs1/privs2 fixed to handle running of

subset of tests

-- regress/seabase/TEST031 updated with new tests

  1. … 30 more files in changeset.
TRAFODION-2498 Add support to run hive stmts from traf interface

Syntax:

process hive statement '<string>';

<string>: hive statement starting with create/drop/alter/truncate.

These are the only stmts currently supported.

Ex:

>>process hive statement 'create database trafsch';

will create hive database 'trafsch'

>>process hive statement 'create table trafsch.t (a int)';

will create hive table 't' in hive schema 'trafsch'.

'process hive statment ..' could be issued from any traf interface

(sqlci/trafci/jdbc...)

  1. … 24 more files in changeset.
TRAFODION-2441 user has only select privilege on a table can do ...

This is the second delivery that updates "get" commands so users can only view

information where they have been granted privileges. This delivery handles:

get components;

get privileges on component;

get privileges on component for <role>;

get privileges on component for <user> [cascade];

cascade returns privileges for user plus any roles the user has been granted

  1. … 5 more files in changeset.
[TRAFODION-2315] Heuristic decision for common subexpressions

This includes various fixes in addition to the heuristic decision,

mainly an improvement on how to navigate between parent and child

common subexpressions.

  1. … 29 more files in changeset.
Changes to support new syntax and implementation empty_blob/empty_clob for insert/updates Changes to support new syntax for updating a lob directly through a lobhandle without scanning the table.This is done via new ExeUtil operator. Changes to add columnname info to LOBMD_ table. This helps in mapping any given lobhandle back to the table and the column. Added new sections to the regression test executor/TEST130 to test these changes

  1. … 40 more files in changeset.
[TRAFODION-2317] Infrastructure for common subexpressions

This is a first set of changes to allow us to make use of CTEs

(Common Table Expressions) declared in WITH clauses and to create

a temp table for them that is then read multiple times in the query.

This also includes a fix for

[TRAFODION-2280] Need to remove salt columns from uniqueness constraints

Summary of changes:

- Adding a unique statement number in CmpContext

- Moving the execHiveSQL method from the OSIM code to CmpContext

- Adding a list of common subexpressions and their references

to CmpStatement

- Adding the ability to the Hive Truncate operator to drop the

table when the TCB gets deallocated

- Adding the ability to the HDFS scan to compute scan ranges at

runtime. Those are usually determined in the compiler. This is

only supported for simple, non-partitioned, delimited tables.

We need this because we populate the temp table and read from

in in the same statement, without the option of compiling

after we inserted into the temp table.

- Special handling in the MapValueIds node of common subexpressions.

See the comment in MapValueId::preCodeGen().

- Moved the binder code to create a FastExtract node into a new

method FastExtract::makeFastExtractTree(), to be able to call

it from another place.

- MapValueIds no longer looks at the "used by MVQR flag" to determine

the method for VEGRewrite. Instead it checks whether a list of

values has been provided to do this.

- Adding a new method, RelExpr::prepareMeForCSESharing, that is

kind of an "unnormalizer", undoing some of the normalizer

transformations.

- Implementing the steps for common subexpression materializations

described below.

- Adding the ability to suppress the Hive timestamp modification

check when truncating a Hive table

- Adding an optimizer rule to eliminate CommonSubExprRef nodes.

These nodes should not normally survive past the SQO phase, but

if the SQO phase gets interrupted by an exception, that could

happen, since we then fall back to a copy of the tree before

SQO. In the future, we can consider cost-based decision on

what to do with common subexpressions.

- Adding CommonSubExprRef nodes in the parser whenever we expand

a CTE reference.

- Adding cleanup code to the "cleanup obsolete volatile tables"

command that removes obsolete Hive tables used for common

subexpressions.

Other changes contained in this change set:

- Optimization for empty scans, like select * from t where 1=0

This now generates a cardinality constraint with 0 rows, which

can be used later to eliminate parts of a tree.

(file OptLogRelExpr.cpp)

- [TRAFODION-2280] Need to remove salt columns from uniqueness

constraints generated on salted tables.

(file OptLogRelExpr.cpp)

- Got rid of the now meaningless "seamonster" display in EXPLAIN.

(file GenExplain.cpp and misc. expected files)

- Suppress display of "zombies" in the cstat command. Otherwise,

these zombies (marked as <defunct>) prevent Trafodion from

starting, because they are incorrectly considered "orphan"

processes. This could require a reboot when no reboot is necessary.

(file core/sqf/sql/scripts/pstat)

Incomplete list of things left to be done:

- TRAFODION-2316: Hive temp tables are not secure. Use volatile

tables instead.

- TRAFODION-2315: Add heuristics to decide when to use the temp table

approach.

- TRAFODION-2320: Make subquery unnesting work with common subexpressions.

Generated Plans

---------------

The resulting query plan for a query Q with n common

subexpressions CSE1 ... CSEn looks like this:

Root

|

MapValueIds

|

BlockedUnion

/ \

Union Q

/ \

... CTn

/

Union

/ \

CT1 CT2

Each of the CTi variables looks like the following, an

INSERT OVERWRITE TABLE tempi ...

BlockedUnion

/ \

Truncate FastExtract TEMPi

TEMPi |

CSEi

The original query Q has the common subexpressions replaced

with the following:

MapValueIds

|

scan TEMPi

Here is a simple query and its explain:

prepare s from

with cse1 as (select d_date_sk, d_date, d_year, d_dow, d_moy from date_dim)

select x.d_year, y.d_date

from cse1 x join cse1 y on x.d_date_sk = y.d_date_sk

where x.d_moy = 3;

>>explain options 'f' s;

LC RC OP OPERATOR OPT DESCRIPTION CARD

---- ---- ---- -------------------- -------- -------------------- ---------

11 . 12 root 1.46E+005

5 10 11 blocked_union 1.46E+005

7 9 10 merge_join 7.30E+004

8 . 9 sort 1.00E+002

. . 8 hive_scan CSE_TEMP_CSE1_MXID11 1.00E+002

6 . 7 sort 5.00E+001

. . 6 hive_scan CSE_TEMP_CSE1_MXID11 5.00E+001

1 4 5 blocked_union 7.30E+004

2 . 4 hive_insert CSE_TEMP_CSE1_MXID11 7.30E+004

. . 2 hive_scan DATE_DIM 7.30E+004

. . 1 hive_truncate 1.00E+000

--- SQL operation complete.

>>

CQDs to control common subexpressions

-------------------------------------

CSE_FOR_WITH is the master switch.

CQD Value Default Behavior

--------------------- --------- ------- ---------------------------------------

CSE_FOR_WITH OFF Y No change

ON Insert a CommonSubExprRef node in the

tree whenever we reference a CTE

(table defined in a WITH clause)

CSE_USE_TEMP OFF Y Disable creation of temp tables

for common subexpressions

SYSTEM Same as OFF for now

ON Always create a temp table for

common subexpressions where possible

CSE_DEBUG_WARNINGS OFF Y No change

ON Emit diagnostic warnings that show why

we didn't create temp tables for

common subexpressions

CSE_CLEANUP_HIVE_TABLES OFF Y No change

ON Cleanup Hive tables used for CSEs with

the "cleanup obsolete volatile tables"

command

CommonSubExprRef relational operators

-------------------------------------

This is a new RelExpr class that is introduced. It marks the common

subexpressions in a RelExpr tree. This operator remembers the name of

a common subexpression (e.g. the name used in the WITH clause).

Multiple such operators can reference to the same name. Each of

these references has a copy of the tree.

Right now, these operators are created in the parser when we expand a

CTE (Common Table Expression), declared in a WITH clause. If the CTE

is referenced only once, then the CommonSubExprRef operator is removed

in the binder - it also doesn't live up to its name in this case.

The remaining CommonSubExprRef operators keep track of various changes

to their child trees, during the binder and normalizer phases. In

particular, it tracks which predicates are pushed down into the child

tree and which outputs are eliminated.

The CmpStatement object keeps a global list of all the

CommonSubExprRef operators in a statement, so the individual operators

have a way to communicate with their siblings:

- A statement can have zero or more named common subexpressions.

- Each reference to a common subexpression is marked in the RelExpr

tree with a CommonSubExprRef node.

- In the binder and normalizer, common subexpressions are expanded,

meaning that multiple copies of them exist, one copy per

CommonSubExprRef.

- Common subexpressions can reference other common subexpressions,

so they, together with the main query, for a DAG (directed

acyclic graph) of dependencies.

- Note that CTEs declared in a WITH clause but not referenced are

ignored and are not part of the query tree.

In the semantic query optimization phase (SQO), the current code makes

a heuristic decision what to do with common subexpressions - to

evaluate them multiple times (expand) or to create a temporary table

once and read that table multiple times.

If we decide to expand, the action is simple: Remove the

CommonSubExprRef operator from the tree and replace it with its child.

If we decide to create a temp table, things become much more difficult.

We need to do several steps:

- Pick one of the child trees of the CommonSubExprRefs as the one to

materialize.

- Undo any normalization steps that aren't compatible with the other

CommonSubExprRefs. That means pulling out predicates that are not

common among the references and adding back outputs that are

required by other references. If that process fails, we fall back

and expand the expressions.

- Create a temp table tmp.

- Prepare an INSERT OVERWRITE TABLE tmp SELECT * FROM cse tree

that materializes the common subexpression in a table.

- Replace the CommonSubExprRef nodes with scans of the temp table.

- Hook up the insert query tree with the main query, such that it

is executed before the main query starts.

Temporary tables

----------------

At this time, temporary tables are created as Hive tables, with a

fabricated, unique name, including the session id, a unique statement

number within the session, and a unique identifier of the common

subexpression within the statement. The temporary table is created at

compile time. The query plan contains an operator to truncate the

table before populating it. The "temporary" Hive table is dropped when

the executor TCB is deallocated.

Several issues are remaining with this approach:

- If the process exits before executing and deallocating the statement,

the Hive table is not cleaned up.

Solution (TBD): Clean up these tables like we clean up left-over

volatile tables. Both are identified by the session id.

- If the executor runs into memory pressure and deallocates the TCB,

then allocates it again at a later time, the temp table is no longer

there.

Solution (TBD): Use AQR to recompile the query and create a new table.

- Query cache: If we cache a query, multiple queries may end up with

the same temporary table. This works ok as long as these queries are

executed serially, but it fails if both queries are executed at the

same time (e.g. open two cursors and fetch from both, alternating).

Solution (TBD): Add a CQD that disables caching queries with temp tables

for common subexpressions.

In the future we also want to support volatile tables. However, those also

have issues:

- Volatile tables aren't cleaned up until the session ends. If we run

many statements with common subexpressions, that is undesirable. So,

we have a similar cleanup problem as with Hive tables.

- Volatile tables take a relatively long time to create.

- Insert and scan rates on volatile Trafodion tables are slower than

those on Hive tables.

To-do items are marked with "Todo: CSE: " in the code.

  1. … 78 more files in changeset.
Merge remote branch 'origin/master' into memleak1

  1. … 8 more files in changeset.
Post merge commit. All files here relate to NAList

  1. … 64 more files in changeset.
similarity check changes, commit #1

  1. … 44 more files in changeset.
[JIRA TRAFODION-2141] Cluster view of hbase stats

-- added column REGION_SERVER to region stats virtual table

-- added cluster view of hbase stats

Syntax: select * from table(cluster stats());

-- added support for 'where' pred in stats queries

Syntax: select * from table(cluster stats()) where schema_name = 'SCH';

  1. … 22 more files in changeset.
metadata access performance improvement for accessing traf objects

Packed version of descriptor structures is now stored in traf TEXT

table during various DDL operations.

During dml, if packed descriptors exist, then they are read and used.

If they dont exist, then descriptors are generated on the fly.

Descriptor classes are defined in file sqlcat/TrafDDLdesc.h and

are based off the older desc structures from desc.h.

These are simplified and class'ified version of the older structures

and are versioned. Changes to these classes should be done carefully

by either using filler bytes or versioning them.

2 new cqds, traf_store_object_descriptor and traf_read_object_descriptor,

are added to control if packed descs should be stored/read.

These are currently set to OFF by default but are enabled during

regressions run. Once this feature is tested, these cqds will be turned

on by default.

In addition, some helper commands have been added to manipulate descrs.

Syntax:

alter table t [check|generate|delete|disable|enable] stored descritors

If authorization is enabled, then auth/priv specific info

is generated on the fly and currently not stored in metadata.

In a later checkin, these will also be stored in metadata.

  1. … 74 more files in changeset.
Fix upgrade so repository upgrade will succeed

  1. … 6 more files in changeset.
JIRA TRAFODION-2022 Some more changes related to hive truncate

-- support for 'TRUNCATE' command for hive tables

-- support for truncate with PARTITION specification

-- separation of traf purgedata and hive truncate code into

different classes and tdbs

-- some cleanup of obsolete older purgedata code

-- new tests added to regress/hive/TEST005

  1. … 23 more files in changeset.
hive data modification detection: commit #4

  1. … 24 more files in changeset.
Merge remote branch 'origin/master' into lob_get_trace

  1. … 8 more files in changeset.
support for get command for LOB tables. Support for tracing LOB operations.

  1. … 30 more files in changeset.
commit #1

  1. … 25 more files in changeset.
update

  1. … 298 more files in changeset.
TRAFODION [1879] - Integrate Library Management into Trafodion Metadata

The initial release of library management has been delivered to Trafodion. This

is follow-on work that integrates library management operations into the

existing Trafodion infrastructure. Currently, with the initial release of

library management, the consumer needs to run a special script to set up

everything. This delivery incorporates the steps into existing SQL commands and

removes the need for the script.

This delivery contains:

-> Support for three new INITIALIZE TRAFODION options

-> Support for a new role (plus infrastructure to make it easier to add roles)

-> Change initialize authorization to handle upgrade better

-> Fixed couple issues found while testing

-> New regression test (udr/TEST102)

*** Support for three new INITIALIZE TRAFODION options:

Three new INITIALIZE TRAFODION options have been added:

CREATE LIBRARY MANAGEMENT - create and populates the libmgr schema

DROP LIBRARY MANAGEMENT - drops the libmgr schema

UPGRADE LIBRARY MANAGEMENT - adds new procedures to the libmgr library

Parser was changed to support these new options, a new keyword - MANAGEMENT

was added.

(parser/ParKeyWords.cpp/sqlparser.y)

DDL compiler was changed to recognize the new options and call implementation

methods.

(optimizer/RelExeUtil, sqlcomp/CmpSeabaseDDLcommon.cpp)

The following implementation methods were added:

createSeabaseLibmgr (code for create library management)

dropSeabaseLibmgr (code for drop library management)

upgradeSeabaseLibmgr(code for upgrade library management)

createLibmgrProcs – a helper method called by create and upgrade code to

create libmgr procedures

grantLibmgrPrivs – a helper method called by createLibmgrProcs and

initialize authorization that add grants to procedures for

the new DB__LIBMGRROLE role.

(CmpSeabaseDDL.h/CmpSeabaseDDLroutine.cpp)

Changes were made to define the new schema, library and procedures.

(common/ComSmallDefs.h, CmpSeabaseDDLroutine.h)

All procedures are defined in a new file called sqlcomp/CmpSeabaseDDLroutine.h.

This file is based on similar support for system metadata tables

(CmpSeabaseDDLmd.h). It contains the text of all the procedures. During

"create library management" and "upgrade library management" operations, this

list is used to create/add procedures.

*** Support for a new role:

Initialize authorization code was changed to create the new DB__LIBMGRROLE role.

As part of this change, role infrastructure was modified to make it easier to

add system roles in the future. The initialize code checks to see what roles

have not been added, and adds them.

(common/NAUserId.h, common/ComUser, sqlcomp/PrivMgrMD,

sqlcomp/CmpSeabaseDDLauth, sqlcomp/PrivMgrRoles)

In addition, a check is made – if the SEABASE_LIBMGR_SCHEMA exists, then

grants are performed on the procedures to allow DB__LIBMGRROLE execute

privilege.

(sqlcomp/CmpSeabaseDDLcommon.cpp)

*** Change initialize authorization to handle upgrade better:

Changes were made so initialize authorization performs an upgrade operation if

called and component privileges already exists.

(sqlcomp/PrivMgrComponentOperations, sqlcomp/PrivMgrComponentPrivileges,

sqlcomp/PrivMgrMD)

*** Fixed couple of issues found while testing:

Invalid role ID generation. Role ID generation code was using the wrong range

to determine the next role ID.

(CmpSeabaseDDLauth.cpp)

Drop schema issues with libraries. Drop library cascade fails when the order

of libraries and functions is incorrect. If you drop function, drop library it

fails with missing procedure.

(CmpSeabaseDDLschema.cpp)

  1. … 26 more files in changeset.