CmpSeabaseDDL.h

Clone Tools
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
iChanges based on review comments

  1. … 14 more files in changeset.
Fixees to support stroring Udr libraries s Blobs.

  1. … 45 more files in changeset.
Only expose supported component operations

There is a set of component operations and many are about features that we do

not support such as CREATE_TRIGGER. This checkin no longer returns these

privileges through

get privileges on component sql_operations

showddl component sql_operations

The is_system metadata column in the component_operations table now supports

three values:

Y - it is a system operation

N - it is a user operation

U - it is an unsupported (unused) operation (new)

An "initialize authorization" or fresh installation is required to make these

changes available.

Installating this code version calls initialize authorization

Running regrinit.sql also calls initialize authorization

Some performance enhancements were made to make "initialize authorization" run

faster.

  1. … 17 more files in changeset.
TRAFODION-3121 Enhance 'initialize trafodion' to return progress status

-- init traf will now return status during various steps.

'Started' indication will be returned when that step starts.

'Completed' will be returned when it completes successfully.

'Failed' will be returned if that step fails.

Elapsed time for each step will be shown in {ET hh:mm:ss} format.

-- method CmpSeabaseDDL::initSeabaseMD is rewritten and moved from

file CmpSeabaseDDLcommon.cpp to new file CmpSeabaseDDLinitraf.cpp.

-- priv mgr tables are now always created during init traf.

-- a 'no return status' option has been added if status is not to be

returned during init traf. This is used when init traf is called

internally, for ex, during upgrade.

It is not an externalized option.

-- Output of init traf is shown below.

Note: output is not a printf/cout to stdout.

It is returned as selected rows to the caller.

Apache Trafodion Conversational Interface 2.3.0

Copyright (c) 2015-2017 Apache Software Foundation

>>initialize trafodion;

Initialize Trafodion: Started

Verify User: Started

Verify User: Completed {ET: 00:00:00}

Version Check: Started

Version Check: Completed {ET: 00:00:00}

Create Metadata Tables: Started

Create Metadata Tables: Completed {ET: 00:00:49}

Update Metadata Tables: Started

Update Metadata Tables: Completed {ET: 00:00:06}

Create Schema Objects: Started

Create Schema Objects: Completed {ET: 00:00:20}

Create Metadata Views: Started

Create Metadata Views: Completed {ET: 00:00:05}

Create Repository Tables: Started

Create Repository Tables: Completed {ET: 00:00:16}

Create PrivMgr Tables: Started

Create PrivMgr Tables: Completed {ET: 00:00:29}

Create Library Manager: Started

Create Library Manager: Completed {ET: 00:00:51}

Initialize Trafodion: Completed {ET: 00:03:09}

--- SQL operation complete.

  1. … 36 more files in changeset.
CTAS fixes for ddl on hive objects

  1. … 26 more files in changeset.
Merge remote branch 'origin/master' into ansharma_nnpk_br

Conflicts:

core/sql/regress/seabase/EXPECTED031

core/sql/sqlcomp/CmpDescribe.cpp

core/sql/sqlcomp/CmpSeabaseDDLtable.cpp

core/sql/sqlcomp/DefaultConstants.h

  1. … 12 more files in changeset.
nonnull pkey, commit #1

  1. … 22 more files in changeset.
TRAFODION-3086 Traf support for DDL operations on Hive objects

-- Support for TRAFODION-3086. Details in document attached to jira.

Other changes:

-- support for "if not exists", "if exists" clause for create/drop view

-- Support for: truncate T, truncate table T.

-- same as purgedata

-- showddl <tab>, detail

-- unregister hive schema <sch>

-- will unregister all objects in specified schema

-- Support for "if not registered", "if registered" clauss for

register/unregister command.

  1. … 90 more files in changeset.
TRAFODION-3046: Privilege support for native HBase tables

-- Grants and revokes against native HBase tables are enforced similar to hive

-- Privilege checking added when creating and dropping native HBase tables

-- Removing dependent Trafodion metadata when native HBase tables are dropped

-- Added regression test (privs2/TEST146)

-- Reorg - split PrivMgrComponents into 2 files:

PrivMgrComponents and PrivMgrUserPrivs

An hbase table can be referenced using one of the following types:

"_CELL_" - references cell data

"_ROW_" - references row data

"_MAP_" - references data defined by the mapped (external) table

Privileges are granted against each of these types, so if you:

select * from hbase."_CELL_".hbase1;

You must have the select privileges on this table

For example: "grant select on hbase."_CELL_".hbase1 to user1"

Likewise for other hbase types, grants are required to gain accessibility

grant select on hbase."_MAP_".hbase1 to user1

grant select on hbase."_ROW_".hbase1 to role1

  1. … 25 more files in changeset.
[TRAFODION-3005] Fix issues with keys on long columns

  1. … 1 more file in changeset.
Merge [TRAFODION-2977] PR 1473 Fix bugs with max key length detection

  1. … 1 more file in changeset.
[TRAFODION-2977] Fix issues with maximum key length detection

  1. … 4 more files in changeset.
[TRAFODION-2974] Make event log reader and JDBC real TMUDFs

Roberta pointed out that we have two predefined UDFs, EVENT_LOG_READER

and JDBC, where the system administrator should have the ability to

control who can execute these functions.

To do this, these two UDFs cannot be "predefined" UDFs anymore, since

those don't have the metadata that's required for doing grant and

revoke.

Roberta also pointed out that the JDBC UDF should refuse to connect to

the T2 driver, for security reasons.

The fix leaves the predefined TMUDFs in place, for now, they will be

removed in R2.4 (see TRAFODION-2975).

The new "real" TMUDFs are in the "_LIBMGR_" schema, mostly for

convenience, as this schema has other UDFs that are created when

Trafodion is initialized.

  1. … 10 more files in changeset.
[TRAFODION-2819] Fix 4247 error on drop table when certain CQDs are present

  1. … 4 more files in changeset.
new COMMENT-ON SQL statement: review change - delete allocated spaces for function CmpSeabaseDDL::getSeabaseObjectComment

1. delete allocated COMMENT spaces after used.

  1. … 4 more files in changeset.
new COMMENT-ON SQL statement: init

1. new SQL syntax: COMMENT ON

2. new component privilege: SQL_OPERATIONS::COMMENT

3. MD table changes: add new column for tables "_MD_".OBJECTS and

"_MD_".COLUMNS

4. SHOWDDL changes: add COMMENT-ON output for each object

5. new build-in views: "_MD_".OBJECT_COMMENT_VIEW and

"_MD_".COLUMN_COMMENT_VIEW

  1. … 25 more files in changeset.
[TRAFODION-2674] Create index sees error 2006 internal assertion

RangePartitionFunction was missing region boundaries for indexes and using table's

boundaries incorrectly

Turned OFF OPT_PCODE_CACHE to avoid mem leak.

Unrelated testware change for compGeneral/TEST023

  1. … 7 more files in changeset.
ra TRAFODION-2584 Add support to register hive schemas in traf MD

This checkin extends jira TRAFODION-2584 to add support to register

hive schemas(referred to as databases in hive) in trafodion metadata.

Until now, only hive tables and views were registered.

Registering hive schemas would allow certain operations, like

granting/revoking privileges, to be done on a hive schema/database.

Syntax to register/unregister hive schema

-- register/unregister hive schema hive.sch;

Syntax to show registred hive schemas

-- get hive registered schemas in catalog trafodion

"showddl schema hive.sch" will now show the ddl for hive database

creation and whether this schema is registered.

showddl and 'get tables in schema' will now return error if that

schema doesn't exist.

regress/hive/TEST007 has been extended with additional tests.

  1. … 15 more files in changeset.
TRAFODION - 2632 Performing update statistics on metadata tables causes issues

Added code to return error and cleanup code a bit:

*** ERROR[9205] UPDATE STATISTICS is not supported for object <metadata table>

Also fixed a problem where privileges on Hive tables that included external

table gave false "no priv" error for the SHOWDDL <hive table> command.

  1. … 5 more files in changeset.
TRAFODION-2610 Register native HBase tables in traf MD plus couple more

-- support for registering hbase native cell/row format tables in traf MD

Syntax:

register [internal] hbase table [if not exists] <table-name>

unregister [internal] hbase table [if exists] <table-name>

-- support to get info related to registered hbase tables

Syntax: get hbase registered table in catalog trafodion

-- enhancements to HBase mapped tables and bug fixes

-- disabling of creation of external tables on native hbase

tables unless they are mapped tables

-- support for case-insensitive format string in TO_DATE function

(exp/exp_datetime.h)

-- regress/seabase/TEST022 enhanced with additional tests

  1. … 48 more files in changeset.
[TRAFODION-2603] Remove obsolete utilities and component privileges

  1. … 15 more files in changeset.
[TRAFODION-2584] Add support to register hive objects in traf metadata

Syntax:

register hive table/view [if not exists] <object-name> [cascade]

unregister hive table/view [if exists] <object-name> [cascade]

cascade option: register/unregister all underlying objects that are

part of the specified view

update statistics, grant/revoke, traf views or external table creation

on hive objects will automatically and internally register those objects

in trafodion metadata.

invoke/showddl will show if this object is registered and whether

that registration was internal or explicit.

Get command extensions:

get hive registered tables/view/objects in catalog trafodion;

get hive external tables in catalog trafodion;

Cleanup command extensions:

cleanup metadata command will clean up inconsistent hive objects

(underlying hive object is missing but object is registered or

an external table exists)

cleanup [hive table | hive view] on <object-name>;

Existing hive objects that had implicit or explicit external tables

created prior to this checkin will have no change in behavior.

ObjectUID of those external tables will continue to be used for

relevant operations.

One can drop those external tables and explicitly register them,

or a subsequent operation(upd stats, grant, etc) that needs objectUID will

automatically register them.

minor changes based on review comments of previous checkin

get all objects command on hive metadata no longer fails.

get views on objects return 3-part name that could be used to

differentiate between a traf and hive view.

regress/hive/TEST007 has been extended.

TBD: Add register/unregister privileges

  1. … 52 more files in changeset.
Various TRAFODION jira fixes, details below.

TRAFODION-2416 HBASE_OPTIONS on create index are now correctly recreated

after data is removed through purgedata command.

TRAFODION-2401 a foreign key referential constraint with NOT ENFORCED

option no longer creates an index.

TRAFODION-2299 A WITH statement now displays number of rows selected

instead of 'sql operation complete'. This fix is currently only done

for sqlci commands. It still needs to be added to trafci interface.

TRAFODION-1796 During regressions run, if default schema 'SCH'

doesn't already exist, then it is automatically created and updated in

the system defaults table when the first create statement is seen.

This change was added a while back.

TRAFODION-1637 vsbb delete was incorrectly being chosen when deleting rows

from hbase tables in _ROW_ or _CELL_ format. That has been fixed.

In addition, the cqd hbase_delete_costing 'OFF' has been removed

from seabase/TEST022.

hive/TEST007 has been fixed to create a trafodion schema where non-hive

tables/views are created.

  1. … 12 more files in changeset.
TRAFODION-2439 add support for traf views on hive tables

support for creation of trafodion views on hive tables.

cqd hive_views is set to ON by default to enable it.

During view creation, an implicit external table is created

on underlying hive table if one doesn't already exists.

This is used to track usage info

in views_usage metadata table and to track privileges.

Usage info is used to return views on tables or tables in views

during GET stmts.

trafodion view is created in trafodion catalog/schema and not in hive

catalog/schema.

delete and update operations are not allowed on traf/hive views.

drop of external hive table with CASCADE option will cause all

dependent views to be dropped.

drop of external table without CASCADE option will not drop it

an a dependent view exists on that table.

If the underlying hive table is dropped from hive shell,

then an error is returned at runtime when that view is accessed.

new test regress/hive/TEST007 contains tests for this feature.

  1. … 15 more files in changeset.
TRAFODION-1586 Add support for external Traf tables mappedd to HBase tables

This is the initial support for HBase mapped tables.

A doc for external manuals will be added later.

regress/seabase/TEST022 contains various testcases.

  1. … 77 more files in changeset.
jira [trafodion-2290] and few other bug fixes

-- trafodion-2290: pcode support for added columns on aligned tables

-- lob conversion error is reported instead of crash

-- alter table rename is restricted to not run under a user xn

-- error during alter table rename are handled correctly

-- error during alter/drop cols on aligned tables are handled correctly.

-- aligned format drop column with delimited names are handled correctly

-- rollup grouping now work correctly on expressions and renamed columns

  1. … 23 more files in changeset.
[TRAFODION-2282][TRAFODION-882] New scheme to invalidate histogram cache

  1. … 27 more files in changeset.
[TRAFODION-2187] Fix DROP SCHEMA CASCADE when sample tables are present

  1. … 5 more files in changeset.
[TRAFODION-2161] Fix issue with upgrade failure when Repository views exist

  1. … 5 more files in changeset.
TRAFODION [2137] Improve metadata access time during query compilation

A change was made to return privilege information in the descriptor structure

instead of getting it when the NATable or NARoutine object is instantiated.

For tables, storing privileges in the descriptor structure allows privileges

to be saved with other table attributes in the metadata. This improves metadata

access time during initial query compilations.

Changes:

--> At create time or when the object's DDL changes (redeftime), the compiler

gets the list of privs for all users. If stored descriptors is enabled,

this list is stored as part of the object definition in the TEXT table.

--> PrivMgr returns a list of bitmaps for all users granted any priv

--> the list of privs is transformed into a VirtTable

--> the VirtTable is transformed into TrafDesc

--> a packed form of the TrafDesc is stored in the TEXT table

--> When an NATable or NARoutine is instantiated, the current user's credentials

are extracted from the TrafDesc and stored in the class thereby eliminating

the need to perform I/O to get privs for the user.

  1. … 22 more files in changeset.