Cli.h

Clone Tools
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
Move core into subdir to combine repos

  1. … 10768 more files in changeset.
Move core into subdir to combine repos

  1. … 10622 more files in changeset.
Move core into subdir to combine repos

Use: git log --follow -- <file>

to view file history thru renames.

  1. … 10837 more files in changeset.
Using the language manager for UDF compiler interface

blueprint cmp-tmudf-compile-time-interface

This change includes new CLI calls, to be used in the compiler to

invoke routines. Right now, only trusted routines are supported,

executed in the same process as the caller, but in the future we may

extend this to isolated routines. Using a CLI call allows us to share

the language manager between compiler and executor, since language

manager resources such as the JVM and loaded DLLs exist only once per

process. This change is in preparation for Java UDFs.

Changes in a bit more detail:

- Added 4 new CLI calls to allocate a routine, invoke it, retrieve

updated invocation and plan infos and deallocate (put) the routine.

The CLI globals now have a C/C++ and a Java language manager that

is allocated on demand.

- The compiler no longer loads a DLL for the UDF compiler interface,

it uses the new CLI calls instead.

- DDL syntax is changed to allow TMUDFs in Java (not officially

supported, so don't use it quite yet).

- TMUDFs in C are no longer supported, only C++ and Java are.

Converted remaining TMUDF tests to C++.

- C++ TMUDFs now do a basic verification at DDL time, so errors

like missing entry points are detected earlier. Validation for

Java TMUDFs is also done through the CLI.

- Make sure we have no memory or resource leaks:

- CmpContext keeps track of UDF-related objects allocated on

system heap and in the CLI, cleaned up at the end of a statement

- CLI keeps a list of allocated trusted routines, cleaned up

when a CLI context is deallocated

- Using ExeCliInterface class to make the new CLI calls (4 new calls

added).

- Removed CmpCli class in the optimizer directory and converted

tracking compiler to use ExeCliInterface as well.

- Compile-time parameter values are no longer baked into the

UDRInvocationInfo. Instead, they are provided as an input row, the

same way as they are provided at runtime.

- Bug fixes in C++ UDR code, mostly related to serialization and

to multiple interactions with the UDF through serialized objects.

- Added more info to UDRInvocationInfo (SQL access type, etc.).

- Since there are multiple plans per invocation, each of which

can have multiple interactions with the UDF, plans need to be

numbered so the UDF side can tell them apart to attach the

right state (owned by the UDF) to it.

- The language manager needs some functions that are provided by

the process it's running in. Added those (empty, for now) functions

as cli/CliImplLmExtFunc.cpp.

- Added a new class for Java TMUDFs, LmRoutineJavaObj. Added methods

to allocate such routines and to load their class as well as to

create Java objects by invoking the default constructor through JNI.

- Java TMUDFs use the new UDR interface (to be provided by Suresh and

Pavani). In the language manager, the container is the class of

the UDF, the external path is the fully qualified jar name. The

Java method name is <init>, the default constructor, with signature

"()V". Some code changes were required to do this.

- Created a new directory trafodion/core/sql/src for Java sources in

the sql engine. Right now, only language manager java

sources are in this directory, but I am planning to move the other

java sources under sql in a future checkin. Suresh and Pavani

will add their UDF-related Java files there as well.

- Renamed the udr jar to trafodion-sql-<version>.jar, in anticipation

of combining all the sql Java sources into this jar.

- Created a maven project file trafodion/core/sql/pom.xml and

changed makefiles to invoke maven to build java sources.

- More work to separate new UDR interface from older SPInfo object,

so that we can get rid of SPInfo if/when we don't support the older

style anymore.

- Small fix to odb makefile, make clean failed when executed twice.

Patch set 2: Adding a custom filter for test regress/udr/TEST108.

Change-Id: Ic827a42ac25505fb1ee451b79636c0f9349d8841

  1. … 98 more files in changeset.
Merge "Miscellaneous DDL and security bug fixes"

  1. … 4 more files in changeset.
cli api to store explain in repository and few more changes.

-- new api to stored packed explain in repository

SQL_EXEC_StoreExplainData.

-- storage of packed explain in encoded form to

mask out any null characters. This helps with

treating of explain data as a string.

-- bug fix in cleanup code

-- fix to handle input param values greater than short max.

-- test code in sqlci frontend to test explain get/store apis.

-- fixed udr/TEST001

Change-Id: I1d501cb93eb7d32808405863afdbdb5c1cec79ee

  1. … 23 more files in changeset.
Miscellaneous DDL and security bug fixes

Fixed a testware issue with fullstack2/TEST062 that occurred during

release testing

Bug 1415196 - Alter volatile table add column cores at CmpSeabaseDDL::alterSeabaseTableAddColumn()

Added a check to not allow add or drop column for volatile tables:

- sqlcomp/CmpSeabaseDDLtable.cpp

Bug 1415232 - A failed create view causes a volatile table to disappear

The code to bind a view does not correctly reset the volatile schema in use

session parameter in case of an error. Subsequent calls do not check for

volatile objects.

parser/BindStmtDDL.cpp

Bug 1371265 - should not allow grants to DB__ROOT or current user

Added a check at grant to prevent this

- sqlcomp/PrivMgrPrivileges.cpp

Bug 1392491 - Unavailability of privmgr metadata error is incomplete

If not all the privmgr metadata is available, then a new Compile context

flag called IS_AUTHORIZATION_READY is set. This flag is adjusted when

a new compiler context is started, and when authorization is enabled and

disabled.

When isAuthorizationEnabled is called and authorization is incomplete,

error 1234 is now returned by default.

After coding changes were added, a request to not check all privmgr metadata

table at context startup was requseted - a performance concern. Fix was

changed to check all tables for debug builds but check only one table for

release builds. If the performance problem is fixed, then we can go back

and check for all privmgr tables.

- arkcmp/CmpContext.h

- arkcmp/CmpContext.cpp

- sqlcomp/CmpSeabaseDDLcommon.cpp

- sqlcomp/nadefaults.cpp

Bug 1402009 - DB__ROOT is unable to grant privilege on object in private schema

When DB__ROOT executes a grant or revoke on objects it does not own, need to

change the grantor from DB__ROOT to the object owner. This matches the same

behavior for other DDL operations such as CREATE.

As part of this fix, the GRANTED BY clause is now allowed for GRANT

statements but it won't be complete until LP bug 1414225 is done.

- sqlcomp/CmpSeabaseDDLtable.cpp (seabaseGrantRevoke)

- sqlcomp/PrivMgrCommands.h

- sqlcomp/PrivMgrCommands.cpp

- sqlcomp/PrivMgrPrivileges.h

- sqlcomp/PrivMgrPrivileges.cpp

- sqlcomp/PrivMgrMD.h

Bug 1414125 - User without priv can view data in metadata tables

Fixes were in place for all metadata tables except the privmgr metadata

tables. The priv information was always being set to none in setupPrivInfo

(NATable) and revoking a privilege was not correctly removing privilege

information from object_privileges.

- optimizer/NATable.cpp

- sqlcomp/PrivMgrCommands.cpp

Bug - create library checking privileges when authorization is not enabled

- CmpSeabaseDDLroutine.cpp

Enhanced the sqlci env command:

- alphabetize the output

- add the following information

-- authentication status

-- authorization status

-- external (LDAP) user connected

A new session parameter called SESSION_EXTERNAL_USER_NAME was added to return

the external user name connected.

A new cli request called SQL_EXEC_GetAuthState_Internal was written to return

the authentication & authorization status. Code was also added, but not yet

supported, for auditing status.

Renamed member/methods that use ldap to external

Changed sqlci env command to return new format

- cli/sqlcli.h

- cli/SQLCLIdev.h

- cli/CliExtern.cpp

- cli/Cli.h

- cli/Cli.cpp

- cli/Context.h

- cli/Context.cpp

- qmscommon/QRQueries.cpp

- sqlci/SqlciEnv.h

- sqlci/SqlciEnv.cpp

- regress/fullstack2/EXPECTED062.SB

- regress/funnstack2/DIFF062.KNOWN.SB.OS

Change-Id: I04627435a0e644c6b14bbf6bd8aa1162d81224fb

  1. … 27 more files in changeset.
repository upgrade and explain enhancements. Details below.

-- repository upgrade infrastructure integrated

with 'initialize trafodion, upgrade'

-- fields have been added/removed/modified in repos tables

-- new repos table has been added

-- upgrade enhanced to only upgrade subset of components that

need to be upgraded. One or more of metadata, repos, views, priv.

-- metadata version now tracks release version and indicates

the traf release where upgrade was needed.

Current metadata version will be 1.1.0.

-- packed explain data is now returned to caller so it could be

stored in repository

-- explain info from repository for a query id could be retrieved

and displayed

explain qid <query-id>;

explain options 'f' qid <query-id>

select * from table(explain(null, 'EXPLAIN_QID=<query-id>'));

-- a sql query could be explained and returned in relational format.

select * from table(explain(null, 'EXPLAIN_STMT=<query-str>));

Users only need to have select permission on tables referenced

in query-str.

-- explain and statistics tables could be invoked

invoke table(explain(null, null));

invoke table(statistics(null, null));

Squashed commit of the following:

commit 09004ff7260b771378bc1a29f0ecb82e5e0c6100

Author: Anoop Sharma <anoop.sharma@hp.com>

Date: Mon Mar 16 12:24:03 2015 -0700

repos and explain, #12

Change-Id: Ia08151b5c7087b6b7daa5b662df2a584e5a7b2a1

commit 46f05ada264094626e45445d3875ccf876cad802

Merge: 3393587 6b3a4c4

Author: Anoop Sharma <anoop.sharma@hp.com>

Date: Sun Mar 15 21:27:48 2015 -0700

Merge remote branch 'gerrit/master' into fix1

commit 3393587de319a3ad43d7d360cd74501e8b8103e4

Author: Anoop Sharma <anoop.sharma@hp.com>

Date: Sun Mar 15 21:26:30 2015 -0700

repos and explain, #11

Change-Id: I895a4b4766e0a92b7472a6b347f0bb3ee07fa9b6

commit cf6ffee549b37ffbcd6ce750a1fdbf7f0c410d61

Author: Anoop Sharma <anoop.sharma@hp.com>

Date: Sun Mar 15 16:07:22 2015 -0700

repos and explain, #10

Change-Id: I19e182ec6600525cec986f173afaf68a44e04af2

commit e331eec68feea59d9fa11e0154e3093762120eb2

Author: Anoop Sharma <anoop.sharma@hp.com>

Date: Fri Mar 13 18:19:03 2015 -0700

repos and explain, #9

Change-Id: I783c75513a432f718d2cd7c6030d410c419a79c8

commit 9e9da221eb45b760963741657fe9cb910a753c84

Merge: 20b64e5 9eb7f37

Author: Anoop Sharma <anoop.sharma@hp.com>

Date: Fri Mar 13 14:52:09 2015 -0700

Merge remote branch 'gerrit/master' into fix1

Conflicts:

sql/common/ComSmallDefs.h

sql/executor/ExExeUtilCli.h

sql/generator/GenRelExeUtil.cpp

sql/sqlcomp/CmpSeabaseDDLcommon.cpp

sql/sqlcomp/CmpSeabaseDDLupgrade.cpp

sql/sqlcomp/CmpSeabaseDDLupgrade.h

Change-Id: Id21b10ff35ad506f409a83a8d955c2e643a2bcf5

commit 20b64e5552d8f614586ed843aad76d87d3473e2e

Author: Anoop Sharma <anoop.sharma@hp.com>

Date: Fri Mar 13 13:44:15 2015 -0700

repos and explain, #8

Change-Id: I2b49d962f08a22ca6c8b437a583855e2344f513c

commit 500b7a9486f23fff6b1c17bb6e79c2f614a7e3e6

Merge: f2c050e fc1c31d

Author: Anoop Sharma <anoop.sharma@hp.com>

Date: Fri Mar 13 08:07:34 2015 -0700

Merge remote branch 'gerrit/master' into fix1

commit f2c050ea048b409b7d6769172e16c4ab26267bd0

Author: Anoop Sharma <anoop.sharma@hp.com>

Date: Fri Mar 13 08:07:15 2015 -0700

repos and explain, #7

Change-Id: I0bc04b6e8f147af1defbfd023c3700e26cc5b6f1

commit fabdb7dadd34238a8a227e880aa868ce484dc580

Merge: 59bb689 74640d7

Author: Anoop Sharma <anoop.sharma@hp.com>

Date: Thu Mar 12 09:01:20 2015 -0700

Merge remote branch 'gerrit/master' into fix1

commit 59bb6893437abe3999a53188503c60cf47633458

Author: Anoop Sharma <anoop.sharma@hp.com>

Date: Thu Mar 12 09:00:44 2015 -0700

repos and explain, #7

Change-Id: I6595a04ad59c1f705a04beee18c877cb81db0fac

commit a86353b45436126915fd2ba74c9af53f8bfd4d19

Author: Anoop Sharma <anoop.sharma@hp.com>

Date: Mon Mar 9 16:21:16 2015 -0700

repos and explain, #6

Change-Id: Ie4fe2b2bece7de3697eb8ada0d1cb1067b89bb88

commit 61272e0d6107536ab8b691415770e856267b6da5

Merge: 1708cb4 ca5dfb4

Author: Anoop Sharma <anoop.sharma@hp.com>

Date: Mon Mar 9 15:52:43 2015 -0700

Merge remote branch 'gerrit/master' into fix1

commit 1708cb44511809bf8e63800d226ac8ea34916726

Merge: 1284f4f 31f25ec

Author: Anoop Sharma <anoop.sharma@hp.com>

Date: Fri Mar 6 14:55:13 2015 -0800

Merge remote branch 'gerrit/master' into fix1

commit 1284f4f78f1f43657f29899cc8b9a96c02f37815

Author: Anoop Sharma <anoop.sharma@hp.com>

Date: Fri Mar 6 14:54:54 2015 -0800

commit: repos and explain, #6

Change-Id: I38d796fd4ab58bf29fa62c4bd5492ce44de9c8a0

commit f639911a1ea4247ae4c297bbbd904b84c7587b3a

Author: Anoop Sharma <anoop.sharma@hp.com>

Date: Mon Mar 2 06:31:21 2015 -0800

repos and explain, #5

Change-Id: Ie97ba3cac8dc78784d0a7eef57176e51b9e16224

commit da69fe9595d4ad5c9adf3b32ae8f848a0e6be964

Merge: 9b95b8c a5caeef

Author: Anoop Sharma <anoop.sharma@hp.com>

Date: Mon Mar 2 06:31:00 2015 -0800

Merge remote branch 'gerrit/master' into fix1

commit 9b95b8c3240a4186704b8b7f2ff8cefebf211732

Merge: ea2f687 8cd8a92

Author: Anoop Sharma <anoop.sharma@hp.com>

Date: Thu Feb 26 08:50:25 2015 -0800

Merge remote branch 'gerrit/master' into fix1

commit ea2f6871b7c7803d0f1e3fa664b474e069a0befd

Author: Anoop Sharma <anoop.sharma@hp.com>

Date: Thu Feb 26 08:50:03 2015 -0800

commit: repos and explain, #5

Change-Id: I44d8f87b3652e6300e41302d3a3e88b9b18652a3

commit 522a47eb991df2245005c693e4d034235adbfdc5

Merge: d986988 f42694e

Author: Anoop Sharma <anoop.sharma@hp.com>

Date: Wed Feb 25 13:13:02 2015 -0800

Merge remote branch 'gerrit/master' into fix1

commit d98698856a4be479a7603b0fb79210e80e88e1c1

Merge: 5841e47 2aad3ef

Author: Anoop Sharma <anoop.sharma@hp.com>

Date: Wed Feb 18 14:00:24 2015 -0800

Merge remote branch 'gerrit/master' into fix1

commit 5841e47f1e3f9a018424298a92cd134f4430e6bf

Author: Anoop Sharma <anoop.sharma@hp.com>

Date: Wed Feb 18 13:57:46 2015 -0800

repos and explain, #4

Change-Id: I01f53003069598381743c9be80b405ed37cefe6d

commit e7b5966d8ce653a57b8128c0b24c15fc05737668

Author: Anoop Sharma <anoop.sharma@hp.com>

Date: Wed Feb 11 11:28:20 2015 -0800

repos and explain, #3

Change-Id: Iff148ef175a7641ea37e27e60435ccf8f613e6db

commit aba98dc8cfda62d4e708cd892ed626f2a3051eb0

Merge: 66f1926 19a0c32

Author: Anoop Sharma <anoop.sharma@hp.com>

Date: Tue Feb 10 14:09:58 2015 -0800

Merge remote branch 'gerrit/master' into fix1

commit 66f1926d279f2b408d436b0aa53a3a6a5388901e

Merge: 9d0ef2a b68f59b

Author: Anoop Sharma <anoop.sharma@hp.com>

Date: Mon Feb 9 10:52:35 2015 -0800

Merge remote branch 'gerrit/master' into fix1

Conflicts:

sql/sqlcomp/CmpSeabaseDDLcommon.cpp

sql/sqlcomp/CmpSeabaseDDLtable.cpp

Change-Id: I6348fec3aece2baad9e9570749f43c086416219e

commit 9d0ef2aa01a62ea3e4e0ab35f8ec9687ef3336ef

Merge: aac268d 2cd8fbb

Author: Anoop Sharma <anoop.sharma@hp.com>

Date: Fri Feb 6 14:15:49 2015 -0800

Merge remote branch 'gerrit/master' into fix1

commit aac268d097afbe506dcf76de4d8f49652a1d4df7

Author: Anoop Sharma <anoop.sharma@hp.com>

Date: Tue Feb 3 16:31:17 2015 -0800

repos and explain, #2

Change-Id: Ia2a50159e577cd6ff8110dca35236fa90ea0d260

commit 2e6160411c8facb448e78d387fa3227e7b759c5d

Merge: c6e2fe5 566706d

Author: Anoop Sharma <anoop.sharma@hp.com>

Date: Tue Feb 3 15:21:03 2015 -0800

Merge remote branch 'gerrit/master' into fix1

commit c6e2fe529f9d1e59d3eb26cb145b260d81814443

Author: Anoop Sharma <anoop.sharma@hp.com>

Date: Tue Feb 3 15:16:36 2015 -0800

repository updated and explain changes, #1

Change-Id: I5daf66a2064b46ad65f1200579385965f7f56808

Change-Id: I0211d754524b8c06a05bb7ade20f884ec91340d2

  1. … 58 more files in changeset.
Query Invalidation triggered by DDL, phase 1

This first check-in implements most of the framework which will

be used to complete the QI DDL feature. It redefines the old

security invalidation key (SQL_SIKEY) to handle DDL operations in

addition to REVOKE. In a limited number of DDL operations, the object

UIDs of affected Seabase objects are propagated to all nodes for

use by the compiler to invalidate NATable cache entries, as

well as a limited number of types of cached queries. Later this

month, the framework will be complete by allowing prepared queries

that have already been returned from the compiler to be invalidated.

Then the next step for the framework will be support for invalidating

the HTable cache. Finally an effort will be made to cover all of

the necessary DDL operations and all types of cached queries.

The check-in include a new regression test (executor/TEST122) that

demonstrates the cases that are covered. Specifically, a table will

be dropped and recreated with the same name but different definition

in one sqlci session. In another session, which has already populated

NATable cache and query cache for INSERT, UPDATE, DELETE, SELECT,

SELECT COUNT(*), INVOKE and SHOWDDL statements, those some types

of statements will be resubmitted and correctly compiled.

Change-Id: Ie61ce751089b57ce1894f1764c338e9400bb7b8a

Closes-Bug: #1329358

Implements: blueprint ddl-query-invalidation

  1. … 41 more files in changeset.
Security changes to support authorization

Added support for authorization commands:

- initialize authorization [, drop]

- create/drop roles

- register/unregister components

- create/drop component operations

- grant/revoke object privileges

- grant/revoke role privileges

- grant/revoke component privileges

- updates to GET and SHOWDDL statements

- checking of privileges for DML requests

- checking of privileges for DDL requests

- regression tests added to catman1 library

Fixed a testware problem in catman1 TEST135 and TEST139

Fixed a parser problem introduced by compGeneral/TESTTOK2 which was recently

introduced.

More details:

This delivery was part of code worked on by many people for several

months on a remote branch. This team held bi-weekly meetings

for several months to design and implement these features. These

meetings also included extensive code reviews.

The security features which include authentication (which was delivered

in June) and authorization is turned off by default. The

traf_authentication_setup script located in $MY_SQROOT/sql/scripts needs

to be run to enable both authentication and authorization. This

procedure is described on the Trafodion Twiki page and will be updated once this

delivery completed to include authorzation.

Delivery updates:

Updated traf_authentication_setup to return consistent error messages

and added a comment to ComSmallDefs.h to address a buf size issue for

metadata tables.

Change-Id: I896f1ee006590284653b2c9882901c05b5f2ba22

  1. … 100 more files in changeset.
added support for externalized Sequence numbers.

-- sql statements: create/drop/alter/get/showddl sequence for sequence objects

-- function 'seqnum' to retrieve sequence numbers.

An external spec has been created.

Launchpad #1349985

Code reviewd by Joanie C, Suresh S, Selva and Sandhya.

Full dev regressions run and passed.

Change-Id: Ie11dbab4d24ff6a1106697f7e2253ea895e6c873

  1. … 71 more files in changeset.
Corrects problems w/authetication error reporting

1) Error not reported when authentication fails (LP 1321059)

2) Authentication error reporting should be moved out of CLI (US

1036)

In addition, the template authentication configuration file refers to

the wrong script for enabling authentication. (LP 1327302)

Analysis and Design:

When support for Zookeeper was added to MXOSRVR, authentication was not

active. The code to retrieve and handle errors was moved later in the

connection logic. Unfortunately, by the time it was invoked, the error

was no longer available, and a generic error was reported by an

exception handler. The check is move back to its previous location.

Previously the CLI would load authentication error codes and text into

the diags area, and then MXOSRVR would extract it. Now, MXOSRVR simply

generates its own error, although currently it looks a lot like the old

error (minus the CLI reference).

After the authentication configuration template was delivered, the name

of the script to enable authentication was chosen. The template was not

updated due to time constraints.

Externals:

Previously the error messages from authentication were of the form:

*** ERROR[8837] CLI Authentication : User: external-username : invalid

username or password

The new format is:

*** ERROR[8837] Invalid username or password. User: external-username

The format of this message will likely change, but for now the original

CLI range error is retained.

Change-Id: I8ab58404941aaf50319883071bb4142cbaafb557

  1. … 11 more files in changeset.
Initial code drop of Trafodion

  1. … 4886 more files in changeset.