CliExtern.cpp

Clone Tools
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
TRAFODION - 3218 User still has privilege after user's role has been revoked ...

Partial support for column level privileges with QI support for:

column select

column insert

column references

column update

Also, as part of this, updated privilege code in a couple of areas:

Changed object caching code in NATable and NARoutine to store all privileges

assigned to the object when the object is cached (privDescs_). During the load

operation, the code creates bitmaps (privInfo_) for the current user. Privilege

checks are performed against the user bitmaps (privInfo_). This is in

anticipation for some performance updates when connecting to Trafodion (mxosrvr)

with different users.

Change getRoleList to include the roleID and the granteeID that granted the

privilege. The grantee can be a user or a role.

When a privilege is revoked from a role, send QI keys for every user that has

been granted to role.

  1. … 40 more files in changeset.
Support to provide a locking mechanism for LOB insert/update operations

  1. … 28 more files in changeset.
[TRAFODION-2783] jdbc_test_cdh fails at times with type 2 JDBC driver

Reworked the code as per the comment in the git

  1. … 4 more files in changeset.
[TRAFODION-2783] jdbc_test_cdh fails at times with type 2 JDBC driver dumping core

In case of Type 2 JDBC driver, the Trafodion SQL engine is a library that is dynamic

loaded into the process. Initialization of C++ static objects in the dynamic loaded libraries

are supposed to be done before dlopen returns. But the behavior seems to be nondeterministic when

there are multiple threads or when there are dependent static objects (An static object expects

another to be initialized before it). I think, the order of the initialization is not guaranteed

by the standard.

Refactored the code to initialize static object CharInfo::builtinCollationDB_ as part of the first

CLI call in a thread safe manner.

  1. … 2 more files in changeset.
[TRAFODION-2768] Make Trafodion code base to compile in RH7

  1. … 126 more files in changeset.
TRAFODION-2731 CodeCleanup: Phase 4. Remove legacy/obsolete pragmas

  1. … 392 more files in changeset.
lcov: commit #1

  1. … 333 more files in changeset.
code cleanup, commit #1

  1. … 128 more files in changeset.
TRAFODION-2731 CodeCleanup: Remove obsolete, legacy and unused code

This phase handles the following:

-- removal of code that dealt with:

-- mpalias, NSK, MP, mploc, resource fork, rfork

-- ARLIB, DISK, VOLUME, PFS, compiler version info

-- interpretasrow/IAR, AuditImage, ExtractColumns functions

-- ARKCMP_SINGLE_PROCESS and oneProcess()

-- recompControl, remoteDefaults, rtdu, module

-- latebind thru nsk defines, guardian names, nametype nsk

-- SHADOW implementation

-- MEASURE

-- older sqlcat ReadTableDef

-- DP2_MEMORY, DP2_SPACE, HGB_DP2_MEMORY_LIMIT

-- internal cli methods no longer used by any caller

Code within the following defines is removed if it is obsolete

or the define itself is removed if that feature is always on:

-- removed NA_EIDPROC

-- removed SQLEXP_LIB_FUNC

-- removed NA_CMPDLL

-- removed SQ_PHANDLE_VERIFIER

-- removed SQ_NEW_PHANDLE

-- removed __EID

-- removed ARKFS_OPEN

-- removed STAND_ALONE

-- removed __TANDEM

-- removed NA_C89

-- removed NA_NSK

-- removed SQLEXPORT_LIB_FUNC

-- removed SQLCLI_LIB_FUNC

-- removed CLI_PRIV_SRL

-- removed PRIV_SRL

-- removed NA_LINUX

-- removed NA_HSC_LINUX

-- removed NA_UNIX

-- removed NA_WINNT

-- removed HAVE_MMAP

-- removed NA_NO_C_RUNTIME

-- removed NA_DEBUG_C_RUNTIME(replaced with _DEBUG)

-- removed NA_64BIT usage except in sqlcli.h

-- removed dg64

-- removed SQLEXPORT_LIB

-- removed NA_ARKFS

-- removed NA_IEEE_FLOAT

-- removed NA_GUARDIAN_MSG

-- removed NA_HSC

-- removed NA_TMFNOTYETSUPPORTED

-- removed ERROR

-- removed ERROR_STATE

-- removed SQLERRORS_LIB_FUNC

Contents of these files have been removed.

Next checkin fill remove the files itself from git:

executor/dmeasql.h

executor/ExMeas.h, ExMeas.cpp

executor/tempfile.h, .cpp

executor/rcb.h

executor/stubs.cpp, stubs2.cpp

exp/srlversion.cpp

cli/rtdu.h, rtdu2.h, rtdu.cpp, rtdu.cpp

cli/VicKeyValuePair.h

cli/CliDll.cpp

cli/CliStubsStaticBuild.cpp

cli/globalsrlversion.cpp

cli/globalstubs.cpp

cli/sqlciSRLStubs.cpp

cli/test.cpp

cli/privsrlversion.cpp

common/SqlExpDllDefines.h

common/SqlExportDllDefines.h

sqlcat/enum.h

sqlcat/ReadTableDef.h, cpp

sqlcat/readRealArk.h, cpp

  1. … 460 more files in changeset.
Merge remote branch 'origin/release2.1'

Conflicts:

core/conn/jdbcT4/pom.xml

docs/sql_reference/src/asciidoc/_chapters/olap_functions.adoc

docs/sql_reference/src/asciidoc/_chapters/sql_functions_and_expressions.adoc

install/python-installer/db_uninstall.py

install/python-installer/scripts/copy_files.py

install/python-installer/scripts/hdfs_cmds.py

install/python-installer/scripts/traf_check.py

install/python-installer/scripts/traf_ldap.py

install/python-installer/scripts/traf_setup.py

install/python-installer/scripts/traf_user.py

install/python-installer/scripts/wrapper.py

  1. … 7 more files in changeset.
[TRAFODION-2596] Improve the log4j and log4cxx infrastructure in Trafodion

The following changes are made in the way Trafodion logs the messages:

Writes to a log file based on a component or set of components.

C++ part of the code base:

Component Default log file Configuration File

TM tm_<nid>.log log4cxx.trafodion.tm.config

SSCP sscp_<nid>.log log4cxx.trafodion.sscp.config

SSMP ssmp_<nid>.log log4cxx.trafodion.ssmp.config

All SQL processes trafodion.sql_<nid>.log log4cxx.trafodion.sql.config

mxosrvr

sqlci

tdm_arkesp

tdm_arkcmp

tdm_udrserv

Java part of the code base

TM trafodion.dtm.log log4j.dtm.config

SQL trafodion.sql.java.log log4j.sql.config

By default, the log level is set to INFO for most of the cases. When a message

dominates the log file and if doesn't add value in the current level, it will be

changed to the higher level in the hierarchy. This should help to improve the

usefulness of the log file at the default INFO level.

The existence of an environment variable TRAF_MULTIPLE_SQL_LOG_FILE will revert back

to the old way of logging into multiple files. Then, the configuration file

log4cxx.trafodion.masterexe.config will be used.

Currently, RollingFileAppender appender is used for both modes. Different config files

are used to change this appender when the need arises.

log4cxx.trafodion.udr.config and log4cxx.trafodion.lob.config are removed.

log4j.hdfs.config used as the config file for SQL is renamed to log4j.sql.config

Foundation components logging will be revamped later.

This closes #1070

  1. … 19 more files in changeset.
As debugged by Arvind: Increase size of the buffer that reads cmdline arg

Conflicts:

core/sql/cli/CliExtern.cpp

Changes to support new syntax and implementation empty_blob/empty_clob for insert/updates Changes to support new syntax for updating a lob directly through a lobhandle without scanning the table.This is done via new ExeUtil operator. Changes to add columnname info to LOBMD_ table. This helps in mapping any given lobhandle back to the table and the column. Added new sections to the regression test executor/TEST130 to test these changes

  1. … 40 more files in changeset.
TRAFODION-2327 Reduce I/O when loading objects into caches

For each authorization ID (user, role, or PUBLIC), a bitmap containing the

accumulated privileges (across all grantors) is stored with the object desc.

When the object desc is loaded into cache, the privilege bitmaps associated

with the current user, PUBLIC, and the current users' roles are extracted and

unioned together to calculate the final set of privileges. This unioned list

is used during privilege checking.

Today, an I/O is performed to retrieve the list of roles granted to the current

user for each object loaded into NATable and NARoutine cache. Since this list

does not change unless the current user changes (a new session with a different

user) or a grant/revoke role for the current user is performed, these extra

I/O's are not needed.

To remove the extra I/O's for each object, the list of roles will be stored in

the ContextCli. Therefore, this in-memory role list can be used instead of

rereading metadata.

This checkin creates two new CLI requests:

- GetRoleList - returns the list of roles associated with the user

If the list exists in ContextCli, it returns the stored values

If the list does not exist, it retrieves them from Metadata, stores

them and returns the values

- ResetRoleList - removes the list of roles from ContextCli

The first time GetRoleList is called in a session, the users' roles are

stored in ContextCli. They remain in memory until the session ends and

restarts as a different user, or another process grants or revokes a role

from the current user.

If another process revokes a role from the current user, a query invalidation

key is created. When the revoke role query invalidation key for the current

user is detected, ResetRoleList is called. The next time GetRoleList is called

an updated role list is retrieved from metadata and stored in ContextCli.

If another process grants a role to the current user, there could be two

outcomes. If the current user already has the privilege from another source

then nothing happens. If the current user does not have the privilege, then

one recompilation is attempted. Prior to performing the retry, code was

added to ResetRoleList. The recompilation then gets the latest role list and

either succeeds or fails depending on the granted privileges.

  1. … 16 more files in changeset.
Merge [TRAFODION-2265] PR 750 ex_conv_clause::findInstruction() linearly searches convInstrInfo[]

[TRAFODION-2296] Consistent error reporting in abort, commit transaction. Changes to wait for all RS to respond when there is an exception in abort and commit transaction requests.

When there is an error returned from commit or rollback transaction, the details of the error can be obtained in the following log files

a) $MY_SQROOT/logs/tm_.log in the node that issued this request contains the error message as seen by the TM process in the JNI side. This event may not have transaction id.

b) $MY_SQROOT/logs/trafodion.dtm.log contains more info about this error with the transaction id. These events are logged from java side of TM.

c) In the region server logs of all the regions that participated in the transaction.

These exceptions are visible as error code on the client side. To get the details about the exception, the above logs need to be browsed.

Fixes to avoid tm core upon double delete of incoming messages to it

This closes #768

  1. … 7 more files in changeset.
[TRAFODION-2262] Mxosrvr or java core with the stack trace pointing to log4Cxx functions

The Logger repository needs to be initialized. The method QRLogger::initLog4cxx

initialized the commonLogger infrastructure also.

PR #745 merged to fix this issue exposed a problem in SSMP. This

caused SSMP to dump core when a node is brought down.

SSMP should have create the connection to SSCP on that node.

  1. … 3 more files in changeset.
TRAFODION-2265

Parse ex_conv_clause::convInstrStrInfo at sql process startup

(in sqInit) into a sparsely populated index for quick reference by

ex_conv_clause::findInstruction.

  1. … 2 more files in changeset.
Merge remote branch 'origin/master' into ansharma_simcheck_br

  1. … 2 more files in changeset.
[TRAFODION-2261] Mxosrvr or java core with starting from Java_org_trafodion_jdbc_t2_SQLMXConnection_close [TRAFODION-2262] Mxosrvr or Java core with the stack trace pointing to log4Cxx functions

IpcServerClass was not getting destructed correctly. When the CLI context is deleted, the ESP

server class object an IpcServerClass instance that manages the ESP server started

in the CLI context are also destroyed. Fixed the IpcServerClass and IpcServer destructors

so that there is no memory corruption.

Log4Cxx infrastructure in SQL was recursively attempting to log the messages when there is an

issue with it. This was causing stack corruption.

Also added a code to initialize the log4cxx infrastructure if it is not done already upon

the first CLI call.

  1. … 9 more files in changeset.
similarity check changes, commit #1

  1. … 44 more files in changeset.
Merge remote branch 'origin/master' into lobglobals_fix

Conflicts:

core/sql/cli/Context.cpp

core/sql/exp/ExpLOBaccess.cpp

  1. … 13 more files in changeset.
Changes to move hdfFs handing to the context globals level and remove all disconnects from hdfs from the various components.

  1. … 27 more files in changeset.
[TRAFODION-2031] At times T2 applications dump core at the time of logging error Cores were dumped when SQL tries to log the error message via log4cxx. Log4cxx instance was not initialized in case of T2 driver

[TRAFODION-1956] session defaults was getting corrupted in mxosrvr. COMPILER_IDLE_TIMEOUT wasn't added

in alphabetical order. Also fixed the possible buffer overrun issue with some of the set

commands in SessionDefaults

  1. … 5 more files in changeset.
Merge [TRAFODION-1944] Externalize SQL_EXEC_GetAuthState CLI call

TRAFODION [1944] Check status of authentication and authorization via a CLI call

The SQL_EXEC_GetAuthState_Internal CLI request has been changed to

SQL_EXEC_GetAuthState and moved from SQLCLIdev.h to sqlcli.h so it is available

to external callers.

  1. … 3 more files in changeset.
Merge remote branch 'origin/master' into lob_get_trace

  1. … 8 more files in changeset.
support for get command for LOB tables. Support for tracing LOB operations.

  1. … 30 more files in changeset.
TRAFODION-1910 mxosrvr crashes on Hive query after reconnect (take 2)

NATableDB is caching a pointer to a HiveClient_JNI object

(HiveMetaData::client_), but that object gets deallocated when a JDBC

client disconnects. Fixing this by keeping the HiveClient_JNI around

across sessions.

Selva and Suresh commented on the first fix and suggested to treat both

HBaseClient_JNI and HiveClient_JNI the same and to remove the CLI

interface that's used to delete these objects.

Therefore, the new fix is to remove this CLI call. It gets called from

two places, one is when an ODBC/JDBC connection closes and the other

is from "initialize trafodion, drop". We believe that neither of them

is needed. Note that we have only one object of each type per CLI

context, and that we delete both objects when we delete the context

(ContextCli::deleteMe()), so there are no leaks.

  1. … 3 more files in changeset.
update

  1. … 298 more files in changeset.