sqlcli.h

Clone Tools
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
jira TRAFODION-3157 Add support for BINARY/VARBINARY datatype

Spec attached to jira.

new test regress/seabase/TEST004 added.

  1. … 86 more files in changeset.
iChanges based on review comments

  1. … 14 more files in changeset.
Fixees to support stroring Udr libraries s Blobs.

  1. … 45 more files in changeset.
[TRAFODION-2430] RMS enhancements

RMS now displays 3 numeric values separated by '|' in "Details"

columns in the default output based on the type of stats entry.

These numeric values are VAL2, VAL3, VAL4 of the statistics virtual

table.

Stats Type VAL2 VAL3 VAL4

ROOT_OPER_STATS MessageBytes MessageCount MemoryAllocated

SE_STATS SE_IO_KBytes SE_IO_SumTime ActRowsAccessed

BMO_STATS Scr_IO_Count bmoHeapAllocated ScrFileCount

VAL1 is operCpuTime and TEXT column contains the PID of the process

ExFastExtractTcb is now instrumented with ExHdfsScanStats.

Hdfs scan and access opearations are now properly instrumented with ExHdfsScanStats

counters.

Lob access is now instrumented and is part of the statistics

entry(ExHdfsScanStats or ExHbaseAccessStats) attached with tcb.

Additionally, the following changes are done

1. Removed obsolete ExLobStats

2. Optimized space usage in RMS shared segments for some of the operators

3. A new class ExStorageEngineStats is now created. This class is

type-defed as ExHbaseAccessStats and ExHdfsScanStats.

  1. … 28 more files in changeset.
Merge PR 1449 [TRAFODION-2963] RMS infrastructure now reads the configured pid Max from /proc/sys/kernel/pid_max

TRAFODION-3121 Enhance 'initialize trafodion' to return progress status

-- init traf will now return status during various steps.

'Started' indication will be returned when that step starts.

'Completed' will be returned when it completes successfully.

'Failed' will be returned if that step fails.

Elapsed time for each step will be shown in {ET hh:mm:ss} format.

-- method CmpSeabaseDDL::initSeabaseMD is rewritten and moved from

file CmpSeabaseDDLcommon.cpp to new file CmpSeabaseDDLinitraf.cpp.

-- priv mgr tables are now always created during init traf.

-- a 'no return status' option has been added if status is not to be

returned during init traf. This is used when init traf is called

internally, for ex, during upgrade.

It is not an externalized option.

-- Output of init traf is shown below.

Note: output is not a printf/cout to stdout.

It is returned as selected rows to the caller.

Apache Trafodion Conversational Interface 2.3.0

Copyright (c) 2015-2017 Apache Software Foundation

>>initialize trafodion;

Initialize Trafodion: Started

Verify User: Started

Verify User: Completed {ET: 00:00:00}

Version Check: Started

Version Check: Completed {ET: 00:00:00}

Create Metadata Tables: Started

Create Metadata Tables: Completed {ET: 00:00:49}

Update Metadata Tables: Started

Update Metadata Tables: Completed {ET: 00:00:06}

Create Schema Objects: Started

Create Schema Objects: Completed {ET: 00:00:20}

Create Metadata Views: Started

Create Metadata Views: Completed {ET: 00:00:05}

Create Repository Tables: Started

Create Repository Tables: Completed {ET: 00:00:16}

Create PrivMgr Tables: Started

Create PrivMgr Tables: Completed {ET: 00:00:29}

Create Library Manager: Started

Create Library Manager: Completed {ET: 00:00:51}

Initialize Trafodion: Completed {ET: 00:03:09}

--- SQL operation complete.

  1. … 36 more files in changeset.
RMS infrastructure now reads the configured pid Max from /proc/sys/kernel/pid_max but limits it to a maximum of 128K. It can be overridden by "PID_MAX" environment variable in ms.env.

If the PID_MAX variable is set to 0, 64K is assumed to be configured pid max.

If the PID_MAX variable is set to less than 32K, then 32K is assumed to be configured pid max.

When a SQL process has a pin greater than the configured pid max, an error message

"Pid , is higher than the configured pid max <configured_pid>" is written to

the SQL log file and disables RMS for those processes. Node won't be brought down.

"Get Statistics for rms " shows the the configured pid max.

The environment variable to set the size of RMS shared segment size is now

changed to RMS_SHARED_SEG_SIZE_MB.

genms change for controlling pid_max and rms shared segment size

  1. … 13 more files in changeset.
TRAFODION-2731 CodeCleanup: Phase 5. Cleanup of Platform.h

Removal of unneeded/unused entries from Platform.h and

their usage from other files.

Removal of some other defines that are no longer used. There

might be a compilation error if an obsolete define is being used.

If that happens, those defines need to be removed.

  1. … 26 more files in changeset.
Ensured that CQD NUM_ESP_FRAGMENTS can take value upto 8. Removed the environment variable concept to set the number of esp fragments. Removed the CQDs ESP_NUM_FRAGMENTS_WITH_QUOTAS and ESP_MULTI_FRAGMENT_QUOTAS. Use the corresponding CQD ESP_NUM_FRAGMENTS and ESP_MUTLI_FRAGMENT instead (cherry picked from commit 72534e89633f3d4b8a61ab26d471aaa8b7f3e12a)

Conflicts:

core/sql/executor/ex_frag_rt.cpp

core/sql/executor/ex_frag_rt.h

core/sql/generator/GenRelMisc.cpp

  1. … 45 more files in changeset.
TRAFODION-2731 CodeCleanup: Remove obsolete, legacy and unused code

This phase handles the following:

-- removal of code that dealt with:

-- mpalias, NSK, MP, mploc, resource fork, rfork

-- ARLIB, DISK, VOLUME, PFS, compiler version info

-- interpretasrow/IAR, AuditImage, ExtractColumns functions

-- ARKCMP_SINGLE_PROCESS and oneProcess()

-- recompControl, remoteDefaults, rtdu, module

-- latebind thru nsk defines, guardian names, nametype nsk

-- SHADOW implementation

-- MEASURE

-- older sqlcat ReadTableDef

-- DP2_MEMORY, DP2_SPACE, HGB_DP2_MEMORY_LIMIT

-- internal cli methods no longer used by any caller

Code within the following defines is removed if it is obsolete

or the define itself is removed if that feature is always on:

-- removed NA_EIDPROC

-- removed SQLEXP_LIB_FUNC

-- removed NA_CMPDLL

-- removed SQ_PHANDLE_VERIFIER

-- removed SQ_NEW_PHANDLE

-- removed __EID

-- removed ARKFS_OPEN

-- removed STAND_ALONE

-- removed __TANDEM

-- removed NA_C89

-- removed NA_NSK

-- removed SQLEXPORT_LIB_FUNC

-- removed SQLCLI_LIB_FUNC

-- removed CLI_PRIV_SRL

-- removed PRIV_SRL

-- removed NA_LINUX

-- removed NA_HSC_LINUX

-- removed NA_UNIX

-- removed NA_WINNT

-- removed HAVE_MMAP

-- removed NA_NO_C_RUNTIME

-- removed NA_DEBUG_C_RUNTIME(replaced with _DEBUG)

-- removed NA_64BIT usage except in sqlcli.h

-- removed dg64

-- removed SQLEXPORT_LIB

-- removed NA_ARKFS

-- removed NA_IEEE_FLOAT

-- removed NA_GUARDIAN_MSG

-- removed NA_HSC

-- removed NA_TMFNOTYETSUPPORTED

-- removed ERROR

-- removed ERROR_STATE

-- removed SQLERRORS_LIB_FUNC

Contents of these files have been removed.

Next checkin fill remove the files itself from git:

executor/dmeasql.h

executor/ExMeas.h, ExMeas.cpp

executor/tempfile.h, .cpp

executor/rcb.h

executor/stubs.cpp, stubs2.cpp

exp/srlversion.cpp

cli/rtdu.h, rtdu2.h, rtdu.cpp, rtdu.cpp

cli/VicKeyValuePair.h

cli/CliDll.cpp

cli/CliStubsStaticBuild.cpp

cli/globalsrlversion.cpp

cli/globalstubs.cpp

cli/sqlciSRLStubs.cpp

cli/test.cpp

cli/privsrlversion.cpp

common/SqlExpDllDefines.h

common/SqlExportDllDefines.h

sqlcat/enum.h

sqlcat/ReadTableDef.h, cpp

sqlcat/readRealArk.h, cpp

  1. … 460 more files in changeset.
Merge PR 1184 [TRAFODION-2420] RMS enhancements

  1. … 1 more file in changeset.
[TRAFODION-2420] RMS enhancements

Added four new counters to provide information about multiple executes

of a prepared statement.

No. of times executed

Min. Execute Time

Max. Execute Time

Avg. Execute Time

  1. … 5 more files in changeset.
[TRAFODION-2688] Need a way to find queries that are stuck in compilation via RMS

When a query is stuck in compilation, the mxosrvr remains in connected state forever.

Connection idle timer doesn't kick in. The query timeout can't cancel this query

because the query hasn't start executing. There is a need to find the process id of the

mxosrvr(master) process that is stuck in compilation via RMS. Then the mxosrvr process can be killed.

To find the queries stuck in compile for more than 30 seconds

cd $TRAF_HOME/export/limited-support-tools/LSO

./offender -s queries_in_compile

This script lists queries that are being compiled for more than the given number of seconds

CURRENT_TIMESTAMP TIME_IN_SECONDS NODE PID QUERY_ID ` SOURCE_TEXT

-------------------------- -------------------- ----------- ----------- ---------------------------------------------------------------------------- ----------------------

2017-07-15 01:54:14.909780 1860 0 5505 MXID11000005505212366841736965000000000000206U3333300_278___SQLCI_DML_LAST__ "select * from partsupp ;

  1. … 4 more files in changeset.
[TRAFODION-2385] linux SQLTYPECODE_BOOLEAN using

  1. … 2 more files in changeset.
[TRAFODION-2385] linux odbc support tinyint,boolean,largeint unsigned

  1. … 4 more files in changeset.
[TRAFODION-2420] RMS enhancements

New counters introduced as part of BMO

InterimRowCount - Counter to keep track intermediate rows

produced by hash join and for intermediate

runs in sort

phase - BMO Phase for hash-join and sort

get statistics for statement s1, options 'cs' now works.

Removed attaching BMO stats to Non-root Paritial and leaf partition hash group by operators.

These operators are non-blocking BMOs.

Introduced 3 phases for hash grby operator to be visible in RMS Stats

options 'sl' now works with get statistics for qid.

The relevant new counters are now part of accumulated statistics type.

Extended the support upto 512 opeartors from 256 operators in RMS

Fixed both Type 2 and mxosrvr code to obtain all the externalized counters in accumualted stats correctly.

  1. … 27 more files in changeset.
[TRAFODION-2294] Fix permissions hole in Explain

  1. … 12 more files in changeset.
[TRAFODION-2420] RMS enhancements

Improved BMO operation accounting. Two new counters are added

a) scratchIOSize - Size in KB of the scratch IO

b) scratchIOMaxTime - Time in microseconds taken by any long pole

ESP instance to do scratch IO operations

The ScrBufSize, ScrBufRead and ScrBufWritten are no longer displayed

in the formatted get statistics command outputs.

Reduced the default value of CQD HBASE_NUM_CACHE_ROWS_MAX to 1024 from

10000 to reduce stress on the client JVM memory.

Disabled the memory pressure triggering in the BMO operators by increasing

the threshold GEN_MEM_PRESSURE_THRESHOLD to 10000 from 100. The memory

pressure detection code was inadvertently enabled when the cap to limit

memory pressure constant to 100 was removed.

Memory pressure triggers in BMO operators will be enabled later when

we understand the memory pressure detection better.

  1. … 14 more files in changeset.
[TRAFODION-2420] RMS enhancements Introduced 2 new counters HeapWM: - "SQL Heap WM" - Tracks the high water mark of the used memory including space buffer and bmo operator memory. It is enough to monitor this counter alone to track the memory utilization of the query.

scrIOTime: "Scr. IO Time" - Tracks the IO time taken to read/write to

the scratch files at BMO operator level

Introduced a 'Single line Format' option to display BMO stats and SE stats

in a single line format to enable exporting the output to Excel easily.

GET STATISTICS FOR QID CURRENT PROGRESS, OPTIONS 'SL'

Removed and cleaned up the code further

Also fixed memory corruption issues seen with GET STATISTICS command.

  1. … 16 more files in changeset.
[TRAFODION-2259] Add TopN stats counter and realign BMO stats.

  1. … 7 more files in changeset.
Merge [TRAFODION-2420] PR-898 RMS enhancements

  1. … 4 more files in changeset.
[TRAFODION-2400] RMS enhancements

Changes as per the review comments

  1. … 1 more file in changeset.
[TRAFODION-2400] RMS enhancements

Removed FsDp2MsgStats and ExeDp2Stats

Introduced a new class ExeSEStats to cater the different

storaage engine supported by Trafodion.

The counters pertaining to storage engine were not getting

in GET STATISTICS FOR QID ... ACCUMULATED mode. Fixed this issuue

Removed references to obsolete Measuree concepts

  1. … 15 more files in changeset.
[TRAFODION-2420] RMS enhancements and cleanup

Removed the following classes pertaining to obsolete storage engine DP2.

ExDP2LeafOperStats

ExDP2InsertStats

ExPertableStats

And also removed some counters belonging to this obsolete storage engine

in other classes.

  1. … 7 more files in changeset.
Changes to support new syntax and implementation empty_blob/empty_clob for insert/updates Changes to support new syntax for updating a lob directly through a lobhandle without scanning the table.This is done via new ExeUtil operator. Changes to add columnname info to LOBMD_ table. This helps in mapping any given lobhandle back to the table and the column. Added new sections to the regression test executor/TEST130 to test these changes

  1. … 40 more files in changeset.
JIRA TRAFODION-2099 Add support for boolean datatype

-- boolean datatype support in create, cast and params

Ex:

create table t (a boolean);

insert into t values (true), (false);

select * from t where a = true;

-- boolean datatype support for hive columns

-- unarith op support. Currently used for '!' operator that

negates a boolean value.

Ex: select * from t where !a = false;

-- support for more tinyint and largeint unsigned conversion/comparison

operations in expressions and pcode

-- some code cleanup: removed reference to downrev compiler code genration

-- create table if not exists as select... syntax support

-- regress/seabase/TEST003 updated with additional tests

  1. … 81 more files in changeset.
support for largeint unsigned: commit #1

  1. … 27 more files in changeset.
JIRA TRAFODION-2060 Add support for tinyint datatype

-- added support for tinyint datatype to be used in create

and cast statements

-- added support for returning tinyint in select list and

input params

-- currently tinyint select list and input params is only enabled for

queries from sqlci. Some support for it has been added to jdbc T4

but need to be completed. Until then, tinyint select/param support

is not enabled for t4/t2/odbc interfaces

-- removed obsolete tandem floating point datatypes

  1. … 62 more files in changeset.
[TRAFODION-2031] At times T2 applications dump core at the time of logging error Cores were dumped when SQL tries to log the error message via log4cxx. Log4cxx instance was not initialized in case of T2 driver

[TRAFODION-1956] session defaults was getting corrupted in mxosrvr. COMPILER_IDLE_TIMEOUT wasn't added

in alphabetical order. Also fixed the possible buffer overrun issue with some of the set

commands in SessionDefaults

  1. … 5 more files in changeset.
TRAFODION [1944] Check status of authentication and authorization via a CLI call

The SQL_EXEC_GetAuthState_Internal CLI request has been changed to

SQL_EXEC_GetAuthState and moved from SQLCLIdev.h to sqlcli.h so it is available

to external callers.

  1. … 3 more files in changeset.