Globals.h

Clone Tools
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
Move core into subdir to combine repos

  1. … 10768 more files in changeset.
Move core into subdir to combine repos

  1. … 10622 more files in changeset.
Move core into subdir to combine repos

Use: git log --follow -- <file>

to view file history thru renames.

  1. … 10837 more files in changeset.
Using the language manager for UDF compiler interface

blueprint cmp-tmudf-compile-time-interface

This change includes new CLI calls, to be used in the compiler to

invoke routines. Right now, only trusted routines are supported,

executed in the same process as the caller, but in the future we may

extend this to isolated routines. Using a CLI call allows us to share

the language manager between compiler and executor, since language

manager resources such as the JVM and loaded DLLs exist only once per

process. This change is in preparation for Java UDFs.

Changes in a bit more detail:

- Added 4 new CLI calls to allocate a routine, invoke it, retrieve

updated invocation and plan infos and deallocate (put) the routine.

The CLI globals now have a C/C++ and a Java language manager that

is allocated on demand.

- The compiler no longer loads a DLL for the UDF compiler interface,

it uses the new CLI calls instead.

- DDL syntax is changed to allow TMUDFs in Java (not officially

supported, so don't use it quite yet).

- TMUDFs in C are no longer supported, only C++ and Java are.

Converted remaining TMUDF tests to C++.

- C++ TMUDFs now do a basic verification at DDL time, so errors

like missing entry points are detected earlier. Validation for

Java TMUDFs is also done through the CLI.

- Make sure we have no memory or resource leaks:

- CmpContext keeps track of UDF-related objects allocated on

system heap and in the CLI, cleaned up at the end of a statement

- CLI keeps a list of allocated trusted routines, cleaned up

when a CLI context is deallocated

- Using ExeCliInterface class to make the new CLI calls (4 new calls

added).

- Removed CmpCli class in the optimizer directory and converted

tracking compiler to use ExeCliInterface as well.

- Compile-time parameter values are no longer baked into the

UDRInvocationInfo. Instead, they are provided as an input row, the

same way as they are provided at runtime.

- Bug fixes in C++ UDR code, mostly related to serialization and

to multiple interactions with the UDF through serialized objects.

- Added more info to UDRInvocationInfo (SQL access type, etc.).

- Since there are multiple plans per invocation, each of which

can have multiple interactions with the UDF, plans need to be

numbered so the UDF side can tell them apart to attach the

right state (owned by the UDF) to it.

- The language manager needs some functions that are provided by

the process it's running in. Added those (empty, for now) functions

as cli/CliImplLmExtFunc.cpp.

- Added a new class for Java TMUDFs, LmRoutineJavaObj. Added methods

to allocate such routines and to load their class as well as to

create Java objects by invoking the default constructor through JNI.

- Java TMUDFs use the new UDR interface (to be provided by Suresh and

Pavani). In the language manager, the container is the class of

the UDF, the external path is the fully qualified jar name. The

Java method name is <init>, the default constructor, with signature

"()V". Some code changes were required to do this.

- Created a new directory trafodion/core/sql/src for Java sources in

the sql engine. Right now, only language manager java

sources are in this directory, but I am planning to move the other

java sources under sql in a future checkin. Suresh and Pavani

will add their UDF-related Java files there as well.

- Renamed the udr jar to trafodion-sql-<version>.jar, in anticipation

of combining all the sql Java sources into this jar.

- Created a maven project file trafodion/core/sql/pom.xml and

changed makefiles to invoke maven to build java sources.

- More work to separate new UDR interface from older SPInfo object,

so that we can get rid of SPInfo if/when we don't support the older

style anymore.

- Small fix to odb makefile, make clean failed when executed twice.

Patch set 2: Adding a custom filter for test regress/udr/TEST108.

Change-Id: Ic827a42ac25505fb1ee451b79636c0f9349d8841

  1. … 98 more files in changeset.
SQL support for phandle verifier, etc.

Changes to use phandle verifier from foundation to distinguish

processes when pids are recycled. Remove some workarounds which

attempted to use process creation timestamps to do the same

thing. Also, use msg_mon_node_down2 to log the reason why the

node is brought down. And removed some unneeded special case

code to free the shared segment semaphore at the startup of

mxssmp.

Change-Id: I4914a26680535222504ecf552fb539b1a817c850

  1. … 21 more files in changeset.
Reworked Transaction handling in SQL

Fixed to inherit the transaction from the received message in ESPs, MXUDR or

any other process when it is passed from master processes.

Reworked switching transaction when the context is switched to handle more than

one connection in a thread. This fixed the following bugs

Bug 1356193 SPJ with resultsets not working thru JDBC T2 driver

Bug 1355038 SPJ with result set failed with ERROR[8841]

ESPs will particpate in a transaction with this change

Change-Id: If91f70060f718818eec25aef206f318262cf47aa

  1. … 4 more files in changeset.
Fix to close cursors in the current context and other changes

When the runtime detects that there is change in the transaction, all the open

cursors associated with the transaction in the context are closed.

Changed the token of the some of the counters in ExHbaseAccessStats

Cleaned up the NAMemory destructor code

Change-Id: Ia9fb7098b408b0eb334771264abca013905f7904

  1. … 6 more files in changeset.
More changes to fix bug 1403410

The globals cli_globals was set in the constructor when the CliGlobals was

being initialzied . This allowed some threads to sneak in and use this

global before it is fully initialized. This was causing unexpected dump at

the time of SQL_EXEC_CreateContext.

This has been fixed.

Also took care of the review comments from the previous check-in. Getting the

unique context id is now made thread safe

Change-Id: I12778c8cfb6e14138ef7349e15e98e2a1741896d

  1. … 3 more files in changeset.
More fixes to make SQL thread safe

Fix for bug 1403334 – T2 multithread tests core dump in SQLCLI_DropContext

NAMemory was not threadsafe in derived memory management. The calls

registerMemory, unRegisterMemory was not threadSafe when the heap is marked

as threadsafe heap. Also, the mutex is now dropped as part of NAHeap

destructor.

Fix for bug 1403410 - With 128 threads thru T2 driver, MTTrans and YCSB tests

crash and generate this core

The context list was being manipulated without acquiring a mutex in some places

These obsolete codes are now removed.

Some of the globals were wrongly declared as thread globals.

Other Fixes

SQL_EXEC_GetDiagnosticsCondInfo2 CLI call is now protected by mutexes

like any other CLI call.

Also removed some of the obsolete codes in the cli layer.

Currently commented out the code to close all the open cursors in all

the contexts when a change in transaction id is detected. We need to detect

the change in transaction at context level and close cursors belonging to

that context only.

Change-Id: I0edb7b7f0a844c8c49995af4c130dd319472a783

  1. … 9 more files in changeset.
SQL syntax to cancel executing query, phase 1

This check-in implements the command "control query cancel qid <qid>".

The command will send a cancel message to the MXSSMP cancel broker,

but since the master executor is not yet sending a query started

message,

no query will be canceled, and if the message is successful SQLCODE

-8026 will be raised. The change also validates the cancel command.

produces new EXPLAIN and SHOWPLAN output and handles errors, as shown

in a new regression test.

Change-Id: I472fca043aa12d684c13581264baf22a7ea5f49b

Implements: blueprint sql-query-cancel

  1. … 11 more files in changeset.
Enable RMS monitoring

The query execution statistics can now be obtained while the query is running

via

a) GET STATISTICS FOR QID <qid>

b) Using STATISTICS table-valued function

c) Using CLI calls – SQL_EXEC_GetStatistics2 and SQL_EXEC_GetStatisticsItems

The statistics can be viewed in accumulated, pertable, progress and operator

stats format.

You can also obtain statistics for each fragment instance and process wise

statistics for each operator of an active query.

Light-weight SQL Offender (LSO) feature is also enabled. With LSO, you can get

1. Queries offending CPU resource

2. Queries blocked in Trafodion Engine

3. Queries that are not active

4. Queries that haven’t finished but blocked in client

5. Queries offending memory resource

Explain in RMS feature can be enabled by setting the

cqd explain_in_rms ‘on’ before compiling a query.

EXPLAIN.. FOR QID <qid> command is now supported.

GET PROCESS STATISTICS FOR <pid> is also supported

Change-Id: I59eb54dd4fd347a37e0b7e7842ee242e553ef60d

  1. … 48 more files in changeset.
SQL Memory allocation tracing and overflow detection

1) Added code to record stack info when given size of memory blocks are

allocated and dump these info out to a file or terminal if the

deallocation is not made at the time the heap is distructed. This works

for heaps constructed from NAMemory.

2) Fixed the memory debug code to detected memory overflow at the

de-allocation time.

These two features are enabled through environment variables in objects

compiled in debug mode. See sql/common/NAMemory.cpp for details.

3) Fixes for memory usage issues found when testing above code.

Change-Id: Id0e180aee3d069de11836904e80a4290b180dc67

  1. … 8 more files in changeset.
Fix for bug 1373179

Native Hbase Table access via Trafodion dumps core.

The error messages passed from Java layer to JNI was not handled

correctly leading to core dump. Fixed the error handling code correctly.

Also flagged three Java Objects in HTableClient for gc earlier to reduce the

java object memory usage.

Separated out the relase and flush HTableClient as two different operations

to ensure that HTableClient can be continued to be used even after flush.

Changed NAString in error routines to use the relevant heap instead of

system heap.

Change-Id: I3fac1b8de4745ec9c6f47fc8f64b483f9cc6ef36

  1. … 10 more files in changeset.
Enabling thread independent JNI objects

This submit includes the following changes:

1.Removes the restrictions that JNI objects can be used in the created

thread only.

2. SQL_EXEC_SwithContext CLI now manages the tidList_ correctly in the

CliGlobals structure.

3. ContextCli is now not associated with a thread.

This should further improve the thread safe concept in the trafodion engine.

This would possibly improve the robustness of mysql front-end to trafodion.

This change is also required to share CmpContext across multiple sessions in

future.

Change-Id: I57d61067f05ad7747159ca8eeede905a27a03c10

  1. … 9 more files in changeset.
Code Drop Update - 5/23/14

Change-Id: If478e8857cbfa9652227af7ed83cd61dd075a889

  1. … 163 more files in changeset.
Initial code drop of Trafodion

  1. … 4886 more files in changeset.