Clone
 

selvaganesang <selva@apache.org> in Trafodion

[TRAFODION-3280] Reduce path length in Trafodion for improved performance and scalability

There are many places in Trafodion code getenv is called repeatedly. Cleaned up code so that

repeated getenv calls are avoided during prepare and execute operations.

    • -3
    • +1
    /core/sql/exp/ExpPCodeOptsNativeExpr.cpp
Fix for phoenix T2 tests in FunkyNamesTest

FunkyNamesTest has been removed temporarily

EsgynDB is not able to support delimited column names correctly. It has always been returning incorrect results.

We have modified FunkyNamesTest to suit our behavior. With the change to move from Array to HashMap for performance

reasons in both T4 and T2 driver code. However, the incorrect results differ between T2 and T4 when there are

more than one delimited column name matches due to uppercasing the column name always. This needs some change

in the descriptor information sent from SQL engine to denote that column names are delimited to preserve case.

We need to use the case preserving key String for delimited column names and case insensitive key String for

regular column names in the HashMap to convert from column names to column index.

    • -2
    • +3
    /core/sqf/sql/scripts/install_traf_components
Fix for the failure of testLTrimFunction, testTrimFunction and testRTrimFunction of VariableLengthPKTest in phoenix T4 tests.

When the statement type is TYPE_UNKNOWN, mxosrvr was executing the extra code

to load controls, GetHashInfo etc. GetHashInfo was corrupting the sqlString by

truncating spaces in between the literals.

Removed these code from mxosrvr because it is specific to QueryStatsServer as part of

NeoView and WMS.

    • -1038
    • +21
    /core/conn/odbc/src/odbc/nsksrvr/SrvrConnect.cpp
Fix to ensure infostats command from trafci works as expected.

Fix to ensure that trafci returns the correct status message of the recent sql command instead of returning "--- SQL operation complete" always.

Optimizing and avoiding unnecessary function calls shown as hot spots in jprofile - part-IIa

Removed ununsed code

Fix for cores seen with T2 JDBC applications caused by PR 1820

New member variables were not initialized in the CliGlobals constructor

Optimizing and avoiding unnecessary function calls shown as hot spots in jprofile - part-II

Changed Hashtable to HashMap where possible with the assumption that the JDBC connection

shouldn't be used from different threads. TrafT4Connection object needs be made thread-safe.

Optimized org.trafodion.jdbc.t4.TrafT4ResultSet.findColumn by using HashMap of columnName to

columnIndex. This should avoid excessive calls to java.lang.String.equalsIgnoreCase.

Removed excessive calls to java.lang.Class.getMethod from TrafT4ResultSet

Cleaned up and ensured that JDBC driver conforms to transaction handling via Connection.commit and Connection.rollback. The transaction is always started by the database engine when it needs the transaction.

However, begin work and commit work is still supported via SQL command but

there is no transaction state is maintained in the driver for these commands.

Optimizing and avoiding unnecessary function calls shown as hot spots in jprofile - part-II

java.lang.String.split and javal.lang.String.replaceAll

JDBC driver was parsing the sql string passed to it by the application to determine the type of

SQL statement and to check if the sql string has parameters. It doesn't parse the sql string anymore

and passes the sql string to SQL engine via mxosrvr. In case of PreparedStatement, the JDBC driver will

set the sqlStmtType based on the query type returned. In case of Statement, the JDBC driver will set the

sqlStmtType as SQL_TYPE_UNKNOWN.

Now, the PreparedStatement is always prepared for setting array values for parameter for all

statement types such as IUD and select. SQL engine will report an error if the array values are

not supported for any statement type.

Optimizing and avoiding unnecessary function calls shown as hot spots in jprofile

java.net.Socket.setSoTimeout

org.trafodion.jdbc.t4.T4LoggingUtilities.log

java.net.Socket.getLocalSocketAddress

Now socket.read waits for the network timeout. It can be configured via a property networkTimeout or

call to connection.setNetworkTimeout. If the networkTimeout is not configured, a default value of 10 secs

is assumed.

The caller is expected to set the timeout depending upon the operation being performed like loginTimeout,

queryTimeout. Socket.read will wake up for every networkTimeout expiration and check if the timeout for the

operation hasn't exceeded. When it exceeds, SocketTimeoutException is thrown.

Changed the T4Statement and T4Restult classes to have "has a" relationship with T4Connection for

better understanding of the code. Earlier, it had "is a" relationship with T4Connection.

Changes as per review comments of PR 1820

Merge branch 'master' of github.com:apache/trafodion into trafodio_3280_2

[TRAFODION-3280] Reduce path length in Trafodion for improved performance and scalability

This PR fixes the following issues to improve performance.

1. When the session is dropped for the default context, we were unnecessarily

dropping HBase connections.

2. There were 2 HBase connections per mxosrvr - one for the default context and another

for repository context. Now it has been changed to one HBase connection per process.

    • -13
    • +12
    /core/sql/executor/HBaseClient_JNI.cpp
[TRAODION-3280] Reduce path length in Trafodion for improved performance and scalability

cli_globals::currContext() is now optimized for default context.

[TRAODION-3280] Reduce path length in Trafodion for improved performance and scalability

Changes to ensure that logging related objects are created and the corresponding

functions are called only when the logging Level is set higher than the recording level

[TRAFODION-3260] SSMP may wait 3 seconds before handling requests The commit 95c2712b131adc2ba06b4e620ad67f93c2658497 introduced a regression with ESPs acting as master to run SQL queries. These SQL queries were stuck at close.

The new and delete operator without namespaces overrides all new and delete operations. This increases the pathlength of memory allocation and deallocation unnecessarily in release mode and the original intent of tracking new and delete not useful anymore. Hence this has been now enabled in debug mode only.

A CQD CANCEL_QUERY_ALLOWED is introduced to complement the similar SSD for ease of setting in defaults table.

1. JULIANTIMESTAMP now uses clock_gettime with CLOCK_REALTIME by default. It can be switched to earlier implementaion via setting env variable USE_LINUX_JULIANTIMESTAMP=0 2. Remove unwanted CQDs set by mxosrvr. GENERATE_EXPLAIN is not turned ON by mxosrvr.

Merge PR 1766[TRAFODION-3234] Refactor hive meta calls

Merge PR 1764 optimize get/set schema

Merge PR 1762 Fix for hive/TEST003 failure

Merge PR 1746 [TRAFODION-3234] Refactor hive meta calls to be less resource intensive to support hive partitions if needed

Merge PR 1735 [TRAFODION-3225] Obscure cores seen in RMS and logger related code when Trafodion is stressed

Merge PR 1706 [TRAFODION-3171] Refactor Hive sequence file reading to use the new implementation

Merge PR 1702 [TRAFODION-3171] Refactor Hive sequence file reading to use the new implementation

Merge PR 1696 [TRAFODION-3110] Refactor LOB access to use the new implementation of HdfsClient

Merge PR 1701 [TRAFODION-3171] Refactor Hive sequence file reading to use the new implementation

Merge PR 1697 [TRAFODION-3171] Refactor Hive sequence file reading to use the new implementation