Trafodion

Clone Tools
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
Avoid 'unnecessary' construction of (multiple) String objects.

- when log4j logging level is not set to TRACE or DEBUG.

E.g.: For this Java log4j call:

LOG.trace("Enter foo var1: " + var1 + " var2: " + var2);

Let's say that the log4j logging level is set to ERROR.

The above code will construct multiple String objects

and append operations on them to create a String object that

is then passed to the 'trace()' method where it will

check whether the log level is not TRACE and simply return.

So, we spent a lot of cycles for nothing. The following

changes will help circumvent that.

Thanks to Selva for this suggestion.

Changes:

a)

LOG.trace(...

to

if (LOG.isTraceEnabled()) LOG.trace(....

b)

LOG.debug(...

to

if (LOG.isDebugEnabled()) LOG.debug(....

Change-Id: I111a6cdd7e88bdcf070c9ad52956b15e65c3cdb4

    • -27
    • +27
    /sql/executor/HBulkLoadClient.java
    • -20
    • +20
    /sql/executor/SequenceFileWriter.java
Bug 1376922 Union query on a view returns wrong results

Qifan investigated this bug and found the problem in replacePivs().

Rather than forcing the parent's partitioning function on the child,

the fix takes the child's function and only replaces the PIVs in it.

Additional changes:

- The replacePivs() method used a ValueIdSet for the PIVs. This should

be a list, since we use multiple PIVs often by position in the list.

- We don't need the code in replacePivs() that fixes predicates in scan

nodes, since we call replacePivs() before calling preCodeGen() on the

child and therefore the child node does not yet have predicates that

refer to PIVs.

- We don't need to replace the partitioning expression anymore, since

it does not refer to any PIVs and we leave the partitioning key

predicates almost unchanged.

- Fixing a small, unrelated, thing: When sourcing in sqenv.sh twice,

it reported an error message, due to a shell variable that didn't

get initialized to an empty string (workstation environment only).

Change-Id: Id8a20c0d958d8ce13edd59849a1418d252b5691d

    • -167
    • +43
    /sql/optimizer/OptPhysRelExpr.cpp
Eliminate heap corruption error in arkcmp

Allocation from heap in ustat/hs_globals.cpp used () instead

[] for char array, which has been corrected.

Change-Id: I630b98b5b01deb28d03931d9c3746d3d3d7741c0

Closes-Bug: 1377260

Adding retry logic to abort,commit,prepare

If a coprocessor makes a call to a region that is not up or

that throws an exception, it fails. The code will add a retry

to that and also add code to refresh the region information if

it is stale.

Change-Id: Ie3d831202f55bcd93d6de3dbf3ba679d2412ebf5

DoP adjustment for small queries. Rework

Change-Id: Idd70130ca07dc4c82eb2a83422093789fd510c81

Fix for bug 1378089

The HTableClient java objects are removed from the cache upon ODBC/JDBC

disconnection. However, in use HTableClient objects are not garbage

collected because of the global reference being held on the JNI side.

This change is to flag the members of the HTableClient for garbage collection

to reduce the amount of leak in java objects. In-use HTableClient objects are

still being leaked. A complete fix will come later.

Change-Id: I10df40ad58cfde9f2c50e612df7c910cbf33726d

Merge "Enable Trafodion 0.98 recovery"

Merge "Bug 1375902 Incorrect range boundaries for queries using ESPs"

Merge "Re-enable Seabase/TEST012 and Seabase/TEST013"

Merge "Removing an extraneous source tree: sqf/src/seatrans/hbase-trx2"

Removing left over processes after HW uninstall

Change-Id: I3b317110cb68d59c4e4523b5017468f799783398

Fix for bug 1376946

Changed ExpHbaseInterface::deleteColumns to

ExpHbaseInterface_JNI::deleteColumns and refactored this method to behave

correctly.

Change-Id: If4f5e88684bf5232e3b5aa605d1071752e8bb68c

Re-enable Seabase/TEST012 and Seabase/TEST013

Change-Id: I56e013370893fbbcf40a1a57d143867789682da6

    • -1
    • +1
    /sql/regress/tools/runregr_seabase.ksh
Enable Trafodion 0.98 recovery

Change-Id: I2c1784bab66b73cd673ded43e88193eaae5edf27

Removing an extraneous source tree: sqf/src/seatrans/hbase-trx2

Reason:

We inadvertently created a directory 'hbase-trx2' (when we were working on HBase0.98 in a side-branch). So want to clean that up.

Change-Id: I74c018e179786abefe80c49c5c0804cf0ab269ff

    • -67
    • +0
    /sqf/src/seatrans/hbase-trx2/Makefile
    • -56
    • +0
    /sqf/src/seatrans/hbase-trx2/README.txt
    • -29
    • +0
    /sqf/src/seatrans/hbase-trx2/build_trx
    • -171
    • +0
    /sqf/src/seatrans/hbase-trx2/pom.xml
  1. … 46 more files in changeset.
Fix for initialize authorization failure

and reenabled catman1 TEST133:

Initialize authorization creates a set of metadata tables and then loads

data into the OBJECT_PRIVILEGES table to specify object ownership values

It also loads data into the ROLE_USAGE table to specify role ownership

values. Sometimes these insert..selects fail with an error 8102. This

change does not fix the 8102 problem but fixes the code so the insert ..

select succeeds. Bascially the insert was changed to an upsert and

sanity checks placed around calls to make sure the correct number of

rows were processed.

insertSelect code changes:

Added select count(*) from target table to make sure target table empty

Changed insert to an upsert command

After upsert added select count(*) on target table to get rows found

Compare rows in target table with expected rows -> return an error if

not equal.

This fixes the problem. The upsert ignores duplicate rows so we

avoid the 8102 error. The inserted versus expected number of rows make

sure the correct number of rows were processed.

Change-Id: Id701d031ab9b9c2ebdc0584b01a2b5af9fc02b26

    • -1
    • +1
    /sql/regress/tools/runregr_catman1.ksh
    • -12
    • +43
    /sql/sqlcomp/PrivMgrPrivileges.cpp
Bug 1375902 Incorrect range boundaries for queries using ESPs

Reading unsalted, multi-region tables in parallel was very imbalanced,

except for the very first time a table was accessed in a session.

The problem was that NATable caching involves unparsing the region

boundaries, and we created these region boundary values in a way

that could not be unparsed. So, when we used an NATable object from

the NATable cache, the region boundaries had dummy values instead

of the real ones.

The fix adds code to create valid SQL literals from binary values,

to be used in the unparse method. Moving to HBase 0.98 also required

an additional fix, to handle partial values in HBase region start keys.

A region start key in an HBase 0.98 table can be a prefix of an

actual row key.

There is still a problem with keys that use the interval type,

but it is probably a day 1 issue. Since it is late in the 0.9

release, I'm deferring that to the next release.

Change-Id: Id0788a87641e7201723b5f8014215a36c87b7e23

Merge "a) undo disable txns for DDL change b) fix USE_TRANSACTIONS_SCANNER setup"

Native Hbase access improvments

Native hbase access via Trafodion SQL engine now utilizes the enhancements

made for Trafodion table like pushing mutliple rows to JNI and pre-fetch.

Also removed the dependency on the ResultIterator for the native hbase access.

This reduces yet another object created in HTableClient and improves java

memory usage on the client side.

Cleaned up state changes in the HBase access operators and removed

redundant code in the JNI layer.

Change-Id: I70ab52917aac64b68b3816b8ad834842a4d8745e

    • -187
    • +134
    /sql/executor/ExHbaseIUD.cpp
    • -181
    • +148
    /sql/executor/ExHbaseSelect.cpp
    • -469
    • +146
    /sql/executor/HBaseClient_JNI.cpp
a) undo disable txns for DDL change b) fix USE_TRANSACTIONS_SCANNER setup

Change-Id: Ib37e202b9239305bd1e38e2761b587a4316ee439

removing quasi secure mode from bulk load

in HBase .98 permissions on the /bulkload will controlled

using Access Control Lists (ACL). Quasi secure was introdiced

because HBase .94 did not have ACLs.

Change-Id: I1427d6e7ab639417010875b10a01b60696790dee

    • -83
    • +0
    /sql/executor/TrafBulkLoadClient.java
Disabling dtmci 'status regions'

Disabling 'status regions' functionality from dtmci since it needs

to be upgraded to work for hbase98.

Change-Id: I97679a6d4a71713130e62cf266318eab041fac98

Code to determine HBase version, renaming of scripts

Now using hbase shell to determine HBase version. trafodion_mods will

run the correct subscripts depending on if HBase is 0.98 or 0.94.

Renamed ambari_uninstall script to hortonworks_uninstall to clear up

some confusion and to make deleting stray ambari processes easier.

Deleting more stray rpm's in traf_cloudera_uninstall.

Change-Id: I1cce1cfdfd652ab56e9e292d92f2cbc0488a66bc

    • -131
    • +0
    /installer/tools/ambari_uninstall
    • -0
    • +131
    /installer/tools/hortonworks_uninstall
    • -209
    • +0
    /installer/tools/traf_ambari_uninstall
    • -1
    • +15
    /installer/tools/traf_cloudera_uninstall
    • -0
    • +215
    /installer/tools/traf_hortonworks_uninstall
Delivery of the Migration to HBase 0.98 branch

Change-Id: I410b90e0730f5d16f2e86a63cbffe4abaf9daa5d

    • -0
    • +26
    /conn/jdbc_type2/native/SQLMXCommonFunctions.cpp
    • -1
    • +1
    /conn/jdbc_type2/native/SrvrOthers.cpp
    • -1
    • +1
    /conn/odbc/src/odbc/nsksrvr/SrvrMain.cpp
    • -1
    • +1
    /sqf/export/include/dtm/tmtransaction.h
    • -0
    • +8
    /sqf/sql/scripts/cp_search
    • -0
    • +5
    /sqf/sql/scripts/cp_separator
    • -54
    • +79
    /sqf/sql/scripts/install_local_hadoop
    • -0
    • +269
    /sqf/sql/scripts/orc/OrcFileRead.java
  1. … 276 more files in changeset.
Support for SALTED index.

CREATE INDEX supports a new clause "SALT LIKE TABLE".

This causes SALT column to be leading column in the index

Duplicate columns in the index table are now eliminated. This is a bugfix.

Showddl and Invoke will show the SALT syntax and column respectively for

the index. CREATE index also supports HBASE_OPTIONS clause.

A bug seen when NULLABLE partitioning columns are used is also fixed.

Patch Set 2 : All rework from Patch Set 1, excpt for NATable.cpp

Patch Set 3 : Rework in NATable.cpp. Thanks to Hans for all help. Nullable

partition key columns will now generate evenly distributed dataflow through

esps.

Patch Set 4: Fix 3 issues found for the work done in Patch Set 3. Change is only in NATable.cpp

Change-Id: If378ffca29ee83dd4b7928c784b8d34d76f50049

  1. … 11 more files in changeset.
Merge "Delivering the first version of the trafodion_scanner utility."

Estmate NHase row count when stats not available

When statistics are not available for a table, give the optimizer

a better estimate of its cardinaly than the default value by

reading summary information from the trailer block of the table's

HFiles.

blueprint estimate-rowcount

Change-Id: Ie4a44bc7c87385c551eef96478147117b151ab9b

Bug 1363162.Added a change #2 suggested after code review.

Change-Id: I8acf9b5598cc8f7ac3dbcdec59b7814bc77c680e

Fixed private key file check and odd characters in license prompt.

The prompt for accepting the license had microsoft word quote characters

in it which did not render as quotes on some terminal connections.

change the characters to standard ascii quote characters.

The test for private key file was missing a logical not operator.

Change-Id: Ia628cde5008674f8b7fba85e221a790e664fe6d7

Merge "Fix for Bug #1373530. Zookeeper quorum was wrong in dcs-site.xml file."