Trafodion

Clone Tools
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
Estmate NHase row count when stats not available

When statistics are not available for a table, give the optimizer

a better estimate of its cardinaly than the default value by

reading summary information from the trailer block of the table's

HFiles.

blueprint estimate-rowcount

Change-Id: Ie4a44bc7c87385c551eef96478147117b151ab9b

Bug 1363162.Added a change #2 suggested after code review.

Change-Id: I8acf9b5598cc8f7ac3dbcdec59b7814bc77c680e

Fixed private key file check and odd characters in license prompt.

The prompt for accepting the license had microsoft word quote characters

in it which did not render as quotes on some terminal connections.

change the characters to standard ascii quote characters.

The test for private key file was missing a logical not operator.

Change-Id: Ia628cde5008674f8b7fba85e221a790e664fe6d7

Merge "Fix for Bug #1373530. Zookeeper quorum was wrong in dcs-site.xml file."

Bug 1335477: Group by query returns different results in 2 builds

Reset the noExePreds flag in the MDAM key when we add

partitioning key predicates during preCodeGen. Those added

predicate can contain executor predicates that must not be

suppressed.

Also fixed an issue found in bug 1306836, where the syntax for

selecting specific partitions (salt buckets) was ignored for

coprocessor aggregation (simple count(*)). This syntax now disables

the coprocessor aggregate.

Change-Id: Id8dc0b7769f31a700bb459c424caada2f447a6ae

Delivering the first version of the trafodion_scanner utility.

Change-Id: I65f97dfaf98cf22943c3b3386cd132980ab4c9de

    • -0
    • +218
    /installer/tools/trafodion_scanner
    • -0
    • +84
    /installer/tools/trafodion_scanner.cfg
Merge "Fix for bug 1373179"

Merge "Start RMS processes by default"

Merge "Removed duplicate file from mpi"

Merge "PrivMgr code review rework/fixes"

Fix for bug 1373591, 1370576, fix for port issue

In traf_ambari_mods the script is now parsing for the port number and using

that to pass into the ambari configs.sh script to change HBase settings. The

script will otherwise use the default of 8080 which may be incorrect.

Bug 1373591 uninstall scripts are now removing distro repos.

Bug 1370576 cloudera and ambari setup scripts are now removing previous

installed files

Change-Id: I4160c8f0b9b246f13fb103465fe15566f541bfc2

    • -0
    • +11
    /installer/tools/traf_ambari_uninstall
    • -3
    • +10
    /installer/tools/traf_cloudera_uninstall
Start RMS processes by default

This change configures instance startup to include an mxssmp and

mxsscp process to run on each node. The performance group has

verified that there is no performance regression when RMS processes

are running.

Even though RMS processes are running, the GET STATISTICS commands

are still not able to return statistics for a query id.

Change-Id: If6321b9e756b5ec94e274d5dc0194fe804973882

Implements: blueprint enable-rms-infrastructure

Merge "1.DCS child server state remains as CONNECT_REJECT"

Fix for Bug #1373530. Zookeeper quorum was wrong in dcs-site.xml file.

The dcs_installer script was incorrectly trying to get the zookeeper

quorum node name in the single node cluster case. In the single

node cluster case the zookeeper quorum is simply the node name.

Change-Id: I69b7f857521ac9fd03fdd63742b6bdbf94ee3c6b

PrivMgr code review rework/fixes

This code has been reviewed and/or tested by some of the members of the

security team.

Overview:

1) Replaced calls to sprintf with calls to std::to_string. This avoids the

problem of the buffer potentially being too small for the written data.

2) Eliminated use of STATUS_INTERNAL error return in PrivMgr code. Usage was

inconsistent and could lead to confusion and errors. Internal errors are

now reported as STATUS_ERROR.

3) Previously construction of internal errors was not supplying the filename.

Two defines were created, one for PrivMgr (PRIVMGR_INTERNAL_ERROR) and one

for SeabaseDDL (SEABASEDDL_INTERNAL_ERROR) to generate the error completely,

using the provided string.

4) Code was aligned where previous changes had left it misaligned.

5) The command GET COMPONENT PRIVILEGES ON component FOR authID was

implemented. Also, the header for the related GET COMPONENT PRIVILEGES ON

component was updated.

6) ALTER USER now supports remapping the DB__ROOT user. Also, the command

now correctly checks the REMAP_USER privilege.

7) Checks for ID/Name mapping were not always checking the return code.

Checks were added and errors (sometimes internal) were added.

8) Buffer size for usernames was previously hardcoded, now

MAX_DBUSERNAME_LEN is used.

9) DROP ROLE now checks for granted privileges prior to removing the system

grant for the role.

10) GRANT/REVOKE ROLE now support the MANAGE_ROLES privilege for using the

GRANTED BY clause. Now a non-DB__ROOT user can grant or revoke a role on

behalf of another user, but only if they have the MANAGE_ROLES privilege.

This authority may be moved to a separate privilege in the future.

11) A member variable (myTable_) was not being freed in the destructor.

12) GRANT/REVOKE ROLE is now checking the return from insert, delete, and

update operations.

Externals:

Previously error messages 1356 and 1357 were missing a parameter.

E.g.

*** ERROR 1356 *** Cannot create the component privilege specified.

Component privilege code for the component already exists.

is now:

*** ERROR 1356 *** Cannot create the component privilege specified.

Component privilege code CR for the component already exists.

Error 1357 reports an existing component operation name.

Internals:

/cli

Context.cpp

Only change is alignment changes from a previous review.

/executor

ExExeUtilGet.cpp

o Fixed the getComponentPrivilegesForUser query-missing a parenthesis.

o Updated the headers generated for GET COMPONENT PRIVILEGES ON component and

GET COMPONENT PRIVILEGES ON component FOR authID.

o Within the work() function, corrected code for handling dual GET COMPONENT

PRIVILEGES command based on presence/absence of param1 (authID).

/generator

GenRelExeUtil.cpp

Reverted component operations back to component privileges for use in dual

command.

/sqlcomp

CmpSeabaseDDL.h

Added SEABASEDDL_INTERNAL_ERROR define.

CmpSeabaseDDLauth.cpp

CmpSeabaseDDLauth.h

o verifyAuthority() in the base class was removed, and the version in the user

class was enhanced to support verifying the authority to remap users.

o Internal errors are now generated using the SEABASEDDL_INTERNAL_ERROR define.

o CmpSeabaseDDLuser::alterUser now supports remap the external username for

DB__ROOT. Previously reserved names were prohibited by the ALTER USER

command, now an exception is made for DB__ROOT if the operation is setting a

new external name.

o CmpSeabaseDDLrole::dropRole now checks for privileges granted to a role

before removing the system grant of the role. Previously the system grant

could have been removed leaving the role ungrantable.

CmpSeabaseDDLcommon.cpp

o Internal errors are now generated using the SEABASEDDL_INTERNAL_ERROR define.

o grantRevokeSeabaseRole() now uses the MANAGE_ROLES privilege to determine if

a user can grant or revoke a role on behalf of another user. Previously

this was restricted to DB__ROOT. This authority may be moved to a separate

component privilege in the future.

o GRANTED BY clause now restricted to DB__ROOT for GRANT/REVOKE COMPONENT PRIVILEGE.

PrivMgrCommands.cpp

o Replaced usage of STATUS_INTERNAL with STATUS_ERROR.

o Removed unused grantRoleToCreator() - CREATE ROLE calls PrivMgrRoles class

directly.

PrivMgrComponentOperations.cpp

o Member variable myTable_ is now deleted when the instance is freed.

o Corrected generation of error message for existing component name or code to

use the correct parameter constructor type.

o Internal errors are now generated with PRIVMGR_INTERNAL_ERROR define.

o UIDToString (which calls std::to_string) used instead of sprintf to convert

component UIDs to strings.

PrivMgrComponentPrivileges.cpp

o Member variable myTable_ is now deleted when the instance is freed.

o Internal errors are now generated with PRIVMGR_INTERNAL_ERROR define.

o UIDToString (which calls std::to_string) used instead of sprintf to convert

component UIDs to strings. Similarly, authIDToString is called to convert

granteeIDs and grantorIDs to strings.

o grantPrivilege() no longer removes WITH GRANT OPTION when a privilege a user

has WITH GRANT OPTION is re-granted to the user without WITH GRANT OPTION.

Instead, the grant is now a nop.

PrivMgrComponents.cpp

o Internal errors are now generated with PRIVMGR_INTERNAL_ERROR define.

o UIDToString (which calls std::to_string) used instead of sprintf to convert

component UIDs to strings.

PrivMgrDefs.h

Removed STATUS_INTERNAL from the list of PrivStatus enums.

PrivMgrMD.h

o Added define PRIVMGR_INTERNAL_ERROR.

o Added static inline functions authIDToString and UIDToString. Respectively

they take an int32_t and int64_t and return a std:string. Internally they

both cast the value to long long int as that is the only equivalent type

supported by the current (4.4.6) gcc compiler.

PrivMgrMD.cpp

o Internal errors are now generated with PRIVMGR_INTERNAL_ERROR define.

o UIDToString (which calls std::to_string) used instead of sprintf to convert

component UIDs to strings.

o A grant of select to the AUTHS table to PUBLIC was removed. This was added

as a workaround, but another workaround (allow SELECT on all metadata tables)

superseded this one.

PrivMgrPrivileges.cpp

o Internal errors are now generated with PRIVMGR_INTERNAL_ERROR define.

o Checks for STATUS_INTERNAL were removed.

PrivMgrRoles.cpp

o Member variable myTable_ is now deleted when the instance is freed.

o Internal errors are now generated with PRIVMGR_INTERNAL_ERROR define.

o authToString is called to convert roleIDs, granteeIDs, and grantorIDs to strings.

o grantRole() no longer removes WITH ADMIN OPTION when a privilege a user has

been granted a role WITH ADMIN OPTION and the role is re-granted to the user

without WITH ADMIN OPTION. Instead, the grant is now a nop.

o The error return from insert or update is checked following a grant as well

as from delete and update following a revoke, and an internal error is

reported if the operation fails.

Change-Id: I6d9fc31455222f28cdc5d0db65dc75fc8bb4a99e

    • -91
    • +75
    /sql/sqlcomp/CmpSeabaseDDLauth.cpp
    • -164
    • +154
    /sql/sqlcomp/CmpSeabaseDDLcommon.cpp
    • -10
    • +13
    /sql/sqlcomp/PrivMgrComponents.cpp
    • -37
    • +14
    /sql/sqlcomp/PrivMgrPrivileges.cpp
    • -101
    • +58
    /sql/sqlcomp/PrivMgrRoles.cpp
Fix for bug 1373179

Native Hbase Table access via Trafodion dumps core.

The error messages passed from Java layer to JNI was not handled

correctly leading to core dump. Fixed the error handling code correctly.

Also flagged three Java Objects in HTableClient for gc earlier to reduce the

java object memory usage.

Separated out the relase and flush HTableClient as two different operations

to ensure that HTableClient can be continued to be used even after flush.

Changed NAString in error routines to use the relevant heap instead of

system heap.

Change-Id: I3fac1b8de4745ec9c6f47fc8f64b483f9cc6ef36

    • -100
    • +16
    /sql/executor/HBaseClient_JNI.cpp
    • -5
    • +26
    /sql/executor/JavaObjectInterface.cpp
Ensure UID returned for MD tables is non-null

When an NATable is created for a metadata table, the UID is not known.

Under certain conditions, this creates a problem whereby the AUTHS

table can not be read, and access to a table is incorrectly denied.

In this fix, the NATable::objectUid() function is changed to do a

lookup of the UID for a metadata table if it is 0, and to store the

result in NATable for future calls.

Change-Id: I5eb58871070e585a7242f92becde7e6a24a8f5f5

Closes-Bug: #1370289

Fix for bug 1356069

Fix for bug 1356069 :

Executor assert may not dump core within embedded compiler

Change-Id: I4fcceb6ba2ae65acc23ff33011e97f8d7c03d2f0

Fix for bug 1370203

Added a check for leftover rpms in traf_cloudera_uninstall. If rpms are

found then they will be removed.

Change-Id: I28f8696405c0147fb422f40138cda7b8d1b3b3fc

    • -12
    • +21
    /installer/tools/traf_cloudera_uninstall
Merge "Fix for bug 1354231."

1.DCS child server state remains as CONNECT_REJECT

DCS server STATE is listed as CONNECT_REJECT after

a failed authentication attempt. It remains stuck

in that state for 10 seconds.

Fixes Bug #1321057

Now mxosrvr state is immediately set to AVAILABLE.

2.DCS child server state is set to CONNECTED even

after user gets "*** ERROR[8837] Internal error

occurred" during authentication (e.g. due to

wrongly configured .traf_authentication_config).

Now mxosrvr state is set to AVAILABLE after the

error is returned to the user.

Change-Id: I55131a2ea6dfb362e6560c86b0398e7ba9e1cc2b

    • -31
    • +31
    /conn/odbc/src/odbc/nsksrvr/SrvrConnect.cpp
Improved performance; less obtrusive logging

Change-Id: I278636f5858b2613ae95a1396af8c13085331649

Fix for leak in Java Objects

The local references weren't explicitly deleted in the JNI functions.

This caused the objects created in the JNI layer to leak. Fixed for all the

functions wherein StringArrayList and ByteArrayList were converted to use

array of objects. Need to investigate other functions as well.

Mike's comments are taken care of. THe comment from the previous submit is

also taken care of

Change-Id: I4c9d0c64c9bb8f4790411ebe80133bfb0944d6ed

Removed duplicate file from mpi

This change removes mpi/mpiCC which was a duplicate of mpi/mpicc.

This fixes bug 1332744, "Two files with same case-insensitive name

in mpi", which made it harder to download Trafodion source to a

Windows system.

Change-Id: I343678e8275cfb820c16cc07df8297b0b8de699d

Fix for bug 1354231.

Perviously, our driver uses NULL, which value is actually 0,

for socket descriptor validation.

POSIX actually allows 0 for a valid socket descriptor.

Thus when the app closes stdin, of which descriptor is 0,

the 1st socket created will use 0 as its descriptor.

The correct behavior is use -1 as invalid descriptor check.

New code initializes the m_hSocket with INVALID_SOCKET(defined

in unix_extra.h, which value is -1),

and as well reset to this value when the socket is closed.

Change-Id: I2f1455d2a107c55da2225313e990f811131a2183

    • -2
    • +2
    /conn/odbc/src/odbc/Common/TCPIPSystemDrvr.cpp
JAVA_HOME now set in dcs-config.sh

The bin/dcs-config.sh script didn't recognize Java 1.7. Both bin/start-dcs.sh

and bin/stop-dcs.sh now use hostname -f when comparing zkutil returned value

to that of hostname.

Change-Id: Ie0a6937099adf5a43cf890ea050b30ba1df066ff

Closes-Bug: 1369686

Merge "Fix for Bug 1367413"

make to_char() SQL function cacheable for SQC

Change-Id: If1b772b7cd1e0990cdc1387b4fa36bc8eae689e7

Closes-Bug: #1370794

Externalized all of dcs.cloud.command to property file dcs-default.xml

Parts of the dcs.cloud.command were hard coded inside of DCS. With

this change all of the command is moved to the existing dcs.cloud.command

property contained in the DCS jar file resources. This allows the user

to manipulate the command without changing any DCS code. It's important

for the user manipulating the command to remember that the command must

return the result back in the form <internal ip addr>,<external id addr>

as DCS parses the string.

Change-Id: Icda9c688b63782a586382ae1f61f2f9c0b763174

Closes-Bug: #1370596

Merge "Better naming consistency in user scripts"