Trafodion

Clone Tools
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
Add hbase.snapshot.enabled=true to Cloudera HBase settings

Change-Id: If7d60b14d9eecf3f29a8c05e98df4ca82f6f459d

delete obsolete files. remove mpiw.h.

Change-Id: If5839df5c676d6a07c01e651139fcbebfc89eda9

  1. … 20 more files in changeset.
obsolete.

Change-Id: Id7bdf0bd52891056e388ca7ac6cc922fe45ad8bf

Adding installation of more rpm packages.

Added the following rpm packages:

pixman libXfont libX11 perl-Params-Validate perl-Time-HiRes

Made the optional group packages into separate installs

so we can more easily see which one failed.

Changed the EPEL rpm installation to use wget which will timeout

if unable to reach the download site. The previous method of

using rpm directly could hang forever.

Change-Id: Ifad64df0f6ba493877ce777890ab17a8b193a3ff

Merge "Save proc info before deletion to avoid core"

Merge "added support for externalized Sequence numbers."

    • -7
    • +111
    /sql/sqlcomp/CmpSeabaseDDLtable.cpp
Update version to next release number: 0.8.4

Also change names of debug flavor of package files (tarballs).

Having "-debug" at the end makes it compatible with install tools

and is more standard naming scheme.

Change-Id: I0d5a03202a6c7d096615268444bb2c06ca9b4422

setting a cqd in sqlci does not have effect in embedded compiler

The changed set of CQDs in a cmp context is not propagated to another

cmp context when a switch takes place. The change ensures that the list

of CQDs that are changed in a cmp context (list kep in ControlDB) are

pushed to the cmp context we switch to. The change also ensures that the

list of reset CQDs are also kep in the ControlDB. The reason we need

this list now is that in case a CQD value was changed and was pushed its

value to a CI in a previous switch, we need to make sure that if the CQD

is reset, the default value is pushed during a switch.

Change-Id: I6519c4c99c086929f482d09759358b9ebf7f2171

Closes-Bug: #1347810

    • -14
    • +16
    /sql/regress/executor/EXPECTED013.SB
    • -1
    • +3
    /sql/regress/fullstack2/EXPECTED062.SB
added support for externalized Sequence numbers.

-- sql statements: create/drop/alter/get/showddl sequence for sequence objects

-- function 'seqnum' to retrieve sequence numbers.

An external spec has been created.

Launchpad #1349985

Code reviewd by Joanie C, Suresh S, Selva and Sandhya.

Full dev regressions run and passed.

Change-Id: Ie11dbab4d24ff6a1106697f7e2253ea895e6c873

    • -0
    • +183
    /sql/common/SequenceGeneratorAttributes.h
  1. … 57 more files in changeset.
Avoid pushdown of sampling to HBase if oversampling is used

In general, when a Sample clause is present that specifies random

sampling, the sampling is pushed down to the HBase layer and performed

using a filter. When oversampling (sampling rate > 100%) is used, this

can not be done because the filter used is incapable of returning

multiple copies of a given row. The result would be to return all

selected rows once, instead of a greater number of rows that is

consistent with the oversampling.

This change opts out of sampling pushdown if the sample percentage is

greater than 100%, performing the sampling instead with a Sample

operator within Trafodion.

Note that the function RelSample::isSimpleRandomRelative() formerly

contained a comment stating that it returned FALSE if oversampling was

used, although this was not the case. Rather than adding that check to

the function, the comment was removed and the check for oversampling

was added to RelSample::bindNode() as a separate pushdown check.

isSimpleRandomRelative() is also used as a condition for parallel

execution, which should not be ruled out simply because random

oversampling is used.

Closes-Bug #1340432

Change-Id: I6894be2735cf3f7e5a5838bfcab4e1055a7274cd

Merge "Updated readme file for seabed"

Merge "Fix dependencies."

remove mpi refs.

Change-Id: If5b401af11fbeafb32bc0c5ba39f6d2e6dbe2cfb

    • -23
    • +0
    /sqf/src/seabed/src/versmpipmon.cpp
Update version to next release number: 0.8.4

Also add gitignore file for files created during build.

Change-Id: I017014a54cd39abca8ad9ecd5265fed8a6ec4da0

Merge "Add installation of EPEL rpm."

Save proc info before deletion to avoid core

Change-Id: I663190869c69fe702f0c3a20fe9196bbc082b179

Table-specific split policy for salted tables:

Added a table-specific split policy when we are creating salted

tables. Also added a max file size for salted tables, if the max file

size split policy is used. Both split policy and max file size are

controlled via 2 new CQDs:

HBASE_SALTED_TABLE_MAX_FILE_SIZE (default: 0 - Use HBase default (10 GB))

HBASE_SALTED_TABLE_SET_SPLIT_POLICY (default: ON)

- ON means that salted tables will use ConstantSizeRegionSplitPolicy.

- OFF means that tables (salted or not) will use the default HBase split

policy, unless specified in the HBASE_OPTIONS clause.

Examples:

CREATE TABLE T ... SALT ...

- Uses ConstantSizeRegionSplitPolicy with the default max file size

(10 GB unless changed in hbase-site.xml)

CREATE TABLE T ... <no salt option>

- Uses the default HBase split policy

(IncreasingToUpperBoundRegionSplitPolicy, unless changed in

hbase-site.xml). I’ll work with Marvin to stop changing this

default in our installer.

CREATE TABLE T ... SALT ... HBASE_OPTIONS (SPLIT_POLICY =

'org.apache.hadoop.hbase.regionserver.IncreasingToUpperBoundRegionSplitPolicy');

- Uses IncreasingToUpperBoundRegionSplitPolicy, overriding the CQD

CQD HBASE_SALTED_TABLE_SET_SPLIT_POLICY OFF;

CREATE TABLE T ... SALT ...

- Uses the default HBase split policy

Change-Id: I4e221479e2c05c97d69b4865fefffba7e3e561fe

    • -3
    • +21
    /sql/sqlcomp/CmpSeabaseDDLcommon.cpp
    • -9
    • +66
    /sql/sqlcomp/CmpSeabaseDDLtable.cpp
Fix histograms for primary key of salted tables

Users attempting to use Update Statistics to create a multi-column

histogram (MC) corresponding to the primary key of a salted table may be

unaware that the "_SALT_" column is implicitly prepended to the key as

stated in the Create Table statement, and omit it. This fix will cause

Update Stats to detect a request for a multi-column histograms that

specifies the primary key columns (or a prefix of the full key), and

add _SALT_ to it if missing, and order the MC to match the order of the

columns in the primary key.

The change only affects salted tables, and is only applied if neither

the ON EVERY KEY nor ON EVERY COLUMN clauses is present, because an MC

matching the full primary key is automatically generated in those cases.

A second part of this fix applies to cases where ON EVERY KEY or ON

EVERY COLUMN is specified in an Update Statistics statement on a salted

table. By default, MCs corresponding to subsets of the primary key will

no longer be generated automatically in this case. The cqd

USTAT_ADD_SALTED_KEY_PREFIXES_FOR_MC may be set to 'ON' to cause MCs

for subsets of the primary key to be generated.

Closes-Bug #1336983

Change-Id: I930fef13371d5d773a0df44601cf99a8a4dc8322

Fixes for JDBC T4 DBMetasample - bug#1342453

Change-Id: I82bc2116d3fff3174b0a4feff957f4dcf60c0f69

    • -801
    • +490
    /conn/jdbc_type4/samples/DBMetaSample/README
Improve IUD performance by using direct buffers

All IUD operations except the Bulk load operations now use direct buffers.

There is a

- Direct buffer for multiple rowIDs

- Direct buffer for one or more rows containing the column names and

column valueis

Format for RowID direct buffer

numRowIds + rowId + rowId + …

rowId len is passed as a parameter to Java functions

Format for Row direct buffer

numCols

__

colNameLen |

colName | For each non-null column value

colValueLen |

colValue __ |

The colValue will have one byte null indicator 0 or -1 if column is nullable

The numRowIds, numCols, colNameLen and colValueLen are byte swapped because

Java always use BIG_ENDIAN format. But, the column value need not be byte

swapped because column value is always interpreted in the JNI side.

These direct buffers are wrapped as ByteBuffer and parsed in the java functions

to do the corresponding Hbase operations. Using direct buffers avoids the

following

- Creating temporary Mutation objects

- Creating RowToInsert java objects from these Mutation objects

- Transitioning from JNI to java call to add columns to RowToInsert object

- Iterating through the columns objects to do put on the java side.

Fix for seabase/TEST022 with the merged version. Rowwise format has been

changed recently and corresponding change is done in direct row buffer.

Multiple row delete was not part of transaction earlier because RMInterface

wasn't exposing a API to do delete using List. In patchset 1, multiple

deletes was converted into delete a single row iteratively using transaction.

In patchset 2, RMInterface is changed to support multiple row delete and hence

Trafodion HbaseClient uses this new interface to do multiple row delete with

transaction.

Change-Id: I1617b4236a823040ec3dfefbc66508f3f01868fc

    • -162
    • +322
    /sql/executor/ExHbaseAccess.cpp
    • -112
    • +138
    /sql/executor/ExHbaseIUD.cpp
    • -39
    • +171
    /sql/executor/HBaseClient_JNI.cpp
    • -47
    • +166
    /sql/executor/HTableClient.java
    • -39
    • +104
    /sql/exp/ExpHbaseInterface.cpp
fix bug 1343661(cleanup HBASE partitioning), bug 1347819 and bug 1343566

1343661: please refer to sql/sqlcomp/DefaultConstConstants.h for the

definition of CQD HBASE_PARTITIONING.

1347819: the change is in ExExeUtilLoad.cpp to disable HASH2 for fast load.

1343566: method NADefaults::getTotalNumOfESPsInCluster() now returns the

correct value if CQD PARALLEL_NUM_ESPS is set to an integer value.

Rework 1 to address Dave's review comments.

Rework 2 to address Khaled's review comments as follows. A Boolean flag

(isTrafLoadPrep_) is added to class BinWA to better control the type of

partitioning functions needed for the traf preparation step. When we are

binding all nodes, the flag is set to TRUE which instructs createNAFileSet()

not to create hash2.

Rework 3 to address Hans's review comments. If force to have hash2 and

the partitioning function in the cached table is range, do not return the

cached object.

Rework 4 to address seabase/TEST015 core, which is reported in bug

1349990. Bug 1349990 is fixed in this rework.

Change-Id: I9f886ce5361ff4ede16c592e8a3c4e5ca89d76b7

    • -53
    • +60
    /sql/optimizer/OptPhysRelExpr.cpp
    • -20
    • +20
    /sql/regress/seabase/EXPECTED015
    • -127
    • +127
    /sql/regress/seabase/EXPECTED016
    • -0
    • +146
    /sql/regress/seabase/EXPECTED018
  1. … 5 more files in changeset.
Commands added to help with HW install/uninstall

Added deletion of files left over from install during uninstall.

Adding command to reset Ambari database during install to insure

all files from previous installation are cleaned up.

Edit: Typo

Change-Id: I4819b82de8560328eda45d32c9007c6360734e2d

    • -1
    • +25
    /installer/tools/traf_ambari_uninstall
increase the trafodion.dtm.log file size and rollover count

Also added a (commented out) DailyRollingFileAppender

and the following property associated with it:

log4j.appender.hbaseclient.DatePattern='.'yyyy-MM-dd

So, if one needs to roll the log file daily, simply uncomment the

DailyRollingFileAppender

Change-Id: I5254ff43246ebd34f4737a79dffb3adc346d5d48

Merge "adding a missing filter file for sebase/test015"

Updated readme file for seabed

Change-Id: I22cc69a80539b8cae22fb0a1b3694ed4512fe05c

Merge "Fix for bug 1336106: HDFS files without permission causes core dump"

Merge "Bug 1315567 bug in salted tables with descending VARCHAR columns"

Cleaned up directory for testing monitor

- Added Apache license in files which did not have it.

- Converted end-of-lines in some files from Windows to Linux so all are Linux style.

- Removed trailing spaces.

- Deleted some obsolete code.

- Corrected comments.

Change-Id: I2d5e454f5b2e47ecdc3b2182888bbbd0a94552e4

    • -58
    • +76
    /sqf/monitor/test/childExitChild.cxx
    • -275
    • +293
    /sqf/monitor/test/childExitCtrl.cxx
    • -147
    • +165
    /sqf/monitor/test/childExitParent.cxx
    • -600
    • +618
    /sqf/monitor/test/deathNotice.cxx
    • -150
    • +168
    /sqf/monitor/test/deathUnreg.cxx
    • -196
    • +214
    /sqf/monitor/test/deathWatch.cxx
    • -1668
    • +1668
    /sqf/monitor/test/montestutil.cxx
    • -8
    • +26
    /sqf/monitor/test/persistentProc.cxx
  1. … 16 more files in changeset.
adding a missing filter file for sebase/test015

Change-Id: I36656b44cfef8a49410d608efd5acf2149eb1231

    • -0
    • +29
    /sql/regress/seabase/FILTER015
Fixing pdsh issue.

Added single quotes to each pdsh command so that it will be executed

correctly.

Edit: Removed single quotes around commands with variables.

Edit 2: Changes made from comments by reviewers.

Edit 3: dcs_installer change.

Edit 4: traf_cloudera_uninstaller change.

Change-Id: I2c0b81d8780b8c0c273217e0b0c88db6fe4f60df

    • -127
    • +121
    /installer/tools/traf_ambari_uninstall
    • -21
    • +22
    /installer/tools/traf_cloudera_uninstall