Clone Tools
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
Migrate from log4cpp to log4cxx

This change is a wholesale removal of log4cpp from source tree.

log4cxx is an external library installed via RPM, or user build

to default /usr/lib64 and /usr/include directories. Some of the

QRLogger and CommonLogger code was changed to use the new log4cxx

APIs.

Change-Id: I248bac0a8ffbfea6cbc1ba847867b30638892eae

  1. … 208 more files in changeset.
Squashed commit of the following:

commit 94e16c214c20c3ab74bf8bf7cef5dedc91285488

Author: qchen <qifan.chen@hp.com>

Date: Fri Jun 5 16:55:51 2015 +0000

port AS, turn of AS for now

commit 1311090436706506c9ce4e8d455c425d896812c8

Author: qchen <qifan.chen@hp.com>

Date: Fri Jun 5 16:54:55 2015 +0000

port AS, rework

commit 7377923f5ef036adcb07e50ec92aca2625ed3fe0

Author: qchen <qifan.chen@hp.com>

Date: Thu Jun 4 01:49:17 2015 +0000

AS porting

Change-Id: I6ba4349409cbee8d344cba97802febf9fec5c1ea

  1. … 6 more files in changeset.
Costing and statistics compiler interfaces for UDFs

blueprint cmp-tmudf-compile-time-interface

bug 1433192

This change adds compiler interfaces for UDFs that give information

about statistics of the result table and also a cost estimate. It also

has more code for the upcoming Java UDF feature, retrieving updated

invocation infos and returning them back to the executor/compiler C++

code.

Description of the changes in more detail:

- Addressed remaining review comments from my last checkin,

https://review.trafodion.org/1655

- Make sure that user-generated exceptions during deallocation of

a routine are reported. These happens in the destructor of the

object derived from tmudr::UDR. For Java, we may need a deallocate

method.

- Java and JNI code to serialize the updated UDRInvocationInfo and

UDRPlanInfo object after calling the user code and return them back

through the JNI interface to the calling C++ code.

- The cost method source files had some inline methods defined in

the .cpp file and used an include file that included other .cpp

files. Make didn't pick up changes made in these files. Removed

this code and changed it to regular methods and inlines.

- Replaced some Context * parameters in costing with PlanWorkSpace *,

to be able to get to UDF-related info that's stored in a special

PlanWorkSpace.

- Changed the behavior or isBigMemoryOperator() for TMUDFs. If the

UDF writer specifies the DoP for the UDF invocation, then consider

it a BMO.

- If possible, synthesize the HASH2 partitioning function of a TMUDF's

child as the partitioning function of the UDF. This can be done if

the partitioning key gets passed through the UDF.

- Statistics interface for TMUDFs:

- TMUDF now populates statistics field in the UDRInvocationInfo

object and calls the describeStatistics() method.

- Added an estimated # of partitions for partitioned input tables

of TMUDFs. Also changed row count methods to "estimated" row count.

- Added code to incorporate the information on row count and UEC

provided by the UDF writer into statistics of the TMUDF. This code

is not that suitable for coding it as the default implementation

of describeStatistics(). Therefore, the default implementation of

describeStatistics() does nothing, but the compiler applies some

heuristics in case the UDF writer provides no statistics.

- Changed cost method for TMUDFs to incorporate an estimated cost

per row from the UDF writer. There is no special compiler interface

call to ask for the cost, it can be set from the

describeDesiredDegreeOfParallelism() call and, once supported, from

the describePlanProperties() call. Note that we don't have immediate

plans to support describePlanProperties(), that might come after 2.0.

Patch Set 3: Addressed Dave's review comments.

Patch Set 4: Fixed misplaced copyright in expected file.

Change-Id: Ia9ae076b7ae1fc2968c3d253d6d2d0e1d9a2ea40

  1. … 45 more files in changeset.
Move core into subdir to combine repos

  1. … 10768 more files in changeset.
Move core into subdir to combine repos

  1. … 10622 more files in changeset.
Move core into subdir to combine repos

Use: git log --follow -- <file>

to view file history thru renames.

  1. … 10837 more files in changeset.
port skew buster to Trafodion

1. add simplified TEST062

2. reuse cached partitioning expression when only doVarCharCast

is the same as the when the expression is created in

TableHashPartitioningFunction::createPartitioningExpressionImp().

3. rework

4. fix a time monitor bug reporting incorrect processor time (CPU time

computed from clock() calls)

5. comment out the assert on the size of NATable cache not decreasing.

This is to fix regression failure with seabase/TEST020. Selva will check

in a formal fix that allocates space for NATable objects from a single

heap, and then reenables this assert check.

Change-Id: I9eeee4f36ba8678e90e0ac68a85bfc733599d932

  1. … 26 more files in changeset.
Manageability changes - event mgmt and stats publication

Implements changes to support event management using log4cpp.

Configuration files are located in $MY_SQROOT/conf folder and all logs

files are located in $MY_SQROOT/logs folder

For more information see the blueprint at:

https://blueprints.launchpad.net/trafodion/+spec/eventmanagement

Implements changes for publication of statistics to repository. For more

information see the blueprint at:

https://blueprints.launchpad.net/trafodion/+spec/repositorymetrics

Note:

In this initial delivery publication of statistics is disabled by

default and it can be enabled via DCS property. This code has been

reviewed internally prior to merging with mainline

Documentation:

https://wiki.trafodion.org/wiki/index.php/Trafodion_Manageability

Included timestamp to be part of the primarykey for metric aggregation

table

Addressed some of the comments and incorporated Anoop's change for

repository

Changed the queryBuf size in sql/sqlcomp/CmpSeabaseDDLrepos.cpp to 20000

Modified the sql/regress/seabase/EXPECTED024

Change-Id: I517575233c10b2a8683cdd1d53a2eec96d7c2a6f

  1. … 781 more files in changeset.
Phase 2 for log reader TMUDF

blueprint cmp-tmudf-compile-time-interface

Log reader TMUDF is mostly working now.

Still need to set cqd NUM_PARALLEL_ESPS '<num of nodes>' on clusters.

Still needs more work and more testing.

Still seeing some issues with non-ASCII characters.

// SQL Syntax to invoke this function:

//

// select * from udf(event_log_reader( [options] ));

//

// The optional [options] argument is a character constant. The

// following options are supported:

// f: add file name output columns (see below)

// t: turn on tracing

// d: loop in the runtime code, to be able to attach a debugger

// (debug build only)

// p: force parallel execution on workstation environment with

// virtual nodes (debug build only)

More detailed explanation of changes:

- PredefUdrReadfile.cpp: Work on event log reader TMUDF

- sqludr.*: New method to add formal parameters, allows TMUDF to

accept optional parameters.

- OptPhysRelExpr.cpp:

Made some changes for TMUDFs with arity 0 to avoid asserts

and to be able to call okToAttemptESPParallelism in method

RelExpr::synthPhysicalProperty(). This is needed for leaf

operators (arity 0) that want to initiate parallel execution

and TMUDFs seem to be first in that situation.

Changed TableMappingUDF::synthPhysicalProperty to generate

a partitioning function with multiple partitions (and no

partitioning key, so far) if required.

- ExUdr.cpp,

GenUdr.cpp,

sqlparser.y,

DefaultConstants.h,

nadefaults.cpp,

BindItemExpr.cpp,

BindRelExpr.cpp,

ValueDesc.ppp:

Addressed review comments from last phase, got rid of ALLOW_UDF CQD

- Rel*.h

Rel*.cpp

OptPhysRelExpr.cpp (has other changes as well)

Simple but messy change to add one more parameter to

RelExpr::synthPhysicalProperty

Change-Id: I5549e47c0f019beefd4ec1695ae7abf8c3bd43e3

  1. … 39 more files in changeset.
Merge "TMUDF C++ compiler interface, part of log-reading TMUDF"

  1. … 1 more file in changeset.
TMUDF C++ compiler interface, part of log-reading TMUDF

This is the infrastructure for a new C++ interface for TMUDFs

(table-mapping UDFs). It is used by a new log-reading TMUDF that

is not yet complete, but should be finished in the next few days.

See blueprint cmp-tmudf-compile-time-interface for more info.

Change-Id: I5a74e461462313b6d9722ac0deb21cd16c4b02ce

  1. … 55 more files in changeset.
NJ costing changes.

Change-Id: I1bab98a9cbb6745ad64fa8e8fd0e0011f6ef55c6

  1. … 15 more files in changeset.
Various LP fixes, bugs and code cleanup.

-- removed obsolete code (label create/alter/delete, get disk/label/buffer stats,

dp2 scan)

-- metadata structs are now created as classes and initialized during

creation. LP 1394649

-- warnings are now being returned from compiler to executor after DDL operations.

-- duplicate constraint names now return error.

-- handle NOT ENFORCED constraints: give warning during creation and not enforce

during use. LP 1361784

-- drop all indexes (enabled and disabled indexes) on a table during drop table

and schema now works. LP 1384380

-- drop constraint on disabled index succeeds. LP 1384479

-- string truncation error is now returned if default value doesn't fit in

column. LP 1394780

-- fixed issue where a failure during multiple constraints creation in a create

stmt was not cleaning up metadata. LP 1389871

-- update where current of is now supported. LP 1324679

Change-Id: Iec1b0b4fc6a8161a33b7f69228c0f1e3f441f330

  1. … 54 more files in changeset.
fix cardinality errors for better DoP for OE

Problems fixed are as follows.

1. TableAnalysis::getLocalPredsOnPrefixOfList() prematurely breaks

out of the loop to collect predicates on key columns.

2. TableDesc::getSaltColumnAsSet() does not return the result in

the form of base columns valueIds.

3. When the dop is estimated lower than the low bound of ESPs, the

low bound value is used.

4. SimpleFileScanOptimizer::computeSingleSubsetSize() incorrectly

applies the computed predicates on salted column which reduces the

subset size by CQD HIST_NO_STATS_UEC (default to 2). A subset size

specifies the # of rows processed.

Change-Id: I85bebf0d31d8f3e2db5bfe7c00cb4467b53308ac

  1. … 6 more files in changeset.
Index-join scan trimming heuristics rework II

Change-Id: I9cf63be7967a012559e27dc4ca950bb28b8ccd3b

  1. … 15 more files in changeset.
rework fix to move global variabls to optDefaults

Change-Id: I70303eb6c2587fe7c151e0737977d2e1802054cf

  1. … 2 more files in changeset.
delete three new data members in ~OptDefaults

Change-Id: Ibd757043e780bb1dddcaebdf70f7fdcd16399f7a

  1. … 1 more file in changeset.
Move three global variables into opDefauls.

Change-Id: I1588c746e1a61418c19d4ded13f5c201c581d0bd

  1. … 13 more files in changeset.
dop check-in rework.

Change-Id: Idb21dfdbdff76228f9634abc0d1feac6e9e1810b

  1. … 1 more file in changeset.
Squashed commit of the following:

commit 221d4199001b3f06d5629b82ed1281bdfb95f043

Author: qchen <qifan.chen@hp.com>

Date: Mon Sep 8 19:32:05 2014 +0000

DoP: use a better version of resource estimator

Change-Id: Idf6ea5caa7e4915c65b4d54f58d7e483c26871f4

commit 75c92f255ad3841102757f6188d749db42eaedf3

Author: qchen <qifan.chen@hp.com>

Date: Fri Sep 5 15:55:11 2014 +0000

do not add ESP partition requirement to partial GB leaf1 and leaf2.

commit f7c652adf2953d18f10da4a952f78d3815b5b3d4

Author: qchen <qifan.chen@hp.com>

Date: Fri Sep 5 15:20:40 2014 +0000

finalize

commit a419954b1528ca0d1f84c5298b806b46f561a713

Author: qchen <qifan.chen@hp.com>

Date: Wed Sep 3 23:45:13 2014 +0000

add a check on stats smart ptr before its use.

commit 8a2dcc24f0081d554ec3d69560c050f720da04dd

Author: qchen <qifan.chen@hp.com>

Date: Wed Sep 3 18:31:50 2014 +0000

partial gb root running in Master; use rowcount from stats for Hbase

in AppliedStatMan::getStatsForCANodeId().

commit 55f46ededa9fcb30d38d17be6692e1535b4a274c

Author: qchen <qifan.chen@hp.com>

Date: Fri Aug 29 17:46:00 2014 +0000

improve the logic to disable parallel GB partial root when rows < 5000

commit 743783720d030f38e50a065737c8c218fa47c6b5

Author: Ravisha Neelakanthappa <Ravisha.Neelakanthappa@hp.com>

Date: Wed Aug 20 17:20:09 2014 +0000

Fix for bug 1348317. Enable Adaptive Segmentation logic to

compute DoP based on resource estimation.

Change-Id: I7c2b0756854d5a86d8bde162dad064b4e924233c

Change-Id: Id68e1d8265e478f128ed61d1cabc4849873dae23

  1. … 19 more files in changeset.
bug 1368271 fix

Change-Id: I7a609467e3192854bd2a31b91c1c4ce150a9bf28

  1. … 6 more files in changeset.
changes to embedded compiler logic to fix compiler GUI debugger feature

The compiler GUI debugger feature was affected by changes to enable the

embedded compiler. Recent behavior in the compiler GUI debugger

include displaying internal queries when displaying a user query

and occasionally crashing the sqlci process. The changes to fix this

issue is to ensure that the GUI debugger is only enabled when the

(compiler) context of the user query is the active context. Another change is

to ensure that the GUI debugger is disabled and related structures are

appropriately reset when the user queries fails during compilation.

fixes to the rentrant compiler logic to support spjs so external

compilers are avoided when compiling "call" statements. Changes

were made to file CmpSeabaseDDLtable.cpp method

CmpSeabaseDDL::getSeabaseRoutineDesc.

Moved the global DisplayGraph to CmpStatement class for

readability and maintainability.

Closes-Bug: #1340960

Change-Id: Ia75f889c2ed24efabc4501432cc6b5706094b3e6

Closes-Bug: #1339205

  1. … 17 more files in changeset.
Code cleanup (06042014)

Change-Id: Ib7f2f39e42085e026ac1000e43048c9a185b6976

  1. … 14 more files in changeset.
Source file cleanup

Change-Id: Iba4e0059e8a0f996ea18f8089a87338bfe1930a9

  1. … 32 more files in changeset.
Code Drop Update - 5/23/14

Change-Id: If478e8857cbfa9652227af7ed83cd61dd075a889

  1. … 163 more files in changeset.
Initial code drop of Trafodion

  1. … 4886 more files in changeset.