Clone Tools
Constraints: committers
Constraints: files
Constraints: dates
Log reading TMUDF, phase 3

blueprint cmp-tmudf-compile-time-interface

- Addressed review comments from phase 2. See

- Added a "parse_status" column to the TMUDF, see

updated syntax below

- Added versioning info to new DLL

- EVENT_LOG_READER TMUDF now should choose the correct

degree of parallelism without the need for CQDs

- Brought back the REPLICATE PARTITION keyword, which

is used in the TMUDF syntax. This should fix the failure

in regression test udf/TEST108.

- Some remaining issues:

- Newlines in the error message are not handled well,

at best the additional lines are lost, at worst

they will cause parse errors

- log_file_node output column is always 0

- Code is not yet integrated with changes to event


- Not yet tested on clusters

Updated syntax for the log reader TMUDF:

SQL Syntax to invoke this function:

select * from udf(event_log_reader( [options] ));

The optional [options] argument is a character constant. The

following options are supported:

f: add file name output columns (see below)

t: turn on tracing

d: loop in the runtime code, to be able to attach a debugger

(debug build only)

p: force parallel execution on workstation environment with

virtual nodes (debug build only)

Returned columns:

log_ts timestamp(6),

severity char(10 bytes) character set utf8,

component char(24 bytes) character set utf8,

node_number integer,

cpu integer,

pin integer,

process_name char(12 bytes) character set utf8,

sql_code integer,

query_id varchar(200 bytes) character set utf8,

message varchar(4000 bytes) character set utf8

if option "f" was specified, we have four more columns:

log_file_node integer not null,

log_file_name varchar(200 bytes) character set utf8 not null,

log_file_line integer not null,

parse_status char(2 bytes) character set utf8 not null

(log_file_node, log_file_name, log_file_line) form a unique key

in the result table. parse_status indicates whether there were

any errors reading the information:

' ' (two blanks): no errors

'E' (as first or second character): parse error

'T' (as first or second character): truncation or over/underflow


'C' (as first or second character): character conversion error

Change-Id: Iee3fc8383d4125f0f9b6c6035aa90bb82ceee92e

  1. … 10 more files in changeset.
Phase 2 for log reader TMUDF

blueprint cmp-tmudf-compile-time-interface

Log reader TMUDF is mostly working now.

Still need to set cqd NUM_PARALLEL_ESPS '<num of nodes>' on clusters.

Still needs more work and more testing.

Still seeing some issues with non-ASCII characters.

// SQL Syntax to invoke this function:


// select * from udf(event_log_reader( [options] ));


// The optional [options] argument is a character constant. The

// following options are supported:

// f: add file name output columns (see below)

// t: turn on tracing

// d: loop in the runtime code, to be able to attach a debugger

// (debug build only)

// p: force parallel execution on workstation environment with

// virtual nodes (debug build only)

More detailed explanation of changes:

- PredefUdrReadfile.cpp: Work on event log reader TMUDF

- sqludr.*: New method to add formal parameters, allows TMUDF to

accept optional parameters.

- OptPhysRelExpr.cpp:

Made some changes for TMUDFs with arity 0 to avoid asserts

and to be able to call okToAttemptESPParallelism in method

RelExpr::synthPhysicalProperty(). This is needed for leaf

operators (arity 0) that want to initiate parallel execution

and TMUDFs seem to be first in that situation.

Changed TableMappingUDF::synthPhysicalProperty to generate

a partitioning function with multiple partitions (and no

partitioning key, so far) if required.

- ExUdr.cpp,








Addressed review comments from last phase, got rid of ALLOW_UDF CQD

- Rel*.h


OptPhysRelExpr.cpp (has other changes as well)

Simple but messy change to add one more parameter to


Change-Id: I5549e47c0f019beefd4ec1695ae7abf8c3bd43e3

  1. … 39 more files in changeset.
Various LP fixes, bugs and code cleanup.

-- removed obsolete code (label create/alter/delete, get disk/label/buffer stats,

dp2 scan)

-- metadata structs are now created as classes and initialized during

creation. LP 1394649

-- warnings are now being returned from compiler to executor after DDL operations.

-- duplicate constraint names now return error.

-- handle NOT ENFORCED constraints: give warning during creation and not enforce

during use. LP 1361784

-- drop all indexes (enabled and disabled indexes) on a table during drop table

and schema now works. LP 1384380

-- drop constraint on disabled index succeeds. LP 1384479

-- string truncation error is now returned if default value doesn't fit in

column. LP 1394780

-- fixed issue where a failure during multiple constraints creation in a create

stmt was not cleaning up metadata. LP 1389871

-- update where current of is now supported. LP 1324679

Change-Id: Iec1b0b4fc6a8161a33b7f69228c0f1e3f441f330

  1. … 54 more files in changeset.
Identity column and sequence numbers support.

Added support for IDENTITY columns.

Finished sequence numbers functionality.

Bug fixes and perf enhancements in those areas.

This code has been pre-reviewed by Joanie C.

Change-Id: I0445bc9765b60becb9adf8c053c05344395aecaa

  1. … 94 more files in changeset.
Changes to support OSS poc.

This checkin contains multiple changes that were added to support OSS poc.

These changes are enabled through a special cqd mode_special_4 and not

yet externalized for general use.

A separate spec contains details of these changes.

These changes have been contributed and pre-reviewed by Suresh, Jim C,

Ravisha, Mike H, Selva and Khaled.

All dev regressions have been run and passed.

Change-Id: I2281c1b4ce7e7e6a251bbea3bf6dc391168f3ca3

  1. … 143 more files in changeset.
Security changes to support authorization

Added support for authorization commands:

- initialize authorization [, drop]

- create/drop roles

- register/unregister components

- create/drop component operations

- grant/revoke object privileges

- grant/revoke role privileges

- grant/revoke component privileges

- updates to GET and SHOWDDL statements

- checking of privileges for DML requests

- checking of privileges for DDL requests

- regression tests added to catman1 library

Fixed a testware problem in catman1 TEST135 and TEST139

Fixed a parser problem introduced by compGeneral/TESTTOK2 which was recently


More details:

This delivery was part of code worked on by many people for several

months on a remote branch. This team held bi-weekly meetings

for several months to design and implement these features. These

meetings also included extensive code reviews.

The security features which include authentication (which was delivered

in June) and authorization is turned off by default. The

traf_authentication_setup script located in $MY_SQROOT/sql/scripts needs

to be run to enable both authentication and authorization. This

procedure is described on the Trafodion Twiki page and will be updated once this

delivery completed to include authorzation.

Delivery updates:

Updated traf_authentication_setup to return consistent error messages

and added a comment to ComSmallDefs.h to address a buf size issue for

metadata tables.

Change-Id: I896f1ee006590284653b2c9882901c05b5f2ba22

  1. … 100 more files in changeset.
Squashed commit of the following:

commit 221d4199001b3f06d5629b82ed1281bdfb95f043

Author: qchen <>

Date: Mon Sep 8 19:32:05 2014 +0000

DoP: use a better version of resource estimator

Change-Id: Idf6ea5caa7e4915c65b4d54f58d7e483c26871f4

commit 75c92f255ad3841102757f6188d749db42eaedf3

Author: qchen <>

Date: Fri Sep 5 15:55:11 2014 +0000

do not add ESP partition requirement to partial GB leaf1 and leaf2.

commit f7c652adf2953d18f10da4a952f78d3815b5b3d4

Author: qchen <>

Date: Fri Sep 5 15:20:40 2014 +0000


commit a419954b1528ca0d1f84c5298b806b46f561a713

Author: qchen <>

Date: Wed Sep 3 23:45:13 2014 +0000

add a check on stats smart ptr before its use.

commit 8a2dcc24f0081d554ec3d69560c050f720da04dd

Author: qchen <>

Date: Wed Sep 3 18:31:50 2014 +0000

partial gb root running in Master; use rowcount from stats for Hbase

in AppliedStatMan::getStatsForCANodeId().

commit 55f46ededa9fcb30d38d17be6692e1535b4a274c

Author: qchen <>

Date: Fri Aug 29 17:46:00 2014 +0000

improve the logic to disable parallel GB partial root when rows < 5000

commit 743783720d030f38e50a065737c8c218fa47c6b5

Author: Ravisha Neelakanthappa <>

Date: Wed Aug 20 17:20:09 2014 +0000

Fix for bug 1348317. Enable Adaptive Segmentation logic to

compute DoP based on resource estimation.

Change-Id: I7c2b0756854d5a86d8bde162dad064b4e924233c

Change-Id: Id68e1d8265e478f128ed61d1cabc4849873dae23

  1. … 19 more files in changeset.
Code cleanup (06042014)

Change-Id: Ib7f2f39e42085e026ac1000e43048c9a185b6976

  1. … 14 more files in changeset.
Initial code drop of Trafodion

  1. … 4886 more files in changeset.