anoop sharma <> in Trafodion

initial support for returning multiple versions and column timestamps

This feature is not yet externalized.

Support added to:

-- return multiple versions of rows

-- select * from table {versions N | MAX | ALL}

-- get hbase timestamp of a column

-- select hbase_timestamp(col) from t;

-- get version number of a column.

-- select hbase_version(col) from t

Change-Id: I37921681fc606a22c19d2c0cb87a35dee5491e1e

    • -26
    • +428
    • -13
    • +15
  1. … 34 more files in changeset.
drop column fix.

We currently dont support drop of a column which has an

index (implicit or explicit) defined on it.

But drop column was also not working in cases where a secondary

index was present on a column that was not being dropped.

That has been fixed.

Change-Id: I53264b630794bef5c99ffb7a57b8fd2ba0f13bb5

    • -0
    • +38
various fixes and enhancements, details below.

-- improved DDL performance by not invalidating internal create/alter


-- added an optimization during CREATE INDEX to not go through

'upsert using load' processing if source table is empty.

-- added support for ISO datetime format (2015-06-01T07:35:20Z)

-- added support for RESET option to ALTER SEQUENCE and IDENTITY.

This will reset generated seq num to the START VALUE.

-- added support for cqd TRAF_STRING_AUTO_TRUNCATE.

If set, strings will be automatically truncated during insert/update.

-- fixed sqlci to pass in correct varchar param len indicator (2 or 4 bytes).

-- changed sizeof(short) to correct vcindlen (2 or 4 bytes)

-- removed some NA_SHADOWCALLS defines

Change-Id: Ie6715435d9c210ae6c2db4ff6bc0545c1b196979

    • -1
    • +10
  1. … 25 more files in changeset.
added regressions tests, not yet enabled.

Change-Id: I197db96c2e6942b3428b3ca779a0bbbdc1c22525

    • -0
    • +23
    • -0
    • +51
    • -0
    • +79
    • -0
    • +45
    • -0
    • +41
    • -0
    • +21
    • -0
    • +21
    • -0
    • +41
    • -0
    • +41
    • -0
    • +316
    • -0
    • +307
    • -0
    • +41
    • -0
    • +43
    • -0
    • +42213
    • -0
    • +42213
  1. … 1023 more files in changeset.
updates based on review of previous MV checkin.

Change-Id: I8297314909ff2f1932f8c43d6202e62350b8f9a4

  1. … 15 more files in changeset.
Added MV and MVQR related code files.

Initial checkin, code not yet tested or externalized.

Change-Id: I6b56c532dea8bda898febf8bef0709afe8f8a8e3

    • -0
    • +127
    • -0
    • +19
    • -0
    • +44
    • -0
    • +259
    • -0
    • +1454
    • -0
    • +460
    • -0
    • +223
    • -0
    • +269
    • -0
    • +139
    • -0
    • +354
    • -0
    • +161
    • -0
    • +422
    • -0
    • +174
  1. … 231 more files in changeset.
added fullstack2/FILTER062

Change-Id: I2ed813d5674897109fb973124c245cdd1aa789f9

    • -0
    • +43
various lp and other fixes, details below.

-- added support for self referencing constraints

-- limit clause can now be specified as a param

(select * from t limit ?)

-- lp 1448261. alter table add identity col is not allowed and now

returns an error

-- error is returned if a specified constraint in an alter/create statement

exists on any table

-- lp 1447343. cannot have more than one identity columns.

-- embedded compiler is now used to get priv info during invoke/showddl.

-- auth info is is not reread if already initialized

-- sequence value function is now cacheable

-- lp 1448257. inserts in volatile table with identity column now work

-- lp 1447346. inserts with identity col default now work if inserted

in a salted table.

-- only one compiler is now needed to process ddl operations with or

without authorization enabled

-- query cache in embedded compiler is now cleared if user id changes

-- pre-created default schema 'SEABASE' can no longer be dropped

-- default schema 'SCH' is automatically created if running regressions

and it doesn't exist.

-- improvements in regressions run.

-- regressions run no longer call a script from another sqlci session

to init auth, create default schema

and insert into defaults table before every regr script

-- switched the order of regression runs

-- updates from review comments.

Change-Id: Ifb96d9c45b7ef60c67aedbeefd40889fb902a131

    • -22
    • +27
  1. … 55 more files in changeset.
LP 1444044, dup col definitions are not detected

During a create table or create view, duplicate col

names are not being detected and an error is not being

returned. This results in an incorect table or view

being created. An error may or may not be returned

later depending on how that object is being used.

That has been fixed to detect dup cols and return an

error message during create time.

Fixing it now so users dont create incorrect objects

by mistake.

Change-Id: I7ad772adcb067159ab80a487dead8dffc62bb546

    • -0
    • +14
LP Bug 1442949

-- alter add col was not running under an internal Xn.

that caused rollback to not work.

-- change to allow internal cleanup command to proceed if

running under a transaction. This is needed for internal

cleanups after error.

-- better error msg if index population fails.

Change-Id: I0fd85be97a2ab96544cd037f920df0a3f5868542

    • -0
    • +11
    • -51
    • +0
additional changes to support ALIGNED row format.

This feature is not externalized yet.

Change-Id: Idbf19022916d437bb7bb69019194de5057cbcb65

    • -111
    • +65
    • -513
    • +273
  1. … 7 more files in changeset.
metadata fixes and 'sqlmp' code cleanup

-- NATable struct for metadata was being created multiple

times whenever information for a new table was read

from metadata. That has been fixed.

-- an 'initialize trafodion, drop' followed by 'initialize traf'

from the same session was failing due to priv info not getting

reset. This would show up if 'initialize authorization' was

done earlier. That has been fixed.

-- code cleanup mostly related to sqlmp legacy code and reference.

Change-Id: I346e3f3bbc6c7784b38e7e2e1f11d487854c281c

  1. … 40 more files in changeset.
explain enhancements and fixes.

-- support to return explain details from packed explain plan.

select * from table(explain(null, 'EXPLAIN_PLAN=<packed-plan>'))

This enables caller to retrieve the packed explain plan, ship

it to another process and then format it there.

DSM will be using this functionality.

-- sqlci syntax added to test explain enhancement functionality:

get qid for statement s;

store explain for s in repository;

set qid <qid> for s;

-- new tests added to seabase/TEST011

-- some bug fixes to handle 4 byte lengths for

explain plan greater than short max.

-- changed err enums CLI_DESC_NOT_EXSISTS and

CLI_STMT_NOT_EXSISTS to the right EXISTS spelling

(this is just for you, Dave).

-- added missing copyrights

Change-Id: Ic60758fe49790516be125cca7f7e23fe1265feb7

  1. … 18 more files in changeset.
perf enhancement for ddl operations.

DDL operations where objects (tables, views) had large number

of columns were running slow due to single row inserts into

metadata COLUMNS table. This showed up during DSM repository

creation which had 150 tables and 500 views.

Changes done:

-- added code to do rowwise rowsets

-- enhanced metadata COLUMNS inserts to use rwrs upserts

-- changed metadata calls to use upsert instead of insert

-- fixed a cleanup bug

Change-Id: I07b619598e05eab80ec965ac0194614b73ecde57

    • -35
    • +200
fix to repository explain_plan definition

explain_plan column length was not being set correctly

from the define value. That has been fixed.

Change-Id: I8b546da139f4a0c79efda8da2cb1f11ff55580dd

cli api to store explain in repository and few more changes.

-- new api to stored packed explain in repository


-- storage of packed explain in encoded form to

mask out any null characters. This helps with

treating of explain data as a string.

-- bug fix in cleanup code

-- fix to handle input param values greater than short max.

-- test code in sqlci frontend to test explain get/store apis.

-- fixed udr/TEST001

Change-Id: I1d501cb93eb7d32808405863afdbdb5c1cec79ee

  1. … 9 more files in changeset.
LP bug 1435479. Explain query looping.

in current code, executor buffer for returned explain row

was hardcoded at 4K. That caused no buffer to be found if

explain description column value was greater

than 3000(the default). Executor would go in a loop

trying to find space in sql buffer.

In this case, query text caused total explain len

to become greater than the buffer size.

It has been fixed to compute buffer len based on the

returned explain virtual table row size.

Change-Id: I33a806d4baffb18f4311e1320f8af1b759ea5863

review updates and more. See below.

--Code updates from previous review.

--Bug fixes and enablement of ALIGNED FORMAT row storage.

This is for testing only.

--Error handling for explain statement.

Change-Id: Ia01b3813e0b72ce66e2c03cc50777d4f1f8d5a78

  1. … 5 more files in changeset.
repository upgrade and explain enhancements. Details below.

-- repository upgrade infrastructure integrated

with 'initialize trafodion, upgrade'

-- fields have been added/removed/modified in repos tables

-- new repos table has been added

-- upgrade enhanced to only upgrade subset of components that

need to be upgraded. One or more of metadata, repos, views, priv.

-- metadata version now tracks release version and indicates

the traf release where upgrade was needed.

Current metadata version will be 1.1.0.

-- packed explain data is now returned to caller so it could be

stored in repository

-- explain info from repository for a query id could be retrieved

and displayed

explain qid <query-id>;

explain options 'f' qid <query-id>

select * from table(explain(null, 'EXPLAIN_QID=<query-id>'));

-- a sql query could be explained and returned in relational format.

select * from table(explain(null, 'EXPLAIN_STMT=<query-str>));

Users only need to have select permission on tables referenced

in query-str.

-- explain and statistics tables could be invoked

invoke table(explain(null, null));

invoke table(statistics(null, null));

Squashed commit of the following:

commit 09004ff7260b771378bc1a29f0ecb82e5e0c6100

Author: Anoop Sharma <>

Date: Mon Mar 16 12:24:03 2015 -0700

repos and explain, #12

Change-Id: Ia08151b5c7087b6b7daa5b662df2a584e5a7b2a1

commit 46f05ada264094626e45445d3875ccf876cad802

Merge: 3393587 6b3a4c4

Author: Anoop Sharma <>

Date: Sun Mar 15 21:27:48 2015 -0700

Merge remote branch 'gerrit/master' into fix1

commit 3393587de319a3ad43d7d360cd74501e8b8103e4

Author: Anoop Sharma <>

Date: Sun Mar 15 21:26:30 2015 -0700

repos and explain, #11

Change-Id: I895a4b4766e0a92b7472a6b347f0bb3ee07fa9b6

commit cf6ffee549b37ffbcd6ce750a1fdbf7f0c410d61

Author: Anoop Sharma <>

Date: Sun Mar 15 16:07:22 2015 -0700

repos and explain, #10

Change-Id: I19e182ec6600525cec986f173afaf68a44e04af2

commit e331eec68feea59d9fa11e0154e3093762120eb2

Author: Anoop Sharma <>

Date: Fri Mar 13 18:19:03 2015 -0700

repos and explain, #9

Change-Id: I783c75513a432f718d2cd7c6030d410c419a79c8

commit 9e9da221eb45b760963741657fe9cb910a753c84

Merge: 20b64e5 9eb7f37

Author: Anoop Sharma <>

Date: Fri Mar 13 14:52:09 2015 -0700

Merge remote branch 'gerrit/master' into fix1








Change-Id: Id21b10ff35ad506f409a83a8d955c2e643a2bcf5

commit 20b64e5552d8f614586ed843aad76d87d3473e2e

Author: Anoop Sharma <>

Date: Fri Mar 13 13:44:15 2015 -0700

repos and explain, #8

Change-Id: I2b49d962f08a22ca6c8b437a583855e2344f513c

commit 500b7a9486f23fff6b1c17bb6e79c2f614a7e3e6

Merge: f2c050e fc1c31d

Author: Anoop Sharma <>

Date: Fri Mar 13 08:07:34 2015 -0700

Merge remote branch 'gerrit/master' into fix1

commit f2c050ea048b409b7d6769172e16c4ab26267bd0

Author: Anoop Sharma <>

Date: Fri Mar 13 08:07:15 2015 -0700

repos and explain, #7

Change-Id: I0bc04b6e8f147af1defbfd023c3700e26cc5b6f1

commit fabdb7dadd34238a8a227e880aa868ce484dc580

Merge: 59bb689 74640d7

Author: Anoop Sharma <>

Date: Thu Mar 12 09:01:20 2015 -0700

Merge remote branch 'gerrit/master' into fix1

commit 59bb6893437abe3999a53188503c60cf47633458

Author: Anoop Sharma <>

Date: Thu Mar 12 09:00:44 2015 -0700

repos and explain, #7

Change-Id: I6595a04ad59c1f705a04beee18c877cb81db0fac

commit a86353b45436126915fd2ba74c9af53f8bfd4d19

Author: Anoop Sharma <>

Date: Mon Mar 9 16:21:16 2015 -0700

repos and explain, #6

Change-Id: Ie4fe2b2bece7de3697eb8ada0d1cb1067b89bb88

commit 61272e0d6107536ab8b691415770e856267b6da5

Merge: 1708cb4 ca5dfb4

Author: Anoop Sharma <>

Date: Mon Mar 9 15:52:43 2015 -0700

Merge remote branch 'gerrit/master' into fix1

commit 1708cb44511809bf8e63800d226ac8ea34916726

Merge: 1284f4f 31f25ec

Author: Anoop Sharma <>

Date: Fri Mar 6 14:55:13 2015 -0800

Merge remote branch 'gerrit/master' into fix1

commit 1284f4f78f1f43657f29899cc8b9a96c02f37815

Author: Anoop Sharma <>

Date: Fri Mar 6 14:54:54 2015 -0800

commit: repos and explain, #6

Change-Id: I38d796fd4ab58bf29fa62c4bd5492ce44de9c8a0

commit f639911a1ea4247ae4c297bbbd904b84c7587b3a

Author: Anoop Sharma <>

Date: Mon Mar 2 06:31:21 2015 -0800

repos and explain, #5

Change-Id: Ie97ba3cac8dc78784d0a7eef57176e51b9e16224

commit da69fe9595d4ad5c9adf3b32ae8f848a0e6be964

Merge: 9b95b8c a5caeef

Author: Anoop Sharma <>

Date: Mon Mar 2 06:31:00 2015 -0800

Merge remote branch 'gerrit/master' into fix1

commit 9b95b8c3240a4186704b8b7f2ff8cefebf211732

Merge: ea2f687 8cd8a92

Author: Anoop Sharma <>

Date: Thu Feb 26 08:50:25 2015 -0800

Merge remote branch 'gerrit/master' into fix1

commit ea2f6871b7c7803d0f1e3fa664b474e069a0befd

Author: Anoop Sharma <>

Date: Thu Feb 26 08:50:03 2015 -0800

commit: repos and explain, #5

Change-Id: I44d8f87b3652e6300e41302d3a3e88b9b18652a3

commit 522a47eb991df2245005c693e4d034235adbfdc5

Merge: d986988 f42694e

Author: Anoop Sharma <>

Date: Wed Feb 25 13:13:02 2015 -0800

Merge remote branch 'gerrit/master' into fix1

commit d98698856a4be479a7603b0fb79210e80e88e1c1

Merge: 5841e47 2aad3ef

Author: Anoop Sharma <>

Date: Wed Feb 18 14:00:24 2015 -0800

Merge remote branch 'gerrit/master' into fix1

commit 5841e47f1e3f9a018424298a92cd134f4430e6bf

Author: Anoop Sharma <>

Date: Wed Feb 18 13:57:46 2015 -0800

repos and explain, #4

Change-Id: I01f53003069598381743c9be80b405ed37cefe6d

commit e7b5966d8ce653a57b8128c0b24c15fc05737668

Author: Anoop Sharma <>

Date: Wed Feb 11 11:28:20 2015 -0800

repos and explain, #3

Change-Id: Iff148ef175a7641ea37e27e60435ccf8f613e6db

commit aba98dc8cfda62d4e708cd892ed626f2a3051eb0

Merge: 66f1926 19a0c32

Author: Anoop Sharma <>

Date: Tue Feb 10 14:09:58 2015 -0800

Merge remote branch 'gerrit/master' into fix1

commit 66f1926d279f2b408d436b0aa53a3a6a5388901e

Merge: 9d0ef2a b68f59b

Author: Anoop Sharma <>

Date: Mon Feb 9 10:52:35 2015 -0800

Merge remote branch 'gerrit/master' into fix1




Change-Id: I6348fec3aece2baad9e9570749f43c086416219e

commit 9d0ef2aa01a62ea3e4e0ab35f8ec9687ef3336ef

Merge: aac268d 2cd8fbb

Author: Anoop Sharma <>

Date: Fri Feb 6 14:15:49 2015 -0800

Merge remote branch 'gerrit/master' into fix1

commit aac268d097afbe506dcf76de4d8f49652a1d4df7

Author: Anoop Sharma <>

Date: Tue Feb 3 16:31:17 2015 -0800

repos and explain, #2

Change-Id: Ia2a50159e577cd6ff8110dca35236fa90ea0d260

commit 2e6160411c8facb448e78d387fa3227e7b759c5d

Merge: c6e2fe5 566706d

Author: Anoop Sharma <>

Date: Tue Feb 3 15:21:03 2015 -0800

Merge remote branch 'gerrit/master' into fix1

commit c6e2fe529f9d1e59d3eb26cb145b260d81814443

Author: Anoop Sharma <>

Date: Tue Feb 3 15:16:36 2015 -0800

repository updated and explain changes, #1

Change-Id: I5daf66a2064b46ad65f1200579385965f7f56808

Change-Id: I0211d754524b8c06a05bb7ade20f884ec91340d2

    • -3
    • +30
    • -15
    • +48
  1. … 44 more files in changeset.
Trafodion Metadata Cleanup command support.

Various changes to support cleanup command has been added.

A separate external spec contains the details.

Summary of syntax:

cleanup [ table t | index i | sequence s | object o] [, uid <value>]

cleanup [private | shared] schema sch

cleanup uid <value>

cleanup metadata, check, return details

In addition, a new command to get names of various hbase objects

has also been added:

get [ all | user | system | external ] hbase objects;

Change-Id: I93f1f45e7fd78091bacd7c9f166420edd7c1abee

  1. … 65 more files in changeset.
memory leak and salted partitions set up fix

change ported from master branch. LP 1418685

Change-Id: I540c6ecda4d44f4ee6112c05cc144061cb50a66e

LP and other fixes.

-- LP 1414074. Added 'cleanup obsolete volatile tables' command

-- Added support for 'get all volatile tables' command

-- LP 1411864. Alter Sequence now correctly returns an error if value

specified exceeds the max value

-- LP 1413743. Error indicating sequence number max is reached is now

being returned instead of a numeric overflow error ff max largeint value

is reached while generating sequence numbers.

-- LP 1418685. Partition information is now being set up correctly for

delimited salted tables.

-- changed copyright message

Change-Id: Ic9e532204890a68ea0616b99a3170a0cc735ad53

Signed-off-by: Anoop Sharma <>

    • -386
    • +51
LP bug 1413676, alter table ignored 'not enforced' constraints.

Change-Id: Ib5a042385ad313442986dc8a27461c0e13c2f5b2

LP bug 1413404, repository tables are inaccessible after upgrade.

As part of upgrade, an internal 'initialize trafodion' stmt

is issued which creates repository tables with salted columns.

It also added salt expression string to TEXT metadata table.

Objects that were created prior to this release did not

have salt expression string in TEXT table. Upgrade code

moves older salt expression from default field of columns

table to TEXT table and updates text type.

It was incorrectly updating text type of newly created

respository tables that were already added to TEXT.

Fix is to not update texttype if salt string has already

been added.

Change-Id: Ib2b618b0e0cbc765737869f49b82e7a1355b1ff2

skipping seabase/test025 test for R1.0.

build/test seem to be hanging off and on after running

this test or during this test.

we dont know what the root cause is, it only happens

on build machines.

disabling this test for now. Cant say for sure if that

will take care of hang but we will try that out.

This test is for IDENTITY cols which is a tech preview


Post 1.0, we will enable this test and find/fix the root


Change-Id: I0dc541c74a7a8a27fd55e6dc14b9f839f6f9d8a3

    • -1
    • +1
LP 1410965 Old schemas do not show after upgrade.

upgrade was not creating schema objects

for older objects. That has been fixed.

Change-Id: I455d9d0f0b6ed5bf068f0cbc72a3721de391d3dc

    • -0
    • +30
additional fix to LP 1408504

fixed output of 'get version of software' to display that

metadata need to be upgraded.

Removed an unneeded error entry that was a leftover.

Change-Id: I8f3117fd643eff5c157bba57e0ac75b9ba1bb9c0

    • -5
    • +11
LP and other fixes. Details below.

-- LP 1408504: Any sql operation done when trafodion is uninitialized or

needs to be upgraded will return error. Until now, some commands

(like get schemas, invoke) were not returning an error.

-- LP 1408506: metadata upgrade was not handling repository tables

and was failing. That has been fixed.

-- index related commands (create, populate) now run in multiple phases.

Metadata update within a xn and row population without a xn.

-- common methods have been added that can be called to begin and

end transactions.

-- HYBRID_QUERY_CACHE cqd is now off by default

Change-Id: I99ef2548998b1a6830d4332db09080df5bcfc1c1

    • -180
    • +151
    • -45
    • +43
    • -6
    • +54
    • -6
    • +37
    • -2
    • +61
    • -53
    • +11
Perf improvement during metadata read.

Metadata information about metadata tables is read in 2 parts.

First part is hardcoded definitions since metadata structs are constant

and do not change. Second part is read from metadata tables(object uid)

as this varies depending on when metadata is initialized.

This checkin reads object uid only once and then stores it in

global metadata struct.

Change-Id: I094bd0c4bba67871bbca9cfdd8b75576841ba630

    • -0
    • +18
Bug fixes

-- LP 1403987. An external arkcmp process was being started to compile

a query. That has been fixe by using embedded arkcmp

-- Code to switch to embedded arkcmp and then switch back can now be

done by calling a method on CmpSeabaseDDL clase

-- get schemas/get tables were showing correlated name instead of

the actual metadata table if an error occured during access of

the metadata table. That has been fixed.

-- During exit of sqlci and master executor process, volatile

tables are dropped. This was being done uncondionally and caused exit

perf degradation. Code has been fixed to check and see if a volatile

schemas was created during that session and if it did, then they

are dropped.

Change-Id: Ie43dbac23e38aeb90849574e70181138bfffebf6

    • -0
    • +51
    • -63
    • +19