EXPECTEDRTS

Clone Tools
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
[TRAFODION-2420] RMS enhancements

New counters introduced as part of BMO

InterimRowCount - Counter to keep track intermediate rows

produced by hash join and for intermediate

runs in sort

phase - BMO Phase for hash-join and sort

get statistics for statement s1, options 'cs' now works.

Removed attaching BMO stats to Non-root Paritial and leaf partition hash group by operators.

These operators are non-blocking BMOs.

Introduced 3 phases for hash grby operator to be visible in RMS Stats

options 'sl' now works with get statistics for qid.

The relevant new counters are now part of accumulated statistics type.

Extended the support upto 512 opeartors from 256 operators in RMS

Fixed both Type 2 and mxosrvr code to obtain all the externalized counters in accumualted stats correctly.

  1. … 27 more files in changeset.
[TRAFODION-2594] Trafodion logs the same message twice in most of its log4j and log4cxx logs [TRAFODION-2420] RMS enhancements [TRAFODION-2478] Memory monitor Thread enhancements

a) Changed the configuration to have log4j.additive property set to false

b) ScratchIOSize is initialized so that this counter doesn't show invalid

value

c) Cleaned up memory monitor concept and added code to log the memory monitoring

counters to sccp log file every 5 minutes or when there is a pressure difference

  1. … 9 more files in changeset.
[TRAFODION-1906] Dop is reported incorrectly in RMS operator stats

DOP should be set to 1 in the ExOperStats constructors when TDB is passed

in. DOP is reset to 0, when the statistics area in the ESP processes

are shipped to higher tier process to avoid double counting.

DOP is restored back to 1 before the next execution again.

scratchIOSize is now initialized to 0 in the BMOStats constructor

to avoid junk value seen at times.

  1. … 5 more files in changeset.
[TRAFODION-2420] RMS enhancements

Improved BMO operation accounting. Two new counters are added

a) scratchIOSize - Size in KB of the scratch IO

b) scratchIOMaxTime - Time in microseconds taken by any long pole

ESP instance to do scratch IO operations

The ScrBufSize, ScrBufRead and ScrBufWritten are no longer displayed

in the formatted get statistics command outputs.

Reduced the default value of CQD HBASE_NUM_CACHE_ROWS_MAX to 1024 from

10000 to reduce stress on the client JVM memory.

Disabled the memory pressure triggering in the BMO operators by increasing

the threshold GEN_MEM_PRESSURE_THRESHOLD to 10000 from 100. The memory

pressure detection code was inadvertently enabled when the cap to limit

memory pressure constant to 100 was removed.

Memory pressure triggers in BMO operators will be enabled later when

we understand the memory pressure detection better.

  1. … 14 more files in changeset.
[TRAFODION-2420] RMS enhancements Introduced 2 new counters HeapWM: - "SQL Heap WM" - Tracks the high water mark of the used memory including space buffer and bmo operator memory. It is enough to monitor this counter alone to track the memory utilization of the query.

scrIOTime: "Scr. IO Time" - Tracks the IO time taken to read/write to

the scratch files at BMO operator level

Introduced a 'Single line Format' option to display BMO stats and SE stats

in a single line format to enable exporting the output to Excel easily.

GET STATISTICS FOR QID CURRENT PROGRESS, OPTIONS 'SL'

Removed and cleaned up the code further

Also fixed memory corruption issues seen with GET STATISTICS command.

  1. … 16 more files in changeset.
[TRAFODION-2854] Improve SET STATISTICS command to display the information in new format

Currently SET STATISTICS ON | OFF alone is supported. It is improved to take some more options

SET STATISTICS PERTABLE ;

SET STATISTICS PROGRESS ;

SET STATISTICS DEFAULT ;

This will display pertable, progress, operator level statistics for the these options respectively.

SET STATISTICS ALL will display both progress and operator level statistics.

These commands are valid both in sqlci and trafci.

  1. … 6 more files in changeset.
Merge branch 'master' of git://git.apache.org/incubator-trafodion into topn

Conflicts:

core/sql/executor/ExExeUtilGetStats.cpp

  1. … 1 more file in changeset.
[TRAFODION-2259] Add TopN stats counter and realign BMO stats.

  1. … 7 more files in changeset.
Squashed commit of the following:

commit f924372b8a8d1f395e227ace804bf32ea18e2437

Author: selvaganesang <selva.govindarajan@esgyn.com>

Date: Fri Jan 6 03:27:53 2017 +0000

[TRAFODION-2420] RMS enhancements

Fix for core/TESTRTS failure

commit 461cdc879b887de437adee253fbeb74eef3204e3

Author: selvaganesang <selva.govindarajan@esgyn.com>

Date: Fri Jan 6 00:02:31 2017 +0000

[TRAFODION-2420] RMS enhancements

Fixed the issue pointed by Steve

This closes #PR 904

  1. … 1 more file in changeset.
[TRAFODION-2400] RMS enhancements

Removed FsDp2MsgStats and ExeDp2Stats

Introduced a new class ExeSEStats to cater the different

storaage engine supported by Trafodion.

The counters pertaining to storage engine were not getting

in GET STATISTICS FOR QID ... ACCUMULATED mode. Fixed this issuue

Removed references to obsolete Measuree concepts

  1. … 15 more files in changeset.
[TRAFODION-2420] RMS enhancements and cleanup

Removed the following classes pertaining to obsolete storage engine DP2.

ExDP2LeafOperStats

ExDP2InsertStats

ExPertableStats

And also removed some counters belonging to this obsolete storage engine

in other classes.

  1. … 7 more files in changeset.
jira TRAFODION-2234 turn aligned format on, phase 1

This is phase 1 of aligned format change.

Dev regressions now run with aligned format tables.

To test both hbase and aligned format, some tests have explicit

specification to create hbase format tables.

Tests that test for features that are currently only available

with hbase format (like pushdown sel expr) create hbase format tables.

Many expected files have been updated to reflect aligned format,

mostly in showddl and explain output.

  1. … 35 more files in changeset.
[TRAFODION-2134] Change the overflow_mode to 'DISK' by default

  1. … 1 more file in changeset.
[TRAFODION-1823] ESP idle timeout does not kick in, leading to too many ESPs on the system

Close message from the master was not sent to ESP by the platform

agnostic messaging layer in Trafodion. Fixed this bug and then ESP idle time

works as expected.

ESP_IDLE_TIMEOUT can now be given as a CQD. Internally, it would be

changed to SET SESSION DEFAULT and used by the IPC layer of Trafodion.

ESP_IDLE_TIMEOUT is set to 30 minutes by default in the CQD too.

Added test in core/TESTRTS to test ESP_IDLE_TIMEOUT

  1. … 5 more files in changeset.
[TRAFODION-1771] Enabled TESTRTS again TESTRTS was failing at times with error 8926. This is due to absence of ROOT_OPER_STATS in statistics area, but the stats descriptor was incorrectly indicating that there is a ROOT_OPER_STATS.

In future, I will get rid of setting the accumulated statistics as part of

cleanup effort of the statistics collection.

  1. … 2 more files in changeset.
Fix issue where optimization on key column addition should be turned off for MDAM scans, and update EXPECTEDTESTRTS to showcase the new value expected as byte read showing a 53% improvement over previous code

  1. … 1 more file in changeset.
ESP-colocation fixes (Ravisha and Qifan)

  1. … 6 more files in changeset.
Move core into subdir to combine repos

Use: git log --follow -- <file>

to view file history thru renames.

  1. … 10837 more files in changeset.