Clone Tools
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
[TRAFODION-3281] Remove virtual nodes configuration

Changed workstation mode (aka install_local_hadoop aka ILH) to use

single-node configuration instead of virtual nodes.

* ILH script generates sqconfig for current node.

* template sqconfig file moved to config template dir.

* Updated expected files accordingly.

** Filed jira [TRAFODION-3283] for bug found.

Removed setting and use of SQ_VIRTUAL_NODES variable.

Removed virtual nodes support in sqgen.

  1. … 17 more files in changeset.
[TRAFODION-3234] Refactor hive meta calls

Hive similarity check was detecting if there is a change in the hive

directory timestamp only. It has been changed to detect the DDL changes too.

Cleaned up code further and removed all the unwanted code.

  1. … 16 more files in changeset.
Fix for hive/TEST003 failure. Created a known failure to accommodate the costing change between Jenkins and contributor's work space environments.

Fix for hive/TEST003 failure seen in recent builds.

[TRAFODION-3234] Refactor hive meta calls to be less resource intensive to support hive partitions if needed

Fixes for hive/TEST005 failure.

This also made some of the earlier failed test cases in this test suite to return correct results.

Data modification check was unnecessarily done in the scan operator. This has been removed.

  1. … 3 more files in changeset.
[TRAFODION-3234] Add support for hive partitioned tables

Refactored the interaction with hive metastore to use the efficient APIs to obtain the necessary

information to construct the hive table descriptors. This refactoring is expected to reduce the

memory requirements, efficient and high performing even when there are thousands of partitions.

  1. … 13 more files in changeset.
known diff for AQR issue with hive timestamp mismatch.

[TRAFODION-3171] Refactor Hive sequence file reading to use the new implementation

Added a known diff file till TRAFODION-3185 is taken care of/

[TRAFODION-3171] Refactor Hive sequence file reading to use the new implementation

Disabled parallel plan in hive/TEST006 that could cause spliting a compressed sequence file

fix hive 009

fix regression failures

  1. … 6 more files in changeset.
TRAFODION-3086 DDL on Hive objs: use database and msck support

-- traf "set schema hive.<sch>" now behaves the same as Hive "use <db>".

1) <sch> is validated to exist at set schema time.

2) <sch> is set in Hive environment for that session. Any unqualified

object reference in ddl uses that default schema (database)

-- added support for msck (meta store check) command.

This command repairs database by refreshing untracked files/partitions

in hive metadata.

  1. … 12 more files in changeset.
fix sql regressions

  1. … 5 more files in changeset.
fix for hive/test008 jenkins regr failure

Merge remote branch 'origin/master' into ansharma_tsnano_br

  1. … 1 more file in changeset.
TRAFODION-3086 additional fixes

  1. … 7 more files in changeset.
TRAFODION-3086 Further enhancements to 'DDL on Hive objects'

-- Hive CTAS enhancements

-- CTAS will do create through Hive and insert...select in Traf

-- a cqd can be set if CTAS need to be passed in completely to Hive

-- CTAS 'no load' option can be used to create a Hive

table LIKE a traf table.

Older 'create hive like traf' is removed.

-- explain for CTAS to show ddl, insert...select, upd...stats stmts

-- Truncate enhancements.

-- traf: purgedata, truncate, truncate table

-- Hive: truncate, truncate table

-- 'if exists' option has been added

-- Hive truncate is through new exeutil operator

-- older operator uses LOB interface to clear data.

Maintained as Legacy operator, will be removed.

-- new operator uses Hive to truncate tables

-- explain for truncate to show trunc query that will be sent

-- removed obsolete parallel label op

-- removed obsolete purgedata (fast delete) op

  1. … 29 more files in changeset.
CTAS fixes for ddl on hive objects

  1. … 23 more files in changeset.
Merge remote branch 'origin/master' into ansharma_tsnano_br

Conflicts:

core/sql/regress/hive/EXPECTED005

  1. … 4 more files in changeset.
nanosecs precision for timestamp/interval datatypes

  1. … 29 more files in changeset.
hive regr fix

  1. … 1 more file in changeset.
jenkins regr failures updates.

Disabled INSERT thru 'process hive statement'

It return an error on HDP platform. Once the issue is fixed,

it will be enabled.

Also, merged remote branch 'origin/master' into ansharma_hiveddl_br

Conflicts:

core/sql/executor/JavaObjectInterface.cpp

core/sql/executor/JavaObjectInterface.h

  1. … 5 more files in changeset.
TRAFODION-3086 Traf support for DDL operations on Hive objects

-- Support for TRAFODION-3086. Details in document attached to jira.

Other changes:

-- support for "if not exists", "if exists" clause for create/drop view

-- Support for: truncate T, truncate table T.

-- same as purgedata

-- showddl <tab>, detail

-- unregister hive schema <sch>

-- will unregister all objects in specified schema

-- Support for "if not registered", "if registered" clauss for

register/unregister command.

  1. … 76 more files in changeset.
[TRAFODION-3065] Trafodion to support compressed Hive Text formatted tables

Compressed text files are now supported via the new implementation using

HDFS java APIs. When the hadoop is not configured to support a particular type

of compression, an error is thrown.

[TRAFODION-2982] JNI HDFS interface should support varied sized large buffers for read/write

A new CQD HDFS_IO_INTERIM_BYTEARRAY_SIZE_IN_KB is introduced to chunk

the read and write when byteArray is involved.

  1. … 15 more files in changeset.
TRAFODION-3027 Enhanced jenkins checktests. Expected file update.

  1. … 1 more file in changeset.
removed regress/hive/EXPECTED030,040

JIRA TRAFODION-3027 Enhance jenkins checktests to include more tests.

Details in jira.

  1. … 30 more files in changeset.
Fix for regression failure for hive/TEST017

various fixes

-- JIRA 2980 support for INSTR function.

INSTR(source_string, pattern, startPos, occurrence)

Search for pattern in source_string.

Start at startPos'th character. Optional, default 1

Return occurrence'th occurrence. Optional, default 1

(exp/exp_function.cpp/h, exp/ExpPCodeClauseGen.cpp,

generator/GenItemFunc.cpp, optimizer/BindItemExpr.cpp, ItemExpr.cpp,

optimizer/ItemFunc.h, SynthType.cpp, parser/sqlparser.y)

-- hivemd returns precision, scale, display_datatype

(comexe/ComTdbExeUtil.h, common/BaseTypes.cpp,NAType.cpp/h,

executor/ExExeUtilGet.cpp, ExExeUtil.h)

-- hive decimal type is treated as sql NUMERIC for performance.

(common/NAType.cpp)

-- hive select from hivemd with prepare and multiple executes work

(executor/ExExeUtilGet.cpp)

-- fix for crash in sql buffer pool when allocating large size rows.

(executor/sql_buffer.cpp)

-- SIGN(op) now returns error if op is not numeric

(BindItemExpr.cpp)

-- strings of length zero are no longer cached. Caching them causes problem

during backpatching.

(optimizer/ItemCache.cpp)

--- compile time predicates on constants were not being created correctly

in some cases. That has been fixed.

(ValueDesc.cpp)

-- trim now supported on scaled numeric, float and bignum datatypes

(BindItemExpr.cpp)

-- CASE is now supported on scaled numerics, float, bignum

(BindItemExpr.cpp)

-- char_length is now supported on numerics

(ItemFunc.h, BindItemExpr.cpp)

-- DAYOFMONTH now allowed only on datetime datatypes

(common/OperTypeEnum.h, sqlparser.y, BindItemExpr.cpp, ItemExpr.cpp)

-- TO_TIME now allowed only on 'datetime with time' or character datatypes.

(BindItemExpr.cpp, bin/SqlciErrors.txt)

-- updated expected file for compGeneral/test042

  1. … 31 more files in changeset.