udrserv.cpp

Clone Tools
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
Merge [TRAFODION-2879] Core dump ...plus four more fixes, see below PR-1384

  1. … 2 more files in changeset.
[TRAFODION-2896] Internal error in stored procedures when a warning is generated in SQL

This problem was seen by Any on a large cluster and Selva found this issue

during debugging. At this point we don't have an easy testcase to reproduce,

but we agree that the code change is something we should do.

  1. … 2 more files in changeset.
[TRAFODION-2888] Streamline setjmp/longjmp concepts in Trafodion

First set of changes to streamline setjmp/longjmp.

a) Removed the setjmp in heap management within Trafodion.

b) Removed obsolete code related to No-wait operation concepts in SQL

  1. … 30 more files in changeset.
lcov: commit #1

  1. … 333 more files in changeset.
TRAFODION-2731 CodeCleanup: Remove obsolete, legacy and unused code

This phase handles the following:

-- removal of code that dealt with:

-- mpalias, NSK, MP, mploc, resource fork, rfork

-- ARLIB, DISK, VOLUME, PFS, compiler version info

-- interpretasrow/IAR, AuditImage, ExtractColumns functions

-- ARKCMP_SINGLE_PROCESS and oneProcess()

-- recompControl, remoteDefaults, rtdu, module

-- latebind thru nsk defines, guardian names, nametype nsk

-- SHADOW implementation

-- MEASURE

-- older sqlcat ReadTableDef

-- DP2_MEMORY, DP2_SPACE, HGB_DP2_MEMORY_LIMIT

-- internal cli methods no longer used by any caller

Code within the following defines is removed if it is obsolete

or the define itself is removed if that feature is always on:

-- removed NA_EIDPROC

-- removed SQLEXP_LIB_FUNC

-- removed NA_CMPDLL

-- removed SQ_PHANDLE_VERIFIER

-- removed SQ_NEW_PHANDLE

-- removed __EID

-- removed ARKFS_OPEN

-- removed STAND_ALONE

-- removed __TANDEM

-- removed NA_C89

-- removed NA_NSK

-- removed SQLEXPORT_LIB_FUNC

-- removed SQLCLI_LIB_FUNC

-- removed CLI_PRIV_SRL

-- removed PRIV_SRL

-- removed NA_LINUX

-- removed NA_HSC_LINUX

-- removed NA_UNIX

-- removed NA_WINNT

-- removed HAVE_MMAP

-- removed NA_NO_C_RUNTIME

-- removed NA_DEBUG_C_RUNTIME(replaced with _DEBUG)

-- removed NA_64BIT usage except in sqlcli.h

-- removed dg64

-- removed SQLEXPORT_LIB

-- removed NA_ARKFS

-- removed NA_IEEE_FLOAT

-- removed NA_GUARDIAN_MSG

-- removed NA_HSC

-- removed NA_TMFNOTYETSUPPORTED

-- removed ERROR

-- removed ERROR_STATE

-- removed SQLERRORS_LIB_FUNC

Contents of these files have been removed.

Next checkin fill remove the files itself from git:

executor/dmeasql.h

executor/ExMeas.h, ExMeas.cpp

executor/tempfile.h, .cpp

executor/rcb.h

executor/stubs.cpp, stubs2.cpp

exp/srlversion.cpp

cli/rtdu.h, rtdu2.h, rtdu.cpp, rtdu.cpp

cli/VicKeyValuePair.h

cli/CliDll.cpp

cli/CliStubsStaticBuild.cpp

cli/globalsrlversion.cpp

cli/globalstubs.cpp

cli/sqlciSRLStubs.cpp

cli/test.cpp

cli/privsrlversion.cpp

common/SqlExpDllDefines.h

common/SqlExportDllDefines.h

sqlcat/enum.h

sqlcat/ReadTableDef.h, cpp

sqlcat/readRealArk.h, cpp

  1. … 460 more files in changeset.
Merge [TRAFODION-2637] Part 2: Rework PR-1147

Merge [TRAFODION-2626] Make conf directory location configurable

This closes #1129

Squash merge of:

commit c679ffdd235d3f53d6f502931f5ab970d938819d

Author: Steve Varnau <steve.varnau@esgyn.com>

Date: Mon Jun 26 21:00:48 2017 +0000

[TRAFODION-2626] Updated to include recent additions to conf

Updated to modify recently added references to conf dir in jira

TRAFODION-2654.

Author: Steve Varnau <steve.varnau@esgyn.com>

Date: Mon Jun 26 20:51:24 2017 +0000

Merge remote branch 'origin/master' into traf2626_conf

commit b0dcf05aa2b2580700b214dfb12543ff0a1102df

Author: Steve Varnau <steve.varnau@esgyn.com>

Date: Tue Jun 20 16:27:44 2017 +0000

[TRAFODION-2626] Make conf directory location configurable

For user configuration files, use TRAF_CONF variable instead of

TRAF_HOME/conf.

In cases where we want to install config files to an alternate location

(say TRAF_CONF=/etc/trafodion), we would copy all the default files from

TRAF_HOME/conf to TRAF_CONF and make modifications in the TRAF_CONF location.

In development workspace, both locations can be one in the same.

One case where I left the original path is for the install_features file.

This file identifies version features and is not a user-configurable file.

Hence it should be read from the TRAF_HOME/conf location.

  1. … 13 more files in changeset.
[TRAFODION-2637] Part 2: Rework

This is cleanup and rework for an earlier PR that somehow got messed

up. The first commit of PR 1141 got merged into Master without

indicating so in the PR.

I am submitting a new commit with the rework from the review comments

here.

  1. … 1 more file in changeset.
[TRAFODION-2637] Library management with Cloudera parcels

Also in this commit:

[TRAFODION-2581] Combine JVM startup code of SQL executor and UDR server

We have library management SPJs that allow a user to deploy library

files from a client machine. These SPJs make use of the pdcp command

that is installed on many clusters, but not always. The new method

uses temporary HDFS files and a TMUDF to distribute the library

file to all nodes of the cluster.

For now I left the old PUT method in place and added a new PUTFILE

method. We can remove the old method later.

A few other things fixed in the process:

- Make sure that when a UDR server calls the executor and uses a JVM,

we share the same JVM. To do this, we need to use the same startup

method, and I needed to change the executor startup method to accept

the extra options used for the UDR server.

- Got rid of the "HBase compatibility mode" in JavaObjectInterface.

- A common method to execute SQL in the library management code.

- removed the obsolete init_libmgmt.sh file.

- We have special code that makes sure a UDR can request to run in

parallel, including a request to run on all nodes. Expanded this

special code from predefined UDFs to any UDF.

  1. … 25 more files in changeset.
[TRAFODION-2625] Memory "leak" for large TMUDFs

This is not a leak in the strict sense, as the memory gets deallocated

once the UDF finishes, but for UDFs that read or produce many rows, we

need to call method IpcBufferedMsgStream::cleanupBuffers() while the

UDF is executing. Otherwise, none of the reply buffers used will be

deallocated until the end. I've seen up to 20 GB of buffers being held

that way.

[TRAFODION-2596] Improve the log4j and log4cxx infrastructure in Trafodion

The following changes are made in the way Trafodion logs the messages:

Writes to a log file based on a component or set of components.

C++ part of the code base:

Component Default log file Configuration File

TM tm_<nid>.log log4cxx.trafodion.tm.config

SSCP sscp_<nid>.log log4cxx.trafodion.sscp.config

SSMP ssmp_<nid>.log log4cxx.trafodion.ssmp.config

All SQL processes trafodion.sql_<nid>.log log4cxx.trafodion.sql.config

mxosrvr

sqlci

tdm_arkesp

tdm_arkcmp

tdm_udrserv

Java part of the code base

TM trafodion.dtm.log log4j.dtm.config

SQL trafodion.sql.java.log log4j.sql.config

By default, the log level is set to INFO for most of the cases. When a message

dominates the log file and if doesn't add value in the current level, it will be

changed to the higher level in the hierarchy. This should help to improve the

usefulness of the log file at the default INFO level.

The existence of an environment variable TRAF_MULTIPLE_SQL_LOG_FILE will revert back

to the old way of logging into multiple files. Then, the configuration file

log4cxx.trafodion.masterexe.config will be used.

Currently, RollingFileAppender appender is used for both modes. Different config files

are used to change this appender when the need arises.

log4cxx.trafodion.udr.config and log4cxx.trafodion.lob.config are removed.

log4j.hdfs.config used as the config file for SQL is renamed to log4j.sql.config

Foundation components logging will be revamped later.

This closes #1070

  1. … 19 more files in changeset.
[TRAFODION-2566] Reduce the virtual memory allocated in Trafodion processes with JDK1.8

Added two new configurable JVM options -XX:CompressedClassSpaceSize=128m and

-XX:MaxMetaSpaceSize=128m.

You can modify the default value of these options using environment variables

JVM_COMPRESSED_CLASS_SPACE_SIZE and JVM_METASPACE_SIZE respectively.

With this change the virtual memory of any SQL processes and tm is reduced by 1GB.

In JDK1.8 this space is used in lieu of permanent generation. Hence, I reduced the

default Java Objects Max heap size for SQL master, ESP and arkcmp processes to 256 MB from 512MB.

The udrserv process continues to have 512MB of java objects space.

  1. … 3 more files in changeset.
convert all MY_SQROOT to TRAF_HOME

  1. … 294 more files in changeset.
TRAFODION-1754 Showddl component does not display grants correctly

Showddl should be displaying grants on an operation in a manner that allows

the privileges to be recreated. Currently, they are returned in the order

in which they are read from the metadata.

While working on this JIRA, several issues were fixed including:

- Consolidated privilege values into one location - NAUserId.h. They used to be

stored in NAUserId.h, ComSmallDefs.h, and PrivMgrDefs.h

- The check for getting the next available role ID was not correct.

- PrivMgrPrivileges::hasColWGO has incorrect indexing

- PrivMgrComponentPrivileges::selectAllWhere has incorrect error checking

  1. … 16 more files in changeset.
AFODION-1492] Allow specification of MaxHeapSize for JVM started by UDR server

This is a fix for the second of the two issues described in this JIRA. The first

has been addressed by narendra in a separate checkin.

In this checkin we allow multiple methods to specify JVM startup options for the

UDR server process. There are three methods applied in this order with the last

specification for each property superceeding previous specifications.

1. A default default of 512MB for UDR JVM's max heap size

2. Specify any desired property including -Xmx in a new configuration file

3. Specify any desired property using TRAF_UDR_JAVA_OPTIONS env variable

The new configuration file is at $MY_SQROT/conf/trafodion.udr.config. It has

detailed comments on how properties should be specified in the file. The location

(including name) of the configuration file can be overridden with the env variable

TRAFUDRCFG.

The TRAF_UDR_JAVA_OPTIONS env variable must be specified in $MY_SQROOT/etc/ms.env

for it to be visible to the UDR server. Multiple properties can be separated by a

'\t' (TAB) chracater or the '\n' (newline) character

  1. … 4 more files in changeset.
Part 2 of changes to update license text in Trafodion source

This check-in updates license text in all source files in the

core/sql directory. Remaining directories will be done in

later check-ins.

  1. … 3147 more files in changeset.
Merge remote branch 'log4cxx/temp-merge' into migrate_log4cpp_to_log4cxx

Conflicts:

core/log4cpp/.gitignore

core/log4cpp/log4cpp-1.1.1/COPYING

core/log4cpp/log4cpp-1.1.1/ChangeLog

core/log4cpp/log4cpp-1.1.1/include/.cvsignore

core/log4cpp/log4cpp-1.1.1/include/Makefile.am

core/log4cpp/log4cpp-1.1.1/include/Makefile.in

core/log4cpp/log4cpp-1.1.1/include/config.h.in

core/log4cpp/log4cpp-1.1.1/include/log4cpp/.cvsignore

core/log4cpp/log4cpp-1.1.1/include/log4cpp/AbortAppender.hh

core/log4cpp/log4cpp-1.1.1/include/log4cpp/Appender.hh

core/log4cpp/log4cpp-1.1.1/include/log4cpp/AppenderSkeleton.hh

core/log4cpp/log4cpp-1.1.1/include/log4cpp/AppendersFactory.hh

core/log4cpp/log4cpp-1.1.1/include/log4cpp/BasicConfigurator.hh

core/log4cpp/log4cpp-1.1.1/include/log4cpp/BasicLayout.hh

core/log4cpp/log4cpp-1.1.1/include/log4cpp/BufferingAppender.hh

core/log4cpp/log4cpp-1.1.1/include/log4cpp/Category.hh

core/log4cpp/log4cpp-1.1.1/include/log4cpp/CategoryStream.hh

core/log4cpp/log4cpp-1.1.1/include/log4cpp/Configurator.hh

core/log4cpp/log4cpp-1.1.1/include/log4cpp/Export.hh

core/log4cpp/log4cpp-1.1.1/include/log4cpp/FactoryParams.hh

core/log4cpp/log4cpp-1.1.1/include/log4cpp/FileAppender.hh

core/log4cpp/log4cpp-1.1.1/include/log4cpp/Filter.hh

core/log4cpp/log4cpp-1.1.1/include/log4cpp/FixedContextCategory.hh

core/log4cpp/log4cpp-1.1.1/include/log4cpp/HierarchyMaintainer.hh

core/log4cpp/log4cpp-1.1.1/include/log4cpp/IdsaAppender.hh

core/log4cpp/log4cpp-1.1.1/include/log4cpp/Layout.hh

core/log4cpp/log4cpp-1.1.1/include/log4cpp/LayoutAppender.hh

core/log4cpp/log4cpp-1.1.1/include/log4cpp/LayoutsFactory.hh

core/log4cpp/log4cpp-1.1.1/include/log4cpp/LevelEvaluator.hh

core/log4cpp/log4cpp-1.1.1/include/log4cpp/LoggingEvent.hh

core/log4cpp/log4cpp-1.1.1/include/log4cpp/Makefile.am

core/log4cpp/log4cpp-1.1.1/include/log4cpp/Makefile.in

core/log4cpp/log4cpp-1.1.1/include/log4cpp/Manipulator.hh

core/log4cpp/log4cpp-1.1.1/include/log4cpp/NDC.hh

core/log4cpp/log4cpp-1.1.1/include/log4cpp/NTEventLogAppender.hh

core/log4cpp/log4cpp-1.1.1/include/log4cpp/OstreamAppender.hh

core/log4cpp/log4cpp-1.1.1/include/log4cpp/PassThroughLayout.hh

core/log4cpp/log4cpp-1.1.1/include/log4cpp/PatternLayout.hh

core/log4cpp/log4cpp-1.1.1/include/log4cpp/Portability.hh

core/log4cpp/log4cpp-1.1.1/include/log4cpp/Priority.hh

core/log4cpp/log4cpp-1.1.1/include/log4cpp/PropertyConfigurator.hh

core/log4cpp/log4cpp-1.1.1/include/log4cpp/RemoteSyslogAppender.hh

core/log4cpp/log4cpp-1.1.1/include/log4cpp/RollingFileAppender.hh

core/log4cpp/log4cpp-1.1.1/include/log4cpp/SimpleConfigurator.hh

core/log4cpp/log4cpp-1.1.1/include/log4cpp/SimpleLayout.hh

core/log4cpp/log4cpp-1.1.1/include/log4cpp/SmtpAppender.hh

core/log4cpp/log4cpp-1.1.1/include/log4cpp/StringQueueAppender.hh

core/log4cpp/log4cpp-1.1.1/include/log4cpp/SyslogAppender.hh

core/log4cpp/log4cpp-1.1.1/include/log4cpp/TimeStamp.hh

core/log4cpp/log4cpp-1.1.1/include/log4cpp/TriggeringEventEvaluator.hh

core/log4cpp/log4cpp-1.1.1/include/log4cpp/TriggeringEventEvaluatorFactory.hh

core/log4cpp/log4cpp-1.1.1/include/log4cpp/Win32DebugAppender.hh

core/log4cpp/log4cpp-1.1.1/include/log4cpp/config-MinGW32.h

core/log4cpp/log4cpp-1.1.1/include/log4cpp/config-openvms.h

core/log4cpp/log4cpp-1.1.1/include/log4cpp/config-win32-stlport-boost.h

core/log4cpp/log4cpp-1.1.1/include/log4cpp/config.h

core/log4cpp/log4cpp-1.1.1/include/log4cpp/convenience.h

core/log4cpp/log4cpp-1.1.1/include/log4cpp/threading/BoostThreads.hh

core/log4cpp/log4cpp-1.1.1/include/log4cpp/threading/DummyThreads.hh

core/log4cpp/log4cpp-1.1.1/include/log4cpp/threading/MSThreads.hh

core/log4cpp/log4cpp-1.1.1/include/log4cpp/threading/Makefile.am

core/log4cpp/log4cpp-1.1.1/include/log4cpp/threading/Makefile.in

core/log4cpp/log4cpp-1.1.1/include/log4cpp/threading/OmniThreads.hh

core/log4cpp/log4cpp-1.1.1/include/log4cpp/threading/PThreads.hh

core/log4cpp/log4cpp-1.1.1/include/log4cpp/threading/Threading.hh

core/log4cpp/log4cpp-1.1.1/src/.cvsignore

core/log4cpp/log4cpp-1.1.1/src/AbortAppender.cpp

core/log4cpp/log4cpp-1.1.1/src/Appender.cpp

core/log4cpp/log4cpp-1.1.1/src/AppenderSkeleton.cpp

core/log4cpp/log4cpp-1.1.1/src/AppendersFactory.cpp

core/log4cpp/log4cpp-1.1.1/src/BasicConfigurator.cpp

core/log4cpp/log4cpp-1.1.1/src/BasicLayout.cpp

core/log4cpp/log4cpp-1.1.1/src/BufferingAppender.cpp

core/log4cpp/log4cpp-1.1.1/src/Category.cpp

core/log4cpp/log4cpp-1.1.1/src/CategoryStream.cpp

core/log4cpp/log4cpp-1.1.1/src/Configurator.cpp

core/log4cpp/log4cpp-1.1.1/src/DllMain.cpp

core/log4cpp/log4cpp-1.1.1/src/DummyThreads.cpp

core/log4cpp/log4cpp-1.1.1/src/FactoryParams.cpp

core/log4cpp/log4cpp-1.1.1/src/FileAppender.cpp

core/log4cpp/log4cpp-1.1.1/src/Filter.cpp

core/log4cpp/log4cpp-1.1.1/src/FixedContextCategory.cpp

core/log4cpp/log4cpp-1.1.1/src/HierarchyMaintainer.cpp

core/log4cpp/log4cpp-1.1.1/src/IdsaAppender.cpp

core/log4cpp/log4cpp-1.1.1/src/LayoutAppender.cpp

core/log4cpp/log4cpp-1.1.1/src/LayoutsFactory.cpp

core/log4cpp/log4cpp-1.1.1/src/LevelEvaluator.cpp

core/log4cpp/log4cpp-1.1.1/src/Localtime.cpp

core/log4cpp/log4cpp-1.1.1/src/Localtime.hh

core/log4cpp/log4cpp-1.1.1/src/LoggingEvent.cpp

core/log4cpp/log4cpp-1.1.1/src/MSThreads.cpp

core/log4cpp/log4cpp-1.1.1/src/Makefile

core/log4cpp/log4cpp-1.1.1/src/Makefile.am

core/log4cpp/log4cpp-1.1.1/src/Makefile.in

core/log4cpp/log4cpp-1.1.1/src/Manipulator.cpp

core/log4cpp/log4cpp-1.1.1/src/NDC.cpp

core/log4cpp/log4cpp-1.1.1/src/NTEventLogAppender.cpp

core/log4cpp/log4cpp-1.1.1/src/OmniThreads.cpp

core/log4cpp/log4cpp-1.1.1/src/OstreamAppender.cpp

core/log4cpp/log4cpp-1.1.1/src/PThreads.cpp

core/log4cpp/log4cpp-1.1.1/src/PassThroughLayout.cpp

core/log4cpp/log4cpp-1.1.1/src/PatternLayout.cpp

core/log4cpp/log4cpp-1.1.1/src/PortabilityImpl.cpp

core/log4cpp/log4cpp-1.1.1/src/PortabilityImpl.hh

core/log4cpp/log4cpp-1.1.1/src/Priority.cpp

core/log4cpp/log4cpp-1.1.1/src/Properties.cpp

core/log4cpp/log4cpp-1.1.1/src/Properties.hh

core/log4cpp/log4cpp-1.1.1/src/PropertyConfigurator.cpp

core/log4cpp/log4cpp-1.1.1/src/PropertyConfiguratorImpl.cpp

core/log4cpp/log4cpp-1.1.1/src/PropertyConfiguratorImpl.hh

core/log4cpp/log4cpp-1.1.1/src/RemoteSyslogAppender.cpp

core/log4cpp/log4cpp-1.1.1/src/RollingFileAppender.cpp

core/log4cpp/log4cpp-1.1.1/src/SimpleConfigurator.cpp

core/log4cpp/log4cpp-1.1.1/src/SimpleLayout.cpp

core/log4cpp/log4cpp-1.1.1/src/SmtpAppender.cpp

core/log4cpp/log4cpp-1.1.1/src/StringQueueAppender.cpp

core/log4cpp/log4cpp-1.1.1/src/StringUtil.cpp

core/log4cpp/log4cpp-1.1.1/src/StringUtil.hh

core/log4cpp/log4cpp-1.1.1/src/SyslogAppender.cpp

core/log4cpp/log4cpp-1.1.1/src/TimeStamp.cpp

core/log4cpp/log4cpp-1.1.1/src/TriggeringEventEvaluatorFactory.cpp

core/log4cpp/log4cpp-1.1.1/src/Win32DebugAppender.cpp

core/log4cpp/log4cpp-1.1.1/src/macros.gmk

core/log4cpp/log4cpp-1.1.1/src/snprintf.c

core/log4cpp/log4cpp-1.1.1/src/verslib.h

core/log4cpp/log4cpp-1.1.1/src/verslog4cpplib.cpp

core/sqf/conf/log4cpp.monitor.mon.config

core/sqf/conf/log4cpp.monitor.mon.snmp.config

core/sqf/conf/log4cpp.monitor.psd.config

core/sqf/conf/log4cpp.monitor.wdg.config

core/sqf/conf/log4cpp.monitor.wdg.snmp.config

core/sqf/conf/log4cpp.trafodion.lob.config

core/sqf/conf/log4cpp.trafodion.masterexe.config

core/sqf/conf/log4cpp.trafodion.sscp.config

core/sqf/conf/log4cpp.trafodion.ssmp.config

core/sqf/conf/log4cpp.trafodion.tm.config

core/sqf/conf/log4cpp.trafodion.udr.config

core/sql/bin/vers_log4cpp.cpp

sql/bin/vers_log4cpp.cpp

sql/bin/vers_log4cxx.cpp

  1. … 220 more files in changeset.
Move core into subdir to combine repos

  1. … 10768 more files in changeset.
Move core into subdir to combine repos

  1. … 10622 more files in changeset.
Move core into subdir to combine repos

Use: git log --follow -- <file>

to view file history thru renames.

  1. … 10837 more files in changeset.