Clone
 

chris sheedy <chris.sheedy@hp.com> in Trafodion

Added missing HP copyright in a single file

Change-Id: Icd16eaf4f0eecfd225f547ff1d2e1053698dd680

Added missing HP copyright in 1 file

Change-Id: Ic112210951a05b3374e0eee22d7938d0e9481e40

Remove obsolete sqlutils references

The sql/nskgmake/Makerules* files had references to the removed

sqlutils, which made those files unnecessarily complicated. Those

references and some other obsolete code were removed, mainly in sql

makefiles. In sql/common, 2 copyright dates were corrected.

This change follows on from change 1724 which removed many

obsolete make files.

Change-Id: I9c5b90d55ae40a74f525c1f9dced0454bf634581

    • -7
    • +5
    /sql/sqlshare/ReservedInternalNames.cpp
Added or updated copyrights in 3 .def files

Change-Id: If6266dced785338d3f3d2c08f46e02e17c9fd682

    • -3
    • +15
    /odbcclient/DSNConverter/DSNConverter.def
    • -0
    • +18
    /odbcclient/Drvr35Res/Drvr35Res.def
Added one missing HP copyright

Change-Id: Ifccaf2e09bfc03cb0b05f48e7a4b56841227945b

Copyright corrections

Put correct form of Apache License into many source files.

The tdm_odbcSrvrMsg.mc file is used on Windows, and is

treated as binary on Linux. However, it is visible in

Linux with "cat", and editable with "notepad" on Windows.

In patch set 2, updated sql/regress/newregr by deleting some

unused files, updating copyrights in scripts, and removing

copyrights from front of expected test results. This is not

a final list of the changes needed, since the test suites in

the "newregr" directory need more work before they run correctly.

Change-Id: If0ea5331b822764191c839c38dfd84218612fe7a

    • binary
    /conn/odbc/src/odbc/SrvrMsg/tdm_odbcSrvrMsg.mc
    • -17
    • +0
    /sql/regress/newregr/exeperf/banner
    • -29
    • +11
    /sql/regress/newregr/exeperf/runepr.ksh
    • -19
    • +17
    /sql/regress/newregr/exeperf/runregr
    • -17
    • +18
    /sql/regress/newregr/exeperf/setup_exeperf.ksh
    • -29
    • +13
    /sql/regress/newregr/mvqr/CropDescriptor.ksh
    • -17
    • +0
    /sql/regress/newregr/mvqr/EXPECTED001
    • -4139
    • +0
    /sql/regress/newregr/mvqr/EXPECTED001.NT
    • -17
    • +0
    /sql/regress/newregr/mvqr/EXPECTED002
    • -17
    • +0
    /sql/regress/newregr/mvqr/EXPECTED003
    • -17
    • +0
    /sql/regress/newregr/mvqr/EXPECTED004
  1. … 319 more files in changeset.
Added and corrected copyright statements

The instructions for building were improved, they are in

the odbcclient/README.txt file, not in wiki.trafodion.org.

Change-Id: I3b33d47fc46f22bbaa8b2468b3290b10b525d049

    • binary
    /odbcclient/drvr35msg/tdm_odbcdrvmsg.mc
Do not run check-copyrights step in make all

Change-Id: I93c056c72117ba9f3b151cb57438ff63bb9db159

Removed obsolete symlinks

Change-Id: I6d406654d5276da7d61a31d07dd667f13861e81f

    • -1
    • +0
    /conn/unixodbc/odbc/dependencies/unixSSL
Add convenience script used to watch processes

Help text is:

Give a list with counts of all the processes running under one or more userids.

Syntax: sqf/tools/sq_check_myuserid [-s <sleep secs>] [-n <num samples] [-c] [-u user ...] [-a prog ...]

-c is used to give a sorted count of the processes running.

-s means to do multiple sampling, sleeping in between.

Defaults to 1 minute if not specified and -n > 0

-n means to do that many samples.

If -s is used, the default is forever, otherwise 1

-u user ... to specify which userid to watch, default is current user.

-a prog ... to add additional program to ignore.

See script source for initial list of ignored programs.

Change-Id: Icb6395c6d52d7a4e98c8c5d0b3932d1875baaf90

    • -0
    • +147
    /sqf/tools/sq_check_myuserid
Small improvements in checking install_local_hadoop options

Change-Id: Ia7e0e661e2b0f54dbd8934a6305707306dd6d30c

Removed about 50 obsolete make files from sql

Security is not built inside sql, and sqlutils is not in Trafodion, so

their makefiles are not needed and have been deleted. Also deleted a few

other unused makefiles. In modified makefiles, removed unused parts.

The sql/nskgmake/Makerules* files still have references to sqlutils, those

lines will be considered in a separate change.

In second patch set, deleted unused directory sql/nskgmake/exeindp2

In third patch, added missing copyrights in

sql/executor/OrcFileReader.java

sql/regress/udr/TEST101.java

and deleted empty file

sql/parser/StmtDDLAlterTableAlterColumnRecalibrateSG.h

Change-Id: Id9ab492d4a23579cd0ab6db7279e0da56c55ea4c

    • -29
    • +0
    /sql/nskgmake/openldap_liblber/Makefile
    • -29
    • +0
    /sql/nskgmake/openldap_libldap/Makefile
    • -29
    • +0
    /sql/nskgmake/openldap_liblutil/Makefile
    • -78
    • +0
    /sql/nskgmake/openldap_test/Makefile
    • -101
    • +0
    /sql/nskgmake/openssl_crypto/Makefile
    • -180
    • +0
    /sql/nskgmake/openssl_crypto1/Makefile
    • -177
    • +0
    /sql/nskgmake/openssl_crypto2/Makefile
    • -142
    • +0
    /sql/nskgmake/openssl_crypto3/Makefile
    • -39
    • +0
    /sql/nskgmake/security_sql_cli/Makefile
  1. … 51 more files in changeset.
Invoking sqenv.sh repeatedly does not change shell environment

Bug 1401216 reported that sqenv.sh is not idempotent. Previously this

was fixed by using shell function remove_duplicates_in_path() for the

variables that grew. However, MANPATH was missed. This change adds

one line which ensures MANPATH does not grow.

Trailing spaces which were added to sqenvcom.sh in the last few months

were deleted in this change.

Change-Id: Id0bca1cfbf79d0f215667cbbf205b1b5180b6e78

Corrected typos in user messages

Corrected about 15 typos in messages which can be reported to the user,

for example, "loaction" which should be "location".

While changing files, also removed unnecessary spaces from the end of

source lines, error messages and strings.

In backup_restore_functions.sh, did not change 9 uses of "hsdf" to "hdfs" in

shell coding, for example, line 212: local hsdf_loc="$1"

While this is user-visible, it is not likely to be seen, and perhaps this

is used with other non-Trafodion scripts which already use the better

spelling, $hdfs_loc.

Change-Id: I311da5560057d441cb6ab80e91bef6d0a4f6ae3c

Corrected typos in user-visible messages and scripts

Corrected 4 misspellings and about 10 uses of "its" which should be "it is".

Deleted some but not all trailing spaces in the modified files.

Change-Id: I382110a24e325686f2bcf48e88a7a2394d34618b

    • -31
    • +31
    /installer/tools/trafodion_scanner.cfg
Improved make rules for log4cpp packaging

Packaging of the source for log4cpp in change 1604 did not handle

a second make in the same directory or "make clean". Those problems

have been fixed.

Change-Id: I4172d6b02092fcf6e943e50a106051e214956ed1

Package log4cpp source into core product tgz file

The log4cpp source files used to build trafodion/core have small differences

from files downloaded from http://sourceforge.net/projects/log4cpp version

1.1.1, so they should be provided to the end user.

This change packages the log4cpp directory of https://github.com/trafodion/core

into a tar file, and puts that in the trafodion core product's .tgz file as:

sources/log4cpp-1.1.1.tar

When Trafodion is installed, this file is available to the end user.

This new file has tar format, not gzip format, and hence its name has .tar

instead of .tgz. Files with the tgz extension are excluded from packaging

into the core product because "*tgz" is in sqf/build-scripts/package.exclude.

In sqf/Makefile, 4 obsolete lines were deleted, because this build does not

compare the package contents with a bom (bill-of-materials) list.

The packaged source files include the license used by log4cpp, in the file

named COPYING which starts with:

GNU LESSER GENERAL PUBLIC LICENSE

Version 2.1, February 1999

Copyright (C) 1991, 1999 Free Software Foundation, Inc.

59 Temple Place, Suite 330, Boston, MA 02111-1307 USA

Everyone is permitted to copy and distribute verbatim copies

of this license document, but changing it is not allowed.

In the second patch set, added the new tar file to two .gitignore files

so builds in Jenkins do not fail the test for untracked files in the

check-git-status.sh script.

Change-Id: I8db0559128e3f57653ad3cf08f2b553b225a550c

Fixed shell script for making mpi files

The zfix shell script had only one name for a temp file, "/tmp/zxx". This did

not allow two builds on the same machine to run the script at the same time.

This is a possible cause of build failures, such as the corrupted mpi.h seen

today in https://jenkins02.trafodion.org/job/build-product-release/141

which had this error:

... /sqf/export/include/mpi.h:837:1: error: unterminated #ifndef ##(MPI)

The temp file name is now "/tmp/zxx.$$" which adds a pid to the end of the name.

Change-Id: I20e6549342fe792fba30524cbc5fe14cca833a1d

Cleanup of comments in log4cpp

Copyright notices in log4cpp files needed some minor changes to make them 100%

correct, since the rules for their copyrights are not the same as elsewhere in

Trafodion. This is preparatory work before adding the source files for log4cpp

to the packaged core product. Those files have small differences from the

files downloaded from http://sourceforge.net/projects/log4cpp version 1.1.1,

and so they should be provided to the end user.

Added the Changelog file which records the differences made by different

people at http://sourceforge.net/projects/log4cpp. The changes for Trafodion

are now listed at the front of this Changelog.

The file include/log4cpp/RemoteSyslogAppender.hh does not have a new copyright

because the change merely corrects a typo. This trivial change does not meet

the threshold for needing a copyright.

In some lines, trailing spaces were restored to reduce the differences from the

original files.

Change-Id: If57d5d394abd664674b7859860ab1ae7ad3d3f1f

    • -0
    • +3825
    /log4cpp/log4cpp-1.1.1/ChangeLog
    • -7
    • +7
    /log4cpp/log4cpp-1.1.1/src/FileAppender.cpp
    • -14
    • +14
    /log4cpp/log4cpp-1.1.1/src/PatternLayout.cpp
    • -6
    • +6
    /log4cpp/log4cpp-1.1.1/src/StringUtil.cpp
  1. … 3 more files in changeset.
Report ports used by install_local_hadoop

Added options -n and -v to install_local_hadoop so it prints the

ports numbers it configures. Also improved the help text which

now is:

install_local_hadoop [ -p {<start port num> | rand | fromDisplay} ]

[ -y ]

[ -n ]

[ -v ]

-p configures non-standard ports, and is one of:

-p <start port num> custom cases

-p rand for shared systems, use a random start port number

between 9000 and 49000 that is divisible by 200

-p fromDisplay if you are running on a VNC session

-y answers interactive questions implicitly with yes

-n takes no action, useful with -v

-v lists the port values used

See script header for use of optional environment variables.

For example:

$ install_local_hadoop -v -n

MY_DCS_MASTER_INFO_PORT=40010

MY_DCS_MASTER_PORT=37800

MY_DCS_SERVER_INFO_PORT=40030

MY_HADOOP_DN_HTTP_PORT_NUM=50075

MY_HADOOP_DN_IPC_PORT_NUM=50020

MY_HADOOP_DN_PORT_NUM=50010

MY_HADOOP_HDFS_PORT_NUM=9000

MY_HADOOP_JOB_TRACKER_HTTP_PORT_NUM=50030

MY_HADOOP_NN_HTTP_PORT_NUM=50070

MY_HADOOP_SECONDARY_NN_PORT_NUM=50090

MY_HADOOP_SHUFFLE_PORT_NUM=8080

MY_HADOOP_TASK_TRACKER_PORT_NUM=50060

MY_HBASE_MASTER_INFO_PORT_NUM=60010

MY_HBASE_MASTER_PORT_NUM=60000

MY_HBASE_REGIONSERVER_INFO_PORT_NUM=60030

MY_HBASE_REGIONSERVER_PORT_NUM=60020

MY_HBASE_REST_PORT_NUM=8080

MY_HBASE_ZOOKEEPER_LEADERPORT_NUM=3888

MY_HBASE_ZOOKEEPER_PEERPORT_NUM=2888

MY_HBASE_ZOOKEEPER_PROPERTY_CLIENTPORT_NUM=2181

MY_REST_SERVER_PORT=4200

MY_REST_SERVER_SECURE_PORT=4201

MY_SQL_PORT_NUM=3346

MY_YARN_ADMIN_PORT_NUM=8033

MY_YARN_HTTP_PORT_NUM=8088

MY_YARN_LOCALIZER_PORT_NUM=8040

MY_YARN_NM_PORT_NUM=8041

MY_YARN_RESMAN_PORT_NUM=8032

MY_YARN_SCHED_PORT_NUM=8030

MY_YARN_TRACKER_PORT_NUM=8031

$

Because more options were added, changed the test of them to a loop instead

of sequential tests.

Removed trailing spaces.

Change-Id: Ia4c7acd89a7556c4f14af7e75b8f28c98abee48c

    • -64
    • +129
    /sqf/sql/scripts/install_local_hadoop
Allow longer compile times in compGeneral/TEST042

In the compGeneral suite, TEST042 checks that sql compile times are fast enough

in the release flavor. This change allows longer times on OpenStack and VMware

virtual machines. Allowed times for physical machines are unchanged.

The specific "failure" which requires this change was run on a VMware

virtual machine. The difference from the expected output was:

1794c1794

< SUCCESS (less than 1ms)

---

> FAIL (at least 1ms) Compile Time 00:00:00.001713

1824c1824

< SUCCESS (less than 1ms)

---

> FAIL (at least 1ms) Compile Time 00:00:00.001821

The largest acceptable compile time on a virtual machine is now 1.999 ms, which

in the above format is "00:00:00.001999".

For physical machines, the largest value is unchanged at 1.399 ms.

This change requires detecting that the test is executing on a virtual machine.

That is found by looking in /sys/class/dmi/id/sys_vendor for "OpenStack"

or "VMware".

Change-Id: Ibd294c7e85917af1ecba993b80103e58f50d5ec7

Fixed typos in user-visible messages and sample code

Fixed spellings and minor grammar in strings used in

in output, and sample code supplied to users.

One possibly harmful misspelling was found in source code,

file sqf/monitor/linux/montim.cxx used "Contianer_ExitProcess"

where it should have used "Container_ExitProcess".

Five files which could be changed for this reason are being held back

because they are also in open change 875. They will be checked in after

that change is merged to the main branch:

conn/odbc/src/odbc/dependencies/linux/smxoevl.h

conn/trafci/src/org/trafodion/ci/InterfaceQuery.java

sqf/src/sp_proxy/sp_common.cpp

sqf/src/tm/tmlibmsg.h

sqf/src/tm/tmtimer.cpp

In patch set 2, modified these files as notde by reviewer:

conn/jdbc_type2/samples/CreateDataSource.java

sqf/monitor/test/montestutil.cxx

sql/cli/Statement.h

sql/optimizer/RelExpr.h

sql/optimizer/RelScan.h

sql/optimizer/ScmCostMethod.cpp

Change-Id: I3761e9e1518ab39415806bba0f4ed93d14e1d41c

    • -13
    • +13
    /conn/jdbc_type2/native/SqlInterface.cpp
    • -8
    • +8
    /conn/jdbc_type2/native/jdbcUtilLib.cpp
    • -10
    • +10
    /conn/jdbc_type2/native/tslxInitialization.cpp
    • -3
    • +3
    /conn/jdbc_type4/src/T4Messages.properties
    • -5
    • +5
    /conn/odbc/src/odbc/Common/QSExceptions.h
    • -1
    • +1
    /conn/odbc/src/odbc/Common/TransportBase.cpp
  1. … 45 more files in changeset.
Remove duplicates in PATH and CLASSPATH to fix bug 1401216

In sqf/sqenvcom.sh :

1. Removed duplicates from environment variables $CLASSPATH and $PATH,

so sqenv.sh can be invoked repeatedly without them growing.

2. Removed test for JAVA_HOME containing exactly "1.7.0_67", which should

help fix bug 1387238.

3. To reduce the places the recommended jdk version "1.7.0_67" appears,

set this variable earlier:

REQ_JDK_VER="1.7.0_67"

This is used only in setting JAVA_HOME if it does not exist.

4. Removed jdk 1.6 lines since we stopped using that when directory

sql/hbaseExtensions was removed about the end of September.

5. Made indentation more consistent.

6. Made formatting of "if" and "do" more consistent.

7. Used double-brackets in conditionals, which is safer as

documented in http://mywiki.wooledge.org/BashFAQ/031

8. Use $(..) for subshells instead of less readable backticks.

In sqf/tools/sqtools.sh, added function remove_duplicates_in_path which

removes second and later appearances of the same value.

Change-Id: Iec48874b14d08ca8155006ab6a6a3f41c1714ca8

Give warning, not error, when Hadoop not found

When Trafodion has just been untarred from a product file obtained from

downloads.trafodion.org, and no Hadoop is found, give a warning instead of

an error and suggest that install_local_hadoop may be needed.

In that situation, do not put 4 colons at front of CLASSPATH.

Change-Id: If207b5999df37d3139015ff88bc4bde4686f7919

Removed duplicate file from mpi

This change removes mpi/mpiCC which was a duplicate of mpi/mpicc.

This fixes bug 1332744, "Two files with same case-insensitive name

in mpi", which made it harder to download Trafodion source to a

Windows system.

Change-Id: I343678e8275cfb820c16cc07df8297b0b8de699d

Better naming consistency in user scripts

Scripts installed into sql/scripts had inconsistent naming of the

software being used, and most have been corrected here.

Change-Id: Ie3f525f2b0760371975ebd0777d2abd7b6dc52e0

Added PTPA to list of Trafodion programs in pstat

The pstat script was missing the name of executable PTPA.

This may have caused problems because pstat is used in these

scripts in trafodion/core :

sqf/monitor/linux/montest_run.virtual

sqf/sql/scripts/psuspendall

sqf/sql/scripts/sqpstack

sqf/sql/scripts/pkillall

sqf/sql/scripts/presumeall

sqf/sql/scripts/bats/runsqlreg.virtual

sqf/sql/scripts/bats/runseabedbats.virtual

sqf/sql/scripts/bats/runtmbats.virtual

sqf/sql/scripts/cstat

sqf/tools/sqtools.sh

The pstat script now includes these comments, which previously were

needed when changing this script but not documented for developers:

# Trafodion program and script command names.

# Add new Trafodion programs and scripts to this list.

# Observe the following:

# 1. KEEP THE LIST IN ALPHABETICAL ORDER: A-Z then a-z

# 2. Separate each entry with a comma.

# 3. NO SPACES WHATSOEVER; this list is an argument to ps -C.

To report what pstat does, the -H option was added. Option -h

is more common for help, but that is already used and these other

scripts use "pstat -h" to suppress a 2-line header:

sqf/sql/scripts/psuspendall

sqf/sql/scripts/pkillall

sqf/sql/scripts/presumeall

The current help text is:

$ pstat -H

usage: pstat [-h] [-l] [-s] [-H]

-h Suppress header

-l List Trafodion programs. When -s is also used,

list only programs used in safe mode.

-s Safe mode, does not show distinguished programs:

monitor,sqwatchdog

-H Show help text

Change-Id: I4f843f5fb128c6d62f8eba8d1db53e33fb9c060c

Typos and misspellings fix 4

Corrected spelling of "disappeared" in 26 user messages in odbc

Change-Id: Ib7613a49aa05a93125d3769d4cdbf3d58ea4356c

Add ability to check for absolute files used as dlls

Added script and makefile target for detecting absolute filenames as

seen in bug 1365658. The make target is not yet used in Jenkins

builds. Because the check takes more than 30 seconds, it was not

added as part of the normal "make package" work, a 20+% increase in

running make is not worthwhile for a rare bug which may never happen

again.

Example:

$ time make find-absolute-dlls

sqf/build-scripts/find-abs-dlls

Searching <my-directory>

for absolute files used as dlls ...

ERROR: Found 6 files with absolute filenames as dlls:

sql/lib/linux/64bit/debug/sqlci

NEEDED /home/tools/jdk1.7.0_09_64/jre/lib/amd64/server/libjsig.so

sql/lib/linux/64bit/debug/tdm_arkesp

NEEDED /home/tools/jdk1.7.0_09_64/jre/lib/amd64/server/libjsig.so

sql/lib/linux/64bit/debug/tdm_udrserv

NEEDED /home/tools/jdk1.7.0_09_64/jre/lib/amd64/server/libjsig.so

sql/lib/linux/64bit/debug/tdm_arkcmp

NEEDED /home/tools/jdk1.7.0_09_64/jre/lib/amd64/server/libjsig.so

sql/lib/linux/64bit/debug/mxssmp

NEEDED /home/tools/jdk1.7.0_09_64/jre/lib/amd64/server/libjsig.so

sql/lib/linux/64bit/debug/mxsscp

NEEDED /home/tools/jdk1.7.0_09_64/jre/lib/amd64/server/libjsig.so

make: *** [find-absolute-dlls] Error 6

real 0m39.801s

user 0m26.196s

sys 0m9.438s

Also made a small cleanup to the existing check_man script.

Second patch set added -v to help text.

Change-Id: I22babe08dbf8a4620603fc670acc6b05e831222f

    • -0
    • +102
    /sqf/build-scripts/find-abs-dlls
Updated expected files for fullstack2/TEST062

The new Jenkins job core-regress-fullstack2-cdh4.4 was failing

because the release flavor expected file for TEST062 was out of date.

Both flavors have been updated, that is, EXPECTED062.SB for debug

and EXPECTED062.SB.RELEASE for release.

Change-Id: Iaabae7ce89f0b1c7a2ac909808d3e1f9090709ea

    • -30
    • +30
    /sql/regress/fullstack2/EXPECTED062.SB