traf_config_setup

1. Added ssh test to make sure able to connect

2. Fixed tar bug when running installer from other directory than

/installer/

3. Testing to make sure user is not running trafodion_installer from

the trafodion user id.

4. Added message to make sure user checks their firewall settings

if they can not access Hadoop.

Edit 1: Changed error reporting. Change locatoin of Trafodion user id

check.

Edit 2: Most check outside the for loop. Changed "much" to "must".

Change-Id: I967c94ff23a0f0d55a1ca3a8ec7d51b7fb48307f

Edit 1: Previous code was overwriting all previous reserved ports.

Edit 2: Typo

Edit 3: Typos. Added support for saving previous reserved ports.

Edit 4: Spacing.

Change-Id: I19294116da0aa062129a90a2e2ae83ca19a26237

************** SPECIAL NOTE **************

With this check-in and going forward, it's necessary to download both the

installer and trafodion tar files in order to run the scanner. In other

words, it's no longer possible to just copy the 2 scanner files,

trafodion_scanner and trafodion_scanner.cfg, onto a cluster and

run the scanner (actually, it's still possible to do this in test

environments, by providing a trafodion configuration file with the

relevant settings via the scanner -t (--trafodion_config_file) option).

Nonetheless, the following scanner use cases are still supported:

1. Scanner is automatically invoked by the installer

2. Scanner is manually invoked

a. Before running the installer

b. After running the installer

************** trafodion_scanner.cfg changes **************

+ added back the HBaseVersion and HBaseJavaVersion checks

--> now the HBaseVersion check takes into account

the CDH_5_3_HDP_2_2_SUPPORT setting from the

the Trafodion configuration file

+ added HadoopDistro check to check for supported Hadoop distribution,

'hortonworks' or 'cloudera'

+ removed RECORD_CONFIG for "Hadoop distribution", because it's no longer

needed

+ fixed Bug 1442258 by adding KernelPidMaxLimit check

+ added RECORD_CONFIG for "Kernel parameters"

************** trafodion_scanner changes **************

+ added logic to read the Trafodion configuration file and set placeholder

variables for use in various checks

--> added -t (--trafodion_config_file) option to specify the

Trafodion configuration filename

(default is /etc/trafodion/trafodion_config)

--> renamed the -c option from --config_file to --scanner_config_file,

for consistency with the new -t (--trafodion_config_file) option

--> removed the -n (--nodes) option, since the node info now comes

from the Trafodion configuration file

--> removed the -x (--execute_local_commands) option for two reasons:

1) it makes less sense now 2) it's increasingly hard to maintain

--> right now the following settings are used from the Trafodion

configuration file: NODE_LIST, CDH_5_3_HDP_2_2_SUPPORT

+ removed the hadoop prerequisite checks because they are no longer needed,

given the following checks in trafodion_scanner.cfg: HadoopDistro,

SameHadoopVersion, HBaseVersion, and SameHBaseVersion

+ changed logger name from "trafodion_scanner_logger" to "trafodion_scanner"

to reduce unnecessary clutter in the log file

+ cleaned up the logging code

+ adjusted the help text

+ fixed a bug in the group_nodes_by_string function

************** trafodion_install changes **************

+ adjusted scanner invocation

--> removed the -n option and added the -t option,

per the changes mentioned above

************** traf_config_setup changes **************

+ fixed how permissions are set for the $TRAF_CONFIG file

Change-Id: I195cb437dece75cca45cc4c221a423c7fc3b8d26

Scanner running by default.

Fixes to trafodion_uninstaller (removing --instance option).

Edit 1: Changed error message, added echo statements to

trafodion_uninstaller to clarify, fixed HDP version bug.

Change-Id: If6fe0bc88abbcb634eabf4a292a0975aa87f4cc0

Corrected 4 misspellings and about 10 uses of "its" which should be "it is".

Deleted some but not all trailing spaces in the modified files.

Change-Id: I382110a24e325686f2bcf48e88a7a2394d34618b

The scanner will now be ran by default. The --no_scanner option can be

given to trafodion_install to by pass the scanner.

Simple security has now been automated. Users will be prompted to give

LDAP config information and then traf_authentication_setup will be ran

before sqstart.

Edit 1: Deleted unused variable and changed log directory for the

scanner logs.

Edit 2: Copyright

Edit 3: Changing the scanner to not be ran by default. Due to errors during

testing.

Change-Id: I60f75d11e7be615e475b3ffe4f9b4227ae6e7971

Removed output redirection

Change-Id: I86ee98d8a7e6ce6b36739ced2f36c770459d5c50

Testing sudo access on all nodes.

Added error checking to make sure all nodes entered are valid.

Edit 1: Small change to error checking

Change-Id: Ife2c6ec9769be94b9f26a9c94fef74c9c1b43d56

Replaced the previous method of distro detection with a more reliable

one and removed an error check that is now made obsolete.

Change-Id: I772dcd042a115535869c6199646653c1e1b0562f

Added error for when user trys to use a JAVA JRE instead of a JDK.

Fixed installer directory copy error that was preventing new files

from being copied to /usr/lib/trafodion/installer.

Edit 1: Changed order of Java checks.

Change-Id: I7d72b3892767e9d37f700dd2ff1b622c6b99476e

Change-Id: I1dcdab43105c7b504ee5985943aa5113fc030668

Trafodion.conf file was not being copied to all nodes. Needed to change

permissions on copy directory.

Edit 1: Changing permissions on $LOCAL_WORKDIR to allow multiple people

to install on one cluster. Fixed bug when setting $HOME_DIR. Fix to bug

1440229.

Edit 2: Fixed typo and permission on set of files.

Edit 3: Fixed Scanner return error bug. THANKS YVES.

Edit 4: Added case if scanner fails.

Change-Id: I5a237cbae4779b91121a467b1afba65f796be4cc

Bug fix 1429923: traf_config_setup will now check if epel path entered

exists and if the file is an rpm. This questions will also only be asked

if this is the first time installing trafodion. Will warn users that if a

file is not entered that we will try to download it later. In traf_setup all nodes

are checked to see if epel is installed, if not and a path to the file

is not given then it will be downloaded and installed on all nodes

missing this rpm.

Added error message to trafodion_uninstaller if config file is not

found.

Change-Id: Ia4f254e238d2f4f6ca4fd2826c8a7a71797d9855

Changed the way Linux system limits are set for trafodion

previously they were set in /etc/security/limits.conf and now

they are set for the trafodion userid in our own file in

/etc/security/limits.d/trafodion.conf in traf_setup

Fixed bug where node list kept getting appended to when user

entered node names on subsequent runs of the installer instead of

resetting the node list in traf_config_setup

Deleted some old code no longer used in traf_add_user

Added a recusive option to a chmod to handle the case when the

directory already existed and had other files in it in

trafodion_install.

Change-Id: I971799c93786b810b878e6434d8aae28c67b4db3

Added REST Server installation/configuration

Fixed JAVA_HOME not getting set correctly on all nodes

Removed LOCAL_WORKDIR from config file which caused problems

when later installs were ran from a different sudo userid.

Fixed problem where NODE_LIST was getting reversed node order

Fixed traf_*_mods98 scripts that were using $PWD instead of

$LOCAL_WORKDIR which caused hbase-trx jar file to not be

copied to all nodes.

patch 1: Added code to support older Trafodion builds that do

not have the REST build tar file.

Also moved some pdsh commands to after pdsh has been installed.

Change-Id: I3a1c51255ca34899a003257f8e442267dddfbdf1

Now able to use multiple command line arguments at one time.

Edit 1: Correcting $STARTING_NODE to be current hostname

Edit 2: Correcting $STARTING_NODE to be current hostname only for single

node install. Adding checkes to make sure Trafodion and Hadoop are

matching versions.

Edit 3: Correcting $STARTING_NODE

Change-Id: Icb29082d6362614f536cd170a13714ec61c359ba

Scanner will now only run if --run_scanner command line option is used.

Edit 1: Fix to get correct Hortonworks repo.

Change-Id: Iac7d6f7a2cb3969ed3ce39bb744aaebd941482fd

Edit 1: Added error message if TRAF_BUILD is not found.

Edit 2: Forgot "then"

Change-Id: I1ea7015709c72ccff5db8f86fafffa39a9d969a1

This code is dependent on a file being added to Trafodion core to work

for Cloudera 5.3 and Hortonworks 2.2. Until that file is added, install

will fail on Cloudera 5.3 and Hortonworks 2.2 but should build fine on

any other version.

Edit 1: Adding check to make nothing greater than Cloudera 5.3.* is

installed or Hortonworks 2.2.* is installed. Adding fix to NFS check.

Edit 2: Copy-paste error.

Edit 3: Changing permissions and type on package name

Edit 4: Error fixing

Change-Id: I065bef3d52651b8cda18deacd0bec6d2343ad1d8

If a build file was specified (as opposed to a package file) by the user

the values from previous runs was not being picked up as the

default values on subsequent runs. This fixes that for the

trafodion build and DCS build tar files.

Change-Id: I7bc941a0789a2619268d2dfb439daf7f593e0c94

Changed how NODE_LIST is created. A list of nodes is entered by the user

and that list is used to ssh to each node given which then the

"hostname" command is used to get hostname which is used to create

NODE_LIST that is used to generate sqconfig. Users can now enter in IP

addresses or node names. All hostname checks are now removed.

The scanner can now be called from trafodion installer.

Edit 1: Removed code used for testing

Edit 2: Corrected TRAF_VERSION (for when a debug build is used) and

added changes to how scanner is called.

Change-Id: I5671be3566c1fc3af0d7441254d9fa52150db2ba

Bug 1415661: Check to make sure trafodion user id to install trafodion

with (normally 'Trafodion') is not the same as the sudo user.

Bug 1421373: Install gzip if not installed

Change-Id: Ib9e0c080a4c3aaa95ef230045437aafbe9d1be10

Removed all hard coded "trafodion" and replaced with $TRAF_USER

Change-Id: I78c5df3830b3264be91f98df3b5aa83c1689a2d6

Changed name of traf_installer to traf_start to avoid confusion with new

script trafodion_install.

Checking to make sure there is a correct Java path in Java version

check.

Added fix to trafodion_uninstaller to all --instance option to be ran by

root or sudo user.

Change-Id: I263f5d9969f3e53e745dc497ceead0c33ece873c

Removed all code checking for HP Cloud since it is not need.

Fixed Java version check as it was not currently working as expected

Made changes to uninstaller.

Change-Id: I4b6a61f14f0870b0d8c921765532f317243a1147

pdsh was being called before it might have been installed. Now

installing pdsh during traf_config_setup since it will be needed.

Change-Id: Ied68568826c0edffb6251313722b4d98586a03ab

This is a checkin of the single script project which basically

wraps all the various scripts into one single script that the user

runs. The script now has the logic to determine which Hadoop distro

is installed as well as to determine if it is a full install or only

an upgrade and many other changes.

This was developed and tested in a private shared repo so

a review of the code has already been done. QA has approved this

to be checked in.

edit1: fixed shift problem in parsing parameters

edit2: Fixed problems with user config file

Added more error checking for config file

edit3: Fixed config directory creation

edit4: Fixed copyright and OpenStack VM package installation

edit5: Changed Java version detection

No longer do yum removes if package is there already

edit6: Refixed OpenStack VM package installation that got

overlaid in a merge

Added chown command to handle case where SQ_ROOT was not

a sub-directory of the Trafodion userid's home dir

Change-Id: I37a331c89c311feee430541b69f5f7dfe2177e60