Clone Tools
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
Add info messages when DcsServer is waiting for DcsManager znode

Closes Bug #1443597

Change-Id: I39c83591ab52a2a2c3795e068122c1df52ce3d4a

    • -0
    • +9
    ./trafodion/dcs/server/ServerManager.java
  1. … 1 more file in changeset.
some MT Servers do not return back to AVAILABLE

Fixed synchronization between Listener and Worker Threads

Change-Id: I9929a2713b262a265556c8e1d2930642ba22bebd

LP 1442730 Eliminating SQL Cancel for PUBLISHING Queries

DCSMaster checks if OID ends with text "_PUBLISHING" and if yes, ignores

SQL Cancel

Patch #1

Fixed reported problems

Change-Id: I4b913517f5cbb20bfca203c6f9bfa9fa9abd5047

Fix for bug 1443561

SQL plan collection enabled by default now.

Fixes bug 1443561

Change-Id: Ibc9cb9e41d37a384a6f0bf4cb34c625ea0a98dd7

Fix for bug 1443561

SQL plan collection enabled by default now.

Fixes bug 1443561

Change-Id: Ibc9cb9e41d37a384a6f0bf4cb34c625ea0a98dd7

Removed check for period in FQDN

When property dcs.dns.interface is specified some systems return

only the short DNS name. DCS assume that returned name always

contains at least one "." This caused a null value to be inserted

into zookeeper nodes for the hostname.

closes bug: #1442402

Change-Id: Idb9abd3c63bba1293adb4797902f7128e3c3f07c

Updated floating IP documentation

Added addtional text in architecture and configuration.

Fixed some nits from last code review

Change-Id: I2183f76a61034c08ab377b53e9f3e3a790b2d5fb

    • -1
    • +1
    ./trafodion/dcs/master/FloatingIp.java
  1. … 2 more files in changeset.
Added floating IP feature

DcsMaster backups are configured in the conf/backup-masters file.

Whenever a backup takes over it now executes the bin/scripts/dcsbind.sh

script to move the floating IP to its own node. There are 3 new

properties necessary. They are:

dcs.master.floating.ip

Disabled by default. This property is used to enable/disable the

the floating IP feature.

dcs.master.floating.ip.external.interface

There is no default. The user must specify the network interface

to use for floating ip.

dcs.master.floating.ip.external.ip.address

There is no default. The user must specify the dotted IP

address to use for floating ip.

Patch set #2

removed hostname from backup-masters file

cleaned up trailing spaces

Patch set #3

updated copyrights

Change-Id: Ib9f9aa618716a0424c2dd160e5dc819c6c1a33eb

    • -445
    • +496
    ./trafodion/dcs/Constants.java
    • -210
    • +262
    ./trafodion/dcs/master/DcsMaster.java
    • -0
    • +121
    ./trafodion/dcs/master/FloatingIp.java
    • -114
    • +91
    ./trafodion/dcs/master/MasterLeaderElection.java
    • -585
    • +696
    ./trafodion/dcs/master/ServerManager.java
    • -50
    • +86
    ./trafodion/dcs/script/ScriptContext.java
    • -37
    • +64
    ./trafodion/dcs/util/DcsNetworkConfiguration.java
  1. … 7 more files in changeset.
Fixed query cancel sent by ODBC Driver

If Sql Cancel is sent by JDBC Driver, srvrObjRef has only portNumber.

When the request is sent by ODBC Driver, its format is following:

"TCP:IpAddress/PortNumber:ODBC".

To get the portNumber we have to parse srvrObjRef string

===============================

Replaced ip Address by <IpAddress> and port number by <portNumber>

Change-Id: Ic839bee2be44e9ca88fa2f8cd2cb1a72ff5ef994

Fixes in serverMT

1. In pom.xml changed jdbct2 version to 1.1.0 and

repository to org.trafodion.jdbc.t2.T2Driver

2. Changed processing of server state to make connect/disconnect process

more robust

3. Cleanup code

4. Fixed problem with connecting timeout

5. Added to Sql Connection two options: ENFORCE_ISO88591 and

DOWNLOAD_CERTIFICATE

============================================

Patch

1. Changed pom.xml back to previous T2 dev repository and version

20150329_0830

Change-Id: I5ca6f2055d1e6fd81b4f65e8d5e6b26df9fe308e

    • -68
    • +78
    ./trafodion/dcs/servermt/ServerUtils.java
  1. … 1 more file in changeset.
partialy fix for query cancel

1. prepared DcsMaster for accepting request sent by T4 Driver to cancel

the query

2. Cleanup the code. Changing tabs to spaces.

3. Added processing unknown requests.

===============================================

Patch

1. To servermt added processing EndTransaction

2. To Master Listener added processing requests in LITTLE_ENDIAN

3. To servermt added fix for Logging

4. Fixed Exception message in RequestCancelQuery

5. In servermt fixed bugs reported in Phoenix Test.

===================================================

Patch

1. Fixed wrong text exception

2. Added checking if LOG is on

3. Removed listing LOG properties

4. Fixed precision for TIME and TIMESTAMP

5. Changed TYPE_EXPLAIN to return result set

6. Changed in SqlFetch process EOD

7. To SqlPrepare added more LOG info

8. Changed TConnection/TStatement closing

================================================

Patch

1. added query cancel procedure.

2. to pom.xml changed hadoop version to 2.5.0-cdh5.3.0 to make T2 driver

work

3. to servermt added SqlClose Api

4. fixed more problems in servermt

================================================

Patch

1. in pom.xml changed hadoop version to 2.6.0 and removed cloudera

repository.

2. fixed connecting timeout.

3. added connecting server state.

4. cleanup debug log messages.

5. restored in ClientData accidently deleted code.

================================================

Patch

1. fixed TAB-SPAC issues

===============================================

Patch

Fixed issues reported in code review

Change-Id: Ifd0b7835c87dbf84d5ea0dde05219a0651f19a74

    • -40
    • +52
    ./trafodion/dcs/master/listener/ClientData.java
    • -257
    • +262
    ./trafodion/dcs/master/listener/ConnectReply.java
    • -0
    • +254
    ./trafodion/dcs/master/listener/RequestCancelQuery.java
    • -0
    • +128
    ./trafodion/dcs/master/listener/RequestGetObjectRef.java
    • -0
    • +87
    ./trafodion/dcs/master/listener/RequestUnknown.java
    • -25
    • +41
    ./trafodion/dcs/servermt/ServerUtils.java
  1. … 12 more files in changeset.
Convert DSC documentation from docbooks to asciidoc

Docbooks was very difficult for contributors to easily add

updates. More time was spent fixing xml tags than writing

the needed documentation. Asciidoc while very feature rich in

its tag set is much much easier to learn.

Change-Id: Ib23ba6c11c8b0f13023119e181ea16efddd07ac6

    • -475
    • +491
    ./trafodion/dcs/rest/model/ServerModel.java
  1. … 34 more files in changeset.
DcsServer checks if Trafodion is running before mxosrvr restart

DcsServer aggresively tried to restart its child mxosrvrs.

In scenarios where Trafodion had been stopped abruptly e.g.,

when "ckillall" command was executed, it would cause many

mxosrvr processes to start then quickly die. This change

adds check to ensure that Trafodion is running before ever

attempting a restart of any mxosrvr.

Change-Id: If430ce94e6071e402786d73d544d2ab23306d1fe

Closes-Bug: #1411475

    • -399
    • +490
    ./trafodion/dcs/server/ServerManager.java
  1. … 1 more file in changeset.
Logging for JdbcT2 driver now directed to DCS log

A companion commit to bug/1419112. DCS log4j.properties now used

when calling Trafodion HBaseClient.

See bug 1419112

Change-Id: Ic160d7307f938e12150bbf9c30f2272bd604f415

    • -17
    • +26
    ./trafodion/dcs/util/JdbcT2Util.java
    • -0
    • +97
    ./trafodion/dcs/util/Log4jUtils.java
  1. … 1 more file in changeset.
Enabled statistics

Change-Id: I0e63a0769c258a63c068771af8f606f52a2b10ed

  1. … 1 more file in changeset.
Enabled statistics

Change-Id: I0e63a0769c258a63c068771af8f606f52a2b10ed

  1. … 1 more file in changeset.
added new DCS Serverna

fixed reported issues:

1. copyright 2015

2. changed tabs to spaces

3. versioning in T2Driver

4. Added detailed description

fixed reported issues.

changed the name of Dcs Serverna to new name Dcs Servermt

changed System.out.println to LOG.debug

changed serverna to servermt in jamon

added property dcs.server.handlers.threads.max. This

property limits number of threads created by Dcs Server.

Default value is 10.

Fixed error in dcs

fixed description of the property

cleanup the code

changed system.out.println to LOG

Change-Id: Ib5b47a5c09171ddd1b1f062fe8e7b26bf31779bd

    • -1
    • +1
    ./trafodion/dcs/master/MasterLeaderElection.java
    • -1
    • +1
    ./trafodion/dcs/rest/client/Client.java
    • -1
    • +1
    ./trafodion/dcs/rest/client/Cluster.java
    • -1
    • +1
    ./trafodion/dcs/rest/client/Response.java
    • -0
    • +248
    ./trafodion/dcs/servermt/DcsServer.java
    • -0
    • +70
    ./trafodion/dcs/servermt/Metrics.java
    • -0
    • +349
    ./trafodion/dcs/servermt/ServerConstants.java
    • -0
    • +328
    ./trafodion/dcs/servermt/ServerManager.java
    • -0
    • +47
    ./trafodion/dcs/servermt/ServerStatusServlet.java
    • -0
    • +456
    ./trafodion/dcs/servermt/ServerUtils.java
  1. … 45 more files in changeset.
Added new property to disable sqlplan for user queries

Changed the property name isolated to statistics

Change-Id: I378270984ef22afc699063341cb30901e19611ab

    • -1
    • +4
    ./trafodion/dcs/server/ServerManager.java
  1. … 1 more file in changeset.
Added new property to disable sqlplan for user queries

Changed the property name isolated to statistics

Change-Id: I378270984ef22afc699063341cb30901e19611ab

(cherry picked from commit f14119cdcb82d59cb8d83a2af7f7485a13fc5f2b)

    • -1
    • +4
    ./trafodion/dcs/server/ServerManager.java
  1. … 1 more file in changeset.
Disabled statistics, updated documentation & copyright notices.

Change-Id: I5bd92fa3a65007be792f7e97f2d66cc90b443823

  1. … 17 more files in changeset.
Enable repository status by default closes bug: 1413382

Change-Id: Idd1433657c8dfec66ad066ec5914f6daa3e7fcb9

  1. … 1 more file in changeset.
Fixed issue with Trafodion query tools connection pool

Added more error checking to handle errors

Change-Id: I9c66c03cb45643b15602afb91f05c2d78c7d5b55

    • -3
    • +15
    ./trafodion/dcs/master/QueryPlanModel.java
    • -3
    • +3
    ./trafodion/dcs/master/ServerManager.java
  1. … 6 more files in changeset.
Trafodion repository fixes

Ability to query 3 repository tables and visual query plan.

Change-Id: Iedfb3c879d6d7f2c1c0a66472a9ae28ad7d64eb9

    • -0
    • +584
    ./trafodion/dcs/master/QueryPlanModel.java
    • -0
    • +35
    ./trafodion/dcs/master/QueryPlanResponse.java
    • -2
    • +7
    ./trafodion/dcs/master/ServerManager.java
    • -0
    • +239
    ./trafodion/dcs/serverna/DcsServer.java
    • -0
    • +70
    ./trafodion/dcs/serverna/Metrics.java
    • -0
    • +449
    ./trafodion/dcs/serverna/ServerManager.java
    • -0
    • +47
    ./trafodion/dcs/serverna/ServerStatusServlet.java
    • -146
    • +84
    ./trafodion/dcs/util/JdbcT4Util.java
  1. … 90 more files in changeset.
Merge "Disable mxosrvr port map Closes-Bug: #1408454"

Disable workload statistics publishing Closes-Bug: #1409091

Disable the publishing of workload statistics in mxosrvr.

This is set in the following property:

dcs.server.user.program.statistics.enabled

Change-Id: I7d6227dbe04f50aa670ceb0a21aaf95dcd4454be

Disable mxosrvr port map Closes-Bug: #1408454

In order to improve mxosrvr start time performance 2 new properties were

added:

dcs.server.user.program.port.map.timeout.seconds = 60

dcs.server.user.program.port.bind.timeout.seconds = 30

They are passed to mxosrvr at start time. The DcsMaster would create a

port map in the servers/registered znode so mxosrvrs could identify

their DcsServer instance/child id and discover the TCP/IP port to use.

During performance testing, it was found that because DCS/MXOSRVRs usage

of ports in the ephemeral range DCS couldn't guarantee dedicated use of

a given range of ports. Setting the 2 properties to -1 disables port

mapping and returns to original herding style port identification

dcs.server.user.program.port.map.timeout.seconds = -1

dcs.server.user.program.port.bind.timeout.seconds = -1

Change-Id: I0497bfbc58878378d67a8cba9293708e88dc271e

Manageability changes in DCS

1) Port mapping

At startup DcsMaster now creates a list of TCP/IP ports based on the

entries in the conf/servers file and writes it to the zookeeper

/<username>/trafodion/dcs/servers/registered znode. Each MXOSRVR reads

the port map list and retrieves its unique port number.

2) Trafodion repository

JDBC type4 is used to access SQL from DcsMaster. The feature is disabled

by default but may be enabled by overriding the unpublished

"dcs.master.trafodion.repository" property with a value of .true. in

your dcs-site.xml file.

3) HA for DcsMaster

A new file, /bin/master-backup.sh, was added that when called from

dcs-start/dcs-stop script look for the presence of optional

/conf/backup-masters file. This feature works like the existing

.servers. in that you can specify hostnames, one per line, e.g.,

.localhost.. The script will start/stop DcsMaster processes on all hosts

defined in the file. The DcsMaster has new leader election code so that

only one DcsMaster process is the .leader.. All others wait, in a

non-herding fashion, for other DcsMasters to fail. The vision is that

the Seaquest .ndcsbind. script may be run prior to starting DCS but

more effort is needed to test this. This commit sets up the

infrastructure. There.s no property for this feature as it.s enabled

only the /conf/backup-masters file exists and has host names defined.

4) Trafodion logs

As a convenience DcsMaster can display the Trafodion log files found in

$MY_SQROOT/logs. There was no requirement for this rather I added for my

development use. Be warned that DcsMaster is not aware of Trafodion

configuration changes so right now it only looks in $MY_SQROOT/logs.

This feature is disabled by default but may be enabled by overriding the

unpublished "dcs.master.trafodion.logs" property with a value of .true.

in your dcs-site.xml file.

5) Basic Security

Provide basic infrastructure to authenticate DcsMaster Web UI users. A

new file /conf/realm.properties was added as a first step. It allows for

definition of users with roles. The embedded Jetty web server reads this

file and uses it to authenticate users attempting to access the

DcsMaster web UI. This feature will surely change over time. It also

supports #3 above as we need username/password for the connection to SQL

in T4 driver. We want users to only see/cancel their own queries. This

feature is disabled by default but may be enabled by overriding the

unpublished "dcs.master.security" property with a value of .true. in

your dcs-site.xml file.

Change-Id: I3e258f698dd828e683f0de03c2fe67c3ce4a5d26

    • -122
    • +205
    ./trafodion/dcs/http/HttpServer.java
    • -7
    • +43
    ./trafodion/dcs/master/DcsMaster.java
    • -0
    • +177
    ./trafodion/dcs/master/MasterLeaderElection.java
    • -6
    • +5
    ./trafodion/dcs/master/MasterStatusServlet.java
    • -0
    • +11
    ./trafodion/dcs/master/RunningServer.java
    • -0
    • +96
    ./trafodion/dcs/master/ServerItem.java
    • -40
    • +251
    ./trafodion/dcs/master/ServerManager.java
    • -15
    • +31
    ./trafodion/dcs/rest/DcsRest.java
    • -1
    • +1
    ./trafodion/dcs/rest/RestConstants.java
    • -67
    • +25
    ./trafodion/dcs/rest/RootResource.java
    • -0
    • +152
    ./trafodion/dcs/rest/ServerResource.java
  1. … 50 more files in changeset.
mxosrvr exit when heapsize become large,dcs part, change after last review

Change-Id: I881aba688404ad704a4c1ea4fe27cae3dd7dda84

    • -0
    • +3
    ./trafodion/dcs/server/ServerManager.java
  1. … 1 more file in changeset.
Externalized all of dcs.cloud.command to property file dcs-default.xml

Parts of the dcs.cloud.command were hard coded inside of DCS. With

this change all of the command is moved to the existing dcs.cloud.command

property contained in the DCS jar file resources. This allows the user

to manipulate the command without changing any DCS code. It's important

for the user manipulating the command to remember that the command must

return the result back in the form <internal ip addr>,<external id addr>

as DCS parses the string.

Change-Id: Icda9c688b63782a586382ae1f61f2f9c0b763174

Closes-Bug: #1370596

    • -19
    • +19
    ./trafodion/dcs/util/DcsNetworkConfiguration.java
Fix start-dcs.sh to ensure only a single DcsMaster is started

This is a companion fix to previous commit for stop-dcs.sh

change.

Closes Bug: 1326130

Change-Id: I8fb4974523f406f8b916d6844d84bf5420c50240

    • -10
    • +14
    ./trafodion/dcs/zookeeper/ZkUtil.java
  1. … 2 more files in changeset.