ExHbaseSelect.cpp

Clone Tools
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
TRAFODION-1586 Add support for external Traf tables mappedd to HBase tables

This is the initial support for HBase mapped tables.

A doc for external manuals will be added later.

regress/seabase/TEST022 contains various testcases.

  1. … 77 more files in changeset.
[TRAFODION-1421] Implement parallel scanner primitive new CQD: hbase_dop_parallel_scanner - default 0.0 = disabled - from 0.x to 1.0 -> the DOP will be dynamically assessed as a percentage of region to scan. If 10 region, and cqd is .3 (30%), then 3 threads will be used... - from 2.0 and above, will take ceiling, and use it as fixed number of threads. Advantage of parallel scanner parallelism over ESP is on resource consumption. Parallel scanner is just creating threads, while ESP are full proceses wth high memory consumption. For now, this feature is just a primitive that can be manually exercised. The next obvious step is to make the compiler aware of it and pick it over ESP DOP when appropriate...

  1. … 21 more files in changeset.
update

  1. … 85 more files in changeset.
Add comments to explain the use of stack variable to minimize impact on runtime code based optimization.

[TRAFODION-1900]

When doing MDAM scans, we are performing interlaced scan for PROBE and for real scan. The probes always return only 1 row, then we close the scanner immediately, therefore should use always small scanner. I will make it conditional on the existing CQD HBASE_SMALL_SCANNER (ether SYSTEM or ON). In addition, caching of blocks retrieved by probe should always we at least receiving one succesfull cache hit on the next MDAM scan, therefore forcing caching ON for MDAM prob is a good idea. Again, will make this forcing conditional on HBASE_SMALL_SCANNER SYSTEM or ON.

Then for the real scan part of MDAM, I will use the following heuristic: If previous scan fitted in one hbase block, then it is likelly than next will also fit in one hbase block, therefore enable small scanner for next scan. Again all this only if CQD above is ON or SYSTEM.

Also includes a fix where SMALL_SCANNER would be turned on for MDAM scan because the compiler for MDAM is not polulating the expected number of rows returned.

Results of using small scanner on MDAM when it make sense showed a 1.39X speed improvement...

  1. … 2 more files in changeset.
Implement TRAFODION-1420 Use ClientSmallScanner for small scans to improve perdormance Hbase implements an optimization for small scan (defined as scanning less than a data block ie 64Kb) resulting in 3X performance improvement. The underlying trick is about cutting down RPC calls from 3 (OpenScan/Next/Close) to 1, and use pread stateless instead of seek/read state-full and locking method to read data. This JIRA is about improving the compiler who can be aware if a scan will be acting on single data block (small) or not, and pass this data to executor so that it can use the right parameter for scan. (scan.setSmall(boolean)). reference: https://issues.apache.org/jira/browse/HBASE-9488 https://issues.apache.org/jira/browse/HBASE-7266

  1. … 27 more files in changeset.
[TRAFODION-1448] Leak in HTableClient objects

Refactored the code to ensure that HTableClient object in java

is released for every batch of gets.

In the process removed unused code in HTableClient_JNI.cpp and

ExpHbaseInterface.cpp

  1. … 6 more files in changeset.
Rework for review comments + resolve merge conflict

Merge branch 'master' of github.com:apache/incubator-trafodion into khaled_cif

Conflicts:

core/sql/generator/GenRelScan.cpp

  1. … 4 more files in changeset.
[Trafodion-1424] Enable CIF for Trafodion scan operator When a row is read is read by the scan operator from native HBase or Trafodion tables (either aligned or regular format) a convert expression is applied to convert the row into a form usable by rest of query tree. Output of convert expression is now in aligned format, with varchars not being blank padded. Change can be disabled with cqd COMPRESSED_INTERNAL_FORMAT 'OFF'. Majority of change is due to Khaled Bouaziz.

  1. … 5 more files in changeset.
Part 2 of changes to update license text in Trafodion source

This check-in updates license text in all source files in the

core/sql directory. Remaining directories will be done in

later check-ins.

  1. … 3147 more files in changeset.
Merge remote branch 'core/master'

  1. … 108 more files in changeset.
Move core into subdir to combine repos

    • -0
    • +1881
    ./ExHbaseSelect.cpp
  1. … 10768 more files in changeset.
Move core into subdir to combine repos

    • -0
    • +1875
    ./ExHbaseSelect.cpp
  1. … 10622 more files in changeset.
Move core into subdir to combine repos

Use: git log --follow -- <file>

to view file history thru renames.

    • -0
    • +1898
    ./ExHbaseSelect.cpp
  1. … 10837 more files in changeset.