Clone
 

spanchamia <spanchamia@maprtech.com> in drill

DRILL-3492: Add support for encoding/decoding of to/from OrderedBytes format

Description:

This change allows encoding/decoding of data from/to 'double', 'float',

'bigint', 'int' and 'utf8' data types to/from OrderedBytes format.

It also allows for OrderedByte encoded row-keys to be stored in

ascending as well as descending order.

The following JIRA added the OrderedBytes encoding to HBase:

https://issues.apache.org/jira/browse/HBASE-8201

This encoding scheme will preserve the sort-order of the native

data-type when it is stored as sorted byte arrays on disk.

Thus, it will help the HBase storage plugin if the row-keys have been

encoded in OrderedBytes format.

This functionality allows us to prune the scan ranges, thus reading much

lesser data from the server.

Testing Done:

Added a new unit-test class TestOrderedBytesConvertFunctions.java which

derives from TestConvertFunctions.java class.

Also add new test cases to TestHBaseFilterPushDown class that will test

if we were able to push-down filters correctly and if the results are

correct.

DRILL-3492 - * Remove repeated allocations of byte arrays and PositionedByteRange objects

on heap(as suggested by Jason).

* Remove OrderedBytes encode/decode operations on UTF8 types.

Reasons -

1. These operations are slow and incur a lot of heap allocations

2. UTF8 types maintain their natural sort order when stored as binary arrays.

DRILL-3492 - Remove test code that creates test tables with UTF8 OrderedByte encoding.

  1. … 4 more files in changeset.