Clone Tools
  • last updated 21 mins ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
DRILL-7337: Add vararg UDFs support

  1. … 37 more files in changeset.
DRILL-2326: Fix scalar replacement for the case when static method which does not return values is called

- Fix check for return function value to handle the case when created object is returned without assigning it to the local variable

closes #1687

  1. … 2 more files in changeset.
DRILL-6524: Prevent incorrect scalar replacement for the case of assigning references inside if block

  1. … 3 more files in changeset.
DRILL-6868: Upgrade Janino compiler to 3.0.11

- Remove workaround where removing adjacent ALOAD-POP instruction pairs

- Remove ModifiedUnparser and use DeepCopier for modifying methods instead of modifying it with custom Unparser implementation

closes #1553

  1. … 3 more files in changeset.
DRILL-6422: Replace guava imports with shaded ones

  1. … 981 more files in changeset.
DRILL-6349: Drill JDBC driver fails on Java 1.9+ with NoClassDefFoundError: sun/misc/VM

closes #1446

  1. … 23 more files in changeset.
DRILL-6320: Fixed license headers.

closes #1207

  1. … 2055 more files in changeset.
DRILL-6094: Decimal data type enhancements

Add ExprVisitors for VARDECIMAL

Modify writers/readers to support VARDECIMAL

- Added usage of VarDecimal for parquet, hive, maprdb, jdbc;

- Added options to store decimals as int32 and int64 or fixed_len_byte_array or binary;

Add UDFs for VARDECIMAL data type

- modify type inference rules

- remove UDFs for obsolete DECIMAL types

Enable DECIMAL data type by default

Add unit tests for DECIMAL data type

Fix mapping for NLJ when literal with non-primitive type is used in join conditions

Refresh protobuf C++ source files

Changes in C++ files

Add support for decimal logical type in Avro.

Add support for date, time and timestamp logical types.

Update Avro version to 1.8.2.

  1. … 201 more files in changeset.
DRILL-3581: Upgrade HPPC to 0.7.1

  1. … 13 more files in changeset.
DRILL-2762: Update Fragment state reporting and error collection

DeferredException

- Add new throwAndClear operation on to allow checking for exceptions preClose in FragmentContext

- Add new getAndClear operation

BufferManager

- Ensure close() can be called multiple times by clearing managed buffer list on close().

FragmentContext/FragmentExecutor

- Update FragmentContext to have a preClose so that we can check closure state before doing final close.

- Update so that there is only a single state maintained between FragmentContext and FragmentExecutor

- Clean up FragmentExecutor run() method to better manage error states and have only single terminal point (avoiding multiple messages to Foreman).

- Add new CANCELLATION_REQUESTED state for FragmentState.

- Move all users of isCancelled or isFailed in main code to use shouldContinue()

- Update receivingFragmentFinished message to not cancel fragment (only inform root operator of cancellation)

WorkManager Updates

- Add new afterExecute command to the WorkManager ExecutorService so that we get log entries if a thread leaks an exception. (Otherwise logs don't show these exceptions and they only go to standard out.)

Profile Page

- Update profile page to show last update and last progress.

- Change durations to non-time presentation

Foreman/QueryManager

- Extract listenable interfaces into anonymous inner classes from body of Foreman

QueryManager

- Update QueryManager to track completed nodes rather than completed fragments using NodeTracker

- Update DrillbitStatusListener to decrement expected completion messages on Nodes that have died to avoid query hang when a node dies

FragmentData/MinorFragmentProfile

- Add ability to track last status update as well as last time fragment made progress

AbstractRecordBatch

- Update awareness of current cancellation state to avoid cancellation delays

Misc. Other changes

- Move ByteCode optimization code to only record assembly and code as trace messages

- Update SimpleRootExec to create fake ExecutorState to make existing tests work.

- Update sort to exit prematurely in the case that the fragment was asked to cancel.

- Add finals to all edited files.

- Modify control handler and FragmentManager to directly support receivingFragmentFinished

- Update receiver propagation message to avoid premature removal of fragment manager

- Update UserException.Builder to log a message if we're creating a new UserException (ERROR for System, INFO otherwise).

- Update Profile pages to use min and max instead of sorts.

  1. … 43 more files in changeset.
DRILL-2389: Remove timestamp with time zone

  1. … 35 more files in changeset.
DRILL-1385, along with some cleanup

Cleaned up option handling. This includes using finals, making member variables

private whenever possible, and some formatting.

- fixed a bug in the string formatting for the double range validator

- OptionValidator, OptionValue, and their implementations now conspire not to

allow the creation of malformed options because the OptionType has been added

to validator calls to handle OptionValues that are created on demand.

Started with updated byte code rewrite from Jacques

Fixed several problems with scalar value replacement:

- use consistent ASM api version throughout

- stop using deprecated ASM methods (actually causes bugs)

- visitMethodInsn()

- added a couple of missing super.visitEnd()s

- fixed a couple of minor FindBugs issues

- accounted for required stack size increases when replacing holders for

longs and doubles

- added accounting for frame offsets to cope with long and double local

variables and value holder members

- fixed a few minor bugs found with FindBugs

- stop using carrotlabs' hash map lget() method on shared constant data

- fixed an incorrect use of DUP2 on objectrefs when copying long or double

holder members into locals

- fixed a problem with redundant POP instructions left behind after replacement

- fixed a problem with incorrect DUPs in multiple assignment statements

- fixed a problem with DUP_X1 replacement when handling constants in multiple

assignment statements

- fixed a problem with non-replaced holder member post-decrements

- don't replace holders passed to static functions as "out" parameters

(common with Accessors on repeated value vectors)

- increased the maximum required stack size when transferring holder members to

locals

- changed the code generation block type mappings for constants for external

sorts

- fixed problems handling constant and non-constant member variables in

operator classes

- in general, if a holder is assigned to or from an operator member variable,

it can't be replaced (at least not until we replace those as well)

- Use a derived ASM Analyzer (MethodAnalyzer) and Frame

(AssignmentTrackingFrame) in order to establish relationships between

assignments of holders through chains of local variables. This effectively

back-propagates non-replaceability attributes so that if a holder variable

that can't be replaced is assigned to from another holder variable, that

second one cannot be replaced either, and so on through longer chains of

assignments.

- code for dumping generated source code

- MergeAdapter dumps before and after results of scalar replacement

(if it's on)

- fixed some problems in ReplacingBasicValue by replacing HashSet with

IdentityHashMap

- made loggers private

- added a retry strategy for scalar replacement

if a scalar replacement code rewriting fails, then this will try to

regenerate the bytecode again without the scalar replacement.

- bytecode verification is always on now (required for the retry strategy)

- use system option to determine whether scalar replacement should be used

- default option: if scalar replacement fails, retry without it

- force replacement on or off

- unit tests for the retry strategy are based on a single known failure case,

covered by DRILL-2326.

- add tests TestConvertFunctions to test the three scalar replacement options

for the failing test case (testVarCharReturnTripConvertLogical)

- made it possible to set a SYSTEM option as a java property in Drillbit

- added a command line argument to force scalar replacement to be on during

testing in the rootmost pom.xml

In the course of this, added increased checking of intermediate stages of code

rewriting, as well as logging of classes that cause failures.

- work around a bug in ASM's CheckClassAdapter that doesn't allow for checking

of inner classes

Added comments, tidied up formatting, and added "final" in a number of places.

Signed-off-by: vkorukanti <venki.korukanti@gmail.com>

    • -0
    • +333
    ./AloadPopRemover.java
    • -134
    • +392
    ./InstructionModifier.java
    • -0
    • +105
    ./MethodAnalyzer.java
    • -11
    • +30
    ./ValueHolderReplacementVisitor.java
  1. … 43 more files in changeset.
DRILL-1904 - Part 1: Package level docs for the common module and a few packages in exec.

Fix line wrapping on package level docs to standardize at 80 character wrap.

Patch updated to address review comments.

Patch updated again to address Sudheesh's review comments.

  1. … 28 more files in changeset.
DRILL-1402: Add check-style rules for trailing space, TABs and blocks without braces

  1. … 440 more files in changeset.
DRILL-634: Cleanup/organize Java imports and trailing whitespaces from Drill code

  1. … 768 more files in changeset.
Switch to DrillBuf Add @Inject DrillBuf Move comparison functions to memory sensitive ones Add scalar replacement functionality for value holders Simplify date parsing function Add local compiled code caching

    • -0
    • +259
    ./InstructionModifier.java
    • -0
    • +57
    ./ReplacingBasicValue.java
    • -0
    • +91
    ./ReplacingInterpreter.java
    • -0
    • +66
    ./ScalarReplacementNode.java
    • -0
    • +117
    ./ScalarReplacementTypes.java
    • -0
    • +69
    ./TrackingInstructionList.java
    • -0
    • +223
    ./ValueHolderIden.java
    • -0
    • +75
    ./ValueHolderReplacementVisitor.java
  1. … 205 more files in changeset.