Clone Tools
  • last updated a few minutes ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
DRILL-634: Cleanup/organize Java imports and trailing whitespaces from Drill code

    • -3
    • +5
    ./OrderedPartitionProjectorTemplate.java
  1. … 765 more files in changeset.
DRILL-991: Limit should terminate upstream fragments immediately upon completion

  1. … 36 more files in changeset.
Enable View persistence, Storage Plugin and System option persistence.

Conflicts:

exec/java-exec/src/main/java/org/apache/drill/exec/ExecConstants.java

exec/java-exec/src/test/java/org/apache/drill/exec/cache/TestCacheSerialization.java

  1. … 167 more files in changeset.
DRILL-865: Interface changes to AbstractRecordBatch to enable easy collection of stats. Add BaseRootExec as a wrapper to collect stats for senders.

    • -89
    • +84
    ./OrderedPartitionRecordBatch.java
  1. … 17 more files in changeset.
status changes

    • -84
    • +89
    ./OrderedPartitionRecordBatch.java
  1. … 76 more files in changeset.
Add support for RepeatedMapVector, MapVector and RepeatedListVector.

    • -2
    • +2
    ./OrderedPartitionProjectorTemplate.java
  1. … 135 more files in changeset.
Switch distributed cache to Infinispan Add Operator level metrics

    • -10
    • +10
    ./OrderedPartitionRecordBatch.java
  1. … 109 more files in changeset.
Move to Optiq 0.6 Also includes: -improve exception catching -move schema path parsing to Antlr -close zookeeper connection on if client created -enhance BaseTestQuery and have other query tests utilize it -Various test fixes for better memory release. still needs client allocator to be closed. -refactor DrillSqlWorker and create multiple SqlHandlers -Add PojoRecordReader and DirectPlan capabilities -Update Antlr to use same quoting rules as SQL: single quote for quoted strings, back ticks for identifiers -Move back to old Sorts until bugs are fixed -Refector SelectionVector management within Prels -Add support for NO_EXCHANGES option -Extract SchemaFactories to use Optiq's new Schema handling capabilities -Add basic handling of cancel in UserServer -Remove output requirement from Project -Add start of usercredentials to User communication

    • -21
    • +21
    ./OrderedPartitionRecordBatch.java
  1. … 181 more files in changeset.
DRILL-620: Memory consumption fixes

accounting fixes

trim buffers

switch to using setSafe and copySafe methods only

adaptive allocation

operator based allocator wip

handle OOM

Operator Context

    • -1
    • +3
    ./OrderedPartitionProjectorTemplate.java
    • -23
    • +52
    ./OrderedPartitionRecordBatch.java
  1. … 155 more files in changeset.
DRILL-335: Implement Hash Aggregation

1. Implementation of the hash aggregation execution operator - this has two main parts: the HashAggTemplate and the HashAggBatch.

2. Implementation of a hash table which is used by the hash aggregation. The hash table hash two main parts: the HashTableTemplate and the ChainedHashTable. The hash table internally uses the notion of 'BatchHolder' to keep track of all keys that can fit within one batch of 64K values. New BatchHolder objects are created as needed. Each BatchHolder has its own vector container. The HashAggregate also has a similar structure and it keeps track of the workspace variables.

(NOTE: An initial design document for the hash aggregation and hash table was already attached with Drill-335. The document has not yet been updated with the latest implementation ... will try to do that in the near future).

3. Jinfeng's changes to use workspace vectors in the generated code for aggregate functions (previously, for streaming aggregate we only needed to maintain workspace variable for 1 running group; however for hash aggregate we need to maintain it for all groups).

4. Fix for Drill-318: because of #3 above, the previous fix for Drill-318 is not valid anymore. I modified the template generation code for the aggregate functions such that they conform to the new infrastructure.

5. The original AggTemplate, AggBatch and Aggregator classes have been moved to corresponding StreamingAggTemplate, StreamingAggBatch and StreamingAggregator in order to differentiate it from hash aggregation. These appear as new files but the code there has not changed.

I have run several tests manually as part of TestHashAggr...these tests use TPC-H data and in particular a relatively large 'Orders' table. However, I have not yet packaged the tests to run as part of JUnit since the location and size of the parquet files needs to be figured out. I will continue to work on that.

  1. … 48 more files in changeset.
DRILL-387: Support using Hive simple UDFs in Drill * As part of this change FunctionDefinition (and related code) * are deleted, instead the same information available in * Function Holders are used whenever required * Freemarker/CodeModel codegen for Drill ObjectInspectors * Comparator function cleanup

  1. … 85 more files in changeset.
DRILL-346: Move constant expressions to setup.

  1. … 18 more files in changeset.
DRILL-450: Add exchange rules, move from BasicOptimizer to Optiq

  1. … 126 more files in changeset.
DRILL-257: Move SQL parsing to server side. Switch to Avatica based JDBC driver. Update QuerySubmitter to support SQL queries. Update SqlAccesors to support getObject() Remove ref, clean up SQL packages some. Various performance fixes. Updating result set so first set of results must be returned before control is return to client to allow metadata to populate for aggressive tools like sqlline Move timeout functionality to TestTools. Update Expression materializer so that it will return a nullable int if a field is not found. Update Project record batch to support simple wildcard queries. Updates to move JSON record reader test to expecting VarCharVector.getObject to return a String rather than a byte[].

    • -11
    • +12
    ./OrderedPartitionRecordBatch.java
  1. … 310 more files in changeset.
DRILL-352: Extract ClassGenerator from CodeGenerator. Use hierarchal set of ClassGenerators to support implementation of an arbitrarily complex inner class hierarchy. Introduce @RuntimeOveridden annotation to allow overrides of inner classes without requiring abstract inner classes. Update operators to use methods. Add test for inner class generation. Remove support for JDK based compilation, focusing entirely on Janino based compilation.

    • -25
    • +51
    ./OrderedPartitionRecordBatch.java
  1. … 34 more files in changeset.
DRILL-334: Subdivide Drillbit control and data messages. Add support for socket backpressure. Add TopLevel and Child memory allocator with debug mode to capture memory leaks. Various memory leak fixes to get build to complete.

Also includes fixes from reviews by Tim.

    • -180
    • +249
    ./OrderedPartitionRecordBatch.java
  1. … 212 more files in changeset.
DRILL-259: Implicit cast functionality

Includes following changes:

Implicit cast : WIP.

Implicit cast: work in progress 2.

implicit_cast: workin in progress 3.

implicit cast : work in progress. Prototype works.

Implicit cast: add unit testcase. Handle null vs non-null type.

Implicit cast: add unit test. change comparision operator's allsame from true to false.

Drill-259: implicit cast. Rebase on Drill-316 explicit cast branch.

DRILL-259: implicit cast . reverse change to codegenerator.

DRILL-259: implicit cast. code clean.

DRILL-259: code clean.

Drill-259: add apache license to 7 new java files. reverse the change to pom.xml

Drill-259: minor change to test case.

Drill-259: implicit cast - process NullExpression. Convert NullExpression into TypedNullConstant.

Drill-259: implicit cast. Revise according to review comments.

  1. … 33 more files in changeset.
DRILL-311: Replace OrderedPartitionBatchCreator with OrderedPartitionSenderCreator

    • -39
    • +0
    ./OrderedPartitionBatchCreator.java
    • -0
    • +47
    ./OrderedPartitionSenderCreator.java
  1. … 2 more files in changeset.
DRILL-271 Address code review comments. VectorAccessibleSerializable now take WritableBatch. Release and reconstruct code now part of Writablebatch class.

  1. … 5 more files in changeset.
DRILL-271 refactor DistributedCache code. Uses hazel cast 3.1 and custom serialization.

    • -10
    • +10
    ./OrderedPartitionRecordBatch.java
  1. … 12 more files in changeset.
DRILL-271 retool VectorContainerSerializable to work with containers and batches. also modify DrillSerializable to work with InputStream, OutputStream

  1. … 4 more files in changeset.
DRILL-254: Add iterator validator and correct interface violations

  1. … 11 more files in changeset.
DRILL-230: Addressing comments in code review, abstract out references to HazelCache and add comments

    • -1
    • +2
    ./OrderedPartitionProjectorTemplate.java
    • -94
    • +148
    ./OrderedPartitionRecordBatch.java
  1. … 32 more files in changeset.
DRILL-230: Build a sampling range partitioner

    • -0
    • +39
    ./OrderedPartitionBatchCreator.java
    • -0
    • +40
    ./OrderedPartitionProjector.java
    • -0
    • +101
    ./OrderedPartitionProjectorTemplate.java
    • -0
    • +403
    ./OrderedPartitionRecordBatch.java
    • -0
    • +63
    ./SampleCopierTemplate.java
    • -0
    • +61
    ./SampleSortTemplate.java
    • -0
    • +101
    ./SortContainerBuilder.java
  1. … 37 more files in changeset.