drill

Clone Tools
  • last updated a few minutes ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
DRILL-2127: make JDBC tests re-use bit instances; reset cached bits on failure

DRILL-1597: refactor empty population logic out & re-use across RepeatedList/Map vectors & add unit test

DRILL-2143: Part 1 - just remove RecordBatch from UDF setup method.

Disable date/time functions as they can no longer access incoming recordbatch to get query start time or timezone.

During rebase DateTypeFunctions.CurrentDate was updated to include a commented out vesion of the fix from DRILL-2372,

this will be fixed in the new patch for part 2 of 2143.

Remove RecordBatch from setup of a new new UDFs.

  1. … 110 more files in changeset.
MD-143: Update DrillFileSystem package location after DRILL-2080.

DRILL-2172: make web UI bound check before accessing a vector, returning null for overflowing indices; fix a non-string type rendering problem;

DRILL-1605: ensure GenericAccessor returns null for non-existent records

DRILL-2169: disable embedded server for tests when not needed

DRILL-1953: alter session set store.json.all_text_mode does not work as documented

    • -0
    • +1
    /exec/java-exec/src/test/resources/jsoninput/big_numeric.json
DRILL-2151: VariableLengthVector.copyFromSafe() unnecessary sets the offsetVector

DRILL-2173: Partition queries to drive dynamic pruning

Adds new interface on the QueryContext as well as individual schemas for exploring partitions of tables.

Adds injectable type for partition explorer for use in UDFs. This is hooked into both to expression

materialization and interpreted evaluation. The FragmentContext throws an exception to tell users to turn on

constant folding if a UDF that uses the PartitionExplorer makes it past planning.

2173 update -Address Chris' review comments.

Change the PartitionExplorer to return an Iterable<String> instead of String[]

Add interface level description to PartitionExplorer and StoragePluginPartitionExplorer.

New inner class in FileSystemPlugin to fulfill the new Iterable interface for partitions.

Formatting/cleanup fixes

Clean up error reporting code in MaxDir UDF. Remove method to get a string from a DrillBuf, as it was already defined in StringFunctionHelpers. Add new utility method to specifically convert a VarCharHolder to a string to remove some boilerplate.

Fixed an errant copy paste in a comment and removed unused imports.

Fix docs in FileSystemPlugin, belongs with the 2173 changes.

Fix references in Javadoc to properly use @link instead of @see.

2173 fixes, correctly return an empty list of sub-partitions if the path requested in the partition explorer interface is a file. Fix a few docs.

More 2173, finishing Chris' comments

2173 update - Add validation for PartitionExplorer injectable in UdfUtiltiers.

small change to fix refactored unit tests.

cleanup for 2173

Fix maxdir UDF so it can compile in runtime generated code as well as the interpreted expression system (needed to fully qualify classes and interfaces). It still fails to execute, as we prevent requesting a schema from a non-root fragment. We do not expect these types of functions to ever be used without constant folding so this should not be an issue.

Update error message in the case where the partition explorer is being used outside of planning.

Adding free marker generated maxdir, imaxdir, mindir and imindir

remove import that violates build checks, fix typo in new test class name

Separate out SubDirectoryList from FileSystemSchemaFactory.

Fix unit test to correctly test all four functions.

Update partition explorer to take List instead of Collection. As the lists are used in parallel it should be explicit that these are expected to be ordered (which Collections do not guarantee).

Drop the extra file generated due to the header in the free marker template and fix a typo and remove an unused import.

  1. … 5 more files in changeset.
DRILL-1988: Create child vectors while initializing complex vectors using Materialized Field

DRILL-2142: Refactor complex vector classes. Move the logic that applies specifically to maps in a separate base class that will be used by RepeatedMap and Map vectors.

DRILL-1872: Assertion failure and empty map on order by with complex data

DRILL-1921: Disable union, intersect, except, natural join, cross join; Disable 'cardinality' ; Disable types tinyint, smallint, real, multiset

DRILL-2187: Single Broadcast Sender

Also includes:

1. Fix merge join planning issue (1c5c810 by jinfengni)

2. ExternalSort: Check the memory available for in-memory sorting or not in making decision to spill or not (36f9dd1)

3. Cleanup in ExternalSortBatch and its helper classes (36f9dd1)

4. MergeJoinBatch: Limit the outgoing record batch size to 2^15 (37dfeb8)

5. StreamingAggBatch: Limit outgoing record batch size to 2^15 (7d8a2e4)

  1. … 43 more files in changeset.
DRILL-2092: For HashJoin and MergeJoin, process the null comparisons for IS NOT DISTINCT FROM operator.

Added test input and modified test to use baseline CSV.

Modified comparator string to reflect calcite's SqlKind.

DRILL-2126: make jdbc connection impl honor parameters passed from user

DRILL-2054: Interpret "||" as string concat operator; Disable `&&`;

DRILL-2236: Optimize hash inner join by swapping inputs based on row count comparison. Add a planner option to enable/disable this feature.

Revise code based on review comments.

DRILL-2109: Fix for parquet dictionary page reading

DRILL-2087: Fix an incorrect join plan by ensure either both sides are distributed, or none of them is distributed.

Add unit testcase.

MD-125: Update HBase storage plugin to support HBase 0.98

DRILL-2090: Update HBase storage plugin to support HBase 0.98

    • -222
    • +12
    /contrib/storage-hbase/pom.xml
DRILL-2088: ReAlloc in FixedWidthVector.setValueCount if current buffer capacity is less than the valueCount.

    • -0
    • +33
    /exec/java-exec/src/test/resources/remover/sv_with_no_filter.json
New blog post

    • -0
    • +40
    /blog/_posts/2015-01-27-schema-free-json-data-infrastructure.md
DRILL-2079: Use AllocationHelper to allocate memory for vectors.

DRILL-2078: Remove deprecated VectorAllocator and modify its users

DRILL-2072: Fix getGroupCount() in RepeatedMapVector

DRILL-2069: Fix star query prefix logic it's used with subquery in where clause. Convert some star query query unit test to verify query results.

Add new unit test case based on review comments.

DRILL-1545: Allow custom extensions for json files

Fix case where compressed files are skipped.

Add jackson annotation to avoid serializing the default extensions list to enable forwards compatibility.