Clone Tools
  • last updated 17 mins ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
DRILL-6540: Updated Hadoop and HBase libraries to the latest versions

Hadoop: 3.2.1

HBase: 2.2.2

closes #1895

  1. … 82 more files in changeset.
DRILL-7310: Move schema-related classes from exec module to be able to use them in metastore module

closes #1816

  1. … 101 more files in changeset.
DRILL-7273: Introduce operators for handling metadata

closes #1886

  1. … 155 more files in changeset.
DRILL-7011: Support schema in scan framework

* Adds schema support to the row set-based scan framework and to the "V3" text reader based on that framework.

* Adding the schema made clear that passing options as a long list of constructor arguments was not sustainable. Refactored code to use a builder pattern instead.

* Added support for default values in the "null column loader", which required adding a "setValue" method to the column accessors.

* Added unit tests for all new or changed functionality. See TestCsvWithSchema for the overall test of the entire integrated mechanism.

* Added tests for explicit projection with schema

* Better handling of date/time in column accessors

* Converted recent column metadata work from Java 8 date/time to Joda.

* Added more CSV-with-schema unit tests

* Removed the ID fields from "resolved columns", used "instanceof" instead.

* Added wildcard projection with an output schema. Handles both "lenient" and "strict" schemas.

* Tagged projection columns with their output schema, when available.

* Scan projection added modes for wildcard with an output schema. The reader projection added support for merging reader and output schemas.

* Includes refactoring of scan operator tests (the test file grew too large.)

* Renamed some classes to avoid confusing reader schemas with output schemas.

* Added unit tests for the new functionality.

* Added "lenient" wildcard with schema test for CSV

* Added more type conversions: string-to-bit, many-to-string

* Fixed bug in column writer for VarDecimal

* Added missing unit tests, and fixed bugs, in Bit column reader/writer

* Cleaned up a number of unneded "SuppressWarnings"

closes #1711

  1. … 224 more files in changeset.
DRILL-5603: Replace String file paths to Hadoop Path - replaced all String path representation with org.apache.hadoop.fs.Path - added PathSerDe.Se JSON serializer - refactoring of DFSPartitionLocation code by leveraging existing listPartitionValues() functionality

closes #1657

  1. … 81 more files in changeset.
DRILL-6901: Move schema builder to src/main

Moves the SchemaBuilder class out of the src/test name space into the src/main namespace. Specifically, into the existing record.metadata package.

Many files changed in this move. Corrected two minor issues: import of the wrong Arrays class and unnecessary annotations.

  1. … 88 more files in changeset.
DRILL-6540: Upgrade to HADOOP-3.0.3 libraries

- accomodate apache and mapr profiles with hadoop 3.0 libraries

- update HBase version

- fix jdbc-all woodox dependency

- unban Apache commons-logging dependency

  1. … 10 more files in changeset.
DRILL-6422: Replace guava imports with shaded ones

  1. … 980 more files in changeset.
DRILL-6656: Disallow extra semicolons and multiple statements on the same line.

closes #1415

  1. … 143 more files in changeset.
DRILL-6644: Don't reserve space for incoming probe batches unnecessarily during the build phase.

git closes #1409

  1. … 6 more files in changeset.
DRILL-6549: batch sizing for nested loop join

closes #1363

  1. … 3 more files in changeset.
DRILL-6310: limit batch size for hash aggregate

closes #1324

  1. … 9 more files in changeset.
DRILL-6512: Remove unnecessary processing overhead from RecordBatchSizer

closes #1341

  1. … 11 more files in changeset.
DRILL-6461: Added basic data correctness tests for hash agg, and improved operator unit testing framework.

git closes #1344

  1. … 33 more files in changeset.
DRILL-6236:Batch sizing for hash join

This closes #1227

  1. … 11 more files in changeset.
DRILL-6385: Support JPPD feature

  1. … 61 more files in changeset.
DRILL-6431: Unnest operator requires table and a single column alias to be specified. Fixing the issues related to star column renaming, same field name renaming and also enforcing that an alias column is required for the unnest operator.

  1. … 17 more files in changeset.
DRILL-6356: batch sizing for union all

closes #1255

  1. … 6 more files in changeset.
DRILL-6389: Fixed building javadocs - Added documentation about how to build javadocs - Fixed some of the javadoc warnings

closes #1276

  1. … 65 more files in changeset.
DRILL-6320: Fixed license headers.

closes #1207

  1. … 2061 more files in changeset.
DRILL-6307: Handle empty batches in record batch sizer correctly

closes #1228

  1. … 5 more files in changeset.
DRILL-6340 Output Batch Control in Project using the RecordBatchSizer

Changes required to implement Output Batch Sizing in Project using the RecordBatchSizer.

closes #1302

  1. … 56 more files in changeset.
DRILL-6275: Fixed direct memory reporting in sys.memory.

closes #1176

  1. … 3 more files in changeset.
DRILL-6231: Fix memory allocation for repeated list vector

closes #1171

  1. … 1 more file in changeset.
DRILL-6162: Enhance record batch sizer to retain nesting information. Refactor record batch sizer and add unit tests for sizing and vector allocation.

  1. … 5 more files in changeset.
DRILL-6210: Enhanced test schema utilities

closes #1150

  1. … 54 more files in changeset.
DRILL-6205: Reduce memory consumption of testFlattenUpperLimit test

closes #1147

DRILL-6164: Heap memory leak during parquet scan and OOM

closes #1122

  1. … 14 more files in changeset.
DRILL-6027: Initial implementation of HashJoin spill, without memory limits checks yet

  1. … 29 more files in changeset.
DRILL-6027: - Added memory claculator - Added unit tests and docs. - Fixed IOB caused by output vector allocation. - Don't double count records that were spilled in HashJoin

  1. … 55 more files in changeset.