Clone Tools
  • last updated 20 mins ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
DRILL-7473: Parquet reader failed to get field of repeated map

closes #1933

    • binary
    ./map/parquet/repeated_struct_with_dict.parquet
  1. … 5 more files in changeset.
DRILL-7096: Develop vector for canonical Map<K,V>

- Added new type DICT;

- Created value vectors for the type for single and repeated modes;

- Implemented corresponding FieldReaders and FieldWriters;

- Made changes in EvaluationVisitor to be able to read values from the map by key;

- Made changes to DrillParquetGroupConverter to be able to read Parquet's MAP type;

- Added an option `store.parquet.reader.enable_map_support` to disable reading MAP type as DICT from Parquet files;

- Updated AvroRecordReader to use new DICT type for Avro's MAP;

- Added support of the new type to ParquetRecordWriter.

    • binary
    ./map/parquet/000000_0.parquet
    • binary
    ./simple_map.parquet
  1. … 107 more files in changeset.
DRILL-4858: REPEATED_COUNT on an array of maps and an array of arrays is not implemented

- Implemented 'repeated_count' function for repeated MAP and repeated LIST;

- Updated RepeatedListReader and RepeatedMapReader implementations to return correct value from size() method

- Moved repeated_count to freemarker template and added support for more repeated types for the function

closes #1641

    • binary
    ./repeated_types.parquet
  1. … 8 more files in changeset.
DRILL-6670: Align Parquet TIMESTAMP_MICROS logical type handling with earlier versions + minor fixes

closes #1428

    • binary
    ./parquet_logical_types_complex.parquet
    • binary
    ./parquet_logical_types_complex_nodict.parquet
    • binary
    ./parquet_logical_types_complex_nullable.parquet
    • binary
    ./parquet_logical_types_complex_nullable_nodict.parquet
  1. … 12 more files in changeset.
DRILL-5797: Use Parquet new reader on all non-complex columns queries

    • binary
    ./complex_special_cases.parquet
  1. … 6 more files in changeset.
DRILL-5971: Fix INT64, INT32 logical types in complex parquet reader

Added the following types : ENUM (Binary annotated as ENUM) INT96 (Dictionary encoded)

Fixed issue with reading Dictionary encoded fixed width reader

Added test file generator

This closes #1049

    • binary
    ./logical_int_complex.parquet
    • binary
    ./parquet_logical_types_complex.parquet
    • binary
    ./parquet_logical_types_complex_nullable.parquet
  1. … 10 more files in changeset.
DRILL-4264: Allow field names to include dots

  1. … 98 more files in changeset.
DRILL-3533: Fix incorrect query result when querying parquet for fields that do not exists.

  1. … 2 more files in changeset.
DRILL-1835: Fix sort and join when results include complex types

    • -0
    • +109
    ./baseline_sorted.json
  1. … 7 more files in changeset.
DRILL-1835: Fix sort and join when results include complex types

    • -0
    • +109
    ./baseline_sorted.json
  1. … 7 more files in changeset.
DRILL-1830: Fix reading map type in parquet reader

  1. … 2 more files in changeset.
DRILL-1830: Fix reading map type in parquet reader

  1. … 2 more files in changeset.
DRILL-1818: Fix complex projection pushdown

Also fixes:

Issue where converted type was lost when projecting two elements in a repeated type

Issue when selecting multiple elements in a repeated type, but using inconsistent casing

  1. … 3 more files in changeset.
DRILL-1818: Fix complex projection pushdown

Also fixes:

Issue where converted type was lost when projecting two elements in a repeated type

Issue when selecting multiple elements in a repeated type, but using inconsistent casing

  1. … 3 more files in changeset.