drill

Clone Tools
  • last updated 12 mins ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
Tool to build documentation DAG (docs.json) and markup to consume that data in documentation nav and breadcrumbs.

    • -10779
    • +30
    /_includes/doc_toc.html
    • -0
    • +78
    /_tools/createdatadocs.py
Fixed paths

Fixed base path

Merge branch 'gh-pages' of https://github.com/tshiran/drill into gh-pages

New doc site style

    • -0
    • +6
    /_includes/breadcrumbs.html
    • -0
    • +17
    /_includes/breadcrumbs.html.gen
    • -0
    • +3
    /_includes/doc_nav.html
    • -0
    • +10789
    /_includes/doc_toc.html
    • -0
    • +4
    /_includes/doc_toc_categories.html
    • -0
    • +15
    /_layouts.old/default.html
    • -0
    • +25
    /_layouts.old/docpage.html
    • -0
    • +9
    /_layouts.old/page.html
    • -0
    • +45
    /_layouts.old/post.html
  1. … 8 more files in changeset.
DRILL-2423: Show proper message when trying to drop an unknown view.

DRILL-2761: ParquetGroupScan's copy constructor should copy the content of a collection data member.

DRILL-2221: Do not skip writing an empty list

small fix for bad resolution of merge conflict in free marker template

Fix test to use dfs_test instead of the default dfs storage plugin

DRILL-2717 (continuation) Fix checkstyle.

DRILL-2766: Removed SystemRecordReader and SystemRecords + Use PojoRecordReader with ThreadsIterator and MemoryIterator + Includes DRILL-2670: Added a test case since DRILL-2714 resolves this issue

DRILL-2350: Improve exception handling and error messages in JSON reader.

    • -0
    • +1
    /exec/java-exec/src/test/resources/jsoninput/DRILL-2350.json
DRILL-2570 - exclude "META-INF/services" dir from jdbc-all

DRILL-2741: Continue waiting for completion of sent batches when interrupted

DRILL-2675: Implement a subset of User Exceptions to improve how errors are reported to the user

  1. … 24 more files in changeset.
DRILL-2613: 1-Hygiene/prep.: Split DrillResultSet impl. vs. intf.; renaming.

Code "hygiene":

- Renamed parameter index to rowOffset, and documented it, in SqlAccessor.

- Renamed index to rowOffset in AbstractSqlAccessor and BoundCheckingAccessor too.

- Fixed a cast to DrillResultSet to be a ResultSet.unwrap(...) call.

Preparation: Split DrillResultSet into interface vs. implementation:

- Moved old implementation class org.apache.drill.jdbc.DrillResultSet

to new implementation class org.apache.drill.jdbc.impl.DrillResultSetImpl.

- Created new interface org.apache.drill.jdbc.DrillResultSet,

declaring method getQueryId(...).

- Relatedly, in nearby JDBC code:

- Renamed various references.

- Added some "public" qualifiers. Most are only for the interim until

DRILL-2089 (moving other implementation classes from org.apache.drill.jdbc

to org.apache.drill.jdbc.impl) is completed sufficiently. (See marking

with "DRILL-2089".)n

- (Files: old DrillResultSet, DrillResultSetImpl, new DrillResultSet;

DrillConnectionImpl, DrillCursor, DrillJdbc41Factory, MetaImpl;

JdbcTestQueryBase.)

DRILL-2729: Fix deserialization issues with Hive decimal type partition columns.

DRILL-2762: Update Fragment state reporting and error collection

DeferredException

- Add new throwAndClear operation on to allow checking for exceptions preClose in FragmentContext

- Add new getAndClear operation

BufferManager

- Ensure close() can be called multiple times by clearing managed buffer list on close().

FragmentContext/FragmentExecutor

- Update FragmentContext to have a preClose so that we can check closure state before doing final close.

- Update so that there is only a single state maintained between FragmentContext and FragmentExecutor

- Clean up FragmentExecutor run() method to better manage error states and have only single terminal point (avoiding multiple messages to Foreman).

- Add new CANCELLATION_REQUESTED state for FragmentState.

- Move all users of isCancelled or isFailed in main code to use shouldContinue()

- Update receivingFragmentFinished message to not cancel fragment (only inform root operator of cancellation)

WorkManager Updates

- Add new afterExecute command to the WorkManager ExecutorService so that we get log entries if a thread leaks an exception. (Otherwise logs don't show these exceptions and they only go to standard out.)

Profile Page

- Update profile page to show last update and last progress.

- Change durations to non-time presentation

Foreman/QueryManager

- Extract listenable interfaces into anonymous inner classes from body of Foreman

QueryManager

- Update QueryManager to track completed nodes rather than completed fragments using NodeTracker

- Update DrillbitStatusListener to decrement expected completion messages on Nodes that have died to avoid query hang when a node dies

FragmentData/MinorFragmentProfile

- Add ability to track last status update as well as last time fragment made progress

AbstractRecordBatch

- Update awareness of current cancellation state to avoid cancellation delays

Misc. Other changes

- Move ByteCode optimization code to only record assembly and code as trace messages

- Update SimpleRootExec to create fake ExecutorState to make existing tests work.

- Update sort to exit prematurely in the case that the fragment was asked to cancel.

- Add finals to all edited files.

- Modify control handler and FragmentManager to directly support receivingFragmentFinished

- Update receiver propagation message to avoid premature removal of fragment manager

- Update UserException.Builder to log a message if we're creating a new UserException (ERROR for System, INFO otherwise).

- Update Profile pages to use min and max instead of sorts.

  1. … 30 more files in changeset.
DRILL-2754: Correctly allocate offset vector size for repeated type based on the number of groups being transferred, not the number of child values.

Test various levels of nesting, the core change fixes all three of these previously failing cases.

Test improvements, helper methods for easy construction of maps and lists in baselines.

    • -0
    • +6
    /exec/java-exec/src/test/resources/store/json/null_list.json
DRILL-2719: ValueVector#getBuffers(clear) must consistently clear vectors & retain buffers

    • -53
    • +35
    /_docs/manage/conf/001-mem-alloc.md
DRILL-1384: Part 2 - Unit test cases change.

Change two JDBC test cases, since the results for query w/o ORDER BY are not semantically deterministic.

Make sure TestHashJoinAdvanced uses HashJoin only. Modify expected results in TestProjectPushDown: the order of pushed columns could be different.

Disable TPCH Q16, due to OPTIQ-373 plan change.

Disable one parquet writer in TestExample, since it need ensure the created file does not exist on the file system.

DRILL-2685: Unique-ify local Hive metastore directory per test JVM instance.

    • -2
    • +2
    /_docs/sql-ref/functions/004-string.md
    • -4
    • +13
    /_docs/sql-ref/nested/004-repeated-contains.md
DRILL-2718: Move counting and tracking of sent batches to FragmentContext

Creates wrapper classes FragmentDataTunnel and FragmentUserDataTunnel which wrap

the DataTunnel and UserClientConnection, respectively, allowing us to use DataTunnels

and UserClientConnections from a global pool, but track pending batches and send status

at the FragmentContext level.

Consolidates the various StatusListener implementations used by the various senders and

instead uses just one implementation.

DRILL-2733: Make dfs_test.tmp schema location on local fs exclusive to test JVM fork.

  1. … 5 more files in changeset.
DRILL-2714: For SystemTable, Enforce Width only if the table is distributed (e.g., sys.memoy, sys.drillbits, sys.threads)

    • -15
    • +15
    /_docs/sql-ref/functions/004-string.md
minor doc changes

    • -6
    • +85
    /_docs/sql-ref/functions/005-aggregate.md
    • -20
    • +20
    /_docs/sql-ref/functions/006-nulls.md
    • -19
    • +23
    /_docs/connect/009-mapr-db-plugin.md
    • -19
    • +10
    /_docs/develop/contribute/002-ideas.md
    • -1
    • +413
    /_docs/manage/conf/001-mem-alloc.md
    • -0
    • +3
    /_docs/sql-ref/002-lexical-structure.md
    • -0
    • +889
    /_docs/sql-ref/functions/002-conversion.md
  1. … 4 more files in changeset.