DRILL-5716: Queue-driven memory allocation * Creates new core resource management and query queue abstractions. * Adds queue information to the Protobuf layer. * Foreman and Planner changes
- Abstracts memory management out to the new resource management layer. This means deferring generating the physical plan JSON to later in the process after memory planning.
* Web UI changes
* Adds queue information to the main page and the profile page to each query. * Also sorts the list of options displayed in the Web UI.
- Added memory reserve
A new config parameter, exec.queue.memory_reserve_ratio, sets aside a slice of total memory for operators that do not participate in the memory assignment process. The default is 20% testing will tell us if that value should be larger or smaller.
* Additional minor fixes
- Code cleanup. - Added mechanism to abandon lease release during shutdown. - Log queue configuration only when the config changes, rather than on every query. - Apply Boaz’ option to enforce a minimum memory allocation per operator. - Additional logging to help testers see what is happening.
DRILL-5512: Standardize error handling in ScanBatch Standardizes error handling to throw a UserException. Prior code threw various exceptions, called the fail() method, or returned a variety of status codes.