http_request.c

Checkout Tools
  • last updated 5 hours ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
core: handle morphing buckets setaside/reinstate and kill request core filter.

The purpose of ap_request_core_filter() is not clear, it seems to prevent

potential morphing buckets to go through AP_FTYPE_CONNECTION filters which

would fail to set them aside (ENOTIMPL), and read them (unbounded) in memory.

This patch allows ap_filter_setaside_brigade() to set morphing buckets aside

by simply moving them, assuming they have the correct lifetime (either until

some further EOR, or the connection lifetime, or whatever). IOW, the module is

responsible for sending morphing buckets whose lifetime needs not be changed

by the connection filters.

Now since morphing buckets consume no memory until (apr_bucket_)read, like FILE

buckets, we don't account for them in flush_max_threshold either. This changes

ap_filter_reinstate_brigade() to only account for in-memory and EOR buckets to

flush_upto.

Also, since the EOR bucket is sent only to c->output_filters once the request

is processed, when all the filters < AP_FTYPE_CONNECTION have done their job

and stopped retaining data (after the EOS bucket, if ever), we prevent misuse

of ap_filter_{setaside,reinstate}_brigade() outside connection filters by

returning ENOTIMPL. This is not the right API for request filters as of now.

Finally, ap_request_core_filter() and co can be removed.

  1. … 6 more files in changeset.
Define ap_method_mask_t (typedef for apr_uint64_t) and use for method

bitmasks rather than apr_int64_t. Fixes UBSan errors shifting to the

top bit of a signed integer.

* include/httpd.h: Add ap_method_mask_t, use it for AP_METHOD_BIT.

(struct ap_method_mask_t): Likewise for method_mask field.

(struct request_rec): Likewise for allowed field.

* include/http_config.h (struct cmd_parms): Likewise for limited field.

* include/ap_mmn.h: Bump MMN major.

* modules/*/*.c: Adjust all method masks to use ap_method_mask_t.

  1. … 8 more files in changeset.
Follow up to r1840265: really privatize ap_filter_{recycle,adopt_brigade}().

Move ap_filter_adopt_brigade()'s declaration to "server/core.h" (private).

For ap_filter_recycle(), make it static/internal to util_filter (renamed to

recycle_dead_filters() which better fits what it does). It's now also called

unconditionally from ap_filter_input_pending() which itself is always called

after the request processing and from MPM event (as input_pending hook).

  1. … 4 more files in changeset.
Follow up to r1840149: core input filter pending data.

Since r1840149 ap_core_input_filter() can't use use f->[priv->]bb directly, so

ap_filter_input_pending() stopped accounting for its pending data.

But ap_core_input_filter() can't (and doesn't need to) setaside its socket

bucket, so ap_filter_setaside_brigade() is not an option. This commit adds

ap_filter_adopt_brigade() which simply moves the given buckets (brigade) into

f->priv->bb, and since this is not something to be done blindly (the buckets

need to have c->pool/bucket_alloc lifetime, which is the case in the core

filter) the function is not AP_DECLAREd/exported thus can be used in core only.

With ap_filter_adopt_brigade() and ap_filter_reinstate_brigade(), the core

input is now ap_filter_input_pending() friendly.

Also, ap_filter_recycle() is no more part of the API (AP_DECLARE removed too),

there really is no point to call it outside core code. MAJOR bumped once again

because of this.

  1. … 4 more files in changeset.
util_filter: protect ap_filter_t private fields from external (ab)use.

Introduce opaque struct ap_filter_private to move ap_filter_t "pending", "bb"

and "deferred_pool" fields to the "priv" side of things.

This allows to trust values set internally (only!) in util_filter code, and

make useful assertions between the different functions calls, along with the

usual nice extensibility property.

Likewise, the private struct ap_filter_conn_ctx in conn_rec (from r1839997)

allows now to implement the new ap_acquire_brigade() and ap_release_brigade()

functions useful to get a brigade with c->pool's lifetime. They obsolete

ap_reuse_brigade_from_pool() which is replaced where previously used.

Some comments added in ap_request_core_filter() regarding the lifetime of the

data it plays with, up to EOR...

MAJOR bumped (once again).

  1. … 7 more files in changeset.
core: follow up to r1839997: some runtime optimizations.

We don't mind about cleaning up a connection filter when its pool is being

cleaned up already. For request filters, let pending_filter_cleanup() do

nothing if the given filter is not pending (anymore), which allows to save a

cleanup kill when the filter is removed.

Clear (zero) the reused filters (ap_filter_t) on reuse rather than cleanup,

then a single APR_RING_CONCAT() can be used to recycle dead_filters in a one

go.

Always call ap_filter_recycle() in ap_filter_output_pending(), even if no

filter is pending, and while at it fix s/ap_filter_recyle/ap_filter_recycle/

silly typo.

  1. … 3 more files in changeset.
core: follow up to r1839997: recycle request filters to a delayed ring first.

We want not only ap_filter_output_pending() to be able to access each pending

filter's *f after the EOR is destroyed, but also each request filter to do

the same until it returns.

So request filters are now always cleaned up into a dead_filters ring which is

merged into spare_filters only when ap_filter_recycle() is called explicitely,

that is in ap_process_request_after_handler() and ap_filter_output_pending().

The former takes care of recycling at the end of the request, with any MPM,

while the latter keeps recycling during MPM event's write completion.

  1. … 4 more files in changeset.
core: axe data_in_in/output_filter from conn_rec.

They were superseded by ap_filter_should_yield() and ap_run_in/output_pending()

in r1706669 and had poor semantics since then (we can't maintain pending

semantics both by filter and for the whole connection).

Register ap_filter_input_pending() as the default input_pending hook (which

seems to have been forgotten in the first place).

On the MPM event side, we don't need to flush pending output data when the

connection has just been processed, ap_filter_should_yield() is lightweight and

enough to determine whether we should really enter write completion state or go

straight to reading. ap_run_output_pending() is used only when write completion

is in place and needs to be completed before more processing.

  1. … 6 more files in changeset.
core: integrate data_in_{in,out}put_filter to ap_filter_{in,out}put_pending().

Straightforward for ap_filter_input_pending() since c->data_in_input_filter is

always checked wherever ap_run_input_pending(c) is.

For ap_filter_output_pending(), this allows to set c->data_in_output_filter in

ap_process_request_after_handler() and avoid an useless flush from mpm_event.

  1. … 5 more files in changeset.
core: Add ap_reuse_brigade_from_pool().

Current RETRIEVE_BRIGADE_FROM_POOL macro from "http_request.c" is turned into

a helper and used in ap_request_core_filter().

We will need it in a subsequent commit in "util_filter.c" too.

  1. … 4 more files in changeset.
* modules/http/http_request.c (ap_process_request_after_handler,

ap_process_request): Cache and retrieve the brigade structure used

to send EOR and FLUSH between requests in c->pool userdata, to avoid

allocating a brigade structure per-request out of c->pool.

Submitted by: rpluem, jorton

  1. … 1 more file in changeset.
PR62186: preserve %<m for ErrorDocument internal redirects

*) core: Preserve the original HTTP request method in the '%<m' LogFormat

when an path-based ErrorDocument is used. PR 62186.

[Micha Lenk <micha lenk.info>]

Submitted By: Micha Lenk

Committed By: covener

  1. … 1 more file in changeset.
Fix some typos reported in PR 59998

Most add already been fixed when PR 59990 had been applied on trunk.

Thx klemens

  1. … 15 more files in changeset.
Fix timeout logging in ap_process_request().

We can't use 'r' after ap_process_request_after_handler(), the core output

filter might have cleaned up its deferred bucket brigade on error, including

the EOR bucket.

Reported by: steffenal

Fixes SpiderLabs/ModSecurity#1542

core: don't send EOR bucket through request filters.

The core request filter is the only one which should take care of it.

In theory the other request filters should have bailed out on EOS already,

but that's not always the case (and even less the case on error).

So be safe by not sending them a bucket which may destroy the request (and

their brigade) underneath them.

core, mod_rewrite: introduce the 'redirect-keeps-vary' note

to allow proper Vary header insertion when

dealing with a RewriteRule in a directory

context.

This change is an attempt to fix a long standing problem,

brought up while working on PR 58231. Our documentation clearly

states the following:

"If a HTTP header is used in a condition this header is added

to the Vary header of the response in case the condition

evaluates to true for the request."

This is currently not true for RewriteCond/Rules working in

a directory context, since when an internal redirect happens

all the outstanding response headers get dropped.

There might be a better solution so I am looking forward to

hear more opinions and comments. My goal for a delicate change

like this one would be to affect the least amount of configurations

possible, without triggering unwanted side effects.

If the solution is good for everybody tests will be written

in the suite asap.

  1. … 2 more files in changeset.
ap_check_pipeline: clarify/simplify !max_blank_lines logic, no functional change.
[mod_proxy_]http: follow up to r1750392.

Export [ap_]check_pipeline() and use it also for ap_proxy_check_connection(),

so that all the necessary checks on the connection are done before reusing it.

  1. … 7 more files in changeset.
Revert r1756064 and r1756060 until fixed (tests framework passes).
  1. … 7 more files in changeset.
http: follow up to r1750392.

Restore AP_MODE_READBYTES cleared inadvertently in r1756060.

[Reverted by r1756065]

[mod_proxy_]http: follow up to r1750392.

Export [ap_]check_pipeline() and use it also for ap_proxy_check_connection().

[Reverted by r1756065]

  1. … 7 more files in changeset.
Follow up to r1734656: restore c->data_in_input_filters usage to

see if it helps unblocking test framework.

  1. … 5 more files in changeset.
core: Extend support for setting aside data from the network input filter

to any connection or request input filter.

  1. … 13 more files in changeset.
core: follow up to r1710095, r1710105.

We can do this in a single (no inner) loop, and simplify again the logic.

core: follow up to r1710095.

Simplify logic in check_pipeline(), and log unexpected errors.

  1. … 1 more file in changeset.
core: Limit to ten the number of tolerated empty lines between request,

and consume them before the pipelining check to avoid possible response

delay when reading the next request without flushing.

Before this commit, the maximum number of empty lines was the same as

configured LimitRequestFields, defaulting to 100, which was way too much.

We now use a fixed/hard limit of 10 (DEFAULT_LIMIT_BLANK_LINES).

check_pipeline() is changed to check for (up to the limit) and comsume the

trailing [CR]LFs so that they won't be interpreted as pipelined requests,

otherwise we would block on the next read without flushing data, and hence

possibly delay pending response(s) until the next/real request comes in or

the keepalive timeout expires.

Finally, when the maximum number of empty line is reached in

read_request_line(), or that request line does not contains at least a method

and an (valid) URI, we can fail early and avoid some failure detected in

further processing.

  1. … 3 more files in changeset.
Make sure we free the main request rather than the final internal

redirect.

core: Extend support for asynchronous write completion from the

network filter to any connection or request filter.

  1. … 15 more files in changeset.
mod_ssl: follow up to r1705194, r1705823, r1705826 and r1705828.

Add CHANGES entry, and restore ap_process_request_after_handler()'s comment

as prior to r1705194 (the change makes no sense now).

  1. … 1 more file in changeset.
mod_ssl: forward EOR (only) brigades to the core_output_filter().
  1. … 1 more file in changeset.