Checkout Tools
  • last updated 5 hours ago
Constraints: committers
Constraints: files
Constraints: dates
Add missing pool tags to help debugging.
  1. … 40 more files in changeset.
malloc -> ap_malloc

bz #63967

convert malloc(3) into ap_malloc

bz 64049

  1. … 4 more files in changeset.
Fix spelling errors found by codespell. [skip ci]

  1. … 99 more files in changeset.
In winnt_accept() (server/mpm/winnt/child.c), there's a call to PostQueuedCompletionStatus() with a buffer (BytesRead) for the number of bytes read.

When compiling the code with Visual Studio 2019 (v 16.3.10), we get a warning than BytesRead is not initialized. When using /RTCu (run-time checks for uninitialzed variables), the program crashes on this line with an exception.

If we initialize the variable to 0, the problem is solved.

PR 63965

MPMs: early initialize scoreboard's child generation number.

Since [mpm]_note_child_killed uses the scoreboard's generation number for

child_status hook (MPM_CHILD_EXITED), we must initialize it early (i.e. in

[mpm]_note_child_started where MPM_CHILD_STARTED is set) to avoid race

conditions on restart (e.g. storm/loop of restarts) leading to AH00546.

PR 62658.

  1. … 5 more files in changeset.
event, worker: runtime pool.

MPMs event and worker both need a dedicated pool to handle the creation of

the threads (listener, workers) and synchronization objects (queues, pollset,

mutexes...) in the start_threads() thread, with at least the lifetime of

the connections they handle, and thus survive pchild destruction (notably

in ONE_PROCCESS mode, but SIG_UNGRACEFUL is concerned too).

For instance, without this fix, the below backtrace can happen in ONE_PROCCESS

mode and a signal/^C is received (with active connections):

Thread 1 "httpd" received signal SIGSEGV, Segmentation fault.

(gdb) bt

#0 <BOOM>

#1 0x00007ffff7c7e016 in apr_file_write (thefile=0x0, ...)

^ NULL (cleared)

at file_io/unix/readwrite.c:230

#2 0x00007ffff7c7e4a7 in apr_file_putc (ch=1 '\001', thefile=0x0)

^ NULL (cleared)

at file_io/unix/readwrite.c:377

#3 0x00007ffff7c8da4a in apr_pollset_wakeup (pollset=0x55555568b870)

^ already destroyed by pchild

at poll/unix/pollset.c:224

#4 0x00007ffff7fc16c7 in decrement_connection_count (cs_=0x7fff08000ea0)

at event.c:811

#5 0x00007ffff7c83e15 in run_cleanups (cref=0x7fffe4002b78)

at memory/unix/apr_pools.c:2672

#6 0x00007ffff7c82c2f in apr_pool_destroy (pool=0x7fffe4002b58)

^ master_conn

at memory/unix/apr_pools.c:1007

#7 0x00007ffff7c82c12 in apr_pool_destroy (pool=0x7fff08000c28)

^ ptrans

at memory/unix/apr_pools.c:1004

#8 0x00007ffff7c82c12 in apr_pool_destroy (pool=0x555555638698)

^ pconf

at memory/unix/apr_pools.c:1004

#9 0x00007ffff7c82c12 in apr_pool_destroy (pool=0x555555636688)

^ pglobal

at memory/unix/apr_pools.c:1004

#10 0x00005555555f4709 in ap_terminate ()

at unixd.c:522

#11 0x00007ffff6dbc8f1 in __run_exit_handlers (...)

at exit.c:108

#12 0x00007ffff6dbc9ea in __GI_exit (status=<optimized out>)

at exit.c:139

#13 0x00007ffff7fc1616 in clean_child_exit (code=0)

at event.c:774

^ pchild already destroyed here

#14 0x00007ffff7fc5ae4 in child_main (child_num_arg=0, child_bucket=0)

at event.c:2869


While at it, add comments about the lifetimes of MPMs pools and their objects,

and give each pool a tag (e.g. "pchild" accordingly to other MPMs).

(follow up for event_pollset in r1835846).

  1. … 3 more files in changeset.
mpm_winnt: Do not redefine the standard CONTAINING_RECORD() macro

in child.c.

This definition has been added in — perhaps,

because not every versions of SDK contained it at that time.

But since then, the macro has been available starting from Windows 2000


and any available version of Windows SDK now should also contain it.

mpm_winnt: Remove an obsolete comment in child.c explaining why the

declarations of the structures and functions to access the completion

contexts reside in a header file.

This no longer holds, as all the necessary functions and structures are

located in the single .c file (child.c).

mpm_winnt: Tweak the names of the variables in child.c which are used to

represent a queue of the completion contexts.

Starting from r1801655, the "queue" isn't really a queue, as all the

access happens with a LIFO order. So, instead of that, call it a "pool

of completion contexts", adjust names of all relevant variables and

tweak the comments.

This patch changes

- qlock to ctxpool_lock,

- qhead to ctxpool_head, and

- qwait_event to ctxpool_wait_event.

mpm_winnt: Tweak the listener shutdown code to use a separate event

instead of the global variable (shutdown_in_progress).

This change has two purposes. First of all, it makes the listener threads

which are blocked waiting for a completion context exit immediately during

shutdown. Previously, such threads would only check for exit every second.

The second reason for this change is to put the child_main() function in

charge of controlling the listeners life cycle. Previously, such relation

was circumvented by the fact that the listeners were also waiting for the

global child exit_event. With the new separate listener_shutdown_event,

only the child_main() function is responsible for shutting down the

listeners, and I think that this makes the code a bit clearer.

All the original behavior, including the special APLOG_DEBUG diagnostic

message when we fail to acquire a free completion context in 1 second,

is kept unchanged.

  1. … 1 more file in changeset.
mpm_winnt: Following up on r1801655, add a comment that explains the

reason to choose the LIFO processing order for completion contexts.

It would be better to keep this important information in the code, instead

of just having it in the log message.

mpm_winnt: Advertise support for preshutdown notifications in the service,

and perform shutdown in respond to SERVICE_CONTROL_PRESHUTDOWN.

The pure shutdown notification leaves a small amount of time for the service

to finish (and the allowed amount of time has been shrinking with every new

version of Windows), and handling only it increases the chance of the process

being killed by SCM, instead of gracefully shutting down. Handling the

preshutdown control code extends this period, and increases the chances of

finishing everything properly when the machine is rebooted or shut down.


Please note that although the preshutdown notifications are available only

starting from Windows Vista, the code is compatible with the previous versions

of Windows, since the SCM ignores unknown SERVICE_ACCEPT codes, and will

still send an ordinary SERVICE_CONTROL_SHUTDOWN under old Windows


mpm_winnt: Remove unused values of the io_state_e enum.

Submitted By: Ivan Zhakov <ivan {at}>

mpm_winnt: Remove a duplicated comment in the child_main() function.

mpm_winnt: Use a LIFO stack instead of a FIFO queue to hold unused

completion contexts, as that may significantly reduce the memory usage.

This simple change can have a noticeable impact on the amount of memory

consumed by the child process in various cases. Every completion context

in the queue has an associated allocator, and every allocator has it's

ap_max_mem_free memory limit which is not given back to the operating

system. Once the queue grows, it cannot shrink back, and every allocator

in each of the queued completion contexts keeps up to its max_free amount

of memory. The queue can only grow when a server has to serve multiple

concurrent connections at once.

With that in mind, consider a case with a server that doesn't encounter many

concurrent connections most of the time, but has occasional spikes when

it has to serve multiple concurrent connections. During such spikes, the

size of the completion context queue grows.

The actual difference between using LIFO and FIFO orders shows up after

such spikes, when the server is back to light load and doesn't see a lot

of concurrency. With FIFO order, every completion context in the queue

will be used in a round-robin manner, thus using *every* available allocator

one by one and ultimately claiming up to (N * ap_max_mem_free memory) from

the OS. With LIFO order, only the completion contexts that are close to

the top of the stack will be used and reused for subsequent connections.

Hence, only a small part of the allocators will be used, and this can

prevent all other allocators from unnecessarily acquiring memory from

the OS (and keeping it), and this reduces the overall memory footprint.

Please note that this change doesn't affect the worst case behavior, as

it's still (N * ap_max_mem_free memory), but tends to behave better in

practice, for the reasons described above.

Another thing worth considering is the new behavior when the OS decides

to swap out pages of the child process, for example, in a close-to-OOM

condition. Handling every new connection after the swap requires the OS

to load the memory pages for the allocator from the completion context that

is used for this connection. With FIFO order, the completion contexts are

used one by one, and this would cause page loads for every new connection.

With LIFO order, there will be almost no swapping, since the same completion

context is going to be reused for subsequent new connections.

mpm_winnt: Drop the APLOG_DEBUG diagnostic saying how many thread

are blocked on the I/O completion port during the shutdown.

Prior to r1801635, the shutdown code required to know the amount of blocked

threads, as it has been dispatching the same amount of completion packets.

But this no longer holds, and the only reason why we maintain the

corresponding g_blocked_threads variable is because of this debug

diagnostic message.

Drop it in order to reduce complexity of the quite critical code in the

winnt_get_connection() function and to reduce the amount of global


mpm_winnt: Remove an unnecessary Sleep() in the winnt_accept() function.

This sleep occured in a situation when:

- We don't have a free completion context in the queue

- We can't add one, as doing so would exceed the max_num_completion_contexts

limit (all worker threads are busy)

- We have exceeded a 1 second timeout while waiting for it

In this case, the Sleep() call is unnecessary, as there is no intermittent

failure that can be waited out, but rather than that, it's an ordinary

situation with all workers being busy. Presumably, calling Sleep() here

can be even considered harmful, as it affects the fairness between the

listeners that are blocked waiting for the completion context.

So, instead of calling Sleep() just check for the possible shutdown and

immediately retry acquiring a completion context. If all worker threads

are still busy, the retry will block in the same WaitForSingleObject() call,

which is fine.

mpm_winnt: Simplify the shutdown code that was waiting for multiple worker

thread handles in batches.

Starting from r1801636, there is no difference between ending the wait with

one or multiple remaining threads. This is because we terminate the process

if at least one thread is still active when we hit a timeout.

Therefore, instead of making an effort to evenly distribute and batch the

handles with WaitForMultipleObjects(), we could just start from one end,

and wait for one thread handle at a time.

mpm_winnt: Avoid using TerminateThread() in case the shutdown routine

hits a timeout while waiting for the worker threads to exit.

Using TerminateThread() can have dangerous consequences such as deadlocks —

say, if the the thread is terminated while holding a lock or a heap lock

in the middle of HeapAlloc(), as these locks would not be released.

Or it can corrupt the application state and cause a crash.


Rework the code to call TerminateProcess() in the described circumstances

and leave the cleanup to the operating system.

mpm_winnt: Make the shutdown faster by avoiding unnecessary Sleep()'s

when shutting down the worker threads.

Previously, the shutdown code was posting an amount of I/O completion

packets equal to the amount of the threads blocked on the I/O completion

port. Then it would Sleep() until all these threads "acknowledge" the

completion packets by decrementing the global amount of blocked threads.

A better way would be to send the number of IOCP_SHUTDOWN completion

packets equal to the total amount of threads and immediately proceed to

the next step. There is no need to block until the threads actually receive

the completion, as the shutdown process includes a separate step that waits

until the threads exit, and the new approach avoids an unnecessary delay.

mpm_winnt: Following up on r1801144, use the new accept_filter_e enum

values in a couple of missed places in winnt_accept().

mpm_winnt: Fix typo in the logged message in winnt_get_connection().

mpm_winnt: Refactor the mpm_get_completion_context() function so that it

would return a proper apr_status_t instead of yielding the result via the

*timeout out variable.

This makes the calling side easier to follow by avoiding an additional

layer of if's.

mpm_winnt: Remove an unnecessary retry after receiving a non-timeout failure

from the mpm_get_completion_context() function.

Currently, the only possible reasons why mpm_get_completion_context() could

fail are real errors such as being unable to WaitForSingleObject(), allocate

memory or create an event. Retrying under such circumstances doesn't make

sense, and could be as well considered harmful.

mpm_winnt: Factor out a helper function to parse the type of an accept

filter and use an appropriate enum for it.

This makes the code in winnt_accept() a bit easier to follow. As a minor

side effect, it also fixes a small bug where the "unrecognized AcceptFilter

'%s'" log entry would always contain "none" instead of the actually

unrecognized kind of the accept filter.

mpm_winnt: Don't forget to close the I/O completion port as part of the

cleanup in the child process.

If the lingering close does not leave the socket in a disconnected state,

do not recycle the socket.

On the trunk:

mpm_winnt: always invoke ap_lingering_close() at connection end.

winnt/service: each log message should use its own APLOGNO.
  1. … 1 more file in changeset.