Checkout Tools
  • last updated 1 hour ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
mod_dav_svn: Set Last-Modified response header for 'external' GET requests

(i.e. requests to URI's not under /!svn). This partially reverts r1724790

which removed it for all GET requests, for performance reasons (however, for

external requests the Last-Modified header is needed for certain use cases).

See this thread on dev@:

https://lists.apache.org/thread.html/51f4f1fb5363c1d1c393f6ab769b12b9c7914af583fc333c652306d0@%3Cdev.subversion.apache.org%3E

Date: 2019/09/02 14:20:16 GMT

From: Johan Corveleyn

To: Subversion dev list

Subject: Re: Last-Modified HTTP header in GET responses

Message-ID: <CAB84uBX4dNCsbaOn1Dzy0N3CB4rfC0_ZVCZ_TpVikt+p1zMGNg@mail.gmail.com>

Review by: brane

* subversion/mod_dav_svn/repos.c

(get_last_modified): New function (reverted its removal in r1724790).

(set_headers): Set Last-Modified response header for 'external' requests.

* subversion/tests/cmdline/mod_dav_svn_tests.py

(last_modified_header): New test.

  1. … 1 more file in changeset.
* subversion/mod_dav_svn/repos.c

(get_resource): Following up on r1850651: Set cleanup handler for

FS warning logging regardless of presence of R->USER.

Patch by: sergey.raevskiy{_AT_}visualsvn.com

Fix a use-after-free in mod_dav_svn's logging of FS warnings.

The FS warning callback could be called with a request context that had

already been deallocated. This resulted in a crash during 'make check'

with ra_serf on OpenBSD. The problem was even documented in a comment:

/* ### hmm. the FS is cleaned up at request cleanup time. "r" might

### not really be valid. we should probably put the FS into a

### subpool to ensure it gets cleaned before the request.

### is there a good way to create and use a subpool for all

### of our functions ... ??

*/

Rather than putting the FS into a subpool, the solution implemented with this

commit installs a pre-cleanup handler on the request pool, which switches the

logging context from the request to its associated connection. This avoids the

use-after-free at the cost of a less precise logging context.

Suggested by: stefan2

https://svn.haxx.se/dev/archive-2018-12/0145.shtml

* subversion/mod_dav_svn/repos.c

(log_warning): Rename to ...

(log_warning_req): ... this.

(log_warning_conn): New logging helper which uses a connection context.

(cleanup_req_logging_baton, cleanup_req_logging): New APR pool cleanup

handler which switches FS logging context from a request to a connection.

(get_resource): Install aforementioned pool cleanup handler.

* subversion/mod_dav_svn/repos.c

(open_stream): Return an error if a non-conforming client sends

a PUT before the corresponding CHECKOUT when attempting a v1

protocol commit. This is required for strict v1 compliance

and does not affect the v2 protocol.

Following up on r1803899, properly negotiate the svndiff version when a

client indicates the preference of svndiff2 format, but the compression

is disabled on the server.

In this case, the server configuration should override the client's

preference (and this is how Subversion 1.9 worked), so tweak the

negotiation scheme so that the server would always be use uncompressed

svndiff0 format with "SVNCompressionLevel 0".

* subversion/mod_dav_svn/repos.c

(negotiate_encoding_prefs): Add special handling for compression level 0.

Add 'http-compression=auto' mode on the client, now used by default.

Following up on the recently added support for LZ4 compression, this patch

introduces the new possible 'auto' mode for our client 'http-compression'

configuration option, and starts using it by default.

The previously used default was 'http-compression=yes', and the new 'auto'

mode is a small, but an important tweak to its behavior. In the new mode,

the HTTP compression is still being used, but the negotiating is tweaked

to favor svndiff2 with LZ4 compression when working over local networks.

The reasoning behind this is that for local networks it probably makes sense

to favor compression speed of LZ4 instead of the better compression ratio

from zlib-5, which has been used previously.

To separate local and wide area networks, we use the connection latency of

the initial OPTIONS request obtained with serf_connection_get_latency().

* subversion/libsvn_subr/config_file.c

(svn_config_ensure): Document the new default and possible values of

the http-compression option.

* subversion/libsvn_ra_serf/ra_serf.h

(svn_ra_serf__session_t.using_compression): Turn into a tristate field.

(svn_ra_serf__session_t.conn_latency): New field.

(svn_ra_serf__is_local_network): Declare new helper function.

(svn_ra_serf__setup_svndiff_accept_encoding): Accept a session instead

of the 'using_compression' boolean argument.

* subversion/libsvn_ra_serf/util.c

(svn_ra_serf__is_local_network): Implement this new function.

(svn_ra_serf__setup_svndiff_accept_encoding): Tweak the Accept-Encoding

to indicate that we'd like to see svndiff2 with http-compression=auto

and when working over a local network. Otherwise, when using compression,

indicate both svndiff2 and svndiff1 support, but tell the server that we

favor svndiff1.

(setup_request): Properly handle the 'using_compression' field, which

is now a tristate.

* subversion/libsvn_ra_serf/serf.c

(load_config): Get the compression setting as a tristate.

(svn_ra_serf__open): Initialize the connection latency to default

value before we make the initial OPTIONS request.

(ra_serf_dup_session): Add placeholder comment for the new field.

* subversion/libsvn_ra_serf/options.c

(options_response_handler): Remember the connection latency in the

session.

* subversion/libsvn_ra_serf/commit.c

(negotiate_put_encoding): Rework to support new http-compression=auto

mode. Update or rewrite the related comments.

* subversion/libsvn_ra_serf/blame.c

(blame_context_t): Store session instead of the 'using_compression' field.

(setup_headers): Update call to svn_ra_serf__setup_svndiff_accept_encoding().

(svn_ra_serf__get_file_revs): Remember the used session in context.

* subversion/libsvn_ra_serf/replay.c

(revision_report_t): Store session instead of the 'using_compression' field.

(setup_headers): Update call to svn_ra_serf__setup_svndiff_accept_encoding().

(svn_ra_serf__replay_range): Remember the used session in context.

* subversion/libsvn_ra_serf/update.c

(fetch_ctx_t): Store session instead of the 'using_compression' field.

(headers_fetch): Update call to svn_ra_serf__setup_svndiff_accept_encoding().

Properly handle the 'using_compression' field, which is now a tristate.

(fetch_for_file): Remember the used session in the fetch context.

(setup_update_report_headers): Update call to

svn_ra_serf__setup_svndiff_accept_encoding().

* subversion/mod_dav_svn/repos.c

(get_svndiff_version): New helper function.

(negotiate_encoding_prefs): Tweak the negotiation logic. Only override

the client's preference if it supports svndiff2 and SVNCompressionLevel

is set to 1. In all other cases, select the svndiff format that the

client prefers to see (the one with the largest ;q= value in the Accept-

Encoding header).

  1. … 10 more files in changeset.
* subversion/mod_dav_svn/repos.c

(negotiate_encoding_prefs): Fix typo in the comment.

Lay the groundwork that would allow ra_serf to stream svndiff deltas

without creating temporary files when working against new servers.

This patch adds a new delta editor callback, apply_textdelta_stream()

that we'll use to stream the deltas, fully implements in the ra_serf's

commit editor, but doesn't yet change the behavior of existing editor

drivers, such as svn_client_import5() or svn_client_commit6().

This requires a minor tweak to the Subversion's HTTP protocol, and

it's the reason why streaming would only work against new servers.

Currently, all PUT requests include a special header that contains the

result checksum, which is used by the server to validate the integrity

of the result after it applies the delta received over the wire. While

this approach works fine if the client first creates a temporary file with

the delta and only then starts sending it to the server (the result checksum

is calculated while preparing the temporary file), it can't be used in the

stream approach, as with it we'd need to know the result checksum _before_

we start sending data.

So we turn the existing scheme inside out, and teach mod_dav_svn to send the

result checksum in the responses to PUT requests. Then, the client would

check if the received checksum matches what it calculated, and, possibly,

return a checksum mismatch error (thus aborting the edit and the transaction).

* subversion/include/svn_delta.h

(svn_txdelta_stream_open_func_t): New callback type.

(svn_delta_editor_t): Add a forward declaration for this type.

(svn_delta_editor_t.apply_textdelta_stream): New vtable member.

* subversion/libsvn_delta/default_editor.c

(apply_textdelta_stream): Provide default implementation for this callback

that performs a fallback to apply_textdelta().

(default_editor): Extend the default instance of an svn_delta_editor_t.

* subversion/libsvn_delta/cancel.c

(apply_textdelta_stream): Implement this forwarding callback, and ...

(svn_delta_get_cancellation_editor): ...install it here.

* subversion/include/svn_dav.h

(SVN_DAV_NS_DAV_SVN_PUT_RESULT_CHECKSUM): New.

* subversion/mod_dav_svn/repos.c

(close_stream): Send the "X-SVN-Result-Fulltext-MD5" header when responding

to successful PUT requests.

* subversion/mod_dav_svn/version.c

(get_vsn_options): Advertise new capability.

* subversion/libsvn_ra_serf/ra_serf.h

(svn_ra_serf__session_t.supports_put_result_checksum): New field.

* subversion/libsvn_ra_serf/options.c

(capabilities_headers_iterator_callback): Remember if the server sends

the result checksum in the response to a successful PUT request.

* subversion/libsvn_ra_serf/commit.c

(file_context_t.svndiff_sent): New field.

(file_context_t.remote_result_checksum): New field.

(apply_textdelta): Update the comment stating that it would be nice to

get rid of temporary files for svndiff delta. Factor out the svndiff

format selection logic ...

(negotiate_put_encoding): ...into this new helper function.

(open_txdelta_baton_t): New.

(txdelta_stream_errfunc): New error function for the stream bucket.

(create_body_from_txdelta_stream): New svn_ra_serf__request_body_delegate_t

that creates the request body by opening an svn_txdelta_stream_t, turning

it into svn_stream_t and giving away a bucket wrapping around that stream.

Use it in ...

(apply_textdelta_stream): ...this new function that performs a PUT, and

streams the request body.

(put_response_ctx_t): New.

(put_response_handler): New, remembers the result checksum we received

from the server.

(close_file): Don't do a PUT if we did it in apply_textdelta_stream().

Check for a checksum mismatch using the checksum returned from the server.

(svn_ra_serf__get_commit_editor): Install the new apply_textdelta_stream()

callback when working against new servers.

  1. … 8 more files in changeset.
Negotiate the use of the svndiff2 format over http://.

On the server-side, start sending svndiff2 deltas to clients that can read

them, but only when SVNCompressionLevel is set to 1. On the client-side,

keep sending svndiff1. Currently we use svndiff1 with default compression

ratio (5), and svndiff2 is not a substitute for it, as that would result

in worse compression.

* subversion/include/svn_dav.h

(SVN_DAV_NS_DAV_SVN_SVNDIFF2): New.

* subversion/libsvn_ra_serf/ra_serf.h

(svn_ra_serf__session_t): Add 'supports_svndiff2' field.

* subversion/libsvn_ra_serf/options.c

(capabilities_headers_iterator_callback): Remember if the server supports

svndiff2.

* subversion/libsvn_ra_serf/commit.c

(apply_textdelta): Add the fallback code for a theoretically possible

situation where the server has advertised _only_ svndiff2 support.

Note why we still use svndiff1 when client-side http compression is

enabled. Adjust a couple of comments.

* subversion/libsvn_ra_serf/util.c

(svn_ra_serf__setup_svndiff_accept_encoding): Advertise support for both

svndiff2 and svndiff1 with identical (q=) values. We'll let the server

decide between svndiff2 and svndiff1.

* subversion/mod_dav_svn/repos.c

(negotiate_encoding_prefs): Select the svndiff2 format if the client

supports it and if the server's SVNCompressionLevel is set to 1.

* subversion/mod_dav_svn/version.c

(get_vsn_options): Advertise svndiff2 support.

  1. … 6 more files in changeset.
* subversion/mod_dav_svn/repos.c

(deliver): Rename 'entry_pool' local variable to 'iterpool'.

Fix the unbounded memory usage when mod_dav_svn is paired up with

mod_headers or mod_deflate.

The problem is caused by how mod_dav passes the output filter list to its

providers. A hook receives the current head of the output filter list for

a particular request. Certain filters, such as the one in mod_headers, are

designed to perform the work only once. When the work is done, a filter

removes itself from the list. If a filter is the first in the list, this

updates the head of the linked list in request_rec (r->output_filters), but

not the ap_filter_t * pointer that was passed as an argument to mod_dav_svn.

So, mod_dav_svn continues to operate with an outdated list of filters, and

every write causes a re-execution of the already removed filter.

In case with mod_headers, this triggers an unbounded memory usage, e.g.,

responding to a GET request for a 500 MB file results in the server

consuming a large amount (~3.2 GB) of memory. Full discussion of the

mod_dav's part of the issue can be found in [1].

The idea behind the fix is that we add a new opaque type, dav_svn__output,

a small corresponding set of private API functions, and adjust the existing

output helpers so that every write would target a dav_svn__output object.

A dav_svn__output is constructed for a particular request, and internally

it ensures that the writes *always* target the actual output filter list

in r->output_filters.

[1] https://mail-archives.apache.org/mod_mbox/httpd-dev/201608.mbox/%3C20160822151917.GA22369%40redhat.com%3E

* subversion/mod_dav_svn/dav_svn.h

(dav_svn__output): New.

(dav_svn__output_create, dav_svn__output_get_bucket_alloc,

dav_svn__output_pass_brigade): New.

(dav_svn__merge_response, dav_svn__update_report,

dav_svn__log_report, dav_svn__dated_rev_report,

dav_svn__get_locations_report, dav_svn__get_location_segments_report,

dav_svn__file_revs_report, dav_svn__replay_report,

dav_svn__get_mergeinfo_report, dav_svn__get_locks_report,

dav_svn__get_deleted_rev_report, dav_svn__get_inherited_props_report,

dav_svn__post_create_txn, dav_svn__post_create_txn_with_props,

dav_svn__brigade_write, dav_svn__brigade_puts,

dav_svn__brigade_printf, dav_svn__brigade_putstrs,

dav_svn__make_base64_output_stream, dav_svn__final_flush_or_error):

Now work with a dav_svn__output, instead of an ap_filter_t.

* subversion/mod_dav_svn/util.c

(dav_svn__output): New.

(dav_svn__output_create, dav_svn__output_get_bucket_alloc,

dav_svn__output_pass_brigade): Implement new functions.

(dav_svn__brigade_write, dav_svn__brigade_puts,

dav_svn__brigade_printf, dav_svn__brigade_putstrs,

dav_svn__final_flush_or_error): Target the actual output filter list

for a particular request.

(brigade_write_baton): Store a dav_svn__output object.

(brigade_write_fn): Target the actual output filter list for a

particular request.

(dav_svn__make_base64_output_stream): Now works with a dav_svn__output,

instead of an ap_filter_t.

* subversion/mod_dav_svn/repos.c

(diff_ctx_t): Store a dav_svn__output object.

(write_to_filter): Call dav_svn__brigade_write() function.

(close_filter): Use the dav_svn__output's API. The apr_brigade_cleanup()

call is no longer required, since dav_svn__output_pass_brigade() also

cleans the bucket brigade.

(emit_collection_head, emit_collection_entry, emit_collection_tail):

Now work with a dav_svn__output, instead of an ap_filter_t.

(deliver): Create a dav_svn__output at the beginning of this function.

Use the dav_svn__output's API for brigade creation and passing to the

output filter stack.

(handle_post_request): Now works with a dav_svn__output, instead of

an ap_filter_t.

(dav_svn__method_post): Create a dav_svn__output and forward it to

handle_post_request().

* subversion/mod_dav_svn/version.c

(deliver_report): Create a dav_svn__output and forward it to the

report handlers.

(merge): Create a dav_svn__output and forward it to

dav_svn__merge_response().

* subversion/mod_dav_svn/merge.c

(send_response, do_resources, dav_svn__merge_response): Now work with

a dav_svn__output and use the new private API.

* subversion/mod_dav_svn/posts/create_txn.c

(dav_svn__post_create_txn, dav_svn__post_create_txn_with_props): Now

work with a dav_svn__output.

* subversion/mod_dav_svn/reports/dated-rev.c

(dav_svn__dated_rev_report): Work with a dav_svn__output. Use the new

private API when creating a bucket brigade.

* subversion/mod_dav_svn/reports/deleted-rev.c

(dav_svn__get_deleted_rev_report): Work with a dav_svn__output. Use

the new private API when creating a bucket brigade.

* subversion/mod_dav_svn/reports/file-revs.c

(file_rev_baton): Store a dav_svn__output object.

(dav_svn__file_revs_report): Work with a dav_svn__output. Use the new

private API when creating a bucket brigade.

* subversion/mod_dav_svn/reports/get-location-segments.c

(location_segment_baton): Store a dav_svn__output object.

(dav_svn__get_location_segments_report): Work with a dav_svn__output.

Use the new private API when creating a bucket brigade.

* subversion/mod_dav_svn/reports/get-locations.c

(send_get_locations_report, dav_svn__get_locations_report): Work with a

dav_svn__output. Use the new private API when creating a bucket brigade.

* subversion/mod_dav_svn/reports/get-locks.c

(send_get_lock_response, dav_svn__get_locks_report): Work with a

dav_svn__output. Use the new private API when creating a bucket brigade.

* subversion/mod_dav_svn/reports/inherited-props.c

(dav_svn__get_inherited_props_report): Work with a dav_svn__output.

Use the new private API when creating a bucket brigade.

* subversion/mod_dav_svn/reports/log.c

(log_receiver_baton): Store a dav_svn__output object.

(log_revision_receiver): Rewrite the forced flush by creating and sending

a flush bucket to the output filters (that's what ap_fflush() does

internally). Drop the aborted connection check, since it's now a

part of dav_svn__output_pass_brigade().

(dav_svn__log_report): Work with a dav_svn__output. Use the new private

API when creating a bucket brigade.

* subversion/mod_dav_svn/reports/mergeinfo.c

(dav_svn__get_mergeinfo_report): Work with a dav_svn__output. Use the

new private API when creating a bucket brigade.

* subversion/mod_dav_svn/reports/replay.c

(edit_baton_t): Store a dav_svn__output object.

(make_editor, dav_svn__replay_report): Work with a dav_svn__output.

Use the new private API when creating a bucket brigade.

* subversion/mod_dav_svn/reports/update.c

(update_ctx_t): Store a dav_svn__output object.

(dav_svn__update_report): Work with a dav_svn__output. Use the new

private API when creating a bucket brigade.

  1. … 16 more files in changeset.
In mod_dav_svn, switch every remaining usage of ap_fputs(), ap_fprintf()

and ap_fputstrs() to dav_svn__brigade functions.

Doing so ensures proper error handling in case of dead sockets (this

is issue 1709, see https://issues.apache.org/jira/browse/SVN-1709).

And with this change, every output that happens in mod_dav_svn now either

uses the dav_svn__brigade functions or manually prepares a bucket and

shoves it to the output filter stack by calling ap_pass_brigade().

This is an important prerequisite to fix the unbounded memory usage in

a case when mod_dav_svn is paired up with mod_headers or mod_deflate.

* subversion/mod_dav_svn/dav_svn.h

(dav_svn__brigade_putstrs): New wrapper for apr_brigade_vputstrs().

* subversion/mod_dav_svn/util.c

(dav_svn__brigade_putstrs): Implement new wrapper.

* subversion/mod_dav_svn/repos.c

(emit_collection_head, emit_collection_entry, emit_collection_tail):

Use dav_svn__brigade functions. Return possible errors, and ...

(deliver): ...handle them here.

* subversion/mod_dav_svn/merge.c

(send_response, dav_svn__merge_response): Use dav_svn__brigade functions.

* subversion/mod_dav_svn/reports/dated-rev.c

(dav_svn__dated_rev_report): Use dav_svn__brigade functions.

* subversion/mod_dav_svn/reports/deleted-rev.c

(dav_svn__get_deleted_rev_report): Use dav_svn__brigade functions.

* subversion/mod_dav_svn/reports/get-location-segments.c

(location_segment_receiver): Use dav_svn__brigade functions.

* subversion/mod_dav_svn/reports/get-locations.c

(send_get_locations_report): Use dav_svn__brigade functions. Return

an svn_error_t instead of an apr_status_t, and handle it ...

(dav_svn__get_locations_report): ...here.

* subversion/mod_dav_svn/reports/get-locks.c

(SVN_APR_ERR): Remove this macro.

(send_get_lock_response): Use dav_svn__brigade functions. Return

an svn_error_t of an apr_status_t, and handle it ...

(dav_svn__get_locks_report): ...here.

  1. … 8 more files in changeset.
In mod_dav_svn, factor out three functions responsible for sending

the header, trailer and a single entry of the collection.

This lays the groundwork required for switching to dav_svn__brigade

functions whenever we write something to the output filter stack.

And this switch would be required to fix the unbounded memory usage

when mod_dav_svn is paired up with mod_headers or mod_deflate.

* subversion/mod_dav_svn/repos.c

(deliver): Factor out parts responsible for sending the header, trailer

and the entry of the collection into ...

(emit_collection_head, emit_collection_tail, emit_collection_entry):

...these new functions.

* subversion/mod_dav_svn/repos.c

(close_filter): Cleanup the brigade after the ap_pass_brigade() call.

Although it's not strictly required in this stream's close_fn, because

we always call apr_brigade_destroy() upon returning from the deliver()

hook, generally, this is the right thing to do.

In mod_dav_svn, update the code responsible for answering to GET requests

with a specified delta base so that it would use apr_brigade_write() instead

of manually creating a bucket and passing it to the output filter stack.

The apr_brigade_write() buffers the data internally before sending it to the

filters. Buffering is important in this particular case, because our svndiff

handler can emit a noticeable amount of small writes. And while some of the

httpd's filters have their own buffering, some of them don't. For instance,

the chunking filter (which is pretty close to the top of the stack) doesn't

do that — so, this resulted in inefficient small chunks of size 4, 5, etc.,

being sent over the wire.

* subversion/mod_dav_svn/repos.c

(write_to_filter): Use apr_brigade_write() instead of sending a single

transient bucket to the output filter stack.

In mod_dav_svn, rewrite the code responsible for answering to GET requests

with a specified delta base, so that it would reuse a single bucket brigade.

This gets rid of the unbounded memory usage caused by creating a new bucket

brigade per *every* call of the svn_stream_write(). The bucket brigades

were allocated in the request pool.

* subversion/mod_dav_svn/repos.c

(diff_ctx_t): Remove `pool`, add a new bucket brigade (`bb`) field.

(write_to_filter, close_filter): Reuse the single bucket brigade from the

stream's context.

(deliver): Create a bucket brigade, pass it to the output stream for

svndiff, and destroy it after we've sent the data.

In mod_dav_svn, reuse a single bucket brigade when responding to GET

requests without an "X-SVN-VR-Base" header, i.e., without a delta base.

Previously we were creating a new brigade per every chunk of data, and

that caused an excessive memory usage of around 7-8 MB while serving a

800 MB-sized response.

This was pointed out by Joe Orton <jorton@redhat.com> as a part of the

discussion in http://svn.haxx.se/dev/archive-2016-08/0028.shtml

* subversion/mod_dav_svn/repos.c

(deliver): Rewrite the code to reuse a single bucket brigade within the

loop, and properly call apr_brigade_destroy() when the brigade is no

longer required.

mod_dav_svn: Do not set Last-Modified response header for GET responses.

This saves a bit of unnecessary work on the server-side, since:

- This header is not used by Subversion clients

- We allow caching GET responses for up-to 1 week without re-validation

- Browsers and proxies support ETag and use it for re-validation instead of

the Last-Modified header

See the discussion in thread "Last-Modified HTTP header in GET responses" [1]

[1] https://www.mail-archive.com/dev@subversion.apache.org/msg34354.html

* subversion/mod_dav_svn/repos.c

(get_last_modified): Remove.

(set_headers): Do set Last-Modifed response header.

Add runtime configuration option to enable FSFS node properties caching

without enabling full-texts caching.

* subversion/include/svn_fs.h

(SVN_FS_CONFIG_FSFS_CACHE_NODEPROPS): New.

* subversion/libsvn_fs_fs/caching.c

(read_config): Parse SVN_FS_CONFIG_FSFS_CACHE_NODEPROPS FS config flag.

(svn_fs_fs__initialize_caches): Enable node properties caching if needed.

* subversion/mod_dav_svn/dav_svn.h

(dav_svn__get_nodeprop_cache_flag): New.

* subversion/mod_dav_svn/mod_dav_svn.c

(dir_conf_t): Add NODEPROP_CACHE member.

(merge_dir_config): Merge NODEPROP_CACHE setting.

(SVNCacheNodeProps_cmd): New.

(dav_svn__get_nodeprop_cache_flag): New.

(cmds): Add SVNCacheNodeProps directive.

* subversion/mod_dav_svn/repos.c

(get_resource): Pass node properties caching setting to svn_repos_open3().

* subversion/svnadmin/svnadmin.c

(open_repos): Enable node properties caching to match Subversion 1.9.0

behavior.

* subversion/svnserve/svnserve.c

(SVNSERVE_OPT_CACHE_NODEPROPS): New.

(svnserve__options): Add '--cache-nodeprops' command line parameter.

(sub_main): Handle '--cache-nodeprops' command line parameter.

* subversion/tests/libsvn_fs_fs/fs-fs-fuzzy-test.c

(fuzzing_1_byte_1_rev): Enable node properties caching to match

Subversion 1.9.0 behavior.

  1. … 7 more files in changeset.
Cache youngest revision during processing one request on mod_dav_svn.

* subversion/mod_dav_svn/dav_svn.h

(dav_svn_repos): Add YOUNGEST_REV member.

(dav_svn__create_txn): Drop const qualifier from REPOS argument.

(dav_svn__get_youngest_rev): New function declaration.

* subversion/mod_dav_svn/util.c

(dav_svn__get_youngest_rev): New function.

* subversion/mod_dav_svn/activity.c

* subversion/mod_dav_svn/liveprops.c

* subversion/mod_dav_svn/lock.c

* subversion/mod_dav_svn/reports/get-location-segments.c

* subversion/mod_dav_svn/reports/update.c

* subversion/mod_dav_svn/repos.c

* subversion/mod_dav_svn/version.c

(dav_svn__create_txn, insert_prop_internal, append_locks,

dav_svn__get_location_segments_report, dav_svn__update_report,

prep_regular, prep_version, parse_querystring,

dav_svn__working_to_regular_resource, get_option, dav_svn__checkout):

Use dav_svn__get_youngest_rev() instead of direct call to

svn_fs_youngest_rev().

(get_parentpath_resource, get_resource): Initialize RESPO->YOUNGEST_REV to

SVN_INVALID_REVNUM.

  1. … 8 more files in changeset.
* subversion/mod_dav_svn/repos.c

(get_last_modified): Use SVN_PROP_REVISION_DATE constant instead of

hard-coded "svn:date".

Fixup the recent solution for issue SVN-4514 by discouraging caching for

requests that include ?r=WORKINGREV, but don't contain ?p=PEGREV in the

url, e.g.:

https://svn.apache.org/repos/asf/subversion/trunk/README?r=1716593

Results for these requests aren't guaranteed to be stable, since mod_dav_svn

can either immediately return the resource at the revision or trace back the

history to the canonical ?p=PEGREV location. The result depends on current

state of the repository, and cannot be cached, so make sure we set the

"Cache-Control: max-age=0" header when responding to such requests.

Other possible ?p= and ?r= requests were producing appropriate Cache-Control

headers even before this changeset, and that behavior is left intact:

https://svn.apache.org/repos/asf/subversion/trunk/README

https://svn.apache.org/repos/asf/subversion/trunk/README?p=1716593

https://svn.apache.org/repos/asf/subversion/trunk/README?p=1716593&r=1716593

Add a new test that specifies current behavior of how and when we set the

Cache-Control header, and as well covers the aforementioned ?r= case.

* subversion/mod_dav_svn/repos.c

(prep_regular): Do not mark the resource with a URL query string as

'idempotent', unless it specifies a peg revision.

* subversion/tests/cmdline/svntest/main.py

(create_http_connection): New utility function, factored out from ...

* subversion/tests/cmdline/lock_tests.py

(http_connection, create_dav_lock_timeout, dav_lock_refresh): ...here.

* subversion/tests/cmdline/mod_dav_svn_tests.py: New file.

  1. … 3 more files in changeset.
Resolve issue SVN-4514. Add 'Cache-Control: max-age=604800' for all requests

to resource that are 'idempotent', i.e. target is fully specified in URI and

cannot change. For other requests add 'Cache-Control: max-age=0' to disable

browser's heuristic caching.

* subversion/mod_dav_svn/dav_svn.h

(dav_resource_private): Add new member IDEMPOTENT.

* subversion/mod_dav_svn/repos.c

(parse_version_uri): Mark resource as 'idempotent'.

(prep_regular): Mark resource as 'idempotent' only if revision is specified

in request URI.

(is_cacheable): New.

(set_headers): Use is_cacheable() to determine Cache-Control header value.

Set Cache-Control for non-existing resources too.

  1. … 1 more file in changeset.
Switch the last FS revprop API users to the rev'ed functions.

This fixes the last deprecation warnings

* subversion/mod_dav_svn/merge.c

(dav_svn__merge_response): Call the latest API variant, always

requesting the latest data.

* subversion/mod_dav_svn/repos.c

(get_last_modified): Same.

  1. … 1 more file in changeset.
Use the typesafe svn_sort__array() helper instead of qsort() in a few places.

* subversion/libsvn_client/merge.c

(get_mergeinfo_paths,

combine_range_with_segments): Use svn_sort__array().

* subversion/libsvn_fs_base/tree.c

(includes): Add svn_sorts_private.h.

(verify_locks): Use svn_sort__array().

* subversion/mod_dav_svn/repos.c

(negotiate_encoding_prefs): Use svn_sort__array(). Remove const from local

variable.

* subversion/tests/libsvn_client/client-test.c

(includes): Add svn_sorts_private.h.

(test_remote_only_status): Use svn_sort__array().

  1. … 3 more files in changeset.
When accessing an APR hash with a non-string key, take the key size from

the key variable instead of its type. This makes the code more robust in

case we want to change the key type in the future.

* subversion/libsvn_client/ra.c

(repos_locations): Take the key size from the key variable.

* subversion/libsvn_repos/log.c

(do_logs): Same.

* subversion/mod_dav_svn/repos.c

(parse_querystring): Same.

* subversion/tests/libsvn_repos/repos-test.c

(file_rev_handler,

verify_locations,

set_expected): Same.

* tools/dev/fsfs-access-map.c

(open_file): Same.

  1. … 4 more files in changeset.
Add an apr_status_t parameter to dav_svn__new_error() and

dav_svn__new_error_svn(). This allows us to avoid dropping the

status when ap_fflush() and ap_pass_brigade() calls fail, and a

few other calls as well. Most calls currently pass zero.

dav_svn__new_error() was originally modelled on dav_new_error() in

httpd 2.0 which did not support passing an apr_status_t but in 2.4

an apr_status_t parameter was added.

* subversion/mod_dav_svn/dav_svn.h

(dav_svn__new_error_svn, dav_svn__new_error): Add APR status parameter.

* subversion/mod_dav_svn/util.c

(dav_svn__new_error, dav_svn__new_error_svn): Add APR status parameter.

(dav_svn__final_flush_or_error): Pass status.

(build_error_chain, dav_svn__test_canonical): Pass zero.

* subversion/mod_dav_svn/repos.c

(deliver): Pass status or zero.

(prep_working, prep_activity, prep_private, prep_resource,

dav_svn_split_uri2, get_parentpath_resource, parse_querystring,

get_resource, get_parent_resource, open_stream, seek_stream,

create_collection, copy_resource, remove_resource, move_resource,

do_walk, dav_svn__create_version_resource, handle_post_request,

dav_svn__method_post): Pass zero.

* subversion/mod_dav_svn/lock.c

(unescape_xml): Pass status.

(dav_lock_to_svn_lock, get_locks, find_lock, has_locks,

append_locks, remove_lock, refresh_locks): Pass zero.

* subversion/mod_dav_svn/merge.c

(dav_svn__merge_response): Pass status or zero.

* subversion/mod_dav_svn/activity.c

(dav_svn__delete_activity): Pass zero.

* subversion/mod_dav_svn/deadprops.c

(save_value, db_open, decode_property_value, db_store): Pass zero.

* subversion/mod_dav_svn/reports/dated-rev.c

(dav_svn__dated_rev_report): Pass zero.

* subversion/mod_dav_svn/reports/deleted-rev.c

(dav_svn__get_deleted_rev_report): Pass zero.

* subversion/mod_dav_svn/reports/file-revs.c

(dav_svn__file_revs_report): Pass zero.

* subversion/mod_dav_svn/reports/get-location-segments.c

(dav_svn__get_location_segments_report): Pass zero.

* subversion/mod_dav_svn/reports/get-locations.c

(dav_svn__get_locations_report): Pass zero.

* subversion/mod_dav_svn/reports/get-locks.c

(dav_svn__get_locks_report): Pass zero.

* subversion/mod_dav_svn/reports/inherited-props.c

(dav_svn__get_inherited_props_report): Pass zero.

* subversion/mod_dav_svn/reports/log.c

(dav_svn__log_report): Pass zero.

* subversion/mod_dav_svn/reports/mergeinfo.c

(dav_svn__get_mergeinfo_report): Pass zero.

* subversion/mod_dav_svn/reports/replay.c

(make_editor, dav_svn__replay_report): Pass zero.

* subversion/mod_dav_svn/reports/update.c

(malformed_element_error, dav_svn__update_report): Pass zero.

* subversion/mod_dav_svn/version.c

(set_auto_revprops, vsn_control, dav_svn__checkout, uncheckout,

dav_svn__checkin, deliver_report, make_activity, merge): Pass zero.

  1. … 18 more files in changeset.
Put jcorvel's workaround for issue #4531 into mod_dav_svn's implementation.

* subversion/mod_dav_svn/repos.c

(get_resource): Set a Depth header with value "0" on copy requests.

This prevents mod_dav's precondition check from recursing over the copy

source tree and thus restores O(1) behaviour for 'svn copy' with current

httpd releases.

The alternative of patching mod_dav was also considered but has been

dismissed. Ideally, there would be an API between mod_dav and its

providers to enable/disable the precondition tree walk.

Patch by: me

gstein

FS API change: Rename svn_fs_node_same to svn_fs_node_unchanged.

Note that we currently (ab-)use the same enum in the FS implementation for

ID relationships but this is a mere convenience. They don't have the same

semantics wrt. to edge cases and don't expose their use of the enum through

the FS API.

* subversion/include/svn_fs.h

(svn_fs_node_relation_t): Rename said element and document the semantics

of all elements in terms of (root, path) pairs

other FS API concepts. Be as strict as feasible.

Remove references to #svn_fs_compare_ids as it

does _not_ have the same strictness guarantees.

* subversion/libsvn_fs_base/id.c

(svn_fs_base__id_compare): Update enum element name.

* subversion/libsvn_fs_base/tree.c

(txn_body_copy): Same.

* subversion/libsvn_fs/editor.c

(can_modify): Same.

* subversion/libsvn_fs_fs/id.c

(svn_fs_fs__id_compare): Same.

* subversion/libsvn_fs/fs-loader.c

(svn_fs_compare_ids): Same.

* subversion/libsvn_fs_fs/tree.c

(fs_node_relation): Same.

* subversion/libsvn_fs_x/fs_id.c

(id_compare): Same.

* subversion/libsvn_fs_x/tree.c

(x_node_relation): Same.

* subversion/libsvn_repos/delta.c

(svn_repos_dir_delta2): Same.

* subversion/mod_dav_svn/repos.c

(do_out_of_date_check): Same.

* subversion/mod_dav_svn/util.c

(dav_svn__get_safe_cr): Same.

* subversion/mod_dav_svn/version.c

(dav_svn__checkout): Same.

* subversion/tests/libsvn_fs/fs-test.c

(check_related,

check_txn_related): Same.

  1. … 13 more files in changeset.
Make the commit editor for all ra layers behave in the same way on

encountering base revisions that are higher than the revision that the

commit is against (read: > HEAD). If callers don't have a real BASE

revision they should provide SVN_INVALID_REVNUM or fetch HEAD, instead

of just passing something above head.

* subversion/libsvn_repos/commit.c

(dir_baton,

file_baton): Add boolean to note that we already did a write check.

(check_out_of_date): New function.

(add_file_or_directory): Remember that we did a write check.

(delete_entry): Perform base_revision check if needed.

(apply_textdelta): Only check authz if we didn't just do that.

(open_file): Use standard out of date check.

(change_file_prop): Only check authz if we didn't just do that.

(change_dir_prop): Only check authz if we didn't just do that.

Improve out of date check.

* subversion/mod_dav_svn/repos.c

(remove_resource): Verify that base revision makes sense.

* subversion/tests/libsvn_ra/ra-test.c

(delete_revision_above_youngest): New test.

(test_funcs): Add delete_revision_above_youngest.

  1. … 2 more files in changeset.