subversion

Checkout Tools
  • last updated 4 hours ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
Increment the trunk version number to 1.14, and introduce a new CHANGES

section, following the creation of the 1.13.x release branch.

* subversion/include/svn_version.h,

subversion/bindings/javahl/src/org/apache/subversion/javahl/NativeResources.java,

subversion/tests/cmdline/svntest/main.py

(SVN_VER_MINOR): Increment to 14.

* CHANGES: New section for 1.14.0.

* CHANGES: pick a release date for 1.13.0.
* CHANGES: update with changes since 1.12.x.
Make the dump stream parser more resilient to malformed dump streams that

do not contain \n characters at all.

Previously, we'd attempt to load the whole input into memory due to how

svn_stream_readline() is currently implemented. Doing so could potentially

choke for large files. The corresponding real-world case is where a user

(accidentally) attempts to load a huge binary file that does not contain \n

characters as the repository dump.

This is the potential cause of the OOM reported in

https://lists.apache.org/thread.html/c96eb5618ac0bf6e083345e0fdcdcf834e30913f26eabe6ada7bab62@%3Cusers.subversion.apache.org%3E

* subversion/libsvn_repos/load.c

(parse_format_version): Read the dump version string directly from

stream, with an upper limit of 80 bytes. Comment on why we don't use

svn_stream_readline() for this particular case.

(svn_repos_parse_dumpstream3): Update the call to parse_format_version().

Fix an issue with the readline implementation for file streams that could

cause excessive memory usage for inputs containing one or multiple \0 bytes.

This is the likely cause of the OOM reported in

https://lists.apache.org/thread.html/c96eb5618ac0bf6e083345e0fdcdcf834e30913f26eabe6ada7bab62@%3Cusers.subversion.apache.org%3E

(I think that the problem itself is a regression introduced by me in 1.10.)

Note: one thing I noticed while preparing the fix is that our `readline_fn`

functions for different streams have inconsistent behavior if the input data

contains \0 bytes. More specifically, they may return different `line` values,

that may either be truncated at \0 or actually contain the whole data between

EOLs, including \0 bytes. For now, this patch only fixes the excessive memory

usage problem, and I noted this related problem in the test and left it for

future work.

* subversion/libsvn_subr/stream.c

(readline_apr_lf, readline_apr_generic): Reallocate the buffer based on its

current size, instead of calculating the new size based on the already

prealloc'd size. There are no actual benefits in reallocating based on

`blocksize`, and in the described case with \0 bytes doing so also backfires

and may cause excessive allocations due to the actual size of the string

being less than we expect it to. A degenerate case of the erroneous

behavior is ...

* subversion/tests/libsvn_subr/stream-test.c

(test_stream_readline_file_nul): ...exploited in this new test.

(test_funcs): Run new test.

* subversion/tests/libsvn_subr

(): Adjust svn:ignore.

* tools/dist/release.py (main): Move '--branch' to be a global option.
* tools/dist/release.py: Remove obsolete distinction between http and https access.
When following an HTTP redirect, use the Location header URL exactly.

Previously we canonicalized the redirect URL, which could lead to a redirect

loop. Then Subversion would report a redirect loop as the error, potentially

hiding a more interesting error such as when the target is not in fact a

Subversion repository.

A manual test case (on a non-repository):

before:

$ svn ls https://archive.apache.org/dist

Redirecting to URL 'https://archive.apache.org/dist':

Redirecting to URL 'https://archive.apache.org/dist':

svn: E195019: Redirect cycle detected for URL 'https://archive.apache.org/dist'

after:

$ svn ls https://archive.apache.org/dist

Redirecting to URL 'https://archive.apache.org/dist/':

svn: E170013: Unable to connect to a repository at URL 'https://archive.apache.org/dist/'

svn: E175003: The server at 'https://archive.apache.org/dist/' does not support the HTTP/DAV protocol

* subversion/libsvn_ra_serf/options.c

(svn_ra_serf__exchange_capabilities): Don't canonicalize the redirect URL.

* subversion/libsvn_ra_serf/util.c

(response_get_location): Don't canonicalize the redirect URL.

* tools/dist/release.py

(get_branch_path): New, factored out from several callers.

* tools/dist/release.py

Use a global constant for archive URL so all ASF URLs are thus defined near the top of file.

Add a hint about mod_dav_svn misconfiguration.

* subversion/mod_dav_svn/mod_dav_svn.c

(merge_dir_config): When warning about an overlapping configuration,

if the two configuration blocks are for the same URL then hint that

the problem may be including the same config twice.

Fix a comment. No functional changes.

* subversion/tests/libsvn_wc/wc-queries-test.c

(test_query_expectations): Minor fix in a comment: too -> to.

* tools/dist/release.py: Add tool versions for 1.13 (same as 1.10 ... 1.12).
Follow up to r1865987: Fix another printf format bug in FSFS.

* subversion/libsvn_fs_fs/verify.c (compare_p2l_to_rev):

Cast entry::type to unsigned integer and use the %u format specifier.

Found by: danielsh

* COMMITTERS: Add myself as a full committer.

mod_dav_svn: Set Last-Modified response header for 'external' GET requests

(i.e. requests to URI's not under /!svn). This partially reverts r1724790

which removed it for all GET requests, for performance reasons (however, for

external requests the Last-Modified header is needed for certain use cases).

See this thread on dev@:

https://lists.apache.org/thread.html/51f4f1fb5363c1d1c393f6ab769b12b9c7914af583fc333c652306d0@%3Cdev.subversion.apache.org%3E

Date: 2019/09/02 14:20:16 GMT

From: Johan Corveleyn

To: Subversion dev list

Subject: Re: Last-Modified HTTP header in GET responses

Message-ID: <CAB84uBX4dNCsbaOn1Dzy0N3CB4rfC0_ZVCZ_TpVikt+p1zMGNg@mail.gmail.com>

Review by: brane

* subversion/mod_dav_svn/repos.c

(get_last_modified): New function (reverted its removal in r1724790).

(set_headers): Set Last-Modified response header for 'external' requests.

* subversion/tests/cmdline/mod_dav_svn_tests.py

(last_modified_header): New test.

Use more of the new _safe variants of canonicalization functions.

This commit converts all relevant function calls in libsvn_repos.

* subversion/libsvn_repos/commit.c

(add_file_or_directory, delete_entry, open_directory, open_file): Use

svn_relpath_canonicalize_safe() instead of svn_relpath_canonicalize().

* subversion/libsvn_repos/delta.c

(svn_repos_dir_delta2): Use svn_relpath_canonicalize_safe() instead of

svn_relpath_canonicalize().

* subversion/libsvn_repos/dump.c

(make_dir_baton): Convert to return an svn_error_t and use

svn_relpath_canonicalize_safe() instead of svn_relpath_canonicalize().

(open_root, add_directory, open_directory): Update callers.

(dump_node): Use svn_relpath_canonicalize_safe() instead of

svn_relpath_canonicalize().

* subversion/libsvn_repos/dump_editor.c

(make_dir_baton): Convert to return an svn_error_t and use

svn_relpath_canonicalize_safe() instead of svn_relpath_canonicalize().

(open_root, add_directory, open_directory): Update callers.

* subversion/libsvn_repos/load-fs-vtable.c

(prefix_mergeinfo_paths, make_node_baton, svn_repos_get_fs_build_parser6):

Use svn_relpath_canonicalize_safe() instead of svn_relpath_canonicalize().

Use more of the new _safe variants of canonicalization functions.

* subversion/svnserve/serve.c

(find_repos): Use svn_dirent_canonicalize_safe() instead of

svn_dirent_canonicalize().

Use more of the new _safe variants of canonicalization functions.

This commit converts relpath function calls in svnserve.

* subversion/svnserve/serve.c

(set_path, delete_path, link_path, add_lock_tokens, unlock_paths, get_file,

get_dir, update, switch_cmd, status, diff, get_mergeinfo, log_cmd,

check_path, stat_cmd, get_locations, get_location_segments, get_file_revs,

lock, lock_many, unlock, unlock_many, get_lock, get_locks, get_deleted_rev,

get_inherited_props, list, find_repos): Use svn_relpath_canonicalize_safe()

instead of svn_relpath_canonicalize().

Introduce more use of the new _safe variants of canonicalization functions.

This commit converts uri function calls in svnserve; more to follow.

* subversion/svnserve/serve.c

(canonicalize_access_file, link_path, reparent, switch_cmd, diff,

construct_server_baton): Call svn_uri_canonicalize_safe() instead

of svn_uri_canonicalize().

* tools/dev/unix-build/Makefile.svn: Use a working URL for junit jar file.

* tools/buildbot/bb-openbsd/svnbuild.sh: Pre-fetch distfiles before building.

* tools/dev/unix-build/Makefile.svn: Use working URL to fetch bzip2-1.0.6.

* tools/dev/unix-build/Makefile.svn: URL to Cyrus SASL distfile has moved.

* tools/buildbot/slaves/bb-openbsd/svnbuild.sh,

tools/buildbot/slaves/bb-openbsd/svnclean.sh: Make these scripts more

resilient against missing files after buildslave system reboot.

Add a test for SVN-4827, svn x-shelve checksum mismatch.

* subversion/tests/cmdline/shelf_tests.py

(shelve_with_kw_translation): New test.

(test_list): Run it.

* build/generator/gen_base.py

(TargetExe.add_dependencies): Add 'svnmover' as another special case,

so 'make davautocheck' will work correctly without 'make svnmover'

or 'make tools' having been run before it.

Found by: danielsh

Issue #4828: by default, hide global options too.

* subversion/libsvn_subr/opt.c

(print_command_info3, subcommand_help): Let 'verbose' apply to global options a well.

* subversion/tests/cmdline/getopt_tests_data/svn_help_log_switch_stdout

Adjust expected help output.

Issue #4828: update test expectations, following r1866272.
Issue #4828: remind user they can use '-v' to show experimental subcommands.

* subversion/libsvn_subr/opt.c

(print_generic_help_body3): Show a reminder if experimental subcommands were

hidden.