subversion

Checkout Tools
  • last updated 3 hours ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates

Changeset 1866951 is being indexed.

Make the dump stream parser more resilient to malformed dump streams that

do not contain \n characters at all.

Previously, we'd attempt to load the whole input into memory due to how

svn_stream_readline() is currently implemented. Doing so could potentially

choke for large files. The corresponding real-world case is where a user

(accidentally) attempts to load a huge binary file that does not contain \n

characters as the repository dump.

This is the potential cause of the OOM reported in

https://lists.apache.org/thread.html/c96eb5618ac0bf6e083345e0fdcdcf834e30913f26eabe6ada7bab62@%3Cusers.subversion.apache.org%3E

* subversion/libsvn_repos/load.c

(parse_format_version): Read the dump version string directly from

stream, with an upper limit of 80 bytes. Comment on why we don't use

svn_stream_readline() for this particular case.

(svn_repos_parse_dumpstream3): Update the call to parse_format_version().

Fix an issue with the readline implementation for file streams that could

cause excessive memory usage for inputs containing one or multiple \0 bytes.

This is the likely cause of the OOM reported in

https://lists.apache.org/thread.html/c96eb5618ac0bf6e083345e0fdcdcf834e30913f26eabe6ada7bab62@%3Cusers.subversion.apache.org%3E

(I think that the problem itself is a regression introduced by me in 1.10.)

Note: one thing I noticed while preparing the fix is that our `readline_fn`

functions for different streams have inconsistent behavior if the input data

contains \0 bytes. More specifically, they may return different `line` values,

that may either be truncated at \0 or actually contain the whole data between

EOLs, including \0 bytes. For now, this patch only fixes the excessive memory

usage problem, and I noted this related problem in the test and left it for

future work.

* subversion/libsvn_subr/stream.c

(readline_apr_lf, readline_apr_generic): Reallocate the buffer based on its

current size, instead of calculating the new size based on the already

prealloc'd size. There are no actual benefits in reallocating based on

`blocksize`, and in the described case with \0 bytes doing so also backfires

and may cause excessive allocations due to the actual size of the string

being less than we expect it to. A degenerate case of the erroneous

behavior is ...

* subversion/tests/libsvn_subr/stream-test.c

(test_stream_readline_file_nul): ...exploited in this new test.

(test_funcs): Run new test.

* subversion/tests/libsvn_subr

(): Adjust svn:ignore.

* tools/dist/release.py (main): Move '--branch' to be a global option.
[On the staging-ng branch]

* BRANCH-README: New file.

Create branch of Subversion site for modernization project.

Remove unused old branch: site-ng.

This branch was created in 2015 and never used. Also it was branched

from the wrong level so it contained other branches. To be replaced by

a new "staging-ng" branch under subversion/site/.

* tools/dist/release.py: Remove obsolete distinction between http and https access.
When following an HTTP redirect, use the Location header URL exactly.

Previously we canonicalized the redirect URL, which could lead to a redirect

loop. Then Subversion would report a redirect loop as the error, potentially

hiding a more interesting error such as when the target is not in fact a

Subversion repository.

A manual test case (on a non-repository):

before:

$ svn ls https://archive.apache.org/dist

Redirecting to URL 'https://archive.apache.org/dist':

Redirecting to URL 'https://archive.apache.org/dist':

svn: E195019: Redirect cycle detected for URL 'https://archive.apache.org/dist'

after:

$ svn ls https://archive.apache.org/dist

Redirecting to URL 'https://archive.apache.org/dist/':

svn: E170013: Unable to connect to a repository at URL 'https://archive.apache.org/dist/'

svn: E175003: The server at 'https://archive.apache.org/dist/' does not support the HTTP/DAV protocol

* subversion/libsvn_ra_serf/options.c

(svn_ra_serf__exchange_capabilities): Don't canonicalize the redirect URL.

* subversion/libsvn_ra_serf/util.c

(response_get_location): Don't canonicalize the redirect URL.

* tools/dist/release.py

(get_branch_path): New, factored out from several callers.

* tools/dist/release.py

Use a global constant for archive URL so all ASF URLs are thus defined near the top of file.

* index.html

(#site-content): Use the correct trademark symbol.

* publish/index.html: Fix TM/(R) attributions on 'Apache Subversion'.
* publish/site-nav.html

Move 'binary packages' above 'source download' as a tiny recognition of

what I suppose are the most common needs.

* publish/site-nav.html: Add a link to 'Source Code' page in nav menu.
* publish/source-code.html

(#source-svn): un-linkify the ASF repo root, as not useful here.

Add a hint about mod_dav_svn misconfiguration.

* subversion/mod_dav_svn/mod_dav_svn.c

(merge_dir_config): When warning about an overlapping configuration,

if the two configuration blocks are for the same URL then hint that

the problem may be including the same config twice.

* branches/1.12.x/STATUS: Vote for r1866425.
Fix a comment. No functional changes.

* subversion/tests/libsvn_wc/wc-queries-test.c

(test_query_expectations): Minor fix in a comment: too -> to.

* tools/dist/release.py: Add tool versions for 1.13 (same as 1.10 ... 1.12).
* STATUS: Nominate r1866425 (Last-Modified header) for 1.12.x too.

* upcoming.part.html: Automatically regenerated
Merge r1865266 from trunk:

* r1865266

mod_dav_svn: Always install cleanup handler for FS warning logging.

Justification:

FS warning logging cleanup handler was only installed in the presence

of an R->USER request field, which is unrelated.

Votes:

+1: stsp, rhuijben, brane

Merge r1864025 from trunk:

* r1864025

fsfs, fsx: Make an error message clearer to developers.

Justification:

Prevent hexadecimal values from being taken for decimal ones.

Votes:

+1: danielsh, rhuijben, brane

Merge the r1863987 group from trunk:

* r1863987, r1863990

Fix format of Requires(.private) fields in pkg-config files

Justification:

Fixes format of .pc files and specifies correct include/link flags

Votes:

+1: jamessan, rhuijben, brane

    • ?
    /branches/1.12.x/build/generator/gen_make.py
Merge r1864440 from trunk:

* r1864440

Fix 'svn patch' setting UNIX permissions to 0600 on files with props.

Justification:

'svn patch' should honour the user's umask.

User complained: https://svn.haxx.se/dev/archive-2019-07/0072.shtml

Votes:

+1: stsp, rhuijben, brane

Merge the r1863987 group from trunk:

* r1863987, r1863990

Fix format of Requires(.private) fields in pkg-config files

Justification:

Fixes format of .pc files and specifies correct include/link flags

Votes:

+1: jamessan, rhuijben, brane

* branches/1.12.x/STATUS: Approve r1865266.
* branches/1.12.x/STATUS: Approve r1864025 (and actually add my vote to the r1863987 group).
* branches/1.10.x/STATUS: Approve r1864440 and the r1863987 group.
* branches/1.12.x/STATUS: Vote for r1864440 and approve the r1863987 group.