Checkout Tools
  • last updated 4 hours ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates

Changeset 1875921 is being indexed.

Introduce 'svnadmin build-repcache' command.

Implement the 'svnadmin build-repcache' CLI and add an ioctl API for building

the representation cache.

The implementation iterates over revisions in the specified range and recursively

processes the changed nodes, starting from the corresponding revision roots.

For each changed node, it ensures that its data and property representations

exist in the rep-cache. The nodes are processed in the same order as when

committing a transaction (see write_final_rev() function in libsvn_fs_fs/transaction.c),

so that the rep-cache.db files are fully consistent.

* subversion/svnadmin/svnadmin.c

(cmd_table): Add and document the 'build-repcache' command.

(subcommand_build_repcache,

build_rep_cache,

build_rep_cache_progress_func): New.

* subversion/include/svn_error_codes.h

(SVN_ERR_FS_REP_SHARING_NOT_ALLOWED,

SVN_ERR_FS_REP_SHARING_NOT_SUPPORTED): New error codes.

* subversion/include/private/svn_fs_fs_private.h

(svn_fs_fs__ioctl_build_rep_cache_input_t,

SVN_FS_FS__IOCTL_BUILD_REP_CACHE): New.

* subversion/libsvn_fs_fs/fs.c

(fs_ioctl): Handle SVN_FS_FS__IOCTL_BUILD_REP_CACHE.

* subversion/libsvn_fs_fs/fs_fs.h

* subversion/libsvn_fs_fs/fs_fs.c

(): Include 'low_level.h'.

(svn_fs_fs__build_rep_cache,

reindex_node,

ensure_representation_sha1): New. Iterate over revisions and recursively

process the changed nodes. For each changed node, ensure that its data

and property representations exist in the rep-cache.

* subversion/tests/cmdline/svnadmin_tests.py

(build_repcache): New test.

(test_list): Add the new test.

* subversion/tests/libsvn_fs_fs/fs-fs-private-test.c

(): Include 'libsvn_fs_fs/rep-cache.h' and 'libsvn_fs/fs-loader.h'.

(build_rep_cache): New test.

(test_funcs): Add the new test.

* tools/client-side/bash_completion

(_svnadmin): Add the 'build-repcache' command.

Patch by: Denis Kovalchuk <denis.kovalchuk@visualsvn.com>

  1. … 7 more files in changeset.
* subversion/tests/cmdline/svnadmin_tests.py

(load_normalize_node_props): Rewrite without literal trailing spaces, to

prevent the bug fixed in r1875620 from recurring.

Increment the trunk version number to 1.15, and introduce a new CHANGES

section, following the creation of the 1.14.x release branch.

* subversion/include/svn_version.h,

subversion/bindings/javahl/src/org/apache/subversion/javahl/NativeResources.java,

subversion/tests/cmdline/svntest/main.py

(SVN_VER_MINOR): Increment to 15.

* CHANGES: New section for 1.15.0.

  1. … 3 more files in changeset.
* subversion/tests/cmdline/svnadmin_tests.py

(load_normalize_node_props): Restore intended trailing whitespace which

was removed in r1875617.

Update svn_ra_open4() to svn_ra_open5() across the tree.

Most of these changes are mechanical. JavaHL gains a redirect cycle fix.

The set_svn_ra_open() callback gets updated to the new API.

* subversion/bindings/javahl/native/CommitEditor.cpp

(open_callback_session): Use svn_ra_open5() and show the redirected URL in

an error message.

* subversion/bindings/javahl/native/RemoteSession.cpp

(RemoteSession::RemoteSession): Use svn_ra_open5() and detect redirect

cycles based on the actual redirection URL sent by the server.

* subversion/libsvn_client/ra.c

(svn_client__open_ra_session_internal): Use svn_ra_open5().

* subversion/libsvn_ra/ra_loader.c

(svn_ra_open5): Pass svn_ra_open5 to the svn_svn_ra_open() callback.

* subversion/libsvn_ra/ra_loader.h

(svn_ra__open_func_t): Update to match svn_ra_open5().

* subversion/libsvn_ra_serf/serf.c

(svn_ra_serf__open): Update comment referring to svn_ra_open4().

* subversion/svnsync/svnsync.c

(do_initialize, open_source_session,

open_target_session): Update to svn_ra_open5().

* subversion/tests/cmdline/atomic-ra-revprop-change.c

(change_rev_prop): Update to svn_ra_open5().

* subversion/tests/libsvn_ra/ra-test.c

(make_and_open_repos, check_tunnel_callback_test, tunnel_callback_test,

tunnel_run_checkout, commit_locked_file): Update to svn_ra_open5().

  1. … 7 more files in changeset.
* everywhere: Run tools/dev/remove-trailing-whitespace.sh to remove

trailing whitespace before we branch 1.14.

  1. … 91 more files in changeset.
* libsvn_subr/mergeinfo-test.c

(test_rangelist_merge_random_non_validated_inputs): Indentation fix.

Plug error leaks in newly added mergeinfo tests.

Found by: svn-bb-openbsd buildbot

https://ci.apache.org/builders/svn-bb-openbsd/builds/510/steps/Test/logs/faillog-ra_svn-fsfs

[[[

subversion/tests/svn_test_main.c:475: (apr_err=SVN_ERR_TEST_FAILED)

svn_tests: E200006: Test crashed (run in debugger with '--allow-segfaults')

FAIL: mergeinfo-test 25: test rangelist merge random non-validated inputs

]]]

Core was generated by `mergeinfo-test'.

Program terminated with signal SIGABRT, Aborted.

#0 *_libc_abort () at /usr/src/lib/libc/stdlib/abort.c:57

57 memset(&sa, 0, sizeof(sa));

(gdb) up

#1 0x0000010070bcff68 in err_abort (data=0x10022671880)

156 abort();

(gdb) list

151 {

152 svn_error_t *err = data; /* For easy viewing in a debugger */

153 SVN_UNUSED(err);

154

155 if (!getenv("SVN_DBG_NO_ABORT_ON_ERROR_LEAK"))

156 abort();

157 return APR_SUCCESS;

158 }

159 #endif

160

(gdb) p *err

$3 = {apr_err = 235000,

message = 0x1001bc9c280 "In file 'subversion/libsvn_subr/mergeinfo.c' line 1096: assertion failed (rangelist_is_sorted(chg))", child = 0x0,

pool = 0x10085adbc00,

file = 0x10070d799b0 "subversion/libsvn_subr/mergeinfo.c", line = 1096}

(gdb)

* subversion/tests/libsvn_subr/mergeinfo-test.c

(clear_failure_mode_errors): New helper function. Clears errors accumulated

during a test run.

(test_rangelist_merge_random_canonical_inputs,

test_rangelist_merge_random_semi_c_inputs,

test_rangelist_merge_random_non_validated_inputs): Clear accumulated errors.

Adjust tests following r1875039 (disable shelving by default).
Merge the 'decouple-shelving-cli' branch to trunk.

Summary:

Add the shelving v2 implementation from Subversion 1.11, as an alternative

to the shelving v3 implementation from Subversion 1.12.

They have substantially different pros and cons, so it is beneficial for the

user to be able to choose.

Make the shelving CLI version selectable by an environment variable:

env. var. not set => shelving v3 enabled

SVN_EXPERIMENTAL_COMMANDS=shelf3 => shelving v3 enabled

SVN_EXPERIMENTAL_COMMANDS=shelf2 => shelving v2 enabled

SVN_EXPERIMENTAL_COMMANDS= => no shelving CLI

  1. … 15 more files in changeset.
Followup to r1874057, escape whitespace instead of quoting filename

As danielsh pointed out, only specific characters can be escaped by a backslash

in quoted shell strings. Rather than surrounding the escaped path with double

quotes, post-process the quoted path and manually escape any whitespace that is

present.

* subversion/libsvn_subr/cmdline.c

(escape_path): New function, wrapper around apr_pescape_shell(), which

handles escaping of whitespace.

(svn_cmdline__edit_file_externally, svn_cmdline__edit_string_externally):

Call the new function instead of apr_pescape_shell()

* subversion/tests/cmdline/update_tests.py

(update_accept_conflicts): Include ';' in renamed path ("p; i"), to ensure

non-whitespace escaping is handled correctly.

  1. … 1 more file in changeset.
Escape filenames when invoking $SVN_EDITOR

Per https://subversion.apache.org/faq.html#svn-editor, $SVN_EDITOR is invoked

through the shell instead of directly executed. The user is expected to

properly escape/quote $SVN_EDITOR, but svn was putting the filename directly

into the command without any escaping. This therefore breaks attempts to,

e.g., run the editor from the merge conflict dialog when a path has special

characters.

Update locations where we invoke the editor to quote the filename as well as

escape shell special characters using apr_pescape_shell(). The quotes are

needed in addition to the escaping, since apr_pescape_shell() does not escape

whitespace.

* subversion/libsvn_subr/cmdline.c

(svn_cmdline__edit_file_externally, svn_cmdline__edit_string_externally):

Quote and escape, via apr_pescape_shell(), the filename in the command line.

* subversion/tests/cmdline/update_test.py

(update_accept_conflicts): Rename "A/D/G/pi" to "A/D/G/p i" before performing

the merge, so the test exercises the changes above.

  1. … 1 more file in changeset.
Fix test failures seen on 32-bit architectures (Fedora Raw Hide, both

i686 and armv7hl) when building with GCC 10 snapshots.

* subversion/tests/libsvn_fs_fs/fs-fs-pack-test.c

(get_rev_contents): Avoid signed integer overflow on platforms with

32-bit long.

Follow-up to r1872925:

* subversion/tests/README: Add missing case 'SKIP' to the synopsis.

Fix issue #4840 "Merge assertion failure in svn_sort__array_insert".

This replaces the implementation of svn_rangelist_merge2() with a correct

and simpler one.

* subversion/libsvn_subr/mergeinfo.c

(adjust_remaining_ranges,

rangelist_merge2): Delete.

(rangelist_interval_kind_t,

rangelist_interval_t,

rangelist_interval_iterator_t,

rlii_update,

rlii_next_any_interval,

rlii_first,

rlii_next,

rangelist_builder_t,

rl_builder_new,

rl_builder_flush,

rl_builder_add_interval,

rangelist_merge): New.

(svn_rangelist_merge2): Adjust to call the new implementation.

* subversion/tests/libsvn_subr/mergeinfo-test.c

(test_rangelist_merge_canonical_result,

test_rangelist_merge_array_insert_failure,

test_rangelist_merge_random_canonical_inputs,

test_rangelist_merge_random_semi_c_inputs): Remove XFAIL.

  1. … 1 more file in changeset.
Fix test expectations of an XFAIL test.

For issue #4840 "Merge assertion failure in svn_sort__array_insert".

Follow-up to r845298 (r5224):

* subversion/tests/README: Add missing cases to the synopsis.

Fix weak test coverage.

In the tests for "semi-canonical" and "canonical" kinds of inputs to

svn_rangelist_merge2(), the input generator previously called "generate a

random non-validated input" repeatedly until it passed certain quality

tests. That made the result heavily biased towards very simple cases.

As a result, the coverage of the "semi-canonical" and "canonical" tests

missed the failure mode of the original bug report #4840.

This generates "semi-canonical" and "canonical" inputs with a fairer

distribution, increasing their coverage to include that failure mode.

For issue 4840 "Merge assertion failure in svn_sort__array_insert"

* subversion/tests/libsvn_subr/mergeinfo-test.c

(rand_interval_triangular): New.

(rangelist_random_non_validated): Use it.

(int_compare,

ascending_values): New.

(rangelist_random_semi_canonical,

rangelist_random_canonical): Rewrite.

(add_failure_mode): Deduplicate 'Attempted insert at index ...' messages.

Use the variable I declared in r1872921.
Add comments about parameters of random input generation.

For issue #4840 "Merge assertion failure in svn_sort__array_insert".

Report each failure mode only once in random-input tests.

Instead of printing each case that fails, print only the first example of

each failure mode. This makes it easier to manually check that test has

enough coverage. (Programmatic checks for particular coverage could be

added.)

For issue 4840 "Merge assertion failure in svn_sort__array_insert"

* subversion/tests/libsvn_subr/mergeinfo-test.c

(new_response): New.

(test_rangelist_merge_random_canonical_inputs,

test_rangelist_merge_random_semi_c_inputs,

test_rangelist_merge_random_non_validated_inputs): Use it.

New test for issue 4840 "Merge assertion failure in svn_sort__array_insert".

* subversion/tests/libsvn_subr/mergeinfo-test.c

(test_rangelist_merge_array_insert_failure): New test.

(test_funcs): Run it, XFAIL.

Make random-input mergeinfo tests repeatable.

The pseudo-random sequence used in each random-input mergeinfo test is now

repeatable and independent of other tests.

This ensures the test results are repeatable and avoids occasional

unexpected pass of expected-fail tests.

For issue #4840, "Merge assertion failure in svn_sort__array_insert".

* subversion/tests/libsvn_subr/mergeinfo-test.c

(rand_less_than): Use a passed-in state ('seed').

(rangelist_random_non_validated,

rangelist_random_semi_canonical,

rangelist_random_canonical,

mergeinfo_random_non_validated): Pass through the state.

(test_rangelist_merge_random_canonical_inputs,

test_rangelist_merge_random_canonical_inputs,

test_rangelist_merge_random_semi_c_inputs,

test_mergeinfo_merge_random_non_validated_inputs):

Keep random state separately in each test.

Fix a test crash in non-debug builds.

A follow-up to r1872121.

Found by: nathanhartman

subversion/tests/libsvn_subr/mergeinfo-test.c

(rangelist_to_string): Properly skip error trace links.

* subversion/tests/cmdline/pegrev_parse_tests.py: Set the executable bit,

as already set on all sibling *_tests.py files.

Random-input testing for svn_mergeinfo_merge2().

For issue #4840, "Merge assertion failure in svn_sort__array_insert".

* subversion/tests/libsvn_subr/mergeinfo-test.c

(mergeinfo_random_non_validated,

mergeinfo_to_string_debug,

mergeinfo_merge_random_inputs): New.

(test_mergeinfo_merge_random_non_validated_inputs): New test.

(test_funcs): Run it.

Avoid aborting on assertion failure in the area of mergeinfo calculations.

Instead, raise a catchable assertion error.

If an error occurs in svn_rangelist_merge2(), produce a more detailed error

message to aid in debugging.

Introduce and use 'svn_sort__array_insert2()' which checks its inputs. The

old version of this function was aborting on out-of-range inputs. Introduce

and use 'svn_sort__array_delete2()' likewise, as similar issues may show up

here too. The old version of this function was ignoring calls with

out-of-range inputs. The old versions of both functions are still in use

elsewhere in the Subversion libraries.

For issue #4840, "Merge assertion failure in svn_sort__array_insert".

* subversion/include/private/svn_sorts_private.h,

subversion/libsvn_subr/sorts.c

(svn_sort__array_insert2,

svn_sort__array_delete2): New.

* subversion/libsvn_client/merge.c

(slice_remaining_ranges,

insert_child_to_merge): Allow returning an error.

Everywhere: use svn_sort__array_insert2() and svn_sort__array_delete2().

* subversion/libsvn_subr/mergeinfo.c

(adjust_remaining_ranges): Allow returning an error.

(dual_dump): Replace this old debug helper...

(rangelist_to_string_debug): ... with this new one.

(svn_rangelist_merge2): Extract the body of this function into a local

'rangelist_merge2', leaving the original as an error-checking

wrapper. If an error occurs, report its inputs.

(rangelist_is_sorted): New.

Everywhere: use svn_sort__array_insert2() and svn_sort__array_delete2().

* subversion/tests/libsvn_subr/mergeinfo-test.c

(test_rangelist_merge_random_non_validated_inputs): Expect assertion

failures; ignore them.

(test_funcs): Expect that test to pass now.

  1. … 4 more files in changeset.
Use a matching integer type instead of casts, as a cleaner follow-up to r1872108.

Suggested by: brane

Avoid integer conversion warning. A follow-up to r1872107.

* subversion/tests/libsvn_subr/mergeinfo-test.c

(rangelist_to_array): Add explicit casts.

Random-input testing for issue #4840, "Merge assertion failure in

svn_sort__array_insert".

This adds tests for svn_rangelist_merge2() with canonical inputs,

with "semi-canonical" inputs which meet criteria described in its

doc string, and with non-validated inputs.

* subversion/tests/libsvn_subr/mergeinfo-test.c

(...): Helper functions.

(test_rangelist_merge_random_canonical_inputs,

test_rangelist_merge_random_semi_c_inputs,

test_rangelist_merge_random_non_validated_inputs): New tests.

(test_funcs): Run them.