Checkout Tools
  • last updated 48 mins ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
Adjust lifetime of the result returned from svn_repos__normalize_prop().

Previously, this function would make an unconditional copy of the value into

the result pool. While this may look convenient in some of the cases, there

is a downside of making unnecessary copies even if no normalization happened.

Instead of that, let's give the calling site control over the results' lifetime

by not making the copy if the input did not change. The callers may still dup()

the value if they need to, but with this approach there are no unavoidable

copies in various situations, such as where we would like to only update the

value if it changed or where we don't have an appropriate scratch pool, but

need to normalize the property value anyway.

* subversion/include/private/svn_repos_private.h

(svn_repos__normalize_prop): Adjust documentation.

* subversion/libsvn_repos/fs-wrap.c

(svn_repos__normalize_prop): Do not duplicate the input if no normalization

is required.

* subversion/libsvn_repos/dump_editor.c

(normalize_props): Adjust call to svn_repos__normalize_prop(), explicitly

duplicate the property value into the result pool.

* subversion/svnrdump/load_editor.c

(set_revision_property, set_node_property): Explicitly duplicate the property

values into the long-living pool.

* subversion/svnrdump/util.c

(svn_rdump__normalize_props): Adjust call to svn_repos__normalize_prop(),

explicitly duplicate the property value into the result pool.

  1. … 4 more files in changeset.
Remove redundant code.

* subversion/svnrdump/load_editor.c

(close_revision): Don't validate rev-props again; set_node_property() did it.

Source code re-organization.

* subversion/svnrdump/load_editor.c

(commit_baton_t,

commit_callback,

lock_retry_func,

fetch_base_func,

fetch_props,

fetch_props_func,

fetch_kind_func,

get_shim_callbacks,

get_lock): Move down, to reflect logical organization.

Refactoring svnrdump's load-editor: interpose a filtering layer.

This filtering layer is currently just a framework: it does nothing

functional. Some rdump-specific functionality will later be moved from

the dumpstream-to-editor layer (get_dumpstream_loader) to here.

For issue SVN-4780: Factor out the dumpstream loader editor driver.

* subversion/svnrdump/load_editor.c

(filter_*_t,

filter_*,

get_dumpstream_filter): New.

(svn_rdump__load_dumpstream): Insert this filtering layer.

Tweak for readability.

For issue SVN-4780: Factor out the dumpstream loader editor driver.

* subversion/svnrdump/load_editor.c

(set_revision_property,

close_revision): Say what we mean directly.

Refactor svnrdump's load-editor towards being reusable.

For issue SVN-4780: Factor out the dumpstream loader editor driver.

* subversion/svnrdump/load_editor.c

(new_revision_record): Move further down the file to match its placement

in the vtable and in the dumpstream processing order.

Refactor svnrdump's load-editor towards being reusable.

For issue SVN-4780: Factor out the dumpstream loader editor driver.

* subversion/svnrdump/load_editor.c

(parse_baton): Move commit editor variables from here...

(revision_baton): ... to here. Add 'head_rev_before_commit' here...

(new_revision_record): ... and initialize it here...

(revision_start_edit,

new_node_record): ... instead of recalculating it here.

(set_fulltext,

apply_textdelta,

close_node,

close_revision): Update uses of the moved variables.

Refactor svnrdump's load-editor towards being reusable.

For issue SVN-4780: Factor out the dumpstream loader editor driver.

This pulls out the 'get a commit editor' and 'get props' functions into

callbacks.

* subversion/svnrdump/load_editor.c

(loader_fns_t,

commit_baton_t): New.

(parse_baton): Add callbacks.

(commit_callback,

fetch_base_func,

fetch_props_func,

fetch_kind_func,

get_shim_callbacks): Take a commit_baton_t instead of a revision_baton.

(fetch_props): New: a callback, split out from fetch_props_func().

(revision_start_edit): New.

(new_node_record,

close_revision): Get a commit editor via callback instead of directly.

(remove_node_props): Get props via callback instead of directly.

(revstart): New: a callback.

(get_dumpstream_loader): Initialize the callbacks.

* subversion/svnrdump/load_editor.c

(commit_callback): Add a doc string.

Introduce a small separation of concerns in svnrdump's dumpstream loader.

* subversion/svnrdump/svnrdump.h,

subversion/svnrdump/load_editor.c

(get_dumpstream_loader): New, separated out...

(svn_rdump__load_dumpstream): ... from here. Improve doc string.

  1. … 3 more files in changeset.
* subversion/svnrdump/load_editor.c

(svn_rdump__load_dumpstream): Remove redundant code.

Update issue tracker links in comments, from Tigris (issuezilla) to Apache (Jira).

URL fragment identifiers like '#desc5' are left in place, not yet updated.

This is a merge of r1828508 from the 'shelve-checkpoint' branch where I

committed it by mistake.

  1. … 43 more files in changeset.
svnadmin: Introduce the `--normalize-props` option for the load and

load-revprops commands.

Currently, the only performed normalization is the automatic translation

of non-LF line endings in the svn: property values. Apparently, this is

a common issue reported by our users, as Subversion versions prior to 1.6

allowed such values for properties like svn:log and so they can be present

in the dump files. The behavior before this changeset is that a user can

either force loading the dump with such invalid properties using the

--bypass-prop-validation (that affects reading them in the future),

manually repair the dump file or use a tool like svnsync that performs

the property normalization. It would be better if the users had an option

to fix this right in the svnadmin load command.

See also the corresponding FAQ entry and related issues that have been

fixed in svnsync and svnrdump:

https://subversion.apache.org/faq.html#fix-nonLF-log

https://issues.apache.org/jira/browse/SVN-3404

https://issues.apache.org/jira/browse/SVN-4263

* subversion/include/svn_repos.h

(svn_repos_load_fs6): New version of svn_repos_load_fs5() accepting the

new `normalize_props` argument.

(svn_repos_load_fs5): Deprecate.

(svn_repos_load_fs_revprops): Accept the new `normalize_props` argument.

(svn_repos_get_fs_build_parser6): New version of

svn_repos_get_fs_build_parser5() accepting the new `normalize_props`

argument.

* subversion/include/private/svn_repos_private.h

(svn_repos__normalize_prop): New private helper. The `normalized_p`

argument will be currently unused by all calling sites, but it's added

as the groundwork to simplify implementing the notifications about

normalized properties in the future.

* subversion/libsvn_repos/deprecated.c

(svn_repos_load_fs5, svn_repos_get_fs_build_parser5): Implement by

forwarding to the new versions of these functions.

* subversion/libsvn_repos/fs-wrap.c

(): Include svn_subst.h.

(svn_repos__normalize_prop): Implement this new helper function.

The implementation is close to how svn_rdump__normalize_prop() is

written, but this function accepts two pools and handles the new

`normalized_p` output argument.

* subversion/libsvn_repos/load-fs-vtable.c

(struct parse_baton): Add new `normalize_props` field.

(change_rev_prop): Accept new `normalize_props` argument, call the

new helper function if it's set.

(close_revision): Normalize the revision properties if necessary.

(svn_repos_get_fs_build_parser6): New, replacing ...

(svn_repos_get_fs_build_parser5): ...this function.

(svn_repos_load_fs6): New, replacing ...

(svn_repos_load_fs5): ...this function.

(revprops_close_revision): Update call to change_rev_prop().

(build_revprop_parser): Accept new `normalize_props` argument, use it

to initialize the new field in the parse_baton.

(svn_repos_load_fs_revprops): Handle the new `normalize_props` argument

by forwarding it to build_revprop_parser().

* subversion/svnadmin/svnadmin.c

(svnadmin__cmdline_options_t, options_table): Add svnadmin__normalize_props.

(cmd_table): Enable the new option for load and load-revprops commands.

(struct svnadmin_opt_state): Add new `normalize_props` field and ...

(sub_main): ...initialize it here.

(subcommand_load, subcommand_load_revprops): Normalize properties when

running with --normalize-props. Use separate hints when encountering

invalid properties with non-LF line endings and all other kinds of

invalid properties.

* subversion/svnrdump/svnrdump.h

(svn_rdump__normalize_prop): Remove this declaration. This function is

now superseded by the new svn_repos__normalize_prop() helper.

* subversion/svnrdump/util.c

(): Include svn_repos_private.h, drop other includes.

(svn_rdump__normalize_prop): Remove. This function is now superseded

by the new svn_repos__normalize_prop() helper.

(svn_rdump__normalize_props): Use the new svn_repos__normalize_prop()

helper function.

* subversion/svnrdump/load_editor.c

(set_revision_property, set_node_property): Use the new

svn_repos__normalize_prop() helper function.

* subversion/tests/libsvn_repos/dump-load-test.c

(test_load_bad_props): Use the new svn_repos_load_fs6() function, keep

the previous behavior unchanged, i.e., do not normalize the properties.

* subversion/tests/cmdline/svntest/actions.py

(run_and_verify_load, load_repo): Accept new `normalize_props` argument.

* subversion/tests/cmdline/svnadmin_tests.py

(load_bad_props): Test loading with --normalize-props. Check for the error

codes reported during load with non-LF line endings and without the new

option. In cases when the load is successful, check the resulting

svn:log property value in the repository.

* tools/client-side/bash-completion

(_svnadmin): Extend completion info.

  1. … 12 more files in changeset.
Avoid a function with a potentially unexpected side effect by introducing

an output parameter instead of changing the value in place.

* subversion/svnrdump/svnrdump.h

(svn_rdump__normalize_prop): Add output parameter and update the docstring.

(svn_rdump__normalize_props): Update the docstring.

* subversion/svnrdump/util.c

(svn_rdump__normalize_prop): Populate the output parameter. Ensure

that the resulting value is always copied (even if unchanged) to the

result pool; that should also allow to sometimes get rid of the double

copying on the calling site.

(svn_rdump__normalize_props): Update call to svn_rdump__normalize_prop().

* subversion/svnrdump/load_editor.c

(set_revision_property, set_node_property): Update these calling sites of

svn_rdump__normalize_prop(). Don't make unnecessary copies, as

now the values are always copied to the result pool during the call

to svn_rdump__normalize_prop().

  1. … 2 more files in changeset.
Follow up on r1696166: More sizeof changes when using APR hashes.

* subversion/svndumpfilter/svndumpfilter.c

(new_node_record,

adjust_mergeinfo): Take the key size from the key variable.

* subversion/svnrdump/load_editor.c

(set_revision_mapping): Same.

* subversion/tests/libsvn_repos/repos-test.c

(check_locations_info,

test_get_file_revs): Same.

  1. … 2 more files in changeset.
* subversion/svnrdump/load_editor.c

(new_node_record,

close_revision): For clarity, introduce a variable 'head_rev_before_commit'

to hold the expression 'rb->rev - rb->rev_offset - 1'.

Inspired by r1664684.

* subversion/svnrdump/load_editor.c

(new_revision_record): Simplify.

Never use atoi() for converting rev nums; use SVN_STR_TO_REV() instead,

which is a wrapper around atol().

* subversion/svnrdump/load_editor.c

(new_revision_record,

new_node_record): s/atoi/SVN_STR_TO_REV/

* tools/dev/fsfs-access-map.c

(open_file): s/atoi/SVN_STR_TO_REV/

  1. … 1 more file in changeset.
* subversion/svnrdump/load_editor.c

(new_node_record): Pass HEAD instead of HEAD+1, as base revision.

Make svn_string_dup() more comfortable to use. Allow the input string

to be NULL and return NULL for the copy in that case.

Right now, it will segfault for NULL pointers and virtually all callers

check for NULL before calling and emulate the improved logic.

So, this change will not break any legal API user code but save us lots

of duplicated logic.

* subversion/include/svn_string.h

(svn_string_dup): Update docstring.

* subversion/libsvn_subr/string.c

(svn_string_dup): Return NULL as copy for NULL inputs.

* subversion/libsvn_client/mtcc.c

(mtcc_prop_getter): Simplify caller.

* subversion/libsvn_client/repos_diff.c

(change_file_prop,

change_dir_prop): Same.

* subversion/libsvn_fs_base/tree.c

(base_node_prop): Same.

* subversion/libsvn_ra_serf/commit.c

(change_dir_prop,

change_file_prop): Same.

* subversion/libsvn_repos/reporter.c

(get_revision_info): Same.

* subversion/libsvn_wc/diff_editor.c

(change_file_prop,

change_dir_prop): Same.

* subversion/libsvn_wc/externals.c

(change_file_prop): Same.

* subversion/libsvn_wc/props.c

(svn_wc__merge_props): Same.

* subversion/libsvn_wc/update_editor.c

(change_dir_prop,

change_file_prop): Same.

* subversion/svnrdump/load_editor.c

(set_node_property): Same.

  1. … 11 more files in changeset.
* subversion/svnrdump/dump_editor.c,

subversion/svnrdump/load_editor.c

Remove debug statements.

  1. … 1 more file in changeset.
Fix a remaining part of issue #4551 'svnrdump load commits wrong properties,

or fails, on a non-deltas dumpfile'. This change avoids making more than one

change-property call to the commit editor per property.

Since r1654194, RA-serf processes multiple change-property calls the same

way as other RA layers, using only the most recent value for a given

property, so this is only a theoretical bug. This fix is most easily tested

against RA-serf prior to r1654194, when svnrdump_tests.py 52 53 55 all fail

(XFail).

* subversion/svnrdump/load_editor.c

(node_baton): Add a prop-changes hash.

(new_node_record): Initialize it.

(set_node_property,

delete_node_property): Accumulate the prop change in the hash instead of

sending it immediately to the commit editor, thus storing only the most

recent value for each property.

(close_node): Send all pending prop changes to the commit editor.

Following up on r1653988, unbreak compilation in non maintainer builds.

* subversion/svnrdump/load_editor.c

(remove_node_props): Replace 4 calls to SVN_DBG() with calls to

LDR_DBG() to resolve build failure in non maintainer builds.

Fix issue #4551 "svnrdump load commits wrong properties, or fails, on a

non-deltas dumpfile", by teaching the svnrdump loader to find the original

properties of a copied node at the copy source location.

* subversion/svnrdump/load_editor.c

(ARE_VALID_COPY_ARGS): New macro.

(directory_baton): Add copyfrom fields.

(node_baton): Add a copyfrom URL field.

(new_revision_record): Explicitly initialize 'rb->db', for clarity, now

that we rely on it starting out as null.

(push_directory): New function, factored out of new_node_record() and

extended to store the copyfrom source of the directory.

(new_node_record): Factor out the push_directory() functionality. Store

the copyfrom URL separately from the copyfrom path. Move the variable

'relpath_compose' into a tighter scope. Tweak comments.

(remove_node_props): Look up the original properties in the copy source

even if it is implicitly copied (in other words a child of a copy).

* subversion/tests/cmdline/svnrdump_tests.py

(load_non_deltas_copy_with_props): Remove XFail marker. Extend.

(load_non_deltas_replace_copy_with_props): Remove XFail marker. Extend

a little.

  1. … 1 more file in changeset.
* subversion/svnrdump/load_editor.c

(new_node_record): Rewrite a block comment and move it to the top of its

block.

Document the specific requirements of commit editors to be given a URL

rather than a relative path in the copyfrom_path argument of their

add_file() and add_directory() methods.

* subversion/include/svn_ra.h

(svn_ra_get_commit_editor3): Add a comment: requires a URL.

* subversion/include/svn_ra_svn.h

(svn_ra_svn_get_editor): Add a comment: passes on either kind, verbatim.

(svn_ra_svn_drive_editor2): Add a comment: canonicalizes either kind.

* subversion/include/svn_repos.h

(svn_repos_get_commit_editor5): Add a comment: requires a URL.

* subversion/libsvn_ra_serf/ra_serf.h

(svn_ra_serf__get_commit_editor): Add a comment: requires a URL.

* subversion/libsvn_repos/commit.c

(add_file_or_directory): Add a comment: requires a URL.

* subversion/svnrdump/load_editor.c

(new_node_record): Add a comment: provides a URL.

  1. … 5 more files in changeset.
Factor out mergeinfo adjustment from 'svnadmin load' and 'svnrdump load'.

Part of issue #4483 "De-duplicate mergeinfo processing code in svnadmin load,

svnrdump load, svndumpfilter".

No functional change.

* subversion/include/private/svn_repos_private.h,

(svn_repos__adjust_mergeinfo_property): New.

* subversion/libsvn_repos/load-fs-vtable.c

(prefix_mergeinfo_paths): Remove note about duplication.

(renumber_mergeinfo_revs): Take separate arguments instead of batons.

Remove note about duplication.

(svn_repos__adjust_mergeinfo_property): Make semi-public, renaming from

'adjust_mergeinfo_property'. Take separate arguments instead of batons.

(set_node_property): Adjust the caller accordingly.

* subversion/svnrdump/load_editor.c

(prefix_mergeinfo_paths,

renumber_mergeinfo_revs,

adjust_mergeinfo_property): Delete.

(set_node_property): Call svn_repos__adjust_mergeinfo_property() instead.

Wrap any error with a simple message.

  1. … 2 more files in changeset.
* subversion/svnrdump/load_editor.c

(parse_baton,

directory_baton): Remove unused fields...

(uuid_record,

new_node_record): ... and the assignments to them.

Cosmetically tweak some duplicated code to be even more identical. No

functional change.

* subversion/libsvn_repos/load-fs-vtable.c

(parse_baton): Rename 'oldest_old_rev' to 'oldest_dumpstream_rev' for

consistency with svnrdump/load_editor.c, and update all references.

(adjust_mergeinfo_property): Correct a comment.

* subversion/svnrdump/load_editor.c

(adjust_mergeinfo_property): New, factored out...

(set_node_property): ... of here.

  1. … 1 more file in changeset.
Fix issue #4538 "'load' strips r1 references in mergeinfo": don't filter out

references to r1 in mergeinfo during 'svnadmin load' and 'svnrdump load'.

r1 is not an invalid mergeinfo source revision, merely an unlikely one, and

we don't treat it as invalid anywhere else.

This reverts r941438 which has been in place since v1.7.0. Note also that

this commit didn't remove r0 mergeinfo, only r1, despite the claim in its

log message.

* subversion/libsvn_repos/load-fs-vtable.c,

subversion/svnrdump/load_editor.c

(renumber_mergeinfo_revs): Don't filter out references to r1.

Canonicalize mergeinfo, where previously the combination of sorting

followed by filtering achieved canonicalization.

* subversion/svndumpfilter/svndumpfilter.c

(adjust_mergeinfo): Remove a comment that said r1 is invalid.

* subversion/tests/cmdline/svndumpfilter_tests.py

(filter_mergeinfo_revs_outside_of_dump_stream): Correct a comment that

said r1 is invalid, written in r941408.

  1. … 3 more files in changeset.