svndumpfilter

Checkout Tools
  • last updated 3 hours ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
Follow up to r1849080: move a private function out of a public header.

* subversion/include/svn_dirent_uri.h: Update file-level docstrin.

(svn_relpath__internal_style): Removed.

* subversion/include/private/svn_dirent_uri_private.h: New file.

(svn_relpath__make_internal): Renamed from svn_relpath__internal_style.

* subversion/libsvn_subr/dirent_uri.c,

subversion/svndumpfilter/svndumpfilter.c,

subversion/tests/libsvn_subr/dirent_uri-test.c,

subversion/bindings/javahl/native/Path.cpp:

Include private/svn_dirent_uri_private.h and use svn_relpath__make_internal

instead of svn_relpath__internal_style.

* subversion/bindings/swig/python/svn/core.py

(__all__): Remove mention of svn_relpath__internal_style from the comment.

  1. … 6 more files in changeset.
Change a private function's signature to prevent potential assertions.

* subversion/include/svn_dirent_uri.h

(svn_relpath__internal_style): Change prototype so that the function can

return an error instead of aborting if anything goes wrong.

* subversion/libsvn_subr/dirent_uri.c

(svn_relpath__internal_style): Update the implementation.

* subversion/bindings/javahl/native/Path.cpp (Relpath::initfunc),

* subversion/svndumpfilter/svndumpfilter.c (sub_main),

* subversion/tests/libsvn_subr/dirent_uri-test.c (test_relpath_internal_style):

Updated all callers.

  1. … 4 more files in changeset.
* subversion/svndumpfilter/svndumpfilter.c

(new_node_record):

Include the target node's path in two error message, to make it

easier to identify.

Factor out the dump-stream header writing functions.

Although these are small functions, consistency is nice.

* subversion/include/private/svn_repos_private.h,

subversion/libsvn_repos/dump.c

(svn_repos__dump_magic_header_record,

svn_repos__dump_uuid_header_record): New.

(svn_repos_dump_fs4): Use them.

* subversion/svndumpfilter/svndumpfilter.c

(magic_header_record,

uuid_record): Use them.

* subversion/svnrdump/svnrdump.c

(replay_revisions): Use them.

  1. … 3 more files in changeset.
Allow splitting long help strings into paragraphs to ease translation.

Issue SVN-4724.

This patch revises the help string definition format that is used in 'svn'

and most of the other command-line tools to svn_opt_subcommand_desc3_t

(introduced in r1826328).

This patch does not split each long string into paragrpahs, it only adds the

mechanism for doing so.

  1. … 16 more files in changeset.
Tweak the help and add Bash completion for 'svnadmin dump --include/exclude'.

A follow-up to r1811992.

* CHANGES

Note this enhancement.

* subversion/svnadmin/svnadmin.c

(options_table): Document the '--pattern' glob syntax, like r1783741 did

for svndumpfilter.

(cmd_table): Say more about exclude/include in the help for 'dump'.

(ary_prefix_match): Note that this...

* subversion/svndumpfilter/svndumpfilter.c

(ary_prefix_match): ... and this are duplicates.

* tools/client-side/bash_completion

(_svnadmin): Add the --exclude, --include and --pattern options.

  1. … 3 more files in changeset.
Use more robust coding style in handling UTF-8 conversion of command-line

arguments.

Convert to UTF-8 as soon as possible, and name variables assuming UTF-8

encoding is the default rather than an exception, and minimize visibility of

non-UTF-8 values.

* subversion/svn/propdel-cmd.c,

subversion/svn/propedit-cmd.c,

subversion/svn/propget-cmd.c,

subversion/svn/propset-cmd.c:

Rename variables assuming UTF-8; minimize storing non-UTF-8 values.

* subversion/svn/svn.c,

subversion/svnadmin/svnadmin.c,

subversion/svnbench/svnbench.c,

subversion/svndumpfilter/svndumpfilter.c,

subversion/svnfsfs/svnfsfs.c,

subversion/svnlook/svnlook.c,

subversion/svnmucc/svnmucc.c,

subversion/svnrdump/svnrdump.c,

subversion/svnsync/svnsync.c

tools/client-side/svnconflict/svnconflict.c

tools/client-side/svn-mergeinfo-normalizer/svn-mergeinfo-normalizer.c

tools/dev/svnmover/svnmover.c

tools/server-side/svnauthz.c

(sub_main): Convert to UTF-8 before matching the subcommand name; no

observable change if environment's encoding is a superset of ASCII.

(...): Rename variables assuming UTF-8; minimize storing non-UTF-8 values.

  1. … 16 more files in changeset.
Document briefly the glob syntax for 'svndumpfilter exclude --pattern'

and 'svndumpfilter include --pattern'.

* subversion/svndumpfilter/svndumpfilter.c

(options_table): Expand the help for '--pattern'.

Following up on r1702983, enable APR buffering for STDIN in svnrdump load

and in svndumpfilter.

Related discussion is in http://svn.haxx.se/dev/archive-2015-09/0092.shtml

(Subject: "Re: svn commit: r1700799 - ...").

* subversion/svndumpfilter/svndumpfilter.c

(parse_baton_initialize): As above.

* subversion/svnrdump/svnrdump.c

(load_revisions): As above.

  1. … 1 more file in changeset.
Use the standard way of initializing STDIN and STDOUT streams in svnrdump

and svndumpfilter with svn_stream_for_stdin2() and svn_stream_for_stdout()

functions respectively.

* subversion/svndumpfilter/svndumpfilter.c

(open_fn_t): Remove this typedef.

(create_stdio_stream): Remove this helper.

(parse_baton_initialize): Use the functions from stream API to open the

standard input and output streams.

* subversion/svnrdump/svnrdump.c

(load_revisions): Use svn_stream_for_stdin2() to open the standard input.

  1. … 1 more file in changeset.
Follow up on r1696166: More sizeof changes when using APR hashes.

* subversion/svndumpfilter/svndumpfilter.c

(new_node_record,

adjust_mergeinfo): Take the key size from the key variable.

* subversion/svnrdump/load_editor.c

(set_revision_mapping): Same.

* subversion/tests/libsvn_repos/repos-test.c

(check_locations_info,

test_get_file_revs): Same.

  1. … 2 more files in changeset.
Make some private dumpfile APIs more opaque.

* subversion/include/private/svn_repos_private.h,

subversion/libsvn_repos/dump.c

(svn_repos__dumpfile_header_entry_t): Move from the header to the

implementation.

(svn_repos__dumpfile_headers_t): New.

(svn_repos__dumpfile_header_push,

svn_repos__dumpfile_header_pushf,

svn_repos__dump_headers,

svn_repos__dump_node_record): Use svn_repos__dumpfile_headers_t instead

of apr_array_header_t. Remove the redundant 'terminate' parameter from

svn_repos__dump_headers. Update callers.

* subversion/svndumpfilter/svndumpfilter.c

(node_baton_t): Update to use svn_repos__dumpfile_headers_t instead

of apr_array_header_t.

* subversion/svnrdump/dump_editor.c

(dir_baton,

get_props_content,

dump_node_delete,

dump_node,

dump_mkdir,

close_file): Update to use svn_repos__dumpfile_headers_t instead

of apr_array_header_t.

  1. … 3 more files in changeset.
Factor out writing a dumpfile node record, from 'svnadmin dump', 'svnrdump

dump' and 'svndumpfilter'.

[Note from the future: this changed the order of headers in a node record.

Prop-content-length and Text-content-length now come last, just before

Content-length. The test suite was adjusted to accommodate this in r1655064.]

* subversion/include/private/svn_repos_private.h

(svn_repos__dump_node_record): New function.

* subversion/libsvn_repos/dump.c

(svn_repos__dump_node_record): New, moved from output_node_record in

svndumpfilter.c. Make the content-length header optional when there are

no properties or text, so we can preserve exact behaviour.

(dump_node): Use it.

* subversion/svndumpfilter/svndumpfilter.c

(output_node_record): Delete; move to svn_repos__dump_node_record.

(set_fulltext,

close_node): Adjust callers.

* subversion/svnrdump/dump_editor.c

(dir_baton): Add a 'headers' field in which to accumulate headers.

(make_dir_baton): Initialize the 'headers' field.

(get_props_content): Don't add a Prop-content-length header here.

(dump_node): Return headers instead of writing them.

(dump_mkdir,

dump_pending_dir): Simplify by using svn_repos__dump_node_record().

(open_root,

add_directory): Adjust a call to dump_node() to store the headers in the

node baton instead of writing them out straight away.

(close_directory): The same, and instead of adjusting the second dump_node

call and then dumping the resulting headers, use dump_node_delete().

(close_file): Adjust the call to dump_node() and use the headers returned

from it as the starting point for creating the rest of the headers.

  1. … 3 more files in changeset.
More work on deduplicating dump/load code. This time, the target is the

functionality of formatting and writing headers, rather than significant

amounts of code. Factor this out of svnadmin, svnrdump and svndumpfilter.

This starts to make the writing of headers a batch action rather than a

series of sporadic writes to the stream from various code paths. In turn,

this paves the way for further refactoring.

No functional change.

* subversion/include/private/svn_repos_private.h,

subversion/libsvn_repos/dump.c

(svn_repos__dumpfile_headers_create,

svn_repos__dumpfile_header_push,

svn_repos__dumpfile_header_pushf,

svn_repos__dump_headers): New functions.

(dump_node): Use them instead of writing directly to the stream.

* subversion/svndumpfilter/svndumpfilter.c

(node_baton_t): Change the type of the headers accumulator. Add a per-node

pool.

(new_node_record): Initialize the new fields. Use the new functions

instead of writing directly to the stream. Don't write the headers to

the stream here; instead leave them in the node baton for writing later.

(output_node_record): Rename from 'output_node'. Take separate parameters

instead of the node baton. Use the new functions instead of writing

directly to the stream.

(set_fulltext,

close_node): Track the changes to output_node.

* subversion/svnrdump/dump_editor.c

(get_props_content

dump_node

dump_mkdir

dump_pending

close_file): Use the new style for writing headers.

  1. … 3 more files in changeset.
Fix a pool usage problem I introduced in r1652987.

* subversion/svndumpfilter/svndumpfilter.c

(headers_dup): New.

(new_revision_record): Duplicate the incoming headers before storing them.

Factor out writing a dumpfile revision record from 'svnadmin load' and

'svnrdump load' and 'svndumpfilter'.

No functional change.

* subversion/private/svn_repos_private.h

(svn_repos__dump_revision_record): New function.

* subversion/libsvn_repos/dump.c

(write_header,

write_revision_headers,

svn_repos__dump_revision_record): New.

(write_revision_record): Don't serialize the headers and props here, just

pass them to svn_repos__dump_revision_record().

* subversion/svndumpfilter/svndumpfilter.c

(revision_baton_t): Store the original headers in the baton. Remove an

unneeded flag.

(new_revision_record): Don't filter and serialize the headers here, just

store them as a hash.

(output_revision): Don't serialize the headers and props here, just pass

them to svn_repos__dump_revision_record().

(set_revision_property): Remove initialization of the unneeded flag.

* subversion/svnrdump/svnrdump.c

(replay_revstart,

replay_revstart_v2,

dump_revision_header): Don't serialize the headers and props here, just

pass them to svn_repos__dump_revision_record().

  1. … 3 more files in changeset.
* subversion/svndumpfilter/svndumpfilter.c

(set_node_property,

remove_node_props): Clarify comments related to the 'has_props' flag.

Fix issue #4538 "'load' strips r1 references in mergeinfo": don't filter out

references to r1 in mergeinfo during 'svnadmin load' and 'svnrdump load'.

r1 is not an invalid mergeinfo source revision, merely an unlikely one, and

we don't treat it as invalid anywhere else.

This reverts r941438 which has been in place since v1.7.0. Note also that

this commit didn't remove r0 mergeinfo, only r1, despite the claim in its

log message.

* subversion/libsvn_repos/load-fs-vtable.c,

subversion/svnrdump/load_editor.c

(renumber_mergeinfo_revs): Don't filter out references to r1.

Canonicalize mergeinfo, where previously the combination of sorting

followed by filtering achieved canonicalization.

* subversion/svndumpfilter/svndumpfilter.c

(adjust_mergeinfo): Remove a comment that said r1 is invalid.

* subversion/tests/cmdline/svndumpfilter_tests.py

(filter_mergeinfo_revs_outside_of_dump_stream): Correct a comment that

said r1 is invalid, written in r941408.

  1. … 3 more files in changeset.
Introduce mergeinfo rangelist canonicalization functions.

After editing rangelists in mergeinfo, the rangelists need to be returned to

their canonical representation. The mergeinfo documentation states only that

the ranges must be sorted, but adjacent ranges usually should be combined

and at least one test assumes that.

In 'svndumpfilter.c' we were just sorting them without combining adjacent

ranges. (It is probable but not proven that the preceding code can in fact

produce combinable ranges.) In the 'renumber_mergeinfo_revs' functions in

libsvn_repos/load-fs-vtable.c and svnrdump/load_editor.c, we are currently

relying on the following 'filter_mergeinfo_by_ranges' call to combine

adjacent ranges. I plan to get rid of that filtering operation later.

* subversion/include/private/svn_mergeinfo_private.h,

* subversion/libsvn_subr/mergeinfo.c

(svn_rangelist__canonicalize,

svn_mergeinfo__canonicalize_ranges): New functions.

(is_rangelist_normalized): Note the possibility of a false negative result.

(parse_revision_line): Simplify by using svn_rangelist__canonicalize.

* subversion/svndumpfilter/svndumpfilter.c

(adjust_mergeinfo): After adjusting mergeinfo, don't just sort it,

canonicalize it.

  1. … 2 more files in changeset.
* subversion/svndumpfilter/svndumpfilter.c

(adjust_mergeinfo): Add a comment.

Following the availability of apr_hash_this_key() etc. in APR v1.5, use

these in Subversion code instead of svn__apr_hash_index_key() etc. Provide

substitutes only when APR is too old to provide them.

* subversion/include/svn_types.h,

subversion/libsvn_subr/iter.c

(svn__apr_hash_index_key,

svn__apr_hash_index_klen,

svn__apr_hash_index_val):

Rename to apr_hash_this_key, apr_hash_this_key_len, apr_hash_this_val.

Declare and define them only if APR is older than 1.5.

Everywhere else: track the renames.

  1. … 102 more files in changeset.
Have svndumpfilter write Node-path without a leading slash, following

r1578670.

Found by: julianfoad

* subversion/svndumpfilter/svndumpfilter.c

(new_node_record): Skip leading slash.

Followup to r1578670, more ordering of svndumpfilter output to match

documented format.

Reported by: "Brown, Jonathan W" <JonathanW.Brown{_AT_}nuance.com>

* subversion/svndumpfilter/svndumpfilter.c

(new_node_record): Correct order for 'Node-kind' and 'Node-action'.

Fix the order of node record headers written by svndumpfilter.

Reported by: "Brown, Jonathan W" <JonathanW.Brown{_AT_}nuance.com>

* subversion/svndumpfilter/svndumpfilter.c

(new_node_record): Output 'Node-path' first.

Add svn_sort__array() -- simple wrapper around qsort() to sort APR array

and use where applicable.

* subversion/include/private/svn_sorts_private.h

(svn_sort__array): New declaration.

* subversion/libsvn_subr/sorts.c

(svn_sort__array): New.

* subversion/libsvn_client/add.c

(): Include svn_sorts_private.h.

(mkdir_urls): Use svn_sort__array().

* subversion/libsvn_client/commit.c

(): Include svn_sorts_private.h.

(determine_lock_targets): Use svn_sort__array().

* subversion/libsvn_client/commit_util.c

(): Remove stdlib.h and include svn_sorts_private.h.

(svn_client__condense_commit_items): Use svn_sort__array().

* subversion/libsvn_client/ra.c

(): Include svn_sorts_private.h.

(svn_client__repos_location_segments): Use svn_sort__array().

* subversion/libsvn_client/resolved.c

(): Remove stdlib.h and include svn_sorts_private.h.

(svn_client__resolve_conflicts): Use svn_sort__array().

* subversion/libsvn_delta/path_driver.c

(): Include svn_sorts_private.h.

(svn_delta_path_driver2): Use svn_sort__array().

* subversion/libsvn_diff/parse-diff.c

(): Include svn_sorts_private.h.

(svn_diff_parse_next_patch): Use svn_sort__array().

* subversion/libsvn_fs_fs/cached_data.c

(read_dir_entries): Use svn_sort__array().

* subversion/libsvn_fs_fs/transaction.c

(verify_locks): Use svn_sort__array().

* subversion/libsvn_ra/compat.c

(svn_ra__locations_from_log): Use svn_sort__array().

* subversion/libsvn_repos/log.c

(): Include svn_sorts_private.h.

(turn_unique_copies_into_moves, combine_mergeinfo_path_lists,

handle_merged_revisions): Use svn_sort__array().

* subversion/libsvn_repos/replay.c

(): Include svn_sorts_private.h.

(svn_repos__replay_ev2): Use svn_sort__array().

* subversion/libsvn_repos/rev_hunt.c

(): Include svn_sorts_private.h.

(find_merged_revisions): Use svn_sort__array().

* subversion/libsvn_subr/mergeinfo.c

(combine_with_lastrange, parse_revision_line, svn_mergeinfo_sort): Use

svn_sort__array().

* subversion/libsvn_wc/revert.c

(): Include svn_sorts_private.h.

(revert_restore_handle_copied_dirs): Use svn_sort__array().

* subversion/svndumpfilter/svndumpfilter.c

(): Include svn_sorts_private.h.

(do_filter): Use svn_sort__array().

  1. … 17 more files in changeset.
Fix accidentally-global variables.

[ Changes to libsvn_subr and named_atomic-test-common.h reverted in r1571500 ]

* configure.ac

(CMAINTAINERFLAGS): Add -Wmissing-variable-declarations.

* subversion/libsvn_fs_fs/hotcopy.c

(hotcopy_body_baton):

* subversion/libsvn_fs_x/hotcopy.c

(hotcopy_body_baton):

* subversion/libsvn_repos/hooks.c

(parse_hooks_env_option_baton, parse_hooks_env_section_baton):

* subversion/libsvn_wc/wc_db.c

(struct op_delete_many_baton_t):

* subversion/svn/file-merge.c

(struct file_merge_baton):

Remove accidental declaration of global variables.

* subversion/libsvn_subr/ctype.c

(svn_ctype_table_internal):

* subversion/svndumpfilter/svndumpfilter.c

(filtering_vtable):

* subversion/svnserve/svnserve.c

(connection_pools, threads):

* subversion/tests/libsvn_fs_x/string-table-test.c

(basic_strings):

* subversion/tests/libsvn_subr/hashdump-test.c

(proplist, new_proplist, review):

* subversion/tests/libsvn_subr/io-test.c

(test_file_definitions_template):

* subversion/tests/libsvn_subr/named_atomic-test-common.h

(name_namespace, name_namespace1, name_namespace2):

* subversion/tests/libsvn_subr/packed-data-test.c

(sub_records, test_data_:

* subversion/tests/libsvn_subr/prefix-string-test.c

(test_cases):

* subversion/tests/libsvn_subr/priority-queue-test.c

(numbers, expected_modified):

* subversion/tests/libsvn_subr/string-test.c

(phrase_1, phrase_2):

* subversion/tests/libsvn_subr/time-test.c

(test_timestamp, test_timestring, test_old_timestring):

* subversion/tests/libsvn_subr/translate-test.c

(base_origin_subtests):

* subversion/tests/svn_test_main.c

(mode_filter):

Make these variables file-static.

* subversion/tests/svn_test_main.c

(test_argc, test_argv):

* tools/server-side/mod_dontdothat/mod_dontdothat.c

(dontdothat_module):

Make these variables truly global by also declaring them extern.

  1. … 21 more files in changeset.
In all of the command-line programs, flush stdout before exiting, in order

to report consistently if any information written to standard output is

being lost.

Standard output would be flushed on exit anyway, but this makes sure that

output is not silently lost if it fails. We will get both an error message

on stderr and a non-zero exit code.

See issue #780 "Check printf return values" and r1499470 "Also flush stdout

when returning an error".

A flush was present in some of the programs already, but was not guaranteed

to be executed in all cases, and only since r1499470 has an error-returning

code path been subject to a flush. This is now consistent across all the

programs, whether exiting successfully or with an error, except for a few

irregular exit paths that still remain, such as some calls to exit().

* subversion/svnadmin/svnadmin.c,

subversion/svnauth/svnauth.c,

subversion/svndumpfilter/svndumpfilter.c,

subversion/svnlook/svnlook.c,

subversion/svnmucc/svnmucc.c,

subversion/svnrdump/svnrdump.c,

subversion/svnserve/svnserve.c,

subversion/svn/svn.c,

subversion/svnsync/svnsync.c,

subversion/svnversion/svnversion.c

(sub_main): Remove the flush on the main successful code path for

svnadmin, svnauth, svndumpfilter, svnlook, svnmucc, svnversion; on the

main successful and main failure code paths for 'svn'.

(main): Flush here, instead, and report both the main error (if any) and

any flush error.

  1. … 9 more files in changeset.
Factor out the error handling and pool initialization and tear-down in the

main() functions of the rest of our command-line programs. Create and

destroy the pool exactly once, in the outer function, instead of destroying

it at every return statement -- where, inevitably, some cases were missed.

This is mainly a source code improvement. The only visible effect AFAIK is

that errors will always have the correct prefix; previously, some errors

from 'svnlook' and 'svnserve', for example, had an 'svn:' prefix.

Equivalent changes to 'svn', 'svnadmin', 'svnauth' and 'svnrdump' began with

r1368252 and the most recent was r1543834.

* subversion/svndumpfilter/svndumpfilter.c,

subversion/svnlook/svnlook.c,

subversion/svnserve/svnserve.c,

subversion/svnsync/svnsync.c,

subversion/svnversion/svnversion.c

(sub_main): New, extracted from main(). Don't do tear-down actions --

specifically, destroying the pool -- before returning. Simplify error

handling by using the standard ways such as SVN_ERR().

(main): Just do initilization and tear-down around a call to sub_main().

* subversion/svnmucc/svnmucc.c

The same, plus:

(init): Deleted.

(check_lib_versions): New, extracted from init(), for consistency with the

other programs.

(sub_main): Call check_lib_versions() instead of init().

  1. … 5 more files in changeset.
Introduce a null-pointer constant that we can use as a termination sentinel

in variable argument lists. This avoids the ugliness of writing

"(char *) NULL" all over the place.

* subversion/include/svn_types.h (SVN_VA_NULL): New symbol.

* subversion/include/svn_dirent_uri.h (svn_dirent_join_many),

subversion/include/svn_path.h (svn_path_join_many),

subversion/include/svn_xml.h (svn_xml_make_open_tag):

Update docstrings, mentioning SVN_VA_NULL as the argument list terminator.

[Lots of implementation files]: Replace typecast-NULL with SVN_VA_NULL in

function calls with variable argument lists.

  1. … 89 more files in changeset.
Make all commands provide a brief description of themselves in help output,

and consistently tell users where to find the program version.

Undo a change made back in 2004 (r851470) to make 'svn' show its version

number in help output. That behaviour was not carried over to other commands

and hence inconsistent. Since r860375 the --version option is prominently

documented in the main help text of many commands, and as of this commit

that is done consistently across all commands.

* subversion/svn/help-cmd.c

(svn_cl__help): Don't show the version number in help output, instructing

people to use the --version option is enough.

* subversion/svnadmin/svnadmin.c,

subversion/svnauth/svnauth.c,

subversion/svndumpfilter/svndumpfilter.c,

(subcommand_help): Describe program's purpose in help output.

* subversion/svnlook/svnlook.c:

(subcommand_help): Describe program's purpose in help output and mention

the --version option in the main help text.

* subversion/svnmucc/svnmucc.c

(usage): Describe program's purpose in help output and mention the

--version option in the main help text.

* subversion/svnrdump/svnrdump.c,

subversion/svnsync/svnsync.c:

(help_cmd): Describe program's purpose in help output.

* subversion/svnserve/svnserve.c,

subversion/svnversion/svnversion.c:

(help): Describe program's purpose in help output and mention the

--version option in the main help text.

* subversion/tests/cmdline/getopt_tests.py

(rep_lines_res): No more need to substititute version number in 'svn help'.

* subversion/tests/cmdline/getopt_tests_data/svn--help_stdout,

subversion/tests/cmdline/getopt_tests_data/svn_help_stdout:

Adjust expected output.

  1. … 12 more files in changeset.