subversion

Checkout Tools
  • last updated 3 hours ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates

Changeset 1701017 is being indexed.

Implement svn_io_file_rename2() with FLUSH_TO_DISK flag to require OS to

wait until rename operation is actually written to disk. Discussed in thread

"svn commit: r1682265 - /subversion/trunk/subversion/libsvn_fs_fs/util.c" on

dev@s.a.o [1]

[1] http://svn.haxx.se/dev/archive-2015-05/0211.shtml

* subversion/include/svn_io.h

(svn_io_file_rename2): New function declaration.

(svn_io_file_rename): Deprecate.

* subversion/libsvn_subr/io.c

(win32_file_rename): Use MOVEFILE_WRITE_THROUGH flag in call to

MoveFileExW if FLUSH_TO_DISK is non-zero.

(svn_io_file_rename2): Revv from svn_io_file_rename(). Add FLUSH_TO_DISK

parameter and perform flush to disk operation depending on the platform:

use MoveFileEx flag on Windows, fsync() target directory on POSIX and

fsync() target file on all other platforms. This logic mostly copied

from svn_fs_fs__move_into_place().

(svn_io_copy_link, svn_io_copy_file, svn_io_write_atomic,

svn_io_write_version_file): Use svn_io_rename2() with

FLUSH_TO_DISK=FALSE instead of svn_io_rename().

* subversion/tests/libsvn_subr/io-test.c

(test_file_rename2): Simple tests for svn_io_file_rename2().

(test_funcs): Add test_file_rename2.

* subversion/libsvn_subr/deprecated.c

(svn_io_file_rename): Call svn_io_file_rename2() with FLUSH_TO_DISK=FALSE.

* subversion/libsvn_client/copy.c

* subversion/libsvn_client/export.c

* subversion/libsvn_client/externals.c

* subversion/libsvn_fs_fs/fs_fs.c

* subversion/libsvn_fs_fs/lock.c

* subversion/libsvn_fs_fs/transaction.c

* subversion/libsvn_fs_fs/util.c

* subversion/libsvn_fs_x/lock.c

* subversion/libsvn_fs_x/transaction.c

* subversion/libsvn_fs_x/util.c

* subversion/libsvn_subr/config_auth.c

* subversion/libsvn_subr/stream.c

* subversion/libsvn_subr/subst.c

* subversion/libsvn_wc/copy.c

* subversion/libsvn_wc/node.c

* subversion/libsvn_wc/upgrade.c

* subversion/libsvn_wc/wc_db_pristine.c

* subversion/libsvn_wc/workqueue.c

(*): Use svn_io_file_rename2() with FLUSH_TO_DISK=FALSE instead of

svn_io_file_rename().

  1. … 7 more files in changeset.
* STATUS: Remove condition that is completed.
* STATUS: Cast vote on fsfs allocation fix. Approved.
* STATUS: Cast vote. Approve textdelta fix.
Update 1.9.1 release announcement links in news items.
swig: Disable wrappers that aren't working.

This disables svn_fs_change_rev_prop2() and its higher-layer wrappers, which

are currently segfaulting when called from Python (other languages weren't

tested).

Found by: Mark Ziesemer <online{_AT_}mark.ziesemer.com>

* subversion/bindings/swig/include/svn_types.swg

(const svn_string_t *const *old_value_p): New typemap.

* STATUS: Cast vote on swig fix. Approved.
* STATUS: Vote for r1700740 group.

* branches/1.8.x/STATUS: Vote for the r1687304 group.
* branches/1.8.x/STATUS: Approve r1691928.
* STATUS: Nominate r1700740, r1700951.
Following up on r1700740, add missing include.

Patch by: James McCoy <jamessan{_AT_}debian.org>

* subversion/libsvn_auth_kwallet/kwallet.cpp

(includes): Add svn_hash.h.

Update site for the 1.9.1 release.

* publish/download.html: Update the current version.

* publish/index.html,

publish/news.html: Update news.

* subversion/svnserve/serve.c

(set_path): Code formatting fix. No functional change.

* subversion/svnserve/serve.c

(construct_server_baton): Call the internal API to prevent parameter

type conversion issues later on.

Resolve the last API dependency that prevents us from chaning the ra_svn

list item type: the (deprecated) command handler API.

This switches the private API to a new svn_ra_svn__command_handler type

which expects an svn_ra_svn__list_t parameter instead of an APR array.

Then there is the usual translation code between public & private API.

However, since we can't easily translate the callbacks in the API-provided

table, we must allow for both callback types in our new command instance.

Upon invocation we decide which one to call (only one of them will be set).

* subversion/include/private/svn_ra_svn_private.h

(svn_ra_svn__command_handler): New callback type, similar to

svn_ra_svn_command_handler.

(svn_ra_svn__cmd_entry_t): New command structure, similar to

svn_ra_svn_cmd_entry_t.

(svn_ra_svn__handle_commands2): Expect commands of the new type.

* subversion/libsvn_ra_svn/deprecated.c

(svn_ra_svn_handle_commands2): Translate command table.

(svn_ra_svn_handle_commands): To reuse code, implement this in terms of

svn_ra_svn_handle_commands2 now.

* subversion/libsvn_ra_svn/marshal.c

(svn_ra_svn__handle_command): Expect the new command type now.

Translate parameters if we have to call

legacy handlers.

(svn_ra_svn__handle_commands2): Switch to the new command type.

* subversion/svnserve/serve.c

(report_commands,

main_commands): Update command table definition.

(serve_interruptable): Switch to the new command type.

Continue work towards using a specialized list type for ra_svn items.

This introduces a typesafe access macro replacing APR_ARRAY_IDX for

svn_ra_svn__list_t. The generated code is still the same but specifics

of the list data type are now hidden in that new macro.

* subversion/include/private/svn_ra_svn_private.h

(SVN_RA_SVN__LIST_ITEM): Declare new macro.

* subversion/libsvn_ra_svn/client.c

(parse_prop_diffs,

parse_iproplist,

ra_svn_get_dir,

perform_ra_svn_log,

ra_svn_get_locks): Use the new macro to access for svn_ra_svn__list_t

throughout the code.

* subversion/libsvn_ra_svn/cyrus_auth.c

(svn_ra_svn__do_cyrus_auth): Same.

* subversion/libsvn_ra_svn/internal_auth.c

(svn_ra_svn__find_mech): Same.

* subversion/libsvn_ra_svn/marshal.c

(svn_ra_svn__to_public_array,

svn_ra_svn__set_capabilities,

vparse_tuple,

svn_ra_svn__parse_proplist,

svn_ra_svn__handle_failure_status,

svn_ra_svn__read_string,

svn_ra_svn__read_cstring,

svn_ra_svn__read_word,

svn_ra_svn__read_revision,

svn_ra_svn__read_boolean,

svn_ra_svn__read_list): Same.

* subversion/svnserve/serve.c

(add_lock_tokens,

unlock_paths,

construct_server_baton): Same.

Followup to r1700844: fix the build.

* build/ac-macros/swig.m4

(SVN_CHECK_SWIG, SVN_FIND_SWIG):

Only require swig when --with-swig was passed.

Follow-up to r1687583: Fix svn_io_file_rename() on OS/2.

* subversion/libsvn_subr/io.c

(svn_io_file_rename): Restore call to apr_file_rename() on OS/2 like it

was before r1687583.

swig: Error out if --with-swig was passed but swig not found.

* build/ac-macros/swig.m4

(SVN_FIND_SWIG): As above.

Also, change the magic string used by SVN_CHECK_SWIG/SVN_FIND_SWIG to

communicate the "--with-swig passed without value" case.

First step towards using a specialized list type for ra_svn items.

This introduces svn_ra_svn__list_t as a mere alias to apr_array_header_t

and replaces all uses of the latter where they contain svn_ra_svn__item_t.

Exception: the command function definition because that one will require

a specific conversion routine. So, lots of code churn here but no

intended functional change.

After a few more of these formal mirgration steps, we will make the new

list type type-safe and more light-weight than the generic APR array.

* subversion/include/private/svn_ra_svn_private.h

(svn_ra_svn__list_t): Define our new list type.

(svn_ra_svn__item_t,

svn_ra_svn__to_public_array,

svn_ra_svn__to_private_array,

svn_ra_svn__set_capabilities,

svn_ra_svn__parse_tuple,

svn_ra_svn__parse_proplist,

svn_ra_svn__read_data_log_changed_entry)

svn_ra_svn__to_public_item): Switch to the new list type for

protocol items.

* subversion/libsvn_ra_svn/ra_svn.h

(svn_ra_svn__handle_failure_status,

svn_ra_svn__do_cyrus_auth,

svn_ra_svn__do_internal_auth,

svn_ra_svn__find_mech): Same.

* subversion/libsvn_ra_svn/client.c

(make_connection,

parse_prop_diffs,

handle_auth_request,

open_session,

ra_svn_rev_proplist,

parse_iproplist,

ra_svn_get_file,

ra_svn_get_dir,

perform_ra_svn_log,

ra_svn_stat,

ra_svn_lock_compat,

ra_svn_lock,

ra_svn_unlock,

ra_svn_get_lock,

ra_svn_get_locks,

ra_svn_replay_range,

ra_svn_get_inherited_props): Same.

* subversion/libsvn_ra_svn/cyrus_auth.c

(svn_ra_svn__do_cyrus_auth): Same.

* subversion/libsvn_ra_svn/editorp.c

(ra_svn_handle_target_rev,

ra_svn_handle_open_root,

ra_svn_handle_delete_entry,

ra_svn_handle_add_dir,

ra_svn_handle_open_dir,

ra_svn_handle_change_dir_prop,

ra_svn_handle_close_dir,

ra_svn_handle_absent_dir,

ra_svn_handle_add_file,

ra_svn_handle_open_file,

ra_svn_handle_apply_textdelta,

ra_svn_handle_textdelta_chunk,

ra_svn_handle_textdelta_end,

ra_svn_handle_change_file_prop,

ra_svn_handle_close_file,

ra_svn_handle_absent_file,

ra_svn_handle_close_edit,

ra_svn_handle_abort_edit,

ra_svn_handle_finish_replay,

blocked_write,

svn_ra_svn_drive_editor2): Same.

* subversion/libsvn_ra_svn/internal_auth.c

(svn_ra_svn__find_mech,

svn_ra_svn__do_internal_auth): Same.

* subversion/libsvn_ra_svn/marshal.c

(svn_ra_svn__to_public_array,

svn_ra_svn__to_private_array,

svn_ra_svn_set_capabilities,

svn_ra_svn__set_capabilities,

vparse_tuple,

svn_ra_svn__parse_tuple,

svn_ra_svn__parse_proplist,

svn_ra_svn__handle_failure_status,

svn_ra_svn__read_cmd_response,

svn_ra_svn__handle_command,

svn_ra_svn__read_string,

svn_ra_svn__read_cstring,

svn_ra_svn__read_word,

svn_ra_svn__read_revision,

svn_ra_svn__read_boolean,

svn_ra_svn__read_list,

svn_ra_svn__read_check_array_size,

svn_ra_svn__read_data_log_changed_entry): Same.

* subversion/svnserve/serve.c

(set_path,

delete_path,

link_path,

finish_report,

abort_report,

reparent,

get_latest_rev,

do_change_rev_prop,

change_rev_prop2,

change_rev_prop,

rev_proplist,

rev_prop,

add_lock_tokens,

unlock_paths,

commit,

get_file,

get_dir,

update,

switch_cmd,

status,

diff,

get_mergeinfo,

log_cmd,

check_path,

stat_cmd,

get_locations,

get_location_segments,

get_file_revs,

lock,

lock_many,

unlock,

unlock_many,

get_lock,

get_locks,

replay,

replay_range,

get_deleted_rev,

get_inherited_props,

construct_server_baton): Same.

* subversion/libsvn_ra_svn/deprecated.c

(svn_ra_svn_parse_tuple,

svn_ra_svn_parse_proplist): Same. A temporary variable is used to

prevent future type conflicts.

[Combines r1698359 and r170078 into a single commit for better review.]

Introduce a stream wrapper object that adds buffering support to any

readable stream. Use it on the stdin streams in our CL tools.

As it turns out, parsing data from a stdin byte-by-byte incurs a

massive overhead of 100% internal and 300% system load over a buffered

stream. 'svnadmin load-revprops' sees a 5 times speedup if all data

is in OS disc caches. This is a realistic assumption in a "final sync

and switch over to new repository" scenario.

The other 2 uses of stdin either have less data to process (svnfsfs

load-index) or parse only a small fraction of the stream (svnadmin load).

* subversion/include/svn_io.h

(svn_stream_wrap_buffered_read): Declare the new stream constructor API.

* subversion/libsvn_subr/stream.c

(read_handler_buffering_wrapper,

data_available_handler_buffering_wrapper,

is_buffered_handler_buffering_wrapper): Internal logic of the new

stream object.

(svn_stream_wrap_buffered_read): New constructor implementation.

* subversion/svnadmin/svnadmin.c

(subcommand_load_revprops): Wrap the stdin stream.

* subversion/svnfsfs/load-index-cmd.c

(subcommand__load_index): Same.

* subversion/tests/libsvn_subr/stream-test.c

(struct stream_baton_t,

read_handler,

data_available_handler,

create_test_read_stream): New configurable test read stream.

(expect_line_content,

test_stream_buffered_wrapper): New test for the new wrapper stream.

(test_funcs): Register the new test.

Revert r1698359 and r1700789 for rehashing.
[Reverted in r1700797 and re-applied as part of r1700799.]

Remove support for mark & seek from "buffered read" stream wrapper as it

was deemed to difficult to handle. See also here:

http://mail-archives.apache.org/mod_mbox/subversion-dev/201509.mbox/%3CCAP_GPNjwhjD1Ds5%2ByYKiqr2NwzbhmnGq%3DqtB8jowBUJty7_Z3Q%40mail.gmail.com%3E

* subversion/include/svn_io.h

(svn_stream_wrap_buffered_read): Remove all references to mark & seek

other than saying we don't support them.

* subversion/libsvn_subr/stream.c

(buffering_stream_wrapper_baton): Remove all elements used for mark & seek.

(buffering_stream_wrapper_mark): Drop.

(read_handler_buffering_wrapper): Simplify as we always exhaust the buffer

and can completely discard it now.

(decrement_mark_count,

mark_handler_buffering_wrapper,

seek_handler_buffering_wrapper,

assert_zero_mark_count): Drop.

(svn_stream_wrap_buffered_read): Update. Allocate the buffer only one and

do that here.

* subversion/svnadmin/svnadmin.c

(subcommand_load_revprops): Update API caller.

* subversion/svnfsfs/load-index-cmd.c

(subcommand__load_index): Update API caller.

* subversion/tests/libsvn_subr/stream-test.c

(test_stream_buffered_wrapper): Update API caller.

Revert r1700305.
In the kwallet code: Initialize the wallet cache once, instead

of every time we get or save a password. Cache the wallet in the

parameter hash and its pool instead of the single invocation pool.

This patch is in response to an issue

Found by: James McCoy <jamessan{_AT_}debian.org>

* subversion/libsvn_auth_kwallet/kwallet.cpp

(kwallet_terminate): Add forward definition.

(get_wallet): Register pool cleanup handler once, and only when we set

the wallet. Register cleanup on the hash pool.

(kwallet_terminate): Completely cleanup.

(kwallet_password_get,

kwallet_password_set): Don't add another cleanup hook on every invocation.

* tools/buildbot/slaves/svn-x64-macosx/setenv.sh:

Update docs about available environment variables.

* tools/buildbot/slaves/svn-x64-macosx:

Remove the svn:ignore property that's no longer needed here.

* tools/buildbot/slaves/svn-x64-macosx/svnclean.sh,

tools/buildbot/slaves/svn-x64-macosx/svnbuild.sh,

tools/buildbot/slaves/svn-x64-macosx/svnbuild-bindings.sh,

tools/buildbot/slaves/svn-x64-macosx/svncheck.sh,

tools/buildbot/slaves/svn-x64-macosx/svncheck-bindings.sh:

Revert thinko in r1700626 ... the scripts are still (relatively)

in the same directory.

Move the expected location of the svn-x64-macosx buildbot scripts

into a subdirectory, to allow for running other builders on the

same buildslave.

* tools/buildbot/slaves/svn-x64-macosx/setenv.sh

(environment): Skip another directory level.

* tools/buildbot/slaves/svn-x64-macosx/svnclean.sh,

tools/buildbot/slaves/svn-x64-macosx/svnbuild.sh,

tools/buildbot/slaves/svn-x64-macosx/svnbuild-bindings.sh,

tools/buildbot/slaves/svn-x64-macosx/svncheck.sh,

tools/buildbot/slaves/svn-x64-macosx/svncheck-bindings.sh

(scripts): Descend into the .../svnbot directory.

Fix link to buildbot and remove link to hudson.