Checkout Tools
  • last updated 3 hours ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates

Changeset 1834943 is being indexed.

* subversion/libsvn_subr/io.c

(svn_io_dir_walk2): Correct an old comment.

Since on Windows Subversion does not handle symlinks, never check for reparse points.

* subversion/libsvn_subr/io.c

(io_check_path): ignore the 'resolve_symlinks' flag on Windows via #ifdef

Update issue tracker links in comments, from Tigris (issuezilla) to Apache (Jira).

URL fragment identifiers like '#desc5' are left in place, not yet updated.

This is a merge of r1828508 from the 'shelve-checkpoint' branch where I

committed it by mistake.

  1. … 43 more files in changeset.
Rename and move the new-for-1.10 function 'svn_io_stdin_readline()' to

'svn_cmdline__stdin_readline()'.

It is just a wrapper around existing API functions and lacks some checking and

options so was deemed not suitable to go into the public API in its current

form.

* subversion/include/private/svn_cmdline_private.h,

subversion/libsvn_subr/cmdline.c

(svn_cmdline__stdin_readline): Rename and move to here...

* subversion/include/svn_io.h

* subversion/libsvn_subr/io.c

(svn_io_stdin_readline): ... from here.

* subversion/svnbench/svnbench.c,

subversion/svnmucc/svnmucc.c,

subversion/svnrdump/svnrdump.c,

subversion/svn/svn.c,

tools/client-side/svnconflict/svnconflict.c,

tools/client-side/svn-mergeinfo-normalizer/svn-mergeinfo-normalizer.c,

tools/dev/svnmover/svnmover.c

(sub_main): Track the rename.

  1. … 10 more files in changeset.
Introduce a new global option: --password-from-stdin

This new option makes Subversion's command line applications read a password

from standard input. It can be used as an alternative to the --password option

in order to provide passwords without leaking them to argv, which on Unix-like

systems can be viewed by anyone with tools such as ps(1).

Patch by: William Orr <will@worrbase.com>

* subversion/include/svn_io.h

(svn_io_stdin_readline): Declare.

* subversion/libsvn_subr/io.c

(svn_io_stdin_readline): New public API which reads a line of input from

standard input.

* subversion/svn/cl.h

(svn_cl__opt_state_t): Declare auth_password_from_stdin option.

* subversion/svn/svn.c,

subversion/svnbench/svnbench.c

(svn_cl__longopt_t): Add opt_auth_password_from_stdin.

(svn_cl__options): Add --password-from-stdin.

(svn_cl__global_options): Add opt_auth_password_from_stdin.

(sub_main): Handle the new option.

* subversion/svnmucc/svnmucc.c

(help): Add --password-from-stdin option to help text.

(sub_main): Handle the new option.

* subversion/svnrdump/svnrdump.c

(svn_svnrdump__longopt_t): Add opt_auth_password_from_stdin.

(svnrdump__options, opt_baton_t): Add --password-from-stdin and --dumpfile.

(opt_baton_t, replay_revisions, dump_cmp, load_cmd, sub_main): Add support

for the new options. Since svnrdump has historically been reading dump file

data from stdin the --password-from-stdin option requires --dumpfile.

* subversion/tests/cmdline/getopt_tests_data/svn_help_log_switch_stdout:

Adjust exected output.

* tools/client-side/svn-mergeinfo-normalizer/svn-mergeinfo-normalizer.c

(svn_min__longopt_t, svn_min__options): Add opt_auth_password_from_stdin.

(svn_min__options): Add --password-from-stdin.

(sub_main): Support the new options.

* tools/client-side/svnconflict/svnconflict.c

(svnconflict_cmd_baton_t, global_options): Add opt_auth_password_from_stdin.

(svnconflict_options): Add --password-from-stdin.

(sub_main): Support the new options.

  1. … 10 more files in changeset.
Fix a potential bug and an API contract violation in the Win32 implementation

of svn_io_file_write_full().

Even in the case when a specific workaround for issue #1789 is not required,

the implementation of this function has been calling apr_file_write() that

can do a short write and return APR_SUCCESS. If that happens, the

guarantee of the "write full" function would not be fulfilled in the sense

that the function would write less bytes than requested without an

associated error, and that could cause issues further down the line.

On Windows, apr_file_write() is implemented as a call to WriteFile() that

shouldn't result in short writes for file handles, but such short writes

are still possible, for example, for pipes. That is where the previous

implementation could be working improperly. (See the "Remarks" section

in https://msdn.microsoft.com/en-us/library/windows/desktop/aa365747).

* subversion/libsvn_subr/io.c

(svn_io_file_write_full): Always attempt a full write in the Win32-specific

code path. Join and tweak the comments on issue #1789, add the issue

number and place the comment right before the code that implements a

workaround.

* subversion/libsvn_subr/io.c

(svn_io_file_trunc): Fix typo in a comment.

Resolve two minor Windows specific corectness warnings found by a newer

compiler version.

* subversion/libsvn_subr/io.c

(svn_io__utf8_to_unicode_longpath): Remove unnecessary truncate.

* subversion/libsvn_subr/win32_crashrpt.c

(write_value): Properly print pointersized value.

  1. … 1 more file in changeset.
Add a workaround for yet another issue with APR's apr_file_trunc.

The previous workaround is ineffective if the last file access had been

a read. Now, we force it into to "write mode" internally to have the

existing workaround kick in.

Luckily, this only affects 'svnadmin pack' for FSFS format 7 and FSX.

The other functions using trunc should have no problem with the added

overhead.

* subversion/libsvn_subr/io.c

(svn_io_file_trunc): Admend the existing workaround with a dummy-write.

* subversion/tests/libsvn_subr/io-test.c

(test_apr_trunc_workaround): New test demonstrating the problem.

(test_funcs): Register the new test.

  1. … 1 more file in changeset.
Follow-up to r1755486: Rename svn_stream_checksum() to

svn_stream_contents_checksum().

Suggested by: danielsh

* subversion/include/svn_io.h

* subversion/libsvn_subr/stream.c

(svn_stream_checksum): Rename to svn_stream_contents_checksum().

(compute_stream_checksum): Update docstring.

* subversion/libsvn_fs/fs-loader.c

* subversion/libsvn_subr/io.c

* subversion/tests/libsvn_fs/fs-test.c

* subversion/tests/libsvn_subr/stream-test.c

(svn_fs_file_checksum, svn_io_file_checksum2, get_file_checksum,

test_stream_checksum): Adapt callers.

  1. … 5 more files in changeset.
Use Unicode Windows API.

* subversion/libsvn_subr/io.c

(win_init_dynamic_imports): Use GetModuleHandleW instead of GetModuleHandleA.

* subversion/libsvn_subr/sysinfo.c

(system_info): Use GetModuleHandleW instead of GetModuleHandleA.

(enum_loaded_modules): Use GetModuleHandleW/LoadLibraryW instead of

GetModuleHandleA/LoadLibraryA.

  1. … 1 more file in changeset.
Remove redundant check.

* subversion/libsvn_subr/io.c

(svn_io_remove_file2): Remove check for !apr_err, because we have the same

check few lines below.

Introduce svn_stream_checksum() function to calculate checksum of specified

stream contents. Use new API where it makes sense.

* subversion/include/svn_io.h

(svn_stream_checksum): New.

* subversion/libsvn_subr/stream.c

(compute_stream_checksum): New. Helper for svn_stream_checksum().

(svn_stream_checksum): New.

* subversion/tests/libsvn_subr/stream-test.c

(test_stream_checksum): New. Simple test for svn_stream_checksum().

(test_funcs): Add test_stream_checksum to test list.

* subversion/libsvn_fs/fs-loader.c

* subversion/libsvn_repos/config_pool.c

* subversion/libsvn_subr/io.c

* subversion/tests/libsvn_fs/fs-test.c

(svn_fs_file_checksum, auto_parse, svn_io_file_checksum2,

get_file_checksum): Use svn_stream_checksum() instead of

svn_stream_checksummed2(READ_ALL=TRUE) + svn_stream_close().

  1. … 6 more files in changeset.
Fix a minor inefficiency when generating a "random" file name.

* subversion/libsvn_subr/io.c

(get_default_file_perms): Use more of the entropy that we are given.

Use svn_io_file_get_offset() instead of svn_io_file_seek(APR_CUR) where it

makes sense.

* subversion/libsvn_client/patch.c

* subversion/libsvn_diff/parse-diff.c

* subversion/libsvn_repos/dump.c

* subversion/libsvn_subr/io.c

* subversion/libsvn_subr/stream.c

* subversion/tests/libsvn_subr/io-test.c

(tell_file, read_handler_base85, hunk_readline_original_or_modified,

svn_diff_hunk_readline_diff_text, parse_next_hunk, parse_binary_patch,

svn_diff_parse_next_patch, store_delta, svn_io_file_readline,

mark_handler_apr, test_file_readline, aligned_seek): Use

svn_io_file_get_offset() instead of svn_io_file_seek(0, APR_CUR).

  1. … 5 more files in changeset.
Promote libsvn_fs_fs private helper svn_fs_fs__get_file_offset() to public

libsvn_subr function svn_io_file_get_offset().

* subversion/libsvn_fs_fs/util.c

* subversion/libsvn_fs_fs/util.h

(svn_fs_fs__get_file_offset): Move/rename to ...

* subversion/include/svn_io.h

* subversion/libsvn_subr/io.c

(svn_io_file_get_offset): ... here.

* subversion/libsvn_fs_fs/cached_data.c

* subversion/libsvn_fs_fs/index.c

* subversion/libsvn_fs_fs/pack.c

* subversion/libsvn_fs_fs/transaction.c

* subversion/libsvn_fs_fs/verify.c

(get_file_offset, stream_error_create, copy_item_to_temp, copy_rep_to_temp,

copy_node_to_temp, rep_write_get_baton, rep_write_contents_close,

write_container_rep, write_container_delta_rep, write_final_rev,

write_final_changed_path_info, commit_body, expect_buffer_nul): Replace

calls to svn_fs_fs__get_file_offset() within svn_io_file_get_offset().

  1. … 8 more files in changeset.
Revert r1719188: It seems APR doesn't report proper position for apr_file_t

with ungotten character while our patch parser relies on that.

* subversion/include/svn_io.h

(svn_io_file_ungetc): Revert r1719188.

* subversion/libsvn_subr/io.c

(svn_io_file_ungetc, svn_io_file_readline): Revert r1719188.

* subversion/tests/libsvn_subr/io-test.c

(test_file_readline, test_file_ungetc,

svn_test_descriptor_t): Revert r1719188.

  1. … 2 more files in changeset.
[Reverted in 1719196]

Use existing APR function in implementation of svn_io_file_readline() for

peeking char after we found '\r' instead of save position and seek back.

* subversion/include/svn_io.h

(svn_io_file_ungetc): New.

* subversion/libsvn_subr/io.c

(svn_io_file_ungetc): New. Wrapper around apr_file_ungetc().

(svn_io_file_readline): Use svn_io_file_ungetc() for peeking char after we

found '\r' instead of save position and seek back.

* subversion/tests/libsvn_subr/io-test.c

(test_file_readline): New test for svn_io_file_readline().

(test_file_ungetc): New test for svn_io_file_ungetc().

(svn_test_descriptor_t): Add test_file_readline and test_file_ungetc.

  1. … 2 more files in changeset.
When getting the default file permissions for a file created outside

the system directory create the necessary temporary files in the

given directory. This removes the disk IO to TMPDIR during the first

commit made by mod_dav_svn and svnserve processes, all disk IO now

happens in the repositories.

* subversion/libsvn_subr/io.c

(get_default_file_perms): Create temporary files in given directory.

(merge_default_file_perms): Add directory parameter.

(svn_io_open_unique_file3): Pass directory.

Implement svn_io_write_atomic2() with FLUSH_TO_DISK flag to control whether

wait or not until file is actually written to disk. The old

svn_io_write_atomic() was flushing data to disk unconditionally.

* subversion/include/svn_io.h

(svn_io_write_atomic2): New function declaration.

(svn_io_write_atomic): Deprecate.

* subversion/libsvn_subr/io.c

(svn_io_write_atomic2): Revv from svn_io_write_atomic() Add FLUSH_TO_DISK

parameter and perform flush to disk only if FLUSH_TO_DISK is non-zero.

* subversion/libsvn_subr/deprecated.c

(svn_io_write_atomic): Call svn_io_write_atomic2() with FLUSH_TO_DISK=TRUE.

* subversion/libsvn_fs_fs/fs_fs.c

* subversion/libsvn_fs_fs/transaction.c

* subversion/libsvn_fs_fs/util.c

* subversion/libsvn_fs_x/fs_x.c

* subversion/libsvn_fs_x/revprops.c

* subversion/libsvn_fs_x/util.c

* subversion/libsvn_wc/workqueue.c

* subversion/mod_dav_svn/activity.c

* subversion/tests/libsvn_fs_fs/fs-fs-pack-test.c

* subversion/tests/libsvn_fs_x/fs-x-pack-test.c

* subversion/tests/libsvn_repos/repos-test.c

(*): Use svn_io_write_atomic2() with FLUSH_TO_DISK=TRUE instead of

svn_io_write_atomic().

  1. … 13 more files in changeset.
Fix some typos in comments, mostly of the 's/then/than' variety.

Found by "grep -i '\([a-z]er\|more\|less\) then'".

  1. … 9 more files in changeset.
Explicitly release file locks on Windows, instead of relying on them to be

automatically unlocked by the OS.

Documentation for LockFileEx() [1] states that, although a process doesn't

necessarily have to release its file locks before closing the handle, it

is not recommended, and a better choice is to explicitly unlock them:

[[[

If a process terminates with a portion of a file locked or closes a file

that has outstanding locks, the locks are unlocked by the operating system.

However, the time it takes for the operating system to unlock these locks

depends upon available system resources. Therefore, it is recommended that

your process explicitly unlock all files it has locked when it terminates.

If this is not done, access to these files may be denied if the operating

system has not yet unlocked them.

]]]

This changeset turns our common locking sequences from

CreateFile → LockFile → (...) → CloseFile

into

CreateFile → LockFile → (...) → UnlockFileSingle → CloseFile

I conducted a couple of performance tests with svnadmin load, setrevprop,

simple commits, parallel short-living svnadmin instances doing work — i.e.,

tests where explicit unlocking could behave differently from delegating this

to OS. There is no noticeable difference, at least on my machine. So, we

do this not because it causes visible effects in common scenarios, but just

to follow the recommended practice and reduce the chance of encountering a

hard-to-diagnose problem.

Here is one of the performance tests that I used:

[[[

def lock_performance(sbox):

"lock performance"

sbox.build(create_wc=False)

input_file = sbox.get_tempname()

svntest.main.file_write(input_file, 'New log message')

def setlog(n):

svntest.actions.run_and_verify_svnadmin([], [], 'setrevprop', '-r1',

sbox.repo_dir, 'prop' + str(n),

input_file)

import multiprocessing.dummy

start = time.time()

p = multiprocessing.dummy.Pool()

results = p.map(setlog, range(2000))

p.close()

p.join();

stop = time.time()

logger.info('<TOTAL TIME = %.6f>' % (stop - start))

]]]

[1] https://msdn.microsoft.com/en-us/library/windows/desktop/aa365203

Found by: ivan

* subversion/libsvn_subr/io.c

(file_clear_locks): Always compile this function.

(svn_io_lock_open_file, svn_io_unlock_open_file): Install and remove pool

cleanup handlers on all platforms. Update the corresponding comments.

Do not attempt convert arbitrary apr_status_t to OS error to check it for

OS specific codes.

* subversion/libsvn_subr/io.c

(svn_io_open_uniquely_named, svn_io_dir_remove_nonrecursive,

temp_file_create): Use 'apr_err == APR_FROM_OS_ERROR(os_error)' pattern

instead of 'APR_TO_OS_ERROR(apr_err) == os_error'.

Fix Access Denied errors on checkout/update with working copies stored on

SMBv1 network shares.

Discussion: http://svn.haxx.se/dev/archive-2015-09/0054.shtml

Patch by: rhuijben

me

* subversion/libsvn_subr/io.c

(svn_io__win_rename_open_file): Return SVN_ERR_UNSUPPORTED_FEATURE if

SetFileInformationByHandle() returns ERROR_ACCESS_DENIED. Windows seems

to do not support performing rename operation twice using same file handle

for SMBv1 network shares and return ERROR_ACCESS_DENIED in this without

even sending request to SMB server. The caller will fall back to normal

close + rename in this case.

Remove unused argument from static function. No functional changes

intended.

* subversion/libsvn_subr/io.c

(win32_file_rename): Remove POOL argument.

(svn_io_file_rename2): Adapt calls to win32_file_rename().

Fix spurious 'Access Denied' errors during checkout or commit on Windows

reported on TortoiseSVN users mailing list [1].

The issue can be reproduced locally even though known problems reports are

with working copies stored on network share. The reproduction script is:

1. Checkout working copy with a file 'foo'

2. Run 'for /L %I in (1,1,50000000) do type foo' command in background to

emulate indexers/antivirus scanners

3. Modify file 'foo' in repository via another working copy.

4. Update the original working copy with 'type foo' running

The real fix is to add retry loop for SetFileInformationByHandle() call. All

other changes are refactoring to move existing platform specific code from

libsvn_subr/stream.c to libsvn_subr/io.c to use WIN32_RETRY_LOOP macro. We

already have retry loops for almost all IO operations on Windows.

[1] http://tortoisesvn.tigris.org/ds/viewMessage.do?dsForumId=4061&dsMessageId=3134203

* subversion/include/private/svn_io_private.h

(svn_io__win_delete_file_on_close, svn_io__win_rename_open_file): Declare

new library private functions.

* subversion/libsvn_subr/io.c

(FILE_RENAME_INFO, FILE_DISPOSITION_INFO, FileRenameInfo,

FileDispositionInfo, SetFileInformationByHandle_t,

set_file_information_by_handle_proc): Move it here from

libsvn_subr/stream.c.

(win_init_dynamic_imports): Obtain pointer to SetFileInformationByHandle().

(win32_set_file_information_by_handle): New helper.

(svn_io__win_delete_file_on_close): New. Implementation extracted from

svn_stream__install_delete().

(svn_io__win_rename_open_file): New. Implementation extracted from

svn_stream__install_stream(). Retry operation on Access Denied errors.

* subversion/libsvn_subr/stream.c

(FILE_RENAME_INFO, FILE_DISPOSITION_INFO, SetFileInformationByHandle_t):

Move to libsvn_subr/io.c.

(SetFileInformationByHandle, SetFileInformationByHandle_a,

find_SetFileInformationByHandle): Remove.

(svn_stream__install_stream): Use svn_io__win_rename_open_file().

(svn_stream__install_delete): Use svn_io__win_delete_file_on_close().

  1. … 2 more files in changeset.
* subversion/libsvn_subr/io.c

(win_init_dynamic_imports): Check return value from

GetModuleHandle("kernel32.dll") for NULL even it's almost impossible

situation.

* subversion/libsvn_subr/io.c: Call svn_io_file_rename2() with

FLUSH_TO_DISK=TRUE and remove posix-specific flushing to disk, since

svn_io_file_rename2() will do this for us.

Implement svn_io_file_rename2() with FLUSH_TO_DISK flag to require OS to

wait until rename operation is actually written to disk. Discussed in thread

"svn commit: r1682265 - /subversion/trunk/subversion/libsvn_fs_fs/util.c" on

dev@s.a.o [1]

[1] http://svn.haxx.se/dev/archive-2015-05/0211.shtml

* subversion/include/svn_io.h

(svn_io_file_rename2): New function declaration.

(svn_io_file_rename): Deprecate.

* subversion/libsvn_subr/io.c

(win32_file_rename): Use MOVEFILE_WRITE_THROUGH flag in call to

MoveFileExW if FLUSH_TO_DISK is non-zero.

(svn_io_file_rename2): Revv from svn_io_file_rename(). Add FLUSH_TO_DISK

parameter and perform flush to disk operation depending on the platform:

use MoveFileEx flag on Windows, fsync() target directory on POSIX and

fsync() target file on all other platforms. This logic mostly copied

from svn_fs_fs__move_into_place().

(svn_io_copy_link, svn_io_copy_file, svn_io_write_atomic,

svn_io_write_version_file): Use svn_io_rename2() with

FLUSH_TO_DISK=FALSE instead of svn_io_rename().

* subversion/tests/libsvn_subr/io-test.c

(test_file_rename2): Simple tests for svn_io_file_rename2().

(test_funcs): Add test_file_rename2.

* subversion/libsvn_subr/deprecated.c

(svn_io_file_rename): Call svn_io_file_rename2() with FLUSH_TO_DISK=FALSE.

* subversion/libsvn_client/copy.c

* subversion/libsvn_client/export.c

* subversion/libsvn_client/externals.c

* subversion/libsvn_fs_fs/fs_fs.c

* subversion/libsvn_fs_fs/lock.c

* subversion/libsvn_fs_fs/transaction.c

* subversion/libsvn_fs_fs/util.c

* subversion/libsvn_fs_x/lock.c

* subversion/libsvn_fs_x/transaction.c

* subversion/libsvn_fs_x/util.c

* subversion/libsvn_subr/config_auth.c

* subversion/libsvn_subr/stream.c

* subversion/libsvn_subr/subst.c

* subversion/libsvn_wc/copy.c

* subversion/libsvn_wc/node.c

* subversion/libsvn_wc/upgrade.c

* subversion/libsvn_wc/wc_db_pristine.c

* subversion/libsvn_wc/workqueue.c

(*): Use svn_io_file_rename2() with FLUSH_TO_DISK=FALSE instead of

svn_io_file_rename().

  1. … 21 more files in changeset.
Follow-up to r1687583: Fix svn_io_file_rename() on OS/2.

* subversion/libsvn_subr/io.c

(svn_io_file_rename): Restore call to apr_file_rename() on OS/2 like it

was before r1687583.