Checkout Tools
  • last updated 5 hours ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates

Changeset 1794523 is being indexed.

* subversion/libsvn_subr/io.c

(svn_io_file_trunc): Fix typo in a comment.

Resolve two minor Windows specific corectness warnings found by a newer

compiler version.

* subversion/libsvn_subr/io.c

(svn_io__utf8_to_unicode_longpath): Remove unnecessary truncate.

* subversion/libsvn_subr/win32_crashrpt.c

(write_value): Properly print pointersized value.

  1. … 1 more file in changeset.
Add a workaround for yet another issue with APR's apr_file_trunc.

The previous workaround is ineffective if the last file access had been

a read. Now, we force it into to "write mode" internally to have the

existing workaround kick in.

Luckily, this only affects 'svnadmin pack' for FSFS format 7 and FSX.

The other functions using trunc should have no problem with the added

overhead.

* subversion/libsvn_subr/io.c

(svn_io_file_trunc): Admend the existing workaround with a dummy-write.

* subversion/tests/libsvn_subr/io-test.c

(test_apr_trunc_workaround): New test demonstrating the problem.

(test_funcs): Register the new test.

  1. … 1 more file in changeset.
Follow-up to r1755486: Rename svn_stream_checksum() to

svn_stream_contents_checksum().

Suggested by: danielsh

* subversion/include/svn_io.h

* subversion/libsvn_subr/stream.c

(svn_stream_checksum): Rename to svn_stream_contents_checksum().

(compute_stream_checksum): Update docstring.

* subversion/libsvn_fs/fs-loader.c

* subversion/libsvn_subr/io.c

* subversion/tests/libsvn_fs/fs-test.c

* subversion/tests/libsvn_subr/stream-test.c

(svn_fs_file_checksum, svn_io_file_checksum2, get_file_checksum,

test_stream_checksum): Adapt callers.

  1. … 5 more files in changeset.
Use Unicode Windows API.

* subversion/libsvn_subr/io.c

(win_init_dynamic_imports): Use GetModuleHandleW instead of GetModuleHandleA.

* subversion/libsvn_subr/sysinfo.c

(system_info): Use GetModuleHandleW instead of GetModuleHandleA.

(enum_loaded_modules): Use GetModuleHandleW/LoadLibraryW instead of

GetModuleHandleA/LoadLibraryA.

  1. … 1 more file in changeset.
Remove redundant check.

* subversion/libsvn_subr/io.c

(svn_io_remove_file2): Remove check for !apr_err, because we have the same

check few lines below.

Introduce svn_stream_checksum() function to calculate checksum of specified

stream contents. Use new API where it makes sense.

* subversion/include/svn_io.h

(svn_stream_checksum): New.

* subversion/libsvn_subr/stream.c

(compute_stream_checksum): New. Helper for svn_stream_checksum().

(svn_stream_checksum): New.

* subversion/tests/libsvn_subr/stream-test.c

(test_stream_checksum): New. Simple test for svn_stream_checksum().

(test_funcs): Add test_stream_checksum to test list.

* subversion/libsvn_fs/fs-loader.c

* subversion/libsvn_repos/config_pool.c

* subversion/libsvn_subr/io.c

* subversion/tests/libsvn_fs/fs-test.c

(svn_fs_file_checksum, auto_parse, svn_io_file_checksum2,

get_file_checksum): Use svn_stream_checksum() instead of

svn_stream_checksummed2(READ_ALL=TRUE) + svn_stream_close().

  1. … 6 more files in changeset.
Fix a minor inefficiency when generating a "random" file name.

* subversion/libsvn_subr/io.c

(get_default_file_perms): Use more of the entropy that we are given.

Use svn_io_file_get_offset() instead of svn_io_file_seek(APR_CUR) where it

makes sense.

* subversion/libsvn_client/patch.c

* subversion/libsvn_diff/parse-diff.c

* subversion/libsvn_repos/dump.c

* subversion/libsvn_subr/io.c

* subversion/libsvn_subr/stream.c

* subversion/tests/libsvn_subr/io-test.c

(tell_file, read_handler_base85, hunk_readline_original_or_modified,

svn_diff_hunk_readline_diff_text, parse_next_hunk, parse_binary_patch,

svn_diff_parse_next_patch, store_delta, svn_io_file_readline,

mark_handler_apr, test_file_readline, aligned_seek): Use

svn_io_file_get_offset() instead of svn_io_file_seek(0, APR_CUR).

  1. … 5 more files in changeset.
Promote libsvn_fs_fs private helper svn_fs_fs__get_file_offset() to public

libsvn_subr function svn_io_file_get_offset().

* subversion/libsvn_fs_fs/util.c

* subversion/libsvn_fs_fs/util.h

(svn_fs_fs__get_file_offset): Move/rename to ...

* subversion/include/svn_io.h

* subversion/libsvn_subr/io.c

(svn_io_file_get_offset): ... here.

* subversion/libsvn_fs_fs/cached_data.c

* subversion/libsvn_fs_fs/index.c

* subversion/libsvn_fs_fs/pack.c

* subversion/libsvn_fs_fs/transaction.c

* subversion/libsvn_fs_fs/verify.c

(get_file_offset, stream_error_create, copy_item_to_temp, copy_rep_to_temp,

copy_node_to_temp, rep_write_get_baton, rep_write_contents_close,

write_container_rep, write_container_delta_rep, write_final_rev,

write_final_changed_path_info, commit_body, expect_buffer_nul): Replace

calls to svn_fs_fs__get_file_offset() within svn_io_file_get_offset().

  1. … 8 more files in changeset.
Revert r1719188: It seems APR doesn't report proper position for apr_file_t

with ungotten character while our patch parser relies on that.

* subversion/include/svn_io.h

(svn_io_file_ungetc): Revert r1719188.

* subversion/libsvn_subr/io.c

(svn_io_file_ungetc, svn_io_file_readline): Revert r1719188.

* subversion/tests/libsvn_subr/io-test.c

(test_file_readline, test_file_ungetc,

svn_test_descriptor_t): Revert r1719188.

  1. … 2 more files in changeset.
[Reverted in 1719196]

Use existing APR function in implementation of svn_io_file_readline() for

peeking char after we found '\r' instead of save position and seek back.

* subversion/include/svn_io.h

(svn_io_file_ungetc): New.

* subversion/libsvn_subr/io.c

(svn_io_file_ungetc): New. Wrapper around apr_file_ungetc().

(svn_io_file_readline): Use svn_io_file_ungetc() for peeking char after we

found '\r' instead of save position and seek back.

* subversion/tests/libsvn_subr/io-test.c

(test_file_readline): New test for svn_io_file_readline().

(test_file_ungetc): New test for svn_io_file_ungetc().

(svn_test_descriptor_t): Add test_file_readline and test_file_ungetc.

  1. … 2 more files in changeset.
When getting the default file permissions for a file created outside

the system directory create the necessary temporary files in the

given directory. This removes the disk IO to TMPDIR during the first

commit made by mod_dav_svn and svnserve processes, all disk IO now

happens in the repositories.

* subversion/libsvn_subr/io.c

(get_default_file_perms): Create temporary files in given directory.

(merge_default_file_perms): Add directory parameter.

(svn_io_open_unique_file3): Pass directory.

Implement svn_io_write_atomic2() with FLUSH_TO_DISK flag to control whether

wait or not until file is actually written to disk. The old

svn_io_write_atomic() was flushing data to disk unconditionally.

* subversion/include/svn_io.h

(svn_io_write_atomic2): New function declaration.

(svn_io_write_atomic): Deprecate.

* subversion/libsvn_subr/io.c

(svn_io_write_atomic2): Revv from svn_io_write_atomic() Add FLUSH_TO_DISK

parameter and perform flush to disk only if FLUSH_TO_DISK is non-zero.

* subversion/libsvn_subr/deprecated.c

(svn_io_write_atomic): Call svn_io_write_atomic2() with FLUSH_TO_DISK=TRUE.

* subversion/libsvn_fs_fs/fs_fs.c

* subversion/libsvn_fs_fs/transaction.c

* subversion/libsvn_fs_fs/util.c

* subversion/libsvn_fs_x/fs_x.c

* subversion/libsvn_fs_x/revprops.c

* subversion/libsvn_fs_x/util.c

* subversion/libsvn_wc/workqueue.c

* subversion/mod_dav_svn/activity.c

* subversion/tests/libsvn_fs_fs/fs-fs-pack-test.c

* subversion/tests/libsvn_fs_x/fs-x-pack-test.c

* subversion/tests/libsvn_repos/repos-test.c

(*): Use svn_io_write_atomic2() with FLUSH_TO_DISK=TRUE instead of

svn_io_write_atomic().

  1. … 13 more files in changeset.
Fix some typos in comments, mostly of the 's/then/than' variety.

Found by "grep -i '\([a-z]er\|more\|less\) then'".

  1. … 9 more files in changeset.
Explicitly release file locks on Windows, instead of relying on them to be

automatically unlocked by the OS.

Documentation for LockFileEx() [1] states that, although a process doesn't

necessarily have to release its file locks before closing the handle, it

is not recommended, and a better choice is to explicitly unlock them:

[[[

If a process terminates with a portion of a file locked or closes a file

that has outstanding locks, the locks are unlocked by the operating system.

However, the time it takes for the operating system to unlock these locks

depends upon available system resources. Therefore, it is recommended that

your process explicitly unlock all files it has locked when it terminates.

If this is not done, access to these files may be denied if the operating

system has not yet unlocked them.

]]]

This changeset turns our common locking sequences from

CreateFile → LockFile → (...) → CloseFile

into

CreateFile → LockFile → (...) → UnlockFileSingle → CloseFile

I conducted a couple of performance tests with svnadmin load, setrevprop,

simple commits, parallel short-living svnadmin instances doing work — i.e.,

tests where explicit unlocking could behave differently from delegating this

to OS. There is no noticeable difference, at least on my machine. So, we

do this not because it causes visible effects in common scenarios, but just

to follow the recommended practice and reduce the chance of encountering a

hard-to-diagnose problem.

Here is one of the performance tests that I used:

[[[

def lock_performance(sbox):

"lock performance"

sbox.build(create_wc=False)

input_file = sbox.get_tempname()

svntest.main.file_write(input_file, 'New log message')

def setlog(n):

svntest.actions.run_and_verify_svnadmin([], [], 'setrevprop', '-r1',

sbox.repo_dir, 'prop' + str(n),

input_file)

import multiprocessing.dummy

start = time.time()

p = multiprocessing.dummy.Pool()

results = p.map(setlog, range(2000))

p.close()

p.join();

stop = time.time()

logger.info('<TOTAL TIME = %.6f>' % (stop - start))

]]]

[1] https://msdn.microsoft.com/en-us/library/windows/desktop/aa365203

Found by: ivan

* subversion/libsvn_subr/io.c

(file_clear_locks): Always compile this function.

(svn_io_lock_open_file, svn_io_unlock_open_file): Install and remove pool

cleanup handlers on all platforms. Update the corresponding comments.

Do not attempt convert arbitrary apr_status_t to OS error to check it for

OS specific codes.

* subversion/libsvn_subr/io.c

(svn_io_open_uniquely_named, svn_io_dir_remove_nonrecursive,

temp_file_create): Use 'apr_err == APR_FROM_OS_ERROR(os_error)' pattern

instead of 'APR_TO_OS_ERROR(apr_err) == os_error'.

Fix Access Denied errors on checkout/update with working copies stored on

SMBv1 network shares.

Discussion: http://svn.haxx.se/dev/archive-2015-09/0054.shtml

Patch by: rhuijben

me

* subversion/libsvn_subr/io.c

(svn_io__win_rename_open_file): Return SVN_ERR_UNSUPPORTED_FEATURE if

SetFileInformationByHandle() returns ERROR_ACCESS_DENIED. Windows seems

to do not support performing rename operation twice using same file handle

for SMBv1 network shares and return ERROR_ACCESS_DENIED in this without

even sending request to SMB server. The caller will fall back to normal

close + rename in this case.

Remove unused argument from static function. No functional changes

intended.

* subversion/libsvn_subr/io.c

(win32_file_rename): Remove POOL argument.

(svn_io_file_rename2): Adapt calls to win32_file_rename().

Fix spurious 'Access Denied' errors during checkout or commit on Windows

reported on TortoiseSVN users mailing list [1].

The issue can be reproduced locally even though known problems reports are

with working copies stored on network share. The reproduction script is:

1. Checkout working copy with a file 'foo'

2. Run 'for /L %I in (1,1,50000000) do type foo' command in background to

emulate indexers/antivirus scanners

3. Modify file 'foo' in repository via another working copy.

4. Update the original working copy with 'type foo' running

The real fix is to add retry loop for SetFileInformationByHandle() call. All

other changes are refactoring to move existing platform specific code from

libsvn_subr/stream.c to libsvn_subr/io.c to use WIN32_RETRY_LOOP macro. We

already have retry loops for almost all IO operations on Windows.

[1] http://tortoisesvn.tigris.org/ds/viewMessage.do?dsForumId=4061&dsMessageId=3134203

* subversion/include/private/svn_io_private.h

(svn_io__win_delete_file_on_close, svn_io__win_rename_open_file): Declare

new library private functions.

* subversion/libsvn_subr/io.c

(FILE_RENAME_INFO, FILE_DISPOSITION_INFO, FileRenameInfo,

FileDispositionInfo, SetFileInformationByHandle_t,

set_file_information_by_handle_proc): Move it here from

libsvn_subr/stream.c.

(win_init_dynamic_imports): Obtain pointer to SetFileInformationByHandle().

(win32_set_file_information_by_handle): New helper.

(svn_io__win_delete_file_on_close): New. Implementation extracted from

svn_stream__install_delete().

(svn_io__win_rename_open_file): New. Implementation extracted from

svn_stream__install_stream(). Retry operation on Access Denied errors.

* subversion/libsvn_subr/stream.c

(FILE_RENAME_INFO, FILE_DISPOSITION_INFO, SetFileInformationByHandle_t):

Move to libsvn_subr/io.c.

(SetFileInformationByHandle, SetFileInformationByHandle_a,

find_SetFileInformationByHandle): Remove.

(svn_stream__install_stream): Use svn_io__win_rename_open_file().

(svn_stream__install_delete): Use svn_io__win_delete_file_on_close().

  1. … 2 more files in changeset.
* subversion/libsvn_subr/io.c

(win_init_dynamic_imports): Check return value from

GetModuleHandle("kernel32.dll") for NULL even it's almost impossible

situation.

* subversion/libsvn_subr/io.c: Call svn_io_file_rename2() with

FLUSH_TO_DISK=TRUE and remove posix-specific flushing to disk, since

svn_io_file_rename2() will do this for us.

Implement svn_io_file_rename2() with FLUSH_TO_DISK flag to require OS to

wait until rename operation is actually written to disk. Discussed in thread

"svn commit: r1682265 - /subversion/trunk/subversion/libsvn_fs_fs/util.c" on

dev@s.a.o [1]

[1] http://svn.haxx.se/dev/archive-2015-05/0211.shtml

* subversion/include/svn_io.h

(svn_io_file_rename2): New function declaration.

(svn_io_file_rename): Deprecate.

* subversion/libsvn_subr/io.c

(win32_file_rename): Use MOVEFILE_WRITE_THROUGH flag in call to

MoveFileExW if FLUSH_TO_DISK is non-zero.

(svn_io_file_rename2): Revv from svn_io_file_rename(). Add FLUSH_TO_DISK

parameter and perform flush to disk operation depending on the platform:

use MoveFileEx flag on Windows, fsync() target directory on POSIX and

fsync() target file on all other platforms. This logic mostly copied

from svn_fs_fs__move_into_place().

(svn_io_copy_link, svn_io_copy_file, svn_io_write_atomic,

svn_io_write_version_file): Use svn_io_rename2() with

FLUSH_TO_DISK=FALSE instead of svn_io_rename().

* subversion/tests/libsvn_subr/io-test.c

(test_file_rename2): Simple tests for svn_io_file_rename2().

(test_funcs): Add test_file_rename2.

* subversion/libsvn_subr/deprecated.c

(svn_io_file_rename): Call svn_io_file_rename2() with FLUSH_TO_DISK=FALSE.

* subversion/libsvn_client/copy.c

* subversion/libsvn_client/export.c

* subversion/libsvn_client/externals.c

* subversion/libsvn_fs_fs/fs_fs.c

* subversion/libsvn_fs_fs/lock.c

* subversion/libsvn_fs_fs/transaction.c

* subversion/libsvn_fs_fs/util.c

* subversion/libsvn_fs_x/lock.c

* subversion/libsvn_fs_x/transaction.c

* subversion/libsvn_fs_x/util.c

* subversion/libsvn_subr/config_auth.c

* subversion/libsvn_subr/stream.c

* subversion/libsvn_subr/subst.c

* subversion/libsvn_wc/copy.c

* subversion/libsvn_wc/node.c

* subversion/libsvn_wc/upgrade.c

* subversion/libsvn_wc/wc_db_pristine.c

* subversion/libsvn_wc/workqueue.c

(*): Use svn_io_file_rename2() with FLUSH_TO_DISK=FALSE instead of

svn_io_file_rename().

  1. … 21 more files in changeset.
Follow-up to r1687583: Fix svn_io_file_rename() on OS/2.

* subversion/libsvn_subr/io.c

(svn_io_file_rename): Restore call to apr_file_rename() on OS/2 like it

was before r1687583.

Where feasible, we want use simple definedness tests with our SVN_*

preprocessor flag macros. So, make SVN_ON_POSIX comply to this policy.

* subversion/include/private/svn_dep_compat.h

(SVN_ON_POSIX): Switch from a value-based preprocessor flag to simple

definedness.

* subversion/libsvn_fs_fs/util.c

(svn_fs_fs__move_into_place): Update SVN_ON_POSIX check.

* subversion/libsvn_fs_x/batch_fsync.c

(svn_fs_x__batch_fsync_new_path): Same.

* subversion/libsvn_subr/io.c

(svn_io_write_atomic): Same.

Suggested by: brane

  1. … 3 more files in changeset.
Add workaround for APR problem that apr_file_rename() performs cross-volume

renames non-atomically on Windows.

* subversion/libsvn_subr/io.c

(win32_file_rename): New. Windows specific re-implementation of

apr_file_rename().

(svn_io_file_rename): Use win32_file_rename() on Windows and keep the

code for other platforms unchanged

* subversion/libsvn_subr/io.c

(svn_io_file_flush_to_disk): Make the error handling consistent to other

functions by including the file name in the

error message.

Remove unnecessary code in svn_io_file_move().

* subversion/libsvn_subr/io.c

(svn_io_file_move): Do not copy to temporary file for cross-volume moves,

because svn_io_copy_file() already performs atomic copy via temporary

file.

Fix our usage of fsync usage on non-Linux POSIX platforms.

They all share the "directory contains the file name" property.

* subversion/include/private/svn_dep_compat.h

(SVN_ON_POSIX): New define.

* subversion/libsvn_fs_fs/util.c

(svn_fs_fs__move_into_place): Always sync the parent directory when

we are on POSIX - not just for Linux.

* subversion/libsvn_subr/io.c

(svn_io_write_atomic): Same.

  1. … 2 more files in changeset.
Use consistent form for cancel callback invocation. No functional changes

intended.

* subversion/libsvn_fs_fs/revprops.c

* subversion/libsvn_fs_x/revprops.c

* subversion/libsvn_subr/io.c

* subversion/libsvn_wc/upgrade.c

(svn_fs_fs__delete_revprops_shard, svn_fs_x__delete_revprops_shard,

svn_io_remove_dir2, svn_wc__wipe_postupgrade): Use "cancel_func(baton)"

form instead of "(*cancel_func)(baton)" like we in all other code.

  1. … 3 more files in changeset.