libsvn_fs_util

Checkout Tools
  • last updated 5 hours ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates

Changeset 1331050 is being indexed.

Store UUID in svn_fs_t rather than in FSAP_DATA. This is a cleaner

implementation of what r1330932 implements in a less robust manner.

Suggested by: gstein

(the particular 'cleaner implementation')

Update libsvn_fs:

* subversion/libsvn_fs/fs-loader.h

(fs_vtable_t.get_uuid): Remove this member, with comment.

(svn_fs_t.uuid): New struct member.

* subversion/libsvn_fs/fs-loader.c

(svn_fs_get_uuid): Track new location of UUID.

Update FSFS for the new location:

* subversion/libsvn_fs_fs/fs_fs.h,

subversion/libsvn_fs_fs/fs_fs.c:

(svn_fs_fs__get_uuid): Delete this function. It is subsumed into

svn_fs_fs__open() and svn_fs_fs__create().

* subversion/libsvn_fs_fs/caching.c

(svn_fs_fs__initialize_caches, svn_fs_fs__initialize_txn_caches):

* subversion/libsvn_fs_fs/fs.c

(fs_serialized_init):

* subversion/libsvn_fs_fs/fs_fs.c

(svn_fs_fs__open, hotcopy_incremental_check_preconditions,

hotcopy_incremental_check_preconditions, svn_fs_fs__set_uuid):

* subversion/libsvn_fs_fs/tree.c

(fs_same_p):

Track new location of UUID.

* subversion/libsvn_fs_fs/fs.c

(fs_vtable): Drop GET_UUID() member.

* subversion/libsvn_fs_fs/fs.h

(fs_fs_data_t.uuid): Remove this member.

Update BDB for the new location:

* subversion/libsvn_fs_base/fs.c

(populate_opened_fs): New function.

(base_create, base_upgrade): Call it to populate svn_fs_t->uuid.

* subversion/libsvn_fs_base/uuid.h

(svn_fs_base__get_uuid): Rename to..

(svn_fs_base__populate_uuid): .. this, and change signature.

* subversion/libsvn_fs_base/uuid.c

(svn_fs_base__get_uuid): Rename to..

(svn_fs_base__populate_uuid): .. this, and adapt to the new signature.

* subversion/libsvn_fs_base/tree.c

(fs_same_p):

* subversion/libsvn_fs_base/uuid.c

(svn_fs_base__set_uuid):

Track new location of UUID.

* subversion/libsvn_fs_base/fs.c

(fs_vtable): Drop GET_UUID() member.

* subversion/libsvn_fs_base/fs.h

(base_fs_data_t.uuid): Remove this member.

Revert r1330932:

* subversion/libsvn_fs/fs-loader.c

(cache_uuid): Remove this function.

(svn_fs_open, svn_fs_create): Update callers.

(svn_fs_open_berkeley, svn_fs_create_berkeley): Update callers.

* subversion/libsvn_fs_util/fs-util.c

(svn_fs__identifier): Remove.

* subversion/include/private/svn_fs_util.h

(svn_fs__identifier): Reimplement as a macro. It will be removed in the

next revision.

  1. … 14 more files in changeset.
Don't leak the repository path to clients.

Suggested by: ivan

* subversion/include/private/svn_fs_util.h,

subversion/libsvn_fs_util/fs-util.c:

(svn_fs__identifier): New function.

* subversion/libsvn_fs/fs-loader.c

(svn_fs_get_uuid): Add cross-refernce.

* subversion/include/private/svn_fs_util.h

(SVN_FS__ALREADY_EXISTS, SVN_FS__ERR_NOT_MUTABLE, SVN_FS__ERR_NOT_FILE,

SVN_FS__ERR_NO_SUCH_LOCK, SVN_FS__ERR_NO_USER):

Log the filesystem UUID instead of diskpath.

  1. … 2 more files in changeset.
Factor out a function.

* subversion/include/private/svn_fs_util.h,

subversion/libsvn_fs_util/fs-util.c

(svn_fs__append_to_merged_froms): New, factored out of libsvn_fs_*/tree.c.

* subversion/libsvn_fs_base/tree.c

(append_to_merged_froms): Removed.

(txn_body_get_mergeinfo_for_path): Adjust caller.

* subversion/libsvn_fs_fs/tree.c

(append_to_merged_froms): Removed.

(get_mergeinfo_for_path): Adjust caller.

  1. … 3 more files in changeset.
Moving membuffer cache related configuration code from deep inside the FS

layer to the svn utilities where all other cache related code lives.

Details on the discussion can be found here:

http://svn.haxx.se/dev/archive-2011-04/0169.shtml

To minimize the risks of breaking builds on other platforms, library

dependencies will be cleaned up in a follow-up commit.

* subversion/include/private/svn_fs_private.h

(svn_fs__get_global_membuffer_cache): remove here

* subversion/include/private/svn_cache.h

(svn_cache__get_global_membuffer_cache): rename and move it to here

* subversion/include/svn_fs.h

(svn_fs_cache_config_t, svn_fs_get_cache_config, svn_fs_set_cache_config):

remove here

* subversion/include/svn_cache_config.h

(svn_cache_config_t, svn_get_cache_config, svn_set_cache_config):

rename and move them to this new header file; adapt docstrings

* subversion/libsvn_fs_util/caching.c

move to:

* subversion/libsvn_subr/svn_cache_config.c

(svn_cache_config_t, svn_get_cache_config, svn_set_cache_config):

rename

* subversion/libsvn_fs_fs/caching.c

(svn_fs_fs__initialize_caches, svn_fs_fs__initialize_txn_caches):

adapt to renamed API functions

* subversion/libsvn_ra/ra_loader.c

(svn_ra_initialize): remove cache config code

* subversion/svn/main.c

(main): move it here and adapt it to renamed API

* subversion/svnadmin/main.c

(#includes): include svn_cache_config.h instead of svn_fs.h

(main): adapt to renamed API functions and structures

* subversion/svnserve/main.c

(#includes, main): dito

* build.conf

(libsvn_subr): add the new svn_cache_config.h header to the DLL exports

  1. … 11 more files in changeset.
LINUX and possibly other OS will over-commit memory. I.e. actual

memory pages allocation will be deferred to the time of the first

access. As a result, the application may simply segfault later.

Therefore, ensure that the membuffer creation actually allocates

memory pages. Also, free the whole cache memory if allocation

of some parts failed. Continue with the cache disabled in that case.

* subversion/libsvn_fs_util/caching.c

(svn_fs__get_global_membuffer_cache): if cache creation fails,

actually free the previously allocated memory

* subversion/libsvn_subr/cache-membuffer.c

(secure_aligned_pcalloc): replacement for apr_pcalloc that will

not crash upon OOM

(svn_cache__membuffer_cache_create): ensure that memory

gets actually allocated despite of overcommitment by the OS;

don't try to continue upon OOM

  1. … 1 more file in changeset.
Explain why the defaults caching settings have been chosen that way.

* subversion/libsvn_fs_util/caching.c

(cache_settings): add extensive commentary to the default values

Followup to r1067687, fix massive memory use.

* subversion/libsvn_fs_util/caching.c

(svn_fs__get_global_membuffer_cache): Store pointer in correct variable.

Fix merge / manual conflict resolution artifacts.

* subversion/libsvn_fs_util/caching.c

(cache_settings): add missing "#endif"

(svn_fs__get_global_membuffer_cache): fix parameters and "volatile" usage

Merged latest caching bug fixes from performance branch:

revisions 1029232, 1032333, 1033040, 1033057 and 1033294

(support for "no threads", typos, error leaks)

  1. … 2 more files in changeset.
Defer cache initialization to first usage: mod_dav_svn may have to

call svn_fs_set_cache_config() more than once if the Apache config

contains multiple caching related parameters (not today, but in the

near future). Also, this reduces the memory footprint if not all server

instances actually access FSFS repositories.

* subversion/libsvn_fs_util/caching.c

(svn_fs__get_global_membuffer_cache): make it thread-safe and

allow for concurrent initialization.

(svn_fs_set_cache_config): don't create caches as a side-effect.

Changing caching defaults to "16MB membuffer cache, fulltext caching on".

* subversion/libsvn_fs_util/caching.c:

(cache_settings): change defaults

Merge all changes (r998649, r998843, r998852) from the

integrate-cache-membuffer branch.

These patches introduce in-process full-text caching.

  1. … 8 more files in changeset.
Fix compiler warning.

* subversion/libsvn_fs_util/fs-util.c

(svn_fs__canonicalize_abspath): Change variable type to size_t instead

of int.

Test out my new and fancy ASF commit priviledges by changing the copyright

wording in our license headers to reflect ownership by the ASF.

* NOTICE:

Change terminology to ASF, and update a link.

* subversion/libsvn_subr/opt.c

(svn_opt__print_version_info): Note that the product as a whole is

copyrighted by the ASF, and update the project website.

* everywhere:

Change license text to reflect ASF ownership.

  1. … 891 more files in changeset.
Update some of the integer types we use internally to remove the "implicit

conversion shortens 64-bit value into a 32-bit value" warnings. Many of these

were due to using integers to store values better characterized as size_t's

or other bitness-agnostic types.

Yay for 64-bit operating systems!

* subversion/libsvn_subr/xml.c,

subversion/libsvn_subr/macos_keychain.c,

subversion/libsvn_ra_local/ra_plugin.c,

subversion/libsvn_ra_svn/cyrus_auth.c,

subversion/libsvn_fs_base/bdb/dbt.c,

subversion/libsvn_fs_base/bdb/dbt.h,

subversion/libsvn_fs_base/bdb/uuids-table.c,

subversion/libsvn_fs_base/bdb/rev-table.c,

subversion/libsvn_fs_util/fs-util.c,

subversion/libsvn_repos/load.c,

subversion/libsvn_repos/log.c,

subversion/libsvn_repos/repos.c,

subversion/libsvn_repos/commit.c,

subversion/libsvn_repos/replay.c,

subversion/libsvn_delta/svndiff.c,

subversion/libsvn_fs_fs/fs_fs.c,

subversion/libsvn_fs_fs/dag.c:

Update various types to be 64 bits wide, where appropriate. Where not

possible (due to external or API requirements) cast.

  1. … 16 more files in changeset.
Relicense Subversion under the Apache License, Version 2.0.

* NOTICE: New.

* LICENSE: New.

* COPYING,

subversion/LICENSE: Remove.

* subversion/libsvn_subr/opt.c

(svn_opt__print_version_info): Note that the product as a whole is

copyrighted by the SVN Corp, and that it contains contributions from

many people, as referenced in NOTICE.

* subversion/bindings/swig/python/LICENSE_FOR_PYTHON_BINDINGS:

Relicense the SVN parts under Apache 2.0.

* everywhere:

Change copyright notices in file headers to reflect the Apache 2.0 license.

  1. … 882 more files in changeset.
Add a few include files to remove some declaration-duplicated warnings.

(I'd still like to know why these warnings are header-file-include-order

dependent. Kinda makes me a bit wary.)

* subversion/libsvn_fs_util/fs-util.c,

subversion/libsvn_fs_fs/tree.c:

Include svn_dirent_uri.h before svn_path.h.

* subversion/include/private/svn_fs_util.h:

Include svn_path.h after svn_dirent_uri.h.

  1. … 2 more files in changeset.
Rename svn_fs__path_change2_create() to svn_fs__path_change_create_internal().

Suggested by: gstein

* subversion/include/private/svn_fs_util.h

(svn_fs__path_change2_create): Rename to ...

(svn_fs__path_change_create_internal): ... this.

* subversion/libsvn_fs_util/fs-util.c

(svn_fs__path_change2_create): Rename to ...

(svn_fs__path_change_create_internal): ... this.

* subversion/libsvn_fs_base/bdb/changes-table.c

(fold_change):

* subversion/libsvn_fs_fs/fs_fs.c

(svn_fs_fs__add_change):

* subversion/libsvn_fs/fs-loader.c

(svn_fs_path_change2_create): Call svn_fs__path_change_create_internal()

instead of svn_fs__path_change2_create().

  1. … 4 more files in changeset.
Fix undefined references to svn_fs_path_change2_create().

* build.conf

(libsvn_fs.libs): Add 'libsvn_fs_util'.

* subversion/include/private/svn_fs_util.h

(svn_fs__path_change2_create): New declaration.

* subversion/libsvn_fs_util/fs-util.c

(svn_fs_path_change2_create): Rename to ...

(svn_fs__path_change2_create): ... this.

* subversion/libsvn_fs/fs-loader.c

(svn_fs_path_change2_create): New function which wraps

svn_fs__path_change2_create().

  1. … 3 more files in changeset.
Cleanup trailing whitespace:

for extsn in c h cpp java py pl rb; do

sed -i -e 's/[ \t]*$//' `find . -name "*.$extsn" | xargs grep '[ \t]$' -l`

done

This should have been done before the 1.6.x branch, so I'm going to merge

it over there, to avoid merge conflicts in the future.

  1. … 83 more files in changeset.
Move svn_fs_path_change2_create to another file to unbreak build.

* subversion/libsvn_fs/util.c: Remove this file.

* subversion/libsvn_fs_util/fs-util.c

(svn_fs_path_change2_create): Function moved here

from libsvn_fs/util.c.

  1. … 1 more file in changeset.
Housekeeping: In the spirit of r27598, remove trailing whitespace in

our repository.

for extsn in c h cpp java py pl rb; do

find . -name "*.$extsn" | xargs perl -pi.bak -e 'next if /^\f$/; s/\s+$/\n/'

done

(This touches files in various and sundry parts of the tree, which I shan't

enumerate here.)

  1. … 175 more files in changeset.
Purge (I think) the source code tree of sqlite stuffs.

* Makefile.in,

* configure.ac,

* gen-make.py,

* build.conf,

* build/win32/make_dist.py,

* build/generator/gen_win.py,

* build/ac-macros/sqlite.m4

Remove sqlite build stuffs.

* subversion/include/private/svn_fs_mergeinfo.h,

* subversion/include/private/svn_fs_sqlite.h,

* subversion/libsvn_fs_util/mergeinfo-sqlite-index.c,

* subversion/libsvn_fs_util/sqlite-util.h,

* subversion/libsvn_fs_util/sqlite-util.c

Remove sqlite-specific source code files.

* subversion/libsvn_fs_base/tree.c,

* subversion/libsvn_fs_base/dag.c,

* subversion/libsvn_fs_fs/tree.c

Stop including now-missing headers.

* subversion/include/svn_error_codes.h

(SVN_ERR_FS_SQLITE_ERROR, SVN_ERR_FS_SQLITE_READONLY): Removed.

(SVN_ERR_FS_NO_SUCH_NODE_ORIGIN): Shift value to fill in a hole.

* subversion/libsvn_fs_fs/fs_fs.c

Stop including now-missing headers.

(svn_fs_fs__hotcopy): Don't copy the sqlite index database.

(commit_body): Don't try to update the sqlite mergeinfo index.

(svn_fs_fs__create): Don't try to create the sqlite index database.

* tools/server-side/svn-populate-node-origins-index.c

(usage_summary): Don't mention the sqlite index file.

* INSTALL

No longer name sqlite as a dependency.

  1. … 16 more files in changeset.
Use a single directory for the FSFS node origin cache instead of a

sqlite database.

Perhaps this should be sharded. (However, my only concern here is

"too many files in one directory", not "too many small files":

there'll only be one file here per file ever made in the repository,

so it'll take up no more space than a checkout of one branch of all

projects.)

(Technically, the node_origins table is still created in

sqlite-util.c, but that whole file is getting the wrecking ball soon.)

* subversion/include/private/svn_fs_node_origins.h

* subversion/libsvn_fs_util/node-origins-sqlite-index.c

Delete.

* subversion/libsvn_fs_fs/fs.h

(PATH_NODE_ORIGINS_DIR): New.

* subversion/libsvn_fs_fs/fs_fs.c

(): Don't include svn_fs_node_origins.h

(path_node_origins): New.

(svn_fs_fs_hotcopy): Copy cache.

(svn_fs_fs__commit): Call FSFS function instead of sqlite function.

(svn_fs_fs__ensure_dir_exists): Moved from ensure_dir_exists in

lock.c.

(svn_fs_fs__get_node_origin, set_node_origin,

svn_fs_fs__set_node_origins, svn_fs_fs__set_node_origin): New.

* subversion/libsvn_fs_fs/fs_fs.h

(svn_fs_fs__get_node_origin, svn_fs_fs__set_node_origins,

svn_fs_fs__set_node_origin): Declare.

* subversion/libsvn_fs_fs/lock.c

(ensure_dir_exists): Move to fs_fs.c and make libsvn_fs_fs-global.

(write_digest_file): Adjust.

* subversion/libsvn_fs_fs/structure

Document node-origins cache.

* subversion/libsvn_fs_fs/tree.c

(): Don't include svn_fs_node_origins.h.

(fs_node_origin_rev): Call FSFS functions instead of sqlite

functions.

  1. … 7 more files in changeset.
Remove no-longer-used svn_*_get_mergeinfo_for_tree APIs.

* subversion/include/private/svn_fs_mergeinfo.h

(svn_fs_mergeinfo__get_mergeinfo): Remove.

* subversion/include/svn_fs.h

* subversion/libsvn_fs/fs-loader.c

(svn_fs_get_mergeinfo_for_tree): Remove.

* subversion/libsvn_fs/fs-loader.h

(struct root_vtable_t): Remove get_mergeinfo_for_tree.

* subversion/libsvn_fs_base/tree.c

(root_vtable): Remove svn_fs_mergeinfo__get_mergeinfo_for_tree.

* subversion/libsvn_fs_fs/tree.c

(collect_mergeinfo_for_tree, get_mergeinfo_hash_for_tree,

fs_get_mergeinfo_for_tree): Remove.

(root_vtable): Remove fs_get_mergeinfo_for_tree.

* subversion/libsvn_fs_util/mergeinfo-sqlite-index.c

(parse_mergeinfo_from_db, get_mergeinfo_for_children,

svn_fs_mergeinfo__get_mergeinfo_for_tree): Remove.

  1. … 6 more files in changeset.
Merge 'reintegrate' branch to trunk.

  1. … 57 more files in changeset.
Fix a compiler warning.

* subversion/libsvn_fs_util/mergeinfo-sqlite-index.c

(get_mergeinfo_for_children): Lose unused 'omit' variable.

Step 3 in removing repos-level checks using mergeinfo created on a copy:

Remove the mergeinfo filter function argument and baton from

svn_fs_get_mergeinfo_for_tree(). The filter was only used in one place, was

a giant hack to begin with, and is now no longer needed.

* subversion/libsvn_repos/log.c

(get_combined_mergeinfo): Remove filter and baton parameters for

svn_fs_get_mergeinfo_for_tree().

* subversion/libsvn_fs_util/mergeinfo-sqlite-index.c

(svn_fs_mergeinfo__get_mergeinfo_for_tree, get_mergeinfo_for_children):

Stop using the filter parameters.

* subversion/include/svn_fs.h,

subversion/libsvn_fs/fs-loader.c

(svn_fs_get_mergeinfo_for_tree): Remove filter and baton parameters.

(svn_fs_mergeinfo_filter_func_t): Remove.

* subversion/include/private/svn_fs_mergeinfo.h

(svn_fs_mergeinfo__get_mergeinfo_for_tree): Same.

* subversion/libsvn_fs/fs-loader.h

(root_vtable_t.get_mergeinfo_for_tree): Same.

  1. … 5 more files in changeset.
Combine two helper functions (one to check that a filesystem is

opened, one to check that it isn't) into a single one that does both

based on a boolean expectation flag.

NOTE: This changes the test when 'expect_open' is set to examine

svn_fs_t->fsap_data instead of svn_fs_t->path (since that's what

is checked when 'expect_open' is not set, too). I think that's

fine, but wanted to note that here anyway just in case it rings

some alarm in another developer's mind.

* subversion/include/private/svn_fs_util.h,

* subversion/libsvn_fs_util/fs-util.c

(svn_fs__check_fs): Add 'expect_open' flag.

* subversion/libsvn_fs_base/fs.c

(check_already_open): Removed as unused.

(base_bdb_set_errcall): Update call to svn_fs__check_fs().

(open_databases): Use svn_fs__check_fs() instead of check_already_open().

* subversion/libsvn_fs_fs/fs.c

(check_already_open): Removed as unused.

(fs_create): Use svn_fs__check_fs() instead of check_already_open().

* subversion/libsvn_fs_base/tree.c,

* subversion/libsvn_fs_base/bdb/rev-table.c,

* subversion/libsvn_fs_base/uuid.c,

* subversion/libsvn_fs_base/revs-txns.c,

* subversion/libsvn_fs_base/lock.c,

* subversion/libsvn_fs_fs/tree.c,

* subversion/libsvn_fs_fs/fs_fs.c,

* subversion/libsvn_fs_fs/lock.c

Update call calls to svn_fs__check_fs(), passing TRUE for 'expect_open'.

  1. … 11 more files in changeset.
Wrap sqlite3_stmt inside a svn_fs__sqlite_stmt_t struct, which also

includes a pointer to the db.

This has two positive results:

- we stop using sqlite3_db_handle, which is relatively new to sqlite

- we introduce a few new helpers for getting values from queries,

which reduces the number of casts we make

* subversion/libsvn_fs_util/mergeinfo-sqlite-index.c

(index_path_mergeinfo, table_has_any_rows_with_rev,

parse_mergeinfo_from_db, get_mergeinfo_for_path,

get_mergeinfo_for_children): Adjust. (Pool usage may

need to be reaudited.)

* subversion/libsvn_fs_util/node-origins-sqlite-index.c

(get_origin, set_origin): Adjust.

* subversion/libsvn_fs_util/sqlite-util.h

* subversion/libsvn_fs_util/sqlite-util.c

(struct svn_fs__sqlite_stmt_t): New.

(svn_fs__sqlite_prepare): Take a pool argument; return a struct

containing the statement and the db.

(step_with_expectation, svn_fs__sqlite_step_done,

svn_fs__sqlite_step_row, svn_fs__sqlite_step,

svn_fs__sqlite_bind_int64, svn_fs__sqlite_bind_text,

svn_fs__sqlite_finalize, svn_fs__sqlite_reset, upgrade_format):

Adjust.

(svn_fs__sqlite_reset): Remove a random declaration.

(svn_fs__sqlite_column_text, svn_fs__sqlite_column_revnum,

svn_fs__sqlite_column_boolean, svn_fs__sqlite_column_int): New.