libsvn_fs_util

Checkout Tools
  • last updated 7 hours ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates

Changeset 1068249 is being indexed.

LINUX and possibly other OS will over-commit memory. I.e. actual

memory pages allocation will be deferred to the time of the first

access. As a result, the application may simply segfault later.

Therefore, ensure that the membuffer creation actually allocates

memory pages. Also, free the whole cache memory if allocation

of some parts failed. Continue with the cache disabled in that case.

* subversion/libsvn_fs_util/caching.c

(svn_fs__get_global_membuffer_cache): if cache creation fails,

actually free the previously allocated memory

* subversion/libsvn_subr/cache-membuffer.c

(secure_aligned_pcalloc): replacement for apr_pcalloc that will

not crash upon OOM

(svn_cache__membuffer_cache_create): ensure that memory

gets actually allocated despite of overcommitment by the OS;

don't try to continue upon OOM

  1. … 1 more file in changeset.
Explain why the defaults caching settings have been chosen that way.

* subversion/libsvn_fs_util/caching.c

(cache_settings): add extensive commentary to the default values

Followup to r1067687, fix massive memory use.

* subversion/libsvn_fs_util/caching.c

(svn_fs__get_global_membuffer_cache): Store pointer in correct variable.

Fix merge / manual conflict resolution artifacts.

* subversion/libsvn_fs_util/caching.c

(cache_settings): add missing "#endif"

(svn_fs__get_global_membuffer_cache): fix parameters and "volatile" usage

Merged latest caching bug fixes from performance branch:

revisions 1029232, 1032333, 1033040, 1033057 and 1033294

(support for "no threads", typos, error leaks)

  1. … 2 more files in changeset.
Defer cache initialization to first usage: mod_dav_svn may have to

call svn_fs_set_cache_config() more than once if the Apache config

contains multiple caching related parameters (not today, but in the

near future). Also, this reduces the memory footprint if not all server

instances actually access FSFS repositories.

* subversion/libsvn_fs_util/caching.c

(svn_fs__get_global_membuffer_cache): make it thread-safe and

allow for concurrent initialization.

(svn_fs_set_cache_config): don't create caches as a side-effect.

Changing caching defaults to "16MB membuffer cache, fulltext caching on".

* subversion/libsvn_fs_util/caching.c:

(cache_settings): change defaults

Merge all changes (r998649, r998843, r998852) from the

integrate-cache-membuffer branch.

These patches introduce in-process full-text caching.

  1. … 8 more files in changeset.
Fix compiler warning.

* subversion/libsvn_fs_util/fs-util.c

(svn_fs__canonicalize_abspath): Change variable type to size_t instead

of int.

Test out my new and fancy ASF commit priviledges by changing the copyright

wording in our license headers to reflect ownership by the ASF.

* NOTICE:

Change terminology to ASF, and update a link.

* subversion/libsvn_subr/opt.c

(svn_opt__print_version_info): Note that the product as a whole is

copyrighted by the ASF, and update the project website.

* everywhere:

Change license text to reflect ASF ownership.

  1. … 891 more files in changeset.
Update some of the integer types we use internally to remove the "implicit

conversion shortens 64-bit value into a 32-bit value" warnings. Many of these

were due to using integers to store values better characterized as size_t's

or other bitness-agnostic types.

Yay for 64-bit operating systems!

* subversion/libsvn_subr/xml.c,

subversion/libsvn_subr/macos_keychain.c,

subversion/libsvn_ra_local/ra_plugin.c,

subversion/libsvn_ra_svn/cyrus_auth.c,

subversion/libsvn_fs_base/bdb/dbt.c,

subversion/libsvn_fs_base/bdb/dbt.h,

subversion/libsvn_fs_base/bdb/uuids-table.c,

subversion/libsvn_fs_base/bdb/rev-table.c,

subversion/libsvn_fs_util/fs-util.c,

subversion/libsvn_repos/load.c,

subversion/libsvn_repos/log.c,

subversion/libsvn_repos/repos.c,

subversion/libsvn_repos/commit.c,

subversion/libsvn_repos/replay.c,

subversion/libsvn_delta/svndiff.c,

subversion/libsvn_fs_fs/fs_fs.c,

subversion/libsvn_fs_fs/dag.c:

Update various types to be 64 bits wide, where appropriate. Where not

possible (due to external or API requirements) cast.

  1. … 16 more files in changeset.
Relicense Subversion under the Apache License, Version 2.0.

* NOTICE: New.

* LICENSE: New.

* COPYING,

subversion/LICENSE: Remove.

* subversion/libsvn_subr/opt.c

(svn_opt__print_version_info): Note that the product as a whole is

copyrighted by the SVN Corp, and that it contains contributions from

many people, as referenced in NOTICE.

* subversion/bindings/swig/python/LICENSE_FOR_PYTHON_BINDINGS:

Relicense the SVN parts under Apache 2.0.

* everywhere:

Change copyright notices in file headers to reflect the Apache 2.0 license.

  1. … 882 more files in changeset.
Add a few include files to remove some declaration-duplicated warnings.

(I'd still like to know why these warnings are header-file-include-order

dependent. Kinda makes me a bit wary.)

* subversion/libsvn_fs_util/fs-util.c,

subversion/libsvn_fs_fs/tree.c:

Include svn_dirent_uri.h before svn_path.h.

* subversion/include/private/svn_fs_util.h:

Include svn_path.h after svn_dirent_uri.h.

  1. … 2 more files in changeset.
Rename svn_fs__path_change2_create() to svn_fs__path_change_create_internal().

Suggested by: gstein

* subversion/include/private/svn_fs_util.h

(svn_fs__path_change2_create): Rename to ...

(svn_fs__path_change_create_internal): ... this.

* subversion/libsvn_fs_util/fs-util.c

(svn_fs__path_change2_create): Rename to ...

(svn_fs__path_change_create_internal): ... this.

* subversion/libsvn_fs_base/bdb/changes-table.c

(fold_change):

* subversion/libsvn_fs_fs/fs_fs.c

(svn_fs_fs__add_change):

* subversion/libsvn_fs/fs-loader.c

(svn_fs_path_change2_create): Call svn_fs__path_change_create_internal()

instead of svn_fs__path_change2_create().

  1. … 4 more files in changeset.
Fix undefined references to svn_fs_path_change2_create().

* build.conf

(libsvn_fs.libs): Add 'libsvn_fs_util'.

* subversion/include/private/svn_fs_util.h

(svn_fs__path_change2_create): New declaration.

* subversion/libsvn_fs_util/fs-util.c

(svn_fs_path_change2_create): Rename to ...

(svn_fs__path_change2_create): ... this.

* subversion/libsvn_fs/fs-loader.c

(svn_fs_path_change2_create): New function which wraps

svn_fs__path_change2_create().

  1. … 3 more files in changeset.
Cleanup trailing whitespace:

for extsn in c h cpp java py pl rb; do

sed -i -e 's/[ \t]*$//' `find . -name "*.$extsn" | xargs grep '[ \t]$' -l`

done

This should have been done before the 1.6.x branch, so I'm going to merge

it over there, to avoid merge conflicts in the future.

  1. … 83 more files in changeset.
Move svn_fs_path_change2_create to another file to unbreak build.

* subversion/libsvn_fs/util.c: Remove this file.

* subversion/libsvn_fs_util/fs-util.c

(svn_fs_path_change2_create): Function moved here

from libsvn_fs/util.c.

  1. … 1 more file in changeset.
Housekeeping: In the spirit of r27598, remove trailing whitespace in

our repository.

for extsn in c h cpp java py pl rb; do

find . -name "*.$extsn" | xargs perl -pi.bak -e 'next if /^\f$/; s/\s+$/\n/'

done

(This touches files in various and sundry parts of the tree, which I shan't

enumerate here.)

  1. … 175 more files in changeset.
Purge (I think) the source code tree of sqlite stuffs.

* Makefile.in,

* configure.ac,

* gen-make.py,

* build.conf,

* build/win32/make_dist.py,

* build/generator/gen_win.py,

* build/ac-macros/sqlite.m4

Remove sqlite build stuffs.

* subversion/include/private/svn_fs_mergeinfo.h,

* subversion/include/private/svn_fs_sqlite.h,

* subversion/libsvn_fs_util/mergeinfo-sqlite-index.c,

* subversion/libsvn_fs_util/sqlite-util.h,

* subversion/libsvn_fs_util/sqlite-util.c

Remove sqlite-specific source code files.

* subversion/libsvn_fs_base/tree.c,

* subversion/libsvn_fs_base/dag.c,

* subversion/libsvn_fs_fs/tree.c

Stop including now-missing headers.

* subversion/include/svn_error_codes.h

(SVN_ERR_FS_SQLITE_ERROR, SVN_ERR_FS_SQLITE_READONLY): Removed.

(SVN_ERR_FS_NO_SUCH_NODE_ORIGIN): Shift value to fill in a hole.

* subversion/libsvn_fs_fs/fs_fs.c

Stop including now-missing headers.

(svn_fs_fs__hotcopy): Don't copy the sqlite index database.

(commit_body): Don't try to update the sqlite mergeinfo index.

(svn_fs_fs__create): Don't try to create the sqlite index database.

* tools/server-side/svn-populate-node-origins-index.c

(usage_summary): Don't mention the sqlite index file.

* INSTALL

No longer name sqlite as a dependency.

  1. … 16 more files in changeset.
Use a single directory for the FSFS node origin cache instead of a

sqlite database.

Perhaps this should be sharded. (However, my only concern here is

"too many files in one directory", not "too many small files":

there'll only be one file here per file ever made in the repository,

so it'll take up no more space than a checkout of one branch of all

projects.)

(Technically, the node_origins table is still created in

sqlite-util.c, but that whole file is getting the wrecking ball soon.)

* subversion/include/private/svn_fs_node_origins.h

* subversion/libsvn_fs_util/node-origins-sqlite-index.c

Delete.

* subversion/libsvn_fs_fs/fs.h

(PATH_NODE_ORIGINS_DIR): New.

* subversion/libsvn_fs_fs/fs_fs.c

(): Don't include svn_fs_node_origins.h

(path_node_origins): New.

(svn_fs_fs_hotcopy): Copy cache.

(svn_fs_fs__commit): Call FSFS function instead of sqlite function.

(svn_fs_fs__ensure_dir_exists): Moved from ensure_dir_exists in

lock.c.

(svn_fs_fs__get_node_origin, set_node_origin,

svn_fs_fs__set_node_origins, svn_fs_fs__set_node_origin): New.

* subversion/libsvn_fs_fs/fs_fs.h

(svn_fs_fs__get_node_origin, svn_fs_fs__set_node_origins,

svn_fs_fs__set_node_origin): Declare.

* subversion/libsvn_fs_fs/lock.c

(ensure_dir_exists): Move to fs_fs.c and make libsvn_fs_fs-global.

(write_digest_file): Adjust.

* subversion/libsvn_fs_fs/structure

Document node-origins cache.

* subversion/libsvn_fs_fs/tree.c

(): Don't include svn_fs_node_origins.h.

(fs_node_origin_rev): Call FSFS functions instead of sqlite

functions.

  1. … 7 more files in changeset.
Remove no-longer-used svn_*_get_mergeinfo_for_tree APIs.

* subversion/include/private/svn_fs_mergeinfo.h

(svn_fs_mergeinfo__get_mergeinfo): Remove.

* subversion/include/svn_fs.h

* subversion/libsvn_fs/fs-loader.c

(svn_fs_get_mergeinfo_for_tree): Remove.

* subversion/libsvn_fs/fs-loader.h

(struct root_vtable_t): Remove get_mergeinfo_for_tree.

* subversion/libsvn_fs_base/tree.c

(root_vtable): Remove svn_fs_mergeinfo__get_mergeinfo_for_tree.

* subversion/libsvn_fs_fs/tree.c

(collect_mergeinfo_for_tree, get_mergeinfo_hash_for_tree,

fs_get_mergeinfo_for_tree): Remove.

(root_vtable): Remove fs_get_mergeinfo_for_tree.

* subversion/libsvn_fs_util/mergeinfo-sqlite-index.c

(parse_mergeinfo_from_db, get_mergeinfo_for_children,

svn_fs_mergeinfo__get_mergeinfo_for_tree): Remove.

  1. … 6 more files in changeset.
Merge 'reintegrate' branch to trunk.

  1. … 57 more files in changeset.
Fix a compiler warning.

* subversion/libsvn_fs_util/mergeinfo-sqlite-index.c

(get_mergeinfo_for_children): Lose unused 'omit' variable.

Step 3 in removing repos-level checks using mergeinfo created on a copy:

Remove the mergeinfo filter function argument and baton from

svn_fs_get_mergeinfo_for_tree(). The filter was only used in one place, was

a giant hack to begin with, and is now no longer needed.

* subversion/libsvn_repos/log.c

(get_combined_mergeinfo): Remove filter and baton parameters for

svn_fs_get_mergeinfo_for_tree().

* subversion/libsvn_fs_util/mergeinfo-sqlite-index.c

(svn_fs_mergeinfo__get_mergeinfo_for_tree, get_mergeinfo_for_children):

Stop using the filter parameters.

* subversion/include/svn_fs.h,

subversion/libsvn_fs/fs-loader.c

(svn_fs_get_mergeinfo_for_tree): Remove filter and baton parameters.

(svn_fs_mergeinfo_filter_func_t): Remove.

* subversion/include/private/svn_fs_mergeinfo.h

(svn_fs_mergeinfo__get_mergeinfo_for_tree): Same.

* subversion/libsvn_fs/fs-loader.h

(root_vtable_t.get_mergeinfo_for_tree): Same.

  1. … 5 more files in changeset.
Combine two helper functions (one to check that a filesystem is

opened, one to check that it isn't) into a single one that does both

based on a boolean expectation flag.

NOTE: This changes the test when 'expect_open' is set to examine

svn_fs_t->fsap_data instead of svn_fs_t->path (since that's what

is checked when 'expect_open' is not set, too). I think that's

fine, but wanted to note that here anyway just in case it rings

some alarm in another developer's mind.

* subversion/include/private/svn_fs_util.h,

* subversion/libsvn_fs_util/fs-util.c

(svn_fs__check_fs): Add 'expect_open' flag.

* subversion/libsvn_fs_base/fs.c

(check_already_open): Removed as unused.

(base_bdb_set_errcall): Update call to svn_fs__check_fs().

(open_databases): Use svn_fs__check_fs() instead of check_already_open().

* subversion/libsvn_fs_fs/fs.c

(check_already_open): Removed as unused.

(fs_create): Use svn_fs__check_fs() instead of check_already_open().

* subversion/libsvn_fs_base/tree.c,

* subversion/libsvn_fs_base/bdb/rev-table.c,

* subversion/libsvn_fs_base/uuid.c,

* subversion/libsvn_fs_base/revs-txns.c,

* subversion/libsvn_fs_base/lock.c,

* subversion/libsvn_fs_fs/tree.c,

* subversion/libsvn_fs_fs/fs_fs.c,

* subversion/libsvn_fs_fs/lock.c

Update call calls to svn_fs__check_fs(), passing TRUE for 'expect_open'.

  1. … 11 more files in changeset.
Wrap sqlite3_stmt inside a svn_fs__sqlite_stmt_t struct, which also

includes a pointer to the db.

This has two positive results:

- we stop using sqlite3_db_handle, which is relatively new to sqlite

- we introduce a few new helpers for getting values from queries,

which reduces the number of casts we make

* subversion/libsvn_fs_util/mergeinfo-sqlite-index.c

(index_path_mergeinfo, table_has_any_rows_with_rev,

parse_mergeinfo_from_db, get_mergeinfo_for_path,

get_mergeinfo_for_children): Adjust. (Pool usage may

need to be reaudited.)

* subversion/libsvn_fs_util/node-origins-sqlite-index.c

(get_origin, set_origin): Adjust.

* subversion/libsvn_fs_util/sqlite-util.h

* subversion/libsvn_fs_util/sqlite-util.c

(struct svn_fs__sqlite_stmt_t): New.

(svn_fs__sqlite_prepare): Take a pool argument; return a struct

containing the statement and the db.

(step_with_expectation, svn_fs__sqlite_step_done,

svn_fs__sqlite_step_row, svn_fs__sqlite_step,

svn_fs__sqlite_bind_int64, svn_fs__sqlite_bind_text,

svn_fs__sqlite_finalize, svn_fs__sqlite_reset, upgrade_format):

Adjust.

(svn_fs__sqlite_reset): Remove a random declaration.

(svn_fs__sqlite_column_text, svn_fs__sqlite_column_revnum,

svn_fs__sqlite_column_boolean, svn_fs__sqlite_column_int): New.

"mergeinfo" consistency: change "merge info" and "merge-info" to

"mergeinfo" in assorted error messages and comments.

  1. … 6 more files in changeset.
Followup to r28065: since we switched from using the "result" hash to

the "cache" hash, we now have to watch out for explicitly represented

negative cache results.

* subversion/libsvn_fs_util/mergeinfo-sqlite-index.c

(index_path_mergeinfo): Treat NEGATIVE_CACHE_RESULT the same as

NULL.

Stylistic tweak, no functional change.

* subversion/libsvn_fs_util/mergeinfo-sqlite-index.c

(svn_fs_mergeinfo__update_index): Add parens when passing the result

of a boolean comparison as a parameter value.

Make all sqlite-style error handling local to sqlite-util.c. The two

table-specific files now only use Subversion's error system, and only

call the sqlite_column_* APIs (which don't error) directly.

* subversion/libsvn_fs_util/sqlite-util.c

(svn_fs__sqlite_stmt_error): Remove.

(SQLITE_ERROR_CODE, SQLITE_ERR): Move and rename from

sqlite-util.h.

(step_with_expectation): Use a more specific error message (since

it's a app-level error, not a db-level error).

(svn_fs__sqlite_exec, svn_fs__sqlite_step,

svn_fs__sqlite_bind_int64, svn_fs__sqlite_bind_text,

svn_fs__sqlite_finalize, svn_fs__sqlite_reset, check_format,

svn_fs__sqlite_open, svn_fs__sqlite_close): Adjust.

* subversion/libsvn_fs_util/sqlite-util.h

(SVN_FS__SQLITE_ERROR_CODE, SVN_FS__SQLITE_ERR): Move and rename to

sqlite-util.c.

(svn_fs__sqlite_stmt_error): Remove.