Checkout Tools
  • last updated 8 hours ago
Constraints: committers
Constraints: files
Constraints: dates

Changeset 1341826 is being indexed.

Make the recursive property retrieval functions in the wc_db api proportional

to the list of targets instead of to the total size of the working copy.

* subversion/libsvn_wc/wc-queries.sql

(STMT_CREATE_TARGETS_LIST): Add primary key, to give Sqlite a primary index.

(STMT_CACHE_NODE_PROPS): Renamed to ...

(STMT_CACHE_TARGET_PROPS): ... this. Join starting from the targets table

to avoid a wc scan on the nodes table to check for each item if it is

in the targets list. Integrate the STMT_CACHE_ACTUAL_PROPS statement

as that gives us the final result in a single step.

(STMT_CACHE_ACTUAL_PROPS): Remove statement.



(STMT_CACHE_TARGET_PRISTINE_PROPS): ... this. Join starting with the targets

table. Use a CASE to only fetch properties from below when we have a

base-deleted node.

* subversion/libsvn_wc/wc_db.c

(cache_props_recursive): Remove usage of STMT_CACHE_ACTUAL_PROPS.

* subversion/tests/libsvn_wc/wc-queries-test.c

(slow_statements): Remove three more slow statements.

  1. … 2 more files in changeset.
Avoid a table scan of ACTUAL_NODES for every directory during wc copy

operations by splitting a query into two separate queries that each

use the right index.

* subversion/libsvn_wc/wc-queries.sql

(STMT_SELECT_CONFLICT_MARKER_FILES): Split this query into ...

(STMT_SELECT_CONFLICT_MARKER_FILES1): ... this, for the directory and ...

(STMT_SELECT_CONFLICT_MARKER_FILES2): ... this, for the nodes within.

* subversion/libsvn_wc/wc_db.c

(get_conflict_marker_files): New function. Using two queries. Extracted

from ...

(svn_wc__db_get_conflict_marker_files): ... this function.

* subversion/libsvn_wc/wc_db.h

(svn_wc__db_get_conflict_marker_files): Remove obsolete comment.

* subversion/tests/libsvn_wc/wc-queries-test.c

(slow_statements): Remove yet another statement from the slow list.

  1. … 3 more files in changeset.
To remove another slow query, remove a dead code path: streamily retrieving

the op_depth 0 properties.

* subversion/include/private/svn_wc_private.h

(svn_wc__prop_list_recursive): Remove argument.

* subversion/libsvn_client/prop_commands.c


svn_client_proplist3): Update caller.

* subversion/libsvn_wc/props.c

(svn_wc__prop_list_recursive): Remove argument. Update caller.

* subversion/libsvn_wc/wc-queries.sql

(STMT_CACHE_NODE_BASE_PROPS): Remove unused query.

* subversion/libsvn_wc/wc_db.c

(cache_props_baton_t): Remove variable.

(cache_props_recursive): Remove dead code path.

(svn_wc__db_read_props_streamily): Remove argument.

* subversion/libsvn_wc/wc_db.h

(svn_wc__db_read_props_streamily): Remove argument.

* subversion/tests/libsvn_wc/wc-queries-test.c

(slow_statements): Remove query.

  1. … 6 more files in changeset.
Split the root node in the target list processing from its children and/or

descendants to avoid a table scan.

Two cheap queries are together much faster than a table scan.

* subversion/libsvn_wc/wc-queries.sql


(STMT_INSERT_TARGET_DEPTH_FILES): Only select files from children.


(STMT_INSERT_TARGET_DEPTH_INFINITY): Only select descendants

(to work the other 7).



Only select files from children.



* subversion/libsvn_wc/wc_db.c

(populate_targets_tree): Separate the root from the rest query to improve

performance on large working copies by using the index.

* subversion/tests/libsvn_wc/wc-queries-test.c

(slow_statements): Remove 6 more previously slow queries.

  1. … 2 more files in changeset.
Switch the few remaining queries that still used the LIKE operator to


* subversion/libsvn_wc/wc-queries.sql





* subversion/libsvn_wc/wc_db.c

(construct_like_arg): Remove unused function.


svn_wc__db_revert_list_notify): Update user.

(wclock_obtain_cb): Remove unused variable. The code was accidentally

already updated to use the local_relpath in r1341690.

* subversion/tests/libsvn_wc/wc-queries-test.c

(slow_statements): Remove 3 more previously slow queries.

(is_node_table): Handle wc_lock as node table for measurements.

  1. … 2 more files in changeset.
Remove a few wcroot specific queries that can now safely use the generic

variant without any measurable performance penalty.

* subversion/libsvn_wc/wc-queries.sql









* subversion/libsvn_wc/wc_db.c


op_revert_recursive_txn): Update caller.

(wclock_obtain_cb): Accidental variable usage change that should have

been part of my next commit.

* subversion/tests/libsvn_wc/wc-queries-test.c

(slow_statements): Remove slow statements.

  1. … 2 more files in changeset.
Redefine our Sqlite IS_STRICT_DESCENDANT_OF() macro to allow operating on the

"" relpath. This removes the need of duplicating queries for the root and

subtrees, without losing performance on the subtree case.

The full tree would use an index to select on wc_id anyway, so adding

an extra key doesn't slow it down.

* build/

(process_file): Redefine IS_STRICT_DESCENDANT_OF().

* subversion/libsvn_wc/wc-queries.sql














Remove now unneeded ?X = '' subexpressions.

* subversion/tests/libsvn_wc/wc-queries-test.c


Remove statements that started using indexes by this change.

  1. … 2 more files in changeset.
Fix the name of the actual table in the wc-queries-test, to properly detect

slow usages of this table.

* subversion/tests/libsvn_wc/wc-queries-test.c

(slow_statements): Add slow tests on the actual_node table.

(is_node_table): Update table name.

Fix a hidden bug in the merge handling and a slow wc.db query.

* subversion/include/private/svn_wc_private.h

(svn_wc__get_server_excluded_subtrees): Rename to ...

(svn_wc__get_excluded_subtrees): ... this. Update documentaton.

* subversion/libsvn_client/merge.c

(get_mergeinfo_paths): Update caller, that really just needs a list

of paths that are not there.

* subversion/libsvn_wc/node.c

(svn_wc__get_server_excluded_subtrees): Rename to ...

(svn_wc__get_excluded_subtrees): ... this. Update caller.

* subversion/libsvn_wc/wc-queries.sql

(STMT_SELECT_ALL_EXCLUDED_NODES): Remove selection of node itself and wcroot.

Also select excluded nodes.


STMT_SELECT_ALL_EXCLUDED_NODES, but specialized to use on the wcroot.

* subversion/libsvn_wc/wc_db.c

(svn_wc__db_get_server_excluded_subtrees): Rename to ...

(svn_wc__db_get_excluded_subtrees): ... this. Use separate new queries.

Don't ask for copying in the scratch pool if we are copying it to

result_pool within its original lifetime

* subversion/libsvn_wc/wc_db.h

(svn_wc__db_get_server_excluded_subtrees): Rename to ...

(svn_wc__db_get_excluded_subtrees): ... this. Update documentaton.

* subversion/tests/libsvn_wc/wc-queries-test.c

(slow_statements): Add STMT_SELECT_ALL_EXCLUDED_WCROOT to the operationg

on the entire wc list. Remove STMT_SELECT_ALL_SERVER_EXCLUDED_NODES as

it properly uses an index instead of a table scan now.

  1. … 6 more files in changeset.
* subversion/tests/libsvn_wc/wc-queries-test.c


Apply result from review of STMT_DELETE_PRISTINE_IF_UNREFERENCED statement.

* subversion/tests/libsvn_wc/wc-queries-test.c

(test_parsable): Print runtime sqlite version, to help in diagnosing errors.

* subversion/tests/libsvn_wc/wc-queries-test.c

(headers): Following up on r1341232, undo accidental commit of my local

amalgamation test.

* subversion/tests/libsvn_wc/wc-queries-test.c

(test_query_expectations): Avoid a possible segfault with Sqlite 3.7.2.

* subversion/tests/libsvn_wc/wc-queries-test.c

(slow_statements): Remove statements that are not slow with the current


(test_query_expectations): Update evaluation to allow reporting not-slow


* subversion/tests/libsvn_wc/wc-queries-test.c

(test_query_expectations): Simplify and consolidate sqlite capability


Make wc-queries-test failures much easier to diagnose by extending the sqlite

transform script a bit. Store names and some optional value during .sql

parsing and make that information available in the test.

* build/

(Processor): Extend regex to capture flags.

(_sub_statement): Generate extra macro.

(main): Generate a DECLARE_STATEMENT_INFO macro.

* subversion/tests/libsvn_wc/wc-queries-test.c

(wc_query_info): New static variable.


test_query_expectations): Extend error messages with statement name.

  1. … 1 more file in changeset.
Following up on r1341130, fix the bug that the code only parsed the statements

until the first statement that contained multiple queries.

Record current 'slow' queries, based on Sqlite 3.7.12 results.

* subversion/tests/libsvn_wc/wc-queries-test.c

(slow_statements): Add a lot of statements.

(parse_explanation_item): Detect 'USE' operation.

(test_query_expectations): Continue instead of break on parse errors and

multi-statements. Check for primary key usage first and allow 'slow'

suppression for all checks.

Fix integer size mismatch.

* subversion/tests/libsvn_wc/wc-queries-test.c

(parse_explanation_item): Use size_t for the result of strlen().

* subversion/tests/libsvn_wc/wc-queries-test.c

(test_query_expectations): Use a different method to detect

SQLITE_OMIT_EXPLAIN compilation option of Sqlite.

Add some simple query plan evaluation test in the wc-queries-test. This

test passes for Sqlite 3.7.12, but might (and probably will) fail for

older Sqlite versions that lack specific optimizations.

Yes it would be much nicer to implement this in a script language...

But no, that wouldn't help as that might test a different Sqlite

version than Subversion.

* subversion/tests/libsvn_wc/wc-queries-test.c

(slow_statements): New list.

(is_schema_statement): Renamed to in_list and add list argument.


is_schema_statement): New macros.

(explanation_item): New struct.

(parse_explanation_item): New function.

(is_node_table): New function.

(test_query_expectations): New function.

(test_funcs): Add test_query_expectations.

Another attempt of fixing the linking/compilation of wc-queries-test.c for

sqlite amalgamation users.

* subversion/tests/libsvn_wc/wc-queries-test.c

(includes): Add svn_private_config.h

(test_parsable): Rename pool argument.

Add a raw test application to test preparing all our working copy queries on

Sqlite. The current version just parses all queries, but I intend to add

support to verify some aspects of the query plan in a later patch.

The fix in r1340998 was a direct result of writing this test.

* build.conf

(wc-queries-test): New test.

(__ALL_TESTS__): Add wc-queries-test.

* subversion/tests/libsvn_wc/wc-queries-test.c

New file.

  1. … 1 more file in changeset.