Checkout Tools
  • last updated 4 hours ago
Constraints: committers
Constraints: files
Constraints: dates

Changeset 1357376 is being indexed.

When done with temporary tables and triggers explicitly drop them, to avoid

dropping unintentional items (or forgetting to drop intentional items).

This leaves similar fixups for when creating the tables and triggers.

* subversion/libsvn_wc/wc-queries.sql

(STMT_FINALIZE_CHANGELIST): Drop the right trigger explicitly.





STMT_FINALIZE_DELETE): Drop explicitly.

* subversion/tests/libsvn_wc/wc-queries-test.c

(schema_statements): Add STMT_CREATE_CHANGELIST_TRIGGER.

  1. … 1 more file in changeset.
Following up on r1353676, introduce an index on the md5 checksum in the pristine


* subversion/tests/libsvn_wc/wc-queries-test.c

(slow_statements): Remove STMT_SELECT_PRISTINE_BY_MD5, as this now

uses an index.

* subversion/tests/libsvn_wc/wc-queries-test.c

(STMT_CREATE_SCHEMA): Add new index for new wc.db instances.

(STMT_UPGRADE_TO_30): Add new index if it doesn't exist yet on the

future bump to format 30.

Patch by: rhuijben

  1. … 1 more file in changeset.
* subversion/tests/libsvn_wc/wc-queries-test.c

(slow_statements): Fix a comment.

Resolve four wc-queries-test warnings by commenting one statement and moving

two others to wc-metadata.sql

* subversion/libsvn_wc/upgrade.c

(migrate_tree_conflict_data): Update statement references.

* subversion/libsvn_wc/wc-metadata.sql

(STMT_UPGRADE_TO_21): Add two bump specific statements.

(STMT_UPGRADE_TO_27): Add one bump specific statements.

* subversion/libsvn_wc/wc-queries.sql

(STMT_SELECT_TARGETS): Comment statement, like its only caller in wc_db.c



STMT_HAS_ACTUAL_NODES_CONFLICTS): Move to wc-metadata.sql and add an upgrade

specific suffix to avoid new users.

* subversion/tests/libsvn_wc/wc-queries-test.c

(slow_statements): Remove 4 statements. (upgrade statements are automatically

ignored). Add some comments.

  1. … 3 more files in changeset.
* subversion/tests/libsvn_wc/wc-queries-test.c

(slow_statements): Following up on r1342984, remove STMT_SELECT_ALL_FILES as

the slowness has been resolved.

Make svn changelist --remove's performance relative to the size of the target

tree instead of relative to the number of ACTUAL records.

* subversion/libsvn_wc/wc-queries.sql

(STMT_DELETE_ACTUAL_EMPTIES): Just delete unneeded ACTUAL nodes within

reach of the operation instead of globally to avoid a table scan.

* subversion/libsvn_wc/wc_db.c

(set_changelist_txn): Pass local_relpath to sql query.

* subversion/tests/libsvn_wc/wc-queries-test.c

(slow_statements): Remove the last 'need review' item.

  1. … 2 more files in changeset.
Rewrite a query to allow using indexes even though Sqlite doesn't understand

a thing of what the query tries to perform.

* subversion/libsvn_wc/wc-queries.sql

(STMT_SELECT_EXTERNAL_PROPERTIES): Rewrite using a UNION to allow using

indexes for all cases.

* subversion/tests/libsvn_wc/wc-queries-test.c

(slow_statements): And remove from the slow list.

  1. … 1 more file in changeset.
Avoid nested full table scans on two tables in a single query by rewriting the

query to a completely different pattern.

Only operate on the relevant subset of the actual table to avoid doing unneeded

work on all other nodes in the working copy.

The IN (select local_relpath FROM .. WHERE .. AND kind = 'qqq') style

queries are not optimized by Sqlite to use a select on local_relpath.

Depending on the number of affected nodes and nodes in the working copy, this

query is now orders of magnitude faster.

* subversion/libsvn_wc/wc-queries.sql


* subversion/libsvn_wc/wc_db.c

(set_changelist_txn): Pass the operation root as second argument.

* subversion/tests/libsvn_wc/wc-queries-test.c

(slow_statements): Remove another slow statement from the list.

  1. … 2 more files in changeset.
* subversion/tests/libsvn_wc/wc-queries-test.c

(slow_statements): Remove the move queries as these use an index now.

Use a #if for a compile-time check, rather than an if().

* subversion/tests/libsvn_wc/wc-queries-test.c

(test_sqlite_version): Use the preprocessor when determining which version

of SQLite we're building with, rather than doing so at runtime.

* subversion/tests/libsvn_wc/wc-queries-test.c

(supported_explain_query_plan): Following up on r1342243, don't return FALSE

when the query plan is empty, as that is the case for the dumb test query.

* subversion/tests/libsvn_wc/wc-queries-test.c

(supported_explain_query_plan): New function, extracted from


(test_query_expectations): Extract support detection code.

Don't produce errors for unexpected fast queries by adding the

query plan to the error chain.

[And I replaced some accidentally added tabs with spaces]

It is cheaper for us to read a few more Sqlite results that might even have

the answer we looking for than for Sqlite to collect all items and provide

them sorted.

Apply this knowledge on the is_wclocked check + query.

* subversion/libsvn_wc/wc-queries.sql

(STMT_SELECT_ANCESTOR_WCLOCKS): Reorder some checks for sanity and remove

the order by, as Sqlite doesn't know that the passed arguments are related.

* subversion/tests/libsvn_wc/wc-queries-test.c

(STMT_SELECT_ANCESTOR_WCLOCKS): And another 'slow' query gone.

  1. … 1 more file in changeset.
Provide the Sqlite query plan as part of the error message, to avoid having

to install all kinds of Sqlite versions when problems are reported.

* subversion/tests/libsvn_wc/wc-queries-test.c

(test_query_expectations): Store the already copied rows in an array and

add them to created warnings.

* subversion/libsvn_wc/wc-queries.sql

(STMT_SELECT_GE_OP_DEPTH_CHILDREN): Avoid creating a temporary table

by using UNION ALL instead of UNION.

(Without ALL duplicates are filtered by using a temp table)

* subversion/tests/libsvn_wc/wc-queries-test.c

(slow_statements): Remove slow marker.

  1. … 1 more file in changeset.
* subversion/tests/libsvn_wc/wc-queries-test.c

(test_sqlite_version): Grammar tweak.

Wc-queries-test: Remove dependency on libsvn_wc itself. Add version check

as a separate test, to provide a bit more information.

* build.conf

(wc-queries-test): Remove dependency.

* subversion/tests/libsvn_wc/wc-queries-test.c

(includes): Add private/svn_dep_compat.h.

(test_sqlite_version): New function.

(test_parsable): Remove printf, that is now part of test_sqlite_version.

  1. … 1 more file in changeset.
Extend the wc-queries-test application to catch another type of

potentially slow queries: Queries that create temporary tables.

* subversion/tests/libsvn_wc/wc-queries-test.c

(slow_statements): Remove STMT_SELECT_CHANGELIST_LIST as that is now

caught as a result table. Add 3 queries that use a temp table.

(explanation_item): Add some variables.

(parse_explanation_item): Parse COMPOUND and some bits of USE. Set

search type.

(is_result_table): Add changelist_list.

(test_query_expectations): Update tests to handle other info items.

Check for search items before testing fields that may be NULL.

Cleanup the svn ci --include-externals query to use indexes for most of its

work. Use the knowledge that 'parent_relpath = ?' is a subset of

IS_STRICT_DESCENDANT_OF() to at least use some indexing.

* subversion/libsvn_wc/externals.c

(is_external_rolled_out): Externals live in BASE, so use the cheap/direct

api towards that layer, without bothering with copies.

* subversion/libsvn_wc/wc-queries.sql


Remove unused result column. Add missing wc_id in subquery. Just obtain

the repos_id from the wc root, as we can't trust it to be available on

the passed node.

* subversion/libsvn_wc/wc_db.c

(svn_wc__db_committable_externals_below): Update column indexes.

* subversion/tests/libsvn_wc/wc-queries-test.c


  1. … 3 more files in changeset.
Help the Sqlite query planner a bit by rewriting two queries in a way that

makes it use indexes, where it didn't before.

* subversion/libsvn_wc/wc-queries.sql


STMT_SELECT_EXTERNALS_DEFINED): Make the OR operation the outer operation

by duplicating some cheap tests.

(STMT_SELECT_COMMITTABLE_EXTERNALS_BELOW): Accidental change, that should

have been part of the r1341851 change.

* subversion/tests/libsvn_wc/wc-queries-test.c

(slow_statements): Remove two slow statements.

  1. … 1 more file in changeset.
Clean up some loose ends in the streamy property handling.

Rename some statements and apply a generic exception in the statement

validator for tables like the property cache.

No functional changes.

* subversion/libsvn_wc/wc-queries.sql


(STMT_CREATE_TARGET_PROP_CACHE): ... as it is more related to target than

node processing. Remove proof of concept note, as this was part of 1.7.

Rename temporary table to target_prop_cache, to more closely match

our other temporary table names.

Define primary key to avoid a sort when reporting.






* subversion/libsvn_wc/wc_db.c

(cache_props_recursive): Update user.

* subversion/tests/libsvn_wc/wc-queries-test.c

(schema_statements): Update reference.

(slow_statements): Remove statement.

(is_node_table): Add final line to allow extending with one line changes.

(is_result_table): New function.

(test_query_expectations): Allow exceptions on scan and index rules on

(temporary) result tables.

  1. … 2 more files in changeset.
Make the recursive property retrieval functions in the wc_db api proportional

to the list of targets instead of to the total size of the working copy.

* subversion/libsvn_wc/wc-queries.sql

(STMT_CREATE_TARGETS_LIST): Add primary key, to give Sqlite a primary index.

(STMT_CACHE_NODE_PROPS): Renamed to ...

(STMT_CACHE_TARGET_PROPS): ... this. Join starting from the targets table

to avoid a wc scan on the nodes table to check for each item if it is

in the targets list. Integrate the STMT_CACHE_ACTUAL_PROPS statement

as that gives us the final result in a single step.

(STMT_CACHE_ACTUAL_PROPS): Remove statement.



(STMT_CACHE_TARGET_PRISTINE_PROPS): ... this. Join starting with the targets

table. Use a CASE to only fetch properties from below when we have a

base-deleted node.

* subversion/libsvn_wc/wc_db.c

(cache_props_recursive): Remove usage of STMT_CACHE_ACTUAL_PROPS.

* subversion/tests/libsvn_wc/wc-queries-test.c

(slow_statements): Remove three more slow statements.

  1. … 2 more files in changeset.
Avoid a table scan of ACTUAL_NODES for every directory during wc copy

operations by splitting a query into two separate queries that each

use the right index.

* subversion/libsvn_wc/wc-queries.sql

(STMT_SELECT_CONFLICT_MARKER_FILES): Split this query into ...

(STMT_SELECT_CONFLICT_MARKER_FILES1): ... this, for the directory and ...

(STMT_SELECT_CONFLICT_MARKER_FILES2): ... this, for the nodes within.

* subversion/libsvn_wc/wc_db.c

(get_conflict_marker_files): New function. Using two queries. Extracted

from ...

(svn_wc__db_get_conflict_marker_files): ... this function.

* subversion/libsvn_wc/wc_db.h

(svn_wc__db_get_conflict_marker_files): Remove obsolete comment.

* subversion/tests/libsvn_wc/wc-queries-test.c

(slow_statements): Remove yet another statement from the slow list.

  1. … 3 more files in changeset.
To remove another slow query, remove a dead code path: streamily retrieving

the op_depth 0 properties.

* subversion/include/private/svn_wc_private.h

(svn_wc__prop_list_recursive): Remove argument.

* subversion/libsvn_client/prop_commands.c


svn_client_proplist3): Update caller.

* subversion/libsvn_wc/props.c

(svn_wc__prop_list_recursive): Remove argument. Update caller.

* subversion/libsvn_wc/wc-queries.sql

(STMT_CACHE_NODE_BASE_PROPS): Remove unused query.

* subversion/libsvn_wc/wc_db.c

(cache_props_baton_t): Remove variable.

(cache_props_recursive): Remove dead code path.

(svn_wc__db_read_props_streamily): Remove argument.

* subversion/libsvn_wc/wc_db.h

(svn_wc__db_read_props_streamily): Remove argument.

* subversion/tests/libsvn_wc/wc-queries-test.c

(slow_statements): Remove query.

  1. … 6 more files in changeset.
Split the root node in the target list processing from its children and/or

descendants to avoid a table scan.

Two cheap queries are together much faster than a table scan.

* subversion/libsvn_wc/wc-queries.sql


(STMT_INSERT_TARGET_DEPTH_FILES): Only select files from children.


(STMT_INSERT_TARGET_DEPTH_INFINITY): Only select descendants

(to work the other 7).



Only select files from children.



* subversion/libsvn_wc/wc_db.c

(populate_targets_tree): Separate the root from the rest query to improve

performance on large working copies by using the index.

* subversion/tests/libsvn_wc/wc-queries-test.c

(slow_statements): Remove 6 more previously slow queries.

  1. … 2 more files in changeset.
Switch the few remaining queries that still used the LIKE operator to


* subversion/libsvn_wc/wc-queries.sql





* subversion/libsvn_wc/wc_db.c

(construct_like_arg): Remove unused function.


svn_wc__db_revert_list_notify): Update user.

(wclock_obtain_cb): Remove unused variable. The code was accidentally

already updated to use the local_relpath in r1341690.

* subversion/tests/libsvn_wc/wc-queries-test.c

(slow_statements): Remove 3 more previously slow queries.

(is_node_table): Handle wc_lock as node table for measurements.

  1. … 2 more files in changeset.
Remove a few wcroot specific queries that can now safely use the generic

variant without any measurable performance penalty.

* subversion/libsvn_wc/wc-queries.sql









* subversion/libsvn_wc/wc_db.c


op_revert_recursive_txn): Update caller.

(wclock_obtain_cb): Accidental variable usage change that should have

been part of my next commit.

* subversion/tests/libsvn_wc/wc-queries-test.c

(slow_statements): Remove slow statements.

  1. … 2 more files in changeset.
Redefine our Sqlite IS_STRICT_DESCENDANT_OF() macro to allow operating on the

"" relpath. This removes the need of duplicating queries for the root and

subtrees, without losing performance on the subtree case.

The full tree would use an index to select on wc_id anyway, so adding

an extra key doesn't slow it down.

* build/

(process_file): Redefine IS_STRICT_DESCENDANT_OF().

* subversion/libsvn_wc/wc-queries.sql














Remove now unneeded ?X = '' subexpressions.

* subversion/tests/libsvn_wc/wc-queries-test.c


Remove statements that started using indexes by this change.

  1. … 2 more files in changeset.
Fix the name of the actual table in the wc-queries-test, to properly detect

slow usages of this table.

* subversion/tests/libsvn_wc/wc-queries-test.c

(slow_statements): Add slow tests on the actual_node table.

(is_node_table): Update table name.

Fix a hidden bug in the merge handling and a slow wc.db query.

* subversion/include/private/svn_wc_private.h

(svn_wc__get_server_excluded_subtrees): Rename to ...

(svn_wc__get_excluded_subtrees): ... this. Update documentaton.

* subversion/libsvn_client/merge.c

(get_mergeinfo_paths): Update caller, that really just needs a list

of paths that are not there.

* subversion/libsvn_wc/node.c

(svn_wc__get_server_excluded_subtrees): Rename to ...

(svn_wc__get_excluded_subtrees): ... this. Update caller.

* subversion/libsvn_wc/wc-queries.sql

(STMT_SELECT_ALL_EXCLUDED_NODES): Remove selection of node itself and wcroot.

Also select excluded nodes.


STMT_SELECT_ALL_EXCLUDED_NODES, but specialized to use on the wcroot.

* subversion/libsvn_wc/wc_db.c

(svn_wc__db_get_server_excluded_subtrees): Rename to ...

(svn_wc__db_get_excluded_subtrees): ... this. Use separate new queries.

Don't ask for copying in the scratch pool if we are copying it to

result_pool within its original lifetime

* subversion/libsvn_wc/wc_db.h

(svn_wc__db_get_server_excluded_subtrees): Rename to ...

(svn_wc__db_get_excluded_subtrees): ... this. Update documentaton.

* subversion/tests/libsvn_wc/wc-queries-test.c

(slow_statements): Add STMT_SELECT_ALL_EXCLUDED_WCROOT to the operationg

on the entire wc list. Remove STMT_SELECT_ALL_SERVER_EXCLUDED_NODES as

it properly uses an index instead of a table scan now.

  1. … 6 more files in changeset.