Checkout
kotkov
committed
on 27 Jul 17
Stream svndiff deltas without creating temporary files during 'svn import'
over HTTP.

Creating the temporary files has a certain overhead (… Show more
Stream svndiff deltas without creating temporary files during 'svn import'

over HTTP.

Creating the temporary files has a certain overhead (time to write them

and disk space), but what is more important, it prevents the client and

server from simultaneously processing the data, i.e., checksumming and

compressing it.

This can have a significant effect for large binary files, and the new

approach can be up to 2x faster.  The actual improvement depends on the

configuration, but is generally more visible if the server and the client

process data with roughly the same speed.  Several tests in my environment

showed that the import time was reduced from 63s to 32s, and from 8s to 5s

for two (different) large binary files.

Technically, this is achieved by teaching the delta editor driver that

is used in svn_client_import5() to use the new apply_textdelta_stream()

editor callback.

* subversion/libsvn_client/import.c

 (open_txdelta_stream_baton_t): New.

 (txdelta_stream_baton_t, txdelta_next_window, txdelta_md5_digest):

  New.  Used to create a custom txdelta stream that retargets the result

  checksum into a supplied buffer.  Used in open_txdelta_stream().

 (send_file_contents): Move the core of this function, i.e., preparation

  of the delta stream into ...

 (open_txdelta_stream): ...this new svn_txdelta_stream_open_func_t

  implementation, and pass the necessary arguments via baton.  Use

  result and scratch pools.  Note that this function must be restartable,

  as it can be called more than once, for example, if the HTTP server

  decides to do a renegotiation, and we would be forced to resend

  the whole request body.

 (import_file): Use the new apply_textdelta_stream() editor callback.

Show less