This can significantly (up to 3 times) improve the speed of commits and other operations with large binary or incompressible files, while still maintaining a decent compression ratio.
Our current use of zlib compression — which, depending on the protocol, can be used multiple times — heavily affects the speed of commits with large binary or incompressible files. According to the Squash benchmark (https://quixdb.github.io/squash-benchmark/) and to my measurements, the zlib compression speed with the default level is about 30-40 MiB/s, and it doesn't matter if the file is incompressible or not.
This patch provides an alternative in the form of the LZ4 compression. While still providing a decent compression ratio, LZ4 offers much faster compression even than zlib with level=1, and can skip incompressible data chunks. Presumably, LZ4 is used for on-the-fly compression in different file systems for these reasons.
With this patch, LZ4 compression will be enabled for fsfs repositories which specify compression-level=1 in fsfs.conf. The interoperability is implemented by bumping the format of svndiff to 2 and the repository file system format to 8. From the client perspective, the patch starts using LZ4 compression only for file:// protocol, and the support/negotiation of the use of svndiff2 with LZ4 compression for http:// and svn:// can be added later.
The tests for LZ4 compression can be run with one of the following commands:
win-tests.py --fsfs-compression=1 make check FSFS_COMPRESSION=1
* subversion/include/svn_error_codes.h (SVN_ERR_LZ4_COMPRESSION_FAILED, SVN_ERR_LZ4_DECOMPRESSION_FAILED): New error codes.
* subversion/include/private/svn_subr_private.h (svn__compress, svn__decompress): Rename to ... (svn__compress_zlib, svn__decompress_zlib): ..this. (svn__compress_lz4, svn__decompress_lz4): Declare new functions.
* subversion/libsvn_subr/compress.c (): Include LZ4 library header. (svn__compress, svn__decompress): Rename to ... (svn__compress_zlib, svn__decompress_zlib): ..this. (svn__compress_lz4, svn__decompress_lz4): Implement new functions.
* subversion/libsvn_subr/packed_data.c (write_stream_data, read_stream_data): Update usages of svn__compress() and svn__decompress().
* subversion/libsvn_delta/svndiff.c (SVNDIFF_V2): New. (get_svndiff_header): Update to support svndiff2 headers. (encode_window, decode_window, write_handler): Support svndiff2 with LZ4 compression. Tweak the relevant comments.
* subversion/libsvn_fs_fs/fs.h (SVN_FS_FS__FORMAT_NUMBER): Bump to 8. (SVN_FS_FS__MIN_SVNDIFF2_FORMAT): New define.
* subversion/libsvn_fs_fs/fs_fs.c (write_config): Tweak the compression-level option description. (svn_fs_fs__create, svn_fs_fs__info_format): Update to handle the format bump.
* subversion/libsvn_fs_fs/transaction.c (txdelta_to_svndiff): New helper to call svn_txdelta_to_svndiff3() with appropriate svndiff version and compression level, depending on the file system configuration. (rep_write_get_baton, write_container_delta_rep): Use new helper.
* subversion/libsvn_fs_fs/revprops.c (parse_packed_revprops, repack_revprops, svn_fs_fs__copy_revprops): Update usages of svn__compress() and svn__decompress().
* subversion/libsvn_fs_fs/structure (Filesystem formats): Update to describe usage of svndiff2.