Checkout
kotkov
committed
on 14 Sep
Fix an issue with the readline implementation for file streams that could
cause excessive memory usage for inputs containing one or multiple… Show more
Fix an issue with the readline implementation for file streams that could

cause excessive memory usage for inputs containing one or multiple \0 bytes.

This is the likely cause of the OOM reported in

 https://lists.apache.org/thread.html/c96eb5618ac0bf6e083345e0fdcdcf834e30913f26eabe6ada7bab62@%3Cusers.subversion.apache.org%3E

(I think that the problem itself is a regression introduced by me in 1.10.)

Note: one thing I noticed while preparing the fix is that our `readline_fn`

functions for different streams have inconsistent behavior if the input data

contains \0 bytes.  More specifically, they may return different `line` values,

that may either be truncated at \0 or actually contain the whole data between

EOLs, including \0 bytes.  For now, this patch only fixes the excessive memory

usage problem, and I noted this related problem in the test and left it for

future work.

* subversion/libsvn_subr/stream.c

 (readline_apr_lf, readline_apr_generic): Reallocate the buffer based on its

  current size, instead of calculating the new size based on the already

  prealloc'd size.  There are no actual benefits in reallocating based on

  `blocksize`, and in the described case with \0 bytes doing so also backfires

  and may cause excessive allocations due to the actual size of the string

  being less than we expect it to.  A degenerate case of the erroneous

  behavior is ...

* subversion/tests/libsvn_subr/stream-test.c

 (test_stream_readline_file_nul): ...exploited in this new test.

 (test_funcs): Run new test.

* subversion/tests/libsvn_subr

 (): Adjust svn:ignore.

Show less