Checkout Tools
  • last updated 4 hours ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
On branch swig-py3: Fix Python exception handling in callback function

* subversion/bindings/swig/python/libsvn_swig_py/swigutil_py.c

(svn_swig_py_status_func2): Clear python exception if it caused in Python

call back function. (To avoid crash in py3)

* subversion/bindings/swig/python/tests/wc.py

(SubversionWorkingCopyTestCase.test_status_editor): Remove an assertion

from call back because it takes no effect, and check result after

svn_wc_get_status_editor2() API call instead.

(SubversionWorkingCopyTestCase.test_status_editor_callback_exception):

New test case for Python exception within svn_wc_status_func2 call back.

Patch by: Jun Omae <jun66j5 at gmail.com>

(Tweaked by me.)

  1. … 1 more file in changeset.
On branch swig-py3: Use bytes for all API string parameters.

Since many string like arguments in the API can contain arbitrary bytes, use the

Python Bytes interface in Python 3, while still using Str in Python 2, to ensure

full compatibility with all possible input data.

Build system change:

* build/ac-macros/swig.m4:

Add "-DPY3" option to SWIG_PY_OPTS for py3 build to define symbol to switch

typemap between py2 and py3.

Typemap changes:

* subversion/bindings/swig/core.i

(%typemap(argout) (char *buffer, apr_size_t *len):

Map buffer arg to Bytes insted of Str.

(%typemap(in) (const char *data, apr_size_t *len) ($*2_type temp):

Map data arg to Bytes instead of Str and pass Py_ssize_t variable as length

argment of PyBytes_AsStringAndSize for for explicit type conversion.

(%typemap(in) const void *value):

Map value arg to Bytes instead of Str for svn_auth_set_parameter().

* subversion/bindings/swig/include/svn_global.swg:

- Define SWIG_PYTHON_STRICT_BYTE_CHAR symbol for swig context for

Python3.

- Switch "in" type typemap for char *, char const *, char * const,

and char const * const "", to use parse parameter "y" for py3 and

"s" for py2.

* subversion/bindings/swig/include/svn_string.swg

(%typemap(argout) RET_STRING, %typemap(in) svn_stringbuf_t *,

%typemap(out) svn_stringbuf_t *, %typemap(in) const svn_string_t *,

%typemap(out) svn_string_t *, %typemap(argout) const char **OUTPUT):

Map args to Bytes instead of Str.

* subversion/bindings/swig/include/svn_swigcompat.swg

(%set_constant(name, value): Use PyDict_SetItem and PyBytes_FromString

instead of PyDict_SetItemString, for SWIG <= 1.3.27 (not tested).

* subversion/bindings/swig/include/svn_types.swg

(%typemap(in) const char *MAY_BE_NULL): Don't use "z" conversion

format and convert to Bytes if it is not NULL.

(%typemap(in) (const char *PTR, apr_size_t LEN): Use PyBytes_Check and

PyBytes_AsStringAndSize instead of PyStr_Check and PyStr_AsUTF8AndSize.

* subversion/bindings/swig/python/libsvn_swig_py/swigutil_py.c

(make_string_from_ob(), make_svn_string_from_ob(),

svn_swig_py_make_file(), svn_swig_py_get_commit_log_func()):

Use PyBytes_Check and PyBytes_AsString instead of PyStr_Check and

PyStr_AsString.

(convert_hash()): Use Dict_SetItem and PyBytes_FromString instead of

DictItemString.

(convert_svn_string_t(), svn_swig_py_proparray_to_dict(),

ra_callbacks_get_wc_prop()):

Use PyBytes_AsStringAndSize instead of PyStr_AsUTF8AndSize.

(cstring_to_pystring(), convert_string(),

svn_swig_py_propinheriteditemarray_to_dict(),

svn_swig_py_proparray_to_dict(),

svn_swig_py_locationhash_to_dict(),

svn_swig_py_c_strings_to_list(),

svn_swig_py_changed_path_hash_to_dict(),

svn_swig_py_changed_path2_hash_to_dict(),

svn_swig_py_unwrap_string(),

svn_swig_py_array_to_list()): Use PyBytes_FromString instead of

PyStr_FromString.

(delete_entry(), add_directory(), open_directory(), change_dir_prop(),

add_file(), open_file(), apply_textdelta(), change_file_prop(),

close_file(), parse_fn3_uuid_record(), parse_fn3_set_revision_property(),

parse_fn3_set_node_property(), parse_fn3_delete_node_property(),

svn_swig_py_notify_func(), svn_swig_py_status_func(),

svn_swig_py_delta_path_driver_cb_func(), svn_swig_py_status_func2(),

svn_swig_py_fs_lock_callback(), svn_swig_py_repos_authz_func(),

svn_swig_py_repos_history_func(), svn_swig_py_proplist_receiver2(),

svn_swig_py_log_receiver(), svn_swig_py_info_receiver_func(),

svn_swig_py_client_blame_receiver_func(),

svn_swig_py_changelist_receiver_func(),

svn_swig_py_auth_gnome_keyring_unlock_prompt_func(),

svn_swig_py_auth_simple_prompt_func(),

svn_swig_py_auth_username_prompt_func(),

svn_swig_py_auth_ssl_server_trust_prompt_func(),

svn_swig_py_auth_ssl_client_cert_prompt_func(),

svn_swig_py_auth_ssl_client_cert_pw_prompt_func(),

svn_swig_py_config_auth_walk_func(), ra_callbacks_get_wc_prop(),

ra_callbacks_push_or_set_wc_prop(), ra_callbacks_invalidate_wc_props(),

svn_swig_py_commit_callback(), svn_swig_py_ra_file_rev_handler_func(),

svn_swig_py_ra_lock_callback(), reporter_set_path(),

reporter_delete_path(), reporter_link_path(),

wc_diff_callbacks2_file_changed_or_added(),

wc_diff_callbacks2_file_deleted(), wc_diff_callbacks2_dir_added(),

wc_diff_callbacks2_dir_deleted(),

wc_diff_callbacks2_dir_props_changed(),

svn_swig_py_config_enumerator2(),

svn_swig_py_config_section_enumerator2()): Use 's' for py2 and use 'y'

for py3 in argment format string.

(ra_callbacks_push_or_set_wc_prop()): Use PyBytes_FromStringAndSize

instead of PyStr_FromStringAndSize.

(ra_callbacks_get_client_string()): Use PyBytes_FromString instaed of

PyStr_AsString.

* subversion/bindings/swig/svn_client.i

(%typemap(argout) apr_array_header_t **props):

Use PyBytes_FromStringAndSize() instead of PyStr_FromStringAndSize.

Helper python code changes:

* subversion/bindings/swig/python/svn/core.py

(svn_path_compare_paths): Do not use cmp, for py3 compatibility.

(Stream.read()): Chunks is now list of bytes, not list of str.

* subversion/bindings/swig/python/svn/fs.py

(FileDiff.get_pipe()):

- Make sure self.tempfile1 and self.self.tempfile2 are bytes.

- Pass bytes to header_encoding arg in _svndiff.file_output_unified4().

* subversion/bindings/swig/python/svn/ra.py

(Callbacks..__doc__):

Treat path as bytes in sample code.

* subversion/bindings/swig/python/svn/repos.py

(ChangeCollector._make_base_path()): Treat path as bytes.

(ChangeCollector.open_root(): Path and basepath should be bytes in

dir_baton.

(RevisionChangeCollector._make_base_path()): Treat path as bytes.

Unit test changes:

* subversion/bindings/swig/python/tests/auth.py:

Replace all str literals with bytes literals except docstring and module name.

* subversion/bindings/swig/python/tests/checksum.py

(ChecksumTestCases.test_checksum()): Check if type of check_val is

bytes instead of str.

* subversion/bindings/swig/python/tests/client.py:

Replace all str literals with bytes literals except for the mode arg of open(),

arguments to utils.Temper methods, docstrings, and module names.

(SubversionClientTestCase.test_uuid_from_url): Check if return value

type client.uuid_from_url() is bytes instead of str.

(SubversionClientTestCase.test_uuid_from_path): Check if return value

type client.uuid_from_path() is bytes instead of str.

(SubversionClientTestCase.test_merge_peg3()): Open the result file in

raw mode.

(SubversionClientTestCase.test_update4()): Convert os.path.sep into

bytes if it is str in py3.

* subversion/bindings/swig/python/tests/core.py:

Replace all str literals with bytes literals except for exception messages,

arguments to utils.Temper methods, docstrigs, and module names.

Add new tests for svn_stream_*()

(SubversionCoreTestCase.test_stream_from_stringbuf()): New test method.

(SubversionCoreTestCase.test_stream_read_full()): New test method.

(SubversionCoreTestCase.test_stream_read2()): New test method.

(SubversionCoreTestCase.test_stream_write_exception()): New test method.

(SubversionCoreTestCase.test_stream_write()): New test method.

(SubversionCoreTestCase.test_stream_readline()): New test method.

* subversion/bindings/swig/python/tests/delta.py

(DeltaTestCase.estTxWindowHandler_stream_IF(),

DeltaTestCase.estTxWindowHandler_Stream_IF()):

Use bytes literals to make stream when using

svn.core.svn_stream_from_stringbuf().

Use bytes file name to make stream when using

svn.core.svn_stream_from_aprfile2().

* subversion/bindings/swig/python/tests/fs.py

(SubversionFSTestCase.log_message_func(): log_msg_func3 callback now

expects bytes to return.

(SubversionFSTestCase.setUp()): Pass bytes path and url to client.import2().

(SubversionFSTestCase.test_diff_repos_paths_internal(),

SubversionFSTestCase.test_diff_repos_paths_external()): Use bytes for

path1 argument to fs.FileDiff.__init__().

* subversion/bindings/swig/python/tests/mergeinfo.py

(SubversionMergeinfoTestCase.TEXT_MERGEINFO1,

SubversionMergeinfoTestCase.TEXT_MERGEINFO2,

SubversionMergeinfoTestCase.MERGEINFO_SRC): Use bytes instead of str.

(SubversionMergeinfoTestCase.test_mergeinfo_get()):

- Pass list of bytes for paths arg to repos.fs_get_mergeinfo().

- Replace str in expected_mergeinfo with bytes.

* subversion/bindings/swig/python/tests/ra.py

Replace all str literals with bytes literals except assertion message,

arguments to utils.Temper methods, docstrings, and module names.

(SubversionRepositoryAccessTestCase.test_get_file): Compare file

content as bytes.

* subversion/bindings/swig/python/tests/repository.py

Replace all str literals with bytes literals when passed into or returned from

the Subversion API, except exception messages.

(SubversionRepositoryTestCase.test_dump_fs2()): Treat dump and

feedback as bytes and compose expected_feedback as bytes.

* subversion/bindings/swig/python/tests/utils.py

(Temper.alloc_empty_dir(),

Temper.alloc_empty_repo): Convert return value of tempfile.mkdtemp() into

bytes.

(Temper.file_uri_for_path()): Return URI as bytes.

* subversion/bindings/swig/python/tests/wc.py

Replace all str literals with bytes literals when passed into or returned from

the Subversion API, except exception messages.

(SubversionWorkingCopyTestCase.test_get_adm_dir()): Check type as

bytes.

(SubversionWorkingCopyTestCase.test_get_pristine_copy_path(),

SubversionWorkingCopyTestCase.test_diff_editor4()): Open file in raw mode.

(SubversionWorkingCopyTestCase.test_commit()): Use bytes literals

instead of str.

* subversion/bindings/swig/python/tests/trac/versioncontrol/main.py

(Node.DIRECTORY, Node.FILE): Replace str with bytes.

(Node.__init__): Do not convert path to str.

(Node.get_name): Treat self.path as bytes.

(Changeset.ADD, Changeset.COPY, Changeset.DELETE, Changeset.EDIT,

Changeset.MOVE): Replace str with bytes.

* subversion/bindings/swig/python/tests/trac/versioncontrol/svn_fs.py

(SubversionRepository.__init__(),

SubversionRepository.get_oldest_rev(),

SubversionRepository.youngest_rev(),

SubversionRepository.next_rev()): Treat self.path and self.scope as bytes.

(SubversionRepository.normalize_path(),

SubversionRepository.get_node()): Treat path as bytes.

(SubversionRepository.get_path_history()): Replace 'unknown' Changeset value

with bytes.

(SubversionRepository.get_deltas()): Replace str 'entry' arg to

repos.svn_repos_dir_delta() with bytes.

(SubversionNode.__init__()):

- Treat self.path and self.scope as bytes.

- Decode path into str for exception message on py3.

(SubversionNode.get_entries()): Treat self.path as bytes.

(SubversionNode.get_properties()): Property values are bytes, not str.

(DiffChangeEditor.open_root()): Return bytes value.

* subversion/bindings/swig/python/tests/trac/versioncontrol/tests/svn_fs.py

Replace all str literals with bytes literals when passed into or returned from

the Subversion API, except exception messages.

(REPOS_URL) Build REPOS_URL as bytes.

Patch By: Yasuhito FUTATSUKI <futatuki@poem.co.jp>

  1. … 25 more files in changeset.
On branch swig-py3: Create and use a wrapper function to efficiently retrieve

lists from dict object on Python 2 and 3.

Wrapping dict methods such as keys() and values() in list() works to create a

independent list, but on Python 2 two distinct lists are created which is

inefficient.

* subversion/bindings/swig/python/svn/core.py

Create a new utility function, _as_list(), and remove now unnecessary

_bi_list alias.

(svn.core._unprefix_names, svn.core.__all__): Use new _as_list() function in

place of list() on dict output.

* subversion/bindings/swig/python/svn/client.py,

subversion/bindings/swig/python/svn/delta.py,

subversion/bindings/swig/python/svn/diff.py,

subversion/bindings/swig/python/svn/fs.py,

subversion/bindings/swig/python/svn/ra.py,

subversion/bindings/swig/python/svn/repos.py,

subversion/bindings/swig/python/svn/wc.py,

subversion/bindings/swig/python/tests/mergeinfo.py,

subversion/bindings/swig/python/tests/ra.py,

subversion/bindings/swig/python/tests/repository.py,

subversion/bindings/swig/python/tests/trac/versioncontrol/svn_fs.py,

subversion/bindings/swig/python/tests/wc.py

(svn.client.__all__,

svn.delta.__all__,

svn.diff.__all__,

svn.fs.__all__,

svn.fs.entries,

svn.ra.__all__,

svn.repos.__all__,

svn.wc.__all__,

SubversionMergeinfoTestCase.test_mergeinfo_leakage__incorrect_range_t_refcounts,

SubversionRepositoryAccessTestCase.test_delta_driver_commit,

SubversionRepositoryAccessTestCase.test_get_log2,

SubversionChangeset.get_changes,

SubversionChangeset.get_properties,

SubversionRepositoryTestCase.test_get_logs,

SubversionWorkingCopyTestCase.test_entries_read,

SubversionWorkingCopyTestCase.test_diff_editor4):

Use new _as_list() function in place of list() on dict output.

Found By: danielsh, brane

  1. … 12 more files in changeset.
On branch swig-py3: Fix additional Bytes versus String inconsistencies.

Be explicit about UTF-8 conversions between bytes and strings. All

swig unit tests now pass under both Python 2 and 3.

* subversion/bindings/swig/include/svn_types.swg

(typemap(argout) unsigned char digest[ANY], unsigned char *digest):

Treat checksum digest data correctly as bytes, not a string.

* subversion/bindings/swig/python/libsvn_swig_py/swigutil_py.c

(write_handler_pyio): Create a compatibility definition, SVN_SWIG_BYTES_FMT, to

ensure raw file data is treated as bytes in Python 3 and string in Python2.

* subversion/bindings/swig/python/tests/delta.py,

subversion/bindings/swig/python/tests/ra.py,

subversion/bindings/swig/python/tests/repository.py,

subversion/bindings/swig/python/tests/utils.py,

subversion/bindings/swig/python/tests/wc.py,

subversion/bindings/swig/python/tests/trac/versioncontrol/tests/svn_fs.py

(DeltaTestCase.testTxWindowHandler,

DeltaTestCase.testTxdeltaWindowT,

SubversionRepositoryAccessTestCase.test_get_file,

SubversionRepositoryTestCase.test_dump_fs2,

Temper.alloc_known_repo,

SubversionWorkingCopyTestCase.test_diff_editor4.write_diff,

SubversionRepositoryTestSetup.setUp):

Use BytesIO instead of StringIO to servce as a source of raw binary data.

(SubversionRepositoryTestCase.test_parse_fns3):

Open raw data file in binary mode.

(SubversionWorkingCopyTestCase.test_commit):

Explicity encode data being written to a file opened in binary mode and

decode binary data in the result from binascii.b2a_hex().

  1. … 7 more files in changeset.
On branch swig-py3: Ensure Python file objects are closed before destruction.

* subversion/bindings/swig/python/tests/trac/versioncontrol/tests/svn_fs.py,

subversion/bindings/swig/python/tests/utils.py,

subversion/bindings/swig/python/tests/wc.py

(SubversionRepositoryTestSetup.setUp,

SubversionWorkingCopyTestCase.test_get_pristine_copy_path,

Temper.alloc_known_repo):

Wrap bare open() calls in a 'with' block to ensure the file object is closed

before destruction.

  1. … 2 more files in changeset.
On branch swig-py3: Update swig python code to be compatible with both Python 2

and Python3.

Change non-generated python code to use syntax that is valid for both Python 3

and Python 2.7. Where this is not possible, provide compatibility wrappers.

Python 2 unit tests all pass after the changes, but more effort remains for full

Python 3 functionality.

* build/generator/swig/header_wrappers.py

(_write_swig_interface_file): For Python3, include an encoding argument to

tempfile.TemporaryFile()

* build/generator/util/executable.py

(output): Ensure the subprocess' output is interpreted as utf8 for proper

Python 3 compatibility.

[in subversion/bindings/swig/python]

* svn/core.py

Import the builtin 'list' as '_bi_list' to aid in compatibility in other

modules.

(_unprefix_names): Wrap dict.items() in list() for py2.7/3 support.

(__all__): Change filter() to list comprehension for py2.7/3 support.

* svn/client.py

svn/delta.py

svn/diff.py

svn/fs.py

svn/ra.py

svn/repos.py

svn/wc.py

(__all__): Change filter() to list comprehension for py2.7/3 support. Use the

_bi_list to provide a new list instance of locals on both Python 2.7 and 3.

* svn/fs.py

(entries): Wrap dict.items() in _bi_list() for py2.7/3 support.

* tests/auth.py

(test_open): Change deprecated assert_ to assertTrue.

(test_credentials_get_username,

test_credentials_get_simple,

test_credentials_get_ssl_client_cert,

test_credentials_get_ssl_client_cert_pw,

test_credentials_get_ssl_server_trust):

Change deprecated assertEquals to assertEqual and assert_ to assertTrue.

* tests/client.py

(setUp, test_log5, proplist_receiver_trunk, proplist_receiver_dir1,

test_inherited_props, test_update4):

Change deprecated assertEquals to assertEqual.

(testBatonPlay): Change deprecated assertEquals to assertEqual and

assertNotEquals to assertNotEqual.

(test_log3_url, test_platform_providers):

Change deprecated assert_ to assertTrue.

(test_log5): Change deprecated assert_ to assertTrue and assertEquals to

assertEqual.

* tests/core.py

(test_config_enumerate2): Replace dict.iteritems() with dict.items() to be

compatible with py2.7/3. Change deprecated assert_ to assertTrue.

(test_config_enumerate_sections2): Change deprecated assert_ to assertTrue.

* tests/delta.py

(testTxdeltaWindowT): Change deprecated assert_ to assertTrue.

* tests/mergeinfo.py

(test_rangelist_reverse,

test_mergeinfo_leakage__incorrect_range_t_refcounts,

test_mergeinfo_leakage__lingering_range_t_objects_after_del,

inspect_mergeinfo_dict):

Change deprecated assertEquals to assertEqual.

(test_mergeinfo_leakage__incorrect_range_t_refcounts): Wrap dict.items() in

list() for py2.7/3 support. Change deprecated assertEquals to assertEqual.

(inspect_mergeinfo_dict): Change deprecated assert_ to assertTrue.

(compare_mergeinfo_catalogs,

compare_mergeinfos):

Drop usage of dict.keys() for py2.7/3 support.

* tests/ra.py

(test_get_dir2, test_delta_driver_commit, test_get_file_revs,

test_lock, test_get_log2, test_namestring):

Change deprecated assert_ to assertTrue.

(test_delta_driver_commit): Wrap dict.items() in list() for py2.7/3 support.

* tests/repository.py

(test_dump_fs2): Change deprecated assertEquals to assertEqual.

(test_get_logs): Wrap dict.values() in list() for py2.7/3 support.

* tests/run_all.py

(suite): Do not use long() cast when running with Python 3.

* tests/utils.py

Support correct import of StringIO and pathname2url from Python 2.7 and 3.

(file_uri_for_path): Use unqualified pathname2url to support py2.7/3.

* tests/wc.py

(test_is_adm_dir, test_set_adm_dir, test_status_editor,

test_is_normal_prop, test_is_wc_prop, test_is_entry_prop):

Change deprecated assert_ to assertTrue and failIf to assertFalse.

(test_get_adm_dir, test_crawl_revisions2, test_check_wc, test_get_prop_diffs,

test_get_ignores):

Change deprecated assert_ with assertTrue.

(test_commit): Change deprecated assertEquals to assertEqual.

(test_diff_editor4): Wrap dict.items() in list() for py2.7/3 support.

* tests/trac/versioncontrol/svn_fs.py

(get_entries): Drop usage of dict.keys() for py2.7/3 support.

(get_properties, get_changes): Wrap dict.items() with list() for py2.7/3

support.

* tests/trac/versioncontrol/tests/svn_fs.py

Support correct import of StringIO and pathname2url from Python 2.7 and 3.

(test_get_dir_entries, test_get_file_entries, test_get_node_history,

test_get_node_history_follow_copy, test_get_path_history,

test_get_path_history_copied_file, test_get_path_history_copied_dir,

test_changeset_added_dirs, test_changeset_file_edit, test_changset_dir_moves,

test_changeset_file_copy):

Replace the series of direct calls to .next() with assertSequenceEqual for

py2.7/3 support.

(test_diff_file_different_revs, test_diff_file_different_files,

test_diff_file_no_change, test_diff_dir_different_revs,

test_diff_dir_different_dirs, test_diff_dir_no_change,

test_changeset_repos_creation):

Replace <iter>.next() with next(<iter>) for py2.7/3 support.

  1. … 21 more files in changeset.