Checkout Tools
  • last updated 2 hours ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
On branch swig-py3: accept both of bytes/str for input char * arguments

* Replace typemap(in) for char * using 'parse' modifier with one using

function allows both of bytes/str in py2/py3 in libsvn_swig_py library

* Fix functions to convert Python objects into char pointer to accept

both of bytes/str object in py2/py3

* Fix to accept None as representation of NULL value on conversion

prop dict into apr_hash_t * and apr_array_header_t * of svn_props_t *

[In subversion/bindings/swig]

* core.i (%typemap(in) (const char *data, apr_size_t *len): Allow str

as well as bytes for data argment of svn_stream_write()

* include/svn_global.swg

(remove)(%typemap(in) char *, char const *, char * const,

char const * const): Move this typemap into include/svn_strings as

typemap (in) IN_STRING

* include/svn_string.swg (new)(%typemap(in) IN_STRING): replacement of

%typemap(in) char *, char const *, char * const, char const * const.

actual processing code is moved new svn_swig_py_string_to_cstring()

function in python/libsvn_swig_py/swigutil_py.c

* include/svn_types.swg (%typemap(in) const char *MAY_BE_NULL):

Move processing code into new svn_swig_py_string_to_cstring() function

in python/libsvn_swig_py/swigutil_py.c

* python/libsvn_swig_py/swigutil_py.c

(svn_swig_py_string_to_cstring): New function to convert Python

bytes or str into const char *, with better TypeError exception message

(svn_swig_py_string_type_exception): New function to construct

TypeError exception for new make_string_from_ob_maybe_null() function

(make_string_from_ob, make_svn_string_from_ob):

- Allow str as well as bytes for ob

- Don't raise TypeError exception because all callers don't expect it

(make_string_from_ob_maybe_null, make_svn_string_from_ob_maybe_null):

New function same as make_string_from_ob() and make_svn_string_fromob()

but allows None input represents NULL value and raise TypeError

if input value don't have appropriate type

(svn_swig_py_stringhash_from_dict, svn_swig_py_mergeinfo_from_dict,

svn_swig_py_proparray_from_dict, svn_swig_py_prophash_from_dict,

svn_swig_py_path_rev_hash_from_dict,

svn_swig_py_struct_ptr_hash_from_dict): separate check of key conversion

result and value conversion result

(svn_swig_py_proparray_from_dict, svn_swig_py_prophash_from_dict):

allow NULL for prop values

(svn_swig_py_unwrap_string):

- Allow str as well as bytes for source argument

(svn_swig_py_make_file):

- Allow str as well as bytes for py_file argument as file path

(svn_swig_py_auth_gnome_keyring_unlock_prompt_func):

- Use new function make_string_from_ob_maybe_null() instead of

make_string_from_ob() to check TypeError

- Report Python exception caused by Python callback function as

callback exception error

* python/libsvn_swig_py/swigutil_py.h

Expose new public function make_string_from_ob_maybe_null(), which is

used by typemap(in) char IN_STRING, typemap(in) const char *MAY_BY_NULL

* python/tests/client.py

(SubversionClientTestCase.log_entry_receiver_whole): New helper

callback function for new SubversionClientTestCase.test_log5_revprops

(SubversionClientTestCase.test_log5_revprops): new test for

typemap(in) apr_array_t *STRINGLIST and its helper function

svn_swig_py_unwrap_string()

* python/tests/core.py

(SubversionCoreTestCase.test_stream_write_exception):

- As unicode input is now valid, use int value as invalid input

- Add case to be expected to cause UnicodeEncodeError

(SubversionCoreTestCase.test_stream_write_str):(Only for Python 3)

New test case for svn_stream_write() to pass str object as data argument

(SubversionCoreTestCase.test_stream_write_bytes):

Renamed from SubversionCoreTestCase.test_stream_write

* python/tests/run_all.py: Register new test module typemap

* python/tests/typemap.py: New unittest module for typemaps

(SubversionTypemapTestCase): New unittest subclass for unit test about

typemaps

(SubversionTypemapTestCase.test_char_ptr_in): New test case

(SubversionTypemapTestCase.test_char_ptr_in_unicode_exception):

New test case

(SubversionTypemapTestCase.test_char_ptr_may_be_null): New test case

(SubversionTypemapTestCase.test_char_ptr_may_be_null_unicode_exception):

New test case

(SubversionTypemapTestCase.test_make_string_from_ob): New test case

(SubversionTypemapTestCase.test_prophash_from_dict_null_value):

New test case

(SubversionTypemapTestCase.test_make-string_ob_unicode_exception):

New test case

(SubversionTypemapTestCase.test_make_svn_string_ob_unicode_exception):

New test case

(suite): New function to drive SubversionTypemapTestCase

  1. … 9 more files in changeset.
On branch swig-py3: Use bytes for all API string parameters.

Since many string like arguments in the API can contain arbitrary bytes, use the

Python Bytes interface in Python 3, while still using Str in Python 2, to ensure

full compatibility with all possible input data.

Build system change:

* build/ac-macros/swig.m4:

Add "-DPY3" option to SWIG_PY_OPTS for py3 build to define symbol to switch

typemap between py2 and py3.

Typemap changes:

* subversion/bindings/swig/core.i

(%typemap(argout) (char *buffer, apr_size_t *len):

Map buffer arg to Bytes insted of Str.

(%typemap(in) (const char *data, apr_size_t *len) ($*2_type temp):

Map data arg to Bytes instead of Str and pass Py_ssize_t variable as length

argment of PyBytes_AsStringAndSize for for explicit type conversion.

(%typemap(in) const void *value):

Map value arg to Bytes instead of Str for svn_auth_set_parameter().

* subversion/bindings/swig/include/svn_global.swg:

- Define SWIG_PYTHON_STRICT_BYTE_CHAR symbol for swig context for

Python3.

- Switch "in" type typemap for char *, char const *, char * const,

and char const * const "", to use parse parameter "y" for py3 and

"s" for py2.

* subversion/bindings/swig/include/svn_string.swg

(%typemap(argout) RET_STRING, %typemap(in) svn_stringbuf_t *,

%typemap(out) svn_stringbuf_t *, %typemap(in) const svn_string_t *,

%typemap(out) svn_string_t *, %typemap(argout) const char **OUTPUT):

Map args to Bytes instead of Str.

* subversion/bindings/swig/include/svn_swigcompat.swg

(%set_constant(name, value): Use PyDict_SetItem and PyBytes_FromString

instead of PyDict_SetItemString, for SWIG <= 1.3.27 (not tested).

* subversion/bindings/swig/include/svn_types.swg

(%typemap(in) const char *MAY_BE_NULL): Don't use "z" conversion

format and convert to Bytes if it is not NULL.

(%typemap(in) (const char *PTR, apr_size_t LEN): Use PyBytes_Check and

PyBytes_AsStringAndSize instead of PyStr_Check and PyStr_AsUTF8AndSize.

* subversion/bindings/swig/python/libsvn_swig_py/swigutil_py.c

(make_string_from_ob(), make_svn_string_from_ob(),

svn_swig_py_make_file(), svn_swig_py_get_commit_log_func()):

Use PyBytes_Check and PyBytes_AsString instead of PyStr_Check and

PyStr_AsString.

(convert_hash()): Use Dict_SetItem and PyBytes_FromString instead of

DictItemString.

(convert_svn_string_t(), svn_swig_py_proparray_to_dict(),

ra_callbacks_get_wc_prop()):

Use PyBytes_AsStringAndSize instead of PyStr_AsUTF8AndSize.

(cstring_to_pystring(), convert_string(),

svn_swig_py_propinheriteditemarray_to_dict(),

svn_swig_py_proparray_to_dict(),

svn_swig_py_locationhash_to_dict(),

svn_swig_py_c_strings_to_list(),

svn_swig_py_changed_path_hash_to_dict(),

svn_swig_py_changed_path2_hash_to_dict(),

svn_swig_py_unwrap_string(),

svn_swig_py_array_to_list()): Use PyBytes_FromString instead of

PyStr_FromString.

(delete_entry(), add_directory(), open_directory(), change_dir_prop(),

add_file(), open_file(), apply_textdelta(), change_file_prop(),

close_file(), parse_fn3_uuid_record(), parse_fn3_set_revision_property(),

parse_fn3_set_node_property(), parse_fn3_delete_node_property(),

svn_swig_py_notify_func(), svn_swig_py_status_func(),

svn_swig_py_delta_path_driver_cb_func(), svn_swig_py_status_func2(),

svn_swig_py_fs_lock_callback(), svn_swig_py_repos_authz_func(),

svn_swig_py_repos_history_func(), svn_swig_py_proplist_receiver2(),

svn_swig_py_log_receiver(), svn_swig_py_info_receiver_func(),

svn_swig_py_client_blame_receiver_func(),

svn_swig_py_changelist_receiver_func(),

svn_swig_py_auth_gnome_keyring_unlock_prompt_func(),

svn_swig_py_auth_simple_prompt_func(),

svn_swig_py_auth_username_prompt_func(),

svn_swig_py_auth_ssl_server_trust_prompt_func(),

svn_swig_py_auth_ssl_client_cert_prompt_func(),

svn_swig_py_auth_ssl_client_cert_pw_prompt_func(),

svn_swig_py_config_auth_walk_func(), ra_callbacks_get_wc_prop(),

ra_callbacks_push_or_set_wc_prop(), ra_callbacks_invalidate_wc_props(),

svn_swig_py_commit_callback(), svn_swig_py_ra_file_rev_handler_func(),

svn_swig_py_ra_lock_callback(), reporter_set_path(),

reporter_delete_path(), reporter_link_path(),

wc_diff_callbacks2_file_changed_or_added(),

wc_diff_callbacks2_file_deleted(), wc_diff_callbacks2_dir_added(),

wc_diff_callbacks2_dir_deleted(),

wc_diff_callbacks2_dir_props_changed(),

svn_swig_py_config_enumerator2(),

svn_swig_py_config_section_enumerator2()): Use 's' for py2 and use 'y'

for py3 in argment format string.

(ra_callbacks_push_or_set_wc_prop()): Use PyBytes_FromStringAndSize

instead of PyStr_FromStringAndSize.

(ra_callbacks_get_client_string()): Use PyBytes_FromString instaed of

PyStr_AsString.

* subversion/bindings/swig/svn_client.i

(%typemap(argout) apr_array_header_t **props):

Use PyBytes_FromStringAndSize() instead of PyStr_FromStringAndSize.

Helper python code changes:

* subversion/bindings/swig/python/svn/core.py

(svn_path_compare_paths): Do not use cmp, for py3 compatibility.

(Stream.read()): Chunks is now list of bytes, not list of str.

* subversion/bindings/swig/python/svn/fs.py

(FileDiff.get_pipe()):

- Make sure self.tempfile1 and self.self.tempfile2 are bytes.

- Pass bytes to header_encoding arg in _svndiff.file_output_unified4().

* subversion/bindings/swig/python/svn/ra.py

(Callbacks..__doc__):

Treat path as bytes in sample code.

* subversion/bindings/swig/python/svn/repos.py

(ChangeCollector._make_base_path()): Treat path as bytes.

(ChangeCollector.open_root(): Path and basepath should be bytes in

dir_baton.

(RevisionChangeCollector._make_base_path()): Treat path as bytes.

Unit test changes:

* subversion/bindings/swig/python/tests/auth.py:

Replace all str literals with bytes literals except docstring and module name.

* subversion/bindings/swig/python/tests/checksum.py

(ChecksumTestCases.test_checksum()): Check if type of check_val is

bytes instead of str.

* subversion/bindings/swig/python/tests/client.py:

Replace all str literals with bytes literals except for the mode arg of open(),

arguments to utils.Temper methods, docstrings, and module names.

(SubversionClientTestCase.test_uuid_from_url): Check if return value

type client.uuid_from_url() is bytes instead of str.

(SubversionClientTestCase.test_uuid_from_path): Check if return value

type client.uuid_from_path() is bytes instead of str.

(SubversionClientTestCase.test_merge_peg3()): Open the result file in

raw mode.

(SubversionClientTestCase.test_update4()): Convert os.path.sep into

bytes if it is str in py3.

* subversion/bindings/swig/python/tests/core.py:

Replace all str literals with bytes literals except for exception messages,

arguments to utils.Temper methods, docstrigs, and module names.

Add new tests for svn_stream_*()

(SubversionCoreTestCase.test_stream_from_stringbuf()): New test method.

(SubversionCoreTestCase.test_stream_read_full()): New test method.

(SubversionCoreTestCase.test_stream_read2()): New test method.

(SubversionCoreTestCase.test_stream_write_exception()): New test method.

(SubversionCoreTestCase.test_stream_write()): New test method.

(SubversionCoreTestCase.test_stream_readline()): New test method.

* subversion/bindings/swig/python/tests/delta.py

(DeltaTestCase.estTxWindowHandler_stream_IF(),

DeltaTestCase.estTxWindowHandler_Stream_IF()):

Use bytes literals to make stream when using

svn.core.svn_stream_from_stringbuf().

Use bytes file name to make stream when using

svn.core.svn_stream_from_aprfile2().

* subversion/bindings/swig/python/tests/fs.py

(SubversionFSTestCase.log_message_func(): log_msg_func3 callback now

expects bytes to return.

(SubversionFSTestCase.setUp()): Pass bytes path and url to client.import2().

(SubversionFSTestCase.test_diff_repos_paths_internal(),

SubversionFSTestCase.test_diff_repos_paths_external()): Use bytes for

path1 argument to fs.FileDiff.__init__().

* subversion/bindings/swig/python/tests/mergeinfo.py

(SubversionMergeinfoTestCase.TEXT_MERGEINFO1,

SubversionMergeinfoTestCase.TEXT_MERGEINFO2,

SubversionMergeinfoTestCase.MERGEINFO_SRC): Use bytes instead of str.

(SubversionMergeinfoTestCase.test_mergeinfo_get()):

- Pass list of bytes for paths arg to repos.fs_get_mergeinfo().

- Replace str in expected_mergeinfo with bytes.

* subversion/bindings/swig/python/tests/ra.py

Replace all str literals with bytes literals except assertion message,

arguments to utils.Temper methods, docstrings, and module names.

(SubversionRepositoryAccessTestCase.test_get_file): Compare file

content as bytes.

* subversion/bindings/swig/python/tests/repository.py

Replace all str literals with bytes literals when passed into or returned from

the Subversion API, except exception messages.

(SubversionRepositoryTestCase.test_dump_fs2()): Treat dump and

feedback as bytes and compose expected_feedback as bytes.

* subversion/bindings/swig/python/tests/utils.py

(Temper.alloc_empty_dir(),

Temper.alloc_empty_repo): Convert return value of tempfile.mkdtemp() into

bytes.

(Temper.file_uri_for_path()): Return URI as bytes.

* subversion/bindings/swig/python/tests/wc.py

Replace all str literals with bytes literals when passed into or returned from

the Subversion API, except exception messages.

(SubversionWorkingCopyTestCase.test_get_adm_dir()): Check type as

bytes.

(SubversionWorkingCopyTestCase.test_get_pristine_copy_path(),

SubversionWorkingCopyTestCase.test_diff_editor4()): Open file in raw mode.

(SubversionWorkingCopyTestCase.test_commit()): Use bytes literals

instead of str.

* subversion/bindings/swig/python/tests/trac/versioncontrol/main.py

(Node.DIRECTORY, Node.FILE): Replace str with bytes.

(Node.__init__): Do not convert path to str.

(Node.get_name): Treat self.path as bytes.

(Changeset.ADD, Changeset.COPY, Changeset.DELETE, Changeset.EDIT,

Changeset.MOVE): Replace str with bytes.

* subversion/bindings/swig/python/tests/trac/versioncontrol/svn_fs.py

(SubversionRepository.__init__(),

SubversionRepository.get_oldest_rev(),

SubversionRepository.youngest_rev(),

SubversionRepository.next_rev()): Treat self.path and self.scope as bytes.

(SubversionRepository.normalize_path(),

SubversionRepository.get_node()): Treat path as bytes.

(SubversionRepository.get_path_history()): Replace 'unknown' Changeset value

with bytes.

(SubversionRepository.get_deltas()): Replace str 'entry' arg to

repos.svn_repos_dir_delta() with bytes.

(SubversionNode.__init__()):

- Treat self.path and self.scope as bytes.

- Decode path into str for exception message on py3.

(SubversionNode.get_entries()): Treat self.path as bytes.

(SubversionNode.get_properties()): Property values are bytes, not str.

(DiffChangeEditor.open_root()): Return bytes value.

* subversion/bindings/swig/python/tests/trac/versioncontrol/tests/svn_fs.py

Replace all str literals with bytes literals when passed into or returned from

the Subversion API, except exception messages.

(REPOS_URL) Build REPOS_URL as bytes.

Patch By: Yasuhito FUTATSUKI <futatuki@poem.co.jp>

  1. … 25 more files in changeset.
On branch swig-py3: Complete Python SWIG bindings for svn_client.h.

* subversion/bindings/swig/include/svn_containers.swg

(apr_array_header_t **OUTPUT_OF_CONST_CHAR_P): Add additional "char *" array

maps.

(%typemap(argout) apr_array_header_t **options): New output mapping for array

of svn_client_conflict_option_t objects.

(%typemap(argout) apr_array_header_t **versions_p): New output mapping for

array of svn_client__shelf_version_t objects.

* subversion/bindings/swig/include/svn_string.swg

(const char **OUTPUT): Add additional char * output parameter for log_message.

* subversion/bindings/swig/include/svn_types.swg

(SWIGTYPE **OUTPARAM): Add several missing output parameter swig types.

* subversion/bindings/swig/python/libsvn_swig_py/swigutil_py.c,

subversion/bindings/swig/python/libsvn_swig_py/swigutil_py.h

Add make_ob_* swig constructor for svn_client_status_dup.

(svn_swig_py_client_status_func): New Function.

* subversion/bindings/swig/svn_client.i

Add callback typemaps for usages of svn_client_status_func_t.

* subversion/bindings/swig/python/tests/client.py

(SubversionClientTestCase.assert_all_instances_of): New helper method.

(SubversionClientTestCase.test_platform_providers): Add usage of

assert_all_instances_of() helper method.

(SubversionClientTestCase.test_conflict): New test method.

(SubversionClientTestCase.test_shelf): New test method.

  1. … 6 more files in changeset.
On branch swig-py3: Move py3c compat fixes to all swig modules.

Resolves compile warnings in Python 3 build for all modules, not just those in

swigutil_py.c.

* subversion/bindings/swig/python/libsvn_swig_py/swigutil_py3c.h:

New file containing the py3c compat undef fixes previously in swigutil_py.c.

* subversion/bindings/swig/python/libsvn_swig_py/swigutil_py.c:

Moved py3c compat undef fixes to swigutil_py3c.h and include the header.

* subversion/bindings/swig/include/svn_global.swg:

Add swigutil_py3c.h include to ensure all swig modules get the compat fixes.

* subversion/bindings/swig/core.i,

subversion/bindings/swig/include/svn_string.swg,

subversion/bindings/swig/include/svn_types.swg,

subversion/bindings/swig/svn_client.i,

subversion/bindings/swig/svn_delta.i,

subversion/bindings/swig/svn_diff.i,

subversion/bindings/swig/svn_fs.i,

subversion/bindings/swig/svn_ra.i,

subversion/bindings/swig/svn_repos.i,

subversion/bindings/swig/svn_wc.i:

Remove redundant py3c.h inclusion, now that svn_global.swg includes

swigutil_py3c.h.

Found By: danielsh

  1. … 12 more files in changeset.
On branch swig-py3: Use py3c library in Python swig bindings.

Add the py3c Python compatibility library and update the python swig bindings

to use the compatibility functions that it provides. This is the first step to

getting the swig bindings to support Python 3.

* build/ac-macros/py3c.m4:

Create a new ac-macro for the py3c library.

* aclocal.m4:

Include the new py3c.m4 file.

* build/ac-macros/swig.m4:

(SVN_FIND_SWIG): Call the new SVN_PY3C macro to add py3c to the build.

[in subversion/bindings/swig]

* include/svn_string.swg

(python typemaps: RET_STRING): Replace PyString_FromStringAndSize with

PyStr_FromStringAndSize.

(python typemaps: (in) svn_stringbuf_t *,

(in) const svn_string_t * (svn_string_t value)):

Replace PyString_Check with PyStr_Check and

PyString_AS_STRING/PyString_GET_SIZE with PyStr_AsUTF8AndSize.

(python typemaps: (out) svn_stringbuf_t *,

(out) svn_string_t *):

Replace PyString_FromStringAndSize with PyStr_FromStringAndSize.

(python typemaps: (argout) const char **OUTPUT):

Replace PyString_FromString with PyStr_FromString.

* include/svn_types.swg

(python typemaps: (in) (const char *PTR, apr_size_t LEN)):

Replace PyString_Check with PyStr_Check and

PyStr_AS_STRING/PyString_GET_SIZE with PyStr_AsUTF8AndSize.

(python typemaps: (argout) unsigned char digest[ANY]):

Replace PyString_FromStringAndSize with PyStr_FromStringAndSize.

(python typemaps: (in) const unsigned char *digest):

Replace PyString_AsString with PyStr_AsString.

* include/libsvn_swig_py/swigutil_py.h,

include/libsvn_swig_py/swigutil_py.c

Include py3c.h to pick up Python 2/3 compatibility defines.

(svn_swig_py_as_file): Add new compatibility function.

(svn_swig_py_unwrap_string): Add clarifying note to function.

(svn_swig_py_svn_exception,

svn_swig_py_propinheriteditemarray_to_dict,

svn_swig_py_proparray_to_dict,

svn_swig_py_locationhash_to_dict,

svn_swig_py_c_strings_to_list,

svn_swig_py_array_to_list,

cstring_to_pystring,

convert_string):

Replace PyString_FromString with PyStr_FromString.

(make_string_from_ob,

make_svn_string_from_ob,

svn_swig_py_unwrap_string,

ra_callbacks_get_client_string,

exeception_to_error):

Replace PyString_AsString with PyStr_AsString.

(make_string_from_ob, make_svn_string_from_ob):

Replace PyString_Check with PyStr_Check.

(convert_svn_string_t,

svn_swig_py_proparray_to_dict,

ra_callbacks_push_or_set_wc_prop):

Replace PyString_FromStringAndSize with PyStr_FromStringAndSize.

(svn_swig_py_make_file):

Replace PyFile_Check/PyFile_AsFile with svn_swig_py_as_file.

(read_handler_pyio, ra_callbacks_get_wc_prop):

Replace PyString_GET_SIZE/PyString_AS_STRING with PyStr_AsUTF8AndSize.

(svn_swig_py_get_commit_log_func):

Replace PyString_AS_STRING with PyStr_AsString and PyString_Check with

PyStr_Check.

* core.i

(python typemap:

(char *buffer, apr_size_t *len)):

Replace PyString_FromStringAndSize with PyStr_FromStringAndSize.

(python typemap:

(const char *data, apr_size_t *len)):

Replace PyString_Check with PyStr_Check, and PyString_AS_STRING/

PyString_GET_SIZE with PyStr_AsUTF8AndSize.

(python typemap:

const void *value (apr_pool_t *_global_pool, PyObject *_global_py_pool)):

Replace PyString_Check with PyStr_Check and PyString_AS_STRING with

PyStr_AsString.

(python typemap: FILE *): Replace PyFile_AsFile with svn_swig_py_as_file.

* svn_client.i

Include py3c.h to pick up Python 2/3 compatibility defines.

(python typemap: (argout) apr_array_header_t **props):

Replace PyString_FromStringAndSize with PyStr_FromStringAndSize.

* svn_delta.i,

svn_diff.i,

svn_fs.i,

svn_ra.i,

svn_repos.i,

svn_wc.i

Include py3c.h to pick up Python 2/3 compatibility defines.

  1. … 14 more files in changeset.