Clone
Hans Zeller
committed
on 01 Mar 17
[TRAFODION-2477] Invalid characters in translation are ignored
Right now we ignore such invalid characters and also may truncate
the string … Show more
[TRAFODION-2477] Invalid characters in translation are ignored

Right now we ignore such invalid characters and also may truncate

the string at the point of the invalid char. The expected behavior

would be an error.

The only type of invalid data I could create with regular SQL syntax

is an invalid UTF-16 surrogate pair. We have no checks that detect

those when we enter the data. Invalid UTF-8, on the other hand, is

rejected when we try to insert it in the database (at least in the

case I tried).

The fix adds a check to generate an error (file conversionLocale.cpp).

It also adds two CQDs to suppress the error (remaining code files)

and to replace the invalid character with a replacement character.

Right now we use "?", even for Unicode, which has a special replacement

character, see TRAFODION-2515.

Show less

default + 4 more