[TRAFODION-2477] Invalid characters in translation are ignored Right now we ignore such invalid characters and also may truncate the string …
[TRAFODION-2477] Invalid characters in translation are ignoredRight now we ignore such invalid characters and also may truncatethe string at the point of the invalid char. The expected behaviorwould be an error.The only type of invalid data I could create with regular SQL syntaxis an invalid UTF-16 surrogate pair. We have no checks that detectthose when we enter the data. Invalid UTF-8, on the other hand, isrejected when we try to insert it in the database (at least in thecase I tried).The fix adds a check to generate an error (file conversionLocale.cpp).It also adds two CQDs to suppress the error (remaining code files)and to replace the invalid character with a replacement character.Right now we use "?", even for Unicode, which has a special replacementcharacter, see TRAFODION-2515.