in exhaustive mode for milestone:4.5.1 multiple platforms
Looks like revision 28193, which refactored testCEValidity() and checkCEValidity() caused this failure. This test now stipulates that the tertiary should be more than 2. For locale "ur", one piece of data has P = 0, S = 0, T = 2.
Need to check more to understand this.
Note: Byte 02 is used as a separator in merged sort keys. When comparing strings, or using sort keys without merging them, 02 is harmless. Still, this is pretty bad.
Consider working on tickets #7757 (use more byte values in collation tailorings) and #7788 (CE: Tertiary Byte out of range) together, creating a more maintainable C++ class for the weight iterator and making it know about the byte value ranges in all levels.
For weight byte values see http://site.icu-project.org/design/collation/bytes
Yes, for locale "ur", this happens for codepoint U+0611
I think yesterday I fixed this bug in C++ as part of the UCA 6.0 work where the code used to turn a 00 lower-bound-weight into 01, potentially resulting in 02 weights when tailoring after an ignorable weight. The fix was to pin the lower-bound-weight to at least 02. This should fix Urdu where we have &\u0610<<<\u0611 and U+0610 is completely ignorable. The old code probably goes back to before we added the merge-sort-key separator byte 02.
I confirm that the exhaustive test is now passing (at least, this particular one)… thanks!