incorrect isSpecialChar() in collation rules parser

General

Trac Data

Other Data

General

Trac Data

Other Data

Description

Java 6 RuleBasedCollator documents that the collation rule syntax characters are `[\u003A-\u0040 \u005B-\u0060 \u007B-\u007E|\u0021-\u002F]` but ICU's isSpecialChar() (C ucol_tok_isSpecialChar()) incorrectly

includes U+0020 space
excludes U+0040 at sign @
excludes U+007C vertical line |

The space should not be both whitespace and reserved for syntax.

We do use | in the LDML/ICU collation syntax, and the parser also recognizes @ (see https://unicode-org.atlassian.net/browse/ICU-9956#icft=ICU-9956) so they really should be "special".

Activity

UnicodeBot

July 1, 2018 at 12:11 AM

Trac Comment 3 by @Markus Scherer—2014-02-28T05:44:03.901Z

The new code has

UBool
CollationRuleParser::isSyntaxChar(UChar32 c) {
    return 0x21 <= c && c <= 0x7e &&
            (c <= 0x2f || (0x3a <= c && c <= 0x40) ||
            (0x5b <= c && c <= 0x60) || (0x7b <= c));
}

Resize issue view side panel

Fixed

Details

Assignee

Markus Scherer

Reporter

Markus Scherer

Components

collation

Labels

fixed-collv2

Priority

medium

Time Needed

Hours

Fix versions

53.1 (release)

Created June 28, 2018 at 5:20 PM

Updated October 3, 2018 at 10:53 PM

Resolved July 1, 2018 at 8:43 PM