Pseudo locale functionality broken for Android integration.

Description

Reported by Nikita Iashchenko:


Failure log (when attempting to build ICU 67.1 using production data created from CLDR 37):

D_LIBRARY_PATH=../lib:../stubdata:../tools/ctestfw:$LD_LIBRARY_PATH ../bin/genrb -s /home/build/icu/icu4c/source/data/locales -d ./out
/build/icudt67l/ -i ./out/build/icudt67l --usePoolBundle ./out/build/icudt67l/ -k eo_001.txt
/home/build/icu/icu4c/source/data/locales/en_XA.txt:19: duplicate key 'AmPmMarkers' in table, first appeared at line 11
/home/build/icu/icu4c/source/data/locales/en_XA.txt:19: parse error. Stopped parsing table with U_UNSUPPORTED_ERROR
/home/build/icu/icu4c/source/data/locales/en_XA.txt:10: parse error. Stopped parsing resource with U_UNSUPPORTED_ERROR
/home/build/icu/icu4c/source/data/locales/en_XA.txt:9: parse error. Stopped parsing resource with U_UNSUPPORTED_ERROR
couldn't parse the file en_XA.txt. Error:U_UNSUPPORTED_ERROR
LD_LIBRARY_PATH=../lib:../stubdata:../tools/ctestfw:$LD_LIBRARY_PATH ../bin/genrb -s /home/build/icu/icu4c/source/data/locales -d ./out
/build/icudt67l/ -i ./out/build/icudt67l --usePoolBundle ./out/build/icudt67l/ -k es.txt
LD_LIBRARY_PATH=../lib:../stubdata:../tools/ctestfw:$LD_LIBRARY_PATH ../bin/genrb -s /home/build/icu/icu4c/source/data/locales -d ./out
/build/icudt67l/ -i ./out/build/icudt67l --usePoolBundle ./out/build/icudt67l/ -k es_419.txt
make[1]: *** [../data/rules.mk:2699: out/build/icudt67l/en_XA.res] Error 16
make[1]: *** Waiting for unfinished jobs....
make[1]: Leaving directory '/home/build/build-icu/data'
make: *** [Makefile:153: all-recursive] Error 2


This is due to the pseudo-locale logic incorrectly returning resolved "root" paths like aliases (which makes it appear that there are two of the same resource bundle keys (e.g. since there are paths for both "AmPmMarkers" and "AmPmMarkers:alias").

There are two things here:
1) The pseudo-locale logic should not be returning aliases (or probably anything else from "root").
2) Steven Lumis thinks it's odd that we're seeing aliases in "resolved" CLDR data at all (need to check with Mark, but it might be an issue in CLDRFile et al).

Activity

Show:
Markus Scherer
October 9, 2020, 7:06 PM

Closing. If someone finds more problems and provides quick fixes soon, in time for 68, we can reopen for those. If later, please create a new ticket.

David Beaumont
October 9, 2020, 6:02 PM

I think so. The aliases should not be being read now, but since the new pseudo locale stuff was not previously well tested I might have missed something. The best way to know would be to repeat the genrb stuff and see.

Markus Scherer
October 9, 2020, 5:14 PM

Is this done, or is there more work to do?

David Beaumont
May 27, 2020, 6:29 PM

The other ticket relates to code that’s now entirely unused, so I don’t think there’s any overlap.

I did discuss the specific use of RLM codes in the pseudo-localizer with Mark though, and what it’s doing now is apparently best practice (I’m not familiar with this enough to do anything other than follow Mark’s suggestions).

Shane Carr
May 27, 2020, 6:17 PM

I thought they might be related because they both have to do with ICU failures when a pseudolocale is used. I've only done a cursory look at both tickets, so maybe the root causes of the two issues are unrelated.

Fixed
Your pinned fields
Click on the next to a field label to start pinning.

Assignee

David Beaumont

Reporter

David Beaumont

Components

Labels

Priority

major

Time Needed

Hours

Fix versions