This is embarrassing...
I found bugs in collperf2.cpp that I introduced in r34836 (2014jan07). The input UTF-16 strings are not NUL-terminated (getData16()), which yields bogus UnicodeString objects which means that the performance results when using UnicodeString are useless. The results from testing NUL-terminated `UChar *` strings are probably also dubious.
In addition, the sorting and shuffling do not work right (sortData16()).
I am re-running the performance tests. It looks like this explains the finding that UTF-16 is at least 3 times as fast as UTF-8 for sorting & searching... it isn't that much faster.
I would like to fix this in the trunk, merge it into maint-53, and also copy the fixed collperf2 code to maint-52 where it does not exist yet but where it will be handy for comparison.
Fixed in trunk r35675, merged into maint-53 r35676, copied collperf2 to maint-52 r35679.