When StringTokenizer is used with returnDelims set to true, it is returning one
token per run of delimiters. The API docs (like Java's API docs) say that it
returns one token per delimiter. This is the Java StringTokenizer behavior.
This is a compatibility issue for people migrating from Java's StringTokenizer
to ours. We should adhere to our docs and Java's behavior.
In order to support existing users we can continue to support the existing
behavior by adding new constructors. This will be a compatibility change for
them and they'd have to migrate to the new APIs.
Wed Feb 15 17:47:16 2006 grhoten moved from incoming to strings
Fri May 5 11:35:15 2006 dougfelt changed notes2: review: "" to "ram",
Mon Aug 7 15:30:39 2006 ram moved from strings to fixed
Sun Oct 22 07:35:29 2006 grhoten moved from fixed to closed