Skip to Content.
Sympa Menu

xom-interest - Re: [XOM-interest] East Asian Character Sets

xom-interest AT lists.ibiblio.org

Subject: XOM API for Processing XML with Java

List archive

Chronological Thread  
  • From: John Cowan <cowan AT ccil.org>
  • To: Elliotte Harold <elharo AT metalab.unc.edu>
  • Cc: xom-interest AT lists.ibiblio.org
  • Subject: Re: [XOM-interest] East Asian Character Sets
  • Date: Thu, 24 Jun 2004 01:17:44 -0400

Elliotte Harold scripsit:

> Comments? Thoughts? Suggestions? I'd be particularly interested in
> hearing from anyone who knows if these issues really do apply to Chinese
> and/or Korean character sets. I'm guessing the same issues arise, but I
> could be wrong about that, especially for Korean which I haven't looked
> into yet.

They apply to all of the CJK encodings. There are two main issues:

1) ASCII variants. ISO 646 used to allow the creation of national
variants by reassigning certain characters (IIRC they were #@[\]^_{|}~).
the CJK encodings exploited this opportunity by doing things like
putting YEN SIGN in the place of \. Localizations of DOS and other
operating systems tended to ignore this, however, so Japanese DOS users
saw pathnames separated by YEN SIGNs instead of \s, and C programmers
saw escapes starting with YEN SIGNs as well. Therefore, the same byte
can represent either of two Unicode characters. AFAIK Chinese was not
affected by this.

2) Definition-by-glyph. There are no definitive mappings for Western
characters included in the CJK encodings, because characters were
defined in those standards by specifying normative glyphs. The Han
unification effort nailed down for all Han characters exactly what
corresponds to which Unicode value (the results of which are available
in http://www.unicode.org/Public/UNIDATA/Unihan.txt (warning: 26 MB).
The other characters, however, have to be mapped by common sense, and
intelligent people can disagree about the most appropriate mappings.

--
John Cowan cowan AT ccil.org www.ccil.org/~cowan www.reutershealth.com
"In the sciences, we are now uniquely privileged to sit side by side
with the giants on whose shoulders we stand."
--Gerald Holton




Archive powered by MHonArc 2.6.24.

Top of Page