Skip to Content.
Sympa Menu

xom-interest - [XOM-interest] Re: Unicode forms for internal storage

xom-interest AT lists.ibiblio.org

Subject: XOM API for Processing XML with Java

List archive

Chronological Thread  
  • From: Markus Scherer <markus.scherer AT jtcsv.com>
  • To: unicode AT unicode.org
  • Cc: xom-interest AT lists.ibiblio.org
  • Subject: [XOM-interest] Re: Unicode forms for internal storage
  • Date: Tue, 20 Jan 2004 09:52:40 -0800

You need not invent something new: Just use a simplified SCSU encoder, and either a regular SCSU decoder or one that only supports the features which your custom encoder uses.

For a tiny SCSU encoder (main function 75 lines of commented C) that also compresses a little better than what you describe see http://www.mindspring.com/~markus.scherer/unicode/tr6/

You could scale that encoder up or down to your liking.

For a full SCSU converter you could use ICU, for example.
http://oss.software.ibm.com/icu/

You could also use BOCU-1.

With ICU you need not write anything new :-)
(If you need only parts of ICU, see
http://oss.software.ibm.com/icu/userguide/packaging.html)

Best regards,
markus

Elliotte Rusty Harold wrote:
Last night it occurred to me it might be possible to design an internal storage format for this class which had better memory usage characteristics. In particular I'd like ASCII data to occupy only a single byte, and all other BMP characters from 128 to 65535 to occupy only two bytes. Non-BMP characters could be stored in surrogate pairs.




Archive powered by MHonArc 2.6.24.

Top of Page