FreeTDS Development Group

["Lowden, James K" <LowdenJK AT bernstein.com>]

Steve, 

Wow.  Thank you for the education.  "Is there more than one 'character
set'?" sounds like a woefully ignorant question after reading the Hastings
article.  I had no idea, really.   

Two native Chinese speakers in my office (Bejing and Shanghai) did not
mention these issues; Unicode meets their day-to-day needs.  Their
complaints have more to do with the variety of encoding schemes and the
inadequacy of internationalization of most applications.  I say this not to
defend anything (or offend anyone), just to say that maybe Unicode is a
little like Windows 98: it meets a need pretty well if you don't press too
hard.  

I think it also points to the need to be able to query the database for its
encoding system?  

Personally, I want the World Wide Web to live up to its name and as a
technical person I hope the issues the article raises get resolved
completely, without compromise, no matter how many bytes it takes.  But as
you say, this is currently not our problem to solve.  We have to go along to
get along.  

There are all kinds of crazy ramifications to embracing any non-ascii
system, as I'm sure you know.  Little things, like: a filesystem can be very
surprised to see a 0x2f as part of a filename because they "know" 0x2f is
'/'.  Oh boy.  This is an area where as far as I can tell NT is way out in
front.  NTFS isn't bothered by Unicode.  

As an English-speaking sometime OLE programmer, I've had more reason to bury
Unicode than praise it, partly -- OK, mostly -- because of Microsoft's
non-transparent often non-helpful "support" for it.  I see now I've only
been exposed to a hint of the future.  For years I've preached "Text! Save
it as text.  Send it as text."  I wonder what that will mean in future, if
there'll be any "text" left.  

Thanks again.

--jkl

-----Original Message-----
From: Steve Langasek [mailto:vorlon AT netexpress.net]
Sent: June 8, 2001 12:24 PM

A good article covering the CJK-Unicode issues was linked to from Slashdot
recently:
<http://www.hastingsresearch.com/net/04-unicode-limitations.shtml>.
What it boils down to is that the opinion that 16-bits is big enough for
everybody is a Western-imposed proclamation; and just because it suffices
for
Chinese /newspapers/ doesn't mean it's good enough for all of China.
Anything
written in classic Chinese, which encompasses all historical and literary
documents written before the 1940s as well as some modern academic work, is
unrepresentable in Unicode.  

> There doesn't seem to be any support out there for UCS-4, unless you count
Gnu's
> 32-bit wchar_t type (which they say isn't strictly compliant).  So,
supporting
> Unicode would put FreeTDS on par with its databases.

Exactly -- we won't be able to support CJK fully using Unicode, but this is
currently not our problem to solve.