Skip to Content.
Sympa Menu

freetds - Re: Unicode, UTF-8, and Greek

freetds AT lists.ibiblio.org

Subject: FreeTDS Development Group

List archive

Chronological Thread  
  • From: Steve Langasek <vorlon AT netexpress.net>
  • To: TDS Development Group <freetds AT franklin.oit.unc.edu>
  • Subject: Re: Unicode, UTF-8, and Greek
  • Date: Wed, 10 Oct 2001 13:50:48 -0500

James,

On Wed, Oct 10, 2001 at 12:23:14PM -0400, Lowden, James K wrote:
> >Ironically, the users who have the most problems with Unicode in general
> >(CJK users, due to incompatibilities with other national encodings) are
> >also the ones most maligned by UCS2 in particular -- but this is not a
> >problem FreeTDS can solve unless we started pushing UTF-16 data into the
> >database instead of UCS2, thus breaking compatibility with all Windows
> >client software.

> Steve,

> Not just "Windows client software". The server side, too.

> Microsoft stores textual data in two forms: Ascii as char and Unicode
> (UCS2) as nchar data types. It collates both and pretty much allows
> char/nchar to be used interchangeably. The only way to store UTF-16 data on
> the server would be either as some kind of blob, or by lying: pretending to
> the server that our varchar field, say, has ascii when in fact it has
> UTF-16. Such fields could be used in joins (to others like them), but
> there's no way to fashion a "where" clause in the native SQL to search them,
> and no (meaningful) way to sort them. No tool could display them legibly,
> either.

I meant to imply that the clients would all be lying to the server about
the kind of data they're storing, yes. If all clients agree to lie and
store UTF-16 where the server expects UCS-2 (since the server can't tell
these apart, any more than you can tell the difference between Latin1
and UTF-8 without using heuristics), then I believe this would be an
effective way of storing full Unicode UTF-16 text in a UCS2 SQL server.
But it requires that *all* client software be involved in this
conspiracy, so anyone who has any Windows clients would not benefit from
such a hack.

> BTW, do you know of any database, free or proprietary, that understands
> anything besides Ascii and UCS2? AFAIK, no one does and no one's trying.

I've heard just today that MySQL will be adding Unicode support. I'll
read up on it -- and threaten bodily harm if I find them using UCS2. ;)

Steve Langasek
postmodern programmer

Attachment: pgpzk8SZLZvf0.pgp
Description: PGP signature




Archive powered by MHonArc 2.6.24.

Top of Page