Skip to Content.
Sympa Menu

freetds - Re: Unicode, UTF-8, and Greek

freetds AT lists.ibiblio.org

Subject: FreeTDS Development Group

List archive

Chronological Thread  
  • From: Brian Bruns <camber AT ais.org>
  • To: TDS Development Group <freetds AT franklin.oit.unc.edu>
  • Subject: Re: Unicode, UTF-8, and Greek
  • Date: Wed, 10 Oct 2001 18:41:40 -0400 (EDT)


On Wed, 10 Oct 2001, Curt Hagenlocher wrote:

> Steve Langasek <vorlon AT netexpress.net> writes:
>
> > FreeTDS is being written today, not
> > five years ago when UCS-2 was being put into NT4. A Unix widechar
> > implementation done today should be using UCS-4. Much better still is
> > the multibyte standard, UTF-8, which almost everyone in the Unix world
> > uses as the Unicode encoding of choice.
>
> As someone else pointed out, you'd still be limited by the server
> implementation even if the client implements the newer, wider spec.
> UTF-8 can, of course, be stored in regular char/varchar fields as
> long as all the clients of the database app agree to use that
> representation.
>
> Would you return an error if someone passed a valid UCS-4 or UTF-8
> string that was not also a valid UCS-2 string?

All data going *to* the server is in the form of a SQL statement (in
UCS2). Again, currently we only support ascii at the moment and thunk it
to 16bit before sending it. UTF-8 would be a very nice fit for sending as
it eliminates differentiating between 8bit codes and unicode for which
there is no API support.

I like UTF-8 on the input side alot. I suppose I'll have to match it on
the bind side too...my main concerns are:

1) UTF-8 may be larger, applications that make assumptions that a
varchar(255) can be no longer than 255 bytes will break. I could say we
just truncate to the set boundary or if it's unlimited, the application
could core dump. If so, app is broken as designed I suppose.

2) Behaviour would differ if the client was configured to use 4.2
(localized character set) or TDS 7.0 (UTF-8).

I think having a default of UTF-8 is perhaps not a bad idea after all but
let's add a configuration setting to convert to local 8bit where
applicable. (We have to call iconv anyway, it'll be pretty easy to just
substitute greek or whatever for UTF-8 where needed).

> What does Sybase support for Unicode?

Not sure, but I'm interested in finding out. I think unicode support is
System 12 and later.

Brian





Archive powered by MHonArc 2.6.24.

Top of Page