Skip to Content.
Sympa Menu

freetds - RE: [freetds] On unicode strings support

freetds AT lists.ibiblio.org

Subject: FreeTDS Development Group

List archive

Chronological Thread  
  • From: "ZIGLIO, Frediano, VF-IT" <Frediano.Ziglio AT vodafone.com>
  • To: "FreeTDS Development Group" <freetds AT lists.ibiblio.org>
  • Subject: RE: [freetds] On unicode strings support
  • Date: Fri, 8 Oct 2004 10:44:42 +0200

>
> > It wouldn't be easy, because of our design choice. We
> > convert the data
> > in libtds as they arrive from the server. We don't preserve the
> > server's representation, and we don't look to the client's column
> > binding type. We realize that's a deficiency -- see my
> > message on this
> > list on 19 May 2004 "About the Unicode support in the FreeTDS ODBC
> > Driver" -- but there's no active work on it.
> >
> > If you want bona fide UCS-2 data out of FreeTDS right now,
> > you have two
> > options. One, you can convert your UCS-2 data to varbinary on the
> > server, and use a binary binding on the client. Two, you can
> > reconvert
> > the converted data to UCS-2 yourself, using libiconv.
> >
> > If you want to move it up my priority list, could you please
> > explain why
> > this is useful? It seems to me that our one-charset-for-everything
> > model is convenient. I'd like to understand the motivation behind
> > treating UCS-2 data differently.
> First of all I shall mention we have just given the first try
> to make a true
> multilanguage product so my knowladge about the problem is
> pretty limited.
> The main idea is to have the things as simple as it possible:
> std::string
> for chars, std::wstring for wide chars. IMO the conversions
> you are talking
> about would require us to provide a locale information (which
> is probably
> may be encoded with UTF8 or something like that). I am not a
> native English
> speaker and know about localization problems not by hearsay.
> We strongly
> believe the 16-bit unicode will help us to avoid at least
> some of problems
> that may be introduced with a conversion. There is a chance a
> client will be
> unable to represent a unicode text but at least it will be
> able to store it
> correctly.
>

Well... UTF-8 can represent more characters than 16-bit unicode so the
real problem it's that 16-bit unicode can't represent all UTF-8, not the
opposite. Also std::wstring do not contains 16-bit unicode but wide
characters (in C++ char it's not 8 bit and a wide character don't have a
defined size!). unixODBC for example can be compiled to use system wide
character (that is 32 bit unicode on Linux) or 16-bit unicode (very
difficult to handle on Unix...)

> > If you want to work on it yourself, we accept patches, and have been
> > known to provide (sometimes helpful) advice.
> I am afraid no. The problem is not that vital for us to spend
> much time for
> it. And it looks like we just do not have that time, sorry.
>
> BTW, while investigating the issue I have noticed that MS
> driver performs
> pretty good conversion of SQL Server nvarchar type to
> SQL_C_CHAR (simple
> char) if I make such a binding - the conversion you are
> talking about. So,
> at least on Windows box I could have nvarchar on SQL Server side and
> std::string on the client side (I just have not tried it on
> Linux or Solaris
> - it may work too). But for I do not know why reason such
> state of affairs
> did not satisfied the guys who run the product architecture.
> They do want to
> see 16-bit letters. And, frankly speaking, I would prefer it too.
>

FreeTDS do a very good translation. "They do want to see 16-bit
letters"... do they have some technical reasons or just sympathy ?? Some
technical reasons why IMHO a modern unix system should use utf-8:
- it's system default, filesystem are encoded on utf-8 api are utf-8
compatible
- it's default on XML and a supported html encoding in all browser (very
useful for PHP or script for html)
- you can encode any known character
- ascii compatible
- tools like telnet or ssh can use utf-8

freddy77




Archive powered by MHonArc 2.6.24.

Top of Page