FreeTDS Development Group

["ZIGLIO, Frediano, VF-IT" <Frediano.Ziglio AT vodafone.com>]

> 
> > It wouldn't be easy, because of our design choice.  We 
> > convert the data
> > in libtds as they arrive from the server.  We don't preserve the
> > server's representation, and we don't look to the client's column
> > binding type.  We realize that's a deficiency -- see my 
> > message on this
> > list on 19 May 2004 "About the Unicode support in the FreeTDS ODBC
> > Driver" -- but there's no active work on it.  
> >
> > If you want bona fide UCS-2 data out of FreeTDS right now, 
> > you have two
> > options.  One, you can convert your UCS-2 data to varbinary on the
> > server, and use a binary binding on the client.  Two, you can 
> > reconvert
> > the converted data to UCS-2 yourself, using libiconv.  
> > 
> > If you want to move it up my priority list, could you please 
> > explain why
> > this is useful?  It seems to me that our one-charset-for-everything
> > model is convenient.  I'd like to understand the motivation behind
> > treating UCS-2 data differently.  
> First of all I shall mention we have just given the first try 
> to make a true
> multilanguage product so my knowladge about the problem is 
> pretty limited.
> The main idea is to have the things as simple as it possible: 
> std::string
> for chars, std::wstring for wide chars. IMO the conversions 
> you are talking
> about would require us to provide a locale information (which 
> is probably
> may be encoded with UTF8 or something like that). I am not a 
> native English
> speaker and know about localization problems not by hearsay. 
> We strongly
> believe the 16-bit unicode will help us to avoid at least 
> some of problems
> that may be introduced with a conversion. There is a chance a 
> client will be
> unable to represent a unicode text but at least it will  be 
> able to store it
> correctly.
> 

Well... UTF-8 can represent more characters than 16-bit unicode so the
real problem it's that 16-bit unicode can't represent all UTF-8, not the
opposite. Also std::wstring do not contains 16-bit unicode but wide
characters (in C++ char it's not 8 bit and a wide character don't have a
defined size!). unixODBC for example can be compiled to use system wide
character (that is 32 bit unicode on Linux) or 16-bit unicode (very
difficult to handle on Unix...)

> > If you want to work on it yourself, we accept patches, and have been
> > known to provide (sometimes helpful) advice.  
> I am afraid no. The problem is not that vital for us to spend 
> much time for
> it. And it looks like we just do not have that time, sorry.
> 
> BTW, while investigating the issue I have noticed that MS 
> driver performs
> pretty good conversion of SQL Server nvarchar type to 
> SQL_C_CHAR (simple
> char) if I make such a binding - the conversion you are 
> talking about. So,
> at least on Windows box I could have nvarchar on SQL Server side and
> std::string on the client side (I just have not tried it on 
> Linux or Solaris
> - it may work too). But for I do not know why reason such 
> state of affairs
> did not satisfied the guys who run the product architecture. 
> They do want to
> see 16-bit letters. And, frankly speaking, I would prefer it too.
> 

FreeTDS do a very good translation. "They do want to see 16-bit
letters"... do they have some technical reasons or just sympathy ?? Some
technical reasons why IMHO a modern unix system should use utf-8:
- it's system default, filesystem are encoded on utf-8 api are utf-8
compatible
- it's default on XML and a supported html encoding in all browser (very
useful for PHP or script for html)
- you can encode any known character
- ascii compatible
- tools like telnet or ssh can use utf-8

freddy77