FreeTDS Development Group

["Lowden, James K" <LowdenJK AT bernstein.com>]

[tried to send this yesterday.  hope it's still useful.]
-----Original Message-----
From: Michael Kochetkov
Sent: Wednesday, October 06, 2004 7:59 AM

SQLDescribeCol returns SQL_VARCHAR - 12 - type for MS SQL Server
nvarchar(...) field at present. The FreeTDS documentation states that
"At
some future time, FreeTDS aims to support Unicode and other multi-byte
character sets. It does not do so at the current time."
Do you plan to path the unicode strings as is through the FreeTDS layer
in
the nearest future? If no then is it considered to be hard if I decide
to
implement it myself?
-----End Original Message-----

Hello Michael, 

Hmmm.  "FreeTDS aims to support Unicode" could certainly be clearer.  

The game is, you declare your client's preferred encoding (its character
set), and FreeTDS converts all data and metadata to that encoding.
That's our model.  

[Just to define our terms:  Datatypes char, varchar, and text use an
8-bit encoding such as ISO-8859-15.  Datatypes nchar, nvarchar, and
ntext use the 16-bit Unicode UCS-2 encoding.]

Our expectation is that clients that need to support Unicode data will
use UTF-8 as their client character set.  If they happen to use high-bit
characters for their non-Unicode data, they'll receive them as UTF-8
just the same.  In short, we blur the distinction between the two
storage types, to the extent that the client is never presented with
Unicode datatypes.  

Your question is: What are we doing to pass UCS-2 data "straight
through" to the client, unconverted.  More succintly, what are we doing
to support (unconverted) binary bindings? And how hard would it be?  

It wouldn't be easy, because of our design choice.  We convert the data
in libtds as they arrive from the server.  We don't preserve the
server's representation, and we don't look to the client's column
binding type.  We realize that's a deficiency -- see my message on this
list on 19 May 2004 "About the Unicode support in the FreeTDS ODBC
Driver" -- but there's no active work on it.  

If you want bona fide UCS-2 data out of FreeTDS right now, you have two
options.  One, you can convert your UCS-2 data to varbinary on the
server, and use a binary binding on the client.  Two, you can reconvert
the converted data to UCS-2 yourself, using libiconv.  

If you want to move it up my priority list, could you please explain why
this is useful?  It seems to me that our one-charset-for-everything
model is convenient.  I'd like to understand the motivation behind
treating UCS-2 data differently.  

If you want to work on it yourself, we accept patches, and have been
known to provide (sometimes helpful) advice.  

Regards, 

--jkl

-----------------------------------------
The information contained in this transmission may contain privileged and 
confidential information and is intended only for the use of the person(s) 
named above. If you are not the intended recipient, or an employee or agent 
responsible for delivering this message to the intended recipient, any 
review, dissemination, distribution or duplication of this communication is 
strictly prohibited. If you are not the intended recipient, please contact 
the sender immediately by reply e-mail and destroy all copies of the original 
message. Please note that we do not accept account orders and/or instructions 
by e-mail, and therefore will not be responsible for carrying out such orders 
and/or instructions.
If you, as the intended recipient of this message, the purpose of which is to 
inform and update our clients, prospects and consultants of developments 
relating to our services and products, would not like to receive further 
e-mail correspondence from the sender, please "reply" to the sender 
indicating your wishes.  In the U.S.: 1345 Avenue of the Americas, New York, 
NY 10105.