FreeTDS Development Group

["Lowden, James K" <james.k.lowden AT alliancebernstein.com>]

> From: Steve Langasek
> Sent: Wednesday, April 19, 2006 11:43 PM
> 
> On Wed, Apr 19, 2006 at 09:09:32PM -0400, James K. Lowden wrote:
> > > There are some points to resolve. Why do patch set client_charset
> > > and also char_set (now server_charset)? IMO should set just
> > > client_charset and let other code find better server_charset
> > > that match client one.
> 
> > Agreed.  The server should announce its charset, and FreeTDS
> > should consistently use GNU names for client_charset.
> 
> Does the server in fact announce its charset?  It was my impression
> that the wire charset was negotiated between the client and server
> with the client requesting a charset?

Ah.  Let me try again.  In TDS 7.0+, the server announces its charset.
In TDS 8.0, it can do so per column.  (Comment: Baroque, because in
Microsoftland both client and server support UCS-2.) Cf.
http://www.freetds.org/tds.html#collate

In TDS 5.0, as you say, the client proposes and the server disposes.
ISTR the server can send a message like "your charset is no good, we're
using this charset", but reviewing the TDS 5.0 functional spec, I see
it's a matter of negotiation during the login sequence.  

In no event do we tell the server what charset it's using; we say only
what charset we'd like to use to communicate with it.  

> > Sybse servers that support only TDS 4.2 are over 10 years old
> > now; Microsoft adopted TDS 7.0 in 2000?  I doubt anyone is running
> > SQL Server 6.5 with a utf-8 client.
> 
> Does that mean you think no one is using FreeTDS with SQL 6.5?  In
> that case, wouldn't it be best to disallow use of TDS 4.2 to prevent
> accidental charset mismatches? :)

I think the population of running 6.5 server is small and shrinking.  I
think the number of UTF-8 clients attached to them *is* zero.  But there
may well be 6.5 servers running, with antique clients connected to them.
They're subject to the same age-old constraints they were 10 years ago,
to match the client's charset to the server's.  

> > To clarify: A TDS 5.0 sends, as part of the login packet, a
> > *client* charset to tell the server how it wants to see character
> > data encoded.  The string is a Sybase name (not, say, a GNU iconv
> > name).  The server will comply, and data on the wire will use
> > the client charset.  (The server's encoding -- how the data are
> > stored on the server -- is a different question.  Perhaps I chose
> > badly; I think I dreamed up "server charset" when what I really
> > meant was "wire charset".)
> 
> > The distinction could matter: Sybase may use a "wire charset" of
> > utf-8 to make its client happy, but if the server's data are
> > encoded as ISO 8859-1, there will be trouble when the client
> > tries to send a Greek Beta symbol (or a Euro sign).
> 
> So is the point here that locales.conf and the locale should not
> influence the wire charset?  

No.  locales.conf is the last line of defense (before the compiled-in
default).  All we ever do in FreeTDS is acertain the client's desired
charset.  In the case of Sybase, we communicate it to the server; in the
case of Microsoft, we engage iconv.  My point was only that we don't
determine the server's charset, and for that reason the server may
reject data sent by the client as unencodable.  

> In general, I would rather trust the
> client system's ability to do charset conversion and so I agree
> that it would be better if the wire charset were chosen based on
> server information rather than information about the client locale;
> I just don't see where we get such information on a per-server
> basis other than configuring it in freetds.conf.

I think we disagree here.  IMHO Sybase has a coherent policy: let the
client be the client and have the server comport itself to the client's
needs.  (Arguably, you could move all conversions to the client library
and use a single canonical form to communicate to the server, but I
would guess that would imply protocol changes.)  Besides, it's not just
charset stuff.  This from Version 3.4 110 of the TDS 5.0 Functional
Specification:

<quote>
TDS presentation conversion is server makes right. This means that the
server
is always responsible for performing any required conversions.
Presentation
conversion is performed for the following cases:

* Character set conversions for character and text datatypes.
* Numeric conversions for float, decimal, and numeric datatypes between
the
client's local representation and the server's.
Sybase Confidential TDS 5.0 Reference Pages
* Date and time conversions between a client's local representation and
the
server's.
* Byte ordering conversions for length fields and integer datatypes.
</quote>

Sybase is making our job easy.  I think we should let them.  :-)

Regards, 

--jkl



-----------------------------------------
The information contained in this transmission may be privileged and
confidential and is intended only for the use of the person(s) named
above. If you are not the intended recipient, or an employee or agent 
responsible
for delivering this message to the intended recipient, any review, 
dissemination,
distribution or duplication of this communication is strictly prohibited. If 
you are
not the intended recipient, please contact the sender immediately by reply 
e-mail
and destroy all copies of the original message. Please note that we do not 
accept
account orders and/or instructions by e-mail, and therefore will not be 
responsible
for carrying out such orders and/or instructions.  If you, as the intended 
recipient
of this message, the purpose of which is to inform and update our clients, 
prospects
and consultants of developments relating to our services and products, would 
not
like to receive further e-mail correspondence from the sender, please "reply" 
to the
sender indicating your wishes.  In the U.S.: 1345 Avenue of the Americas, New 
York,
NY 10105.