[freetds] Charset question
James K. Lowden
jklowden at freetds.org
Wed Apr 19 21:09:32 EDT 2006
Frediano Ziglio wrote:
> Il giorno dom, 16/04/2006 alle 14.01 -0400, James K. Lowden ha scritto:
> > Steve Langasek wrote:
> > > I'd like to see freetds 0.64 released with
> > > the fixed version of the locale patch that I proposed on the list.
> > You and Frediano both suggest applying your patch to 0.64. I think
> > you've both thought about this more carefully than I have
> The patch touch a part of FreeTDS that is quite "sensible" however I
> don't like to have a "fork" on network that are just 10 line
> > I'll apply it and post 0.64rc3. If there's something more that needs
> > doing, tell me.
> There are some points to resolve. Why do patch set client_charset and
> also char_set (now server_charset)? IMO should set just client_charset
> and let other code find better server_charset that match client one.
Agreed. The server should announce its charset, and FreeTDS should
consistently use GNU names for client_charset.
> Another question is: why have we the server charset in locales.conf??
> This charset should be a charset detected by FreeTDS so FreeTDS should
> be able to detect proper client charset.
We shouldn't. locales.conf is (or will be) a fallback for clients that
don't have nl_langinfo and don't set the client charset in freetds.conf.
locales.conf should govern the client (not server) charset, using GNU (not
> Perhaps is useful if we want to
> specify the server charset when server is not able to handle client
> translated correctly (this happen for instance using utf8 for older
Sybse servers that support only TDS 4.2 are over 10 years old now;
Microsoft adopted TDS 7.0 in 2000? I doubt anyone is running SQL Server
6.5 with a utf-8 client.
> > 1. Absent an override, use nl_langinfo(3) to determine the client
> > charset.
> > 2. The environment settings are overridden by 'client charset' in
> > freetds.conf.
> Agree, locales.conf and "client charset" in freetds.conf are mainly for
> > 3. nl_langinfo() is not available and 'client charset' is absent,
> > locales.conf will be used to infer a client character set.
> Agree, locales.conf should replace nl_langinfo. Note however that it can
> happen that nl_langinfo is present but is not valid ("C C C" or
> similar). In case we want to override valid nl_langinfo or detected
> server charset we can override in freetds.conf.
If nl_langinfo(3) exists but returns "C C C" or somesuch, we can fall back
to the default ISO 8559-1. The user can override with freetds.conf.
> > 4. Charset information is used both by iconv and in the TDS 5.0
> > login.
> Yes, and I would add that server_charset is used only for Sybase (or
> former mssql versions) and only as an override for some server settings.
To clarify: A TDS 5.0 sends, as part of the login packet, a *client*
charset to tell the server how it wants to see character data encoded.
The string is a Sybase name (not, say, a GNU iconv name). The server will
comply, and data on the wire will use the client charset. (The server's
encoding -- how the data are stored on the server -- is a different
question. Perhaps I chose badly; I think I dreamed up "server charset"
when what I really meant was "wire charset".)
The distinction could matter: Sybase may use a "wire charset" of utf-8 to
make its client happy, but if the server's data are encoded as ISO 8859-1,
there will be trouble when the client tries to send a Greek Beta symbol
(or a Euro sign).
> > The situation AIUI is
> > that the Sybase server is initially asked to use a character set it
> > doesn't suppport. It feeds that back to the client in a message, and
> > FreeTDS responds accordingly. The only problem is that it can't be
> > overridden, and causes log messages to accumulate on the server.
> I would be perfect if FreeTDS detect this message and use proper iconv.
> But let this for a future release...
Maybe. Sybase has a "server makes right" policy and understands utf-8.
The only clients helped by iconv on a TDS 5.0 connection are those that
want to use a client charset that Sybase doesn't support. I have other
> Cfr. "0.64rc2 Debian prerelease packages available", 31 March.
Thanks for pointing that out. I remember the message but didn't, um,
connect the dots.
More information about the FreeTDS