FreeTDS Development Group

[Steve Langasek <vorlon AT netexpress.net>]

On Sat, 14 Jul 2001, Brian Bruns wrote:

> That was very insightful. I think we have three major issues here:

> 1) TDS 7/8.  Everything is unicode, so we should just be able to read the
> field in freetds.conf and use iconv to convert between UCS2 and
> that.  If there is no entry in freetds.conf, we could try to grab the
> value of $LANG, although I'm not sure how that maps to the iconv
> labels. Failing there we simply use the present 7bit ASCII conversion

I rather favor defaulting to UTF-8 instead of ASCII when no charset has been
specified.  If there's non-ascii data in a database, it's because someone put
it there; doesn't it make more sense to treat this data losslessly by default,
passing it through for the client app to deal with?

I also wonder if it's necessary to support language settings within
freetds.conf itself.  If we're going to make use of any sort of a 'default', I
think either C or an appropriate UTF-8 locale are the best choices; and
anything other than a default can be specified using either the $LANG
setting or the appropriate bits of the dblib API.

Choosing default charsets for a given language can be done using a
charset.map, as with Sybase.  This seems to me to be the only portable way of
detecting per-session charset preferences, in any case; my pokings into glibc,
at least, show that glibc doesn't expose information to applications about its
own language->charset mappings.  This would also apply below.

> 2) TDS 4.2/5. This gets a little complicated as FreeTDS needs to
> communicate to the server what character set to use.  Problem is the label
> for this (iso_1, cp850) are not the same as the iconv labels (do
> a iconv --list to see them). I would suggest we create a 'charset.map'
> file to map sybase style charset names to the iconv ones, perhaps also
> dealing with $LANG similarly.  Sybase does something similar with the
> locales.dat file.

> 3) multibyte char sets.  I'm afraid I just don't know enough about usage
> of these to make a good attempt at it.  We'll shelve it until a native
> user comes along. ;-)

For the most part, I think iconv will be doing all the hard work.  If we have
working conversion from UCS2->UTF8 (wchar->mbyte) and UCS2->ISO-8859-x
(wchar->char), I expect we'll get working conversion to other multibyte
charsets for free.


ObApology: Debian packages for freetds 0.52 are coming along.  Didn't have as
much time to start on them before the wedding as I had hoped. :)

Steve Langasek
postmodern programmer