FreeTDS Development Group

["Brian Bruns" <camber AT ais.org>]

The only problem with supporting UTF-8 against a sybase (12.x) server
would be the more general problem of multibyte sets, specifically
ct_describe() returning sane values back to the client app. Otherwise, I
could continue to receive iso_1, greek, or whatever format I specify via
charset login field.  I really don't think FreeTDS should have any problem
with supporting this *today*.

The TDS 7 issue is different though.  There is no charset field. 
Everything is unicode. So we must convert it to something usable by the
client app.  Today that means 7bit ascii by a simple high byte strip
mechanism.  But, obviously, that is only a valid option for english and a
few other languages.  (Even most western european language need the
128-255 range and don't work with 7.0 currently).  The safest thing would
be to default to latin_1 with a freetds.conf field allowing other
character sets.  I was leaning towards UTF-8, but that's an easy conf file
change for non-western langs and gives a lot of flexibility to those folks
to choose from a variety of charsets and not be pinned into UTF-8 if the
app is not unicode-aware.  Therefore I propose a 'tds7 client charset'
field in freetds.conf to have a value of any iconv charset label (iconv
--list on linux to view).

I'm a little annoyed with needing one scheme for TDS 4.2/5.0 (server side)
and one for TDS 7.0 (client side) but that's out of our control.  Another
question becomes what to do with DBSETLCHARSET() and friends under tds
7.0?  one could use it as an iconv set, but then you have a problem of
people passing in legal 4.2 sets (iso_1) and not having them work on 7.0. 
I know OpenClient uses the locales.dat file to map en_US for instance to a
language/charset pairing.  Maybe we need something similar for Sybase
charset -> iconv charset mapping.

> Lowden, James K writes:
>  > All, 
>  > 
>  > For anyone who cares and hasn't seen it, there's a fairly succinct
>  > description of Sybase's character set handling in their International
>  > Developer's Guide at
>  > 
> http://manuals.sybase.com/onlinebooks/group-cn/cng1250e/i_and_l/@Generic__Bo
>  > okView     
>  > 
>  > Specifically, 
>  > 
> http://manuals.sybase.com/onlinebooks/group-cn/cng1250e/i_and_l/@Generic__Bo
>  > okTextView/3170;pt=3129
>  > 
>  > In a nutshell, from what I can see, the server converts between several
>  > character sets.  The client's default character set is determined from 
> the
>  > locale environment variables, and ct_bind can be used to coerce other
>  > conversions.  
>  > 
>  > I found no localization or character set support in the db-lib docs.
> 
> You can set the client charset via DBSETLCHARSET() in DBlib.
> 
> For CTlib charset handling is generally handled via the cs_locale()
> interface.
> 
> The server can do conversions where the size of characters is the
> same, or where the target is narrower than the source. It can't (in
> general) do conversions where the target is wider than the source.
> 
> Michael