FreeTDS Development Group

["James K. Lowden" <jklowden AT schemamania.org>]

[new thread because new topic]

On 16 Mar 2003 09:15:30 +0100, Frediano Ziglio <freddyz77 AT tin.it> wrote:
> Il dom, 2003-03-16 alle 05:01, James K. Lowden ha scritto:

> >     client charset = ISO-8859-1

> So, client charset is used only converting from unicode, not for other
> charset...

Yes, the whole thing is a jumble.  We've solved it intellectually, IMHO,
but we have yet to put fingers to keyboard.  I don't think there's any
disagreement.  

At present, we use libiconv only to deal with UCS-2.  The conversion is
driven by the entry in freetds.conf.  While our a approach is very
precise, it requires the user to fiddle with freetds.conf.  Ideally, that
wouldn't be necessary, except as an override.  

What we should do is both harder and easier.  We should inspect the
environment with 
nl_langinfo(3).  The string we're returned is not standardized, but the
domain of values is finite.  Using it, we should look up the equivalent 1)
libiconv string and 2) Sybase string.  Then, we can inform libiconv of the
client's local character set, and tell Sybase servers in what form we'd
like textual data.  

As an added bonus, we could record the Language, to provide translated
error messages one day.  

> I think our program should set only client charset and libtds should
> select proper translation according to the nearest charset.

Yes. 

> Also we should translate single-byte characters, not only unicode...

Yes.  

> Cannot find the requested character set in Syscharsets:  name =
> 'ANSI_X3.4-1968'.
> Msg 2411, Level 10, State 1, Server , Line 0
> No conversions will be done.
> 

Right.  That why my automagical lookup function for equivalent strings is
necessary.  

That message is from Sybase, and 'ANSI_X3.4-1968' is not a valid character
set identification string, as far as Sybase is concerned.  (It's perfectly
fine for libiconv, though).  I think the string Sybase wants in your case
is 'iso_1'.  

Microsoft's framework is: convert nothing.  Store UCS-2, transmit UCS-2,
display UCS-2.  We have to intercede on our clients' behalf, because
practically no FreeTDS client (with apologies to Mark Lilback) wants it. 
That's why we turned to libiconv.  

Sybase's framework is: convert on the server.  If we tell the server what
character set we'd like to use, it *might* do the work for us (and set
error messages to the appropriate language, too, AIUI), depending on its
capabilities.  If if cannot, we can always engage libiconv, but as you
point out, we don't bother yet.  

> After some research ANSI_X3.4-1968 is better known as US-ASCII.

Correct, you should be able to ascertain that from iconv(1).  These are
all equivalent:

$ iconv -l |grep ANSI_X3.4-1968 |sed 's/ /\
/g' 
ANSI_X3.4-1968
ANSI_X3.4-1986
ASCII
CP367
IBM367
ISO-IR-6
ISO646-US
ISO_646.IRV:1991
US
US-ASCII
CSASCII

None of which, naturally, are in syscharsets.  

Regards, 

--jkl