FreeTDS Development Group

["Craig A. Berry" <craigberry AT mac.com>]

At 1:46 PM -0400 5/1/03, Lowden, James K wrote:
> > From: Craig A. Berry [mailto:craigberry AT mac.com]
>> Sent: May 1, 2003 11:28 AM
>>
>> One workaround that got me over one obstacle was adding the following
>> line to the global section of my freetds.conf file:
>>
>>    client charset = ISO8859-1
>>
>> However, the GNUish "ISO-8859-1" variant is hard-coded various places
>> in the sources. 
>
>Craig, thanks for trying this out.  I know about the hard-coding, but could
>you elaborate on why "ISO8859-1" helps you?  I think I know. 

Well, before I did that (with the snapshot from 20 April and so rather moot 
now), I got:

iconv will convert client-side data to the "ISO-8859-1" character set
2003-04-20 19:00:01.886844 iconv_open: cannot convert to "ISO-8859-1": 
invalid argument

(I think seeing "invalid argument" required the addition of strerror(errno) 
to the tdsdump_log call.)  After adding the line to my freetds.conf file, I 
just saw:

iconv will convert client-side data to the "ISO8859-1" character set

and no error message.  But if the initialization is done from either 
locale_charset or nl_langinfo as you note below, the default should work.

>To convert character sets, we use iconv.  To tell iconv what we'd like it to
>convert from/to, we pass charset names.  These names are not standardized
>across iconv implementations (or operating systems). 

So I discovered.  I was about to cry foul when I did a bit of digging and 
found that "ISO8859-1" is what VMS, Tru64, and AIX use, but "ISO-8859-1" is 
what GNU and some others use, and Solaris just uses "88591".  Or at least 
that's what I remember seeing somewhere.  That plus various Open Group 
mailing list archives lament the lack of a standard but don't indicate 
anything has been done about it yet. 

>
>There are two kinds of charset names.  Well, three, sort of:
>
>1.  Server names, from master..syscharsets. 
>2.  OS names, from nl_langinfo(CODESET).
>3.  iconv names. 
>
>I suppose it's obvious that server charset names need to be mapped to iconv
>names.  That mapping is provided by src/tds/sybase_character_sets.h. 
>
>Many OS's provide both nl_langinfo() and iconv(). 

VMS is one of them. 

>  Not all systems will have
>locale_charset(), of course, notably those not running GNU iconv.  The logic
>becomes:
>
>       if (_HAVE_LOCALE_CHARSET_)
>               /* query OS, get GNU charset name */
>               name = locale_charset();
>       else
>               /* query OS, get OS charset name */
>               name = nl_langinfo(CODESET);
>
>Which I think will Just Work on any rightly configured system.  For
>misconfigured systems, we still have "client charset" in freetds.conf, which
>we'll accept verbatim. 

nl_langinfo(CODESET) returns "ASCII" on my stock 7.3-1 VMS system with no 
special language kits installed. 

>Status: we're between "knowing the answer" and "doing the work".
>Initialization is also shaky. 
>
>Status: we're shaking the tree to get one thing right: UTF-8 clients with
>any server.  We're starting with GNU iconv because that's what I've got.
>Generalization isn't my focus right now, but patches are welcome. 

The Apache project has an iconv library as part of the APR library.  I 
haven't looked at the details but it may have some generalizations you could 
borrow.

>Perhaps for ASCII we can find a single string that all iconv implementations
>recognize.


>Is "ASCII" perchance on your list, too?  I like the idea of calling ASCII,
>"ASCII". 

Unfortunately not (even though that's what nl_langinfo returns).  It looks 
like the options for passing to iconv_open are:

ISO8859-1
UCS-2
UCS-4
VTF-7

Installing additional language kits would probably grow that list 
considerably.

>I know, "ASCII silly question, get a silly ANSI". 

ISO love that idea, but are you POSIX it will work?

-- 
________________________________________
Craig A. Berry
mailto:craigberry AT mac.com

"... getting out of a sonnet is much more
 difficult than getting in."
                 Brad Leithauser