FreeTDS Development Group

["James K. Lowden" <jklowden AT schemamania.org>]

All, 

Background.  There's been a little debate here at FreeTDS about how to
identify character sets i.e., how to determine the correct name to use. 
We can query the operating system for the client character set name, with
nl_langinfo(CODESET), but the string it returns is not standardized. 
That's OK, if your operating system also provides an iconv implementation,
one that accepts whatever string nl_langinfo(CODESET) produces.  Problems
crop up when GNU libiconv is used instead.  

For example, nl_langinfo() might return "utf8" or "UTF8" or "UTF-8".  Of
these, only "UTF-8" is recognized by GNU libiconv.  

I had embarked on an over-ambitious plan to map any and all names,
beginning with tds_canonical_charset_name() [now correctly spelled].  At
the same time, I wrote to Bruno Haible, who maintains GNU libiconv,
explaining what I was doing, and offering to help or be helped, depending
on what he had to offer.  

Came today his reply, and the news is good.  Thank you, Bruno!  

Included in the GNU libiconv distribution is a function Bruno wrote,
locale_charset(), that does what we want.  From libcharset.h:

<quote>
/* Determine the current locale's character encoding, and canonicalize it
   into one of the canonical names listed in config.charset.
   The result must not be freed; it is statically allocated.
   If the canonical name cannot be determined, the result is a
non-canonical
   name.  */

extern const char * locale_charset (void);
</quote>

In other words, if we call "locale_charset()" instead of
"nl_langinfo(CODESET)", we get the canonical charset name instead of the
OS's peculiar alias for it.  Happy Days!  

By way of demonstration:

        $ cat >t.c
        #include <stdio.h>
        #include <langinfo.h>
        #include <libcharset.h>

        int main()
        {
                puts( nl_langinfo(CODESET) );
                puts( locale_charset() );
                return 0;
        }
        ^D
        $ cc -O2 -g     -I/usr/pkg/include  \
                        -L/usr/pkg/lib \
          -Wl,--rpath -Wl,/usr/pkg/lib -o t -lcharset t.c 
        $ uname; ./t
        NetBSD
        646
        ASCII

That is, on my NetBSD system, nl_langinfo(CODESET) returns "646", which
GNU libiconv doesn't accept, but locale_charset() returns "ASCII", which
it does.  

locale_charset() includes support for most operating systems, even Win32. 
Should you choose to link FreeTDS against the iconv provided by your OS
instead of GNU's, we'll assume it expects the string returned by your
libc's nl_langinfo().  In either case, I consider the problem of
determining the client charset name solved.  

This isn't the end of our charset name woes, but it's the biggest chunk. 
More on what remains in my next missive.  

Regards, 

--jkl