Skip to Content.
Sympa Menu

freetds - [freetds] Solved: client charset names for iconv

freetds AT lists.ibiblio.org

Subject: FreeTDS Development Group

List archive

Chronological Thread  
  • From: "James K. Lowden" <jklowden AT schemamania.org>
  • To: TDS Development Group <freetds AT lists.ibiblio.org>
  • Cc: Bruno Haible <bruno AT clisp.org>
  • Subject: [freetds] Solved: client charset names for iconv
  • Date: Tue, 15 Apr 2003 23:04:16 -0400

All,

Background. There's been a little debate here at FreeTDS about how to
identify character sets i.e., how to determine the correct name to use.
We can query the operating system for the client character set name, with
nl_langinfo(CODESET), but the string it returns is not standardized.
That's OK, if your operating system also provides an iconv implementation,
one that accepts whatever string nl_langinfo(CODESET) produces. Problems
crop up when GNU libiconv is used instead.

For example, nl_langinfo() might return "utf8" or "UTF8" or "UTF-8". Of
these, only "UTF-8" is recognized by GNU libiconv.

I had embarked on an over-ambitious plan to map any and all names,
beginning with tds_canonical_charset_name() [now correctly spelled]. At
the same time, I wrote to Bruno Haible, who maintains GNU libiconv,
explaining what I was doing, and offering to help or be helped, depending
on what he had to offer.

Came today his reply, and the news is good. Thank you, Bruno!

Included in the GNU libiconv distribution is a function Bruno wrote,
locale_charset(), that does what we want. From libcharset.h:

<quote>
/* Determine the current locale's character encoding, and canonicalize it
into one of the canonical names listed in config.charset.
The result must not be freed; it is statically allocated.
If the canonical name cannot be determined, the result is a
non-canonical
name. */

extern const char * locale_charset (void);
</quote>

In other words, if we call "locale_charset()" instead of
"nl_langinfo(CODESET)", we get the canonical charset name instead of the
OS's peculiar alias for it. Happy Days!

By way of demonstration:

$ cat >t.c
#include <stdio.h>
#include <langinfo.h>
#include <libcharset.h>

int main()
{
puts( nl_langinfo(CODESET) );
puts( locale_charset() );
return 0;
}
^D
$ cc -O2 -g -I/usr/pkg/include \
-L/usr/pkg/lib \
-Wl,--rpath -Wl,/usr/pkg/lib -o t -lcharset t.c
$ uname; ./t
NetBSD
646
ASCII

That is, on my NetBSD system, nl_langinfo(CODESET) returns "646", which
GNU libiconv doesn't accept, but locale_charset() returns "ASCII", which
it does.

locale_charset() includes support for most operating systems, even Win32.
Should you choose to link FreeTDS against the iconv provided by your OS
instead of GNU's, we'll assume it expects the string returned by your
libc's nl_langinfo(). In either case, I consider the problem of
determining the client charset name solved.

This isn't the end of our charset name woes, but it's the biggest chunk.
More on what remains in my next missive.

Regards,

--jkl



  • [freetds] Solved: client charset names for iconv, James K. Lowden, 04/15/2003

Archive powered by MHonArc 2.6.24.

Top of Page