Skip to Content.
Sympa Menu

freetds - RE: [freetds] tds_canonical_charset_name futures?

freetds AT lists.ibiblio.org

Subject: FreeTDS Development Group

List archive

Chronological Thread  
  • From: "Lowden, James K" <LowdenJK AT bernstein.com>
  • To: "FreeTDS Development Group" <freetds AT lists.ibiblio.org>
  • Subject: RE: [freetds] tds_canonical_charset_name futures?
  • Date: Thu, 29 May 2003 16:24:03 -0400

> From: Craig A. Berry [mailto:craigberry AT mac.com]
> Sent: May 29, 2003 3:15 PM
>
> In other words, shouldn't something like what
> tds_iconv_init does for ISO-8859-1 and UTF-8 be done more generally
> at configuration time, i.e., permute through all of the aliases until
> we find the one that works?

Frediano hit on a general solution that works by trial and error. The notion
is to cache the iconv (not necessarily GNU's) name at runtime.

How will it work? src/tds/alternative_character_sets.h is our database of
character set names. On receipt of a name, look up the canonical name and
try iconv_open(3). If that one fails, try each alternative in turn. If one
succeeds, cache it for future use; it becomes the canonical name
thenceforward. If none succeeds, well, it's curtains for iconv.

The static array src/tds/iconv.c::iconv_names is the cache. It's up to
tds_iconv_open() to find the right iconv name, using logic similar to (and
more general than) that found in tds_iconv_init(). As you can see, that's
not done yet.

Other concerns in this area:

1. To support UCS-2 clients (e.g. OS X) we have to reserve a forth member of
the socket's iconv_info array, to hold ASCII->client conversions. This will
be needed for error messages.

2. iconv_info[client2ucs2] need to be initialized when the server tells us
its single-byte encoding.

3. Some iconv() implementations emit a byte order mark (BOM) when converting
to UCS-2. Some don't. Some let you control endianism. Some don't. I don't
know how if or when to control or record these things. We always want the
same thing: little endian, no BOM.

4. iconv-less compilation is currently broken, pending a naive
LATIN1<->UCS-2 implementation in src/replacements/iconv.c.

Anyone who wants to take on any of these things is more than welcome.

I myself got caught in the La Brea error message morass while recoding
tds_iconv to handle unconvertible sequences reasonably. I think I'll punt --
use tds_client_message() for now, even though it's patently seriously broken
for ct-lib clients -- in order to move forward with UTF-8 clients (and
servers). I hope Bill Thompson decides to unify our client-side message
system as part of his ct-lib renewal. However, it's not clear he'll have
that particular itch.

I hope that answers your question. :-)

--jkl
-----------------------------------------
The information contained in this transmission may contain privileged and
confidential information and is intended only for the use of the person(s)
named above. If you are not the intended recipient, or an employee or agent
responsible for delivering this message to the intended recipient, any
review, dissemination, distribution or duplication of this communication is
strictly prohibited. If you are not the intended recipient, please contact
the sender immediately by reply e-mail and destroy all copies of the original
message. Please note that we do not accept account orders and/or instructions
by e-mail, and therefore will not be responsible for carrying out such orders
and/or instructions.






Archive powered by MHonArc 2.6.24.

Top of Page