FreeTDS Development Group

[Steve Langasek <vorlon AT netexpress.net>]

Brian,

All excellent news!  With your permission, I would like to upload a
freetds package built from current CVS to the Debian archive.  As we all
know, there are a number of showstopper bugs in the 0.52 that's
currently included in Debian; current CVS addresses enough of these
issues to be of some value to users of Debian unstable, and I don't
think it would hurt matters to give that same group of dedicated beta
testers the opportunity to help stamp out bugs before the 0.53 release.

On Sun, Nov 25, 2001 at 02:36:06PM -0500, Brian Bruns wrote:

> Well, I *finally* got around to finishing off iconv support (four day 
> weekends are great).

> It works like this:

> . By default TDS 7.0 streams will have their high order byte stripped like 
>   before.
> . Support for iconv must be compiled in. (autoconf *should* pick up 
>   whether you have it or not).
> . iconv conversion needs to be turned on using the 'client charset' string 
>   in the freetds.conf file.  Users of the old interfaces file will not be 
>   able to use iconv conversion.
> . Unfortunately iconv converts unconvertable characters to null, and thus 
>   a mismatch of SQL and iconv charset will result in a truncation at the 
>   first unconvertable character.  Solutions anyone?

This seems a rather compelling reason for consumers of the library to
prefer UTF-8 as the client charset. ;)

Otherwise, the GNU recode library might be an option; the library is
LGPL, and when characters in the source stream have no equivalent in the
target charset, recode will attempt to map them to multibyte equivalents
(although this practice is not universally liked; I understand that not
all Western languages that have ü [ü] as part of their charset
agree on a common asciification of this character, for instance).
Recode could also be shipped with freetds, thus sidestepping any
portability issues between a multitude of imperfect iconv
implementations.

Iconv's status as a Unix standard makes it an appealing choice; but I
know, for instance, that the Samba Team were so underwhelmed by the
quality of available iconv implementations that they rolled a known-good
implementation into the source code they ship.

In the meantime, I'm sure the current iconv setup will make a suitable
basis for working out many charset-related issues.

> . I'm a bit nervous on the multibyte support including UTF-8.  I tried  
>   UTF-8 with sqsh and got garbage characters (SQSH is not unicode clean), 
>   but I fear if the UTF-8 strings exceed the size of the column from which 
>   they came there could be a buffer overrun in the application.  So, 
>   please ensure your apps can handle it before using.  I'm looking at what 
>   support needs to be added in ctlib/dblib (specifically ct_describe) to 
>   alleviate this problem.  However, many apps will make assumptions based 
>   on single character charsets, and dblib especially allow the apps to not 
>   pass a buffer length when binding. Be careful!

Hmm, sounds lke this should be fun to play with. :)

Actually, wrt sqsh being unicode clean -- the problem is more likely to
be with your terminal than with sqsh itself, I suspect.

Cheers,
Steve Langasek
postmodern programmer

Attachment: pgpG7cllwEuIG.pgp
Description: PGP signature