Skip to Content.
Sympa Menu

freetds - Re: iconv support

freetds AT lists.ibiblio.org

Subject: FreeTDS Development Group

List archive

Chronological Thread  
  • From: Brian Bruns <camber AT ais.org>
  • To: TDS Development Group <freetds AT franklin.oit.unc.edu>
  • Subject: Re: iconv support
  • Date: Sun, 25 Nov 2001 17:22:14 -0500 (EST)


On Sun, 25 Nov 2001, Steve Langasek wrote:

> Brian,
>
> All excellent news! With your permission, I would like to upload a
> freetds package built from current CVS to the Debian archive. As we all
> know, there are a number of showstopper bugs in the 0.52 that's
> currently included in Debian; current CVS addresses enough of these
> issues to be of some value to users of Debian unstable, and I don't
> think it would hurt matters to give that same group of dedicated beta
> testers the opportunity to help stamp out bugs before the 0.53 release.
>

I'll bundle up a 0.53pre2 (are we on pre2? ...looking...yes) if it'll help
giving it a name instead of 'FreeTDS CVS2001-11-25-17:01' ;-)

Ok, its on ftp.ibiblio.org/pub/Linux/ALPHA/freetds/freetds-0.53pre2.tgz I
only made sure it built, so hopefully all is well and I'll update the
webpage tommorow.

The only other major thing left for 0.53 is making sure DBD::Sybase 0.93
is working completely. Michael used some as yet unimplemented features in
the new version. Personally, I don't think he really had a real reason to
do so, he's just trying to push us towards better compliance. ;-)

> > . Unfortunately iconv converts unconvertable characters to null, and thus
> > a mismatch of SQL and iconv charset will result in a truncation at the
> > first unconvertable character. Solutions anyone?
>
> This seems a rather compelling reason for consumers of the library to
> prefer UTF-8 as the client charset. ;)

Philosophically, I agree. again, devil in the details.

> Otherwise, the GNU recode library might be an option; the library is
> LGPL, and when characters in the source stream have no equivalent in the
> target charset, recode will attempt to map them to multibyte equivalents
> (although this practice is not universally liked; I understand that not
> all Western languages that have ΓΌ [&uuml;] as part of their charset
> agree on a common asciification of this character, for instance).
> Recode could also be shipped with freetds, thus sidestepping any
> portability issues between a multitude of imperfect iconv
> implementations.
>
> Iconv's status as a Unix standard makes it an appealing choice; but I
> know, for instance, that the Samba Team were so underwhelmed by the
> quality of available iconv implementations that they rolled a known-good
> implementation into the source code they ship.
>
> In the meantime, I'm sure the current iconv setup will make a suitable
> basis for working out many charset-related issues.

Points taken. I believe 80% or so of the users are Linux/*BSD users.
Perhaps for the rest we should include a pointer to libiconv in the
afore-mentioned User Guide entry.

> > . I'm a bit nervous on the multibyte support including UTF-8. I tried
> > UTF-8 with sqsh and got garbage characters (SQSH is not unicode clean),
> > but I fear if the UTF-8 strings exceed the size of the column from
> > which
> > they came there could be a buffer overrun in the application. So,
> > please ensure your apps can handle it before using. I'm looking at
> > what
> > support needs to be added in ctlib/dblib (specifically ct_describe) to
> > alleviate this problem. However, many apps will make assumptions based
> > on single character charsets, and dblib especially allow the apps to
> > not
> > pass a buffer length when binding. Be careful!
>
> Hmm, sounds lke this should be fun to play with. :)

And a little dangerous.

> Actually, wrt sqsh being unicode clean -- the problem is more likely to
> be with your terminal than with sqsh itself, I suspect.

Ah, my bad...of course sqsh is only dumping what freetds gives it to the
screen (terminal). I should have said, my setup is not unicode clean. ;-)

> Cheers,
> Steve Langasek
> postmodern programmer
>

Cheers,

Brian





Archive powered by MHonArc 2.6.24.

Top of Page