Skip to Content.
Sympa Menu

freetds - Re: [freetds] Character conversion error withspecialcharacters(umlauts)

freetds AT lists.ibiblio.org

Subject: FreeTDS Development Group

List archive

Chronological Thread  
  • From: "ZIGLIO, Frediano, VF-IT" <Frediano.Ziglio AT vodafone.com>
  • To: "FreeTDS Development Group" <freetds AT lists.ibiblio.org>
  • Subject: Re: [freetds] Character conversion error withspecialcharacters(umlauts)
  • Date: Fri, 18 Jul 2008 15:09:07 +0200


> >> Hello everyone,
> >>
> >> After about a day of googling I've given up on tha path, so I'll
> >> bother you people instead.
> >>
> >> Basically I've got an Ubuntu server with Apache, mod_perl and the
> >> usual stuff. I've written a simple webapp that
> communicates with an
> >> MSSQL 8 on a Win2K3 box. This is being done through the (I guess
> >> typical) method of Perl package
> >> (DBI) talking to unixODBC/FreeTDS that talks to the server.
> >>
> <> Normal operation works great, no problems at all. But as soon as I
> >> showed my small app to my boss he broke it. I had totally
> forgotten to
> >> use non-english words when testing it (I'm a swede, but I code "in
> >> english" and thus I add test data in english too, old habit). So,
> >> adding words with åäö in an INSERT statement broke the application.
> >>
> >> Checking logs gave me an error: [unixODBC][FreeTDS][SQL
> Server]Error
> >> converting client characters into server's character set. Some
> >> character(s) could not be converted.
> >>
> >> Google helped me narrow it down, but I found no way to
> solve it. I've
> >> checked that iconv can translate it, it does so perfectly.
> I've tried
> >> some "common" special characters (like the ' ) and it
> works just fine.
> >> I've made sure I use nvarchar in the database (missed that
> at first,
> >> but changing it didn't help). My best guess atm is that
> the characters
> >> I want to use will "grow" during conversion (i.e. each
> byte becomes
> >> two
> >> bytes) between UTF-2 and UCS-2LE.
> >>
> >> I've put the logfile for you to have a look at if that helps:
> >> http://pastebin.com/m3a90fc7b
> >>
> >> Many thanks in advance,
> >> Stefan Axelsson
> >>
> >
> >I think it would be useful to see which characters are inserted.
> >Also did you try ClientCharset option (I don't know your
> configuration) ??
> >
> >freddy77
>
>
> None are inserted if I use the more national characters. (In
> extended ASCII that would be (in dec) 132, 134,, 142, 143,
> 148, 153 (åäöÅÄÖ) <-- If they get through correctly formated.
> aaoAAO with dots and rings...
>

Yes, using Latin1 or "similar" encoding.

> If I've missunderstoof you now, do tell me. I'm still quite
> fresh when it comes to this (old MySQL user realy), and thus
> I might just missunderstand a few things here and there. ;)
>
> I've set the Client Charset option (both in freetds.conf and
> in odbc.ini to be sure) to UTF-8. Also tried a few others
> just to make sure, doesn't help any.
>

Remember that in utf8 above characters are not encoded using single characters

> Basically this is my configs: (defaults are kept, so I'll
> just post the system specifics)
>
> ----- /etc/freetds/freetds.conf -----
>
> [CMDB]
> host = anonymized.host.name.com
> port = 1433
> tds version = 8.0
> client charset = UTF-8
>
> -------------------------------------
>
>
> ----- /etc/odbc.ini -----------------
>
> [ODBC Data Sources]
> CMDB = Microsoft SQL Server 2003
>
> [CMDB]
> Driver = /usr/lib/libtdsodbc.so
> Description = CMDB Data
> Trace = Yes
> TraceFile = /tmp/cmdb_tds.log
> Server = anonymized.host.name.com
> Database = cmdb
> TDS_Version = 8.0
>
> -------------------------------------
>

This is odbc-only so freetds.conf should not be used. Default charset is
iso8859-1 so encoding like you suggested should work.

freddy77

Attachment: smime.p7s
Description: S/MIME cryptographic signature




Archive powered by MHonArc 2.6.24.

Top of Page