FreeTDS Development Group

[Frediano Ziglio <freddy77 AT gmail.com>]

2011/9/13 LacaK <lacak AT zoznam.sk>:
> Hi,
> I have problems with correct character data encoding, when reading data from
> MS SQL Server (using DB-Lib), where I have table with char/varchar columns,
> which use CP1250 encoding .
> My freetds.conf is:
> [global]
> tds version = 7.2
> client charset = UTF-8
> dump file = freetds.log
>
> From freetds.log:
> log.c:196:Starting log file for FreeTDS 0.83.dev
>   on 2011-09-13 08:45:51 with debug flags 0x4fff.
> iconv.c:337:tds_iconv_open(026B0860, UTF-8)
> iconv.c:312:Using trivial iconv

This tell that you compile FreeTDS without iconv support. In this case
CP1250 is not supported. You should enable iconv support (probably you
don't have development libraries for libiconv).

> iconv.c:197:local name for ISO-8859-1 is ISO-8859-1
> iconv.c:197:local name for UTF-8 is UTF-8
> iconv.c:197:local name for UCS-2LE is UCS-2LE
> iconv.c:197:local name for UCS-2BE is UCS-2BE
> iconv.c:365:setting up conversions for client charset "UTF-8"
> iconv.c:367:preparing iconv for "UTF-8" <-> "UCS-2LE" conversion
> iconv.c:408:preparing iconv for "ISO-8859-1" <-> "UCS-2LE" conversion
> iconv.c:411:tds_iconv_open: done
> net.c:205:Connecting to 10.0.0.2 port 1433 (TDS version 7.1)
> ...
> iconv.c:1025:setting server single-byte charset to "CP1250"
> iconv.c:487:"CP1250" not supported by iconv, using "ISO-8859-1" instead
>  <--- may be here is source of problem ???
> ...

This is due to trivial iconv use.

> net.c:600:Received packet
> 0000 04 01 00 92 00 3c 01 00-81 02 00 00 00 10 00 38 |.....<.. .......8|
> 0010 0b 69 00 64 00 65 00 6e-00 74 00 5f 00 66 00 69 |.i.d.e.n .t._.f.i|
> 0020 00 65 00 6c 00 64 00 00-00 09 00 a7 0a 00 1b 04 |.e.l.d.. ........|
> 0030 f0 00 00 09 73 00 74 00-72 00 5f 00 66 00 69 00 |....s.t. r._.f.i.|
> 0040 65 00 6c 00 64 00 d1 03-00 00 00 09 00 73 74 72 |e.l.d... .....str|
> 0050 33 *9a e8 9d 9e* 33 d1 04-00 00 00 0a 00 64 65 66 |3....3.. .....def|
>  <--- here data comes (9a e8 9d 9e which corresponds to "ה‹יל" in Win1250
> encoding)
> ...
> Why data comes in server original (SBCS) encoding ?
> My understanding of TDS 7+ was, that server always send UCS-2 encoded
> character data (may be, that it is true only for nchar/nvarchar and
> metadata) (http://www.freetds.org/userguide/localization.htm) ?

Metadata are always UCS-2/UTF-16, varchar/char/text use SBCS on wire.

> (so iconv can always convert them into UTF-8) ... but now it seems, that
> character data (other than nchar/nvarchar) are in server SBCS.
> (so task convert to utf-8 is leaved to freeTDS)
>
> If it is so. Then can I set up freeTDS to not handle this data as ISO-8859-1
> encoded ?
> (I would prefer if unsupported conversion will be detected, then use for
> example current system character encoding rather than ISO-8859-1 ... IMO in
> most cases real character encoding will be different than ISO-8859-1 ...
> which will lead to wrong results)
> Or is there any other way how to receive correct results ?
>
> TIA
> -Laco.
>

Regards
  Frediano