Skip to Content.
Sympa Menu

freetds - [freetds] client charset, character data encoding

freetds AT lists.ibiblio.org

Subject: FreeTDS Development Group

List archive

Chronological Thread  
  • From: LacaK <lacak AT zoznam.sk>
  • To: FreeTDS Development Group <freetds AT lists.ibiblio.org>
  • Subject: [freetds] client charset, character data encoding
  • Date: Tue, 13 Sep 2011 09:51:57 +0200

Hi,
I have problems with correct character data encoding, when reading data from MS SQL Server (using DB-Lib), where I have table with char/varchar columns, which use CP1250 encoding .
My freetds.conf is:
[global]
tds version = 7.2
client charset = UTF-8
dump file = freetds.log

From freetds.log:
log.c:196:Starting log file for FreeTDS 0.83.dev
on 2011-09-13 08:45:51 with debug flags 0x4fff.
iconv.c:337:tds_iconv_open(026B0860, UTF-8)
iconv.c:312:Using trivial iconv
iconv.c:197:local name for ISO-8859-1 is ISO-8859-1
iconv.c:197:local name for UTF-8 is UTF-8
iconv.c:197:local name for UCS-2LE is UCS-2LE
iconv.c:197:local name for UCS-2BE is UCS-2BE
iconv.c:365:setting up conversions for client charset "UTF-8"
iconv.c:367:preparing iconv for "UTF-8" <-> "UCS-2LE" conversion
iconv.c:408:preparing iconv for "ISO-8859-1" <-> "UCS-2LE" conversion
iconv.c:411:tds_iconv_open: done
net.c:205:Connecting to 10.0.0.2 port 1433 (TDS version 7.1)
...
iconv.c:1025:setting server single-byte charset to "CP1250"
iconv.c:487:"CP1250" not supported by iconv, using "ISO-8859-1" instead <--- may be here is source of problem ???
...
net.c:600:Received packet
0000 04 01 00 92 00 3c 01 00-81 02 00 00 00 10 00 38 |.....<.. .......8|
0010 0b 69 00 64 00 65 00 6e-00 74 00 5f 00 66 00 69 |.i.d.e.n .t._.f.i|
0020 00 65 00 6c 00 64 00 00-00 09 00 a7 0a 00 1b 04 |.e.l.d.. ........|
0030 f0 00 00 09 73 00 74 00-72 00 5f 00 66 00 69 00 |....s.t. r._.f.i.|
0040 65 00 6c 00 64 00 d1 03-00 00 00 09 00 73 74 72 |e.l.d... .....str|
0050 33 *9a e8 9d 9e* 33 d1 04-00 00 00 0a 00 64 65 66 |3....3.. .....def| <--- here data comes (9a e8 9d 9e which corresponds to "ščťž" in Win1250 encoding)
...
Why data comes in server original (SBCS) encoding ?
My understanding of TDS 7+ was, that server always send UCS-2 encoded character data (may be, that it is true only for nchar/nvarchar and metadata) (http://www.freetds.org/userguide/localization.htm) ?
(so iconv can always convert them into UTF-8) ... but now it seems, that character data (other than nchar/nvarchar) are in server SBCS.
(so task convert to utf-8 is leaved to freeTDS)

If it is so. Then can I set up freeTDS to not handle this data as ISO-8859-1 encoded ?
(I would prefer if unsupported conversion will be detected, then use for example current system character encoding rather than ISO-8859-1 ... IMO in most cases real character encoding will be different than ISO-8859-1 ... which will lead to wrong results)
Or is there any other way how to receive correct results ?

TIA
-Laco.





Archive powered by MHonArc 2.6.24.

Top of Page