Skip to Content.
Sympa Menu

freetds - RE: [freetds] Character set problem

freetds AT lists.ibiblio.org

Subject: FreeTDS Development Group

List archive

Chronological Thread  
  • From: "Lowden, James K" <LowdenJK AT bernstein.com>
  • To: "FreeTDS Development Group" <freetds AT lists.ibiblio.org>
  • Subject: RE: [freetds] Character set problem
  • Date: Wed, 15 Sep 2004 11:59:30 -0400

-----Original Message-----
From: Hannu Niemi
Sent: Wednesday, September 15, 2004 5:12 AM

In short, the problem is our national characters (with umlauts) that
are converted to question marks (in SELECT queries at least). When
debugging for example 'a umlaut' is hex 84 (CP850?). The server
collation is SQL_Scandinavian_CP850_CI_AS.
---End Original Message---

Hej Hannu!

I assume you're using TDS 0.62 or later.

Show us your freetds.conf and a short TDSDUMP log, just to make sure things
are set up right. It would be good if the log included a query with a failed
conversion.

What is the column's datatype? Unicode datatypes (nchar, nvarchar) have seen
more testing than single-byte charsets. Are both broken for you?

http://www.microsoft.com/globaldev/nlsweb/default.asp?submitted=40b
http://www.microsoft.com/globaldev/reference/oem/850.htm
http://www.microsoft.com/globaldev/reference/sbcs/1252.htm


As a Finnish speaker, is your server configured for Locale ID 0x40B? In
src/tds/iconv.c::lcid2charset(), that's interpreted as code page 1252. That
may not be right. You said your 'a umlaut' is hex 84, which implies CP850;
for CP1252, it's 0xE4. 0x84 is a DOUBLE LOW-9 QUOTATION MARK, which quite
likely isn't in your client character set, causing FreeTDS to render it as
'?'.

IOW, if your server uses Locale ID 0x40B, I think FreeTDS may be mapping your
server's char/varchar data according to the wrong charset. It's using
charset 1252, when it should use charset 850.

What do these queries return?

1> sp_helpsort
2> select char(0xE4), char(0x84)
3> go
Server default collation
------------------------------
Latin1-General, binary sort

- -
รค ,

I don't see a conversion to any 850 charset, meaning FreeTDS won't tell
iconv(3) that char/varchar data arriving from the server are encoded as
CP850. If that's the problem, your TDSDUMP log should provide enough
information for us to generate a patch.

By the way, as long as it's broken ;-) you shouldn't be able to insert 'a
umlaut', either. The INSERT statement should fail (not convert your data to
'?').

Halsingar,

--jkl

-----------------------------------------
The information contained in this transmission may contain privileged and
confidential information and is intended only for the use of the person(s)
named above. If you are not the intended recipient, or an employee or agent
responsible for delivering this message to the intended recipient, any
review, dissemination, distribution or duplication of this communication is
strictly prohibited. If you are not the intended recipient, please contact
the sender immediately by reply e-mail and destroy all copies of the original
message. Please note that we do not accept account orders and/or instructions
by e-mail, and therefore will not be responsible for carrying out such orders
and/or instructions.
If you, as the intended recipient of this message, the purpose of which is to
inform and update our clients, prospects and consultants of developments
relating to our services and products, would not like to receive further
e-mail correspondence from the sender, please "reply" to the sender
indicating your wishes. In the U.S.: 1345 Avenue of the Americas, New York,
NY 10105.






Archive powered by MHonArc 2.6.24.

Top of Page