Skip to Content.
Sympa Menu

freetds - Re: [freetds] connecting and character sets

freetds AT lists.ibiblio.org

Subject: FreeTDS Development Group

List archive

Chronological Thread  
  • From: "Bryan Olmstead (imap)" <bryan AT chfund.org>
  • To: FreeTDS Development Group <freetds AT lists.ibiblio.org>
  • Subject: Re: [freetds] connecting and character sets
  • Date: Thu, 27 Sep 2007 08:59:20 -0700

> Bryan Olmstead (imap) wrote:
>> i recently ran a query that returned a row with a question mark and the
>> line: "WARNING! Some character(s) could not be converted into client's
>> character set. Unconverted bytes were changed to question marks ('?')".
>>
>> searching google and the mailing list i found that i should be able to
>> fix this by either setting "client charset" in the freetds.conf file or
>> the LANG environment variable. i've tested this out and neither seems
>> to have worked as i still receive the warning and the question mark. if
>> anyone could give me some advice i would greatly appreciate it.
>>
>> and before i forget, it's version 0.64 on slackware connecting to mssql.
> ...
>> just in case anyone is wondering, these are the en_US currently:
>>
>> en_US
>> en_US.iso88591
>>
>> and the freetds.conf section:
>>
>> [test]
>> host = 127.0.0.1
>> port = 1434
>> tds version = 7.0
>> client charset = UTF-8
>
> I think what's happening is caused by setting LC_ALL to POSIX. That
> supersedes freetds.conf and tells libc you want ASCII (a/k/a
> ANSI_X3.4-1968).
>
> http://www.opengroup.org/onlinepubs/007908799/xbd/envvar.html
>
> "LANG
> This variable determines the locale category for native language, local
> customs and coded character set in the absence of the LC_ALL"
>
> I.e. LC_ALL supersedes LANG.
>
> tsql reports the results of setlocale(3) and nl_langinfo(3). If you set
> your LC_ALL to en_US.iso88591 or en_US.UTF8 (maybe, depends on your iconv
> implementation) tsql should report using it instead of 'locale charset is
> "ANSI_X3.4-1968"'.

thanks, that really helps put things into perspective. unfortunately,
i'm still getting 'locale charset is "ANSI_X3.4-1968"' when connecting!

i set both LANG and LC_ALL to en_IN.utf8 to test out a different one and
just to make sure:

olmstead@alpha:~/ecw-sql/t> setenv | grep LC ; setenv | grep LANG
LC_ALL=en_IN.utf8
LANG=en_IN.utf8
olmstead@alpha:~/ecw-sql/t> tsql -S penchart -U sa
locale is "C"
locale charset is "ANSI_X3.4-1968"
Password:
1> exit
olmstead@alpha:~/ecw-sql/t> head -12 dump.log
util.c:288:Starting log file for FreeTDS 0.64
on 2007-09-27 08:54:02 with debug flags 0x4fff.
iconv.c:195:names for ISO-8859-1: ISO-8859-1
iconv.c:195:names for UTF-8: UTF-8
iconv.c:195:names for UCS-2LE: UCS-2LE
iconv.c:195:names for UCS-2BE: UCS-2BE
iconv.c:361:iconv to convert client-side data to the "ANSI_X3.4-1968"
character set
iconv.c:514:tds_iconv_info_init: converting "US-ASCII"->"UCS-2LE"
iconv.c:514:tds_iconv_info_init: converting "ISO-8859-1"->"UCS-2LE"
net.c:168:Connecting to 127.0.0.1 port 1434.
write.c:134:tds_put_string converting 5 bytes of "alpha"
write.c:162:tds_put_string wrote 10 bytes


then i tried unsetting both just to see:

olmstead@alpha:~/ecw-sql/t> setenv | grep LC ; setenv | grep LANG
olmstead@alpha:~/ecw-sql/t> tsql -S penchart -U sa
locale is "C"
locale charset is "ANSI_X3.4-1968"
Password:
1> exit
olmstead@alpha:~/ecw-sql/t> head -12 dump.log
util.c:288:Starting log file for FreeTDS 0.64
on 2007-09-27 08:51:47 with debug flags 0x4fff.
iconv.c:195:names for ISO-8859-1: ISO-8859-1
iconv.c:195:names for UTF-8: UTF-8
iconv.c:195:names for UCS-2LE: UCS-2LE
iconv.c:195:names for UCS-2BE: UCS-2BE
iconv.c:361:iconv to convert client-side data to the "ANSI_X3.4-1968"
character set
iconv.c:514:tds_iconv_info_init: converting "US-ASCII"->"UCS-2LE"
iconv.c:514:tds_iconv_info_init: converting "ISO-8859-1"->"UCS-2LE"
net.c:168:Connecting to 127.0.0.1 port 1434.
write.c:134:tds_put_string converting 5 bytes of "alpha"
write.c:162:tds_put_string wrote 10 bytes

thanks again,

-bryan





Archive powered by MHonArc 2.6.24.

Top of Page