[freetds] connecting and character sets

Bryan Olmstead (imap) bryan at chfund.org
Wed Sep 26 20:51:46 EDT 2007


first i want to apologize if i've overlooked something obvious and the
answer is sitting right in front of me.

i recently ran a query that returned a row with a question mark and the
line: "WARNING! Some character(s) could not be converted into client's
character set. Unconverted bytes were changed to question marks ('?')".

searching google and the mailing list i found that i should be able to
fix this by either setting "client charset" in the freetds.conf file or
the LANG environment variable.  i've tested this out and neither seems
to have worked as i still receive the warning and the question mark.  if
anyone could give me some advice i would greatly appreciate it.

and before i forget, it's version 0.64 on slackware connecting to mssql.

here are some examples and the conf file.

this one is with no LANG changes, just the default C:

olmstead at alpha:~/ecw-sql/t> setenv | grep LANG ; setenv | grep LC
LANG=C
LC_ALL=POSIX
olmstead at alpha:~/ecw-sql/t> tsql -S test -U sa
locale is "C"
locale charset is "ANSI_X3.4-1968"
Password:
1> exit

olmstead at alpha:~/ecw-sql/t> head -15 dump.log
util.c:288:Starting log file for FreeTDS 0.64
        on 2007-09-26 17:45:35 with debug flags 0x4fff.
iconv.c:195:names for ISO-8859-1: ISO-8859-1
iconv.c:195:names for UTF-8: UTF-8
iconv.c:195:names for UCS-2LE: UCS-2LE
iconv.c:195:names for UCS-2BE: UCS-2BE
iconv.c:361:iconv to convert client-side data to the "ANSI_X3.4-1968"
character set
iconv.c:514:tds_iconv_info_init: converting "US-ASCII"->"UCS-2LE"
iconv.c:514:tds_iconv_info_init: converting "ISO-8859-1"->"UCS-2LE"


now i changed LANG to en_US:

olmstead at alpha:~/ecw-sql/t> setenv LANG en_US
olmstead at alpha:~/ecw-sql/t> setenv | grep LANG ; setenv | grep LC
LANG=en_US
LC_ALL=POSIX
olmstead at alpha:~/ecw-sql/t> tsql -S test -U sa
locale is "C"
locale charset is "ANSI_X3.4-1968"
Password:
1> exit

olmstead at alpha:~/ecw-sql/t> head -15 dump.log
util.c:288:Starting log file for FreeTDS 0.64
        on 2007-09-26 17:23:57 with debug flags 0x4fff.
iconv.c:195:names for ISO-8859-1: ISO-8859-1
iconv.c:195:names for UTF-8: UTF-8
iconv.c:195:names for UCS-2LE: UCS-2LE
iconv.c:195:names for UCS-2BE: UCS-2BE
iconv.c:361:iconv to convert client-side data to the "ANSI_X3.4-1968"
character set
iconv.c:514:tds_iconv_info_init: converting "US-ASCII"->"UCS-2LE"
iconv.c:514:tds_iconv_info_init: converting "ISO-8859-1"->"UCS-2LE"
net.c:168:Connecting to 127.0.0.1 port 1434.
write.c:134:tds_put_string converting 5 bytes of "alpha"
write.c:162:tds_put_string wrote 10 bytes


and i also tried it with LANG en_US.iso88591

olmstead at alpha:~/ecw-sql/t> setenv LANG en_US.iso88591
olmstead at alpha:~/ecw-sql/t> setenv | grep LANG ; setenv | grep LC
LANG=en_US.iso88591
LC_ALL=POSIX
olmstead at alpha:~/ecw-sql/t> tsql -S test -U sa
locale is "C"
locale charset is "ANSI_X3.4-1968"
Password:
1> exit

olmstead at alpha:~/ecw-sql/t> head -20 dump.log
util.c:288:Starting log file for FreeTDS 0.64
        on 2007-09-26 17:32:48 with debug flags 0x4fff.
iconv.c:195:names for ISO-8859-1: ISO-8859-1
iconv.c:195:names for UTF-8: UTF-8
iconv.c:195:names for UCS-2LE: UCS-2LE
iconv.c:195:names for UCS-2BE: UCS-2BE
iconv.c:361:iconv to convert client-side data to the "ANSI_X3.4-1968"
character set
iconv.c:514:tds_iconv_info_init: converting "US-ASCII"->"UCS-2LE"
iconv.c:514:tds_iconv_info_init: converting "ISO-8859-1"->"UCS-2LE"
net.c:168:Connecting to 127.0.0.1 port 1434.
write.c:134:tds_put_string converting 5 bytes of "alpha"


it looks like all three behaved the same.

just in case anyone is wondering, these are the en_US currently:

en_US
en_US.iso88591

and the freetds.conf section:

[test]
        host = 127.0.0.1
        port = 1434
        tds version = 7.0
        client charset = UTF-8

localhost because i connect through a ssh tunnel.

please let me know if anymore information would be helpful.

thanks in advance,

-bryan


More information about the FreeTDS mailing list