FreeTDS Development Group

["Bryan Olmstead (imap)" <bryan AT chfund.org>]

first i want to apologize if i've overlooked something obvious and the
answer is sitting right in front of me.

i recently ran a query that returned a row with a question mark and the
line: "WARNING! Some character(s) could not be converted into client's
character set. Unconverted bytes were changed to question marks ('?')".

searching google and the mailing list i found that i should be able to
fix this by either setting "client charset" in the freetds.conf file or
the LANG environment variable.  i've tested this out and neither seems
to have worked as i still receive the warning and the question mark.  if
anyone could give me some advice i would greatly appreciate it.

and before i forget, it's version 0.64 on slackware connecting to mssql.

here are some examples and the conf file.

this one is with no LANG changes, just the default C:

olmstead@alpha:~/ecw-sql/t> setenv | grep LANG ; setenv | grep LC
LANG=C
LC_ALL=POSIX
olmstead@alpha:~/ecw-sql/t> tsql -S test -U sa
locale is "C"
locale charset is "ANSI_X3.4-1968"
Password:
1> exit

olmstead@alpha:~/ecw-sql/t> head -15 dump.log
util.c:288:Starting log file for FreeTDS 0.64
        on 2007-09-26 17:45:35 with debug flags 0x4fff.
iconv.c:195:names for ISO-8859-1: ISO-8859-1
iconv.c:195:names for UTF-8: UTF-8
iconv.c:195:names for UCS-2LE: UCS-2LE
iconv.c:195:names for UCS-2BE: UCS-2BE
iconv.c:361:iconv to convert client-side data to the "ANSI_X3.4-1968"
character set
iconv.c:514:tds_iconv_info_init: converting "US-ASCII"->"UCS-2LE"
iconv.c:514:tds_iconv_info_init: converting "ISO-8859-1"->"UCS-2LE"


now i changed LANG to en_US:

olmstead@alpha:~/ecw-sql/t> setenv LANG en_US
olmstead@alpha:~/ecw-sql/t> setenv | grep LANG ; setenv | grep LC
LANG=en_US
LC_ALL=POSIX
olmstead@alpha:~/ecw-sql/t> tsql -S test -U sa
locale is "C"
locale charset is "ANSI_X3.4-1968"
Password:
1> exit

olmstead@alpha:~/ecw-sql/t> head -15 dump.log
util.c:288:Starting log file for FreeTDS 0.64
        on 2007-09-26 17:23:57 with debug flags 0x4fff.
iconv.c:195:names for ISO-8859-1: ISO-8859-1
iconv.c:195:names for UTF-8: UTF-8
iconv.c:195:names for UCS-2LE: UCS-2LE
iconv.c:195:names for UCS-2BE: UCS-2BE
iconv.c:361:iconv to convert client-side data to the "ANSI_X3.4-1968"
character set
iconv.c:514:tds_iconv_info_init: converting "US-ASCII"->"UCS-2LE"
iconv.c:514:tds_iconv_info_init: converting "ISO-8859-1"->"UCS-2LE"
net.c:168:Connecting to 127.0.0.1 port 1434.
write.c:134:tds_put_string converting 5 bytes of "alpha"
write.c:162:tds_put_string wrote 10 bytes


and i also tried it with LANG en_US.iso88591

olmstead@alpha:~/ecw-sql/t> setenv LANG en_US.iso88591
olmstead@alpha:~/ecw-sql/t> setenv | grep LANG ; setenv | grep LC
LANG=en_US.iso88591
LC_ALL=POSIX
olmstead@alpha:~/ecw-sql/t> tsql -S test -U sa
locale is "C"
locale charset is "ANSI_X3.4-1968"
Password:
1> exit

olmstead@alpha:~/ecw-sql/t> head -20 dump.log
util.c:288:Starting log file for FreeTDS 0.64
        on 2007-09-26 17:32:48 with debug flags 0x4fff.
iconv.c:195:names for ISO-8859-1: ISO-8859-1
iconv.c:195:names for UTF-8: UTF-8
iconv.c:195:names for UCS-2LE: UCS-2LE
iconv.c:195:names for UCS-2BE: UCS-2BE
iconv.c:361:iconv to convert client-side data to the "ANSI_X3.4-1968"
character set
iconv.c:514:tds_iconv_info_init: converting "US-ASCII"->"UCS-2LE"
iconv.c:514:tds_iconv_info_init: converting "ISO-8859-1"->"UCS-2LE"
net.c:168:Connecting to 127.0.0.1 port 1434.
write.c:134:tds_put_string converting 5 bytes of "alpha"


it looks like all three behaved the same.

just in case anyone is wondering, these are the en_US currently:

en_US
en_US.iso88591

and the freetds.conf section:

[test]
        host = 127.0.0.1
        port = 1434
        tds version = 7.0
        client charset = UTF-8

localhost because i connect through a ssh tunnel.

please let me know if anymore information would be helpful.

thanks in advance,

-bryan