FreeTDS Development Group

[Niklas Lampén <Niklas.Lampen AT eventusmedia.fi>]

> You might want to set TDSDUMPCONFIG and TDSDUMP in your environment
> and review the resulting logs to see what client character set you're
> actually getting.

I applied the patch, build freetds again and tried and...not luck.

Commands run:

# tsql -S AavaDb2 -U ***** -P ***** -D *****
locale is "en_US.UTF-8"
locale charset is "UTF-8"
using default charset "UTF-8"
Default database being set to *****
1> UPDATE Mod_Texts SET ContentData='foo Найти' WHERE ContentId=6824;
2> SELECT ContentData FROM Mod_Texts WHERE ContentId=6824;
3> GO
ContentData
foo ?????
(1 row affected)
1> exit


Here's what TDSDUMPCONFIG says:
-----
log.c:196:Starting log file for FreeTDS 0.83.dev.20100522
        on 2010-06-02 16:06:59 with debug flags 0x4fff.
config.c:185:Getting connection information for [AavaDb2].
config.c:189:Attempting to read conf files.
config.c:346:... $FREETDSCONF not set.  Trying $FREETDS/etc.
config.c:359:... $FREETDS not set.  Trying $HOME.
config.c:285:Could not open '/root/.freetds.conf' ((.freetds.conf)).
config.c:289:Found conf file '/usr/local/etc/freetds.conf' (default).
config.c:475:Looking for section global.
config.c:532:   Found section global.
config.c:535:Got a match.
config.c:557:   text size = '64512'
config.c:557:   port = '1433'
config.c:557:   tds version = '8.0'
config.c:557:   client charset = 'UTF-8'
config.c:602:tds_parse_conf_section: client charset is UTF-8.
config.c:532:   Found section aavadb.
config.c:532:   Found section aavadb2.
config.c:546:   Reached EOF
config.c:475:Looking for section AavaDb2.
config.c:532:   Found section global.
config.c:532:   Found section aavadb.
config.c:532:   Found section aavadb2.
config.c:535:Got a match.
config.c:557:   tds version = '8.0'
config.c:557:   host = '*****'
config.c:584:Found host entry *****.
config.c:588:IP addr is *****.
config.c:557:   client charset = 'UTF-8'
config.c:602:tds_parse_conf_section: client charset is UTF-8.
config.c:557:   dump file = '/var/log/freetds/freetds.log'
config.c:557:   dump file append = 'yes'
config.c:546:   Reached EOF
config.c:293:Success: [AavaDb2] defined in /usr/local/etc/freetds.conf.
config.c:694:Setting 'dump_file' to '/var/log/freetds/freetds.log' from 
$TDSDUMP.
config.c:217:Final connection parameters:
config.c:218:            server_name = AavaDb2
config.c:219:       server_host_name = *****
config.c:220:                ip_addr = *****
config.c:221:          instance_name =
config.c:222:                   port = 1433
config.c:223:          major_version = 7
config.c:224:          minor_version = 1
config.c:225:             block_size = 0
config.c:226:               language = us_english
config.c:227:         server_charset = iso_1
config.c:228:        connect_timeout = 0
config.c:229:       client_host_name = *****
config.c:230:         client_charset = UTF-8
config.c:231:               app_name = TSQL
config.c:232:              user_name = *****
config.c:235:                library = TDS-Library
config.c:236:              bulk_copy = 0
config.c:237:      suppress_language = 0
config.c:238:          encrypt level = 0
config.c:239:          query_timeout = 0
config.c:242:               database =
config.c:243:              dump_file = /var/log/freetds/freetds.log
config.c:244:            debug_flags = 0
config.c:245:              text_size = 64512
config.c:246:           broken_dates = 0
config.c:247:     emul_little_endian = 0
-----

And this is the result from debug:
-----
log.c:196:Starting log file for FreeTDS 0.83.dev.20100522
        on 2010-06-02 16:06:59 with debug flags 0x4fff.
iconv.c:337:tds_iconv_open(0x8086a70, UTF-8)
iconv.c:197:local name for ISO-8859-1 is ISO-8859-1
iconv.c:197:local name for UTF-8 is UTF-8
iconv.c:197:local name for UCS-2LE is UCS-2LE
iconv.c:197:local name for UCS-2BE is UCS-2BE
iconv.c:365:setting up conversions for client charset "UTF-8"
iconv.c:367:preparing iconv for "UTF-8" <-> "UCS-2LE" conversion
iconv.c:408:preparing iconv for "ISO-8859-1" <-> "UCS-2LE" conversion
iconv.c:411:tds_iconv_open: done
net.c:225:Connecting to ***** port 1433 (TDS version 7.1)
net.c:281:tds_open_socket: connect(2) returned "Operation now in progress"
net.c:321:tds_open_socket() succeeded
util.c:162:Changed query state from DEAD to IDLE
net.c:792:Sending packet
0000 12 01 00 34 00 00 00 00-00 00 15 00 06 01 00 1b |...4.... ........|
0010 00 01 02 00 1c 00 0c 03-00 28 00 04 ff 08 00 01 |........ .(......|
0020 55 00 00 02 4d 53 53 51-4c 53 65 72 76 65 72 00 |U...MSSQ LServer.|
0030 4e 48 00 00            -                        |NH..|

net.c:608:Received header
0000 04 01 00 25 00 00 01 00-                        |...%....|

net.c:662:Received packet
0000 04 01 00 25 00 00 01 00-00 00 15 00 06 01 00 1b |...%.... ........|
0010 00 01 02 00 1c 00 01 03-00 1d 00 00 ff 0a 00 09 |........ ........|
0020 e3 00 00 02 00         -                        |.....|

login.c:1044:detected flag 2
login.c:771:quietly sending TDS 7+ login packet
token.c:328:tds_process_login_tokens()
net.c:608:Received header
0000 04 01 01 a5 00 38 01 00-                        |.....8..|

net.c:662:Received packet
<CLIP>

token.c:337:looking for login token, got  e3(ENVCHANGE)
token.c:122:tds_process_default_tokens() marker is e3(ENVCHANGE)
token.c:337:looking for login token, got  ab(INFO)
token.c:122:tds_process_default_tokens() marker is ab(INFO)
token.c:2708:tds_process_msg() reading message 5701 from server
token.c:2781:tds_process_msg() calling client msg handler
token.c:2794:tds_process_msg() returning TDS_SUCCEED
token.c:337:looking for login token, got  e3(ENVCHANGE)
token.c:122:tds_process_default_tokens() marker is e3(ENVCHANGE)
token.c:2539:tds_process_env_chg(): 5 bytes of collation data received
token.c:2540:tds->collation was
0000 00 00 00 00 00         -                        |.....|

iconv.c:1025:setting server single-byte charset to "CP1252"
token.c:2550:tds->collation now
0000 0b 04 f0 20 00         -                        |... .|

token.c:337:looking for login token, got  e3(ENVCHANGE)
token.c:122:tds_process_default_tokens() marker is e3(ENVCHANGE)
token.c:337:looking for login token, got  ab(INFO)
token.c:122:tds_process_default_tokens() marker is ab(INFO)
token.c:2708:tds_process_msg() reading message 5703 from server
token.c:2781:tds_process_msg() calling client msg handler
token.c:2794:tds_process_msg() returning TDS_SUCCEED
token.c:337:looking for login token, got  ad(LOGINACK)
token.c:372:server reports TDS version 71.0.0.1
token.c:374:Product name for 0x71000001 is 2000 SP1
token.c:337:looking for login token, got  e3(ENVCHANGE)
token.c:122:tds_process_default_tokens() marker is e3(ENVCHANGE)
token.c:2597:changing block size from 4096 to 4096
token.c:337:looking for login token, got  fd(DONE)
token.c:122:tds_process_default_tokens() marker is fd(DONE)
token.c:2459:tds_process_end: more_results = 0
                was_cancelled = 0
                error = 0
                done_count_valid = 0
token.c:2475:tds_process_end() state set to TDS_IDLE
token.c:2490:                rows_affected = 0
token.c:437:tds_process_login_tokens() returning TDS_SUCCEED
mem.c:615:tds_free_all_results()
util.c:162:Changed query state from IDLE to QUERYING
write.c:140:tds_put_string converting 19 bytes of "set textsize 64512 "
write.c:168:tds_put_string wrote 38 bytes
util.c:162:Changed query state from QUERYING to PENDING
net.c:792:Sending packet
0000 01 01 00 2e 00 00 00 00-73 00 65 00 74 00 20 00 |........ s.e.t. .|
0010 74 00 65 00 78 00 74 00-73 00 69 00 7a 00 65 00 |t.e.x.t. s.i.z.e.|
0020 20 00 36 00 34 00 35 00-31 00 32 00 20 00       | .6.4.5. 1.2. .|

token.c:539:tds_process_tokens(0x8086a70, 0xffffb9ac, 0xffffb9a8, 0x100)
util.c:162:Changed query state from PENDING to READING
net.c:608:Received header
0000 04 01 00 11 00 38 01 00-                        |.....8..|

net.c:662:Received packet
0000 04 01 00 11 00 38 01 00-fd 00 00 be 00 00 00 00 |.....8.. ........|
0010 00                     -                        |.|

token.c:554:processing result tokens.  marker is  fd(DONE)
token.c:2459:tds_process_end: more_results = 0
                was_cancelled = 0
                error = 0
                done_count_valid = 0
token.c:2475:tds_process_end() state set to TDS_IDLE
util.c:162:Changed query state from READING to IDLE
token.c:2490:                rows_affected = 0
util.c:110:logic error: cannot change query state from IDLE to PENDING
token.c:539:tds_process_tokens(0x8086a70, 0xffffb9ac, 0xffffb9a8, 0x100)
token.c:542:tds_process_tokens() state is COMPLETED
mem.c:615:tds_free_all_results()
util.c:162:Changed query state from IDLE to QUERYING
write.c:140:tds_put_string converting 128 bytes of "UPDATE Mod_Texts SET 
ContentData='foo Найти' WHERE ContentId=6824;
SELECT ContentData FROM Mod_Texts WHERE ContentId=6824;
"
write.c:168:tds_put_string wrote 246 bytes
util.c:162:Changed query state from QUERYING to PENDING
net.c:792:Sending packet
0000 01 01 00 fe 00 00 01 00-55 00 50 00 44 00 41 00 |........ U.P.D.A.|
0010 54 00 45 00 20 00 4d 00-6f 00 64 00 5f 00 54 00 |T.E. .M. o.d._.T.|
0020 65 00 78 00 74 00 73 00-20 00 53 00 45 00 54 00 |e.x.t.s.  .S.E.T.|
0030 20 00 43 00 6f 00 6e 00-74 00 65 00 6e 00 74 00 | .C.o.n. t.e.n.t.|
0040 44 00 61 00 74 00 61 00-3d 00 27 00 66 00 6f 00 |D.a.t.a. =.'.f.o.|
0050 6f 00 20 00 1d 04 30 04-39 04 42 04 38 04 27 00 |o. ...0. 9.B.8.'.|
0060 20 00 57 00 48 00 45 00-52 00 45 00 20 00 43 00 | .W.H.E. R.E. .C.|
0070 6f 00 6e 00 74 00 65 00-6e 00 74 00 49 00 64 00 |o.n.t.e. n.t.I.d.|
0080 3d 00 36 00 38 00 32 00-34 00 3b 00 0a 00 53 00 |=.6.8.2. 4.;...S.|
0090 45 00 4c 00 45 00 43 00-54 00 20 00 43 00 6f 00 |E.L.E.C. T. .C.o.|
00a0 6e 00 74 00 65 00 6e 00-74 00 44 00 61 00 74 00 |n.t.e.n. t.D.a.t.|
00b0 61 00 20 00 46 00 52 00-4f 00 4d 00 20 00 4d 00 |a. .F.R. O.M. .M.|
00c0 6f 00 64 00 5f 00 54 00-65 00 78 00 74 00 73 00 |o.d._.T. e.x.t.s.|
00d0 20 00 57 00 48 00 45 00-52 00 45 00 20 00 43 00 | .W.H.E. R.E. .C.|
00e0 6f 00 6e 00 74 00 65 00-6e 00 74 00 49 00 64 00 |o.n.t.e. n.t.I.d.|
00f0 3d 00 36 00 38 00 32 00-34 00 3b 00 0a 00       |=.6.8.2. 4.;...|

token.c:539:tds_process_tokens(0x8086a70, 0xffffbc48, (nil), 0x6914)
util.c:162:Changed query state from PENDING to READING
net.c:608:Received header
0000 04 01 00 86 00 38 01 00-                        |.....8..|

net.c:662:Received packet
0000 04 01 00 86 00 38 01 00-fd 11 00 c5 00 01 00 00 |.....8.. ........|
0010 00 81 01 00 00 00 09 00-63 00 fc 00 00 09 04 d0 |........ c.......|
0020 00 34 09 00 4d 00 6f 00-64 00 5f 00 54 00 65 00 |.4..M.o. d._.T.e.|
0030 78 00 74 00 73 00 0b 43-00 6f 00 6e 00 74 00 65 |x.t.s..C .o.n.t.e|
0040 00 6e 00 74 00 44 00 61-00 74 00 61 00 d1 10 64 |.n.t.D.a .t.a...d|
0050 75 6d 6d 79 20 74 65 78-74 70 74 72 00 00 00 64 |ummy tex tptr...d|
0060 75 6d 6d 79 54 53 00 12-00 00 00 66 00 6f 00 6f |ummyTS.. ...f.o.o|
0070 00 20 00 3f 00 3f 00 3f-00 3f 00 3f 00 fd 10 00 |. .?.?.? .?.?....|
0080 c1 00 01 00 00 00      -                        |......|

token.c:554:processing result tokens.  marker is  fd(DONE)
token.c:2459:tds_process_end: more_results = 1
                was_cancelled = 0
                error = 0
                done_count_valid = 1
token.c:2490:                rows_affected = 1
util.c:162:Changed query state from READING to PENDING
token.c:539:tds_process_tokens(0x8086a70, 0xffffbc48, (nil), 0x6914)
util.c:162:Changed query state from PENDING to READING
token.c:554:processing result tokens.  marker is  81(TDS7_RESULT)
token.c:1570:processing TDS7 result metadata.
mem.c:615:tds_free_all_results()
token.c:1594:set current_results (1 column) to tds->res_info
token.c:1601:setting up 1 columns
token.c:3561:adjust_character_column_size:
        Server charset: UCS-2LE
        Server column_size: 64512
        Client charset: UTF-8
        Client column_size: 129024
token.c:1541:tds7_get_data_info:
        colname = ContentData (11 bytes)
        type = 35 (text)
        server's type = 99 (UCS-2 text)
        column_varint_size = 4
        column_size = 129024 (64512 on server)
token.c:1610: name                 size/wsize      type/wtype      utype
token.c:1611: -------------------- --------------- --------------- -------
token.c:1621: ContentData           129024/64512        35/99            0
util.c:162:Changed query state from READING to PENDING
token.c:539:tds_process_tokens(0x8086a70, 0xffffbc48, (nil), 0x6914)
util.c:162:Changed query state from PENDING to READING
token.c:554:processing result tokens.  marker is  d1(ROW)
token.c:665:tds_process_tokens::SET_RETURN stopping on current token
util.c:162:Changed query state from READING to PENDING
token.c:539:tds_process_tokens(0x8086a70, 0xffffbc48, (nil), 0x1508)
util.c:162:Changed query state from PENDING to READING
token.c:554:processing result tokens.  marker is  d1(ROW)
token.c:2424:tds_process_row(): reading column 0
token.c:2169:tds_get_data: type 35, varint size 4
token.c:2230:tds_get_data(): wire column size is 18
util.c:162:Changed query state from READING to PENDING
token.c:539:tds_process_tokens(0x8086a70, 0xffffbc48, (nil), 0x1508)
util.c:162:Changed query state from PENDING to READING
token.c:554:processing result tokens.  marker is  fd(DONE)
token.c:2459:tds_process_end: more_results = 0
                was_cancelled = 0
                error = 0
                done_count_valid = 1
token.c:2475:tds_process_end() state set to TDS_IDLE
util.c:162:Changed query state from READING to IDLE
token.c:2490:                rows_affected = 1
util.c:110:logic error: cannot change query state from IDLE to PENDING
token.c:539:tds_process_tokens(0x8086a70, 0xffffbc48, (nil), 0x6914)
token.c:542:tds_process_tokens() state is COMPLETED
mem.c:615:tds_free_all_results()
util.c:162:Changed query state from IDLE to DEAD
-----

I hope this info helps us to solve this annoying issue I'm having.


Niklas