FreeTDS Development Group

["David Nichols" <nicholsman AT gmail.com>]

Hi,

I am using freetds' ct-lib to provide a database driver to the qore
programming language for sybase and ms sql server databases.  For the
moment, I am now working on Sybase connectivity.

Everything is going pretty well with the port to FreeTDS, however,
with freetds, I get an "Unknown marker" error if I set a character
encoding different to the server's encoding in my freetds.conf file
(client message 52: severity 78: Unknown marker) and I make a simple
select.  This error is returned when the server returns a result set.
It does not happen when I compile my driver with the real Sybase
ct-lib, and it does not happen if I set the character encoding in
freetds.conf to the same encoding as the server; it only happens if
the encoding is different.  Also the problem does not go away if I
change the TDS protocol version level in the freetds.conf file (or by
using the TDSVER environment variable).

For example, I have a Sybase ASE 15 server called "barney" using
"iso_1" (iso-8859-1) encoding.  If I set my freetds.conf entry for
barney to the following:

[barney]
       host = barney
       port = 4112
       tds version = 5.0
       charset = utf8

I can see from the server messages that the character encoding for the
connection is being set properly - from my own output of the server
messages:

TID 0: server: line:0, severity:10, n:5704: Changed client character
set setting to 'utf8'.
TID 0: server: line:0, severity:10, n:5701: Changed database context
to 'master'.

I can execute SQL statements without any problem as long as they do
not return a result set.  With the first statement that returns a
result set, I get the "Unknown marker" error.

As I mentioned before, it does not happen wtih Sybase' ct-lib, it does
not happen if I set the encoding to the same value as the server (i.e.
with charset = iso_1 in the freetds.conf entry for "barney"), and the
problem does not go away if I change the tds protocol version number.

I've included a dump file from setting TDSDUMP when I got this error;
if anyone can help it would be greatly appreciated.

thanks,
David
util.c:288:Starting log file for FreeTDS 0.64
	on 2007-05-28 10:30:06 with debug flags 0x4fff.
iconv.c:195:names for ISO-8859-1: ISO-8859-1
iconv.c:195:names for UTF-8: UTF-8
iconv.c:195:names for UCS-2LE: UCS-2LE
iconv.c:195:names for UCS-2BE: UCS-2BE
iconv.c:361:iconv to convert client-side data to the "ISO-8859-1" character set
iconv.c:514:tds_iconv_info_init: converting "ISO-8859-1"->"UCS-2LE"
net.c:168:Connecting to 192.168.20.2 port 4112.
net.c:673:Sending packet
0000 02 00 02 00 00 00 00 00-61 70 6f 6c 6c 6f 00 00 |........ apollo..|
0010 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 |........ ........|
0020 00 00 00 00 00 00 06 73-61 00 00 00 00 00 00 00 |.......s a.......|
0030 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 |........ ........|
0040 00 00 00 00 00 02 00 00-00 00 00 00 00 00 00 00 |........ ........|
0050 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 |........ ........|
0060 00 00 00 00 00 33 37 38-37 36 00 00 00 00 00 00 |.....378 76......|
0070 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 |........ ........|
0080 00 00 00 05 02 00 06 04-08 01 00 00 00 00 00 00 |........ ........|
0090 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 |........ ........|
00a0 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 |........ ........|
00b0 00 00 00 62 61 72 6e 65-79 00 00 00 00 00 00 00 |...barne y.......|
00c0 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 |........ ........|
00d0 00 06 00 00 00 00 00 00-00 00 00 00 00 00 00 00 |........ ........|
00e0 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 |........ ........|
00f0 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 |........ ........|
0100 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 |........ ........|
0110 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 |........ ........|
0120 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 |........ ........|
0130 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 |........ ........|
0140 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 |........ ........|
0150 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 |........ ........|
0160 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 |........ ........|
0170 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 |........ ........|
0180 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 |........ ........|
0190 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 |........ ........|
01a0 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 |........ ........|
01b0 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 |........ ........|
01c0 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 |........ ........|
01d0 00 02 05 00 00 00 43 54-2d 4c 69 62 72 61 72 79 |......CT -Library|
01e0 0a 05 00 00 00 00 0c 10-75 73 5f 65 6e 67 6c 69 |........ us_engli|
01f0 73 68 00 00 00 00 00 00-00 00 00 00 00 00 00 00 |sh...... ........|

net.c:673:Sending packet
0000 02 01 00 61 00 00 00 00-00 00 00 00 00 00 0a 00 |...a.... ........|
0010 00 00 00 00 00 00 00 00-00 00 00 00 00 75 74 66 |........ .....utf|
0020 38 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 |8....... ........|
0030 00 00 00 00 00 00 00 00-00 00 00 04 01 35 31 32 |........ .....512|
0040 00 00 00 03 00 00 00 00-e2 00 16 01 09 00 00 06 |........ ........|
0050 6d 7f ff ff ff fe 02 09-00 00 00 00 0a 68 00 00 |m....... .....h..|
0060 00                     -                        |.|

token.c:310:tds_process_login_tokens()
net.c:446:Received header
0000 04 01 00 fd 00 00 00 00-                        |........|

net.c:542:Received packet
0000 e3 00 0c 03 04 75 74 66-38 05 69 73 6f 5f 31 e5 |.....utf 8.iso_1.|
0010 00 4b 00 00 16 48 01 0a-05 5a 5a 5a 5a 5a 00 00 |.K...H.. .ZZZZZ..|
0020 01 00 30 43 68 61 6e 67-65 64 20 63 6c 69 65 6e |..0Chang ed clien|
0030 74 20 63 68 61 72 61 63-74 65 72 20 73 65 74 20 |t charac ter set |
0040 73 65 74 74 69 6e 67 20-74 6f 20 27 75 74 66 38 |setting  to 'utf8|
0050 27 2e 0a 06 62 61 72 6e-65 79 00 00 00 e3 00 0f |'...barn ey......|
0060 01 06 6d 61 73 74 65 72-06 6d 61 73 74 65 72 e5 |..master .master.|
0070 00 41 00 00 16 45 02 0a-05 5a 5a 5a 5a 5a 00 00 |.A...E.. .ZZZZZ..|
0080 01 00 26 43 68 61 6e 67-65 64 20 64 61 74 61 62 |..&Chang ed datab|
0090 61 73 65 20 63 6f 6e 74-65 78 74 20 74 6f 20 27 |ase cont ext to '|
00a0 6d 61 73 74 65 72 27 2e-0a 06 62 61 72 6e 65 79 |master'. ..barney|
00b0 00 00 00 e3 00 06 04 03-35 31 32 00 ad 00 14 05 |........ 512.....|
00c0 05 00 00 00 0a 73 71 6c-20 73 65 72 76 65 72 0f |.....sql  server.|
00d0 00 00 00 e2 00 16 01 09-00 00 02 61 7f ff ff ff |........ ...a....|
00e0 e6 02 09 00 00 00 00 0a-00 00 00 00 fd 00 00 00 |........ ........|
00f0 02 00 00 00 1a         -                        |.....|

token.c:314:looking for login token, got  e3(ENVCHANGE)
token.c:105:tds_process_default_tokens() marker is e3(ENVCHANGE)
read.c:179:tds_get_string: reading 4 from wire to give 4 to client.
read.c:179:tds_get_string: reading 5 from wire to give 5 to client.
iconv.c:514:tds_iconv_info_init: converting "ISO-8859-1"->"UTF-8"
iconv.c:514:tds_iconv_info_init: converting "ISO-8859-1"->"UTF-8"
token.c:314:looking for login token, got  e5(EED)
token.c:105:tds_process_default_tokens() marker is e5(EED)
token.c:2439:tds_process_msg() reading message from server
read.c:179:tds_get_string: reading 48 from wire to give 48 to client.
read.c:179:tds_get_string: reading 6 from wire to give 6 to client.
token.c:2501:tds_process_msg() calling client msg handler
token.c:2514:tds_process_msg() returning TDS_SUCCEED
token.c:314:looking for login token, got  e3(ENVCHANGE)
token.c:105:tds_process_default_tokens() marker is e3(ENVCHANGE)
read.c:179:tds_get_string: reading 6 from wire to give 6 to client.
read.c:179:tds_get_string: reading 6 from wire to give 6 to client.
token.c:314:looking for login token, got  e5(EED)
token.c:105:tds_process_default_tokens() marker is e5(EED)
token.c:2439:tds_process_msg() reading message from server
read.c:179:tds_get_string: reading 38 from wire to give 38 to client.
read.c:179:tds_get_string: reading 6 from wire to give 6 to client.
token.c:2501:tds_process_msg() calling client msg handler
token.c:2514:tds_process_msg() returning TDS_SUCCEED
token.c:314:looking for login token, got  e3(ENVCHANGE)
token.c:105:tds_process_default_tokens() marker is e3(ENVCHANGE)
read.c:179:tds_get_string: reading 3 from wire to give 3 to client.
token.c:314:looking for login token, got  ad(LOGINACK)
read.c:179:tds_get_string: reading 10 from wire to give 10 to client.
token.c:314:looking for login token, got  e2(CAPABILITY)
token.c:105:tds_process_default_tokens() marker is e2(CAPABILITY)
token.c:314:looking for login token, got  fd(DONE)
token.c:105:tds_process_default_tokens() marker is fd(DONE)
token.c:2143:tds_process_end: more_results = 0
		was_cancelled = 0
		error = 0
		done_count_valid = 0
token.c:2159:tds_process_end() state set to TDS_IDLE
util.c:119:Changing query state from IDLE to IDLE
token.c:387:leaving tds_process_login_tokens() returning 1
util.c:119:Changing query state from IDLE to QUERYING
mem.c:519:tds_free_all_results()
util.c:119:Changing query state from QUERYING to PENDING
net.c:673:Sending packet
0000 0f 01 00 21 00 00 00 00-21 00 00 00 14 00 73 65 |...!.... !.....se|
0010 74 20 74 65 78 74 73 69-7a 65 20 36 34 35 31 32 |t textsi ze 64512|
0020 20                     -                        | |

util.c:119:Changing query state from PENDING to READING
net.c:446:Received header
0000 04 01 00 11 00 00 00 00-                        |........|

net.c:542:Received packet
0000 fd 00 00 00 02 00 00 00-00                      |........ .|

token.c:526:processing result tokens.  marker is  fd(DONE)
token.c:2143:tds_process_end: more_results = 0
		was_cancelled = 0
		error = 0
		done_count_valid = 0
token.c:2159:tds_process_end() state set to TDS_IDLE
util.c:119:Changing query state from READING to IDLE
util.c:119:Changing query state from IDLE to PENDING
token.c:514:tds_process_tokens() state is COMPLETED
ct.c:592:leaving ct_connect() returning 1
ct.c:603:ct_cmd_alloc()
ct.c:214:setting command state from IDLE to IDLE
ct.c:622:ct_cmd_alloc() : allocating command list to head
ct.c:644:ct_command()
ct.c:214:setting command state from IDLE to IDLE
ct.c:214:setting command state from IDLE to IDLE
ct.c:214:setting command state from IDLE to READY
ct.c:808:ct_send() command_type = 148
util.c:119:Changing query state from IDLE to QUERYING
mem.c:519:tds_free_all_results()
util.c:119:Changing query state from QUERYING to PENDING
net.c:673:Sending packet
0000 0f 01 00 1c 00 00 00 00-21 00 00 00 0f 00 73 65 |........ !.....se|
0010 74 20 63 68 61 69 6e 65-64 20 6f 6e             |t chaine d on|

ct.c:214:setting command state from READY to SENT
ct.c:930:ct_send() succeeded
ct.c:1064:ct_results()
util.c:119:Changing query state from PENDING to READING
net.c:446:Received header
0000 04 01 00 11 00 00 00 00-                        |........|

net.c:542:Received packet
0000 fd 00 00 00 02 00 00 00-00                      |........ .|

token.c:526:processing result tokens.  marker is  fd(DONE)
token.c:2143:tds_process_end: more_results = 0
		was_cancelled = 0
		error = 0
		done_count_valid = 0
token.c:2159:tds_process_end() state set to TDS_IDLE
util.c:119:Changing query state from READING to IDLE
util.c:119:Changing query state from IDLE to PENDING
ct.c:1121:ct_results() process_result_tokens returned 1 (type 4052) 
ct.c:1226:ct_results() results state = 0
ct.c:1227:ct_results() command type  = 148
ct.c:1228:ct_results() dynamic cmd   = 0
ct.c:1064:ct_results()
ct.c:1064:ct_results()
token.c:514:tds_process_tokens() state is COMPLETED
ct.c:1121:ct_results() process_result_tokens returned 2 (type 4052) 
ct.c:214:setting command state from SENT to READY
ct.c:1765:ct_cmd_drop()
ct.c:1799:ct_cmd_drop() : command entry found in list
ct.c:1804:ct_cmd_drop() : relinking list
ct.c:1811:ct_cmd_drop() : relinked list
ct.c:603:ct_cmd_alloc()
ct.c:214:setting command state from IDLE to IDLE
ct.c:622:ct_cmd_alloc() : allocating command list to head
ct.c:644:ct_command()
ct.c:214:setting command state from IDLE to IDLE
ct.c:214:setting command state from IDLE to IDLE
ct.c:214:setting command state from IDLE to READY
ct.c:808:ct_send() command_type = 148
util.c:119:Changing query state from IDLE to QUERYING
mem.c:519:tds_free_all_results()
util.c:119:Changing query state from QUERYING to PENDING
net.c:673:Sending packet
0000 0f 01 00 2c 00 00 00 00-21 00 00 00 1f 00 73 65 |...,.... !.....se|
0010 6c 65 63 74 20 6e 61 6d-65 20 66 72 6f 6d 20 6f |lect nam e from o|
0020 6d 71 5f 73 65 71 75 65-6e 63 65 73             |mq_seque nces|

ct.c:214:setting command state from READY to SENT
ct.c:930:ct_send() succeeded
ct.c:1064:ct_results()
util.c:119:Changing query state from PENDING to READING
net.c:446:Received header
0000 04 01 01 24 00 00 00 00-                        |...$....|

net.c:542:Received packet
0000 ee 00 12 00 01 04 6e 61-6d 65 10 00 00 00 02 af |......na me......|
0010 00 00 00 50 00 ae 00 01-00 d1 00 00 00 0b 73 65 |...P.... ......se|
0020 71 5f 63 6c 61 73 73 65-73 d1 00 00 00 0d 73 65 |q_classe s.....se|
0030 71 5f 63 6f 6e 73 74 61-6e 74 73 d1 00 00 00 12 |q_consta nts.....|
0040 73 65 71 5f 65 72 72 6f-72 5f 69 6e 73 74 61 6e |seq_erro r_instan|
0050 63 65 d1 00 00 00 11 73-65 71 5f 66 6c 6f 77 5f |ce.....s eq_flow_|
0060 69 6e 73 74 61 6e 63 65-d1 00 00 00 09 73 65 71 |instance .....seq|
0070 5f 66 6c 6f 77 73 d1 00-00 00 15 73 65 71 5f 66 |_flows.. ...seq_f|
0080 75 6e 63 74 69 6f 6e 5f-69 6e 73 74 61 6e 63 65 |unction_ instance|
0090 d1 00 00 00 0a 73 65 71-5f 71 75 65 75 65 73 d1 |.....seq _queues.|
00a0 00 00 00 13 73 65 71 5f-73 65 72 76 69 63 65 5f |....seq_ service_|
00b0 6d 65 74 68 6f 64 73 d1-00 00 00 0c 73 65 71 5f |methods. ....seq_|
00c0 73 65 72 76 69 63 65 73-d1 00 00 00 0c 73 65 71 |services .....seq|
00d0 5f 73 65 73 73 69 6f 6e-73 d1 00 00 00 09 73 65 |_session s.....se|
00e0 71 5f 73 74 65 70 73 d1-00 00 00 15 73 65 71 5f |q_steps. ....seq_|
00f0 77 6f 72 6b 66 6c 6f 77-5f 69 6e 73 74 61 6e 63 |workflow _instanc|
0100 65 d1 00 00 00 0d 73 65-71 5f 77 6f 72 6b 66 6c |e.....se q_workfl|
0110 6f 77 73 fd 00 14 00 01-00 00 00 0d             |ows..... ....|

token.c:526:processing result tokens.  marker is  ee(RESULT)
mem.c:519:tds_free_all_results()
read.c:179:tds_get_string: reading 4 from wire to give 4 to client.
token.c:1526:processing result. type = 47(char), varint_size 2
token.c:1547:processing result. column_size 0
util.c:119:Changing query state from READING to PENDING
ct.c:1121:ct_results() process_result_tokens returned 1 (type 4049) 
util.c:119:Changing query state from PENDING to READING
token.c:526:processing result tokens.  marker is  50()
token.c:105:tds_process_default_tokens() marker is 50()
token.c:2252:tds_client_msg: #20020: "Unknown marker".  Connection state is now 3.  
util.c:119:Changing query state from READING to DEAD
token.c:237:Unknown marker: 80(50)!!
util.c:119:Changing query state from DEAD to PENDING
ct.c:1121:ct_results() process_result_tokens returned 0 (type 4055) 
ct.c:2049:ct_cancel()
ct.c:2197:CS_CANCEL_ALL with cmd
ct.c:2207:ct_cancel() command state SENT
ct.c:2208:ct_cancel() sending a cancel 
ct.c:214:setting command state from SENT to IDLE
ct.c:1765:ct_cmd_drop()
ct.c:1799:ct_cmd_drop() : command entry found in list
ct.c:1804:ct_cmd_drop() : relinking list
ct.c:1811:ct_cmd_drop() : relinked list
ct.c:1822:ct_close()
mem.c:519:tds_free_all_results()
ct.c:1834:ct_con_drop()
ct.c:241:ct_exit()