[freetds] Status of UTF-8 support in 0.83 dev

Sebastien FLAESCH sf at 4js.com
Wed Aug 13 05:58:31 EDT 2008


Thank you James for looking at this!

UTF-8 support is critical for one of of customers in Asia...

I have identified a first issue: I must of course use N'xxx' string
constants when inserting UNICODE/UTF-8 data in the database.
In previous sample program, the string constants where identified as
non-Unicode strings in the current database locale. Sorry...

I have added a SELECT to fetch inserted rows, and it seems to be fine
so far. I have checked the values in the database with SQL Server's
query tool and the characters are correct.

Still the insert statement using UTF-8 data fails with:

Error -1 at: SQLExecute 1.2 StmtH
Diagnostic info:
   SQL State: 42000
   SQL code : 8005
   Message  : [FreeTDS][SQL Server]The incoming tabular data stream
              (TDS) remote procedure call (RPC) protocol stream is
              incorrect. Parameter 4: The parameter name is invalid.


Note that the first insert with latin characters "abcéÓÎ" / "abcÛÊé"
(2 bytes long) works, it's the second insert using Asian characters
(3 bytes long)...


FYI I have rebuild freetds with iconv 1.12 now:

[root at fox ~]# ldd -r /opt3/dbs/tds/0.83/lib/libtdsodbc.so
         libiconv.so.2 => /usr/local/lib/libiconv.so.2 (0x00111000)
         librt.so.1 => /lib/tls/librt.so.1 (0x00a7b000)
         libpthread.so.0 => /lib/tls/libpthread.so.0 (0x00957000)
         libc.so.6 => /lib/tls/libc.so.6 (0x001ed000)
         /lib/ld-linux.so.2 (0x00739000)

Thanks,
Seb


James K. Lowden wrote:
> Sebastien FLAESCH wrote:
>> Log shows now:
>>
>> log.c:190:Starting log file for FreeTDS 0.83.dev.20080812
>>          on 2008-08-12 18:11:48 with debug flags 0x4fff.
>> iconv.c:197:names for ISO-8859-1: ISO-8859-1
>> iconv.c:197:names for UTF-8: UTF-8
>> iconv.c:197:names for UCS-2LE: UCS-2LE
>> iconv.c:197:names for UCS-2BE: UCS-2BE
>> iconv.c:363:iconv to convert client-side data to the "UTF-8" character
>> set iconv.c:516:tds_iconv_info_init: converting "UTF-8"->"UCS-2LE"
>> iconv.c:516:tds_iconv_info_init: converting "ISO-8859-1"->"UCS-2LE"
> 
> Hi Sebastien, 
> 
> I modified the log messages from iconv.c to make them clearer.  HTH. 
> 
>> But still, I get:
>>
>>
>> Error -1 at: SQLExecute 1.2 StmtH
>> Diagnostic info:
>>    SQL State: 42000
>>    SQL code : 8005
>>    Message  : [FreeTDS][SQL Server]The incoming tabular data stream
>>    (TDS) remote procedure call (RPC) protocol stream is incorrect.
>>    Parameter 4: The 
>> parameter name is invalid.
> 
> This happens because of these errors in the log:
> 
>> query.c:1326:tds_put_data_info putting status 
>> query.c:1420:tds_put_data: colsize = 9
>> query.c:1449:tds_put_data: not null param varint_size = 2
>> util.c:334:tdserror(0x98cb050, 0x98cc3d8, 2402, 0)
>> odbc.c:2106:msgno 2402 20003
>> util.c:368:tdserror: client library returned TDS_INT_CANCEL(2)
>> util.c:389:tdserror: returning TDS_INT_CANCEL(2)
>> query.c:1326:tds_put_data_info putting status 
>> query.c:1420:tds_put_data: colsize = 9
>> query.c:1449:tds_put_data: not null param varint_size = 2
>> util.c:334:tdserror(0x98cb050, 0x98cc3d8, 2402, 0)
>> odbc.c:2106:msgno 2402 20003
>> util.c:368:tdserror: client library returned TDS_INT_CANCEL(2)
>> util.c:389:tdserror: returning TDS_INT_CANCEL(2)
>> util.c:162:Changed query state from QUERYING to PENDING
>> net.c:768:Sending packet
> 
> The resulting sp_execute call is incorrectly formed, and the server
> complains.  
> 
> 2402 is TDSEICONVO and indicates an invalid multibyte input sequence was
> encountered while sending data to the server.  
> 
> I'm too tired to debug it further.  I would guess the parameter data are
> assumed to be in ISO-8859-1, not in the client's encoding.  
> 
> Regards, 
> 
> --jkl
> _______________________________________________
> FreeTDS mailing list
> FreeTDS at lists.ibiblio.org
> http://lists.ibiblio.org/mailman/listinfo/freetds
> 

-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: odbctest11.c
Url: http://lists.ibiblio.org/pipermail/freetds/attachments/20080813/53142a9e/attachment.c 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: tds.log
Type: text/x-log
Size: 47362 bytes
Desc: not available
Url : http://lists.ibiblio.org/pipermail/freetds/attachments/20080813/53142a9e/attachment.bin 


More information about the FreeTDS mailing list