FreeTDS Development Group

[Andrew Victor <avictor.za AT gmail.com>]

hi Frediano,

> I saw patch about CTLib and nvarchar. I forgot to comment and review
> this patch but looking at unitests results it seems that this patch
> fix nothing and break too much.

The patch actually allows you to store UTF8-encoded data in an
NVARCHAR field using CTLib.


> The patch should try to fix ct_param
> with wide charset source

No, the issue is at a lower level than ct_param().
The source data is already 'wide" UTF-8 ("client charset = UTF-8").
FreeTDS must just remember the data is "wide" so it calls iconv() to
convert the data to UCS2, and not CP1252 when it submits it to the
server.



Here follows the original bug report email with trace I submitted to
the mailing list ...


What is the correct CTLib bind type to use when calling a
stored-procedure having a NVARCHAR parameter?

I set the CS_DATAFMT.datatype field to CS_UNICHAR_TYPE in the call to
ct_param().
The data being bound is a UTF-8 string (and can be converted to UCS-2
using the command-line iconv utility).
My freetds.conf contains "client charset = UTF-8"

When calling ct_send(), the following client callback is generated:

[2008/11/19 16:32:59.069] ERROR : Client message:
[2008/11/19 16:32:59.070] ERROR : Message number: 98, Severity = 9,
Layer = 0, Origin = 0
[2008/11/19 16:32:59.070] ERROR : Message: Error converting characters
into server's character set. Some character(s) could not be converted

The dump file contains:
16:32:59.069348 4673 (ct.c:215):setting command state from IDLE to READY
16:32:59.069377 4673 (ct.c:3364):ct_param()
16:32:59.069391 4673 (ct.c:3365):ct_param() data addr = 0x804d6b7 data
length = 12
16:32:59.069405 4673 (ct.c:4341): _ct_fill_param() status = 256
16:32:59.069418 4673 (ct.c:2009):_ct_get_server_type(0)
16:32:59.069430 4673 (ct.c:3396): ct_param() added rpc parameter @Id
16:32:59.069576 4673 (ct.c:3364):ct_param()
16:32:59.069590 4673 (ct.c:3365):ct_param() data addr = 0x891e7d8 data
length = 4
16:32:59.069604 4673 (ct.c:4341): _ct_fill_param() status = 256
16:32:59.069618 4673 (ct.c:2009):_ct_get_server_type(25)
16:32:59.069631 4673 (ct.c:3396): ct_param() added rpc parameter @eMail
16:32:59.069645 4673 (ct.c:844):ct_send() command_type = 149
16:32:59.069664 4673 (ct.c:4256):paraminfoalloc: status = 256, maxlen 0
16:32:59.069678 4673 (ct.c:4257):paraminfoalloc: name = @Id, varint
size 2 column_type 47 size 12, 12 column_cur_size 12 column_output = 0
16:32:59.069692 4673 (ct.c:4085):paramrowalloc, size = 12, data =
0x891eca0, row_size = 0
16:32:59.069707 4673 (ct.c:4256):paraminfoalloc: status = 256, maxlen 0
16:32:59.069719 4673 (ct.c:4257):paraminfoalloc: name = @eMail, varint
size 2 column_type 39 size 4, 4 column_cur_size 4 column_output = 0
16:32:59.069733 4673 (ct.c:4085):paramrowalloc, size = 4, data =
0x89416b8, row_size = 0
16:32:59.069749 4673 (mem.c:563):tds_free_all_results()
16:32:59.069762 4673 (util.c:162):Changed query state from IDLE to QUERYING
16:32:59.069777 4673 (iconv.c:634):      XX - tds_iconv() charset
to_server: UTF-8 -> UCS-2LE
16:32:59.069791 4673 (query.c:1292):tds_put_data_info putting param_name
16:32:59.069804 4673 (iconv.c:634):     XX - tds_iconv() charset
to_server: UTF-8 -> UCS-2LE
16:32:59.069818 4673 (query.c:1326):tds_put_data_info putting status
16:32:59.069831 4673 (query.c:1420):tds_put_data: colsize = 12
16:32:59.069843 4673 (query.c:1449):tds_put_data: not null param varint_size 
= 2
16:32:59.069856 4673 (iconv.c:634):     XX - tds_iconv() charset
to_server: UTF-8 -> CP1252
16:32:59.069870 4673 (query.c:1292):tds_put_data_info putting param_name
16:32:59.069883 4673 (iconv.c:634):     XX - tds_iconv() charset
to_server: UTF-8 -> UCS-2LE
16:32:59.069904 4673 (query.c:1326):tds_put_data_info putting status
16:32:59.069917 4673 (query.c:1420):tds_put_data: colsize = 4
16:32:59.069930 4673 (query.c:1449):tds_put_data: not null param varint_size 
= 2
16:32:59.069942 4673 (iconv.c:634):     XX - tds_iconv() charset
to_server: UTF-8 -> CP1252
16:32:59.069956 4673 (util.c:334):tdserror(0x891e878, 0x891ea00, 2402, 0)
16:32:59.070035 4673 (util.c:368):tdserror: client library returned
TDS_INT_CANCEL(2)
16:32:59.070049 4673 (util.c:389):tdserror: returning TDS_INT_CANCEL(2)
16:32:59.070063 4673 (util.c:162):Changed query state from QUERYING to PENDING
16:32:59.070076 4673 (net.c:849):Sending packet

The first parameter (@Id) is an ASCII string (CS_CHAR_TYPE) and the
2nd parameter (@eMail) is a UTF8 (CS_UNICHAR_TYPE) string.

The lines of debugging marked with "XX" I added.
So it looks like the CS_UNICHAR_TYPE parameter is being converted to
CP1252, which is the server's single-byte charset.

Am I using the wrong types, or is this a bug in FreeTDS?


Regards,
  Andrew Victor