FreeTDS Development Group

[Sebastien FLAESCH <sf AT 4js.com>]

Frediano,

I am not sure to get what you mean by:

> I think that column size == count in bytes is required cause:

Do you mean the ColumnSize parameter of SQLBindColumn() should be provided as 
a number of bytes instead of characters?
I am not sure this is a good idea, as it is used to define the SQL type 
(NVARCHAR(ColumnSize)) ...
I mean, SQL Native Client expect a number of characters, not bytes.

FYI: Not sure this helps:

If you want to known the maximum number of bytes a multi-byte character can 
use in the current locale,
you have the standard MB_CUR_MAX macro... (man MB_CUR_MAX).

Thanks for your great help.
Seb

ZIGLIO, Frediano, VF-IT wrote:
> > Frediano,
> >
> > Sorry my YES was too fast: I should have checked with a 
> > simple ODBC program.
> >
> > Even inserting does not fully work for now:
> >
> > When binding with a small ColumnSize (like 2), I get invalid 
> > data in the
> > SQL Server Profiler trace (only pieces of the value).
> >
> > Looks like the conversion from UTF-8 to UCS-2 relies on the ColumnSize
> > and does not work properly when that size is small...
> > (Maybe the conversion buffer used is allocated according to 
> > ColumnSize,
> > as a number of bytes?)
>
> Bingo, yes, code in src/odbc/sql2tds.c.
> Oh... I committed patch I sent you yesterday.
>
> > See attached example: Using ColumnSize=2, it does not insert all data,
> > but when using ColumnSize>=6 it works...
>
> I updated utf8 test, see 
> http://freetds.cvs.sourceforge.net/freetds/freetds/src/odbc/unittests/utf8.c?r1=1.3&r2=1.4
> currently it doesn't work, it give 
>
> $ ./utf8
> IF OBJECT_ID(N'mytab王鴻') IS NOT NULL DROP TABLE mytab王鴻
> CREATE TABLE mytab王鴻 (k int, c NCHAR(10), vc NVARCHAR(10))
> INSERT INTO mytab王鴻 VALUES (1,N'aaa',N'aaa')
> INSERT INTO mytab王鴻 VALUES (2,N'abcéáô',N'abcéîô')
> INSERT INTO mytab王鴻 VALUES (3,N'abc王鴻',N'abc王鴻傑王鴻傑')
> DELETE FROM mytab王鴻
> insert #1
> insert #2
> insert #3
> DELETE FROM mytab王鴻
> insert #1
> insert #2
> utf8.c:106 SQLExecute
> SQL error 42000 -- [FreeTDS][SQL Server]Character set conversion is not 
> available between client character set '%.*s' and server character set '%.*s'
>
>
> mmm... error is a bit wrong...
>
> > Binding first value (4 bytes) with ColumnSize=4, it works, and second
> > value (6 bytes) it works with ColumnSize=6...
> >
> > Since ColumnSize defines a number of CHARACTERS, something is 
> > wrong here.
> >
> > Seb
>
> I got it working replacing (sql2tds.c)
>
>     } else {
>         if (is_unicode_type(dest_type))
>             curcol->on_server.column_size = curcol->column_size * 2;
>         else
>             curcol->on_server.column_size = curcol->column_size;
>     }
>
> with
>
>     } else {
>         if (is_unicode_type(dest_type)) {
>             curcol->on_server.column_size = curcol->column_size * 2;
>             curcol->column_size *= 4;
>         } else
>             curcol->on_server.column_size = curcol->column_size;
>     }
>
>
> It merely allocate correct buffer... I think in this case it would be better 
> to either
> - avoid buffer allocation, fill column_data if possible... I think the main problem here is 
> - use server encoding for buffer, I think this is related to bcp todo ("Native bcp has no iconv support; character bcp files are assumed be encoded with the client's charset.  More flexibility one both sides would be good.").
> I think that column size == count in bytes is required cause:
> - its possible to compute characters number from byte count and buffer but 
> it's not easy from characters count and buffer to byte count (it require 
> terminated strings)
> - currently on_server.column_size is always a byte count
> - memcpy/memmove it's easier if you have byte count
> - client libraries provide byte count for strings and buffers
>
> There are some cases where however having different encoding in libTDS buffer 
> is useful
> - having different client encoding
> - avoiding converting characters twice (for instance if client use multibyte 
> and wide at the same time as odbc does)
>
> freddy77
>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> FreeTDS mailing list
> FreeTDS AT lists.ibiblio.org
> http://lists.ibiblio.org/mailman/listinfo/freetds