Skip to Content.
Sympa Menu

freetds - Re: [freetds] Force use of UTF-8 instead of ISO8859-1

freetds AT lists.ibiblio.org

Subject: FreeTDS Development Group

List archive

Chronological Thread  
  • From: Frediano Ziglio <freddy77 AT gmail.com>
  • To: FreeTDS Development Group <freetds AT lists.ibiblio.org>
  • Subject: Re: [freetds] Force use of UTF-8 instead of ISO8859-1
  • Date: Fri, 5 Feb 2010 16:38:31 +0100

2010/2/5 <jklowden AT schemamania.org>:
> On Fri, Feb 05, 2010 at 03:57:03PM +0100, Frediano Ziglio wrote:
>> 2010/2/5 James K. Lowden <jklowden AT freetds.org>:
>> > $ printf "select * from test_utf8\ngo\n" | bsqlodbc -S$S -U$U
>> > -P$(password
>> > $S $U)
>> >        id  name
>> > ----------  ----------
>> > "[FreeTDS][SQL Server]Data truncated"
>> >         1  hall
>> > "[FreeTDS][SQL Server]Data truncated"
>> >         2  adj
>> > "[FreeTDS][SQL Server]Data truncated"
>> >         3  rhus
>>
>> Using nchar(10) "foo" get stored as "foo       " ("foo" followed by 7
>> spaces). Using utf-8 encoding a 5 character get stored in 10 nchar but
>> if a character is not ascii get encoded in more than 10 characters.
>> SQLDescribeCol return 10 characters and you provided 10 character
>> buffer which is insufficient to store original data. Now... On
>> SQLDescribeCol
>>
>>  ColumnSizePtr
>>
>>     [Output] Pointer to a buffer in which to return the size (in
>> characters) of the column on the data source.
>>
>> so is fine to return 10 or we should return that maximum buffer space
>> (40 in this case) ?? Or perhaps we should return 0 (undeterminated) ?
>
> SQLDescribeCol is correct.  It returns the logical size, the size as
> reported by the server.  The column size is measured in characters, by
> definition; the server's storage requirements are not the client's concern.
>
> To determine the appropriate buffer size, bsqlodbc should use SQLGetDescRec
> and SQL_DESC_OCTET_LENGTH.
>
> http://msdn.microsoft.com/en-us/library/ms712499%28VS.85%29.aspx
> http://msdn.microsoft.com/en-us/library/ms713979%28VS.85%29.aspx
>
>        "Similarly, the values for transfer octet length do not come from
> SQL_DESC_LENGTH. They come from the SQL_DESC_OCTET_LENGTH of a field of a
> descriptor for all character and binary types."
>
> The things you learn....
>

Yes... I know octet length is that bsqlodbc use SQLDescribeCol to
compute buffer length. However... correctly our imlpementation returns
10 and.... (rumble!) SQL_WCHAR... I forget this SMALL detail.... I had
to write some code on describecol test... so I think that our driver
is 100% correct... it detects truncation! At this point I would ask me
if SQL_DESC_OCTET_LENGTH is correct... but perhaps we can't even use
this value to compute character buffer... this cause
SQL_DESC_OCTET_LENGTH describe the binary length using SQL_WCHAR not
SQL_CHAR... still conversion can lead to truncation... or not
(currently I have no reply) ???

freddy77




Archive powered by MHonArc 2.6.24.

Top of Page