[freetds] Force use of UTF-8 instead of ISO8859-1

Frediano Ziglio freddy77 at gmail.com
Fri Feb 5 10:38:31 EST 2010

2010/2/5  <jklowden at schemamania.org>:
> On Fri, Feb 05, 2010 at 03:57:03PM +0100, Frediano Ziglio wrote:
>> 2010/2/5 James K. Lowden <jklowden at freetds.org>:
>> > $ printf "select * from test_utf8\ngo\n" | bsqlodbc -S$S -U$U -P$(password
>> > $S $U)
>> >        id  name
>> > ----------  ----------
>> > "[FreeTDS][SQL Server]Data truncated"
>> >         1  hall
>> > "[FreeTDS][SQL Server]Data truncated"
>> >         2  adj
>> > "[FreeTDS][SQL Server]Data truncated"
>> >         3  rhus
>> Using nchar(10) "foo" get stored as "foo       " ("foo" followed by 7
>> spaces). Using utf-8 encoding a 5 character get stored in 10 nchar but
>> if a character is not ascii get encoded in more than 10 characters.
>> SQLDescribeCol return 10 characters and you provided 10 character
>> buffer which is insufficient to store original data. Now... On
>> SQLDescribeCol
>>  ColumnSizePtr
>>     [Output] Pointer to a buffer in which to return the size (in
>> characters) of the column on the data source.
>> so is fine to return 10 or we should return that maximum buffer space
>> (40 in this case) ?? Or perhaps we should return 0 (undeterminated) ?
> SQLDescribeCol is correct.  It returns the logical size, the size as reported by the server.  The column size is measured in characters, by definition; the server's storage requirements are not the client's concern.
> To determine the appropriate buffer size, bsqlodbc should use SQLGetDescRec and SQL_DESC_OCTET_LENGTH.
> http://msdn.microsoft.com/en-us/library/ms712499%28VS.85%29.aspx
> http://msdn.microsoft.com/en-us/library/ms713979%28VS.85%29.aspx
>        "Similarly, the values for transfer octet length do not come from SQL_DESC_LENGTH. They come from the SQL_DESC_OCTET_LENGTH of a field of a descriptor for all character and binary types."
> The things you learn....

Yes... I know octet length is that bsqlodbc use SQLDescribeCol to
compute buffer length. However... correctly our imlpementation returns
10 and.... (rumble!) SQL_WCHAR... I forget this SMALL detail.... I had
to write some code on describecol test... so I think that our driver
is 100% correct... it detects truncation! At this point I would ask me
if SQL_DESC_OCTET_LENGTH is correct... but perhaps we can't even use
this value to compute character buffer... this cause
SQL_DESC_OCTET_LENGTH describe the binary length using SQL_WCHAR not
SQL_CHAR... still conversion can lead to truncation... or not
(currently I have no reply) ???


More information about the FreeTDS mailing list