freetds AT lists.ibiblio.org
Subject: FreeTDS Development Group
List archive
Re: [freetds] Force use of UTF-8 instead of ISO8859-1
- From: Frediano Ziglio <freddy77 AT gmail.com>
- To: FreeTDS Development Group <freetds AT lists.ibiblio.org>
- Subject: Re: [freetds] Force use of UTF-8 instead of ISO8859-1
- Date: Fri, 5 Feb 2010 16:38:31 +0100
2010/2/5 <jklowden AT schemamania.org>:
> On Fri, Feb 05, 2010 at 03:57:03PM +0100, Frediano Ziglio wrote:
>> 2010/2/5 James K. Lowden <jklowden AT freetds.org>:
>> > $ printf "select * from test_utf8\ngo\n" | bsqlodbc -S$S -U$U
>> > -P$(password
>> > $S $U)
>> > id name
>> > ---------- ----------
>> > "[FreeTDS][SQL Server]Data truncated"
>> > 1 hall
>> > "[FreeTDS][SQL Server]Data truncated"
>> > 2 adj
>> > "[FreeTDS][SQL Server]Data truncated"
>> > 3 rhus
>>
>> Using nchar(10) "foo" get stored as "foo " ("foo" followed by 7
>> spaces). Using utf-8 encoding a 5 character get stored in 10 nchar but
>> if a character is not ascii get encoded in more than 10 characters.
>> SQLDescribeCol return 10 characters and you provided 10 character
>> buffer which is insufficient to store original data. Now... On
>> SQLDescribeCol
>>
>> ColumnSizePtr
>>
>> [Output] Pointer to a buffer in which to return the size (in
>> characters) of the column on the data source.
>>
>> so is fine to return 10 or we should return that maximum buffer space
>> (40 in this case) ?? Or perhaps we should return 0 (undeterminated) ?
>
> SQLDescribeCol is correct. It returns the logical size, the size as
> reported by the server. The column size is measured in characters, by
> definition; the server's storage requirements are not the client's concern.
>
> To determine the appropriate buffer size, bsqlodbc should use SQLGetDescRec
> and SQL_DESC_OCTET_LENGTH.
>
> http://msdn.microsoft.com/en-us/library/ms712499%28VS.85%29.aspx
> http://msdn.microsoft.com/en-us/library/ms713979%28VS.85%29.aspx
>
> "Similarly, the values for transfer octet length do not come from
> SQL_DESC_LENGTH. They come from the SQL_DESC_OCTET_LENGTH of a field of a
> descriptor for all character and binary types."
>
> The things you learn....
>
Yes... I know octet length is that bsqlodbc use SQLDescribeCol to
compute buffer length. However... correctly our imlpementation returns
10 and.... (rumble!) SQL_WCHAR... I forget this SMALL detail.... I had
to write some code on describecol test... so I think that our driver
is 100% correct... it detects truncation! At this point I would ask me
if SQL_DESC_OCTET_LENGTH is correct... but perhaps we can't even use
this value to compute character buffer... this cause
SQL_DESC_OCTET_LENGTH describe the binary length using SQL_WCHAR not
SQL_CHAR... still conversion can lead to truncation... or not
(currently I have no reply) ???
freddy77
-
Re: [freetds] Force use of UTF-8 instead of ISO8859-1,
Michael Mata, 02/02/2010
-
Re: [freetds] Force use of UTF-8 instead of ISO8859-1,
James K. Lowden, 02/04/2010
-
Re: [freetds] Force use of UTF-8 instead of ISO8859-1,
Frediano Ziglio, 02/05/2010
-
Re: [freetds] Force use of UTF-8 instead of ISO8859-1,
jklowden, 02/05/2010
-
Re: [freetds] Force use of UTF-8 instead of ISO8859-1,
Frediano Ziglio, 02/05/2010
-
Re: [freetds] Force use of UTF-8 instead of ISO8859-1,
jklowden, 02/05/2010
-
Re: [freetds] Force use of UTF-8 instead of ISO8859-1,
Frediano Ziglio, 02/06/2010
- Re: [freetds] Force use of UTF-8 instead of ISO8859-1, James K. Lowden, 02/06/2010
- Re: [freetds] Force use of UTF-8 instead of ISO8859-1, Frediano Ziglio, 02/06/2010
-
Re: [freetds] Force use of UTF-8 instead of ISO8859-1,
Frediano Ziglio, 02/06/2010
-
Re: [freetds] Force use of UTF-8 instead of ISO8859-1,
jklowden, 02/05/2010
-
Re: [freetds] Force use of UTF-8 instead of ISO8859-1,
Frediano Ziglio, 02/05/2010
-
Re: [freetds] Force use of UTF-8 instead of ISO8859-1,
jklowden, 02/05/2010
-
Re: [freetds] Force use of UTF-8 instead of ISO8859-1,
Frediano Ziglio, 02/05/2010
-
Re: [freetds] Force use of UTF-8 instead of ISO8859-1,
James K. Lowden, 02/04/2010
- Re: [freetds] Force use of UTF-8 instead of ISO8859-1, jklowden, 02/05/2010
Archive powered by MHonArc 2.6.24.