Skip to Content.
Sympa Menu

freetds - Re: [freetds] Force use of UTF-8 instead of ISO8859-1

freetds AT lists.ibiblio.org

Subject: FreeTDS Development Group

List archive

Chronological Thread  
  • From: "James K. Lowden" <jklowden AT freetds.org>
  • To: FreeTDS Development Group <freetds AT lists.ibiblio.org>
  • Subject: Re: [freetds] Force use of UTF-8 instead of ISO8859-1
  • Date: Sat, 6 Feb 2010 14:17:07 -0500

Frediano Ziglio wrote:
> >> correctly our imlpementation returns
> >> 10 and.... (rumble!) SQL_WCHAR... I forget this SMALL detail....
> >
> > What should it return if the encoding is UTF-8?  SQL_WCHAR seems
> > correct, unless it means UCS-2LE.  I think SQL_WCHAR means "Unicode".
> >
>
> Not only this, it also specify an encoding. For instance if you use
> unixODBC is UCS-2 or system wchar_t using iODBC.
...
> > SQL_DESC_OCTET_LENGTH is the length in bytes of the buffer needed hold
> > the data.  The driver knows how the buffer is encoded.  It should
> > return -- as dbcollen() does -- the maximum size that could be
> > required to hold any value that the column could hold.  For nchar(10)
> > in UTF-8, that's 40.
> >
>
> True and false I think... octet length is the buffer size using
> default client type for a given server type.... 20 for a
> nchar(10) if client have sizeof(SQLWCHAR) == 2 cause output is
> supposed to be a "wide" character set (UCS-2/UCS-4... or sometimes
> UTF-16... never UTF-8). Client character set affect how we translate
> to multi-byte character set but not how library encodes wide character
> set.

Wouldn't it be 40 for USC-4?

> Client character set affect how we translate
> to multi-byte character set but not how library encodes wide character
> set.

No, "how we translate" === "how library encodes". Character data have
only two forms: server encoding and client encoding. The library has no
encoding. All data that the client exchanges with the library always uses
one encoding: the client's.

The DM does not know the encoding FreeTDS is using, but the application
does. The application is asking SQL_DESC_OCTET_LENGTH for the size of the
buffer it needs to allocate. FreeTDS has already converted UCS-2 to
utf-8. It should return 40, else the application is likely to allocate
too small a buffer.

Should SQLDescribeCol return SQL_WCHAR for utf-8 data? I think Yes. If
it returned a new constant e.g. SQL_U8CHAR, most applcations wouldn't know
how to deal with it. If it returns SQL_CHAR, the application might assume
1 byte/char. SQL_WCHAR is the best option. If the application assumes
SQL_WCHAR means UCS-2, then the client should use UCS-2 encoding instead.


I think we basically agree here. You see the changes to bsqlodbc, which
now works. I don't see another way for the application to determine the
buffer size.

Regards,

--jkl




Archive powered by MHonArc 2.6.24.

Top of Page