Skip to Content.
Sympa Menu

freetds - Re: [freetds] Status of UTF-8 support in 0.83 dev

freetds AT lists.ibiblio.org

Subject: FreeTDS Development Group

List archive

Chronological Thread  
  • From: Sebastien FLAESCH <sf AT 4js.com>
  • To: FreeTDS Development Group <freetds AT lists.ibiblio.org>
  • Subject: Re: [freetds] Status of UTF-8 support in 0.83 dev
  • Date: Thu, 14 Aug 2008 16:59:06 +0200

Hello James,

James K. Lowden wrote:
Hi Sebastien,
When I print the client and the server charset in tds_convert_string()
called from tds_put_data(), I see:

char_conv->client_charset.name = "UTF-8"
char_conv->server_charset.name = "CP1252"

I expected UCS-2 for the server charset...????

Hence it's normal that the conversion of non-CP1252 compatible chars
like é´» result in error...

That's OK. I think you misunderstand the meaning of
client2server_chardata: "server_chardata" is the server's single-byte
encoding, used for char/varchar/text columns. The server announces its
encoding as a message immediately after logon.
(Off the top of my head, I can't think why we care about the server's
single-byte encoding. Maybe for TDS 4.2. TDS 7+ uses UCS-2; we never
send the server anything else IIRC.)
As I said yesterday, CP1252 could explain your 2-byte/3-byte observation.

First assertion: in unprefixed strings, *only* characters represented by
CP1252 will be stored correctly on your server. All others will be
converted to CP1252 first by the server, with varying results. Pick *any*
non-CP1252 character and see what happens.
Second assertion: all prefixed strings embedded in SQL text will be
handled correctly on both ends.
Final assertion: parameterized query buffers are not correctly converted
when sent to the server.
That's where I think we are. Agreed?

Yes, I think so...

And as I wrote in a previous mail, we should clarify how UTF-8 buffers of
parameterized queries have to be bound...

Should ParameterType be SQL_[VAR]CHAR or SQL_W[VAR]CHAR ?

Looking at the code, client2ucs2 TDSICONV is only used if SQL_W[VAR]CHAR
is used, but that does not fit the IBM DB2 CLI way and confuses me since
SQL_W[VAR]CHAR is for me the MS "Wide Char Unicode", not UTF-8...

The fact that I see unexpected chars at the end of the UTF-8 string in
tds_put_data (from curcol->column_data), is probably because the string
is not null terminated at the same place as my original string (just
guessing).

UTF-8 = [abcéô鴻01000] ( size= 10 )

...

Thanks for you answer, I really appreciate your help.




Regards,
--jkl
_______________________________________________
FreeTDS mailing list
FreeTDS AT lists.ibiblio.org
http://lists.ibiblio.org/mailman/listinfo/freetds






Archive powered by MHonArc 2.6.24.

Top of Page