[freetds] Status of UTF-8 support in 0.83 dev
Sebastien FLAESCH
sf at 4js.com
Thu Aug 14 10:59:06 EDT 2008
Hello James,
James K. Lowden wrote:
> Hi Sebastien,
>
>> When I print the client and the server charset in tds_convert_string()
>> called from tds_put_data(), I see:
>>
>> char_conv->client_charset.name = "UTF-8"
>> char_conv->server_charset.name = "CP1252"
>>
>> I expected UCS-2 for the server charset...????
>>
>> Hence it's normal that the conversion of non-CP1252 compatible chars
>> like é´» result in error...
>
> That's OK. I think you misunderstand the meaning of
> client2server_chardata: "server_chardata" is the server's single-byte
> encoding, used for char/varchar/text columns. The server announces its
> encoding as a message immediately after logon.
>
> (Off the top of my head, I can't think why we care about the server's
> single-byte encoding. Maybe for TDS 4.2. TDS 7+ uses UCS-2; we never
> send the server anything else IIRC.)
>
> As I said yesterday, CP1252 could explain your 2-byte/3-byte observation.
>
>
> First assertion: in unprefixed strings, *only* characters represented by
> CP1252 will be stored correctly on your server. All others will be
> converted to CP1252 first by the server, with varying results. Pick *any*
> non-CP1252 character and see what happens.
>
> Second assertion: all prefixed strings embedded in SQL text will be
> handled correctly on both ends.
>
> Final assertion: parameterized query buffers are not correctly converted
> when sent to the server.
>
> That's where I think we are. Agreed?
Yes, I think so...
And as I wrote in a previous mail, we should clarify how UTF-8 buffers of
parameterized queries have to be bound...
Should ParameterType be SQL_[VAR]CHAR or SQL_W[VAR]CHAR ?
Looking at the code, client2ucs2 TDSICONV is only used if SQL_W[VAR]CHAR
is used, but that does not fit the IBM DB2 CLI way and confuses me since
SQL_W[VAR]CHAR is for me the MS "Wide Char Unicode", not UTF-8...
The fact that I see unexpected chars at the end of the UTF-8 string in
tds_put_data (from curcol->column_data), is probably because the string
is not null terminated at the same place as my original string (just
guessing).
UTF-8 = [abcéô鴻01000] ( size= 10 )
...
Thanks for you answer, I really appreciate your help.
>
> Regards,
>
> --jkl
> _______________________________________________
> FreeTDS mailing list
> FreeTDS at lists.ibiblio.org
> http://lists.ibiblio.org/mailman/listinfo/freetds
>
More information about the FreeTDS
mailing list