FreeTDS Development Group

[Steve Langasek <vorlon AT netexpress.net>]

On Sun, Sep 22, 2002 at 11:06:59AM -0500, Steve Langasek wrote:

> There's a pretty visible bug on SELECT, though -- for whatever reason,
> all of the multibyte characters are being rendered to the terminal as
> either � (the UTF-8 marker for an invalid multibyte sequence) or as ?
> (an unrepresentable character).  The ones represented with a '?' in sqsh
> are also seen that way when viewing the data on the server, so that may
> be a problem with the initial UTF-8 -> UCS-2 conversion; I'll have to
> look into it more to see.  Doesn't bother me too terribly much, I'm
> tickled that everything works as well as it does at this point.

Ok, I understand now where these '?' substitutions are coming from. The
UTF8->UCS2 conversion is done correctly by libtds, and we are sending
valid UCS2 to the server, but the charset information from the locale
isn't sent to the server as part of a TDS7 login packet -- only the
language setting is.  As a result, the *server* is choosing a charset for
us, apparently based on the language setting (in this case, iso_1), and
all input is mapped to that character set -- so any characters outside
the available ISO-8859-1 range are munged!

Below is the snippet of the 0xe3 packet from the SQL server where the
charset is pronounced.

Is there any way to specify the charset as part of the TDS7 login
sequence?

Steve Langasek
postmodern programmer

0040  00 01 0a 6c 00 6f 00 63  00 61 00 6c 00 65 00 74   ...l.o.c .a.l.e.t
0050  00 65 00 73 00 74 00 06  6d 00 61 00 73 00 74 00   .e.s.t.. m.a.s.t.
0060  65 00 72 00 ab 64 00 45  16 00 00 02 00 29 00 43   e.r..d.E .....).C
0070  00 68 00 61 00 6e 00 67  00 65 00 64 00 20 00 64   .h.a.n.g .e.d. .d
0080  00 61 00 74 00 61 00 62  00 61 00 73 00 65 00 20   .a.t.a.b .a.s.e. 
0090  00 63 00 6f 00 6e 00 74  00 65 00 78 00 74 00 20   .c.o.n.t .e.x.t. 
00a0  00 74 00 6f 00 20 00 27  00 6c 00 6f 00 63 00 61   .t.o. .' .l.o.c.a
00b0  00 6c 00 65 00 74 00 65  00 73 00 74 00 27 00 2e   .l.e.t.e .s.t.'..
00c0  00 03 53 00 51 00 4c 00  00 00 00 e3 17 00 02 0a   ..S.Q.L. ........
00d0  75 00 73 00 5f 00 65 00  6e 00 67 00 6c 00 69 00   u.s._.e. n.g.l.i.
00e0  73 00 68 00 00 ab 60 00  47 16 00 00 01 00 27 00   s.h...`. G.....'.
00f0  43 00 68 00 61 00 6e 00  67 00 65 00 64 00 20 00   C.h.a.n. g.e.d. .
0100  6c 00 61 00 6e 00 67 00  75 00 61 00 67 00 65 00   l.a.n.g. u.a.g.e.
0110  20 00 73 00 65 00 74 00  74 00 69 00 6e 00 67 00    .s.e.t. t.i.n.g.
0120  20 00 74 00 6f 00 20 00  75 00 73 00 5f 00 65 00    .t.o. . u.s._.e.
0130  6e 00 67 00 6c 00 69 00  73 00 68 00 2e 00 03 53   n.g.l.i. s.h....S
0140  00 51 00 4c 00 00 00 00  e3 0f 00 03 05 69 00 73   .Q.L.... .....i.s
0150  00 6f 00 5f 00 31 00 01  00 00 e3 0b 00 05 04 31   .o._.1.. .......1

Attachment: pgpdT7lzPXKoD.pgp
Description: PGP signature