FreeTDS Development Group

["Lowden, James K" <LowdenJK AT bernstein.com>]

> From: James K. Lowden [mailto:jklowden AT schemamania.org]
> Sent: March 31, 2003 7:28 AM
> 
> I think we should isolate all client-server charset conversions (and
> endian handling, while we're at it) in a single module.  The set of
> functions in that module should be the only ones that deal with the
> server's character set and with conversion to/from the client charset.
> Only they should deal with the char/byte ratios.  They should also be
> responsible for fixing up the metadata, so that the rest of the code
> doesn't worry about dividing by two or even about what the server's
> character set happens to be.  

As sometimes happens when I have an idea about how libtds "should" be, I
discover further into the analysis it's already done, or mostly done.  

Unicode conversion is done inside tds_[get|put]_string.  tds_get_string() is
called from login.c and by several functions (most static) in token.c;
tds_put_string() likewise from login.c, mostly from query.c, and once from
token.c.  There are also calls from bcp.c, but much of that module should
arguably be in libtds.  

Functions such as tds7_get_data_info() that set the column width should
fixup the width according to the client charset.  We need to keep
server-side and client-side representations of column sizes; most of libtds
only needs to know about the client side.  The only time server-side sizes
are important is when calling functions that perform conversions.  

Fixing up the metadata and adding charset information to TDSICONVINFO will
let us remove ad hoc hard-coded fixups sprinkled throughout the code.  It
won't be easy, but it will be possible.  

Freddy: tds7_get_data_info() discards 5 collation bytes because we haven't
installed our collation structure in TDSCOLINFO.  When we do, we'll have
server-reported per-column charset information.  We'll then have to make a
design decision about whether or not to convert all such columns to a single
client charset.  

Radical note.  Suppose we succeed in deasciifying FreeTDS, and it becomes
fully capable of handling UCS-2 and UTF-8 clients.  Then, for a UCS-2
client, do we convert varchar data to UCS-2?  (They arrive in the server's
single-byte charset.)  If we mimic Microsoft's Win32 client libraries, the
answer is: No.  I propose that if in freetds.conf we find "client charset =
server charset", we do no conversion at all.  

--jkl


The information contained in this transmission may contain privileged and
confidential information and is intended only for the use of the person(s)
named above.  If you are not the intended recipient, or an employee or agent
responsible for delivering this message to the intended recipient, any
review, dissemination, distribution or duplication of this communication is
strictly prohibited. If you are not the intended recipient, please contact
the sender immediately by reply e-mail and destroy all copies of the
original message. Please note that we do not accept account orders and/or
instructions by e-mail, and therefore will not be responsible for carrying
out such orders and/or instructions.