FreeTDS Development Group

["Lowden, James K" <LowdenJK AT bernstein.com>]

Hi Freddy, 

> From: Frediano Ziglio [mailto:freddyz77 AT tin.it]
> Sent: April 29, 2003 2:37 PM
> 
> Current CVS do not work with TDS8 (mssql2k protocol).

I don't have access to a 2K server, so this will be a challenge.  :-(

> If you want to try FreeTDS snapshot use TDS7 even for mssql2k.
> Problem is due to conversion stuff. During login TDS8 server send
> default collation for database using env type 7. This collation is
> readed and used to setup iconv according (using collation charset)

To do this right, we need a map from Microsoft's charset names to iconv
names.  

> however all next conversation use still ucs2...
> TDS7 do not have this problem cause return collation information in a
> different way (not still supported by FreeTDS, env types 5 and 6).

UTF-8 clients really need (at least) two TDSICONVINFO structures: 

1.  client <-> UCS-2 [nchar]
2.  client <-> server charset [char]

TDS 8.0 clients need more, potentially, one for each distinct client-server
charset pair.  

My solution so far:

TDSSOCKET::iconv_info becomes an array (not done yet).  

TDSCOLINFO gets a new member, TDSICONVINFO *iconv_info, that points to one
of the elements in TDSSOCKET::iconv_info.  (Done already, but only for UCS-2
columns, and not committed).  If the server column doesn't need iconv
intervention, its iconv_info is NULL.  

I set up the TDSICONVINFO* while adjusting the column_length, when I know
the server's and client's character set.  ATM is relies on the one-and-only
TDSICONVINFO structure, in the TDSSOCKET.  To make it general, we will need
utility functions to search the socket's array based on a pair of charset
names.  

I think an array for TDSSOCKET::iconv_info is better than a linked list,
because it will be reallocated infrequently.  Tell me if you think
different.  

> TDS7 and TDS8 handle collation in slightly different ways:
> 
> TDS7
> - every database have it's collation
          ^^^^^^^^ every *server*
...

> I think we should have some fixed iconv structures (like database one
> and ucs2) and cache others conversions.

Right.  That's my array in TDSSOCKET.  :-)

> Also tds_get_string is not sufficient 

Cf. tds_get_char_data().  Takes a TDSCOLINFO structure.  Soon, that
structure will have a pointer to a valid (cached) TDSICONVINFO structure.
Then it will be easy to decide what kind of conversion is needed.  

If tds_get_char_data() encounters a charset pair not already present in the
TDSSOCKET::iconv_info array, it will have to ask for one to be added.  That
function doesn't exist yet, either.  

tds_get_string() is only useful if you *know* the inbound string is UCS-2
and you don't have a TDSCOLINFO structure.  I.e., at login and when reading
metadata.  

We had a long discussion about handling conversion errors.  Per my earlier
message, we cannot silently insert a '?' whenever the conversion fails.  We
must emit an error message.  For reads, we can proceed, but for writes we
must refuse to process the data.  

I think it's OK to abort reading BIG5 data on failed conversions, too.  

Regards, 

--jkl


The information contained in this transmission may contain privileged and
confidential information and is intended only for the use of the person(s)
named above.  If you are not the intended recipient, or an employee or agent
responsible for delivering this message to the intended recipient, any
review, dissemination, distribution or duplication of this communication is
strictly prohibited. If you are not the intended recipient, please contact
the sender immediately by reply e-mail and destroy all copies of the
original message. Please note that we do not accept account orders and/or
instructions by e-mail, and therefore will not be responsible for carrying
out such orders and/or instructions.