FreeTDS Development Group

["Lowden, James K" <LowdenJK AT bernstein.com>]

> From: Frediano Ziglio [mailto:freddyz77 AT tin.it]
> Sent: May 13, 2003 4:00 PM
> 
> Well... I don't personally consider libTDS stable to test in
> production... Not using variable charset and not using no-GNU iconv
> version... HP/UX iconv do not run at all. Mac OS X do not work
> (ISO-8859-1 is not supported in Mac OS X.. strange).

OK, subject to those constraints.  Is there anything 0.61 does that can't be
done with current CVS?  

Currently, if we can't lookup a name, we return NULL for the canonical name
, and fail.  Perhaps better would be to return the input, assume it's a
single-byte charset, and use it if iconv_open() succeeds.  

> Problem with variable size is sending. We compute data size 
> to send very lazy... 

Yes.  I like your callback model.  

> Problem with HP/UX and Max OS X (Linux work, others not tested) is
> charset names. Mac OS X recognize ISO8859-1 but not ISO-8859-1. HP/UX
> has its names (like iso81, iso87, roman8, ucs2 and so on), do not have
> UCS-2LE (only ucs2 and it's big endian) and also can't convert from
> ISO8859-7 (iso87) to ISO8859-1 (iso81) directly... so requiring
> ISO8859-A on client with ISO8859-7 on server fail...

There are 3 translations:

1.  locale -> canonical.  Needed for bytes_per_char().
2.  server -> canonical.  Same use.
3.  canonical -> iconv.   needed for non-GNU iconv.  

For GNUers, #3 is a no-op.  To support non-GNU iconv, it's necessary, unless
we substitute the OS's definition of "canonical".  

I had recommended patching alternative_charsets.h, but unfortunately that
would present a problem with charsets.h (which holds the character sizes).
They share a key.  You use the output of tds_canonical_charset_name() as
input to bytes_per_char().  Changing the canonical name means changing the
same string in two places.  

One answer to *that* might be to use enumerated names (enum), and map
various strings to them.  We need the strings, see, for #2 above, to get an
iconv-acceptable name from whatever the server sends us.  Enumerated names
wouldn't save any effort (probably the opposite).  They would make the
comparisons easier, though.  

Instead, I suggest 

        const char * 
        tds_iconv_charset_name(const char *canonical_name);

Having looked up the canonical name for charsize purposes, call this
function to get the iconv name.  For GNU systems, make this a no-op macro.
For non-GNU systems, do a reverse lookup on
src/tds/alternative_character_sets.h, looking down the first column
(canonical names).  For the first matching canonical name, return the
alternative name.  We just have to be sure to keep the OS names ahead of the
server names in that file.  

That's the simplest solution I can see.  

> I updated tests on configure to test standard names. This 
> should help to update code. 

Yes, but you assume iconv(1) and iconv(3) use the same library.  Someone's
going to break that assumption sooner or later.  

> If a system does not have UCS-2LE we should set a flag like
> "convert UCS-2BE to UCS-2LE"... another suitable flag is pass 
> from UCS2
> (if direct conversion is not supported convert first to ucs2 
> and then to destination...)

tds_iconv_open() should set such a flag in the TDSICONVINFO structure, and
tds_iconv should honor it.  That way, no other code has to bother with it.  

--jkl
-----------------------------------------
The information contained in this transmission may contain privileged and 
confidential information and is intended only for the use of the person(s) 
named above. If you are not the intended recipient, or an employee or agent 
responsible for delivering this message to the intended recipient, any 
review, dissemination, distribution or duplication of this communication is 
strictly prohibited. If you are not the intended recipient, please contact 
the sender immediately by reply e-mail and destroy all copies of the original 
message. Please note that we do not accept account orders and/or instructions 
by e-mail, and therefore will not be responsible for carrying out such orders 
and/or instructions.