freetds AT lists.ibiblio.org
Subject: FreeTDS Development Group
List archive
- From: Frediano Ziglio <freddyz77 AT tin.it>
- To: FreeTDS Development Group <freetds AT lists.ibiblio.org>
- Subject: RE: [freetds] Status
- Date: Thu, 11 Sep 2003 14:27:09 -0000
Il lun, 2003-09-08 alle 23:15, Lowden, James K ha scritto:
> > From: Frediano Ziglio [mailto:freddyz77 AT tin.it]
> > Sent: September 8, 2003 1:32 PM
> >
> > I noted James is back from holiday :)
> > I won't be able to use computer for some time (I'll have an
> > operation).
>
> Hi, Freddy. I hope your operation is not serious and that it goes well.
>
As you can see it's going very well :)
> > In the last two/three weeks
> > I applied some patches (mainly for ODBC) and did appropriate
> > updates/fixes.
>
> I noticed you've been busy. No blackouts in Italy? ;-)
>
There was some risk some month ago cause it hadn't rain for months but
now is ok. Wet and no rain... very terrible...
> > tds - iconv stuff
> > I didn't change this stuff. There are still a lot FIXME in
> > code. I noted
> > that someone (I think James) changed fixed characters set from
> > iso8859-1, utf8, ucs2le/be to iso885901, ascii, ucs2le/be.
> > However this
> > is not correct (see tds_iconv_init). Some iconv implementations (like
> > HP/UX one) refuse to convert from single-byte to single-byte to
> > replacing iso8859-1 -> utf8 conversion with iso8859-1 ->
> > ascii one just fails.
>
> Mea culpa. Gads, this is knotty. Let's make sure we're solving only what
> we need to solve.
>
> You were testing for utf8 capacity, which we don't need for most cases. I
> accept it's a reasonable expectation of an iconv implementation, but our
> trivial one lacks it. FreeTDS wants to know, "Do we have iconv?" and I want
> the answer to always be "Yes", or at least, "Yes, we have a minimal iconv
> that will handle ISO-8859-1 <-> UCS2", because that's all most people need
> to use M$ servers.
>
> A person using HP/UX iconv might or might not need UTF-8, but if he's using
> a M$ server, he definitely needs:
>
> 1. ASCII --> UCS2 (for hard-coded queries in libtds)
> 2. ASCII --> client (for FreeTDS-generated messages)
> 3. client <-> UCS2 (for metadata and nchar data)
> 4. client <-> server (for char data)
>
5. client <-> utf8 (sybase utf8, still not supported by FreeTDS)
> Cases 2 and 4 are also useful for Sybase servers.
>
> Testing for UTF-8 is too much, because it's not fundamentally needed. Even
> though "every" iconv implementation supports it, we shouldn't require it
> because we don't need it.
>
> OTOH, we definitely need ASCII <-> client, especially if the client is
> UCS-2! I want to pass all character data through iconv before handing them
> to the client, including internally-generated ASCII messages. If the client
> is ISO-8859-1, and HP/UX iconv won't do #2 or #4, it's important to know
> that. There'll have to be logic in tds_iconv to deal with that case.
>
Well.. time ago I added some declaration for flags:
- indirect conversion. Convert source -> ucs2 -> destination if source
-> destination is not supported.
- byte copy. Use memcpy if same charset (we can use indexes and just
compare them)
- byte swap. HP/UX support only "ucs2" charset and is big endian (little
endian not supported)
> By the same token, I expect iconv to be able to handle like-to-like (e.g.
> ISO-8859-1 <-> ISO-8859-1). In short, it seems tds_iconv() has to know when
> a conversion is trivial (or nil) but unsupported, and revert to memcpy(3) as
> needed.
>
> There's nothing special about ISO-8859-1, either. It's just the most common
> client character set. As far as I'm concerned, it shouldn't be mentioned in
> tds_iconv_init(), unless....
>
Well iso8859-1 has also another property: it can be easily converted to
unicode/utf8 (just same code, only different packing).
> > Also for our purpose we can use iso8859-1
> > instead of ascii (cause it's just an extension).
>
> That might be a useful simplification. All our messages and hard-coded SQL
> text are ASCII, but it's also true they're ISO-8859-1. I doubt there are
> any places where we rely on 7-bit encoding. I'd be happy to replace "ASCII"
> with "ISO-8859-1" in cases #1 and #2, above. We could delete all references
> to ASCII in the code. That might simplify tds_iconv_init() somewhat. I'll
> leave any adjustments to tds_iconv_init() to you.
>
I can use PC very much...
> To cope with a recalcitrant "we-don't-do-no-trivial-conversions" iconv (e.g.
> HP/UX), I suppose we'll have to add a flag to TDSICONVINFO to indicate it's
> OK to use memcpy() if the conversion descriptor is -1.
>
IMHO we should use memcpy even with GNU iconv... just performance :)
> What do you think?
>
I don't think 0.62 will be ready before September end...
freddy77
-
[freetds] Status,
Frediano Ziglio, 09/08/2003
- <Possible follow-up(s)>
-
RE: [freetds] Status,
Lowden, James K, 09/08/2003
- RE: [freetds] Status, Frediano Ziglio, 09/11/2003
- RE: [freetds] Status, Lowden, James K, 09/11/2003
- [freetds] Status, freddyz77, 09/19/2003
- [freetds] status, ZIGLIO Frediano, 09/25/2003
- RE: [freetds] status, Lowden, James K, 09/25/2003
- RE: [freetds] status, Thompson, Bill D (London), 09/25/2003
- RE: [freetds] status, Lowden, James K, 09/25/2003
- RE: [freetds] status, Castellano, Nicholas, 09/25/2003
- RE: [freetds] status, ZIGLIO Frediano, 09/26/2003
Archive powered by MHonArc 2.6.24.