Skip to Content.
Sympa Menu

freetds - RE: [freetds] Status

freetds AT lists.ibiblio.org

Subject: FreeTDS Development Group

List archive

Chronological Thread  
  • From: Frediano Ziglio <freddyz77 AT tin.it>
  • To: FreeTDS Development Group <freetds AT lists.ibiblio.org>
  • Subject: RE: [freetds] Status
  • Date: Thu, 11 Sep 2003 14:27:09 -0000

Il lun, 2003-09-08 alle 23:15, Lowden, James K ha scritto:
> > From: Frediano Ziglio [mailto:freddyz77 AT tin.it]
> > Sent: September 8, 2003 1:32 PM
> >
> > I noted James is back from holiday :)
> > I won't be able to use computer for some time (I'll have an
> > operation).
>
> Hi, Freddy. I hope your operation is not serious and that it goes well.
>

As you can see it's going very well :)

> > In the last two/three weeks
> > I applied some patches (mainly for ODBC) and did appropriate
> > updates/fixes.
>
> I noticed you've been busy. No blackouts in Italy? ;-)
>

There was some risk some month ago cause it hadn't rain for months but
now is ok. Wet and no rain... very terrible...

> > tds - iconv stuff
> > I didn't change this stuff. There are still a lot FIXME in
> > code. I noted
> > that someone (I think James) changed fixed characters set from
> > iso8859-1, utf8, ucs2le/be to iso885901, ascii, ucs2le/be.
> > However this
> > is not correct (see tds_iconv_init). Some iconv implementations (like
> > HP/UX one) refuse to convert from single-byte to single-byte to
> > replacing iso8859-1 -> utf8 conversion with iso8859-1 ->
> > ascii one just fails.
>
> Mea culpa. Gads, this is knotty. Let's make sure we're solving only what
> we need to solve.
>
> You were testing for utf8 capacity, which we don't need for most cases. I
> accept it's a reasonable expectation of an iconv implementation, but our
> trivial one lacks it. FreeTDS wants to know, "Do we have iconv?" and I want
> the answer to always be "Yes", or at least, "Yes, we have a minimal iconv
> that will handle ISO-8859-1 <-> UCS2", because that's all most people need
> to use M$ servers.
>
> A person using HP/UX iconv might or might not need UTF-8, but if he's using
> a M$ server, he definitely needs:
>
> 1. ASCII --> UCS2 (for hard-coded queries in libtds)
> 2. ASCII --> client (for FreeTDS-generated messages)
> 3. client <-> UCS2 (for metadata and nchar data)
> 4. client <-> server (for char data)
>

5. client <-> utf8 (sybase utf8, still not supported by FreeTDS)

> Cases 2 and 4 are also useful for Sybase servers.
>
> Testing for UTF-8 is too much, because it's not fundamentally needed. Even
> though "every" iconv implementation supports it, we shouldn't require it
> because we don't need it.
>
> OTOH, we definitely need ASCII <-> client, especially if the client is
> UCS-2! I want to pass all character data through iconv before handing them
> to the client, including internally-generated ASCII messages. If the client
> is ISO-8859-1, and HP/UX iconv won't do #2 or #4, it's important to know
> that. There'll have to be logic in tds_iconv to deal with that case.
>

Well.. time ago I added some declaration for flags:
- indirect conversion. Convert source -> ucs2 -> destination if source
-> destination is not supported.
- byte copy. Use memcpy if same charset (we can use indexes and just
compare them)
- byte swap. HP/UX support only "ucs2" charset and is big endian (little
endian not supported)

> By the same token, I expect iconv to be able to handle like-to-like (e.g.
> ISO-8859-1 <-> ISO-8859-1). In short, it seems tds_iconv() has to know when
> a conversion is trivial (or nil) but unsupported, and revert to memcpy(3) as
> needed.
>
> There's nothing special about ISO-8859-1, either. It's just the most common
> client character set. As far as I'm concerned, it shouldn't be mentioned in
> tds_iconv_init(), unless....
>

Well iso8859-1 has also another property: it can be easily converted to
unicode/utf8 (just same code, only different packing).

> > Also for our purpose we can use iso8859-1
> > instead of ascii (cause it's just an extension).
>
> That might be a useful simplification. All our messages and hard-coded SQL
> text are ASCII, but it's also true they're ISO-8859-1. I doubt there are
> any places where we rely on 7-bit encoding. I'd be happy to replace "ASCII"
> with "ISO-8859-1" in cases #1 and #2, above. We could delete all references
> to ASCII in the code. That might simplify tds_iconv_init() somewhat. I'll
> leave any adjustments to tds_iconv_init() to you.
>

I can use PC very much...

> To cope with a recalcitrant "we-don't-do-no-trivial-conversions" iconv (e.g.
> HP/UX), I suppose we'll have to add a flag to TDSICONVINFO to indicate it's
> OK to use memcpy() if the conversion descriptor is -1.
>

IMHO we should use memcpy even with GNU iconv... just performance :)

> What do you think?
>

I don't think 0.62 will be ready before September end...

freddy77






Archive powered by MHonArc 2.6.24.

Top of Page