Skip to Content.
Sympa Menu

freetds - RE: [freetds] Status

freetds AT lists.ibiblio.org

Subject: FreeTDS Development Group

List archive

Chronological Thread  
  • From: "Lowden, James K" <LowdenJK AT bernstein.com>
  • To: 'FreeTDS Development Group' <freetds AT lists.ibiblio.org>
  • Subject: RE: [freetds] Status
  • Date: Mon, 8 Sep 2003 17:15:27 -0400

> From: Frediano Ziglio [mailto:freddyz77 AT tin.it]
> Sent: September 8, 2003 1:32 PM
>
> I noted James is back from holiday :)
> I won't be able to use computer for some time (I'll have an
> operation).

Hi, Freddy. I hope your operation is not serious and that it goes well.

> In the last two/three weeks
> I applied some patches (mainly for ODBC) and did appropriate
> updates/fixes.

I noticed you've been busy. No blackouts in Italy? ;-)

> tds - iconv stuff
> I didn't change this stuff. There are still a lot FIXME in
> code. I noted
> that someone (I think James) changed fixed characters set from
> iso8859-1, utf8, ucs2le/be to iso885901, ascii, ucs2le/be.
> However this
> is not correct (see tds_iconv_init). Some iconv implementations (like
> HP/UX one) refuse to convert from single-byte to single-byte to
> replacing iso8859-1 -> utf8 conversion with iso8859-1 ->
> ascii one just fails.

Mea culpa. Gads, this is knotty. Let's make sure we're solving only what
we need to solve.

You were testing for utf8 capacity, which we don't need for most cases. I
accept it's a reasonable expectation of an iconv implementation, but our
trivial one lacks it. FreeTDS wants to know, "Do we have iconv?" and I want
the answer to always be "Yes", or at least, "Yes, we have a minimal iconv
that will handle ISO-8859-1 <-> UCS2", because that's all most people need
to use M$ servers.

A person using HP/UX iconv might or might not need UTF-8, but if he's using
a M$ server, he definitely needs:

1. ASCII --> UCS2 (for hard-coded queries in libtds)
2. ASCII --> client (for FreeTDS-generated messages)
3. client <-> UCS2 (for metadata and nchar data)
4. client <-> server (for char data)

Cases 2 and 4 are also useful for Sybase servers.

Testing for UTF-8 is too much, because it's not fundamentally needed. Even
though "every" iconv implementation supports it, we shouldn't require it
because we don't need it.

OTOH, we definitely need ASCII <-> client, especially if the client is
UCS-2! I want to pass all character data through iconv before handing them
to the client, including internally-generated ASCII messages. If the client
is ISO-8859-1, and HP/UX iconv won't do #2 or #4, it's important to know
that. There'll have to be logic in tds_iconv to deal with that case.

By the same token, I expect iconv to be able to handle like-to-like (e.g.
ISO-8859-1 <-> ISO-8859-1). In short, it seems tds_iconv() has to know when
a conversion is trivial (or nil) but unsupported, and revert to memcpy(3) as
needed.

There's nothing special about ISO-8859-1, either. It's just the most common
client character set. As far as I'm concerned, it shouldn't be mentioned in
tds_iconv_init(), unless....

> Also for our purpose we can use iso8859-1
> instead of ascii (cause it's just an extension).

That might be a useful simplification. All our messages and hard-coded SQL
text are ASCII, but it's also true they're ISO-8859-1. I doubt there are
any places where we rely on 7-bit encoding. I'd be happy to replace "ASCII"
with "ISO-8859-1" in cases #1 and #2, above. We could delete all references
to ASCII in the code. That might simplify tds_iconv_init() somewhat. I'll
leave any adjustments to tds_iconv_init() to you.

To cope with a recalcitrant "we-don't-do-no-trivial-conversions" iconv (e.g.
HP/UX), I suppose we'll have to add a flag to TDSICONVINFO to indicate it's
OK to use memcpy() if the conversion descriptor is -1.

What do you think?

--jkl


-----------------------------------------
The information contained in this transmission may contain privileged and
confidential information and is intended only for the use of the person(s)
named above. If you are not the intended recipient, or an employee or agent
responsible for delivering this message to the intended recipient, any
review, dissemination, distribution or duplication of this communication is
strictly prohibited. If you are not the intended recipient, please contact
the sender immediately by reply e-mail and destroy all copies of the original
message. Please note that we do not accept account orders and/or instructions
by e-mail, and therefore will not be responsible for carrying out such orders
and/or instructions.






Archive powered by MHonArc 2.6.24.

Top of Page