FreeTDS Development Group

["Lowden, James K" <LowdenJK AT bernstein.com>]

> From: Frediano Ziglio [mailto:freddyz77 AT tin.it]
> Sent: September 11, 2003 10:28 AM
> 
> > Hi, Freddy.  I hope your operation is not serious and that 
> > it goes well.  
> 
> As you can see it's going very well :)

Pleased to hear!  

> > A person using HP/UX iconv might or might not need UTF-8, 
> > but if he's using
> > a M$ server, he definitely needs:
> > 
> > 1.   ASCII --> UCS2   (for hard-coded queries in libtds)
> > 2.   ASCII --> client (for FreeTDS-generated messages)
> > 3.  client <-> UCS2   (for metadata and nchar data)
> > 4.  client <-> server (for char data)
> 
> 5. client <-> utf8 (sybase utf8, still not supported by FreeTDS)

I see your point: a Sybase server might be configured with UTF-8 encoding,
letting it handle many varied clients.  My point is different: support of
TDS 7.0 absolutely requires the above 4 conversions.  FreeTDS supports TDS
7.0 out of the box, without libiconv, for ISO-8859-1 clients.  UTF-8 support
(and libiconv) is an important feature, but it is an *option*, and shouldn't
be part of the basic initialization.  IOW, UCS-2 is special, and UTF-8 is
not.  

> Well.. time ago I added some declaration for flags:
> - indirect conversion. Convert source -> ucs2 -> destination if source
> -> destination is not supported.

I'm OK with that.  I think it's a bit esoteric, because it's a corner case.
If you weren't working on it, I'd tell anyone whose iconv didn't support
direct conversion to use GNU instead.  Or send a patch, as you're doing.
;-)

> - byte copy. Use memcpy if same charset (we can use indexes and just
> compare them)

Let us please try to avoid too much optimization.  I would like to avoid
branches testing whether or not we need iconv.  There's no performance gain
to be had.  The iconv() call will be fast, and the buffer pointers and
counts need to be updated by someone.  It's better to call iconv() *every*
possible time, even if memcpy()+fixups could accomplish the same thing.  

> - byte swap. HP/UX support only "ucs2" charset and is big 
> endian (little endian not supported)

This is clearly important.  

> Well iso8859-1 has also another property: it can be easily 
> converted to
> unicode/utf8 (just same code, only different packing).

For some value of "easily".  The range 128-255 in ISO-8859-1 requires two
bytes in UTF-8.  IIRC the first 32 bytes in that range are special, too.  

> I don't think 0.62 will be ready before September end...

Surely not.  I just received notice the replacement parts for my devbox will
be delivered next week.  I would expect nightly snapshots to resume on 21
September, assuming the hard disks weren't also injured in the blackout.  

Regards, 

--jkl
-----------------------------------------
The information contained in this transmission may contain privileged and 
confidential information and is intended only for the use of the person(s) 
named above. If you are not the intended recipient, or an employee or agent 
responsible for delivering this message to the intended recipient, any 
review, dissemination, distribution or duplication of this communication is 
strictly prohibited. If you are not the intended recipient, please contact 
the sender immediately by reply e-mail and destroy all copies of the original 
message. Please note that we do not accept account orders and/or instructions 
by e-mail, and therefore will not be responsible for carrying out such orders 
and/or instructions.