FreeTDS Development Group

[ZIGLIO Frediano <Frediano.Ziglio AT vodafoneomnitel.it>]

> 
> Well, that would be interesting.  A FreeTDS that doesn't care 
> about nulls
> in its string buffers and doesn't ever make any assumptions about
> character width, of course, could pass UCS-2 straight 
> through.  That would
> be better than dual conversion, for sure.  
> 

Many patch I have applied work in this direction (see dbconvert/cs_convert
and convert.c)

> 
> > What's needed? unicode columns in SQL Server and a terminal 
> that support
> > 
> > utf-8 with sqsh.  
> 
> Yup.  AIUI, UTF-8 has been hacked into the Linux console, but 
> that's about
> it.  There's no free xterm that understands it.  You'd have 
> to pipe the
> output to a browswer or something.  
> 

I think xfree xterm and even console can handle utf8.
I'd like to remember that utf8 IS unicode. Do not make confusion from the ms
way of view. utf8 can represent much more characters than ucs2 (well,
additional are quite useless but uft8 -> ucs2 conversion can fail while ucs2
-> utf8 can't).

I don't like poor performance, so double conversion is not so good.
You have taking into account:
- data in mssql2000 can be coded in different format (this is the the
collating) for every column in a table (a column with iso_1 a column with
latin2 and so on)
- data can be returned (wire) in ucs2(2
byte)/utf8(multiple-byte)/single-byte
- iconv can't sometime convert from a single-byte to another single-byte
(see also iconv code in samba, they use a two pass convert if direct
conversion do not exist)
- using ucs2 internal coding is a waste for european/english people
- no conversion at all work but may result in garbage (is server is coded in
Latin1 and client require Latin2 all not english character are replaced with
wrong characters)
- using an internal representation with only single byte coding loose
characters. If client can require utf8 FreeTDS have to support multibyte
character (like utf8) in FreeTDS.
- on ML I have seen Chinese, european (east, west and south) and people from
other country...
- ODBC calls can require single byte or unicode ucs2 result.

So:
- FreeTDS MUST handle multibyte characters
- i propose reallocate row buffer if too short (resolve waste of space for
european/english)

If we don't want an internal representation I think we must use client
representation as internal (so internal -> client is just a copy).

> 
> Made some progress on the UG.  Am a little concerned about cancel
> processing wrt the release.  What do you think?
> 

I think that the problem should have a simple solution: discard results
saved in TDSSOCKET on a cancel :).
I see also that a cancel operation can be issues from another thread
(separate from thread that read data).. this will be a future feature..

bye
  freddy77

=================================
"STRICTLY PERSONAL AND CONFIDENTIAL

This message may contain confidential and proprietary material for the sole
use of the intended recipient. Any review or distribution by others is
strictly prohibited. If you are not the intended recipient please contact
the sender and delete all copies.
The contents of this message that do not relate to the official business of
our company shall be understood as neither given nor endorsed by it."

=================================