FreeTDS Development Group

[Brian Bruns <camber AT ais.org>]

On Mon, 16 Jul 2001, Steve Langasek wrote:

> On Sat, 14 Jul 2001, Brian Bruns wrote:
> 
> > That was very insightful. I think we have three major issues here:
> 
> > 1) TDS 7/8.  Everything is unicode, so we should just be able to read the
> > field in freetds.conf and use iconv to convert between UCS2 and
> > that.  If there is no entry in freetds.conf, we could try to grab the
> > value of $LANG, although I'm not sure how that maps to the iconv
> > labels. Failing there we simply use the present 7bit ASCII conversion
> 
> I rather favor defaulting to UTF-8 instead of ASCII when no charset has been
> specified.  If there's non-ascii data in a database, it's because someone 
> put
> it there; doesn't it make more sense to treat this data losslessly by 
> default,
> passing it through for the client app to deal with?

Ok, my concern is application support, does PHP, Perl (DBD::Sybase,
DBD::ODBC), sqsh, Gnome-DB, SybSQL, sybtcl, Python (although we don't have
this working yet), support UTF-8 out of the box in some reasonable fashion
that is going to make this work?

My belief is that several of them are ok, but some may not be.  

When accessing a database via TDS 4.2 the data is returned using the
default character set of the database. I would assume the appropriate
thing to do in this case is preserve that charset, what ever it may be.

So, if we have a MS SQL server being accessed with 4.2, it comes down in a
national charset, but the same database under TDS 7.0 is unicode and is
converted to UTF-8.  That's a big change for just switching the protocol
version.

I guess at this point I don't have many answers, only a lot of stupid
questions. ;-)

> > 3) multibyte char sets.  I'm afraid I just don't know enough about usage
> > of these to make a good attempt at it.  We'll shelve it until a native
> > user comes along. ;-)
> 
> For the most part, I think iconv will be doing all the hard work.  If we 
> have
> working conversion from UCS2->UTF8 (wchar->mbyte) and UCS2->ISO-8859-x
> (wchar->char), I expect we'll get working conversion to other multibyte
> charsets for free.
>
iconv does most of it, yes. However, I have no idea what multibyte or
variable byte (UTF-8) streams look like on the wire, so handling these in
libtds is currently very broken, and unlikely to be fixed anytime soon.

> ObApology: Debian packages for freetds 0.52 are coming along.  Didn't have 
> as
> much time to start on them before the wedding as I had hoped. :)

I can relate to that.

Brian