FreeTDS Development Group

["Lowden, James K" <LowdenJK AT bernstein.com>]

> From: ZIGLIO Frediano [mailto:Frediano.Ziglio AT vodafone.com]
> Sent: September 25, 2003 6:18 AM

Hi Freddy, 

Thanks for the summary.  We have a few things that need addressing before I
think we're ready to start releasing 0.62.  

>  - support string conversions for Sybase
> 
>    We can skip this for now...

OK. 

>  - support different column charset under mssql2k
>     
>    Post or implement in 0.62 ?

I'd like to see this in 0.62.  TDSCOLINFO supports it; we just have to make
sure that each column's charset is initialized.  

>  - convert string using input parameters
> 
>    Input parameters are not converted. Code do not core but 
> someone can report 
>    strange problems if server charset <-> client charset are very
> different

Up to you.  I think it can wait, unless a UTF-8 client would be affected.  

>  - finish support for no-GNU iconv (indirect conversion see
> src/tds/iconv.c)
>    
>    HP/UX and other require indirect conversions...

This you're doing.  

>  - borrow GNU's libiconv m4 macro to test/provide EILSEQ

Some iconv libraries don't set EILSEQ, and some OS's don't define it.  It
came up with QNX.  GNU handles the situation with an elaborate m4 macro that
autoconf calls.  I want to use the same logic.  

>  - add test for locale_charset() to configure.in.  Use it if available
>    instead of nl_langinfo, because it implies we're using GNU iconv
> 
>    In previous versions we always used ISO-8859-1 as our default
> charset,
>    only tsql set another charset. Should freetds use system default
>    charset as its default ?

It is up to the application to set the locale with setlocale(3).  FreeTDS
only reads it.  tsql is an example of an application that sets its locale.  

I thought once that we'd have to know whether or not we're using GNU iconv.
You demonstrated otherwise, that we could infer/lookup the charset names.
Right now, I don't see any need to know if we're using GNU iconv.  

>  - walk through charset name initialization.  Make sure it 
> works in the absence of an entry in freetds.conf

I think this is fixed.  

>  - pass any internal hard-coded charset names through
>    tds_canonical_charset_name()

superseded and moot.  

>  - the supporting .h file will need adjustment for non-GNU iconv

You did this (believe it or not!).  

>  - input name "ASCII" must always have a canonical form.
> 
>    now we use ISO-8859-1 to convert freetds string to client.
>    I don't think this is a problem so we can post this stuff

Right, no problem.  


Other issues:

1.  Defeat iconv.  Right now, if autoconf finds libiconv, it uses it.  Bill
needs a --without-libiconv switch, to *force* use of the built-in
replacement.  

2.  Ignore iconv.  We go to a lot of effort to determine the client
character set and intend to convert even singlebyte encodings to match the
server's.  BCP, however, presents a different problem, something that's
obvious if you look at Sybase's documentation for their bcp utility.   When
bcp'ing data, there's no reason to assume the encoding of the file's data is
the same as the client charset.  It would be best to follow Sybase and let
freebcp explicitly set the encoding information, but at the very least we
should be able to turn off iconv.  That would put the application in charge.


In short, we need at least to support dbcharsetconv().  We don't absolutely
have to have it for 0.62, but I think it might actually not be that hard,
now that your have a flag in TDSICONVINFO.  

3.  The most worrisome problem is not iconv related; it keeps cropping up
with PHP: this issue of handling an empty result set (I think).  I haven't
had time to make sure that at least our ct-lib implementation is right, and
I'm hoping that our PHP friends here will help us track it through PHP.  

Recent related messages:

        Damian Kramer, September 23, "Possible bug in freeTDS"
        Steve Hanselman September 16 "Issue with freetds 0.61.2 ..."

I know there were others, and ISTR there was some issue with PHP (even a
patch?), but it needs to be flushed out.  If it's an issue with a particular
PHP release, we should document it.  

> Prepared statement need adjustement (blobs for Sybase, 
> SQLPutData). This stuff
> used to work in 0.61 so it's a good thing to fix it before 0.62.

Agreed.  No point in going backwards. ;-)  Too much confusion on the list
otherwise.  

> The good thing I discovered is that output parameters under ODBC are
> only supported
> for RPC, so I can use TDS_LANGUAGE under Sybase for binary data (very
> useful for 
> optimizing bandwidth on blobs instead of converting all to a language
> string)

        :-)

Tests.  I think we should have a UTF-8 unit test.  The iconv integration was
spurred by UTF-8 clients, and we said 0.62 would support them.  We should
know for sure that it does.  

You've put a lot of work into the iconv stuff this month, and it shows.  :-)
Are you satisfied with that state of affairs?  

I would like to fix the PHP issue and make all the autoconf changes before
we start a release cycle.  

Also, the documentation needs updating (we don't need --with-iodbc and
--with-unixodbc, right?).  There's no mention of the iconv strategy, the
automatic charset detection, the built-in replacement, the UTF-8 (?)
readiness.  There's no pointer in the UG to the nacent reference manual, and
no statement about its status.  

Once the release is (in our humble opinion) ready, it will want broad
testing.  I would like to include in the README a list of OS's, iconvs, and
charsets that have been used successfully, and I'd really like a UFT-8
client on that list, and anyone who's not ISO-8859-1.  

Definitely the worst is over.  There are many issues, but nothing
overwhelming, and some things can be postponed for 0.63.  Do you agree?  

--jkl
-----------------------------------------
The information contained in this transmission may contain privileged and 
confidential information and is intended only for the use of the person(s) 
named above. If you are not the intended recipient, or an employee or agent 
responsible for delivering this message to the intended recipient, any 
review, dissemination, distribution or duplication of this communication is 
strictly prohibited. If you are not the intended recipient, please contact 
the sender immediately by reply e-mail and destroy all copies of the original 
message. Please note that we do not accept account orders and/or instructions 
by e-mail, and therefore will not be responsible for carrying out such orders 
and/or instructions.