FreeTDS Development Group

["Varley, David (CBorn at Alcoa)" <David.Varley AT alcoa.com.au>]

Jim,

>From what I can see, the conversions are:

ISO8859-1-GL_ISO8859-1
ISO8859-1-GR_ISO8859-1
ISO8859-1_ISO8859-1-GL
ISO8859-1_ct

with aliases by symbolic links as:

ct_ISO8859-1ASCII-GR_ISO8859-1 -> ISO8859-1_ISO8859-1-GL
ISO8859-1_ASCII-GR -> ISO8859-1-GL_ISO8859-1
ISO8859-1_ISO8859-1-GR -> ISO8859-1-GR_ISO8859-1
ISO8859-1_fold7 -> ISO8859-1_ct
ISO8859-1_fold8 -> ISO8859-1_ct
fold7_ISO8859-1 -> ct_ISO8859-1
fold8_ISO8859-1 -> ct_ISO8859-1

if this is of any help. My version of iconv doesn't support the
--list option of the GNU one, I've just taken these from the
/usr/lib/nls/loc/iconv directory (algorithmic converters).


> From: Lowden, James K [mailto:LowdenJK AT bernstein.com]
[...]
> Thanks for the report.  I've never seen it expressed this 
> way, but istm the
> development cycle has two phases: getting it to work correctly when
> everything's OK, and getting it to cope successfully when 
> they're not.  I
> hope we're in Phase II.  ;-)

My first job out of college was on control systems for secure laboratories
containing potentially lethal pathogens. Motorola 6802 based systems,
programmed in assembler, and the assumption was that anything that could go
wrong would. Bootstrapping was a nightmare, the CPU couldn't trust its own
registers, let alone RAM or ROM on startup until it had tested them. And you
would be surprised at the failures found, from CPUs where the Carry bit
wasn't being set on some operations, to EPROMS that would fail in one
location only following a specific instruction sequence. Those were the days
:)))

David

> > As it
> > stands, the package builds without problems, but bombs out on 
> > the first
> > test, with freetds.log containing:
> > 
> > Starting log file for FreeTDS 0.62.dev.
> >     on 2003-05-14 08:28:29 with debug level 99.
> > iconv to convert client-side data to the "ISO-8859-1" character set
> > 08:28:29.631897 tds_iconv_info_init: cannot convert 
> "CP819"->"UCS-2LE"
> > 08:28:29.631897 Connecting to 192.168.221.5 port 1433, TDS 7.0.
> > 08:28:29.632873 tds_put_string converting 6 bytes of "myhost"
> > 08:28:29.632873 tds_put_string converting 6 bytes of "myhost"
> > [...]
> > 
> > which continues until I get a "device full" message.
> > 
> > This can be traced to tds_put_string() in tds/write.c which 
> > loops forever
> > waiting for tds_iconv() to decrement 'len' below zero, which 
> > never happens.
> > Now while the problem is doubtless in iconv due to different 
> > character set
> > naming on Tru64, having the code loop forever because 
> > tds_iconv() doesn't convert anything is not good.
> 
> Right.  :-( It's important on any call to tds_iconv() to check if no
> conversions were made.  Passing it the same inputs over and 
> over is not a
> recipe for progress.  That I know how to fix.  
> 
> As things stand, we have effectively no support for non-GNU 
> iconv.  However
> we decide to address the issue technically, this much is 
> certain:  We need a
> list of charset names for any iconv we're going to support, 
> mapped to our
> canonical name (or at least a GNU name).  If you'll provide 
> that list, I'll
> work it into the system.  
> 
> The basic problem is, you see, we need to know how many bytes 
> to alot for
> each character.  We maintain a lookup table, in 
> src/tds/character_sets.h,
> with this information.  Its key is a string, our canonical 
> name for the
> character set.  That's why we need to map proprietary names to  our
> canonical ones.  
> 
> The fact that there's no way to obtain this information from the iconv
> system leads me to think that other applications just use a 
> worst-case, 4
> byte/char, assumption.  A database library doesn't have that 
> luxury.  We'd
> be allocating 32 KB for every 8000 byte varchar, of which half would
> typically be wasted.  
> 
> > 08:28:29.631897 tds_iconv_info_init: cannot convert 
> "CP819"->"UCS-2LE"
> 
> CP819 is in our table, and UCS-2LE has always been the name 
> we passed to
> iconv_open, going back at least to the 0.61 release.  My 
> guess is that Tru64
> iconv doesn't recognize "UCS-2LE".  Can you confirm?  
> 
> In the 0.61 release, if iconv failed to open, we reverted to 
> our "strip the
> high byte" algorithm.  That would be very complex to support 
> in the new
> framework, and not much use besides.  If we're going to 
> support Tru64, we
> might as well figure out how to engage its iconv.  
> 
> Regards, David, and thanks for your continued support.   
> 
> --jkl
> 
> 
> -----------------------------------------
> The information contained in this transmission may contain 
> privileged and confidential information and is intended only 
> for the use of the person(s) named above. If you are not the 
> intended recipient, or an employee or agent responsible for 
> delivering this message to the intended recipient, any 
> review, dissemination, distribution or duplication of this 
> communication is strictly prohibited. If you are not the 
> intended recipient, please contact the sender immediately by 
> reply e-mail and destroy all copies of the original message. 
> Please note that we do not accept account orders and/or 
> instructions by e-mail, and therefore will not be responsible 
> for carrying out such orders and/or instructions.
> 
> 
> _______________________________________________
> FreeTDS mailing list
> FreeTDS AT lists.ibiblio.org
> http://lists.ibiblio.org/mailman/listinfo/freetds
>