FreeTDS Development Group

["James K. Lowden" <jklowden AT schemamania.org>]

On Tue, 13 May 2003 00:38:57 +0300, "Spyros Ioakim" wrote:
> 
> I have added client charset = ISO-8859-7 in freetds.conf and executed
> tsql:
> /usr/local/freetds/bin/tsql -Saceoffice -Usa -I
> /usr/local/freetds/etc/freetds.conf 
> locale is "en_US"
> charset is "ISO-8859-1"

Hi Spyros, 

That's a bit misleading, I'm afraid.  tsql is reporting your locale
information, not what it read from freetds.conf.  BUT: atm libtds will use
only what's in freetds.conf ignoring your locale settings.  Basically,
it's not all wired up yet.  Welcome to the development cycle.  ;-)

Still, that's the least of the trouble.  The real problems show up in the
log.    

> Then changed my locale to el_GR:
> 
> /usr/local/freetds/bin/tsql -Saceoffice -Usa
> locale is "el_GR"
> charset is "ISO-8859-7"

Right.  Eventually -- 0.62, that is -- tsql will report what's actually
going to be used, and the libtds will check both your locale and
freetds.conf for client character set preferences.  Right now, you can get
tsql to report things, but you can't have any effect except through
freetds.conf.  

> The log is full of the following:
> Starting log file for FreeTDS 0.62.dev.20030511
>         on 2003-05-13 00:16:07 with debug level 99.
> iconv to convert client-side data to the "ISO-8859-7" character set
> 00:16:07.741573 tds_iconv_info_init: cannot convert
> "ISO-8859-7"->"UCS-2LE"

Ouch ouch ouch.  What I want to know is, Why the !@#$ Not?  

The error is ambiguous.  I just updated CVS to fix that.  You can use
tonight's tarball, or apply the attached patch.  The patch doesn't *fix*
anything; it just makes the log clearer.  Thta will distinguish between
iconv.c not recognizing the character set -- our bug, easy to fix -- and
some underlying libiconv problem.  

In case it's an iconv issue, here's one way to test your system.  

FreeTDS is relying on your iconv(3) to perform the conversions.  You never
mentioned, that I can see, what your host OS is, but I rather assumed it
was some flavor of Linux since you were working with PHP.  Let's see about
your iconv.

1. What does "iconv --version" show?  

        $ iconv --version
        iconv (GNU libiconv 1.8)
        ...

2. If similar, can iconv(1) convert from iso-8859-7 to UCS-2LE? 
        
If "yes", then let's see how iconv works in isolation of freetds.  I've
attached a tarball of a little test system I used.  Please untar it and

        cd iconv; make iso-8859-7.bin

That should yield a binary file of all valid values in the ISO-8859-7
character set.  My system doesn't have Greek fonts installed, but you
should be able to cat the file to your screen and recognize the output.  

See if you can reproduce the following with it:

$ hexdump -C iso-8859-7.bin |grep -i f0
000000a0  e6 e7 e8 e9 ea eb ec ed  ee ef f0 f1 f2 f3 f4 f5

Our friend =f0, Greek small letter PI, is at offset 0xAA in this file. 
That's 170 for you landlubbers.  

$ printf "%x\n" $(expr 2 \* `printf "%d\n" 0xaa`)
154

We would expect the UCS-2 equivalent of that character (=03c0) to be at
offset 0x154, because each UCS-2 character is twice as wide as the
corresponding 8859 one.  

$ iconv -f iso-8859-7 -t UCS-2LE iso-8859-7.bin \
        |hexdump  |grep -i 03C0
0000150 03be 03bf 03c0 03c1 03c2 03c3 03c4 03c5
                  ^^^^ how sweet it is.  

Let us know what you find out.  

--jkl

Attachment: iconv.c.diff
Description: Binary data

Attachment: iso-8859-7.tgz
Description: Binary data