Skip to Content.
Sympa Menu

freetds - Re: [freetds] Unicode problem

freetds AT lists.ibiblio.org

Subject: FreeTDS Development Group

List archive

Chronological Thread  
  • From: Vano Beridze <vano.beridze AT silkroad.ge>
  • To: FreeTDS Development Group <freetds AT lists.ibiblio.org>
  • Subject: Re: [freetds] Unicode problem
  • Date: Sat, 14 Sep 2013 13:57:53 +0400

It was the first case.
tsql displayed characters correctly.

then I looked into php and it seems the problem was with odbc library.
I could not get it working and then switched to mssql library and finally got it working.

Thank you for your time and attention.

Vano Beridze
Software Developer
Silk Road Group

On 9/10/2013 7:58 AM, James K. Lowden wrote:
On Mon, 09 Sep 2013 22:48:48 +0400
Vano Beridze <vano.beridze AT silkroad.ge> wrote:

This is the first lines of /tmp/freetds.log

log.c:196:Starting log file for FreeTDS 0.91
on 2013-07-05 23:20:33 with debug flags 0x4fff.
iconv.c:330:tds_iconv_open(0x7fd771f79e10, UTF-8)
iconv.c:187:local name for ISO-8859-1 is ISO-8859-1
iconv.c:187:local name for UTF-8 is UTF-8
iconv.c:187:local name for UCS-2LE is UCS-2LE
iconv.c:187:local name for UCS-2BE is UCS-2BE
iconv.c:349:setting up conversions for client charset "UTF-8"
iconv.c:351:preparing iconv for "UTF-8" <-> "UCS-2LE" conversion
iconv.c:391:preparing iconv for "ISO-8859-1" <-> "UCS-2LE" conversion
OK, you're using an iconv library, and UTF-8 for a client character
set.

If you select just one row with tsql, is it displayed correctly? If
not, does tsql print an error about not being able to convert?

If the FreeTDS can't convert the data received from the server to the
client's encoding, it raises an error. tsql will print that message if
it occurs. There are 3 possibilities:

1. tsql displays the characters correctly in the terminal. Hooray!
Go look into PHP.

2. tsql does not display the characters correctly, and produces a
message saying it encountered problems during conversion. That's an
iconv problem. Check your documentation and make sure iconv(1) works
on known data.

You can also execute a query like

select cast('hello' as nvarchar(5)) as hello

If tsql displays that correctly, you might have corrupt "Unicode"
data in the nvarchar column you're working with.

3. tsql does not display the characters correctly, but produces no
error message. If you can't read the characters and don't see an error,
then FreeTDS encountered no difficulty converting, but your system
can't actually display the data. Perhaps because of fonts or
something.

As for UCS2 data, table column type is nvarchar(30) and as I know if
the column is nvarchar or nchar it is stored as UCS2
Figure out what the bit representation is for one row, and satisfy
yourself that it's little-endian 2-byte Unicode. You can use "select
cast( foo as varbinary )" to get a hex representation, or look at the
data packet in a TDSDUMP log.

HTH.

--jkl

On 9/5/2013 6:51 AM, James K. Lowden wrote:
On Tue, 03 Sep 2013 14:20:28 +0400
Vano Beridze <vano.beridze AT silkroad.ge> wrote:

The script works fine. it connects and gets the data. The only
problem I have is that data is shown as question marks, so it's not
converted to UTF-8. I've read all the information I could find on
the internet but can not solve the problem.
I would set TDSDUMP and examine the log. You're right that
FreeTDS would substitute question marks for any non-convertible
character. Perhaps the character set name you specified in
freetds.conf isn't exactly right, perhaps because you're linked to
the system iconv library.

The status of iconv is listed at the top of the log. If it says it
set the client charset to UTF-8, that's what it will use.

A different problem that sometimes crops up: sometimes the "Unicode"
data on the server isn't UCS2. In that event, the server will send
the data, but the iconv library will be unable to convert the
characters correctly.

--jkl
_______________________________________________
FreeTDS mailing list
FreeTDS AT lists.ibiblio.org
http://lists.ibiblio.org/mailman/listinfo/freetds








Archive powered by MHonArc 2.6.24.

Top of Page