FreeTDS Development Group

["Lowden, James K" <LowdenJK AT bernstein.com>]

> From: FlavianoMa
> Sent: Thursday, May 05, 2005 1:31 PM
> 
> In my system is not available the iconv library, so I modified
> the built-in iconv substitute in FreeTDS to accept different
> charset ( CP1250, CP1254, CP1256, CP1251 ).  

My, my.  I'd be interested in applying your patches, if you'd like to
contribute them.  But you really wouldn't have needed to do all that
work if you'd installed the GNU libiconv library.  

> Now when I read
> data from the SQL Server I receive data correctly.  When I
> write on the server, I write the correct Unicode buffer ( I
> checked on freetds.log), but the server convert my character
> before save on DB, so after if I'll read the same buffer it
> will be different ( checked on freetds.log).

I think this is what's happening: you write UCS-2 data to the server, to
be inserted in a column with a datatype of char, varchar, or text -- a
single-byte encoding.  Depending on the server version, the character
set of a column is defined by server or (in 2000) by column.  The server
obviously can't store your UCS-2 data in a  varchar column verbatim, so
it converts it to the encoding defined for that column.  When you then
SELECT the same data from the table, you find it's not encoded
"correctly" i.e., not according to the client's charset.  

With FreeTDS 0.63 and a complete iconv implementation, even the above
scenario should work.  When you SELECT the data, it comes labelled with
its encoding (determined at login with TDS 7.0, per resultset column
with TDS 8.0).  FreeTDS converts the data -- insofar as possible -- to
the client's encoding as specified in freetds.conf.  

Assuming I'm right, you have these choices:

1.  Use nchar, nvarchar, or ntext columns.  Then the server can store
your USC-2 data without conversion.  

2.  Match your server's encoding to the client.  

3.  Use GNU iconv.  

For the record, Microsoft servers never convert encodings headed for the
client.  They expect their clients to support UCS-2 and to be configured
with the same charset used by the server.  

HTH.  

--jkl


> 
> So if I write a polish char to SQL Server installed in US
> WinXp it doesn't work.  If I write a polish char to SQL Server
> installed in Polish WinXp it works.
> 
> It seems like if I need to specify to the Server the charset
> that I using on client.
> 
> Why?
> 
> Thanks Flaviano
> 
> -----Original Message----- From: freetds-bounces AT lists.ibiblio.org
> [mailto:freetds-bounces AT lists.ibiblio.org] On Behalf Of Frediano
> Ziglio Sent: Saturday, April 16, 2005 5:22 PM To: FreeTDS
> Development Group Subject: Re: [freetds] Charset conversion
> Polish
> 
> Il giorno sab, 16-04-2005 alle 10:45 +0200, FlavianoMa ha
> scritto:  > I have my SQL Server installed on Windows Xp in
> Polish.  > When I connect from TDS I can't read correctly the
> Polish chars.  > If I set client charset as ISO-8859-1 I
> received an error and the > Polish chars are translated in ?.
> > If I set client charset as UTF-8 I received the Polish chars
> as 2 byte
> 
> > in my string.  > If I set client charset as UCS-2LE or
> UCS-2BE I can't connect.  > I use FreeTDS 0.62.4 and below
> the FreeTDS.LOG: the 1st is when I > connected with ISO-8859-1,
> the 2nd is when I connected with UTF-8 > > The correctly
> conversion for T_PATTNAME should be:  > 0x4F 0x42 0xA3 0x2E
> 0x20 0x50 0x52 0x5A 0x59 0x8C 0x52 0x2E 0x20 0x50 > 0x52 0x5A
> 0x45 0x44 0x4E 0x49 0x41 0x00 >
> 
> You can read polish character correctly only if conversion
> charset support your charset and you select the right charset.
> iso-8859-1 cannot support polish so it replace with ?, utf-8
> correctly return 2 byte for polish, ucs2 still not supported
> (and will never be using dblib...). Use correct iconv library
> and charset. So, don't use --disable-libiconv and set correct
> charset.


-----------------------------------------
The information contained in this transmission may contain privileged and
confidential information and is intended only for the use of the person(s) 
named
above. If you are not the intended recipient, or an employee or agent 
responsible
for delivering this message to the intended recipient, any review, 
dissemination,
distribution or duplication of this communication is strictly prohibited. If 
you are
not the intended recipient, please contact the sender immediately by reply 
e-mail
and destroy all copies of the original message. Please note that we do not 
accept
account orders and/or instructions by e-mail, and therefore will not be 
responsible
for carrying out such orders and/or instructions.  If you, as the intended 
recipient
of this message, the purpose of which is to inform and update our clients, 
prospects
and consultants of developments relating to our services and products, would 
not
like to receive further e-mail correspondence from the sender, please "reply" 
to the
sender indicating your wishes.  In the U.S.: 1345 Avenue of the Americas, New 
York,
NY 10105.