Skip to Content.
Sympa Menu

freetds - Re: [freetds] unicode

freetds AT lists.ibiblio.org

Subject: FreeTDS Development Group

List archive

Chronological Thread  
  • From: Sebastien FLAESCH <sf AT 4js.com>
  • To: FreeTDS Development Group <freetds AT lists.ibiblio.org>
  • Subject: Re: [freetds] unicode
  • Date: Fri, 01 Aug 2008 10:31:07 +0200

Laura,

Just some remarks...

With SQL Server, you must use NCHAR or NVARCHAR to store UNICODE chars.
SQL Server uses UCS-2 on the server side, not UTF-8... However FreeTDS
will support UCS-2/UTF-8 conversion (according to last discussions).

The encoding SQL_Latin1_General_CP1_CI_AS is for CHAR/VARCHAR columns.

What do you use on the Solaris machine to display the Jap text file?

What is the encoding of that text file?

If it's UTF-8, and you use for example an xterm with vi, you need to:

A- Define LANG or LC_ALL to something like en_US.utf8 or ja_JP.utf8.

B- Make sure the xterm is configured to display the correct code set.

Seb

Murray, Laura (US) wrote:
Hi
Hello -
Our OS is Solaris 10.
Encoding within the Oracle database is UTF-8
Encoding of MS SQL Server is: SQL_Latin1_General_CP1_CI_AS

I tried in vain to do the tsql test. The Japanese characters do not
display at all in the Solaris environment (i.e. in a text file on the
server). I xferred a small file from my laptop (where the Japanese
characters appeared properly) to Solaris and on Solaris the Japanese
characters are garbled.
However, within the Oracle Database (hosted by this same Solaris system)
the Japanese characters appear properly.

I am also working with Oracle Support on this problem, but no luck yet.

--Laurie

-----Original Message-----
From: freetds-bounces AT lists.ibiblio.org
[mailto:freetds-bounces AT lists.ibiblio.org] On Behalf Of James K. Lowden
Sent: Wednesday, July 30, 2008 12:41 PM
To: FreeTDS Development Group
Subject: Re: [freetds] unicode

Murray, Laura (US) wrote:
"Please try to change FreeTDS's client char set setting from UTF-8 to
EUCJP-MS as below.

1) add code EUCJP-MS to 'src/tds/encodings.h'.

2) change strings buffer from 256 to 100000 in 'src/tds/write.c'.

3) modify freetds.conf as such: client charset=EUCJP-MS


I implemented 2) and 3) but for 1) did not know where to put the code
in
the file. Also, just wanted to know if you approve of this approach,
in
general. If so.....after these modifications, do I need to redo the
freetds "make" and "make install"?

Hi Laura,
If these changes are necessary, yours would be the first case, and I'd
like to understand why. Adding code to encodings.h isn't baby simple:
the
file is generated because there are interlocking parts. I suspect it
would be better to add the name -- assuming that's required at all -- to
alternative_character_sets.h and regenerate encodings.h. And, yes, it
would mean rebuilding and re-installing FreeTDS.
But how do we know the solution lies in that direction? To answer that,
we'd have to know more about your environment. To wit:

1. OS and version
2. iconv library implementation
3. encoding of data coming from the Oracle server
4. encoding of single-byte data on the Microsoft server
5. the gadget that's using FreeTDS (maybe)

I would also pursue getting a TDSDUMP log.
I don't see how #2 will change anything. Whether the buffer is 10 bytes
or 10 million, the data are all converted and written.
#1 and #3 are related. They're based on the premise that FreeTDS is not
#recognizing the encoding being used by the "client" i.e. the Oracle
#server. That could be true but, as I said, I'm skeptical.
Can you log onto the gateway machine and use tsql to insert Japanese
characters correctly? That would demonstrate none of the above changes
are needed.
Regards,
--jkl
_______________________________________________
FreeTDS mailing list
FreeTDS AT lists.ibiblio.org
http://lists.ibiblio.org/mailman/listinfo/freetds


This email is for the use of the intended recipient(s) only. If you have received this email in error, please notify the sender immediately and then delete it. If you are not the intended recipient, you must not keep, use, disclose, copy or distribute this email without the author's prior permission. We have taken precautions to minimize the risk of transmitting software viruses, but we advise you to carry out your own virus checks on any attachment to this message. We cannot accept liability for any loss or damage caused by software viruses. The information contained in this communication may be confidential and may be subject to the attorney-client privilege. If you are the intended recipient and you do not wish to receive similar electronic messages from us in the future then please respond to the sender to this effect.

_______________________________________________
FreeTDS mailing list
FreeTDS AT lists.ibiblio.org
http://lists.ibiblio.org/mailman/listinfo/freetds






Archive powered by MHonArc 2.6.24.

Top of Page