Skip to Content.
Sympa Menu

freetds - RE: Charset conversion with Sybase ASE 12.0

freetds AT lists.ibiblio.org

Subject: FreeTDS Development Group

List archive

Chronological Thread  
  • From: "Lowden, James K" <LowdenJK AT bernstein.com>
  • To: "'TDS Development Group'" <freetds AT franklin.metalab.unc.edu>
  • Subject: RE: Charset conversion with Sybase ASE 12.0
  • Date: Thu, 14 Nov 2002 12:19:36 -0500


> From: Maxim V Rayevskiy [mailto:ra-max AT yandex.ru]
> Sent: November 14, 2002 2:53 AM
>
> I have set up freetds 0.60 on my FreeBSD system to connect to
> Sybase ASE
> 12.0 running on a Windows server. The character set on Sybase
> is set to be
> cp 1251 (Windows cyrillic). I have edited the locales.conf to
> include char
> set = cp1251. Yet, when I connect to the server I get
>
> Character set conversion is not available between client character set
> 'iso_1' and server character set 'cp1251'.

Maxim,

Thank you for your message. It's clear to me that the documentation, and
maybe the code, needs some revision on this score.

Short answer: Use the "client charset" option in freetds.conf, not in
locales.conf, if you want to use a nonstandard server encoding for your
locale.

Medium answer: Set your LOCALE properly and add an entry to locales.conf:

http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/using-localization
.html
[ru_RU]
date format = %b %d %Y %I:%M%p # may want to change this
language = us_english # may want "russian"?
char set = cp1251 # I think

and of course "export LANG=ru_RU".

If that doesn't work, please provide:

* 0.60 release, or date of snapshot?
* Are you using db-lib, ct-lib, or ODBC?
* TDSDUMPCONFIG log
* TDSDUMP log

Although you should remove the password from your log, please don't munge
the log too much, because the login packet is what we're looking for.

Long answer follows.

locales.conf maps locale information found in the environment to FreeTDS's
settings. That is, if your LANG environment variable is set to "en_US",
FreeTDS will know to tell the server that you want an ISO-1 character set,
US English messages, and a US date format. It's meant to be static, not
site-specific. The notion is to take cues from the client's LOCALE settings
to make not-stupid assumptions about the default settings. Each
language-country combination will get appropriate date, language, and
character set defaults.

I realize Russian is represented by more than one character set; I don't
know how the LOCALE system deals with that. If you would send a four-line
patch for locales.conf suitable to your LOCALE settings, I'll include it in
our distributed file from now on.

freetds.conf overrides the character set inferred from locales.conf. (It
should be possible, on reflection, to override the date format and language,
too, but that'll have to wait.) That's why I think the Short Answer will
solve your immediate problem.

I don't think the documentation is terribly clear on this point, and may
even be misleading. And I'm not positive the code is right, but it looks
it, at least in cvs.

Background to the long answer follows.

When a connection to the server is established, the TDSLOGIN record includes
a "char_set" field.

Sybase servers (TDS 5.0) transmit in 8-bit format, normally ASCII but also
other charsets including CP1251 and UTF-8. Any conversion from the server's
charset to the client's is done by the server, in accordance with what was
requested at login time.

Microsoft servers (TDS 7.0) have a charset too, but they send their data in
Unicode (UCS-2 ). Because most applications using FreeTDS aren't prepared
to deal with 16-bit characters, FreeTDS employs libiconv to convert from
UCS-2 to the client's character set. To the best of my knowledge, the
conversion of server_charset<->USC-2 is performed by the server, and any
conversion of USC-2<->client_charset is performed by the client libary
(that's us). The server ignores the TDSLOGIN "char_set" field (unless it's
a TDS 4.2) login.

The cascading effects of locales.conf and freetds.conf to set the charset
are not well-tested. It may be that the system does not work as described,
because most of the developers don't depend on it. However, it's important
to us that it does work, and I at least would be glad to correct anything we
can isolate.

Regards,

--jkl


>
> It would seem that my charset settings are ignored. Is there
> anything I am
> missing?
>
> My freetds.conf is left untouched except that I added this:
> [main]
> host=prog-srv
> port=5000
> tds version = 5.0
> client charset = cp1251
>
> My locales.conf:
> [default]
> date format = %b %d %Y %I:%M%p
>
> [en_US]
> date format = %b %d %Y %I:%M%p
> language = us_english
> char set = cp1251
> ~
>
> ---
> You are currently subscribed to freetds as: [LowdenJK AT bernstein.com]
> To unsubscribe, forward this message to
> $subst('Email.Unsub')
>


The information contained in this transmission may contain privileged and
confidential information and is intended only for the use of the person(s)
named above. If you are not the intended recipient, or an employee or agent
responsible for delivering this message to the intended recipient, any
review, dissemination, distribution or duplication of this communication is
strictly prohibited. If you are not the intended recipient, please contact
the sender immediately by reply e-mail and destroy all copies of the
original message. Please note that we do not accept account orders and/or
instructions by e-mail, and therefore will not be responsible for carrying
out such orders and/or instructions.






Archive powered by MHonArc 2.6.24.

Top of Page