FreeTDS Development Group

["Ed Edgar" <eedgar AT review.co.jp>]

Brian wrote:
| I think having a default of UTF-8 is perhaps not a bad idea after all but
| let's add a configuration setting to convert to local 8bit where
| applicable. (We have to call iconv anyway, it'll be pretty easy to just
| substitute greek or whatever for UTF-8 where needed).

That sounds right to me, assuming there's no way to find out the default
encoding that the database expects, if there is one.

(I'm a bit out of my depth here, so ignore me if I start talking
nonsense...)

Sensible Japanese developers will often want to do all their internal data
processing (and by extension their data storage) in UTF8 in any case, rather
than the Microsoft-standard native encoding shift-jis, because shift-jis is
a gruesome mixture of 7-bit and 8-bit bytes, which insufficiently localized
applications (eg. PHP) can confuse with ascii. This gets really fun when
somebody uses the letter "so", the one byte of which looks exactly like an
ascii backslash...

Of course, there are plenty of situations where we want to send and retrieve
text in all kinds of formats; In a purely UNIX environment, you could do
everything in EUC, so you wouldn't want to be able to do any conversion, and
want to send and retrieve your data in EUC. (Which works OK because EUC is
purely 8-bit, so it doesn't clash with ascii.) So the option to use a local
8-bit (or 7- and 8- bit?) encoding is very important, I think.

As I think I mentioned before, in my own situation, for legacy reasons, I'm
dependent on my own "client conspiracy". We just dump text in any of the
character sets we work with (Shift-JIS, Big5, EUC-KR, whatever they use for
Thai...) into our (non-localized) database, without telling the database
what it is. This has been working more-or-less OK for our purposes in our
other platform, a Windows-ODBC environment, building apps in a web
development kit called Tango, which is 8-bit-clean but 2-byte-ignorant. I
have no idea what our data this looks like internally to MSSQL2000, and
obviously all bets are off when it comes to sorting, indexing, etc. It's an
ugly hack, but it lets us store our data and get it back unscathed. And I
bet we're not the only people who are doing it.

So it would be great if you could retain an option to let us do the same
thing as we can do with FreeTDS 4.2 - just chuck a bunch of 8-bit characters
at our database, and be able to get them back unbroken.

Anyhow, keep up the good work.

Ed Edgar