FreeTDS Development Group

[Sebastien FLAESCH <sf AT 4js.com>]

Steve Langasek wrote:
> On Wed, Jul 23, 2008 at 01:59:54PM -0400, James K. Lowden wrote:
> > Sebastien FLAESCH wrote:
> > > > I see no reason support multibyte characters.  I have no problem
> > > > converting to UCS-2, UTF-8, etc.  But I see no value in using
> > > > pre-iconv functions or supporting weird encodings.  Unicode is making
> > > > them obsolete.
> > ...
> > > So the idea is to make FreeTDS support multi-byte characters sets, even
> > > if it's just UTF-8:
>
> > Hi Sebastien, 
>
> > We agree; we're just tripping over terminology.  By "multibyte characters"
> > I meant pre-Unicode encodings, such as input to mbtowc(3).  I'm happy to
> > support UTF-8.  
>
> FWIW, UTF-8 is a multibyte character set, and is understood by mbtowc() from
> glibc.

That's true.

I think FreeTDS could support all single-byte or multi-byte character sets if:

1) The application (main) calls setlocale() to define the current charset for
libc functions like mbtowc, wctomb, mblen (that would include UTF-8 support).

2) Any (char*) string manipulation in FreeTDS lib uses mblen() to know the 
size
in bytes of the characters (= no byte-based parsers/lexers).

3) Before sending on TDS protocol, convert from current charset to WideChar
(UTF-16/UCS-2) with mbstowcs() and when receiving TDS responses, convert from
WideChar to current charset with wcstombs().

...

From my understanding, a C application on UNIX can be written in:

A- Pure ASCII (LC_ALL=POSIX)
B- A specific single-byte charset (LC_ALL=fr_FR.iso88591)
C- A specific multi-byte charset (LC_ALL=ja_JP.eucjp or en_US.utf8)
D- Wide Char (UTF-16/UCS-2), using the wchar_t type.

Ideally, FreeTDS should allow all of that... no?

In cases A, B, C, FreeTDS can determine the current charset with 
nl_langinfo().

Maybe FreeTDS could work in wchar_t internally and do charset conversion
at the ODBC API level? I think this is how MS ODBC drivers work... no?

I mean (casts are just there for understanding):

SQLPrepare(hp, (char *) "my SQL string in current SBCD or MBCS charset", 
SQL_NTS);

would be mapped to:

SQLPrepareW(hp, (wchar_t *) "my SQL string in UCS-2", SQL_NTS);

and just use the mbstowcs() standard C lib function to convert current charset
to wide char...

WARNING: wchar_t on UNIX is 4 bytes while UCS-2 and wchar_t on Windonws is 2 
bytes.

?

Cheers,
Seb