Skip to Content.
Sympa Menu

freetds - RE: [freetds] Re: SQL_C_BINARY type issue

freetds AT lists.ibiblio.org

Subject: FreeTDS Development Group

List archive

Chronological Thread  
  • From: "Lowden, James K" <LowdenJK AT bernstein.com>
  • To: "FreeTDS Development Group" <freetds AT lists.ibiblio.org>
  • Subject: RE: [freetds] Re: SQL_C_BINARY type issue
  • Date: Mon, 23 Feb 2004 13:16:30 -0500

> From: Jonathan Monroe [mailto:monroej_freetds AT actualtechnologies.com]
> Sent: February 23, 2004 11:47 AM
> > > On Sun, 22 Feb 2004 at 10:03:02 +0100, Frediano Ziglio wrote:
> > > >>> On Feb 18, 2004, at 2:51 PM, Jonathan Monroe wrote:
> > > >>>
> > > >>>> I think I have found an issue with the way the SQL_C_BINARY
> > > >>> data type
> > > >>>> is handled in the ODBC code.
> >
> > Mm.... here the behavior is different. We convert
> characters as we get
> > from wire so the behavior it's exactly the same of
> varchar/char/text.
>
> IMHO the purpose of doing a SQL_C_BINARY bind is to get access to the
> raw bytes of the field - otherwise you would just bind using
> SQL_C_CHAR. I realize that much thought has preceded this
> conversation
> and MS compatibility isn't always a desired goal. But to me,
> SQL_C_BINARY implies raw bits, while SQL_C_CHAR implies (possibly
> transformed) text. In this case, it seems that a SQL_C_BINARY bind
> should return both bytes of each character in an nvarchar field.


Johnathan,

First of all, welcome to the project and thanks for your efforts. I'm glad
to see Actual Technologies joining the fray. You've raised a thorny issue
here regarding a binary bind of UCS-2 data; let's see what can be done.

Our approach has been to convert all UCS-2 data (nchar, nvarchar, ntext, and
metadata strings) into the client's encoding upon arrival, and to adjust the
column buffer sizes accordingly. The UCS-2 data are converted
instantaneously, before they're available for fetching by the client library.
As far as the application can see, they're locally encoded (UTF-8, ISO
8859-1, whatever), and that's that. We don't support UCS-2 clients yet, so
there's no possibility of seeing UCS-2 data.

Now comes the requirement to *not* convert, to faithfully convey the bits as
bits and let the, uh, chips fall where they may. Problem being, of course,
that we've long discarded the pre-converted data. What to do?

1. We could keep the server's data around. We don't need to discard them;
it's just (usually) more efficient.

2. We could unconvert them. We have the server's metadata. It's not
terribly efficient, but it's not terribly slow, either.

3. We could have a no-convert option, per server. We need this for BCP.
That would impose a significant burden on the client, however: it would have
do *all* data encoding conversions. Potentially, we could add
convert-to-client-encoding as a binding option, and to the tds_convert() and
its brethren.

I think #1 is too wasteful of client buffer memory. I can hear Frediano from
here choking on the idea of keeping two encodings of a 500 MB text column
around, just in case someone wants the pre-converted version. Stand well
clear of that one.

Option #2 is probably the most feasible in the short run. Additional binding
options and the choice of defeating built-in conversions (#3) seem like the
best architectural answers.

> > Keep in mind that endianism change the order...
>
> Yeah. Again, it seems like I should be able to get the raw bytes as
> they are stored in the database, but I'll concede that the correct
> behavior is probably undefined.

Well, you really don't know how the bytes are stored in the database; at any
rate afaik it's undocumented. You really only know how they're represented
on the wire. Our endian policy, of necessity, is "library makes right".
That is, FreeTDS presents data to the client in the client's native form. If
the server happens to deliver little endian data to a big endian client, we
fix it. Simple like that.

> I'm more than happy to help with the work here. I've spent
> considerable time crawling around in the code

The ODBC driver is a big piece of work, and we always have room for capable
hands. It seems to me expanding tds_convert() to deal with convert-to-client
and convert-to-server encodings would be an excellent place to start. And,
yes, unit tests are a good way to ensure that a given functionality
starts/continues to work as expected. :-)

BTW, the best reference point for any patch is CVS HEAD. It's sometimes
feasible to apply them to the release, and sometimes we do, but our focus is
always on the next release.

--jkl

-----------------------------------------
The information contained in this transmission may contain privileged and
confidential information and is intended only for the use of the person(s)
named above. If you are not the intended recipient, or an employee or agent
responsible for delivering this message to the intended recipient, any
review, dissemination, distribution or duplication of this communication is
strictly prohibited. If you are not the intended recipient, please contact
the sender immediately by reply e-mail and destroy all copies of the original
message. Please note that we do not accept account orders and/or instructions
by e-mail, and therefore will not be responsible for carrying out such orders
and/or instructions.
If you, as the intended recipient of this message, the purpose of which is to
inform and update our clients, prospects and consultants of developments
relating to our services and products, would not like to receive further
e-mail correspondence from the sender, please "reply" to the sender
indicating your wishes. In the U.S.: 1345 Avenue of the Americas, New York,
NY 10105.






Archive powered by MHonArc 2.6.24.

Top of Page