Skip to Content.
Sympa Menu

freetds - RE: [freetds] Longstanding issue and 0.64...

freetds AT lists.ibiblio.org

Subject: FreeTDS Development Group

List archive

Chronological Thread  
  • From: "Lowden, James K" <LowdenJK AT bernstein.com>
  • To: "FreeTDS Development Group" <freetds AT lists.ibiblio.org>
  • Subject: RE: [freetds] Longstanding issue and 0.64...
  • Date: Tue, 30 Nov 2004 15:12:01 -0500

> From: ZIGLIO, Frediano, VF-IT
> Sent: Tuesday, November 30, 2004 10:43 AM
> > From: Lowden, James K
> > Sent: lunedì 29 novembre 2004 20.03
> >
> > I think the log format should be standardized; it would be easier to
> > grep and scan. I don't see much value in logging the time on every
> > line. It's enough that we know when packets were sent and received.
> >
> > Here's a format suggestion:
> >
> > LEVEL PID Function (File:Line): text
> >
> > LEVEL would be one of:
> >
> > LOGIN
> > API
> > ASYNC
> > DIAG
> > ERROR
> > PACKET
> > LIBTDS
> > CONFIG
> >
>
> What difference between API and LIBTDS ??

If you set the API bit, you get db-lib/ct-lib/odbc logging: a log entry for
every API call (and probably some internal ones). If you set the LIBTDS bit,
you get a log entry for libtds function calls.

> > Surely stdio synchronizes writes to a file?
> >
>
> stdio ?? If I open 2 connection I need to write to the same file. If I use
> same handle I'm sure it write correctly informations but if I open two
> handles for the same file probably one write overwrite other write. Well...
> I'm not sure but this thing need a bit of testing.

[Interesting test program omitted.]

I had in mind opening the file once and writing through a global handle.

> Also we need a macro to support file:line syntax...

Well, if TDSDUMP() is itself a macro, __FILE__ and __LINE__ are no problem.
I'd use __FUNCTION__, too, as long as it's supported by the compiler (which
we can determine with autoconf). If the compiler can't provide the function
name, it won't appear in the log.

> > I don't think FreeTDS should catch signals. It's up to the
> > application
> > to catch signals and call the API accordingly. If we decide to use
> > O_ASYNC, that's another story.
>
> I know that it's up the application however dbcancel docs state that a code
> inside a signal can call this function (to support alarm or similar) and
> SQLCancel docs state that this function can be called from another thread
> while connection it's busy.

So we have to make sure that dbcancel() only does things that are allowed
within a signal-handler.

I think the low-level networking code will have to be reworked to support
timeouts and cancels correctly.

> > We need to support binary bindings of character data. We lost that
> > feature when we implemented iconv throughout.

> There are many solution/optimization/changes that we can combine
> to relax these difficulties. For example thinking about inserting
> (bulk, rpc, dynamic call) we call call a libTDS function to start
> sending data and then call different functions to insert a
> parameter/column at a time. Or we can provide a way to libTDS to
> grab data directly from binded data (calling for example a function
> that convert data for use or similar). Or a mix of the two (libTDS
> stop when it don't know who to convert parameters and so higher
> API can do the job...).

This is an interesting problem. We should discuss it, and try to devise an
internal API. Like you, I see layers:

1. net. open, read/write, close. Timeouts, interrupt handlers, O_ASYNC.
Also, for BCP purposes, netlib should let the caller use sockets and files in
exactly the same way, so that we can BCP from server to server or from file
to file (to do format/encoding conversions). This layer only deals with
buffers (not files or callback functions).

2. tds. The current code has no packet layer. Each packet is hand-built
(if you will): put the token, put the length, put this byte, put these two
bytes, put this string, etc. It would be clearer if there were functions
like "write_<token>_packet(struct <token>_packet_type* data)". Most packets
-- everything except rows -- are quite small and can easily be described by a
structure.

3. row. Using metadata and data, compose a row, sending pieces via the
"net" layer. Read data from "net", constructing metadata and filling data
buffers. The column read/write functions allocate buffers to communicate
with the "net" layer. No buffers would be passed to/from callers. Instead,
they should take function pointers as arguments, and those functions will
provide/accept data. This would allow BCP to work directly with files, using
only one buffer.

The "row" layer has a lot of work to do:

* iconv. We put all iconv work in libtds, next to the wire. We convert
immediately to client's encoding. That's OK, be we have to honor the
client's binding, too. I guess this is part of "row".

* text. Also part of "row". As you say, sending a row is really just
calling several send-a-column functions, same for reading a row. Text is
tricky, because a partial column can be read/written.

* column. ct-lib and ODBC both provide for column-wise result sets. I
haven't read Bill's code, so I don't know how he does it, but if we're
clever, the row-reading function will know how to build columns, too.
Without double buffering.

Does that look like a good direction to you?

--jkl



-----------------------------------------
The information contained in this transmission may contain privileged and
confidential information and is intended only for the use of the person(s)
named above. If you are not the intended recipient, or an employee or agent
responsible for delivering this message to the intended recipient, any
review, dissemination, distribution or duplication of this communication is
strictly prohibited. If you are not the intended recipient, please contact
the sender immediately by reply e-mail and destroy all copies of the original
message. Please note that we do not accept account orders and/or instructions
by e-mail, and therefore will not be responsible for carrying out such orders
and/or instructions.
If you, as the intended recipient of this message, the purpose of which is to
inform and update our clients, prospects and consultants of developments
relating to our services and products, would not like to receive further
e-mail correspondence from the sender, please "reply" to the sender
indicating your wishes. In the U.S.: 1345 Avenue of the Americas, New York,
NY 10105.






Archive powered by MHonArc 2.6.24.

Top of Page