FreeTDS Development Group

["Lowden, James K" <LowdenJK AT bernstein.com>]

> From: ZIGLIO, Frediano, VF-IT
> Sent: Tuesday, November 30, 2004 10:43 AM
> > From: Lowden, James K
> > Sent: lunedì 29 novembre 2004 20.03
> > 
> > I think the log format should be standardized; it would be easier to
> > grep and scan.  I don't see much value in logging the time on every
> > line.  It's enough that we know when packets were sent and received.
> > 
> > Here's a format suggestion:
> > 
> >     LEVEL PID Function (File:Line): text
> > 
> > LEVEL would be one of:
> > 
> >     LOGIN 
> >     API   
> >     ASYNC 
> >     DIAG 
> >     ERROR 
> >     PACKET
> >     LIBTDS
> >     CONFIG
> > 
> 
> What difference between API and LIBTDS ??

If you set the API bit, you get db-lib/ct-lib/odbc logging: a log entry for 
every API call (and probably some internal ones).  If you set the LIBTDS bit, 
you get a log entry for libtds function calls.  

> > Surely stdio synchronizes writes to a file?  
> > 
> 
> stdio ?? If I open 2 connection I need to write to the same file. If I use 
> same handle I'm sure it write correctly informations but if I open two 
> handles for the same file probably one write overwrite other write. Well... 
> I'm not sure but this thing need a bit of testing.

[Interesting test program omitted.]

I had in mind opening the file once and writing through a global handle.

> Also we need a macro to support file:line syntax...

Well, if TDSDUMP() is itself a macro, __FILE__ and __LINE__ are no problem.  
I'd use __FUNCTION__, too, as long as it's supported by the compiler (which 
we can determine with autoconf).  If the compiler can't provide the function 
name, it won't appear in the log.  

> > I don't think FreeTDS should catch signals.  It's up to the 
> > application
> > to catch signals and call the API accordingly.  If we decide to use
> > O_ASYNC, that's another story.  
> 
> I know that it's up the application however dbcancel docs state that a code 
> inside a signal can call this function (to support alarm or similar) and 
> SQLCancel docs state that this function can be called from another thread 
> while connection it's busy.

So we have to make sure that dbcancel() only does things that are allowed 
within a signal-handler.  

I think the low-level networking code will have to be reworked to support 
timeouts and cancels correctly.  

> > We need to support binary bindings of character data.  We lost that
> > feature when we implemented iconv throughout.  

> There are many solution/optimization/changes that we can combine
> to relax these difficulties. For example thinking about inserting
> (bulk, rpc, dynamic call) we call call a libTDS function to start
> sending data and then call different functions to insert a
> parameter/column at a time. Or we can provide a way to libTDS to
> grab data directly from binded data (calling for example a function
> that convert data for use or similar). Or a mix of the two (libTDS
> stop when it don't know who to convert parameters and so higher
> API can do the job...).

This is an interesting problem.  We should discuss it, and try to devise an 
internal API.  Like you, I see layers:

1.  net.  open, read/write, close.  Timeouts, interrupt handlers, O_ASYNC.  
Also, for BCP purposes, netlib should let the caller use sockets and files in 
exactly the same way, so that we can BCP from server to server or from file 
to file (to do format/encoding conversions).  This layer only deals with 
buffers (not files or callback functions).  

2.  tds.  The current code has no packet layer.  Each packet is hand-built 
(if you will): put the token, put the length, put this byte, put these two 
bytes, put this string, etc.  It would be clearer if there were functions 
like "write_<token>_packet(struct <token>_packet_type* data)".  Most packets 
-- everything except rows -- are quite small and can easily be described by a 
structure. 

3.  row.  Using metadata and data, compose a row, sending pieces via the 
"net" layer.  Read data from "net", constructing metadata and filling data 
buffers.  The column read/write functions allocate buffers to communicate 
with the "net" layer.  No buffers would be passed to/from callers.  Instead, 
they should take function pointers as arguments, and those functions will 
provide/accept data.  This would allow BCP to work directly with files, using 
only one buffer.  

The "row" layer has a lot of work to do:

*   iconv.  We put all iconv work in libtds, next to the wire.  We convert 
immediately to client's encoding.  That's OK, be we have to honor the 
client's binding, too.  I guess this is part of "row".  

*   text.  Also part of "row".  As you say, sending a row is really just 
calling several send-a-column functions, same for reading a row.  Text is 
tricky, because a partial column can be read/written.  

*   column.  ct-lib and ODBC both provide for column-wise result sets.  I 
haven't read Bill's code, so I don't know how he does it, but if we're 
clever, the row-reading function will know how to build columns, too.  
Without double buffering.  

Does that look like a good direction to you?  

--jkl



-----------------------------------------
The information contained in this transmission may contain privileged and 
confidential information and is intended only for the use of the person(s) 
named above. If you are not the intended recipient, or an employee or agent 
responsible for delivering this message to the intended recipient, any 
review, dissemination, distribution or duplication of this communication is 
strictly prohibited. If you are not the intended recipient, please contact 
the sender immediately by reply e-mail and destroy all copies of the original 
message. Please note that we do not accept account orders and/or instructions 
by e-mail, and therefore will not be responsible for carrying out such orders 
and/or instructions.
If you, as the intended recipient of this message, the purpose of which is to 
inform and update our clients, prospects and consultants of developments 
relating to our services and products, would not like to receive further 
e-mail correspondence from the sender, please "reply" to the sender 
indicating your wishes.  In the U.S.: 1345 Avenue of the Americas, New York, 
NY 10105.