[freetds] bulk copy and row buffer

Thompbil18 at aol.com Thompbil18 at aol.com
Mon Jun 7 16:09:26 EDT 2004


Hey Freddy,

> I found a bcp_column_data field in TDSCOLUMN...

Yes - I'll explain why I put it there.

> - dblib bulk code use BCP_COLINFO to hold bulk information
> - ctlib bulk code use TDSCOLUMN to hold bulk information
> - ctlib use bcp_column_data to store data buffer instead of current_row

dblib won't be using BCP_COLINFO for much longer - it's gone with my next 
patch.
ctlib uses both the bcp_column_data structure in TDSCOLUMN and current_row in 
the blk-library processing. here's what it does, and why....

A program sends data to the server using bulk library by binding host program 
variables, along with a couple of indicators (data length and null), to a 
database table column. The host program variables and indicators are then 
populated, and the row_xfer function is called to send the data.

*we* then have to do the following (not necessarily in this order):

Interpret the data and indicators to work out whether the column data is NULL 
or not;
convert the data from one format to another (e.g. host variable of char type 
to database datetime column)
build a bcp "row" to send to the server (SQL server format of "bcp" row is 
easy - Sybase is horrendous).

I've decided to do this in two stages. 

The first stage does everything it can to prepare each column of data. the 
results of this stage are stored in the bcp_column_data structure. I needed to 
store this information on a column-by-column basis, and I wanted to cut down on 
memory (re)allocation, so I needed it to persist somewhere. TDSCOLUMN seemed 
the obvious choice, as that is where I store my bind information too.

The second stage is to build and send the bcp "row" data. One of the 
requirements of my rewrite was to build the whole row of data before sending anything 
down the wire. I needed to store this somewhere, and again I wanted to cut 
down on memory (re)allocation, so I decided to reuse the current_row buffer that 
had already been allocated in the results structure. What I put in the 
current_row isn't (necessarily) in the same format as results data - I'm just using 
the buffer. 

I'll think about your other ideas....

Bill




More information about the FreeTDS mailing list