FreeTDS Development Group

[christos AT zoulas.com (Christos Zoulas)]

On Jan 6,  1:58am, jklowden AT schemamania.org ("James K. Lowden") wrote:
-- Subject: [freetds] bcp rumination

| Looking forward to moving bcp into libtds, and grappling through the code
| again tonight, it occurs to me our next generation bcp code might benefit
| from some redesign, and some new abstractions.  
| 
| Fundamentally, bcp is about moving data from one stream to another with
| the least wasted motion.  The native format hacking we've been up to
| lately has made that especially clear: I usually think about bcp as a
| row-by-row conversion operation, but that's because I rely exclusively on
| delimited files (that are always in character form).  A native bcp
| transfer involves hardly more than reading and writing buffers.  
| 
| That leads me to this picture:
| 
| data ->        stream
|       ---------
|       transform
|       ---------
|        stream   -> data
| 
| It really shouldn't matter which stream is attached to a file, and which
| to a server.  I mean, a handle's a handle, right?  Apart from the gritty
| details of socket handling, the bcp library should be content copying
| file->socket, socket->file, file->file, or socket->socket.  Shouldn't
| really matter, should it?  
| 
| The "transform" has many components: character/binary format, dbconvert()
| conversions, column order mapping with format files, not to mention
| iconv().  I can think of many uses of a freebcp-like utility that could:
| 
| 1.  decode a native file, writing a delimited one, or
| 2.  reorder the columns of a file, or
| 3.  convert the character encoding of a file
| 
| Substitute "server" for "file" anywhere in the above list.  Mix and match.
|  
| I think all that's needed is a really thin layer masking the difference
| between sockets and disk files.  Does that sound feasible or infeasible to
| you?  

That is a very good idea james, but what do we do about fixing bcp for
0.62? Obviously my changes don't work in all cases, and break existing
functionality. As such, they should be backed out. On the other hand,
shipping 0.62 without the changes keeps native bcp broken. Unfortunately,
I don't have a sybase server to test this, and my mssql server seems to
pass the regression test.

christos