Skip to Content.
Sympa Menu

freetds - [freetds] bcp rumination

freetds AT lists.ibiblio.org

Subject: FreeTDS Development Group

List archive

Chronological Thread  
  • From: "James K. Lowden" <jklowden AT schemamania.org>
  • To: TDS Development Group <freetds AT lists.ibiblio.org>
  • Subject: [freetds] bcp rumination
  • Date: Tue, 6 Jan 2004 01:58:44 -0500

Looking forward to moving bcp into libtds, and grappling through the code
again tonight, it occurs to me our next generation bcp code might benefit
from some redesign, and some new abstractions.

Fundamentally, bcp is about moving data from one stream to another with
the least wasted motion. The native format hacking we've been up to
lately has made that especially clear: I usually think about bcp as a
row-by-row conversion operation, but that's because I rely exclusively on
delimited files (that are always in character form). A native bcp
transfer involves hardly more than reading and writing buffers.

That leads me to this picture:

data -> stream
---------
transform
---------
stream -> data

It really shouldn't matter which stream is attached to a file, and which
to a server. I mean, a handle's a handle, right? Apart from the gritty
details of socket handling, the bcp library should be content copying
file->socket, socket->file, file->file, or socket->socket. Shouldn't
really matter, should it?

The "transform" has many components: character/binary format, dbconvert()
conversions, column order mapping with format files, not to mention
iconv(). I can think of many uses of a freebcp-like utility that could:

1. decode a native file, writing a delimited one, or
2. reorder the columns of a file, or
3. convert the character encoding of a file

Substitute "server" for "file" anywhere in the above list. Mix and match.


I think all that's needed is a really thin layer masking the difference
between sockets and disk files. Does that sound feasible or infeasible to
you?

--jkl







Archive powered by MHonArc 2.6.24.

Top of Page