FreeTDS Development Group

[Frediano Ziglio <freddyz77 AT tin.it>]

Il mar, 2004-01-06 alle 07:58, James K. Lowden ha scritto:
> Looking forward to moving bcp into libtds, and grappling through the code
> again tonight, it occurs to me our next generation bcp code might benefit
> from some redesign, and some new abstractions.  
> 
> Fundamentally, bcp is about moving data from one stream to another with
> the least wasted motion.  The native format hacking we've been up to
> lately has made that especially clear: I usually think about bcp as a
> row-by-row conversion operation, but that's because I rely exclusively on
> delimited files (that are always in character form).  A native bcp
> transfer involves hardly more than reading and writing buffers.  
> 
> That leads me to this picture:
> 
> data ->        stream
>       ---------
>       transform
>       ---------
>        stream   -> data
> 
> It really shouldn't matter which stream is attached to a file, and which
> to a server.  I mean, a handle's a handle, right?  Apart from the gritty
> details of socket handling, the bcp library should be content copying
> file->socket, socket->file, file->file, or socket->socket.  Shouldn't
> really matter, should it?  
> 
> The "transform" has many components: character/binary format, dbconvert()
> conversions, column order mapping with format files, not to mention
> iconv().  I can think of many uses of a freebcp-like utility that could:
> 
> 1.  decode a native file, writing a delimited one, or
> 2.  reorder the columns of a file, or
> 3.  convert the character encoding of a file
> 
> Substitute "server" for "file" anywhere in the above list.  Mix and match.
>  
> 
> I think all that's needed is a really thin layer masking the difference
> between sockets and disk files.  Does that sound feasible or infeasible to
> you?  
> 

I would say "see my discussion on zero-copy feature" :) Yes, it's
feasible. I don't know if it's the right time to introduce this feature
very deeply however if you don't remember my idea was to introduce the
stream concept in libTDS to be able to pass a stream to it, just
something like

struct TDSREAD {
        /** start of buffer */
        char *start
        /** end of buffer */
        char *end;
        /** current read position. it should be start <= current <= end */
        char *current;
        /** read next chunk of data */
        int (*read_next)(TDSREAD*);
};

This can be extended (like C++ or gnome) as

struct FILEREAD {
        TDSREAD read;
        FILE *f;
}

static int 
file_read_next(FILEREAD *fr)
{
        ...
}
...
FILEREAD fr;

// well, here we should init buffer and assign to FILE* or whatever...

fr.read.read_next = file_read_next;

tds_stream_func(...(TDSREAD*) &fr...)

freddy77