FreeTDS Development Group

[lihui zhao <lzhao AT n-able.com>]

Hi James,

I just went through the datacopy program, it's pretty much identical to 
what you described here. 

I see your point to rewrite freebcp this way, it's a lot simpler. 
Meanwhile, seems it's better to move some functions in datacopy to a 
utility, and modify them to be general enough to support import from 
files. Otherwise, the new freebcp will do a log copy/paste..

Thanks,

Lihui

James K. Lowden wrote:

> > How do you compare removing fseek/ftell from dblib vs. implementing 
> > freebcp in ctlib in term of amount of work, lib stability etc.?
> >    
>
> Hi Lihui, 
>
> It would be much easier to improve freebcp than to rewrite it in terms of
> ct-lib.  For one thing, ct-lib lacks some of the API support you need.  
>
> Sending a file to a server via bcp should have the same code structure as
> reading a result set: set up a row (by parsing a line in the file), send
> it, repeat.  In fact, it should use the public bcp* functions.  It would
> be much simpler if it did.  
>
> Something like:
>
> Get table metadata.
> foreach column {
>         malloc buffer
>         bcp_bind buffer
> }
> foreach line of data in the file {
>         foreach field in the line {
>                 foreach byte {
>                         fgetc
>                         buffer[i++] = c;
>                         break if field terminator
>                 }
>                 break if row terminator
>         }
>         bcp_sendrow
>         if irow++ > batchsize {
>                 bcp_batch
>         }
> }
>
> There's no need to seek, and bcp_bind() will call dbconvert() and iconv()
> as appropriate.  
>
> We also don't handle text/image data intelligently.  That is, we allocate
> one buffer for the whole thing, which is fine if we have the memory.  To
> use a smaller buffer, we need to use dbmoretext().  If you're interested
> in working on db-lib, writing a unit test for dbmoretext() with bcp would
> be a good place to start.  (t0013.c uses the function, but not for bcp. 
> I'm not sure it works with bcp.)  
>
> I'm happy to help further if you want to work on it.  I'd prefer to use
> the mailing list, but we can work off-list if that's your preference.  
>
> Regards, 
>
> --jkl
>