[freetds] More freebcp -h changes

Konrad J Hambrick konrad at jax01.payplus.com
Mon May 15 14:17:57 EDT 2006


All --

I found that the freebcp -h handler in function bcp_options()
in dblib/bcp.c would only handle one, simple, scalar hint.

When there were multiple hints or if ORDER, ROWS_PER_BATCH or
KILOBYTES_PER_BATCH were invoked via the -h command line flag,
the bcp_options() function copied only the first hint KEYWORD
into the hint structure member (KEYWORD Only, no data went in).

I needed the ORDER( col1, col2, col3 ) hint for loading a set
of tables with (very) large data sets and the existing code in
bcp_options() did not work.

I found that if I pre-sorted the data 'offline' then loaded the
sorted .TXT files via BCP.EXE with the ORDER(...) hint that the
load time was reduced about 50-fold !

Without the ORDER(...) hint, the SQL Server Machine was all but
unusable AFTER BCP.EXE COMPLETED THE LOAD because SQL Server was
apparently building a clustered index in the background.

Same thing happened with freebcp.

Anyhow, the attached patch to bcp.c simply copies the pointer to
the -h commandline string into dbproc->bcpinfo->hint WITHOUT ANY
CHECKING !

A proper patch would require parsing the pdata->hint string then
checking the syntax of (possibly) a compound hint-string.

So ...

Is there already a 'simple text parser' somewhere in the freetds
source tree ?

If so, I could look at adding sanity checks for the hint string
using the existing text parser.

Otherwise, what I have done works for me and has reduced my load +
index time by a factor of fifty, not to mention that the ORDER(...)
hint leaves the server available for other users while I load data ... <G>

Thanks again for freetds !

-- kjh
-------------- next part --------------
A non-text attachment was scrubbed...
Name: bcp.c-patch.diff
Type: text/x-patch
Size: 2573 bytes
Desc: not available
Url : http://lists.ibiblio.org/pipermail/freetds/attachments/20060515/cb7acc1d/attachment.bin 


More information about the FreeTDS mailing list