[freetds] More freebcp -h changes
Konrad J Hambrick
konrad at jax01.payplus.com
Mon May 15 14:17:57 EDT 2006
All --
I found that the freebcp -h handler in function bcp_options()
in dblib/bcp.c would only handle one, simple, scalar hint.
When there were multiple hints or if ORDER, ROWS_PER_BATCH or
KILOBYTES_PER_BATCH were invoked via the -h command line flag,
the bcp_options() function copied only the first hint KEYWORD
into the hint structure member (KEYWORD Only, no data went in).
I needed the ORDER( col1, col2, col3 ) hint for loading a set
of tables with (very) large data sets and the existing code in
bcp_options() did not work.
I found that if I pre-sorted the data 'offline' then loaded the
sorted .TXT files via BCP.EXE with the ORDER(...) hint that the
load time was reduced about 50-fold !
Without the ORDER(...) hint, the SQL Server Machine was all but
unusable AFTER BCP.EXE COMPLETED THE LOAD because SQL Server was
apparently building a clustered index in the background.
Same thing happened with freebcp.
Anyhow, the attached patch to bcp.c simply copies the pointer to
the -h commandline string into dbproc->bcpinfo->hint WITHOUT ANY
CHECKING !
A proper patch would require parsing the pdata->hint string then
checking the syntax of (possibly) a compound hint-string.
So ...
Is there already a 'simple text parser' somewhere in the freetds
source tree ?
If so, I could look at adding sanity checks for the hint string
using the existing text parser.
Otherwise, what I have done works for me and has reduced my load +
index time by a factor of fifty, not to mention that the ORDER(...)
hint leaves the server available for other users while I load data ... <G>
Thanks again for freetds !
-- kjh
-------------- next part --------------
A non-text attachment was scrubbed...
Name: bcp.c-patch.diff
Type: text/x-patch
Size: 2573 bytes
Desc: not available
Url : http://lists.ibiblio.org/pipermail/freetds/attachments/20060515/cb7acc1d/attachment.bin
More information about the FreeTDS
mailing list