FreeTDS Development Group

["Lowden, James K" <LowdenJK AT bernstein.com>]

> From: Lowden, James K [mailto:LowdenJK AT bernstein.com]
> Sent: February 4, 2003 5:20 PM
> 
> I'm experiencing a significant bcp performance differential 
> between FreeTDS
> and Microsoft's libraries.  
> 
> On Windows NT, using Microsoft's BCP, I get:
> 
> $ bcp MSCI..vEQSecurityNames out vEQSecurityNames.txt -c -T 
> ...
> 63744 rows copied.
> Network packet size (bytes): 4096
> Clock Time (ms.): total     3875 Avg        0 (16450.06 rows per sec.)
> 
> Under 4 seconds.  

Further information.  Using freebcp -n (native mode, don't convert the data
to character format), performance is comparable to Microsoft's bcp:

$ time freebcp MSCI..vEQSecurityNames out vEQSecurityNames.bcp -n -U$U -P$P
-S$S

Starting copy...

firstrow = 0 row_of_query = 63744 rows written 63744
63744 rows copied.

real    0m4.620s
user    0m3.325s
sys     0m0.220s

$ time freebcp MSCI..vEQSecurityNames out vEQSecurityNames.txt -c -U$U -P$P
-S$S

Starting copy...

firstrow = 0 row_of_query = 63744 rows written 63744
63744 rows copied.

real    1m4.006s
user    0m38.862s
sys     0m0.388s

This tells me that of the 1:04 elapsed time, 0:38 is spent in user space,
and probably 0:35 of that in tds_convert().  It also leads me to suspect
that the slow reception at the client end introduces delays in sending data
from the server.  How else to account for the gap between elapsed and user
time?  That gap is persistent.  Here's another view, using NetBSD's time(1)
utility:

$ /usr/bin/time -l freebcp MSCI..vEQSecurityNames out vEQSecurityNames.txt
-c -U$U -P$P -S$S

Starting copy...

firstrow = 0 row_of_query = 63744 rows written 63744
63744 rows copied.

       59.07 real        38.97 user         0.23 sys
         0  maximum resident set size
         0  average shared memory size
         0  average unshared data size
         0  average unshared stack size
       221  page reclaims
         0  page faults
         0  swaps
         0  block input operations
       485  block output operations
         5  messages sent
         0  messages received
         0  signals received
        11  voluntary context switches
       602  involuntary context switches

I would be interested to hear other reports.  I think we need to look at the
conversion functions.  

--jkl


The information contained in this transmission may contain privileged and
confidential information and is intended only for the use of the person(s)
named above.  If you are not the intended recipient, or an employee or agent
responsible for delivering this message to the intended recipient, any
review, dissemination, distribution or duplication of this communication is
strictly prohibited. If you are not the intended recipient, please contact
the sender immediately by reply e-mail and destroy all copies of the
original message. Please note that we do not accept account orders and/or
instructions by e-mail, and therefore will not be responsible for carrying
out such orders and/or instructions.