Skip to Content.
Sympa Menu

freetds - RE: [freetds] BCP performance

freetds AT lists.ibiblio.org

Subject: FreeTDS Development Group

List archive

Chronological Thread  
  • From: "Lowden, James K" <LowdenJK AT bernstein.com>
  • To: "'freetds AT lists.ibiblio.org'" <freetds AT lists.ibiblio.org>
  • Subject: RE: [freetds] BCP performance
  • Date: Wed, 5 Feb 2003 15:08:43 -0500

http://www.swox.com/gmp/

I'm going to compare some results. It's hard to believe we shouldn't use
gmp, even though it introduces a dependency.

--jkl




> -----Original Message-----
> From: Lowden, James K [mailto:LowdenJK AT bernstein.com]
> Sent: February 5, 2003 2:31 PM
> To: 'freetds AT lists.ibiblio.org'
> Subject: RE: [freetds] BCP performance
>
>
> > From: Lowden, James K [mailto:LowdenJK AT bernstein.com]
> > Sent: February 4, 2003 5:20 PM
> >
> > I'm experiencing a significant bcp performance differential
> > between FreeTDS
> > and Microsoft's libraries.
> >
> > On Windows NT, using Microsoft's BCP, I get:
> >
> > $ bcp MSCI..vEQSecurityNames out vEQSecurityNames.txt -c -T
> > ...
> > 63744 rows copied.
> > Network packet size (bytes): 4096
> > Clock Time (ms.): total 3875 Avg 0 (16450.06
> rows per sec.)
> >
> > Under 4 seconds.
>
> Further information. Using freebcp -n (native mode, don't
> convert the data
> to character format), performance is comparable to Microsoft's bcp:
>
> $ time freebcp MSCI..vEQSecurityNames out
> vEQSecurityNames.bcp -n -U$U -P$P
> -S$S
>
> Starting copy...
>
> firstrow = 0 row_of_query = 63744 rows written 63744
> 63744 rows copied.
>
> real 0m4.620s
> user 0m3.325s
> sys 0m0.220s
>
> $ time freebcp MSCI..vEQSecurityNames out
> vEQSecurityNames.txt -c -U$U -P$P
> -S$S
>
> Starting copy...
>
> firstrow = 0 row_of_query = 63744 rows written 63744
> 63744 rows copied.
>
> real 1m4.006s
> user 0m38.862s
> sys 0m0.388s
>
> This tells me that of the 1:04 elapsed time, 0:38 is spent in
> user space,
> and probably 0:35 of that in tds_convert(). It also leads me
> to suspect
> that the slow reception at the client end introduces delays
> in sending data
> from the server. How else to account for the gap between
> elapsed and user
> time? That gap is persistent. Here's another view, using
> NetBSD's time(1)
> utility:
>
> $ /usr/bin/time -l freebcp MSCI..vEQSecurityNames out
> vEQSecurityNames.txt
> -c -U$U -P$P -S$S
>
> Starting copy...
>
> firstrow = 0 row_of_query = 63744 rows written 63744
> 63744 rows copied.
>
> 59.07 real 38.97 user 0.23 sys
> 0 maximum resident set size
> 0 average shared memory size
> 0 average unshared data size
> 0 average unshared stack size
> 221 page reclaims
> 0 page faults
> 0 swaps
> 0 block input operations
> 485 block output operations
> 5 messages sent
> 0 messages received
> 0 signals received
> 11 voluntary context switches
> 602 involuntary context switches
>
> I would be interested to hear other reports. I think we need
> to look at the
> conversion functions.
>
> --jkl
>
>
> The information contained in this transmission may contain
> privileged and
> confidential information and is intended only for the use of
> the person(s)
> named above. If you are not the intended recipient, or an
> employee or agent
> responsible for delivering this message to the intended recipient, any
> review, dissemination, distribution or duplication of this
> communication is
> strictly prohibited. If you are not the intended recipient,
> please contact
> the sender immediately by reply e-mail and destroy all copies of the
> original message. Please note that we do not accept account
> orders and/or
> instructions by e-mail, and therefore will not be responsible
> for carrying
> out such orders and/or instructions.
>
>
>
> _______________________________________________
> FreeTDS mailing list
> FreeTDS AT lists.ibiblio.org
> http://lists.ibiblio.org/mailman/listinfo/freetds
>


The information contained in this transmission may contain privileged and
confidential information and is intended only for the use of the person(s)
named above. If you are not the intended recipient, or an employee or agent
responsible for delivering this message to the intended recipient, any
review, dissemination, distribution or duplication of this communication is
strictly prohibited. If you are not the intended recipient, please contact
the sender immediately by reply e-mail and destroy all copies of the
original message. Please note that we do not accept account orders and/or
instructions by e-mail, and therefore will not be responsible for carrying
out such orders and/or instructions.






Archive powered by MHonArc 2.6.24.

Top of Page