FreeTDS Development Group

["Lowden, James K" <LowdenJK AT bernstein.com>]

> From: Li, Maggie (IT)
> Sent: Tuesday, December 06, 2005 9:16 AM
> 
> > Also, what does "ident $(command -v freebcp)" tell you?
> 
> $Id: tds.h,v 1.167 2003/12/31 11:33:10 freddy77 Exp $
...
> $Id: freebcp.c,v 1.28 2003/12/09 10:19:16 freddy77 Exp $

I thought that might be the case.  

Upgrade.  You're using 0.62, code that's two years old; we'll be
releasing 0.64 soon.  A lot happens in two years.  

I'm glad you got your file loaded, though.  

--jkl


> -----Original Message-----
> From: freetds-bounces AT lists.ibiblio.org
> [mailto:freetds-bounces AT lists.ibiblio.org] On Behalf Of James K.
> Lowden
> Sent: Monday, December 05, 2005 8:38 PM
> To: FreeTDS Development Group Subject: Re: [freetds] freebcp file
> limit
> 
> Li, Maggie (IT) wrote:
> > I need to freebcp a large file (about 15 million lines, 4 gb in
> > size),
> 
> > but after about 5 million lines, it gives segmentation fault.
> 
> Wow.  Thanks for the report.
> 
> It would take quite a bit of effort for me to reproduce that.  My
> testing facilities are across the Internet.  Even if they have that
> much space, it would take hours just to transmit the data.
> 
> I do have Sybase installed around here somewhere, on a machine
> that's normally booted in something else.  If you could send me
> your DDL, I'll work on it.  It would be very helpful, too, if you
> could send a backtrace.
>  
> 
> Also, what does "ident $(command -v freebcp)" tell you?
> 
> >  Is there a size limit for freebcp?
> 
> No, there's not supposed to be.  The file is read in chunks and
> transmitted to the server piecemeal.  You apparently have tickled
> a bug, some bit of memory that isn't being freed.
> 
> The good news is there may be a workaround.  As long as your data
> file is in plain text (not a "native" bcp file), you can split it
> up.  You could use sed(1) to do the job, or run something this,
> which takes advantage of the first/last options of freebcp:
> 
> #! /bin/sh BCP='freebcp db.own.table in file.dat -S server -c -U
> -P' for FIRST in $(jot 16 0) do
>       LAST=$(( ${FIRST} + 1 )) printf  "${BCP} -b 100000 -F %8d
>       -L %8d \n" \
>               ${FIRST}000001 ${LAST}000000
> done
> 
> Pipe the output to sh, as in
> 
>       $ ./foo | sh -e -x
> 
> and go to lunch.

-----------------------------------------
The information contained in this transmission may be privileged and
confidential and is intended only for the use of the person(s) named
above. If you are not the intended recipient, or an employee or agent 
responsible
for delivering this message to the intended recipient, any review, 
dissemination,
distribution or duplication of this communication is strictly prohibited. If 
you are
not the intended recipient, please contact the sender immediately by reply 
e-mail
and destroy all copies of the original message. Please note that we do not 
accept
account orders and/or instructions by e-mail, and therefore will not be 
responsible
for carrying out such orders and/or instructions.  If you, as the intended 
recipient
of this message, the purpose of which is to inform and update our clients, 
prospects
and consultants of developments relating to our services and products, would 
not
like to receive further e-mail correspondence from the sender, please "reply" 
to the
sender indicating your wishes.  In the U.S.: 1345 Avenue of the Americas, New 
York,
NY 10105.