Skip to Content.
Sympa Menu

freetds - RE: Memory leak when doing many INSERTs

freetds AT lists.ibiblio.org

Subject: FreeTDS Development Group

List archive

Chronological Thread  
  • From: "Eric Deutsch" <edeutsch AT systemsbiology.org>
  • To: "TDS Development Group" <freetds AT franklin.oit.unc.edu>
  • Subject: RE: Memory leak when doing many INSERTs
  • Date: Thu, 13 Sep 2001 18:51:11 -0700



Hi again, I've done a little more investigating, and I've come back to plead
for help. Situation is the same as below:

If I do in perl with DBD::Sybase 0.91:

$querystring="INSERT INTO leaktest (intfield,charfield,realfield) VALUES (
20,'chicken',3.1415 )";
while ( 1 == 1 ) {
$dbh->do($SQLstring);
print ".";
}

and watch my process with `top` it grows forever at a rate of 6 MB per
minute. However, if I do the following in a test C program:

static char sql_query[] = "INSERT INTO dbaccesstests
(intfield,charfield,realfield) VALUES ( 20,'butter',3.1415 )";
while ( 1 == 1 ) {
rc = run_query(tds,sql_query);
if (rc != TDS_SUCCEED) {
fprintf(stderr, "run_query() failed\n");
return 1;
}
fprintf(stdout, ".");
fflush(stdout);
}

I can run this program for many minutes without a single kB being added to
the resident memory of the process. I'm using the run_query() function from
t0004.c.

So it appears that the FreeTDS C API functions aren't leaking as such, but
somehow the interaction between Perl DBI/DBD and FreeTDS is.

As a further test, I did a loop where the SQL command was:

$querystring="INSERT INTO dbaccesstests (intfield,charfield,realfield)
VALUES (
20,'butter',3.1415 ) /*
****************************************************************************
****************************************************************************
****************************************************************************
****************************************************************************
******************************************************************** */";

and as I guessed, the leak rate tripled! Thus is looks very much like
somewhere a copy of the query string is being made and not freed.

I really don't know how to go about figuring this out. Anyone out there
able to help or repro? I can send actual C and Perl programs used.

many thanks,
Eric




> -----Original Message-----
> From: bounce-freetds-127899 AT franklin.oit.unc.edu
> [mailto:bounce-freetds-127899 AT franklin.oit.unc.edu]On Behalf Of Eric
> Deutsch
> Sent: Monday, September 10, 2001 6:33 PM
> To: TDS Development Group
> Subject: [freetds] Memory leak when doing many INSERTs
>
> Hi all, I've noticed what appears to be some memory leaking while
> doing lots
> of INSERTS. I'm using Perl 5.6.0 + DBD::Sybase 0.91 + FreeTDS (both 0.51a
> and latest CVS) under Red Hat 7.0 connecting to SQL Server 2000.
>
> If I do:
>
> $querystring="INSERT INTO leaktest (intfield,charfield,realfield) VALUES (
> 20,'chicken',3.1415 )";
> while ( 1 == 1 ) {
> $dbh->do($SQLstring);
> print ".";
> }
>
> after a suitable setup, and watch my process with top, the memory
> used grows
> and grows at a rate of maybe 5 MB/min. There's no difference if I use
> prepare() and execute().
>
> I have some real code (as opposed to the above example) that is
> parsing and
> loading some huge datasets from files into a my DB and after one
> particularly large set, my process was killed because it hogged all the
> client machine's memory (probably after running for an hour).
> This is a bit
> of a snag in what I'm trying to do. I could start breaking up
> the work, but
> this is awkward for various reasons.
>
> Has anyone encountered/studied/noticed this before? Any advice?
> I suppose
> I could try taking Perl out of the loop to see if it's a FreeTDS
> issue or a
> Perl DBI + DBD::Sybase issue.
>
> I'm assuming it is not unreasonable to expect a Perl script to be able to
> run INSERTs until SQL Server explodes without running into memory
> issues, is
> that true?
>
> thanks,
> Eric
>
>
>
>
>
> ---
> You are currently subscribed to freetds as: [edeutsch AT systemsbiology.org]
> To unsubscribe, forward this message to
> $subst('Email.Unsub')
>





Archive powered by MHonArc 2.6.24.

Top of Page